You are on page 1of 26

U.S.

Department of Education Federal Student Aid

Production Readiness Review (PRR) Process Description

Version7.0 (Final) August 20, 2007

Production Readiness Review Process, Version 7.0 TABLE OF CONTENTS Table of Contents .......................................................................................................................... 2 I. Legislative Background ............................................................................................................ 3 II. Purpose ..................................................................................................................................... 3 III. Applicability............................................................................................................................ 3 IV. Production Readiness Review Process Steps ....................................................................... 4 V. PRR Presentation Outline ....................................................................................................... 6 VI. Sign-Off ................................................................................................................................... 7 VII. Sign-Off Responsibilities ...................................................................................................... 8 VIII. Deliverables.......................................................................................................................... 8 Appendix A - Summary Checklist............................................................................................... 9 Appendix B - Checklist Definitions ........................................................................................... 14 Appendix C - Sample PRR Sign-Off Memo ............................................................................. 24 Revision History .......................................................................................................................... 25

Page 2 of 26

Production Readiness Review Process, Version 7.0 I. LEGISLATIVE BACKGROUND The Production Readiness Review (PRR) Process supports the responsibilities of Federal Student Aid's Chief Information Officer, as described by the Clinger-Cohen Act. These include: • Developing, maintaining, and facilitating the implementation of a sound and integrated information technology architecture. • Promoting the effective and efficient design and operation of all major information resources management processes. II. PURPOSE The purpose of the PRR Process is to establish a common process that defines the activities and the roles of all groups supporting the government's decision to implement an application, release of an application, or information system. The PRR serves as the final, formal, and documented decision point before a new application or a significant release of an existing application enters Federal Student Aid's production environment and is exposed to end-users. Completion of a Production Readiness Review accomplishes the following: • Demonstrates the readiness of the application to enter production to Federal Student Aid's senior management. • Reviews the testing approach, participation, and test results to ensure that the application has been adequately tested and is ready for use by end-users. • Discloses areas of risk associated with the application moving into production and the associated risk mitigation strategies. This ensures that Federal Student Aid Management is aware of all risks associated with a release and has accepted the risk of implementing the application. For applications where users are members of the public (including students, parents, educational institutions, non-profit organizations, or corporations), completion of a Production Readiness Review acknowledges that Federal Student Aid management has accepted the risks associated with public exposure of the application. • Review of Lessons Learned and Process Improvements. • Acquire formal authorization to move the application into production as indicated by completion of a PRR Sign-Off Memorandum (Appendix C). III. APPLICABILITY The Production Readiness Review Process applies to all initial releases of new production applications or information systems. The PRR Process also applies to significant enhancements to existing applications as determined by the application owner or EOCM process. Regularly scheduled releases of applications that have end-users that are not government employees (e.g. the public, students, parents, schools, G.A.s, etc) are required to complete a PRR prior to each release. The Production Readiness Review Process is optional for emergency releases, operating system patches, minor web content updates, security patches, and upgrades to commercial-off-the-shelf software (COTS) that are less than a full version (i.e. upgrade from version 1.1. to 1.2 of a product).

Page 3 of 26

Production Readiness Review Process, Version 7.0

Completion of a PRR is required for any release, regardless of size, if it is requested by any of the following: • Federal Student Aid's Chief Operating Officer (COO) • Federal Student Aid's Chief Information Officer (CIO) • The project sponsor of the application/release entering production. • The project manager of the application/release entering production. IV. PRODUCTION READINESS REVIEW PROCESS STEPS Step 1: Preparation The project team prepares for a Production Readiness Review (PRR) by reviewing and preparing internal documentation, and reviewing the status of the application. The team also prepares a briefing and sign-off memorandum for the formal PRR presentation. It is strongly recommended that large applications or releases hold a "pre-PRR" within the project team, including the IV&V vendor and security certification and accreditation (C&A) team if applicable, to ensure that all documentation is ready for the formal PRR and that there are no outstanding issues that need to be resolved prior to the PRR. Documentation as selected by the Project Manager that may be reviewed and included on the PRR summary checklist (based on project need, but is not limited to): • Business Case • Requirements documents • Design documentation • Test plans (and comparison to requirements documents) • Test results • Security plans and documents • Configuration Management plans and documents • Disaster Recovery Plan • Risk Management Plan • Risk list or other risk tracking associated with the project • ED LCM Verification Matrix • Technical architectural review Appendix A shows a PRR Summary Checklist for use in performing a gap analysis. The gap analysis provides the basis for the risk identification and the development of the risk mitigation strategy. Appendix B shows each section’s point of contact (POC) and checklist definitions.

Page 4 of 26

Production Readiness Review Process, Version 7.0 Step 2: Collaboration The project team should collaborate with other areas of Federal Student Aid that support the project. • Identify all external organizations and representatives who will participate in the production readiness process, including data center managers, other impacted applications (if any), security staff, the CIO/Enterprise Quality Assurance Team, and others as appropriate. Coordinate with the EOCM process as required for all enterprise events associated with any operational system or application. An enterprise event is a change that impacts Federal Student Aid operations across multiple applications/systems or interfaces, including changes to enterprise architecture components, and Federal Student Aid -wide business requirements. The EOCM process will be used for all enterprise events associated with any operational system or application. An enterprise event is a change that impacts Federal Student Aid operations across multiple applications/systems or interfaces, including changes to enterprise architecture components, and Federal Student Aid -wide business requirements. Reference: EOCM Plan, Version 1.0 (May 2, 2007) Initiate discussions with each external organization and representative to identify their individual needs and requirements, a joint PRR may be appropriate. Creation of required data center documents (should be accomplished in cooperation with the appropriate data center). Comprehensive gap analysis (identification of all incomplete activities from the Appendix A – Summary Checklist, assessment of risk associated with the gap and time and schedule implications associated with eliminating the gaps. There is a cost-benefit factor to the elimination of gaps that should be considered. Not all gaps need to be eliminated, but Federal Student Aid approval is required with an action plan provided to close the gap). Risk analysis and risk mitigation strategy (based on comprehensive gap analysis). The PRR presentation outline should be discussed with the Project Manager, Project Sponsor, and the CIO prior to the formal presentation to address specific concerns that senior managers may have with the release.

• • •

• •

Step 3: Presentation and Sign-Off A Production Readiness Review meeting must be scheduled to provide a forum for formal discussion and approval of the release moving to the production environment. The Federal Student Aid Project Manager (PM) / Technical Lead is responsible for directing the preparation of the presentation and selecting the appropriate presenter(s). The presentation is presented by the Project Manager / Technical Lead to the Project Sponsor / Business Owner and Federal Student Aid's CIO. The presentation should not exceed one hour in length, including questions and answers. The presentation is an executive overview of the production readiness. Detailed supporting documentation, including but not limited to a completed PRR checklist (Appendix A), must be available at the PRR meeting to back up any claims, assertions, or metrics presented and will be made available to all attendees at the start of the presentation.

Page 5 of 26

Production Readiness Review Process, Version 7.0 V. PRR PRESENTATION OUTLINE The presentation outline may be customized to meet the needs and interests of the project sponsor and Federal Student Aid's CIO; however, all information included in the outline below must be covered, in some form, during the presentation. Critical issues may vary based on the type of release and additional information may be included in the presentation. Section 1: Business Background • Business drivers for the release • Expected Business Benefits Section 2: Schedule • Planned Development Schedule (Baseline) • Actual Schedule • Schedule information must specifically show the time that was allotted for testing activities. Section 3: Testing Summary • Type of Tests Conducted (Unit, System, Acceptance, Integration, Regression, Alpha, Beta, Performance, Stress, Capacity) • Summary of Results (number of problems, number and criticality of outstanding problems) Section 4: Collaboration and Coordination • Appropriate Security Reviews have occurred (C&A complete, if applicable) • Data Center Readiness • Applications Maintenance Readiness • Help Desk Readiness • Training • ECR / EOCM Related Activities Section 5: Independent Verification & Validation (IV&V), if applicable • Summary of approach, activities, and results. • IV&V Recommendation (Go, Go with Reservations (list reservations), No Go) Section 6: Risk Summary • Risks and mitigation strategies • Outstanding Issues/Action Items • Specific identification of known risks that are being accepted with the decision to implement the release. Section 7: Lessons Learned • General Lessons Learned • Project-Specific Lessons Learned

Page 6 of 26

Production Readiness Review Process, Version 7.0 Meeting Closure: • Completion of formal sign-off memorandum • Delivery of sign-off memorandum and supporting documentation to CIO/Enterprise Quality Assurance Team (may be completed after meeting) VI. SIGN-OFF The primary output of the Production Readiness Review is a memorandum that formally authorizes an application or release to move into the production environment. A sample sign-off memorandum is presented in Appendix C. The sign-off memorandum may be tailored to the needs of the project, subject to the following conditions: 1. Only government employees may sign the memorandum (contractors may not sign). PRR represents the government's decision to implement a release and the government's acceptance of risk associated with that implementation. It is suggested that government staff obtain a separate memo from contractors recommending go or no-go at a PRR, but the final decision to implement a release must be made by government staff. 2. The following are the minimum signatures required for sign-off: • The Project Sponsor or designee • Federal Student Aid's CIO or designee • If IV&V has participated in reviewing the release, the CIO/Enterprise Quality Assurance Program Manager or designee. • The Project Manager • The System Security Officer • Government Data Center Manager (e.g. VDC manager or designee). In the event sign-off does not occur, concurrence should be reached on the actions, activities, or deliverables that must be completed prior to implementation. At the discretion of the Project Sponsor and the CIO, sign-off may occur provided these agreed upon provision are met, or another presentation and production readiness meeting may be required. There are four potential outcomes of the sign-off: • • Unconditional Sign-off means that all required signatures are obtained at the production readiness meeting and there are no outstanding issues the must be resolved prior to implementation. Conditional Sign-off means that although all required signatures are obtained, implementation may not occur until certain conditions are met. These conditions should be written on the front of the memorandum or certificate before the signatures, or on the back of the memorandum or certificate with a notation on the front that the approval is conditional. Provisional Sign-off means that one or more signatures were NOT obtained at the production readiness review. Conditions have been identified that must be met before the missing signatures can be obtained. Once the conditions are met, the signatures can be obtained individually; an additional Production Readiness Presentation is not required.

Page 7 of 26

Production Readiness Review Process, Version 7.0 • No Sign-off. Insufficient information exists at this time to approve the system for production. No signatures are obtained and a new production readiness presentation will be required. The missing information should be clearly identified and understood so that the next Production Readiness Presentation will be successful.

VII. SIGN-OFF RESPONSIBILITIES Project Manager - The Project Manager's signature certifies that all reasonable due diligence has been exercised to assure system stability/operability, that known risks have been identified/ described in the presentation, and testing has been performed, with the results indicating that business benefit will be derived by the implementation of the system. System Security Officer - The System Security Officer's signature certifies that all reasonable due diligence has been exercised to assure system security, and known risks have been identified/described in the presentation and in the supporting documentation. Government Data Center Manager - The Government Data Center Manager's signature certifies that all data center issues and concerns have been addressed and the data center is ready to accept the system into the production environment. Data center refers to any location where a FSA system resides, including VDC, ACS, TSYS, EEMC, or others. Enterprise Quality Assurance Program Manager (IV&V) - The Enterprise Quality Assurance Program Manager's signature certifies that the independent quality assurance activities were performed and that the methods used and the findings identified are described in the presentation/supporting documentation. This signature is required only if the FSA/CIO/QA Team (and IV&V vendor) has been involved in the project’s development effort. If there has been no IV&V involvement this signature shall be marked "Not Applicable" or "N/A." Federal Student Aid's CIO - The Federal Student Aid CIO's signature certifies that all reasonable due diligence has been exercised to assure system stability, operability, and that risks identified and described in the presentation/supporting documentation are reasonable given the expected business benefit. The CIO's signature also certifies that the implementation of the application or release is in alignment with Federal Student Aid's strategy for alignment of information technology investments, as required by the Clinger-Cohen Act. Project Sponsor - The project sponsor's signature certifies acceptance of all business risks associated with implementation of the application or release. This specifically includes the risk of exposing the application or release to end users, including the public for certain releases. VIII. DELIVERABLES The Project Manager should retain the original completed Sign-off memorandum, along with detailed supporting documentation. A copy of the sign-off memorandum and supporting documentation should be delivered to the Contracting Officer Representative (COR) for the project and another copy should be delivered to the CIO/Enterprise Quality Assurance Team.

Page 8 of 26

Production Readiness Review Process, Version 7.0 - Appendix A APPENDIX A - SUMMARY CHECKLIST Purpose: The purpose of this checklist is to provide a guideline for the items that are appropriate to support a PRR. This checklist may be tailored to the particular needs of each application/release. A final version of the checklist may indicate that a gap exists (e.g. user training was not completed); in such cases the managers attending the PRR will make a decision about signing off on the PRR with a gap present. A gap in the checklist simply indicates a risk that needs to be considered prior to moving to production. COMMENTS/ TARGET GAP HOW WORK CMPL CRITERIA DESCRIPTION (Y/N) VALIDATED EFFORT DATE STATUS (Document Name) 1. INVESTMENT INFORMATION 1. IPC Approved Business Case (LCM) 2. Contract or Task Order 3. Project Plan 4. Project Schedule 2. DATA CENTER 1. Request for VDC Services Form 2. Responsibility Matrix 3. Preliminary Requirements Document (Data Center-CMDB) 4. Application Service Level Agreement (SLA) (Data Center-CMDB) 5. Maintenance and Operations Plan (LCM) 6. Implementation Plan (LCM) 7. Solution User Manual (LCM) PROPOSED RISKS RISK IDENTIFIED MITIGATION

RESP

Page 9 of 26

Production Readiness Review Process, Version 7.0 - Appendix A COMMENTS/ TARGET GAP HOW WORK CMPL CRITERIA DESCRIPTION (Y/N) VALIDATED EFFORT DATE STATUS (Document Name) 8. Data Conversion Plan (LCM) 9. System Start-up Plan (LCM) 10. Application Help Desk established 11. Service Delivery Readiness Review (SDR)/Operational Readiness Review (ORR) 12. Change Control Management 13. ITA New Application Questionnaire (only applicable to applications that reside in the ITA environment) 14. Java Code meets FSA Java Code Standards 3. SUPPORT PROCESSES 1. Requirements Management Plan 2. Requirements Traceability Matrix 3. Configuration Management Plan (LCM) 4. Version Control Procedures 5. Configuration Item Library 6. Operational Change Control Procedures 4. TECHNICAL ARCHITECTURE PROPOSED RISKS RISK IDENTIFIED MITIGATION

RESP

Page 10 of 26

Production Readiness Review Process, Version 7.0 - Appendix A COMMENTS/ TARGET GAP HOW WORK CMPL CRITERIA DESCRIPTION (Y/N) VALIDATED EFFORT DATE STATUS (Document Name) 1. Architecture Design 2. Development (i.e. coding) Standards 3. Enterprise Architecture Review 5. LICENSING 1. Software License Requirements (incl. Paid Licenses) 6. CODE REVIEW 1. User Specifications 2. Functional Specifications 3. Technical Specifications 7. SECURITY 1. Application Security Requirements 2. Security Officer Identified by appointment memo. 3. Rules of Behavior for System Users 4. Personnel Security Classifications for users, developers, testers, and others 5. Role based access identified by job position 6. Disaster Recovery/Continuity of Support Plans (LCM) 7. Data Integrity/Validation PROPOSED RISKS RISK IDENTIFIED MITIGATION

RESP

Page 11 of 26

Production Readiness Review Process, Version 7.0 - Appendix A COMMENTS/ TARGET GAP HOW WORK CMPL CRITERIA DESCRIPTION (Y/N) VALIDATED EFFORT DATE STATUS (Document Name) Controls 8. Audit Trails 9. System Security Plan (LCM) 10. Certification and Accreditation complete, or Interim Approval to Operate (IATO) memo signed. 11. Security Risk Assessment complete (LCM) 12. Mitigation Plan implemented (LCM) 13. Critical Infrastructure Protection Survey complete 14. Inventory Worksheet complete 15. MOU/MOA/SLA (if applicable) 16. Privacy Act Systems of Records Review (if applicable) 8. TESTING 1. Test Plan(s) (LCM) 2. Test Scripts 3. Test Data 4. Security Testing 5. Test Execution Results and Reports Summary (LCM) 6. Accessibility Testing 7. System Investigation Report PROPOSED RISKS RISK IDENTIFIED MITIGATION

RESP

Page 12 of 26

Production Readiness Review Process, Version 7.0 - Appendix A COMMENTS/ TARGET GAP HOW WORK CMPL CRITERIA DESCRIPTION (Y/N) VALIDATED EFFORT DATE STATUS (Document Name) (SIR) Log (Defect Log). 8. Client and User Sign-Off of all testing 9. APPLICATION TRAINING 1. Training Plan (LCM) 2. User Training Conducted 3. User Installation and Setup Procedures 4. On-going Training Function Available 10. SUPPORT STAGE 1. Open SIR Responsibility Identified and Agreed Upon 2. Support available for Software Package 3. Organizational Design and Skills Identified 4. Knowledge Transfer Plan 5. Post-Implementation Review (PIR) Advance Packet received PROPOSED RISKS RISK IDENTIFIED MITIGATION

RESP

Page 13 of 26

Production Readiness Review Process, Version 7.0 - Appendix B APPENDIX B - CHECKLIST DEFINITIONS

CRITERIA DESCRIPTION

Point of Contact (POC)/Definition

1. INVESTMENT INFORMATION 1. IPC Approved Business Case POC: Project Manager Business Case that was developed for the IT Investment and was approved by the IPC (OMB 300, Business Case, Reference: ACS Directive OCIO:1-106 - Lifecycle Business Case Light, etc). Management (LCM) Framework. Page 7 of 27, Table 1. 2. Contract or Task Order POC: Project Manager Contract vehicle to obtain required services or development effort. 3. Project Plan POC: Project Manager Project plan document. 4. Project Schedule POC: Project Manager Project schedule with start, end, and milestone dates. 2. DATA CENTER General POC: FSA/CIO/IT Services for VDC (Slawko Senaszczuk, Yolanda Brooks - Alternate) Note: For applications/releases in the VDC, see the VDC Migration Checklist to cover the items listed below. For applications/releases in other data centers, different terms may be used for the items below. POC: Project Manager/Data Center Manager This form requests an overview of services required from the VDC. Note that this form is VDC-specific and may not apply to other data centers. POC: Project Manager/Data Center Manager The matrix indicates which role an organization has during a development effort. (Primary Lead, Support Role, or Joint Responsibility) POC: Project Manager/Data Center Manager This document specifies what Technical Support, Operations, Automation, and Communications are required for the new application. The data center environment needed by the application and estimated number of users are also described. (Ref: VDC Configuration Management Database Data Dictionary, Version 1.0 (May 22, 2007))

1. Request for VDC Services Form

2. Responsibility Matrix

3. Preliminary Requirements Document (Data Center-CMDB)

Page 14 of 26

Production Readiness Review Process, Version 7.0 - Appendix B

CRITERIA DESCRIPTION Point of Contact (POC)/Definition 4. Application Service Level POC: Data Center Manager Agreement (SLA) This document defines production service level (Data Center-CMDB) requirements between FSA business owner and the Data Center for the application in the production environment. (Ref: VDC Configuration Management Database Data Dictionary, Version 1.0 (May 22, 2007)) 5. Maintenance and Operations POC: Project Manager Plan Key components of this plan include: Plan to maintain the Reference: ACS Directive technical solution, Points of Contact (Call Out List), OCIO:1-106 - Lifecycle Notification and Escalation Process with Issues Management (LCM) Resolution, Job Execution Workflow Schedule, Run Framework. Page 14 of 27, Book, and Restart Procedures. Table 13. 6. Implementation Plan POC: Project Manager Reference: ACS Directive OCIO:1-106 - Lifecycle Management (LCM) Framework. Page 14 of 27, Table 12. 7. Solution User Manual Reference: ACS Directive OCIO:1-106 - Lifecycle Management (LCM) Framework. Page 14 of 27, Table 12. 8. Data Conversion Plan Reference: ACS Directive OCIO:1-106 - Lifecycle Management (LCM) Framework. Page 14 of 27, Table 12. 9. System Start-up Plan Reference: ACS Directive OCIO:1-106 - Lifecycle Management (LCM) Framework. Page 14 of 27, Table 12. The implementation plan should include (but not limited to) documentation of the configuration of any software required to support the application in the production environment. This plan also documents the steps required to build the configuration of the application in the production environment. POC: Project Manager

POC: Project Manager

POC: Project Manager

Page 15 of 26

Production Readiness Review Process, Version 7.0 - Appendix B

CRITERIA DESCRIPTION Point of Contact (POC)/Definition 10. Application Help Desk POC: Project Manager established Identify the help desk location and phone number to assist users with resolving application errors, issues and concerns. 11. Service Delivery Readiness POC: Project Manager/Data Center Manager Review (SDR) /Operational Data Center's review of operational support for the Readiness Review (ORR) application in the production environment. It is generally recommended that the ORR be completed prior to the PRR; however in some circumstances it may be appropriate to conduct the PRR first. PRR represents the government's approval to move to production while the ORR ensures that the Data Center provider is prepared to receive the application/release. 12. Change Control POC: Data Center Manager Management Process to manage change requests required by the application in the production environment at the data processing facility. 13. ITA New Application POC: ITA Lead (Sandy England) Questionnaire (only applicable If the application is to reside in the ITA environment, a to applications that reside in the New Application Questionnaire should be completed. ITA environment) 14. Java Code meets FSA Java POC: ITA Lead (Sandy England) Java Code should meet Federal Student Aid's Java Coding Code Standards Standards. 3. SUPPORT PROCESSES 1. Requirements Management POC: Project Manager Plan Plan that documents how the requirements process is managed. 2. Requirements Traceability POC: Project Manager Matrix Matrix that traces high-level requirements to detailed requirements then to specific test cases and the results of testing. 3. Configuration Management POC: Project Manager Plan Configuration Management (CM) is the process of identifying, organizing and managing the integrity of the Reference: ACS Directive project work products throughout the project’s life cycle. OCIO:1-106 - Lifecycle Key Components include: Baseline Work Products, Mechanism to Track and Control Changes, Change Management (LCM) Framework. Page 10 of 27, Requests, and Mechanism to Establish and Maintain Table 6. Baseline Integrity. 4. Version Control Procedures POC: Project Manager Procedures that describe how document and code versions will be managed.

Page 16 of 26

Production Readiness Review Process, Version 7.0 - Appendix B

CRITERIA DESCRIPTION Point of Contact (POC)/Definition 5. Configuration Item Library POC: Project Manager A configuration item library is a repository for storing configuration items and their associated records. This library includes source code. 6. Operational Change Control POC: Project Manager Procedures Procedures to ensure application changes are coordinated and documented. 4. TECHNICAL ARCHITECTURE 1. Architecture Design POC: Project Manager Description of the development, test, and production environments available in project documentation. 2. Development (i.e. coding) POC: Project Manager Standards Verification of appropriate coding standards used. Reference particular standards used. 3. Enterprise Architecture POC: FSA/CIO Enterprise Architecture (Carol review Seifert) The business case includes both the business goals and a technical architecture, including enumeration of software, hardware, etc. The enterprise architecture review checks to see that: • the business case is aligned with the Enterprise Vision and is not duplicative, • the described scope adequately addresses business needs, • the business case recognizes potential external-toprogram impacts, • the business case include both an “as-is” and “tobe” architecture, • the technical architecture is reasonable for meeting the described business need and scope, • the technical architecture conforms to the technology standards, • the supporting documentation has been provided, and that the business case is consistent with ED and FSA Guiding Principles. See “EA Review Template” 5. LICENSING 1. Software License Requirements POC: FSA/CIO/IT Services (VDC Mainframe Christine Williams, VDC Middleware/ITA - Yvette Bryant)

Page 17 of 26

Production Readiness Review Process, Version 7.0 - Appendix B

CRITERIA DESCRIPTION

Point of Contact (POC)/Definition Licenses are in place and verification that licenses are current.

6. CODE REVIEW 1. User Specifications

POC: Project Manager Documentation available for review at PRR. 2. Functional Specifications POC: Project Manager Documentation available for review at PRR. 3. Technical Specifications POC: Project Manager Documentation available for review at PRR. 7. SECURITY General POC: FSA/CIO/Security & Privacy (Bob Ingwalson, FSA's CSO) 1. Application Security POC: Project Manager/Project SSO Each system must have the appropriate security Requirements requirements included in the system design. Security requirements come from Federal, Departmental, and FSA policy and also from application or functional requirements. These security requirements should also trace through the test documentation and trace to the appropriate test cases through the Requirement Traceability Matrix (RTM). 2. System Security Officer POC: Project Manager/Project SSO Identified by appointment Every FSA system must have a System Security Officer memo. (SSO) assigned to coordinate all security aspects of the system. The SSO must be assigned in writing. 3. Rules of Behavior for POC: Project Manager/Project SSO Rules of Behavior is a document that users sign, stating System Users which actions are allowed or not allowed on the system and the consequences for violating the Rules of Behavior. Consequences should be system related (example: denied access – [not will be terminated]) 4. Personnel Security POC: Project Manager/Project SSO Classifications for users, Individuals requiring access to an FSA system must have developers, testers, and others appropriate clearances. Depending on the sensitivity of the data being accessed and the ability of the individual to bypass security controls, the individual will need a low, medium or high (1C, 5C, or 6C) clearance. Specific guidance can be found in the Department’s Handbook 11. 5. Role based access identified POC: Project Manager/Project SSO by job position Each job position must be categorized based on role, and system access and privileges must be based on this categorization. This information must be documented, at a minimum, in the System Security Plan. 6. Disaster POC: Project Manager/Project SSO

Page 18 of 26

Production Readiness Review Process, Version 7.0 - Appendix B

CRITERIA DESCRIPTION Point of Contact (POC)/Definition Recovery/Continuity of All tier 3 and 4 systems must have Disaster Recovery and Support Plans Continuity of Support Plans. Tier 1 and 2 systems must have Continuity of Support Plans. Additional guidance is provided in the Department’s IT Contingency Planning Reference: ACS Directive OCIO:1-106 - Lifecycle Guide and FSA’s IT Contingency Planning Checklist. Tier 0 systems should have its General Support System Management (LCM) provide documented DR/COS support. Framework. Page 12 of 27, Table 9. 7. Data Integrity/Validation POC: Project Manager/Project SSO Controls Describe the system’s data integrity requirements. Describe the controls in place to ensure data integrity in storage and in transit. 8. Audit Trails POC: Project Manager/Project SSO System documentation must describe the system’s ability to log system activity and preserve such logging data, in compliance with the FSA IT Security and Privacy Policy. 9. System Security Plan POC: Project Manager/Project SSO Each Tier 1-4 system must have a document containing a Reference: ACS Directive system’s security controls and procedures. ED requires OCIO:1-106 - Lifecycle this document to be formatted to comply with NIST 80018 Guide for Developing Security Plans for Information Management (LCM) Technology Systems. Tier 0 systems either need their Framework. Page 12 of 27, Table 9. own plan or ensure they are included in their General Support System’s plan. 10. Certification and POC: Project Manager/Project SSO Accreditation complete, or All Tier 1- 4 systems must receive certification and Interim Approval to Operate accreditation prior to operating. Use ED’s Certification memo signed by the system and Accreditation Guide to complete this process. A owner system may receive an interim approval to operate even though all low risk findings are not mitigated but the DAA is willing to allow the system to operate for a limited period while the findings are corrected. 11. Security Risk Assessment POC: Project Manager/Project SSO complete All tier 1-4 systems must complete independent risk assessment consistent with ED’s risk assessment guide Reference: ACS Directive and FSA’s Risk Assessment template. OCIO:1-106 - Lifecycle Management (LCM) Framework. Page 12 of 27, Table 9. 12. Mitigation Plan POC: Project Manager/Project SSO implemented After completing a risk assessment, each system must evaluate the findings and develop a Corrective Action Reference: ACS Directive Plan (CAP) or mitigation plan. Before PRR, the system

Page 19 of 26

Production Readiness Review Process, Version 7.0 - Appendix B

CRITERIA DESCRIPTION OCIO:1-106 - Lifecycle Management (LCM) Framework. Page 12 of 27, Table 9. 13. Critical Infrastructure Protection Survey complete

Point of Contact (POC)/Definition must mitigate all medium and high risks found during the assessment.

POC: Project Manager/Project SSO The CIP survey is a form designed to assess the system’s impact on the mission(s) of the Department and FSA. From the responses to the survey, FSA CIO will calculate the system’s criticality (Mission Critical, Important, Supportive). This criticality will be used to complete the system’s criticality section in the Inventory worksheet. 14. Inventory Worksheet POC: Project Manager/Project SSO To assess a system’s data sensitivity and system complete criticality, each system must complete an Inventory Worksheet. The worksheet will describe the confidentiality, integrity, and availability requirements of the system, and the worksheet will classify the system as a general support system, major application, or an application. The output from the worksheet will define the tier level of the system and what security controls and documentation is required for the system. 15. MOU/MOA/SLA (if POC: Project Manager/Project SSO applicable) Each system must have documented Memoranda of Understanding (MOU’s), Trading Partner Agreements and/or Interconnection Security Agreements with every system that connects, shares data, or interfaces with this system. 16. Privacy Act Systems of POC: Project Manager/Project SSO Records Review (if applicable) Each system must complete a Privacy Impact Assessment to evaluate any Privacy Act data maintained in the system and to determine if the system is a system of record. If Privacy Act data is maintained in the system, a notice must be posted in the Federal Register before the system becomes operational. 8. TESTING 1. Test Plan(s) POC: Project Manager Approved testing plan for all levels and types of testing. Reference: ACS Directive (Sample Format: Standard IEEE 829) OCIO:1-106 - Lifecycle Management (LCM) Framework. Page 12 of 27, Table 10.

Page 20 of 26

Production Readiness Review Process, Version 7.0 - Appendix B

CRITERIA DESCRIPTION Point of Contact (POC)/Definition 2. Test Scripts POC: Project Manager All test scripts utilized during testing. 3. Test Data POC: Project Manager Test Data, under control and sanitized. 4. Security Testing POC: Project SSO Self identified. Test the security control for the system – each system could have different tests. Minimum test: access controls using password strength and the controls to limit access by user type. Any change to the existing system, which impact system requirements for access control and data security protection. 5. Test Results and Reports POC: Project Manager Summary The Test Summary brings together all pertinent information about the testing, including an assessment Reference: ACS Directive about how well the testing has been done, the number of OCIO:1-106 - Lifecycle incidents raised and outstanding, and crucially an Management (LCM) assessment about the quality of the system. Also recorded Framework. Page 13 of 27, for use in future project planning is details of what was Table 11. done, and how long it took. 6. Accessibility Testing: POC: ED/OCIO/ATG (Don Barret) All applications being integrated into the ED operating Section 508 requirements tested environment; EdNet, Ed.gov, ConnectEd, Federal Student and verified by ED Assistive Aid Data Centers, and stand-alone desktops, must be tested for accessibility. Each product, application, and/or Technology Group (ATG) located in FOB-6. POC: Joe web page (software) is tested to determine how accessible Tozzi the software’s business functions are and to determine if it is accessible to the disabled using the assistive technology currently in use at the Department and how well the software meets the ED Requirements for Accessible Design and the federal accessibility standard; Electronic and Information Technology Accessibility Standards, 36 CFR Part 1194. What/when is tested: Pre-acquisition testing of COTS products to determine degree of accessibility BEFORE purchase, post-acquisition/CCRB testing before integration, and for in-house developed software, interim testing during the build/test phases. Testing is performed with the software sponsor, the developer, and the OCIO AT Team testers, including disabled testers present. Where remediation is found to be necessary, additional tests must be performed.

Page 21 of 26

Production Readiness Review Process, Version 7.0 - Appendix B

CRITERIA DESCRIPTION

Point of Contact (POC)/Definition Any exemption justification must include an AT Accessibility Review Report of all products considered to meet the agency need.

See OCIO Directive; OCIO: 3:105 Procuring Electronic and Information Technology (EIT) In Conformance with Section 508 of the Rehabilitation Act of 1973, as amended, for guidance. 7. System Investigation Report POC: Project Manager (SIR) Log (Defect Log). All defects accounted for and disposition noted prior to PRR. 8. Client and User Sign-Off of POC: Project Manager all testing Test results completed and approved by Project Manager and User Acceptance Testing participants. 9. APPLICATION TRAINING 1. Training Plan POC: Project Manager Plan documenting all training activities for the Reference: ACS Directive application/release. OCIO:1-106 - Lifecycle Management (LCM) Framework. Page 14 of 27, Table 12. 2. User Training Conducted POC: Project Manager Any new functionality for new release that affect relevant system operation and written procedures. 3. User Installation and Setup POC: Project Manager Procedures Any new functionality information for installation set-up must be documented. 4. On-going Training Function POC: Project Manager Available Any on-going training, including security is documented. 10. SUPPORT STAGE 1. Open SIR Responsibility Identified and Agreed Upon 2. Support available for Software Package 3. Organizational Design and Skills Identified 4. Knowledge Transfer Plan POC: Project Manager List of known risks agreed to be resolved during and after implementation. POC: Project Manager Maintenance support for the application in the production environment. POC: Project Manager List of specific and unique skills within FSA necessary to maintain the application in the production environment. POC: Project Manager

Page 22 of 26

Production Readiness Review Process, Version 7.0 - Appendix B

Point of Contact (POC)/Definition Plan for transition the application from the development and testing environments to the staging and production environments. 5. Post-Implementation Review POC: FSA/CIO/QA Team (Francis Tang) (PIR) Advance Packet received The advance packet helps the project team preparing in advance the PIR after deployment of the project. The checklist for PIRs and the list of documents required to support PIRs are, especially, useful.

CRITERIA DESCRIPTION

Page 23 of 26

Production Readiness Review Process, Version 7.0 - Appendix C APPENDIX C - SAMPLE PRR SIGN-OFF MEMO

PRR Sign-Off Memorandum
[Date] This certifies that the _[application/release name]_ has been appropriately tested and that all known risks have been disclosed to Federal Student Aid's Management. By signature below, Federal Student Aid accepts the risks associated with implementing this release in the production environment. Development Project Manager: ________________________ {Name} {Title}

System Security Officer:

________________________ {Name} {Title}

Government Data Center Manager:

________________________ {Name} {Title}

Enterprise Quality Assurance (IV&V): (If QA/IV&V is supporting project)

________________________ {Name} {Title}

Federal Student Aid's CIO:

________________________ {Name} {Title} ________________________ {Name} {Title}

Project Sponsor:

Sign-off Type:

Unconditional

Conditional

Provisional

No Sign-off

Page 24 of 26

Production Readiness Review Process, Version 7.0 - Change History REVISION HISTORY
Revision # Date Change Description/Purpose

1.0 - 3.0 4.0 - 5.0 6.0 6.0 6.0

12/28/2005 9/18/2006 9/18/2006 9/18/2006 9/18/2006

6.0 6.0 6.0 6.0 6.0 6.0 6.0

9/18/2006 9/19/2006 9/19/2006 9/19/2006 9/19/2006 9/19/2006 9/19/2006

6.0

9/19/2006

6.0 6.0 6.0 7.0

9/19/2006 9/19/2006 9/25/2006 5/17/2007

For revision history of Versions 1.0 - 3.0, see Version 5.0, Release 1 For revision history of Versions 4.0 - 5.0 Release 2, see Version 5.0, Release 2 Removed table of contents. Removed "Process Sponsor" section. Removed extraneous citations to laws, policies, and regulations, including: GPRA, Circular A-11, Circular A-130, and GAO ITIM Framework for Assessing Process Maturity. Revised "Legislative Background" Section to focus on most relevant legislation, the Clinger-Cohen Act. Updated headings to allow for an automatically generated table of contents rather than a manual one. Created table of contents. Revised "Applicability" section based on CIO Focus Group comments to clarify which releases are covered by PRR. Revised the "Purpose" section based on CIO Focus Group comments to clarify the reasons for conducting a PRR. Removed "Responsibilities" section. Removed "Source Documents" section. Modified "Process" section, resulting in expansion to four new sections: "Production Readiness Review Process Steps," "Presentation Outline," "Sign-off," and "Sign-off Responsibilities." These sections are largely a re-work of the previous "Process" section combined with feedback from the CIO Focus Group. Slight modification to clarify "Deliverables" section. Indicating that Project Manager should keep original sign-off memo and that COR and QA Team get a copy. Updated Appendix A and Appendix B based on feedback from CIO Focus Group and associated follow-up. Updated Appendix C with new sample sign-off memo based on CIO Focus Group feedback. Updated Appendix A and Appendix B to bring item names and definitions in-line with the LCM Framework. Updated document: • Removed language in applicability section about financial impact triggering a PRR. • Add term “Service Delivery Readiness Review (SDR)” in addition to ORR (SDR is the term used by Perot in the new VDC).

Page 25 of 26

Production Readiness Review Process, Version 7.0 - Change History 7.0 6/19/2007 Updated document: • Added bullet under Step 2: Collaboration to describe EOCM coordination activities and the concept of Joint PRRs. • Added terms Technical Lead and Business Owner to Step 3: Presentation and Sign-Off from VDC Configuration Management Database Data Dictionary, V.1 (May 22,07) • Added Training and ECR / EOCM Related Activities to Section V. sub-section 4, page 6. • Removed the word “user” from Section VII. Sign-Off Responsibilities under Project Manager component. • Added to Appendix A – Summary Checklist: 1.Data Center Items 3 & 4 – Reference to CMDB • Added to Appendix B – Checklist Definitions: 1. Data Center Items 3 & 4 – Reference to CMDB 2. Testing, Item 1 – Reference to IEEE Standard 829 • Added to Appendix C – Sample Sign-off Memo: 1. Reference “If QA/IV&V is supporting project”

Page 26 of 26