Southwest Texas State University

Internal Audit and Advisory Services

Disaster Recovery and Business Continuity Planning Audit

May 2, 2003 SWT is a member of the Texas State University System

Internal Audit and Advisory Services Dr. Denise M. Trauth President Re: Disaster Recovery and Business Continuity Planning Audit

May 2, 2003

Internal Audit and Advisory Services has completed a Category II audit of Disaster Recovery and Business Continuity Planning. The audit was conducted as a part of the FY 2003 Audit Plan for Southwest Texas State University. The original objective of this audit was to assess the preparedness of the University in responding to disasters. It was later expanded to include business continuity planning and ensuring compliance with recent laws, such as the Public Health Security and Bioterrorism Preparedness Act of 2002. The objectives were accomplished by conducting interviews with management; and confirming readiness in various areas of disaster recovery and business continuity per University policies and procedures and research from other sources. The scope initially focused on information technology but was expanded to include other areas that may create and/or be impacted by a disaster. A prior audit of hazardous materials resulted in additional staff for the Risk Management Department, and a follow-up audit is currently being conducted to determine the status of recommendations. A representative of the State Fire Marshall’s Office just completed an assessment of fire safety compliance at the University, and a report will be issued. In addition, the status of the 1995 application impact analysis recommendations made by SunGard Planning Solutions, Inc., was assessed. The recent focus on terrorism in the United States resulted in increased campus awareness, and updates were made of some emergency policies during the conduct of the audit. The following areas of concern were noted during the audit: • The official University policies for security of SWT Information Resources has not been updated since 1996, including the related Information Resources Security Manual. There is a brief description of the procedures used for recovery and backup on the Technology Resources website dated June, 2002, but it is not included in official University policies. Although a new Operations Command Center was included in the new Police Department facility that opened last summer, there has not been a practice run or disaster simulation involving University or community emergency personnel in several years.

Page 2 Dr. Denise M. Trauth May 2, 2003 • • No business impact analyses for all departments have been conducted to assist in business continuity planning, although a limited number were assessed in 1995. Study Abroad and international students are not fully incorporated into all University records to ensure immediate recognition of their status by University officials should problems arise either on campus or overseas. There is no contingency fund to cover liability issues, such as providing assistance to a student who becomes ill or requires surgery overseas.

The details of these issues are provided on the following pages. We appreciate the cooperation and suggestions of all the individuals involved in the assessment of the University’s level of preparedness. Sincerely,

Yvonne Eixmann Director c: Dr. Michael L. Abbott Dr. Robert D. Gratz Mr. Gerald W. Hill Mr. William A. Nance Dr. James D. Studer Dr. Carl Van Wyatt Mr. J. Wiley Thedford Mr. Jerry W. Neef

Southwest Texas State University
601 University Drive San Marcos, Texas 78666-4615 Telephone: 512-245-2533 Fax: 512-245-3620 SWT is a member of the Texas State University System


1. In 1995, the University contracted with SunGard Planning Solutions to conduct a Business Impact Analysis of mission critical functions. In response to the analysis, SunGard then worked with Computing Services’ management to prepare an Administrative Computing Disaster Recovery Plan. In 1996, Computing Services management established University Policy and Procedure Statement (UPPS) 04.01.01 Security of SWT Information Resources, including a supporting Information Resources Security Manual. UPPS 04.01.01 and Attachment I, the Information Resources Security Manual have not been updated. According to the Director, Technology Resources Operations, the Technology Resources Disaster Recovery Coordinator is responsible for keeping the Administrative Computing Disaster Recovery Plan up-to-date, but a Work Group Team Leader is responsible for the identification of all modifications and enhancements which may be required to continually and accurately reflect their Recovery Team’s recovery responsibilities and procedures. The Technology Resources Incident Management Team provides overall coordination of response and recovery support activities for the resumption of administrative computing. If an incident occurs, the team evaluates which response and recovery actions should be invoked and activates the corresponding Recovery Teams. The last formal drill was May 16, 1996. However, the configuration of the cluster and mirrored disk systems applications are accessed on a daily basis in the same manner as during a formal test. Recommendation: The UPPS and its attachment should be reviewed and updated to better reflect current operations and procedures. In addition, consideration should be given to having a centralized procedure within the policy for ensuring that all Work Group Recovery Plans are up to date and any exposures have been rectified to ensure a viable state of readiness. This recommendation should be extended to incorporate any changes that are required as a result of the future administrative system currently being implemented, such as the reduced redundancy. Management’s Response: Management agrees. The UPPS is under review which should be completed by September 30, 2003. The Information Resources Security Manual needs extensive revision that will be completed by January 1, 2004. We propose to remove the manual as a policy attachment and simply provide a link to a web page which will include the manual for easy access and update.


2. The Crisis Management Center in the Math and Computer Science Building is the backup facility for administrative computing resumption should the primary facility at J.C. Kellam become unavailable. In an application impact analysis completed by SunGard Planning Solutions in 1995, one of the requirements for endorsing an internal alternate site recovery was sufficient, dedicated recovery hardware resources that are installed and maintained at the alternate site. This requirement has been met according to Information Technology personnel. The other requirement was that the strategy should be successfully tested and the results verified by an independent University third party on an annual basis, but this has not occurred. The Internal Auditor was involved in the last formal drill on May 16, 1996. In the event of a disaster impacting the entire campus or particularly these two facilities, no other “hotsite/coldsite” is readily available to mitigate the loss of business continuity. Given the increasing presence and reputation of the University in the community since the release of this study eight years ago, the risk of not having of a hotsite/coldsite off campus has become more critical. The Department of Information Resources operates the West Texas Disaster Recovery and Operations Center (Center) on the campus of Angelo State University, but the University does not utilize the Center for testing disaster recovery plans, for disaster recovery services, or for data center operations. Recommendation: The University should consider alternative “hotsites/coldsites” off campus to help ensure recovery capabilities. The cost to mitigate loss of technological resources should be considered in comparison to the cost of a significant service disruption exceeding 72 hours. No appropriated funds may be expended for this purpose without a waiver from the Legislative Budget Board. Management’s Response: Management agrees in principle with this recommendation. Alternative hot/cold sites were presented to the President’s Cabinet in 1995. It was decided that with the separate centers on campus and the low risk of both centers becoming inoperative for over 72 hours that risk versus cost of hot site/cold site was not justified. Technology Resources’ staff has attempted on several occasions to obtain cost information from the Department of Information Resources (DIR) Disaster Recovery Services for the West Texas Disaster Recovery and Operations Center, but was unable to obtain the pricing needed to make a cost/benefit risk assessment. Technology Resources’ management will solicit the information again and will obtain cost estimates for a hot site/cold site from other service providers. The information will be provided to management for evaluation by January 1, 2004. 3. The UPPS 05.04.03 Emergency Operations and its attachments, Crisis Management Plan and the Crisis Management Plan for Study Abroad, were


updated in March, 2003, as a reflection of the University’s concern about the increased terrorism threats in the United States. This past year, an Operations Command Center (OCC) was opened within the University Police Department’s new facility. The OCC has an uninterrupted power supply for all critical computers in the Center. The University’s 911 service is a duplicate of the city of San Marcos’ system, and either can take over for the other at any time. In addition, UPD’s radio frequency is the same as the city and Hays County. The 1998 flood and the 2002 flooding of College Inn activated parts of the plan, but a comprehensive practice drill involving all players at the University and within the community has not occurred since 1996. However, since the Operations Command Center became operational, there has been no practice run or disaster simulation. Recommendation: The University should, at a minimum, schedule a practice drill at the Operations Command Center to ensure the coordination of effort results in an effective disaster recovery that mitigates the risk to University facilities and students, faculty and staff. Consideration should also be given to organizing a disaster recovery simulation with city and county personnel. Management’s Response: Management concurs. Regular practice drills will be held and the participation of the city and county will be sought. The next practice drill will be conducted by December 31, 2003. 4. The 1995 Sungard Application Impact Analysis included an in depth review of a limited, albeit critical, number of departments. The Administrative Computing Disaster Recovery Plan was subsequently developed for recovering centrally administered data storage, programs, and processing capability. However, this plan does not address the needs of individual operating units beyond the restoration of access to their critical centrally administered applications. As servers and other independent systems have proliferated over the years, these individual needs have become more critical. The Information Resources Security Manual encourages all university departments to develop individual service disruption plans for protecting their information resource assets and operating capability, but no such program has been undertaken for the University as a whole. Business continuity is dependent on more than technological resources restoration. It also involves requirements such as new locations, vital records, supplies, equipment needs (hardware and other), expenses for relocation and vendor commitment information.


Recommendation: Consideration should be given to conducting business impact analyses for each department on campus to ensure comprehensive coverage of business continuity issues at the University. Such an analysis should also be beneficial in increasing awareness of potential problems and providing opportunities for training in various emergency procedures. If such an endeavor were undertaken, the University should fund a Business Continuity Office with full time staffing. As mentioned earlier, business continuity planning is dependent on far more than technological resources. Management Response: Management has no disagreement with this recommendation. The Information Resources Security Manual provides guidelines to university departments and places responsibility on the individual units for having internal contingency plans. We agree with the recommendation that business continuity planning is dependent on far more than technological resources recovery. Consequently, these plans must be developed by the departments. The Administrative Computing Disaster Recovery Plan was intended to provide recovery for critical core MIS application. Auditor’s Comments: Although guidelines are provided to university departments, there is no assurance that individual units are developing internal contingency plans. Given current budget constraints, the hiring of a Business Continuity Office may not be feasible at this time. However, someone or some office must be responsible for verification of departmental development of plans. 5. The UPPS 05.04.03 Attachment X addresses the Crisis Management Plan for Study Abroad. It is currently being updated and reflects much research into the Study Abroad programs at other universities. In the recent past, foreign exchange students at the University didn’t have the same privileges as other students, but they are now officially enrolled and well supported by the International Office. However, the extension courses utilized for Study Abroad programs are not generating semester credit hours and not used for formula funding, so the students are in a nonreportable class. Anything off-campus can be incorporated into the system, but Study Abroad students are still often not included. If there is an emergency, the University may not be able to identify the individual as an SWT student. Some personnel in the Registrar’s office recognize this problem and know where to look in the system, but if it is after hours or an individual is not familiar with the problem, they may not be able to locate the student. In addition, there is no contingency fund available to protect the University’s reputation and cover potential liabilities. At some other schools, the common practice is to send a university representative to accompany the student’s family and stay with them until the situation is resolved.


But at the University, individuals who are contacted to assist students in Study Abroad programs have to fund the expenses out of their own pockets. Recommendation: Further efforts should be made to ensure all Study Abroad students can be identified through the system. In addition, consideration should be given to having all incidents involving Study Abroad students should be initially directed to the University Police Department, particularly those involving drug related problems. A source of University contingency funds should be made available for emergencies involving Study Abroad students. Management’s Response: Management agrees that efforts need to be made to ensure that all students enrolled in faculty-led study abroad programs offered through extension can be identified through the system. The Director of Extended and Distance Learning, the Director of Correspondence and Extension, the Registrar, and Associate Vice President for Academic Affairs will develop a strategy to address this concern. We believe that some of the system modifications already made in implementing SEVIS to identify and track ESL students may facilitate a short-term solution to this problem while a long-term solution that is a part of our new Student Information System is developed. Management also agrees that all incidents involving Study Abroad students are directed to the University Police Department initially. The Office of Extension and Correspondence will review current procedures in an effort to identify any additional steps that can be taken to address this concern. And, as part of the deliberations regarding the fiscal year 2004 annual operating budget and the possible reallocation of existing funds, consideration will be given to setting up a regular contingency from Institutional Funds for the purposes noted in the recommendation.