Final Audit Report Knox County Internal Audit File Number 02-2014-01 Knox County Internal Audit TABLE OF CONTENTS EXECUTIVE SUMMARY BACKGROUND 1 OBJECTIVES, SCOPE, AND METHODOLOGY 1 IDENTIFIED GOOD OPERATING PRACTICES 2 DEFINED USER ROLES IN THE JUSTICE INFORMATION MANAGEMENT SYSTEM (JIMS) 2 DOCUMENT PREPARATION FOR DAILY COURT PROCEEDINGS 3 FINDINGS 3 QUALITY ASSURANCE CONTROLS IN THE CRIMINAL COURT CLERK AND GENERAL SESSIONS COURT CLERK OFFICES SHOULD BE STRENGTHENED 3 ORIGINAL DOCUMENTS IN GENERAL SESSIONS COURTS SHOULD BE BETTER PROTECTED 4 INTERNAL PROCESSES SHOULD BE DOCUMENTED 4 JIMS TRAINING PROGRAM AND MATERIALS SHOULD BE DOCUMENTED 5 TRAINING PROGRAMS AND MATERIALS IN CRIMINAL COURT CLERK AND GENERAL SESSIONS COURT CLERK OFFICES SHOULD BE DOCUMENTED 5 OBSERVATIONS 5 IMPROVED SYSTEM INTEGRATION 6 REAL-TIME DATA ENTRY 6 RECOMMENDATIONS 7 MANAGEMENT RESPONSE 7 APPENDICES APPENDIX A CRIMINAL COURT / GENERAL SESSIONS COURT CLERK MANAGEMENT RESPONSE APPENDIX B CRIMINAL COURT MANAGEMENT RESPONSE APPENDIX C GENERAL SESSIONS MANAGEMENT RESPONSE APPENDIX D INFORMATION TECHNOLOGY MANAGEMENT RESPONSE APPENDIX E ATTORNEY GENERAL MANAGEMENT RESPONSE
File Number 02-2014-01
EXECUTIVE SUMMARY Criminal Justice Process Audit
WHY WE DID THIS AUDIT The audit was conducted as a result of a request from the Knox County Commission through Resolution R-14-1-905.
WHAT WE FOUND We found the criminal justice process could be improved to operate more efficiently and effectively, with one significant issue identified and four other areas where improvement is needed. Specifically, we found needed improvements related to: Quality assurance controls in the General Sessions Court Clerk- Criminal Division (General Session Court Clerk) and Criminal Court Clerk offices. Protection of original documents in General Sessions Court proceedings. Documentation of departmental internal criminal justice processes. Training programs and materials for JIMS training. Training programs and materials for General Sessions Court Clerk and Criminal Court Clerk offices.
We also identified two areas we consider good operating practices related to defined JIMS user roles and daily document preparation. We also observed two areas for management to consider for improving the process related to better system integration with JIMS and entering data in real time.
WHAT WE RECOMMEND We recommend all departments involved in the process (1) implement stronger document management to better protect original documents in the General Sessions Courts, (2) document internal criminal justice processes, and (3) work with Information Technology to develop a better defined system training program and manual. We recommend the Criminal Court Clerk and General Sessions Court Clerk offices (1) identify critical points in their process where quality controls would improve the accuracy and timeliness of information entered into JIMS, (2) implement quality assurance controls for the identified areas, and (3) better define training requirements and programs for all employees.
BACKGROUND The criminal justice system serves the approximately 430,000 Knox County citizens by handling tens of thousands of criminal cases each year. Recently, problems in the criminal justice system have received increased media coverage.
The process in Knox County includes multiple County departments including the General Sessions Courts, Criminal Courts, Attorney Generals Office, General Sessions Court Clerk Criminal Division, and Criminal Court Clerk. The Information Technology Department assists the process through the design and support of an in-house system, the Justice Information Management System (JIMS), to manage criminal justice information.
The objective of the audit was to evaluate the efficiency and effectiveness of the current criminal justice process. The scope of the audit was limited to the process in place at the initiation of the audit (March 13, 2014) to handle a defendants case from the time the warrant is officially issued until the case is completed. The audit methodology included obtaining and reviewing applicable documentation, conducting interviews with key personnel, performing process walkthroughs, and observing court proceedings.
File Number 02-2014-01 Page i
Knox County Internal Audit BACKGROUND The criminal justice system serves the approximately 430,000 1 Knox County citizens by handling criminal cases in seven courts, including four criminal divisions of General Sessions Court and three divisions of Criminal Court. According to the General Sessions Court Clerk Criminal Division (General Sessions Court Clerk) office, in 2013 dispositions were entered for more than 36,000 cases passing through the Knox County criminal justice system. Due to recent increased media scrutiny regarding potential problems in the criminal justice system, this audit was conducted at the request of the Knox County Commission through Resolution R-14-1-905.
The process in Knox County includes multiple County departments including the General Sessions Courts, Criminal Courts, Attorney Generals Office, General Sessions Court Clerk, and Criminal Court Clerk. The Information Technology Department assists the process through the design and support of an in-house system, the Justice Information Management System (JIMS), to manage criminal justice information.
JIMS is the official information management software of the Knox County criminal justice process and should contain all information decided for each case during court proceedings. The information decided during court proceedings is recorded on paper warrants for General Sessions Courts and in case files for Criminal Courts. The information is entered into JIMS by General Sessions Court Clerk or Criminal Court Clerk employees, depending on in which court the case was heard. The General Sessions Court Clerk and Criminal Court Clerk offices also have bookkeeping systems to record costs assigned and collected.
OBJECTIVES, SCOPE, AND METHODOLOGY The objective of the audit was to evaluate the efficiency and effectiveness of the current criminal justice process. To achieve our objective, we:
Obtained and reviewed applicable procedural documents describing the criminal justice process and other documents as needed.
Conducted interviews and process walkthroughs with key personnel to establish the current practices within the criminal justice process.
Observed various court proceedings to gain an understanding of the environment in which the criminal justice system operates.
To evaluate the significance of audit findings we used both quantitative and qualitative factors. The qualitative factors used in evaluating the significance of a finding were whether the identified issue (1) poses a systematic operational or controls risk in Knox Countys ability to accurately or timely process defendants within the criminal justice system, (2) exposes Knox County to multiple (more than 10) lawsuits or class action, or (3) would result in a high level of reputational damage or inquiry from a state or federal agency. The quantitative factor considered was whether a finding exceeded 3 percent ($369,721) of the fiscal year (FY) 2014 budgets for the five departments included in the audit. A finding only has to meet one of the above four factors to be considered significant. 1 Based on the 2010 census data for Knox County as reported on Knox County Governments website. File Number 02-2014-01 Page 1
Knox County Internal Audit
The scope of the audit was the criminal justice process in place as of March 13, 2014, limited to the handling of a defendants case from the time the warrant is issued until the case is completed 2 . This performance audit was conducted in accordance with generally accepted government auditing standards except for requirement 3.96 related to an external peer review of the Internal Audit Department being conducted every three years. The last peer review was performed in December 2009 and covered internal audit work performed from January 1, 2006, through December 31, 2008. According to the three-year peer review rule, the Internal Audit Department should have had a peer review performed in calendar year 2012 to review audit work performed from January 1, 2009, through December 31, 2011, with an additional peer scheduled for calendar year 2015 to cover audit work performed from January 1, 2012, through December 31, 2014. The audit department changed Directors in December 2013 and it is not known by the current Director why the peer review was not performed in calendar year 2012 in cadence with a three-year review cycle. A peer review is planned either late in calendar year 2014 or early in calendar year 2015 depending on the amount of audit work available to review under revised procedures put in place in February 2013, the schedule of the peer review team, and approval by the Knox County Audit Committee. We do not believe the audit or assurance provided by the audit have been affected by the lack of a current peer review. The Internal Audit Department revised audit procedures used to conduct this review are directly tied through reference to the generally accepted government auditing standards. In addition, each audit is subject to a quality control process that ensures that generally accepted government auditing standards were met within the audit prior to issuing a draft report.
The standards also require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our observations and conclusions based on our audit objectives.
IDENTIFIED GOOD OPERATING PRACTICES During the course of the audit, we identified two areas considered good operating practices that promote efficiency and effectiveness in the criminal justice process. These identified good operating practices are (1) defined user roles in JIMS, and (2) document preparation for daily court proceedings.
DEFINED USER ROLES IN JIMS We found JIMS is restricted through defined user roles which limit the information each user has access to view or update. Defining user roles ensures that each user only enters and has access to the information needed to perform their individual part of the criminal justice process. For example, certain General Sessions Court Clerk and Criminal Court Clerk office employees have the ability to enter sentencing information for a case while other roles in JIMS do not have this ability. Roles are set-up by the Information Technology Department when a user requests to gain access to the system. The request must be made using the proper form and signed by the users supervisor. This security feature provides proper protection over the vast amount of information contained in JIMS and how information can be altered and used.
2 Completion for purposes of this audit can be sentence fully served, probation requirements met, or case diverted / dismissed. All costs must be collected for the case to be considered completed. File Number 02-2014-01 Page 2
Knox County Internal Audit DOCUMENT PREPARATION FOR DAILY COURT PROCEEDINGS The Knox County criminal justice system handles tens of thousands of cases each year and it is the responsibility of General Sessions Court Clerk and Criminal Court Clerk employees to ensure all warrants, case files, and other relevant documentation are ready for use in court each day. Applicable court records and documents are pulled using the daily scheduled docket available in JIMS and brought to the proper court by the Clerk office employees. Having all required documentation readily available in the court proceedings promotes efficiency in the process rather than having to wait for files to be retrieved during proceedings.
FINDINGS We found the criminal justice process could be improved to operate in a more efficient and effective manner. While the criminal justice process included (1) departmental understanding of their role in the criminal justice process and their internal practices in place to perform the activities for which they are responsible, and (2) reliance on JIMS as the information management system to make case information more readily available and prepare the daily docket for court each day, we identified one significant issue and four other areas where the process could be improved.
QUALITY ASSURANCE CONTROLS IN THE CRIMINAL COURT CLERK AND GENERAL SESSION COURT CLERK OFFICES SHOULD BE STRENGTHENED We found stronger quality assurance controls should be implemented in the General Sessions Clerk and Criminal Court Clerk offices. The General Session Clerk and Criminal Court Clerk offices have a predominately manual process for entering information determined during court proceedings. In addition, the majority of this information is entered after the court date using notes on warrants, daily dockets, and case files. The General Sessions Clerk and the Criminal Court Clerk offices handle the bulk of data entry and file processing in the criminal justice process. Most of the information entered is not reviewed or quality assurance controls are not performed to prevent errors. There is an inherent risk in a predominately manual process that information could not entered accurately or timely. As a result, predominately manual processes should include strong quality assurance controls to minimize entry errors during the process and reduce time delays to enact judicial orders.
Examples identified during the audit of areas where quality assurance controls could be strengthened include missing criminal histories, incorrect sentences / dispositions, incorrect scheduling of a case, and delays in processing information. Although we identified some limited automated and manual quality assurance controls in the process including inability to post-date information in JIMS, checks for existing identification number in JIMS, manual review of cases to be expunged cases in the General Sessions Clerk office, and independent comparison of data entered into JIMS to case file notes in the Criminal Court Clerk office, additional quality assurance controls are needed to ensure the criminal justice data is processed accurately and timely. Gaps in quality assurance controls may not necessarily lead to errors in defendant cases; however, there is a systematic risk that lack of quality assurance controls could result in errors to criminal cases. Such errors could include incorrect information included on criminal histories, incorrect court costs assigned, incorrect attachment issued, or delays in releasing defendants.
File Number 02-2014-01 Page 3
Knox County Internal Audit Stronger quality control processes (1) shift focus from correcting errors when identified to preventing / minimizing errors in the process and (2) provide a way to evaluate performance. Examples of quality assurance controls include logic checks in systems, required supervisory approval for critical activities, spot checking notes to data entered, monitoring system generated reports for defined areas, well documented and defined internal processes and training programs, validating that practices match the defined process, and real-time data entry. Since it is not efficient or practical to review 100 percent of the data entered, quality assurance controls would provide points in the process deemed critical or high risk to undergo additional scrutiny for accuracy and timeliness.
This issue is considered significant due to (1) the systematic operational risk the lack of strong quality assurance controls poses to each criminal case in the criminal justice system, (2) exposure of Knox County to potentially multiple lawsuits due to data entered without strong quality assurance controls in place to catch or prevent errors, and (3) exposure to reputation damage or additional inquiry from state or federal agencies.
ORIGINAL DOCUMENTS IN GENERAL SESSIONS COURTS SHOULD BE BETTER PROTECTED We found warrants and other original documents should be better protected. Warrants serve as the official court record for each case in General Sessions Courts. Original warrants and attachments are brought into the courtrooms each day for use with the daily docket of cases. All information pertaining to the case is documented on the warrant to be later entered into JIMS. Information such as charges, new appearance dates, sentencing, disposition, penalties, and probation requirements are documented on the warrants. The General Sessions Court Clerk is the official custodian of court records including warrants. We observed warrants bundled with other original case documentation removed from the General Sessions Court Clerks custody during court proceedings by defending and prosecuting attorneys. At times, this documentation was observed being taken out of the courtroom by attorneys. If warrants were lost or altered, the official court record of the case would be lost or altered. Removing original documents from the General Sessions Court Clerks custody poses a risk that information related to a case may not be processed accurately and / or timely.
INTERNAL PROCESSES SHOULD BE DOCUMENTED We found procedures should be documented detailing each departments role and practices for performing duties in the criminal justice system or referencing how each process interacted with other organizations. While the departments involved in the criminal justice process understood and explained their role and practices within the criminal justice process, no department had written procedures to document the practices in place. Each department should document procedures governing how work is to be performed, the roles and responsibilities of the department and department employees, and where the process intersects with or relies upon other departments. Documented procedures provide employees and departments clear expectations of performance, strengthen consistency of internal operations, assist in training new employees, and provide a way to hold those involved in the process accountable for their responsibilities. Documenting internal processes reduces the risk that the criminal justice process within each department will not be carried out as the department intends or will be performed inconsistently.
File Number 02-2014-01 Page 4
Knox County Internal Audit JIMS TRAINING PROGRAM AND MATERIALS SHOULD BE DOCUMENTED We found JIMS training programs and materials should be documented. The development and implementation of JIMS training is currently the user departments responsibility and varies from department to department. There are currently neither a documented and defined user guide nor training programs developed in conjunction with the Information Technology Department to ensure JIMS is being used consistently and in the manner intended. Having designed the JIMS program, the Information Technology Department is in the unique position of having intimate knowledge not only of the system capabilities but how the system was intended to be used in the process. A better defined training program and user guide should be built in conjunction with the expertise offered by the Information Technology department. A consistent and universal IT developed help guide and training package would ensure that the system is being used as designed, all capabilities of the system are known to users, and all users are operating within the system consistently. Lack of consistent training could lead to information being placed incorrectly in the system and the system not being utilized as designed.
TRAINING PROGRAMS AND MATERIALS IN THE CRIMINAL COURT CLERK AND GENERAL SESSIONS COURT CLERK OFFICES SHOULD BE DOCUMENTED We found training programs and materials in the Criminal Court Clerk and General Sessions Court Clerk offices should be documented for all positions. Both Clerk offices include a variety of positions such as bench clerks, computer clerks, docket clerks, bookkeepers, and customer service personnel. No formalized and documented training programs exist within the Criminal Court Clerk or General Sessions Court Clerk offices for any of the positions. Most training is done on-the job through job shadowing or by working through the ranks of the offices. Without a formalized, documented training manual and program, the training provided may be inconsistent or incomplete which could lead to errors in performing duties. Consistent training and knowledge is especially important in the performance of bench clerk and computer clerk duties that require a thorough knowledge of the criminal justice system in order to understand information decided during court proceedings, make the appropriate notes, and enter the information into JIMS. A formalized, documented training program and manual ensures all employees are required to have the same level of base knowledge before assuming sole responsibility for job duties and sets expectations for performance.
OBSERVATIONS During the course of this audit, we identified two areas the departments involved in the criminal justice process should consider to further improve and strengthen the current process. Observations are not considered findings but are simply areas identified by us during the audit for consideration by management. These areas relate to (1) improved system integration and (2) real-time data entry.
File Number 02-2014-01 Page 5
Knox County Internal Audit IMPROVED SYSTEM INTEGRATION We observed JIMS could be improved through (1) better integration of bookkeeping or accounting systems and (2) the addition or integration of a document management system. Currently the General Sessions Court Clerk and Criminal Court Clerk offices operate bookkeeping and accounting systems outside of JIMS to record assignment and collection of costs 3 on cases. Some, but not all, of the cost information for cases is entered into JIMS through automated and manual processes. As a result, not all cost information associated with a case is always readily available in JIMS to departments involved in the criminal justice process other than General Sessions Court Clerk and Criminal Court Clerk offices. In addition, a document management system that allows electronic viewing of scanned documentation pertinent to cases is not currently available in JIMS or as a separate system. The General Sessions Court Clerk and Criminal Court Clerk offices have considered the addition of such a system. Fully integrated systems with JIMS would (1) provide instant access to information for all parties involved in a criminal case, (2) place more reliance on one system for information rather than multiple systems, (3) protect original documents by offering an electronic view of information, and (4) reduce the amount of cost information and documentation requested from the General Sessions Court Clerk and Criminal Court Clerk offices. Fully integrated systems are needed in order to move toward the real-time data entry environment suggested below. We suggest the departments involved in the criminal justice process consider (1) completely integrating cost information pertaining to criminal cases into JIMS to provide a more complete picture of case information without having to obtain information from a separate bookkeeping or accounting system and (2) use a document management system within JIMS or integrated with JIMS to handle all documents relevant to criminal cases.
REAL-TIME DATA ENTRY We observed real-time data entry would improve the efficiency and accuracy of the current criminal justice process. As described in the report previously, much of the information for criminal cases is entered into JIMS after court proceedings have concluded based on notes written on warrants, dockets, or case files. Entering information after court proceedings may affect timeliness and accuracy of information entered. A real-time data entry system would allow information to be processed as it happens in the courtroom, provide instant information to all involved in the process, improve quality assurance through immediate review of information, remove delays in enacting judicial orders, and expedite processing of the case. A real time data entry system would require the criminal justice process to be examined from a real-time data entry perspective in order to be successfully implemented.
We suggest the departments involved in the criminal justice process consider entering all information determined during court proceedings in a real-time manner.
3 The term costs include litigation tax, fines, court costs, and restitution. File Number 02-2014-01 Page 6
Knox County Internal Audit RECOMMENDATIONS We recommend the General Sessions Clerk and Criminal Court Clerk offices:
1. Identify critical or high-risk activities or points within their internal processes where quality assurance controls are needed to improve and / or promote accuracy and timeliness.
2. For those areas identified in recommendation 1, develop and implement quality assurance controls.
3. Develop a well-defined training manual and formalized training program for each position.
We recommend the General Sessions Court Clerk, Criminal Court Clerk, Information Technology, Attorney Generals Office, Criminal Courts, and General Sessions Courts:
4. Develop and implement a process to manage original documents to ensure the information contained within is adequately protected.
5. Document unwritten internal practices for performing criminal justice process activities for which they are responsible to include duties and responsibilities of employees within the department and where the process relies on or intersects with another department.
6. Work with Information Technology to develop a JIMS user guide and training program to promote consistent and correct use of the system.
MANAGEMENT RESPONSE Management of the departments involved generally agreed with the recommendations of this audit report. See responses in Appendices A through E at the end of this report.
File Number 02-2014-01 Page 7
Knox County Internal Audit APPENDIX A CRIMINAL COURT / GENERAL SESSIONS COURT CLERK MANAGEMENT RESPONSE
File Number 02-2014-01 Appendix A
Knox County Internal Audit APPENDIX B CRIMINAL COURT MANAGEMENT RESPONSE
File Number 02-2014-01 Appendix B
Knox County Internal Audit APPENDIX C GENERAL SESSIONS COURT MANAGEMENT RESPONSE
File Number 02-2014-01 Appendix C
Knox County Internal Audit
File Number 02-2014-01 Appendix C
Knox County Internal Audit APPENDIX D INFORMATION TECHNOLOGY MANAGEMENT RESPONSE
File Number 02-2014-01 Appendix D
Knox County Internal Audit APPENDIX E ATTORNEY GENERAL MANAGEMENT RESPONSE