You are on page 1of 48

Auditing Application Systems

Monica OReilly Deloitte Enterprise Risk Services

Deloitte

May 20, 2005

Which ERP program is the best?


What software program should be used to ensure a company achieves world-class performance in finance? A. SAP B. PeopleSoft C. Oracle D. None of the above

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 1

Which ERP program is the best?


What software program should be used to ensure a company achieves world-class performance in finance? A. SAP B. PeopleSoft C. Oracle D. None of the above

A research study by the Hackett Group showed that more than 96% of companies that implement one of these software packages doesnt achieve world-class finance status.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 2

Which ERP program is the best?


Why?
Companies must incorporate best-practice processes, organization design, and

enabling technology.

They must make process, organization, and policy changes during the

implementation process, and they must configure their ERP systems to support best practices in finance. support these decisions and best practices.

The ERP system will not effectively support the organization unless configured to

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 3

Controls Framework

Quality Cost Delivery Effectiveness and Efficiency of operations Reliability of Information Compliance with laws and regulations Confidentiality Integrity Availability
Copyright 2003, Deloitte & Touche LLP. All Rights Reserved. Page 4

COBIT INFORMATION CRITERIA

Effectiveness
Efficiency Confidentiality Integrity Availability Compliance Reliability of Information

IT Resources (COBIT Definition)


DATA are objects in their widest sense (i.e., external and internal), structured and nonstructured, graphics, sound, etc. APPLICATION SYSTEMS are understood to be the sum of manual and programmed procedures. TECHNOLOGY covers hardware, operating systems, database management systems, networking, multimedia, etc.

FACILITIES are all the resources to house and support information systems.
PEOPLE include staff skills, awareness and productivity to plan, organize, acquire, deliver, support and monitor information systems and services.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 5

Objectives
Discuss the key risks and controls generally associated with specialized business applications List a number of discussion topics that can assist in initial inquiries and detailed discussions with specialized business applications team members Explain tests and procedures typically performed to complete an audit in specialized business applications Describe key documents that can be collected to support audit objectives in specialized business applications

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 6

General Areas of Risk


Data Integrity
Invalid or inaccurate data is entered into the system. Invalid or inaccurate master data is entered into the system. Critical business data is lost, altered, or corrupted. Data is archived offline; not available.

Transaction Processing
Invalid transactions are processed. Transactions are processed using incorrect data. Critical transactions are lost within the application and never processed. Transactions are processed incorrectly. Online, real-time processing of transactions.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 7

General Areas of Risk


Application Configuration
Business process rules are configured incorrectly. Transactions are not processed. Transactions are processed incorrectly. Business process rules are not configured at all. Transactions processed inconsistently. Transactions processed outside the system not recorded.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 8

General Areas of Risk


Security
Unauthorized personnel have access to critical business data or transactions. Personnel have too much access within the system. Violates segregation of duties Application security settings (e.g., password length) do not provide adequate security

restrictions.

Systems administration personnel have access to critical business data or transactions. Information can be accidentally overwritten. Unauthorized or incorrect changes to information are not detected.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 9

General Areas of Risks


Reporting
Reports do not provide information needed to support critical decision making by

management.

Reports are not available. Reports provide access to sensitive or confidential information.

Compliance and reporting issues will increasingly become a critical component of organizations IT systems and will spur upgrades and enhancements, said Lucie Draper, program manager for IDCs Enterprise Technology Trends (ETT) research service. In North America, 43.3% of organizations said that they have purchased new software or upgraded, changed, or enhanced technology associated with regulatory compliance requirements and activities.
Source: IDC, Vertical Markets Demand Regulatory Compliance Solutions That Address Industry-Specific Issues, IDC Survey Shows, June 9, 2004 at http:www.idc.com

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 10

Identification of Key Controls


All key controls should be driven by risk, and/or control objectives. Example: Only valid orders are input and processed.
Control activities will help ensure this. May be IT-oriented within the relevant application E.g., security May be manual E.g., review of faxed order forms

All key control activities should be identified and tested.


Need to determine how to divide tasks between operational and financial auditors and IT

auditors

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 11

Examples of Application System Controls


Only valid orders are input and processed.
SAP R/3 edits and validates financial documents on line. Orders are sequentially numbered by the system. The sequence of orders processed is

accounted for.

Customers enter and/or cancel orders automatically using EDI protocols. SAP R/3 restricts to authorized personnel the ability to create, change, or delete sales

orders, contracts, and delivery schedules.

Which one of the following data entry controls provides the greatest assurance that the data is entered correctly? A. Using key verification B. Segregating the data entry function from data entry validation C. Maintaining a log detailing the time, date, employees user id, and progress of various data preparation and verification tasks D. Adding check digits
Copyright 2003, Deloitte & Touche LLP. All Rights Reserved. Page 12

Auditing Application Systems Approach


Understand the business process or function that the application supports
e.g. Sales commission, order processing

Meet with the business personnel to understand the business process


Map out the process Identify application inputs, outputs, interfaces Identify application controls and manual controls Identify application outputs Highlight any issues or concerns from the business owners

Are these off-the-shelf packages?

Is the application custom developed?


When were the business applications implemented? Have there been any significant processing problems with the application?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 13

Auditing Application Systems Approach


What documentation exists for the application?
How current and accurate is the documentation?

Who are the responsible personnel?


Who is the owner of the application? Who are the primary business superusers? Who are the application administrators? Who is the security administrator?

Where are the key data entry points?


Interfaces? Manual?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 14

Auditing Application Systems Approach


How is application level security designed and administered?
Group vs. individual profiles? Documentation available?

How were controls designed and implemented?


Documentation available?

What are the key outputs from the system?


Reports? Data (interfaces)?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 15

Audit Plan Considerations


Application Configuration
Application configuration changes should follow a well controlled change methodology: Appropriate documentation Approval processes

Testing
Promotion to production Manual workarounds should be limited to application functionality gaps only. Controls should be configured if possible.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 16

Detailed Discussion Topics


Application Configuration
What is the process for application configuration? Who defines application configuration requirements? Is a consistent process followed? Are there forms and approvals for each step of the process?

Design, testing, promotion to production, etc.


How are business analysts assigned?

By process, function, etc.


How many application configuration changes are made on a monthly basis?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 17

Detailed Discussion Topics


Application Configuration
Were business process controls expressly considered in the initial application configuration

design?

Were control specialists on the project team? Was a controls implementation methodology followed?

What methodology? Who implemented?


Does documentation exist? Was a risk analysis performed?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 18

Application Input Controls


Authorization of end users

Unique user ids and passwords


Segregation of duties enforced by the application Audit trails turned on and reviewed periodically Review of system admin privileges Input authorization

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 19

Application Input Controls


Input data validation checks
Limit test a test of reasonableness
Validity test a comparison against master files Self checking number a check for accuracy

Batch integrity of online or database systems

Input controls for batch processing


Item count to assure all items are processed Control total to assure a critical field of data is entered correctly Hash total may be a total of all order numbers

Error reporting and handling

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 20

Audit Plan Considerations


Data
Legacy data should be cleansed and purged prior to loading into the application. Access to master data maintenance should be restricted to authorized personnel. Data validation routines should be used to enhance data input accuracy. Data archiving and retention strategies should support business objectives. Batch input and interface input should be monitored and controlled.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 21

Detailed Discussion Topics


Data How is master data maintenance managed?
If centralized in a shared services function, what processes have been established for

communicating adds, deletes, and changes to the master data maintenance team?

Who approves master data changes? Is documentation retained? Are master data audit trail reports reviewed regularly?

What steps were taken to cleanse and purge legacy data during implementation?
How were black holes addressed?

What is the corporate data retention strategy? What transactions are archived? On what frequency? Is there a corporate data classification strategy in place?
What data has been identified as sensitive, confidential, or critical?
What steps have been taken to secure that data?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 22

Detailed Discussion Topics


Data What interfaces exist?

What data is processed via those interfaces?


Are there hash totals or other controls in place to identify problems with interfaced

transactions?

Who reviews and remediates interface exceptions?

What data validation routines exist within the system? What additional validation routines have been implemented?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 23

Audit Plan Considerations


Transaction Processing
Batch processing of transactions should be monitored and controlled. Business logic should adequately control event-driven transactions. Anomalous transactions should be identified and held for review if possible prior to posting. Access to process transactions should be limited to those personnel responsible for those

transactions only.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 24

Detailed Discussion Topics


Transaction Processing
Are any critical transactions processed in batch? Which transactions are they? What is the batch processing schedule? What dependencies exist? Who is responsible for scheduling and monitoring batch processes? How are problems identified? What business logic rules have been configured to block anomalous transactions from

processing?

What reports are run to identify blocked transactions?


How often are these run? Who is responsible for monitoring and resolving blocked transactions? What are the current outstanding balances of blocked transactions?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 25

Application Processing Controls


Online systems test the validation checks are applied to data being input Batch processing
Use of computer file identification systems (headers) to prevent use of improper data

and program files (only payroll systems will process data labeled as payroll data)

Comparison of control totals gathered during input with total transactions processed.

Processing Controls
Editing Validation Routines

Review exception or error processing reports

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 26

Audit Plan Considerations


Security
A defined, structured methodology should be used to design and implement application

level security. duties.

Application level security should be granular enough to properly restrict segregation of Is access granted during the implementation still appropriate? The level of access granted to superusers and application administrators. There should be a structured approach to granting, removing, and changing users access.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 27

Detailed Discussion Topics


Security
What are the system security settings? Minimum password length, password complexity, expiration, multiple logins allowed,

etc.?

How is user security designed? Are user security profiles designed at the job level or task level? Are profiles designed for individual users or by job function? Are there user groups? Is access granted on a least-privileged basis?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 28

Detailed Discussion Topics


Security
What is the process for adding, changing, or removing users access? Are forms required? How does the security administrator know when to make changes? Who approves changes? Are data owners assigned to each functional area? Are periodic security reviews performed by the data owners? How is the security administrator prevented from making unauthorized changes to

access?

Are security audit trail reports reviewed and approved?

How often? By whom?


What other security reports are used to manage the application? Invalid login attempts, login times, etc.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 29

Audit Plan Considerations


Reporting
There should be a defined reporting strategy. Reports should be limited to authorized personnel only. Ad hoc reporting tools should not allow access to confidential or sensitive data.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 30

Detailed Discussion Topics


Reporting
Are external reporting tools used? Cognos, Crystal Reports, etc. How do these tools interface to the production environment? How are these controlled and managed? Are ad hoc querying tools used? SQL, etc. Who can use these tools? How is sensitive data protected?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 31

Detailed Discussion Topics


Reporting
Is a reporting strategy in place? Have standard reports been defined and developed? Packaged reports or custom? What are the critical reports used by management to support decision making? Are users satisfied with the data reporting functionality? Do users generate their own reports as needed, or are reports generated centrally and

distributed?

If centrally, how are sensitive reports controlled and managed?

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 32

Verify Application Controls


Observe and Test Users Performing Procedures:

Review and test access authorizations and capabilities Separation of duties Authorization of input

Balancing
Error control and correction Distribution of reports Manual controls

Recovery procedures

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 33

Requested Documentation
Data
List of data classification strata List of data owners Data conversion testing results documentation Master data maintenance policies and procedures Master data audit trail reports List of all ingoing and outgoing interfaces Data used, transactions driven, timing, interface type (real time vs. batch), security Data archiving policies and procedures Data archiving reports

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 34

Requested Documentation
Transaction Processing
Listing of key batch jobs or other scheduled processes Processing management policies and procedures Escalation procedures Listing of batch owners Reports of transactions that the system has blocked or flagged with error messages

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 35

Requested Documentation
Application Configuration
Systems design documentation Systems development life cycle policies and procedures List of all custom objects in the application List of current projects or open maintenance tickets For open projects, all supporting documentation Business case, design specifications, testing scripts, etc. Configurable business process controls documentation

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 36

Requested Documentation
Security
Application security design documentation List of security profiles, groups, etc. Security policies and procedures Security administration policies and procedures Copies of security request and change forms Sample for specific users List of all data owners Listing of terminated users for defined period (e.g., last 60 days) Current user listing Copies of reports used for security management

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 37

Requested Documentation
Reporting
Reporting strategy documentation List of all key reports used External tool documentation Interface methods Design documents

Specifications, timing, etc.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 38

Summary
Application System controls comprise manual controls and IT controls.
Need to test both; coordination can be a challenge

Controls should be linked to a framework; typically based on control objectives. Any business process may have one or more primary applications driving the process.
Need to identify the key applications

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 39

Section Summary
Most applications have common elements that are key for auditors to focus on:
Data Transaction processing Configuration Security Reporting

Audits should encompass elements of all of the above.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 40

Example Application System Controls


5
Order Customer Sales Order Staff Regional Sales Manager Order Entry and Pricing Reports European Operations Manager

EDI Orders
Managing & Processing Orders

Input Order

4 6

Order Entry and Pricing reports

2
Accounts Receivable Manager

Review of blocked Orders


SAP

3 1
Manually Released Sales Orders Credit Manager

Key Controls
1 SAP performs certain validations before orders are processed further. For example, customer reference details are checked against Customer Master file records, system ensures completeness of order details and checks stock availability. Validation failures result in orders being placed in incomplete order queues which is checked daily by sales office staff.

Key Controls
4 Order reports are reviewed daily by Regional Sales Managers (CPD) and European Operations Manager (BSD) for unusual order quantities + prices. Some customers orders are processed through EDI. Sequential number checks are performed to ensure all orders are captured with error messages sent to the European Operations Manager. Order pricing reports detailing prices < cost are sent to the Regional Sales Manager and European Operations Manager on a regular basis.

All orders are automatically credit checked by SAP against agreed credit limits before they are processed further. Blocked orders appear on the blocked sales distribution listing which is reviewed throughout the day by the Accounts Receivable Manager.
Blocked orders that are manually released are reviewed daily by the Credit Manager.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 41

Example Application System Controls


Sister Companies Suppliers Invoices SAP Invoice Matching Process Invoices Posted to General Ledger module

Review of Blocked Invoices

2
Report of unblocke d invoices

Stock Receipts

1
External Suppliers Processing Accounts Payable Accounts Payable + Finance Manager

HELEN

3
Divisional Accountant Mismatch Reports

Key Controls
1

Key Controls
3 Although they do not result in the blocking of invoices, the SAP system reports mismatches between the following which are investigated and actioned: invoice and purchase order quantities - all suppliers; invoice, supplier shipping notification and GRN quantities - Sister companies only.

All Suppliers - Invoice price details are matched to standard

costs on purchase orders. Any difference (zero tolerance) will cause the invoice to be blocked. Blocked invoices are reviewed by Accounts Payable and the Finance Manager on a daily basis.

External Suppliers Only - Invoices are blocked for payment


2

where there are quantity mismatches between supplier invoices and goods physically received and recorded on SAP (zero tolerance). Divisional Accountants review month-end reports identifying all blocked invoices that have since been cleared.

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 42

Resources
ISACA Members can access the Generic Application Review, Audit Program and Internal Control Questionnaire.
One of the goals of ISACAs Education Board is to ensure that educational products

developed by ISACA support member and industry information needs. Responding to member requests for useful audit programs, the Education Board has recently released audit programs and internal control questionnaires on various topics for member use through the member-only web site and K-NET. These products are intended to provide a basis for audit work.

http://www.isaca.org

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 43

Auditors Sharing Audit Programs

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 44

Selected Resources
Information Systems Audit and Control Association www.isaca.org IT Governance Institute www.ITgoverance.org Auditors Sharing Audit Programs www.auditnet.org MIS Training Institute www.misti.com

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 45

Q&A

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 46

Contact Information

Monica OReilly Senior Manager Enterprise Risk Services Deloitte 415 783 5780

Copyright 2003, Deloitte & Touche LLP. All Rights Reserved.

Page 47