Jas Peter 10681078 MSC ACC

Amsterdam
Business School
The Impact of ERP Systems on Internal Audit Planning: a

TeamMate Perspective.
Name: Peter Jas

Student number: 10681078
Supervisor: drs. Ed H. Jansen RA MCM
Date: June 22, 2015
Word count: 79672, 0
MSc Accountancy & Control, variant Control
Faculty of Economics and Business, University of Amsterdam
Statement of Originality
This document is written by student Peter Jas, who declares to take full responsibility for the
contents of this document.
I declare that the text and the work presented in this document is original and that no sources
other than those mentioned in the text and its references have been used in creating it.
The Faculty of Economics and Business is responsible solely for the supervision of completion
of the work, not for the contents.
Abstract
The objective of this study is to research to which extend the use of an ERP system has an
impact on internal audit planning. In-depth knowledge of the internal audit planning process and
how ERP systems impact on this by semi-structured interviews on TeamMate experts.
Additionally, reviews of documents from an internal audit department and TeamMate surveys,
give support to the interview findings.
The conclusions from this research are: 1. The internal audit planning process and the
related risk assessment, is performed at a high level, to create a risk rating for each auditable
entity, and at granular level, to review specific risks within the entity when the engagement audit
takes place. 2. The use of an ERP system does have limited to no impact on the high level
internal audit planning. 3. The use of an ERP system does have impact on the audit planning in
the way that less time is required to audit an ERP environment. This is because of smaller and
fewer audit samples, uniform availability of all data and the possibility to grow towards
continuous auditing.
2
Content:
1 Introduction ........................................................................................................................................... 5
2 Research Method ................................................................................................................................... 8
2.1 Semi-structured Interviews....................................................................................................... 8
2.2 Document Review ..................................................................................................................... 9
3 Literature Review ................................................................................................................................ 10
3.1 General information about the internal audit process ....................................................... 10
3.2 Main concerns in ERP auditing ............................................................................................. 12
3.3 ERP impact on high level audit planning ............................................................................ 14
3.4 ERP impact on granular level audit planning ...................................................................... 15
3.5 Other findings .......................................................................................................................... 17
4 Background .......................................................................................................................................... 18
4.1 Wolters Kluwer Financial Services........................................................................................ 18
4.2 TeamMate ................................................................................................................................. 19
4.3 Why TeamMate ........................................................................................................................ 19
5 Findings ................................................................................................................................................ 21
5.1 General information internal audit planning process ........................................................ 21
5.2 Main concerns in ERP systems ............................................................................................. 25
5.5 Other findings .......................................................................................................................... 32
6 Discussion ............................................................................................................................................ 35
6.1 General information internal audit process ......................................................................... 35
6.2 Main concerns ERP systems .................................................................................................. 37
6.5 Other findings .......................................................................................................................... 41
3
7 Conclusion ............................................................................................................................................ 43
References ................................................................................................................................................... 45
8 Appendices ........................................................................................................................................... 48
8.1 Appendix I: Mind map to specify research topic................................................................ 48
8.2 Appendix II: Thesis structure ................................................................................................ 49
8.3 Appendix III: Interview #1 ................................................................................................... 50
8.4 Appendix IV: Interview #2.................................................................................................... 62
8.5 Appendix V: Interview #3 ..................................................................................................... 74
8.6 Appendix VI: Interview #4.................................................................................................... 90
8.7 Appendix VII: Interview #5 ................................................................................................ 103
8.8 Appendix VIII: Interview #6 .............................................................................................. 115
8.9 Appendix IX: Interview #7.................................................................................................. 130
8.10 Appendix X: Interview #8 ................................................................................................... 137
8.11 Appendix XI: Interview #9.................................................................................................. 151
8.12 Appendix XII: Interview #10 .............................................................................................. 159
8.13 Appendix XIII: Interview #11 ............................................................................................ 167
4
1 Introduction
ERP systems link related business processes to one another through workflow automation and
the use of one single database, which can facilitate real-time recording and reporting of economic
events. Any single error, unintended or not, can have a significant effect on the accuracy of the
data and as a result also on the reporting. Internal auditors have the task to assure that the data
does not contain uncalculated risks, in order for senior management to make decisions on
adequate information.
Organizations require Accounting Information Systems to support Management
Accounting Controls with timely and correct information. Accounting Information Systems need
to collect and store data, transform data into information and provide controls to safeguard
assets. Internal Control relies on Accounting Information Systems to monitor risk as well as
compliance to regulations. COSO (2013) provides a framework for organizations to ensure that
the businesses and their risks are in control.
Alsop (1998) provides a brief history of enterprise computing. He states that computers
are invented around 1940 and were used by companies in the 1950’s. This “Big Computing”
contained large and complicated mainframe machines, which could only be used by specialized
people. In the 1980’s the “Personal Computing” was introduced and made the world of
computers accessible for everyone. Limited connectivity and various languages made
communication between the PC’s challenging. The World Wide Web resolved this and moved
the enterprise computing into the age of “Networked Computing”, in which we currently are.
The Networking Computing is making highly integrated information systems, like Enterprise
Resource Planning (ERP), possible.
ERP systems can be seen as an integrated set of applications from various business
procedures and departments and is sharing one single database. This is creating two main
advantages: the elimination of multiple data entry and the increase in flexibility and real-time
information to support Management Accounting (Kanellou and Spathis, 2013). According to
Grabski, Leech and Schmidt, 2011) ERP systems also have some downsides: implementation is
expensive, no long-term benefit compared to competitors and not always recognized to the full
potential. Scapens and Jazayeri (2003) conclude in their research that Management Accounting is
not changing because of ERP systems, but the role of the management accountant is.
As multiple data entry is eliminated with further integration of Accounting Information
Systems, an Internal Control is fading (Sayana, as cited in Grabski et al., 2011). In organizations
without any integration, and so with multiple data entry, the results from various databases can
be intermediately verified and used as control method to guarantee completeness and correctness
5
of data. For audits, as great part of the internal control process, this will have an impact on the
risk assessment and control activities (Bedard, Graham & Jackson, 2005).
This leads to the research question:
What impact has the use of ERP systems on Internal Audit Planning?
Teammate is part of the Wolters Kluwer enterprise and creates audit tools for internal
auditors around the world. I’m currently working for Wolters Kluwer as financial analyst, which
helped in having access to the TeamMate expertise. As financial professional I make use of ERP
systems and frequently communicate with internal auditors. The research question is therefor
interesting in my profession. Another reason for this research question is because there has been
a lot of research on the benefits and downsides of ERP Systems, but there is limited in-depth
research available on the impact of ERP systems on audit planning. This research can give
further insights to an organization on how an ERP system can have an impact on the internal
audit planning and in particular the risk assessment.
In order to answer this question this research question can be broken down in detailed
questions. ERP systems are characterized by the use of one single database throughout the
organization. As a result from this characteristic, the data is entered only once in the ERP system
and this may be done in various physical locations. The detailed question resulting from this
knowledge is:
a. What are the main concerns of risk in an ERP system?
TeamMate experts and internal auditors indicate that audit planning can be split into two
levels: the annual high level audit planning and the engagement granular audit planning. A risk
assessment is performed at both levels of audit planning. The detailed research questions
resulting from these aspects are:
b. How does the use of an ERP system impact the annual or high level risk
assessment and audit planning?
c. How does the use of an ERP system impact the engagement or granular level risk
assessment and audit planning?
6
In the next chapter, I will give an explanation of the research methodology. After that, I
will give a literature review on the research question. In the background chapter, I will give a
brief outline of TeamMate, the expertise on which my research will be based. This will give a
further explanation why TeamMate expertise adds value to this research. In the following
chapter, the findings of the interviews and the documentation review are reflected, followed by
the discussion between the findings and the literature research. The final chapter will state the
conclusions of the research are stated, together with the limitations and possible future research
directions.
7
2 Research Method
This research has as goal to gain in-depth knowledge of the relation between ERP systems and
the internal audit planning. TeamMate experts and users are selected to provide further
information about the research question. Chapter 4 explains why TeamMate is suitable for this
research. For robustness two non-TeamMate users, which perform audit planning, are added. An
iterative process of research has been used as newly found information from the semi-structured
interviews may require further literature research. The qualitative approaches of semi-structured
interviews and documentation review will be most suitable to gain an in depth understanding.
For both approaches a brief description is given below.
2.1 Semi-structured Interviews

The main part of the research is performed by interpreting interviews. Interviews are
held with developers, consultants and users of TeamMate. As mentioned above, two non-
TeamMate users, which are performing risk assessment and audit planning, can be added to gain
robustness in this research.
Semi-structured interviews will start from topics as described in the literature section of
this research. The questions will be open and not formulated too specific, to give room for the
interviewees to add topics and give a wide critical opinion of the impact of ERP systems on risk
assessment and audit planning. The interviews start with questions about their role in the
organization, their expertise in internal audit and in internal planning tools as TeamMate. The
interview continues with discussions about audit planning and ERP systems. This gives room for
a good understanding of both aspects and for possible findings outside the research area of this
paper. When the mindset of the research is created, questions about the impact of ERP systems
on audit planning are finalizing the interview.
The professional roles of the interviewees are: Product Manager (Interviewees #7 & 9),
Director of Product Management (Interviewee #6), Manager Consulting (Interviewee #1),
Consultant (Interviewees #4 & 5), Director of Internal Audit (Interviewee #8) and Internal
Auditor (Interviewees #2 & 3). As mentioned before, Internal Auditors (Interviewees #10 & 11)
who are not using TeamMate, are added for robustness of the research.
The interviews have taken place in the April / May time frame in 2015. The interviews
have been recorded, transcribed and send to the interviewees for review. Interviews 8, 10 and 11
have been in Dutch. Any citations coming from those interviews have been translated in
agreement with the interviewees. After interview #5 a mind map has been created (see Appendix
I) to specify the general direction of the interviews and to review the direction of this research.
8
2.2 Document Review
TeamMate consultants and developers are in constant communication with their clients, which
are internal auditors all over the world in any type of industry, including governmental
organizations. They annually have surveys and interviews about the internal audit process. The
documentation resulting from these surveys and interviews are used in this research in order not
only to confirm findings from the interviews, but possibly also for new information to answer
the research question.
Another part of the document review sources from an internal audit department. A
document is used, showing the criteria in the annual risk assessment as used by this internal audit
department. This documentation is used to mainly answer the question if an ERP system has an
impact on the annual risk assessment and audit planning.
9
3 Literature Review
This chapter researches how an Enterprise Resource Planning system has an impact on internal
audit planning based on existing literature. The first paragraph will give a general overview of the
internal audit process. After that, three paragraphs will give a literature review of the detailed
research questions. A final paragraph has been added after the interviews have taken place, to
reflect the additional findings. Appendix II provides an overview of the structure in this chapter.
3.1 General information about the internal audit process

Goal internal audit
Audits generally produce assurance and increased confidence in the organization or parts of the
organization (Power, 2003). Kanellou and Spathis (2011) give a further explanation that internal
auditing is an independent and objective validation of the organization, which improves the
performance of the processes and assists in aligning the processes to achieve the goals of an
organization. Auditors make use of electronic audit planning tools in order to make the audit
process more efficient and this will give internal audit more room to perform the additional
advisory task (Barret, Cooper and Jamal, 2005).
The COSO framework states that internal control, and therefor internal audit as well, can
be seen as a process (COSO, 2013). Ditsmith & Haskins (as cited in Power, 2003) contradict by
stating that internal audit cannot be seen as a logical series of steps, but is more “a social enterprise
relying on deeply embedded perspectives”. They explain that there is more to internal auditing then just
following a formal process approach, because there are parts of the organization, which will not
be in the scope of this formal process. In agreement is the statement from Power (2003), who
states that in spite of programs to standardize the audit process, differences in audit routines are
found. The continuous necessity for change in audits, sourced from economic, regulatory and
political pressures, is another reason why it is challenging to standardize the audit process.
COSO framework
The COSO Executive Summary (2013) states: “Internal control is a process, effected by an entity’s board
of directors, management, and other personnel, designed to provide reasonable assurance regarding the achievement
of objectives relating to operations, reporting and compliance.” Organizations can develop controls, like
internal auditing, based on the COSO framework to mitigate risk to acceptable levels (COSO,
10
2013). The COSO framework contains the components control environment, risk assessment,
control activities, information & communication and monitoring activities.
In their research for the nature of specific control risks and the auditor response on risks,
Bedard et al. (2005) find evidence that control environment risk factors are most frequently
identified in the management information quality risk area. In their research, they identify two
important areas of risk: Electronic Data Processing (EDP) security and management information
quality. From their studies can be concluded that not all risk factors of the COSO structure are
included in the audit planning. For example control environment appears to be less tangibly
related to audit planning. In line this this statement, Hsu, Sylvestre and Sayed (2006) explain that
the COSO framework is used for consideration of risks that are relevant to business, accounting
and auditing & assurance and that the assessment of risks involved in an ERP environment can
be classified into four categories: business, control, system and security. These statements
conclude that if the COSO framework can be used as a basis for auditing, but is not necessarily
used completely.
Risk assessment
One of the five integrated components in the COSO framework is the risk assessment. COSO
(2013) explains about the risk assessment that every organization faces risks, either sourced
internally or externally. Risk is further defined as “the possibility that an event will occur and adversely
affect the achievement of objectives”, which means that events may occur that will have an impact on
the possibility to achieve the objectives of that organization. COSO (2013) also includes that the
risk assessment should include objectives at all levels of the organization and is determining what
level of risk is being accepted by the organization. The risk assessment in the COSO framework
in relation to internal audit can be interpreted as a way to identify and rate risks at all levels of an
organization, to specify what areas the internal audit department should focus on.
Auditors already have been performing some sort of risk assessment before the first
COSO framework was introduced. The COSO organization has been setup in 1985, while
Gaumnitz et al. (1982) already concluded that auditors are performing some sort of evaluation of
internal control in order to determine the audit plan. Whether or not the risk assessment is
originating from the COSO framework, it is certain that the risk assessment is an important part
of the audit planning process (Bedard et al., 2005). In his research for the impact of information
system risk on the audit planning, he finds evidence that the risk assessments increases if the risk
factors related to management information quality, increase as well.
11
Power (2003) further concludes that if the risk assessment is performed with an
unstructured approach, than a wider range of risk factors will be used in the risk assessment.
Low (2004) makes the additional statement that with the current complexity in organizations,
auditors specialized by industry are better capable of recognizing risk factors to be used in the
risk assessment.
3.2 Main concerns in ERP auditing

Accounting Information Systems are important in creating support for management accounting
and have evolved throughout the years. They started out to automate processes as posting
transactions to journals and sorting the transactions according to the chart of accounts of the
general ledger (Rom & Rhode, 2006, p. 40). In the 1980’s each department has its own
Information System including a stand-alone database. In order to make processes more efficient,
interfaces are created to communicate between the different databases. This communication is
challenging, as the different systems might not be using the same language (Davenport, 1998).
Characteristics of ERP
Kanellou et al. (2013) describe the need for integration between the systems of the various
departments has become bigger as management accounting requires more real-time information
for decision-making. They find in their research evidence that with introduction of ERP systems,
organizations are capable to increase flexibility in information generation, increase integration of
data throughout the organization in the accounting information system, improve quality of the
reports and improve the decisions based on timely and reliable accounting information. Hsu et
al. (2006) add that ERP systems are implemented to gain efficiency in the processes. These ERP
systems contain cross-functional modules to integrate all information systems from the different
departments and are using one database (Robey, Ross & Boudreau, 2002).
As a result of these characteristics, Scapens et al. (2003) conclude that ERP systems lead
to more forward-looking information and line managers gain accounting knowledge. An ERP
system also eliminates multiple data entry, as data is being processed decentralized by operating
personnel in each department, which is automatically generating the appropriate accounting
entries, instead of the centralized routine task in the accounting department. In their research
they also conclude that the elimination of the routine tasks makes room for a different role for
the management accountant and become support for the business managers.
Another characteristic of an ERP system is that they are including their own unique
features (Hsu et al., 2006) and it is challenging to find the ERP system, which is closest aligned
12
with the organizations requirements. The research from Grabski et al. (2011) is in line with this
statement, as they explain that the implementation of an ERP system results in an iterative
process between how an ERP system is shaping the processes of an organization and how the
ERP system is modified to meet the organizations requirements.
Access Management
An ERP system is making use of a single database, which is containing organization wide
confidential information. This raises the concern of internal and external access to the corporate
data (Grabski et al., 2011). Hsu et al. (2006) refer to this security risks containing unauthorized
access to equipment, software or the database, which should be mitigated by physical and logical
security controls. The physical controls relate to equipment and are very similar in an ERP
environment as in a non-ERP environment. The logical controls relate to passwords, encryption
and firewalls, and are used to prevent unauthorized access to a system or database.
Hsu et al. (2006) explain that the logical controls also contain the segregation of duties,
which prevents errors and fraudulent activities. The segregation of duties in a non-ERP
environment is more easily monitored as access of an individual can be captured by fragmented
system and in an ERP system there is more staff and are more access points in the system to be
controlled. About the segregation of duties COSO (2013) states that it is part of the control
activities and if the segregation of duties is not practical, management should install alternative
control activities.
Hunton, Wright and Wright (2004) state that a weakness in the access controls will have
a greater significance, because if these controls are not properly configured, then unauthorized
access to confidential information or the possibility to unauthorized adjustment to the data, can
have a big impact on the organization. Hsu et al. (2006) are in agreement with this statement and
add that internal audit needs to determine the potential risk and develop mitigating controls.
Process flows
Decisions are not better than the data which they are based on. Haug, Stentoft & Pedersen
(2009) describe that data are created in every step of the business process and that decisions are
based on this data. In order to make the appropriate decisions it is vital that the data is of
adequate quality. In their research, they describe that for non-ERP the main concern of data
quality is caused by the use of inconsistency between the various used systems in an
organization. In line with this are the conclusions of Hunton et al. (2004) and Hsu et al. (2006),
who explain that with the integrated modules of an ERP system, processes become
13
interdependent and the fact that data is entered only once can heighten the control risk
requirements.
Sanaya (as cited in Grabski et al., 2011) points out that with the integration between
modules and the seamless data collection, there is no longer the opportunity to verify data by
comparing the same data from the different databases in the steps of a business process. This
contradicts with the statement from Grabski et al. (2011), who explain that an ERP system is
offering advantages for risk management as these controls are integrated and tested within the
system. Hsu et al. (2006) explain that the standard set of controls are not always configured as
intended by design, because controls have a negative impact on efficiency and in the trade-off
between controls and productivity, management usually chooses for efficiency.
3.3 ERP impact on high level audit planning

IT Controls
ERP systems encompass the risk of depending on one single system. COSO (2013) specifically
states that an organization should have control activities in place that mitigate the risk to
acceptable levels for all parts of the organization, including the technology environment. In a
study of audit risk assessment, Hunton et al. (2004) conclude that auditors identify high risks in
business interruption, process interdependency and overall control risks in ERP systems
compared to non-ERP systems. The research claims that financial auditors underestimate ERP
system risk, and specifically the risk in system security, and that audits should be a combined
effort of both financial as IT experts.
ERP implementation or adjustments

ERP systems are expensive and complicated to implement (Davenport, 1998; Grabski et al.,
2011; Hunton et al., 2004). To have implementation success, Grabski & Leech (2007) conclude
that internal audit should be part of the implementation process. Internal auditors can review if
the control methods are configured in line with the company policy. Grabski et al. (2011) explain
that the ERP systems do not create a status quo after the implementation. ERP systems continue
to be reconfigured, updated and extended after the implementation. This raise concerns in an
organization how these changes impact the flow of data and the accuracy of information. COSO
(2013) also recognizes the risk in system alterations and there should be included in the risk
assessment.
14
3.4 ERP impact on granular level audit planning
Single manual data entry
As ERP systems are tightly integrated, any single data entry error, regardless whether or not
intentional, will have an effect on the quality of data and may result on decisions in the business
operation (Hsu et al, 2006). It is important that managers are aware of these business risks and
properly address these risks. Hsu et al. (2006) continue that the human factor has a great impact
on the business risk and is caused by lack of user involvement, lack of adequate knowledge and
high stress levels. In detail they describe that in an ERP system processes are interconnected and
automated, so any single data entry will have an impact on connected cycles and that there is
usually no verification or rectification possibilities at a later stage in the process.
Haug et al. (2009) agrees with this statement and adds that outside an ERP environment
data quality problems occur as a result of inconsistency of the same data in various systems.
Kang and Sanhtanam (2004) also agree with the statement that errors in the data entry will cause
misinformation and conclude that users need adequate training to have a better understanding of
not only the system, but also about the business impact of the performed data entry. They
conclude that not enough attention is paid to training about interdependencies of tasks in an
ERP environment. Hsu et al. (2006) also recognize a solution can be found in adequate training.
System controls
ERP systems already contain adequate controls in order to mitigate security issues and process
interdependencies, although these system controls should be adequately configured (Hunton et
al., 2004). An impact on the audit planning comes from the complexity of the audit environment
(Gaumnitz et al., 1982). Their research concludes that there should be an inverse relationship
between the strength of internal control and the audit hours planned. Hunton et al. (2004) futher
claims that management should recognize the risk in the system controls and internal audit
should be testing them.
In their research Hunton et al. (2004) find evidence that financial auditors underestimate
the ERP system risk, where IT auditors do recognize the risk in system controls. The solution
which Hunton et al. (2004) provide is to combine expertise of financial and IT auditors in the
audits of an ERP environment. Kanellou et al. (2011) state that the auditors’ role is changing and
gets a higher focus on IT auditing, because of the complex IT settings in ERP systems. Not only
should IT auditors get more involved in the audit process and the correlated risk assessment, but
also financial auditors should be adequately trained to be able to perform audit tests in an
efficient and effective manner.
15
Audit preparation
One of the characteristics of an ERP system is the fact that it uses only one database containing
all data of the organization in one single format. Grabski et al. (2011) explain that the internal
audit department can benefit as data will be available more quickly, although this depends on the
system user knowledge and system access authorizations. They also state that the internal audit
department will benefit from the data being in one single format. The data will be easier to read
and recognize. Also no reformatting of data will be required to fit into any analytical tools. This
will decrease the required audit time and for this reason impact the audit planning.
Continuous auditing
In the last two decades the need for continuous auditing of business information has increased
(Kuhn and Sutton, 2010). Continuous auditing can be described as the methodology to review
and report on all transactions and system settings on a real time basis in order to gain assurance
on the data and information accuracy for a company (Alles, Brennan, Kogan and Vasarhelyi,
2006). As mentioned above ERP systems contain all company data in a single database and in a
uniform format. Kuhn et al. (2010) state that this provides the critical infrastructure which is
required for internal audit to use electronic tools created to perform continuous auditing, with
implemented modules called Embedded Audit Modules (EAM).
Kuhn et al. (2010) also explain that organizations focus more on strategic enterprise risk
management and for this reason the demand for continuous auditing is increasing. ERP
environments also demand for increased control procedures; because a lot of reliance is placed
on system controls and many errors may remain undetected in the enormous amount of data
(Kanellou et al., 2011; Alles et al., 2006).
Jans, Alles and Vasarhelyi (2013) raise some concerns related to continuous auditing.
They reason that in the data analysis every transaction can be seen as an anomaly in some
perspective. The internal auditor should have a thorough understanding of the possible point of
error and that the logic in the analysis is focused on this point. Debreceny, Gray, Jun-Jin NG,
Siow-Ping Lee and Yau (2005) add concerns about the independence of the auditor with the use
of EAM. If the internal audit department no longer makes use of a separate system, the
independence of the auditor might become questionable. They also state that with the use of an
EAM, the performance of the entire ERP system will be negatively impacted. A final point of
concern is raised by Kuhn et al. (2010), who raise the issue that large organizations make use of
16
various ERP systems and in each instance an EAM should be implemented and maintained. This
will have a negative impact on the availability of audit hours.
3.5 Other findings

Audit risk
The COSO framework states that internal control procedures should provide a reasonable
assurance for an organization to meet its objectives (COSO, 2013). It also states that there are
limitations to this, because internal control does not have an influence on external influences or
internal bad judgments and decisions. This means that the COSO framework recognizes that
internal auditors cannot identify and/or mitigate all company risks. Bedard et al. (2005) note a
contradiction on this for the IT controls, because of ISA regulations, at least in the US, which
highlights the importance of thorough understanding of IT, it’s no longer acceptable to default
to the conclusion of high risk on controls issues and avoid assessment of controls systems
weakness in audit planning.
Power (2009) concludes in his research for challenges on Enterprise Risk Management,
that the risk management in an organization is mainly focused on accounting and auditing norms
of control. This indicates that there would be less or no focus on risks, which are outside the
financial area of an organization. Power also states that the risk appetite, which can be translated
as the acceptable risk level of an organization, is becoming a tick-box exercise for management
instead of having a true focus on the organization’s risks.
Future of ERP systems

As stated above, ERP systems are not always matching the requirements of an organization (Hsu
et al., 2006); Grabski et al., (2011). A way to work around this issue is to make use of several
ERP systems, because one single ERP system cannot meet the requirements for all parts of an
organization (Kuhn et al., 2010). These points raise the concern if those organizations should
adopt an expensive and complicated ERP solution. Peng and Gala (2014) see a trend for
organizations to migrate their ERP systems into applications and databases in the cloud, as a
reaction on high investments and maintenance of ERP systems. Cloud computing is a network
model to use a pool of configurable computing services and in the form of hybrid cloud, various
clouds can be combined to make use of data from various environments (Mel and Grance,
2010).
17
4 Background
In this chapter I describe the TeamMate organization. I will also explain why this organization is
used as subject in this paper.
4.1 Wolters Kluwer Financial Services

Wolters Kluwer Financial Services (WKFS) provides software, expertise and service available to
organizations around the world to assist in with critical business decisions as compliance & risk
management and save & profitable growth. WKFS is the worldwide leader of compliance, risk
management, finance and audit solutions for the financial industry and utilize this expertise in
other industry segments. These solutions can help at all levels of the organization, including risk
and compliance challenges related to growth of the business by new or existing customers,
manage risk and performance of portfolios or optimizing risk based performance in the entire
organization.
The risk solutions provide a more comprehensive view of risk across multiple disciplines
within an organization and a deeper understanding of risk affecting financial organization’s
business. This includes risk items like: credit risk, enterprise risk, financial crime control, liquidity
risk, market risk and operational risk. The compliance solutions enable organizations to balance
increasing regulatory and risk management obligations with improving business performance.
The technology systems and services allow organizations to more efficiently adapt to changing
regulations, enhance data quality and break down operational silos. The compliance solutions
contain information about the various dimensions of regulations, policies, procedures and the
compliance and reporting related to these items.
The finance solutions bring risk management, compliance, finance and performance
together in a single architecture. This allows organizations to better control and manage financial
data as well as getting a clear organizational view and enhanced management of risk and
performance. The audit process is more efficiently managed by the audit solutions and allows
auditors to spend less time on documenting and reviewing and more time on providing value
added services.
18
4.2 TeamMate
TeamMate is a part of WKFS, which is developing audit management software systems to
increase the efficiency and productivity of the internal control process, including risk assessment,
scheduling, planning, execution, review, report generation, trend analysis, audit committee
reporting and storage. The TeamMate software contains a paperless strategy to manage audits
and eliminates the barriers between paper filled binders and disconnected electronic files, leading
to an efficient internal control workflow. The TeamMate software can be used as one fully
integrated system or as three stand-alone pieces of software:
- TeamMate Audit Management

- TeamMate Analytics
- TeamMate Control Management
TeamMate Audit Management (TAM) contains several modules, which provide a tool for
a streamlined audit process. In the software there is a seamless dataflow for the different audit
aspects as: risk assessment (to develop a risk based audit plan), audit documentation system,
scheduling of staff and audits and tracking of audit projects. TeamMate Analytics (TA) is a set of
tools which provide auditors a quick and easy analysis to identify unusual patterns and anomalies
in data. Internal auditors, fraud examiners, finance managers and accountants, in organizations
ranging from small single person departments to Big 4 audit companies, make use of TA.
TeamMate Control Management (TCM) is software developed to address financial reporting
standards compliance as SOX. The TCM software provides a flexible relationship between
entities, processes, financial statement accounts and other reporting structures to facilitate
filtering and sorting of key information.
4.3 Why TeamMate

TeamMate is relevant to this research as the audit planning tool is used by more than 90,000
auditors around the world. Both the support to the implementation of the software as the annual
surveys held with the users, which are used to further develop the software, give TeamMate the
expertise in Internal Control Planning. The interviewees from the TeamMate development and
consulting departments are not performing internal audit planning currently, but they are aware
of the internal audit planning process, because they work closely with clients to properly
configure their audit planning process in the TeamMate system and most of the interviewees
have experience in internal audit planning from prior internal auditing roles.
19
The applicable systems settings can be found in:
• TeamRisk: a risk assessment tool to generate audit plans and compare risk with COSO,
Basel Committee on Banking Supervision, Institute of Internal Auditors. Score of
selected risks, populate custom measures.
• TeamEWP: a documentation system to spend less time on documenting.
• TeamCentral: an issues tracking database of audit findings and key statistics.
• TeamTEC: a time and expense capture and reporting tool.
• TeamSchedule: a tool to schedule staff and audits.
• TeamStore: a companion tool that houses best practice work programs and workpaper
templates
The settings that will be most applicable to the research question are within TeamRisk and
TeamSchedule. These areas contain detailed information which part of the processes within the
organization contain the biggest risks and how the audit planning is designed to cover these
risks.
5 Findings
This chapter contains the results from the interviews and documentation review. This chapter
has the same structure as the interviews. It starts with general questions about the audit process,
after which it will continue with paragraphs per detailed research question. It ends with a
paragraph with additional findings. The general interpretation of the interviews is described and
will lead to a conclusion to answer the detailed research questions. Interviewee citations are
placed at the end of each section and will support the interpretations coming from the
interviews. Each paragraph ends with a conclusion coming from the interviews.
5.1 General information internal audit planning process

Goal internal audit
The main goal of internal audit is to give assurance to the board on the risks in the organization
and the reliability of information by performing an independent review. This includes testing of
business processes that these are working as intended and to report if processes are not working
as intended. Internal auditors also assist in the mitigation of risks and remediation of
irregularities in the processes. Any errors in transactions, whether these are intended or
unintended, will be investigated by internal audit and they will advise mitigations to prevent
damaging actions to reoccur. As long as internal audit is only advising in this process, they will
keep their independence, which is important to properly review those mitigations. Internal audit
is aware that it is close to impossible to prevent any errors ever to occur at all.
Apart from this classical role of internal audit, in the last decennium the role is shifting to
a more advisory role. Internal audit keeps track of companywide best practices, which can be
used to assist departments to organize their processes. When systems or processes are newly
implemented or changed, internal audit departments are asked to assist in testing the setups and
controls of these systems and processes up front, as a post launch adjustment to this system or
process is always more complicated.
Interviewee #9: “I would say the main goal of an audit is that the chief audit executive gains an
understanding as to how certain parts of the business or a certain process of the business works. Ideally to gain
assurance that it’s working or that things are as they should be, but if they are not that they identify those issues,
identify problems that might impact the business. And they work with the management to put in place a process
for remediating them.”
21
COSO framework
The TeamMate system used to be built aligned to the COSO framework. From customer
experience, the TeamMate products are now more focused on the risk assessment part from the
COSO framework. There has certainly been a shift over the years towards the use of the COSO
framework in organizations. To the question if internal auditors perform their profession with
the use of the COSO framework, no uniform conclusion an be formed.
According to some of the interviewees the COSO framework is not used, because
internal auditors will have to spend time on explaining the principles of that framework to the
various stakeholders. As internal auditors are already pressed for time, they would rather use
terminology the company is familiar with. Aside from the terminology, there is also a concern if
COSO is used as intended: to monitor the entire internal control system of a company. The risk
assessment of the COSO framework is used, but maybe that is too much focused on the
economic side of an organization. Other parts of the COSO framework, for example the
monitoring activities are not used that frequently in internal audit.
Other interviewees respond positively to the question and state that the COSO
framework is focused on risk in the company environment and how those risks are controlled.
They state that internal control is also focused on control of risk, so the COSO framework is
used in internal control. The terminology used by internal controllers, like risk assessment and
control activities, is used in the COSO framework as well.
The interviewees contradict in their answers, which leads to an interesting discussion
outside the scope of this research. An alignment between the COSO framework and internal
audit is recognized in all interviews. If the COSO framework is not entirely used by internal
auditors, at least it’s a starting point and used within systems as TeamMate. For sure the risk
assessment is used as can be interpreted from the COSO framework.
Interviewee #6: “Corporately they will tell you that they follow the COSO framework. They’re monitoring
risks and measuring them and they will identify controls. But if you take a look at what the COSO framework
was supposed to be for, you realize they don’t really follow it.”
Interviewee #11: “You'd say that most of it is based on the COSO framework. It comes back a lot in
literature and I think there is quite a lot of reference to it. In fact it is also a question of what controls you have in
your environment designed to hedge risk and that’s what COSO is all about.”
22
Risk assessment
By definition, an internal audit department only exists in larger organizations and larger
organizations tend to be more complex. The internal auditors do not have sufficient resources to
perform full companywide audits and for this reason, internal auditors want to focus on the areas
of high risk. From the interviews it is not certain if the COSO framework or a risk assessment is
obliged for larger companies, but performing a risk assessment will identify the areas of high risk
in large complex organizations.
The majority of internal auditors are no longer checklist driven departments and are
performing a true risk assessment. In the past the risk assessment would only take up to 25% of
the budget of an audit, nowadays it takes up to 40% of the audit budget. This states that the risk
assessment has become a more important part of the audit process and the audit planning
specifically.
The organizations that are audited are large and complex. The organization is split up in
various entities, which can be a business unit or a project, on which an audit be performed.
These entities together are called the audit universe and each of these entities has specific risks.
For a risk assessment it is impossible to identify each specific risk and to compare all risks to
state which has a higher risk ranking. The internal audit department will perform a risk
assessment at high level, with similar risk factors, to identify which entities are to be audited.
When an entity is being audited, a risk assessment is performed at a granular level.
Interviewee #1: “What I’m seeing now, how internal audit has evolved, is that true focus on risk. I would say
that probably 90% of the clients I work with do a true risk assessment as part of their audit planning.”
Interviewee #4: “The main goal for a risk assessment, is to really to be able to stand back and from a very
high level to be able to focus in on areas that are of higher risk. So that way we can then perform and audit during
that particular year that will further assess those risks.”
High level audit planning

The high level audit planning is performed on an annual basis, or a similar timeframe, and results
in a list of entities to be audited in that timeframe. The audit planning process starts with the
creation of an audit universe, which is listing all auditable entities. If an audit universe is already
identified, it only needs updating by adding new investments or projects, eliminating any
divestments and possibly combining auditable entities which have been merged.
Then the risk assessment is performed, starting with a review of the strategy for each
auditable entity within the audit universe to get a good understanding of the environment or
23
business that it’s in. Internal audit defines the risk factors, from both the company wide policies
as well as the entity strategy. These risk factors are given pre-defined specific rating criteria. The
risk assessment continues by rating all the entities of the audit universe on these risk factors. This
creates a priority list of entities with the highest risk. The outcome of the risk assessment is
discussed with the audit committee and the board of directors. Any concerns may change the
priority of the entities. When the audit planning is set, then this is discussed with the entity
management to create a more exact planning of when the audit can take place.
Interviewee #2: “So what we do is we try to list all those entities and processes, create an audit universe and
then we have defined risk criteria and we rate all the entities based on those criteria. So we have defined those risks
criteria and we have defined how we rate those criteria. And then based on the outcome of that we have the riskier
entities and those are the ones we should be focusing on.”
Granular level audit planning

The high level audit plan has been set and internal auditors start with the granular level audit
planning. The auditors will perform a review of the entity and by interviews with management
and senior staff and reading of reports a better understanding is gained of the product portfolio,
business processes, objectives and the managers opinion of risk within that entity. This can be
compared with the review of the previous audit and the high level review of that entity to see if
anything has changed.
A risk assessment is performed at granular level to identify controls and potential risks
within that entity. Based on that risk assessment a granular audit plan or testing plan is created.
The auditors will then execute the testing from the audit plan and if new risks are identified, then
a new risk assessment is performed including the new knowledge. From the findings of the audit
execution, an opinion is formed. Finally a report is issued which is stating the issues and possible
advice for mitigation.
Conclusions
From this paragraph can be concluded that the interviewees have a very aligned vision of the
reason for internal audit and which processes are used. All interviewees state that the risk
assessment has grown to become an important part of the audit planning process and that the
audit planning and the risk assessment as part of it, is performed at a high annual level and at a
granular engagement level. Although there is a discussion if the audit planning is sourced from
the COSO framework, generally can be concluded from the interviews that they are related.
24
5.2 Main concerns in ERP systems
ERP systems are defined as large complicated systems, which have an impact across the entire
organization. They are modular in setup and almost every department is using that same system
and all data is stored in one database. This places a lot of reliance on one system as all staff is
depending on the same system and database.
The interviewees consider ERP systems expensive and difficult to implement. The
challenge in the implementation is to properly setup system controls which are matching the
requirements from each department working in that system. Another point of attention is that an
ERP system is basically forcing an organization to adapt the process flows as designed in that
system. This means that the system is not adapting to the requirements of the organization, but
the organization squeezes it’s processes in the designed process flows of the system.
Organizations should be aware of this when making the choice of purchasing such an ERP
system.
Access management
ERP systems are using one database which many departments and their staff are using. One of
the major concerns from the interviewees was related to access management. There is a big
concern on who has access to data and who can change it. An ERP system and its database
contains company wide information and every single person, internal or external, is allowed to
have access to view or even change that information.
A proper setup of the segregation of duties in system controls together with security
controls become essential. If the system controls such as access controls are not setup correctly,
then a person could have access or even change company information. A strict segregation of
duties is required between the maintenance of master data and the usage of it. For example you
cannot have a purchaser have access to bank information of the vendors, because he or she
might change that, which may result in incorrect payments. However if access controls are setup
properly then the use of an ERP system gives a solid mitigation for fraud.
Process flows
Another point of concern are the process flows within an ERP system. ERP system process
flows are usually well tested before going to market, but those process flows always need to be
tested to ensure the proper information is coming from the ERP system. Especially if the ERP
system is unfit for the organization and moderations in the system or in the process have been
25
adjusted to make it fit. This gives great concerns about the accuracy of data and the reliability of
the information. When an ERP system is used as intended and with a fit to the company process
flows, then this will give more assurance to the accuracy of data and reliability of the
information.
Interviewee #2: “Access management. Because if access is not managed correctly you have segregation of duties
issues. Second thing is how the process flows in the system.”
Interviewee #7: “The first concern is probably going to be the accuracy of the data. You want to look at the
reporting that is coming out of it. Making sure there is a correct security in place. So making sure that people don’t
have access to information they shouldn’t have.”
Conclusions
This paragraph describes that organizations that use ERP systems are gaining communication
between departments, because of the integration of the various modules. They also raises the
risks of improper fit with the organization and incorrect configuration of those systems. From an
internal audit point of view ERP systems raise concerns about access management, including
segregation of duties, and process flows within the ERP systems, resulting in the reliability of
information. If an ERP system is matching the organizations needs and setup properly, then this
will give powerful system controls to mitigate risks.

IT Controls
As stated in the previous paragraph, a lot of reliance is placed on an ERP system as it is a big
part of the organization. If the system is not operating or data has been corrupted then this could
cost millions for an organization, because the entire organization will not be able to operate.
In a non-ERP environment the systems and databases are more scattered and for that
reason the risk of an entire organization not to operate, is scattered as well. If in such an
environment a system is not operating or a database is corrupted, only part of the business might
not be able to operate and that will make the financial impact lower. For this reason a backup
procedure or a disaster recovery procedure is more important for an organization with an ERP
system then for an organization which makes use of more scattered systems. In the high level
risk assessment this can be taken into account and therefore impact the high level audit planning.
26
Interviewee #11: “You look what are your critical systems and how is the backup procedure, recovery
procedure and alternate location. What happens if there is a power failure and everything is down? Costs could be
millions a day, globally said. What do you have as an alternative? “
ERP implementation or adjustments

In the high level audit planning and risk assessment the only concern which is related to systems
is the implementation of a new system or changes in the process flows of existing systems. The
concern is mainly in the process flow and in the control settings. For example if a change has
been made in the process flow on the input side, then how does that change impact the
information on the output side. This concern is not limited to ERP environments, but as ERP
systems contain a high complexity, the impact of this change could result in a higher risk ranking
in the risk assessment.
Interviewee #3: “One of the big things that would trigger for a specific entity a higher ranking in the overall
risk in the annual planning, if it changed systems. When something is business as usual, you can have a little bit
more comfort that everything is running ok and you can assume that they are setup ok. There’s a lot more risk in
an entity that is going to roll out a new system, to completely replaces an old system. So that would cause an entity
to be rated a lot more risky.”
ERP no impact
In the interviews no other impact of ERP systems on the high level audit planning has been
raised. Both mentioned items can be included in the high level risk assessment and especially the
risk of improper IT controls can have a major impact, but the likelihood is small and for this
reason often does not impact the risk assessment much.
Interviewee #3: “I wouldn’t say it particularly impacts the planning in a sense that we know that regardless
of whether there is a monolithic system or multiple systems in place, we will still be looking at the same scope areas
if we go to an entity.”
In the documentation from the internal audit department the risk factors, which all give
rating values between 1 and 5, are listed based on which auditable entities are rated. Most of the
risk factors are purely financial, in example the variance between last year EBITA and budget.
27
Other risk factors focus on the change of product mix, acquisitions or customer assurance.
These type of risk factors have no relation with he use of an ERP system. Out of the fourteen
risk factors only one can be related to ERP systems, which is the risk factor to score changes or
transformations in the processing. In this risk factor entities which are going through a change of
applications or do a system implementation will get a higher risk rate. This is in line with the
statements above, but is not limited to ERP systems.
Conclusions
This paragraph concludes that the impact of ERP systems on high annual level risk assessment is
minimal. The IT controls of disaster recovery and backup procedures should be taken into
consideration, because of the dependence on one single system. An additional note is that the
implementation of an ERP system or changes within the ERP settings can trigger an entity to
become rated with a higher risk.

Manual data entry is the area which raises more concerns to internal auditors then automated
data entry, such as using scanning devices. More testing or bigger samples are tested at a manual
data entry process and this increases the time required to perform the audit. In an ERP
environment data is entered only once in the system, which means that data needs to be entered
correctly in that entry. There is no opportunity to match data input from various databases, as
there is in a non-ERP environment. This raises the question on how this will impact the granular
audit planning, not if it will impact the granular audit planning.
The data entry is performed by decentralized departments, which might not have an
understanding of the impact of these entries. For example a sales order is entered by the sales
department and this eventually impacts the financial reporting. The sales department does not
have specific finance knowledge and for that reason is not aware of the impact of an entry.
Opposed to that, the sales person does have expert knowledge of a sale and likely has better
knowledge if a sales order has actually taken place. From that perspective the data entry may
contain less risk.
In a non-ERP environment where multiple data entries are used, the data is entered
centrally at the accounting department, which does have specific knowledge of the financial
reports, but lacks the knowledge of the actual sales order. In such an environment there will be
double the quantity of manual data entries and double the samples to be testing as internal
28
auditor, increasing the audit time required and by that also the audit planning. With the use of
interfaces, eliminating the multiple manual data entry, there is the great concern of matching
data. What happens to the data which is in transit? It raises a lot more concern over data
accuracy. The fact that the same data is in two or more databases, which can be used as a
reconciling method, does not benefit in audit planning as much as the additional work auditors
have in testing, because comparing different databases in a non-ERP environment is not that
effective or easy.
The fact that data might be entered in different geographical locations does not have an
impact on the audit planning. There are risks of communication issues or cultural differences,
but in a non-ERP environment that potential risk will be the identical.
Interviewee #6: “It depends on whether the people who are doing this data entry understand the implications
of everything they do. If the people who enter the data understand what the information is used for, then it will be
ok to have them enter the information. But if they don’t understand the purpose of it, therefore they don’t think
they need to be 100% accurate on things, it will impact all the way down the chain.”
System controls
In an ERP environment the risks which manual data entry contains, as raised above, can be
mitigated by system controls, as briefly mentioned in paragraph 5.2. The risk of incorrect data
entry can be mitigated by having a second person checking on the data entry. Training can also
help to ensure that data is entered in the appropriate fields and at the same time create awareness
of the impact of their data entry. These two mitigation methods are not as strong as the system
controls, which an ERP system offers.
Form masks or field limitations can be set in the system controls of an ERP system.
These system controls can ensure that all fields required are populated at the data entry and that
fields are entered with a certain logic to it, for example using thresholds in amounts or not being
able to use future or past dates. A proper configuration of these system control settings will lead
to a powerful mitigation. These system control settings will raise concern in an audit and will be
thoroughly tested, but that will save a lot of time in the overall audit time and therefor will have
an impact on the audit planning. The testing of data control settings requires specific knowledge,
which differs from the knowledge when testing manual entry samples. The internal auditors
expertise will shift to become less operational auditors to more IT auditors.
29
Interviewee #8: “In fact an ERP has a single database with multiple points of entry. If you configure that
correctly then it’s really powerful. If you don’t configure that correctly or if you’re using more databases, then you
have a problem. Then you don’t have the advantage of an ERP system. The more unambiguous you configure the
ERP, the better the controls are and less risk and less audit. And the other way around, if you increase
complexity, then that increases exponentially.”
Interviewee #3: “If you have a fully integrated sales order entry and bookkeeping system and fulfillment
system. If that is all in one, we will then don’t need to spend quite as much time looking at that, because you know
if the order was entered right and if it’s been fulfilled, then in theory everything in between went well. We might
focus more on change in processes, systems, discount procedures, credit notes.”
Audit preparation
An ERP system contains all its data into one database. This characteristic is very beneficial for
the internal audit department as data is more quickly available and easier to interpret. All the data
is stored in one place, which results in a single place to retrieve the data as well. There are
possibilities for internal auditors to retrieve the data themselves, although that does raise some
concerns in how the retrieving data script is impacting the data base. The data will be available
in a consistent format and for that reason easier to use. If the same database is used over the
years, then the benefit only increases. The internal auditors will only have to evaluate one set of
data, which means they don’t have to familiarize themselves with the different outputs from
different systems.
Another beneficial point is that the sample size will be stable. If you have a maximum
sample of 10.000 entries, in an ERP environment you will only have 10.000 entries to test, no
matter if you have 2 million or 10 million data entries. Whereas in a non-ERP environment for
every database there will be a sample of 10.000 data entries. For this reason the use of an ERP
system reduces sample size tremendously and therefore audit time is reduced and audit planning
as well.
Continuous auditing:
On September 22, 2014 Wolters Kluwer announced the launch of an analytical tool within the
TeamMate systems (Wolters Kluwer Financial Services, 2015). The tool itself is briefly described
in paragraph 3.2 of this paper. Wolter Kluwer Financial Services see TeamMate Analytics as a
tool which allows internal auditors to easily analyze big data and to limit time in engagement
testing.
30
It becomes a requirement for internal audit departments to have skilled staff and to have
the proper tools in place to perform data analytics. More and more internal auditors are
performing data analytics in their audit testing. Big data is retrieved from a system and is
analyzed for certain criteria. This way of auditing is replacing the sample testing, because auditors
can, with the use of data analytics, review more transactions in less time, then they could with
sample testing. It will save the auditors a lot of time, while still reviewing more transactions.
This seems to be an auditors dream to put data in a tool and that the tool will identify the
problem areas. Data analytics does raise the issue that the tool is only as good as it has been
setup. If an auditor is using the wrong criteria on the data, then the tool cannot identify the
problem areas properly. Another prerequisite is that the data is in the same format. In non-ERP
environments it will be more challenging as the various databases will contain data in different
formats.
With the use of an ERP system, data analytics can be very powerful and internal auditors
will not be limited by sample size. This means that they will be able to audit 100% of the
transactions. Then internal auditors will move towards continuous auditing. Entity controllers
will perform continuous monitoring, which is reviewing 100% of the transactions in real-time.
The roles of the controllers and the internal auditors will be very aligned, where the controller is
checking all the transactions of accuracy and the internal auditor is assuring the accuracy of all
the transactions.
Interviewee #5: “People are becoming more aware of analytical procedures and being able to administer and
then save time and being able to look at more things. And just become more efficient in your process. It’s not so
much as becoming limited, it’s more that you will be able to cover more. When you’re dealing with analytical
procedures and being able to rely on those results, you’ll be able to test a 100%. Using an analytical process. I’m
not talking about comparison from this year to last year, but digging much deeper. Looking for information, using
statistical methods. Like if there is a normal distribution to evaluate information. Being able to look at outliers.
Things like that.”
Interviewee #9: “I think the advantage to this type of testing is that you don’t need to limit yourself to a
sample size. There are tools now that auditors can use where they can use analytics to test an entire data set. So
they don’t necessarily have to rely on small samples of data to gain assurance. Ideally or what is the trend in the
industry is towards empowering the business so that they can have their own controls in place, so towards
continuous monitoring.”
31
Conclusions
The conclusion paragraph of this paragraph is that manual data entry is considered the main
cause for incorrect data and if the number of manual data entries is increasing, then the time to
perform an audit to increase accordingly. Interfaces limit that risk, but raise other types of risk,
such as consistency of data between the different systems and this will lead to more checks and
so audit time as well.
With the use of an ERP system data is only entered once and by several, decentralized,
departments. The single data entry, without the possibility of comparing data between systems,
does not necessarily lead to a higher risk of incorrect data, because that risk can easily be
mitigated by access controls, field entry controls and training. Important is that these mitigating
controls will have to be tested in the audits. The fact that data can be entered on different
geographical locations in an ERP system is no different from non ERP systems, so this does not
have an impact on the use of an ERP system on audit planning.
Beneficial from ERP systems is that data is more quickly collected and easier to interpret
as all data is centralized and in the same format. This will increase the possibilities for internal
auditors to use data analytics and grow more towards continuous auditing.
5.5 Other findings

Audit risks:
One concern outside the scope of the research which was mentioned most frequently was the
lack of available time to perform audit testing. Internal audit departments are focusing on the
areas with the highest risk and perform their audit planning based on capacity of the internal
audit department. A result from this is that indeed the highest risks are reviewed, but not all risks
identified are covered in the annual audit plan. That also applies to the engagement audit as the
auditors feel stressed for time to perform the tests and not have sufficient time to thoroughly
review the entity. As a contradiction to this is the possibility that internal auditors are
perfectionist, because if in an entity 99% is going well and 1 % is not going so well, then this 1%
will still be reported. Although the task of an internal auditor seems to be of that nature.
Another source of audit risk is not reviewing the highest risks. Internal auditors are using
a lot of their own judgment in the risk assessment and in the reporting on an audited entity. It is
possible that internal auditors do not receive all information from entity management or that the
information is misinterpreted. The result will be that present risks are unidentified or rated lower
in the risk assessment and therefore not be included in the annual audit plan. This risk can be
32
mitigated by trying to objectify the risk assessment instead of performing it on subjective audit
judgments.
Experience in the internal audit department will mitigate both risks mentioned above.
Another way to mitigate these risks is by use of an electronic audit planning tool such as
TeamMate. In this system the audit universe is registered and the risk factors of the high level
risk assessment can be placed. This ensures that every entity is rated with the same logic and
gives a good assurance that no risks are missed. Another benefit is that previous audits are kept
in a database and can be used for granular audit planning. It can show what audit time is required
for entities with a certain magnitude or what has been an issue at a particular entity in the
previous audit. A downside of using such a tool is that it might increase the dogmatism of an
internal auditor. This means that the internal auditor will perform his tasks as required, but is not
really looking into the true risks of an entity and becomes more checklist driven, because the
auditor is more focused on the audit tool instead of the auditable entity.
Interviewee #6: “What we do see across the board is underfunding and understaffing of some of those activities
that would provide some more insight into the corporate as a full. Whether that training on monitoring on the
back end of the process, or giving them more time to do a better or more detailed job at the risk assessment up
front. It’s really about how much work can an internal audit department take on in a year. As they do a lot more
then just audit, like follow up on issues. So if you plan the audits for a year, it’s based on the best guess on how
long one single audit is going to take. A lot of internal audit managers will say that it is a calculated risk in low
risk areas. That does make the assumption that your risk assessment value up front was correct. And if it has
been assessed with low risk, that doesn’t mean it’s no risk. Maybe it’s not a huge financial risk, but it might be a
reputational issue.”
Future of ERP systems:

Paragraph 5.2 already shows that the use of an ERP system is not always the best choice. Some
organizations are simply too scattered or too specific in their processes and products, that an
ERP system does not suit them. In practice its very rare that a company makes use of an ERP
system in the true essence: various modules using one single database. In almost all companies
who use an ERP system, have some part of the organization working outside that system. A
well-known example is the payroll process. This is hardly ever integrated in the ERP system.
Also internal audit systems are never integrated, as this may raise a discussion of the
independence. It seems that ERP systems are only useful to large companies who have very
industry standard processes. These organizations can really make use of an ERP system even
33
though some activities, which are not part of the core business, will remain outside that ERP
system.
This raises the question of where the future is for ERP systems. Current technology
seems to be growing towards a cloud solution, where every department has its own database and
all these databases use the same structure and language. The data can then be combined in a data
warehouse in the cloud.
Interviewee #8: “It's actually a bit the tragedy of yesterday. In the sense that what you would expect nowadays
is very much a best of breed from cloud. I think that that is the future of ERP systems. What you see in history is
that an ERP is a straitjacket, within which manufacturers try as best as possible to facilitate different industries
by an ERP system in modules to cut and which to tailor to modules that this industry expect to need. But that has
massive limitations.”
Conclusions of this paragraph:

From this paragraph can be concluded that audit departments are facing various audit risks and
that the use of an audit planning tool can be a solution, provided that the audit universe and risk
factors of the risk assessment are properly maintained. The final conclusion is that new
technology may eliminate the use of ERP systems.
34
6 Discussion
The findings from the interviews will be reflected on the literature in this chapter. The
paragraphs in this chapter are identically structured as the chapter from the findings and the
literature. In the subparagraphs there is a small variation in the literature, where the high level
and granular level audit planning subparagraphs are combined.
6.1 General information internal audit process

Goal internal audit
All interviews started with a general view about the internal audit purpose, tasks and processes.
The interviews result into a uniform view of internal auditing. The goal of internal auditing is
providing an independent review of the processes and data within the organization to give
assurance to the board. The interviews also indicate that an advisory role is added to the tasks of
an internal auditor. The auditors are more and more involved in system implementations and
changes. In the interviews unified process steps were given for the audit planning process,
indicating that internal auditing can be seen as a process which can be standardized.
The literature review provide the same description about the audit goal: producing
assurance in the organization, as a result from an independent and objective validation of that
organization (Power, 2013; Kanellou et al., 2011). Additional tasks are also identified in the
research of Barret et al. (2005), in line with the result from the interviews. From the COSO
framework can be interpreted that internal auditing can be seen as a process (COSO 2013),
which is in line with the findings from the interviews. This is in contradiction with Power (2003)
and Ditsmith et al. (as cited in Power, 2003), who explain that internal auditing cannot be
standardized.
COSO framework
In the interviews a discussion arises whether internal auditing is following the steps of the COSO
framework. Where some interviewees respond that the COSO framework is not used, others
answer that internal auditing is so closely aligned with COSO, that it is. The consensus from this
discussion is that the COSO framework might not be used to its full potential, at least the COSO
framework can be the basis of internal auditing and some parts, like the risk assessment, are
used.
Bedard et al. (2005) find evidence that not all risk factors of the COSO framework are
used in audit planning and Hsu et al. (2006) state that the COSO framework can be used as
35
consideration. These statements are in complete agreement with the discussion from the
interviews and lead to the same discussion.
Risk assessment
The interviews state that internal audit departments only exist in larger organizations and those
are usually more complex. As a result it is impossible for an internal audit department to review
all auditable entities in detail. Most internal auditors are now performing a risk assessment to
identify the risk areas to audit in order to plan where audits will take place, instead of being
checklist driven. The risk assessment is taking a relative big portion of the audit budget,
indicating the importance of the risk assessment as part of the audit planning. It is impossible to
rate every detailed risk in every entity and compare these risks with each other, for that reason a
high level risk assessment is performed on annual basis. This high level risk assessment reviews
and rates company wide risk factors on each auditable entity and those with the highest risk rate,
are planned to be audited in the following year. When the internal auditors are actually starting
the engagement audit, another risk assessment is performed at a granular level, to identify the
specific areas to test within that entity.
Low (2004) recognizes the complexity within organizations in the same way as the
interviewees. The COSO framework defines risk assessment as one of the five integrated
components, which is focused on identifying risks in an organization, which is in line with the
interviews. Auditors have been performing some sort of risk assessment for each year, as can be
concluded from Gaumnitz et al. (1982), which differs from the conclusion from the interviews,
although in his research he focused on external auditors. Bedard et al. (2005) and Power (2003)
agree with the interviews that the risk assessment is an important part of the audit planning
process. COSO (2013) does state that the risk assessment should include risks at all levels of the
organization, but there is no evidence in the literature that the risk assessment is performed at
both a high and granular level, as described by the interviews.
High level audit planning & Granular level audit planning

In the interviews detailed process descriptions are given about the high level and granular level
audit planning. In both processes the risk assessment takes a prominent stand. In the literature
no process description is found and COSO (2013) only states that a risk assessment should
include risks at all levels of an organization, which to some extend agrees with the interviews, as
all entities should be reviewed.
36
6.2 Main concerns ERP systems
The interviewees escribe ERP systems as large, complicated and expensive. These systems are
modular in setup and are used throughout the entire organization, while making use of one single
database. This places a lot of reliance on the ERP system. ERP systems are difficult to
implement, because the system setups should answer to the needs of all departments and often
the organization needs to change its process flow in order to fit in the ERP system.
The literature explains that ERP systems are integrated cross-functional modules, used by
multiple departments of an organization (Kanellou et al., 2013; Hsu et al., 2006; Robey et al.,
2002; Scapens et al., 2003). Hsu et al. (2006) and Grabski et al. (2011) further explain that ERP
systems are monolithic systems, which are difficult to adjust to the needs of an organization and
as a result the organization is shaping its processes to fit into the ERP system. These statements
are in complete agreement with the conclusions from the interviews.
Access management
As a result of the data being stored in one single database, the interviewees have major concerns
about access to this data. This concern relates to both the misuse of confident company wide
information and the possibility to change the centralized data. System controls which maintain
the segregation of duties are becoming essential for an organization and proper setup is required,
for example the split between usage and maintenance of master data.
The conclusions from Grabski et al. (2011), Hsu et al. (2006) and Hunton et a. (2004) are
the same as from the interviews. They also state that with the use of an ERP system, where
company wide information is centrally stored, there is a big concern about the access to this
confidential information. Firewalls can mitigate the access risk from outside, while a proper
segregation of duties will mitigate the access risk from within the organization.
Process flows
The second major concern which the interviewees raise is about the process flows within an
ERP system. The process flows should be reliable as the ERP system is tested on proper
processes before going to market, but the internal auditors cannot rely on that and will be testing
the process flows intensively to be able to give assurance about the accuracy of data and
reliability of the information from an ERP system. The fact that ERP systems are not always
aligned with the company’s requirements and therefor one or the other is changing processes,
gives more reason to test the process flow.
37
Haug et al. (2009) explain that data is created in every step of a process and that the
quality should be of adequate level in order to make proper decisions. The integration of the
various modules in an ERP system, make the process steps more interdependent and this raises
the concern of control risks (Hunton et al., 2004; Hsu et al., 2006). This is in line with the
conclusion from the interviews. Grabski et al. (2011) contradict to this statement as they claim
that ERP systems have reliable controls built within. Hsu et al. (2006) replies that the standard
controls within an ERP system are not always properly configured, because controls are slowing
down processes. The interviewees recognize the existence of the build in controls and that those
need to be tested, although they give a different reason of why intensive testing is required on
the process flows.

IT controls
The interviews explain that the reliance on one system and one database includes risks like
availability of the system and corruption or loss of data. Internal auditors will look at the
mitigation of the risk that the ERP system is not operating and as a result the company is at a
standstill. The auditors will also review and test the backup procedure to mitigate the risk that
data is corrupted.
COSO (2013) states that an organization should have controls in place, also for the
technological environment, to mitigate the risk to acceptable levels. This can be interpreted as
being in line with the interviewees. Hunton et al. (2004) clearly state that there is a high risk in
business interruption, which is clearly in agreement with the interviews, although they do not
specifically mention back up procedures.
ERP implementation or adjustment

Adjustments to an existing system or implementations of new systems raise the rating in the risk
assessments, as can be concluded from the interviews. Entire process flows will need to be tested
or re-tested. As it impacts the high level risk assessment, this will also impact the high level audit
planning. The interviewees also remark that this applies to both ERP environments as non-ERP
environments.
Any changes, including implementations, to systems should be included in the risk
assessment (COSO 2013). Grabski et al. (2007) continue that the internal audit expertise should
be used in an ERP implementation and Grabski et al. (2011) recognize that ERP systems
continue to change after the implementation. These statements combined are in agreement with
38
the interviews that system changes or implementations raise the concern of an entity in the risk
assessment.
ERP no impact
The interviews give an indication that the above two concerns may have result in an impact of
ERP systems on high level audit planning. The interviewees also state that if there is an impact,
then it will be limited. From the review of the high level risk assessment ratings of an internal
audit department, becomes clear that only the change or implementation of a system has an
impact on the high level risk assessment, although no distinction is placed between an ERP
system or a non-ERP system. This results in the conclusion that an ERP system has generally no
impact on the high level audit planning.

Manual data entry is being tested a more and with bigger samples then automatic data entries,
such as the use of scanning devices, as stated in the interviews. In an ERP system the data is
entered only once, instead of multiple times in a non-ERP environment, so this will reduce
testing time in the granular audit planning. The fact that an ERP environment does not offer the
possibility to compare inputs of the same data in the different databases, does not raise any
concern for the interviewees, also because training and system controls on the data entry fields
can mitigate that issue.
Also the fact that data is entered by various departments, while impacting on centralized
reports, such as the financial statement, does not raise any concerns for the interviewees. They
reason that it will increase the quality of the reports as specific knowledge will be used for the
data entry. The use of various geographical locations with different cultures, is not different with
the use of an ERP system.
Hsu et al. (2006) explain that the human factor has a big impact on the business risk and
is caused by lack of involvement, lack of knowledge and high stress levels. The lack of
knowledge can be interpreted as human error in the data entry, as was raised in the interviews.
Any error by staff will have an effect on the rest of the process, because between the connected
cycles no verification or rectification is possible (Hsu et al., 2006). This contradicts with the
interviewees as they do recognize that this control method is not possible, but they don’t see this
as an issue to impact the audit planning. Haug et al. (2009) continues more in line with the
interviews, as he states that outside an ERP system inconsistencies may occur between databases,
39
which lead to more testing samples. Kang et al. (2004) and Hsu et al. (2006) conclude that
adequate training will mitigate the manual system entry risk, which is comparable to the specific
knowledge and training of decentralized department staff who make the data entry.
System controls
The interviews explain that the risk of data entry errors can be mitigated by inserting a check by a
second person in the process or by training, but also by system controls. System controls can be
configured in the ERP system by creating data forms in such a way that all fields need to be
populated and that these fields do not accept completely incorrect data. If these settings are
configured properly, this will generate a powerful control tool to limit the risk of data entry error.
The internal auditors will test if these system controls are working as intended. To be able to
review these system controls, specific IT knowledge is required. As a result more IT auditors are
involved in the risk assessment and the testing of an ERP environment.
Hunton et al. (2004) and Hsu et al. (2006) recognize that the single point of data entry
and the interdependencies of departments, raises the requirement of using system controls, in the
same way as the interviewees do. Grabski et al. (2011) state that the system controls within an
ERP system are already tested and very powerful, although Hsu et al. (2006) contradict this
statement by adding that those system controls are not always configured as intended and that
the system controls need to be tested by internal auditors. The last statement is inline with the
conclusion from the interviews. Hunton et al. (2004) finds evidence that IT auditors should be
included in ERP testing, as financial auditors do not recognize the risks in system controls in an
ERP environment. Kanellou et al. (2011) also acknowledges the requirement for IT expertise in
an ERP audit.
Audit preparation
The use of an ERP system has benefits for an internal audit department, as the total sample size
will decrease. Internal audit departments are using a maximum of transactions within a sample
and as there is only one database to sample from, instead of various in a non-ERP environment,
this will limit the total sample size. Another benefit from the use of one system and one database
is that the data is more quickly available and all data is in the same format, which decreases the
time an internal auditor requires to perform the sample testing.
Grabski et al. (2011) also recognizes the benefit of all data coming from one system and
one database. Data is more quickly available, although this depends on user specifics as system
access and knowledge, and easier to use as all data is in the same format.
40
Continuous auditing
In the interviews is stated that internal auditors are using data analytics more often and the
profession is even requiring internal auditors to gain knowledge of this. With the use of data
analytics auditors can review a lot more transactions, up to full population testing of a database.
A next step is to perform continuous auditing, although from the interviews can be interpreted
that it is only possible if all entities of an organization are making use of one single system and
database: an ERP system. A note is also made that the analytical tools area only as good as
they’re setup. If there is compliance to this prerequisite, then continuous auditing, with the use
of data analytics, can be a powerful tool and the role of the controller and the internal auditor are
growing more closely together.
Kuhn et al. (2010), Kanellou et al. (2011) and Alles et al. (2006) all recognize the
increasing demand for continuous auditing, as management is more focusing on strategic
enterprise risk. They also state that the use of highly integrated systems, as ERP, are a
requirement to grow into that direction. Jans et al. (2013) points out that in order for continuous
auditing to work properly, the internal auditors should be well aware of the tests which they are
performing, because if the setups are incorrect, the auditors will not cover all risks. Although the
interviews to not state that there is an increased demand for continuous auditing, the statement
that the use of ERP systems are a requirement and that continuous auditing requires proper use
of data analytics, are in line with the literature.
6.5 Other findings

Audit risk
The interviewees raised additional concerns about the internal audit profession and audit risk was
a concern to most interviewees. Audit risk was defined as not identifying the highest risk in an
organization. The concern most mentioned related to audit risk, is audit capacity. The
interviewees explain that auditors are planning to look only at the highest risks, however not at
all risks. This implicates that some risks may not be the biggest risks in an organization, but can
still be a big risk. Another result from limited audit capacity is that the auditors may feel stressed
for time and perform their audits and for this reason not thoroughly review an entity.
Another audit risk is sourced from internal auditors judgment. The risk assessment is
performed by the internal auditor and based on his or her judgment for a big portion. Especially
if the management of entities are not completely open to expose their risks, as they don’t want to
be audited. Experience of the internal auditors and the use of electronic internal audit tools can
41
mitigate these risks. The downside of using such a tool leads to the third mentioned audit risk
that it will increase the dogmatism, which can be present in internal auditing. This dogmatism is
explained as performing the internal audit tasks in an automated way, without a thorough review
of the entity audited.
From COSO (2013) can be interpreted that internal audits should lead to assurance for
an organization to meet its objectives and that there are limitations to this, because the
framework cannot cover bad judgments or bad decisions. The concerns from the interviewees
may result from this statement. Although Bedard et al. (2005) state that there are regulations in
place to ensure that IT controls are covered, which can be interpreted that the concern should
not relate to information systems. Power (2009) agrees with the concern from the interviewees,
as he states that the risk assessment is very economical orientated and other risks might not be in
scope of auditors.
Future of ERP systems

The interviewees note that ERP systems do not always fit the requirements of an organization. A
true ERP environment is also hardly seen, as there are always some parts of an organization
outside of the ERP system. This raises the concern if different solutions should become available
for more complex organizations. One of the interviewees believes in a best of breed from cloud
solutions.
Hsu et al. (2006) and Grabski et al. (2011) acknowledge that ERP systems do not always
match with the requirements of an organization. As a solution some organizations will use
several ERP systems in order to cover all requirements (Kuhn et al., 2010). Then you do loose
the true strength of an ERP system, which is to gain efficiency in information availability. Peng
et al. (2014) conclude that organizations are migrating their applications and databases into the
cloud. Mel et al. (2010) explain hybrid cloud solutions, where several databases in the cloud can
combined into one data warehouse solution.
42
7 Conclusion
In this thesis I researched the impact of ERP systems on internal audit planning. This research
can add value to organizations who consider to purchase or already make use of ERP systems,
because it will give those organizations further insight of the impact of an ERP system on an
organization.
The research starts with the understanding of internal audit planning and ERP systems. I
conclude about the internal audit planning that this is a process closely aligned with COSO and
this process is split up in two levels: the high level annual audit planning and the granular level
engagement planning, where both levels contain a risk assessment as important component. ERP
systems are huge, complex and expensive systems and contain interrelated modules which use
one single database. From an internal audit perspective the main concerns relate to access
management, including the segregation of duties, and process flows, including accuracy of data
and information.
The use of ERP systems do raise concerns about the reliability of the system and data in
the form of system failure and data corruption. The impact of these risks are higher in an ERP
environment, but the likelihood is small and these risks can be mitigated by backup- and disaster-
recovery procedures. Only implementations and changes to an ERP system lead to a higher risk
rating in the risk assessment and therefore impact the internal audit planning, although this is not
limited or different from non-ERP systems. This leads to the conclusion that ERP systems have
no impact on the high level internal audit planning.
On granular level the use of ERP systems does have an impact in various ways. Manual
data entry raises the risk of error and for this reason leads to more and bigger audit samples and
with the use of an ERP system data is entered only once, which leads to one single sample,
where non-ERP environments have the same data entered multiple times and lead to various
samples. The conclusion from this information is that the use of ERP systems lead to less audit
hours required. ERP systems give a good possibility for the use of system controls. Risk will be
reduced if system controls are properly configured and so reduce required audit hours, although
the system controls need to be reviewed and tested with the use of IT auditors. This results that
in the audit planning less audit hours are required from financial auditors, but more are required
from IT auditors, although this will be a lower amount of audit hours.
Internal auditors also benefit from the use of an ERP system, because the data will be
available quicker and in one uniform format, which reduces the preparation time for an audit.
Taking one additional step, this can lead to testing of 100% of the data population or even to
43
continuous auditing. Organizations become more focused on risk management and for this
reason continuous auditing becomes a requirement for internal auditors.
In summary the use of ERP systems have a positive impact on internal audit planning,
because of three reasons: the first reason is that internal auditors will have smaller and fewer
samples to test, because manual data entry is performed less frequently and system controls can
mitigate risks of fraud and errors. The second reason is that the data is quickly available and in
one single format. The third reason is that internal auditing will grow towards continuous
auditing with the use of data analytics. The first two reasons lead to a decrease of required audit
hours to be planned. The third reason results in a different use of audit hours, which will change
audit planning completely. An additional conclusion is that with the use of an ERP system,
internal auditors will require to gain IT audit and data analytics skills.
Two additional subjects are included in this paper, but are outside the scope of the
research: audit risk and the future of ERP systems. Audit risks contain the possibility that
internal auditors do not address risks in an organization. Lack of audit capacity, incorrect
judgments and dogmatism are risks which internal auditors are facing in their own processes.
These can be mitigated by experience or the use of electronic internal audit tools.
ERP systems as they are, don’t seem to have a long future ahead of them, because they
are expensive to implement and to maintain. It also forces most organizations to adjust their
processes to fit in an ERP system, instead of the other way around. The future of ERP systems
is in the cloud. Especially hybrid cloud solutions give the opportunity to use various systems and
make these intercommunicate in the cloud, creating a data warehouse for the entire organization.
A limitation of this research is the fact that only half the interviews were held with people
who actually perform the audit planning currently. The justification for this approach is
mentioned in paragraph 4.3. Another limitation can be found in the fact that not all literature has
been used in the subjects of ERP environments, audit planning and continuous auditing, because
of time constraints. A third limitation is that the research does include information about the
impact of non-ERP environments on internal audit planning, but is only used to support
findings about ERP systems and is not included in the scope of this research.
Suggestions for future research can be identified in the area of high level and granular
level audit planning process. Another possible research could be a survey approach to find proof
if the COSO framework is followed as intended, or can only been seen as a notification.
44
References
Alles, M. & Brennan, G & Kogan, A. & Vasarhelyi, M.A. (2006). Continuous monitoring of
business protocols: A pilot implementation of a continuous auditing system at Siemens.
International Journal of Accounting Information Systems, 7, 137 – 161.
Alsop, S. (1998). Is there life after ERP? For the valley, maybe not. Fortune, 138, (3), 231- 232.
Barret, M. & Cooper, D.J. & Jamal, K. (2005). Globalization and the coordinating of work in
multinational audits. Accounting, Organizations and Society, 30, 1 – 24.
Bedard, J.C. & Graham, L. & Jackson, C. (2005). Information Systems Risk and Audit Planning.
International Journal of Auditing, 9, 147 – 163.
COSO, (2013). Internal Control – Integrated Framework. Executive Summary. Committee of

Sponsoring Organizations of the Treatway Commission.
Davenport, T.H. (1998). Puttingthe enterprise into the enterprise system. Harvard Business Review,
July – August, 121 – 131.
Debreceny, R.S. & Gray, G.L. & Jun-Jin Ng, J. & Siow-Ping Lee, K. & Yau, W. (2005).
Embedded Audit Modules in Enterprise Resource Planning Systems: Implementation
and Functionality. Journal of Information Systems, 19, (2), 7 – 27.
Gaumnitz, B.R. & Nunamaker, T.R. & Surdick, J.J. & Thomas, M.F. (1982). Auditor consensus
in internal control evaluation and audit program planning. Journal of Accounting Research,
1982, 20, (2), 745 – 755.
Grabski, S.V. & Leech, S.A. (2007). Complementary controls and ERP implementation success.
Grabski, S.V. & Leech, S.A. & Schmidt, P.J. (2011). A review of ERP research: a future agenda
for accounting information systems. Journal of Information Systems, 25, (1), 37 – 78.
Haug, A. & Stentoft, J. & Pedersen, A.A. (2009). A classification model of ERP system data
quality. Industrial Management & Data Systems, 109, (8), 1053 – 1068.
Hsu, K. & Sylvestre, J. & Sayed, E.N. (2006). Avoiding ERP pitfalls. Journal of Corporate Accounting
& Finance, 17, (4), 67 – 74.
Hunton, J.E. & Wright, A.M. & Wright, S. (2004). Are financial auditors overconfident in their
ability to assess risk associated with Enterprise Resource Planning systems? Journal of
Information Systems, 18, (2), 7 – 28.
45
Jans, M. & Alles, M. & Vasarhelyi, M. (2013). The case for process mining in auditing: Sources of
value added and areas of application. International Journal of Accounting Information Systems,
14, 1- 20.
Kang, D. & Santhanam, R. (2004). A longitudinal field study of training practices in a

collaborative application environment. Journal of Information Systems, 20, (3), 257 – 281.
Kuhn, J.R. Jr. & Sutton, S.G. (2010). Continuous auditing in ERP system environments: the
current state and future directions. Journal of Information Systems, 24, (1), 91 – 112.
Kanellou, A. & Spathis, C. (2011). Auditing the enterprise system and environment: a synthesis.
Journal of Enterprise Information Management, 24, (6), 494 – 519.
Kanellou, A. & Spathis, C. (2013). Accounting benefits and satisfaction in an ERP environment.
Low, K.Y. (2004). The effect of industry specialization on audit risk assessments and audit
planning decisions. The Accounting Review, 79, (1), 201 – 219.
Mel, P. & Grance, T. (2010). The NIST Definition of Cloud Computing. Communications of the
ACM, 53, (6), 50.
Peng, G.C.A. & Gala, C. (2014). Cloud ERP: A new dilemma to modern organizations? Journal of
Computer Information Systems, 54, (4), 22 – 30.
Power, M.K. (2003). Auditing and the production of legitimacy. Accounting, Organizations and
Society, 28, 379 – 394.
Power, M. K. (2009). The risk management of nothing. Accounting, Organizations and Society, 34,
849 – 855.
Rom, A. & Rhode, C. (2006). Management accounting and integrated information systems: A
literature review. International Journal of Accounting Information Systems, 8, 40 – 68.
Robey, D. & Ross, J.W. & Boudreau, M.C. (2002). Learning to implement enterprise systems: an
exploratory study of the dialects of change. Journal of Management Information Systems, 19,
(1), 17 – 46.
Scapens, R.W. & Jazayeri, M. (2003). ERP systems and management accounting change:
opportunities or impacts? A research note. European Accounting Review, 12, (1), 201 – 233.
TeamMate. 2015. General Information. Available on February 15, 2015, at:

http://teammatesolutions.com
46
Wolters Kluwer Financial Services. 2015. General Information. Available on February 15, 2015, at:
http://www.wolterskluwerfs.com
47
8 Appendices
8.1 Appendix I: Mind map to specify research topic

8.2 Appendix II: Thesis structure
49
8.3 Appendix III: Interview #1
We will start up with a couple of questions and the first ones will be of a more introducing kind
of nature. I’m doing my research about Audit Planning and this research should help me with
getting my university degree. So let’s get started.
What can you tell me about your role in the TeamMate organization?
My official title is manager of FS professional services and actually what I do. I was recently
promoted into this position. It’s only been in this position from the beginning of this year. What
my team especially does is what sales for self services. Either to new a user of TeamMate or to
existing users of TeamMate who to help with making better use of the product or bullet training
to staff because of high turnover of staff, whatever it might be. So a service award will come in
to our sales team and then we have a team of will be 13 consultants from the US that are
assigned to work with a client, specifically on sight. Most of our services at this point at least are
delivered on sight. And we basically learn the clients audit methodology and then help them
implement that within TeamMate and then generally wait about two weeks to a month and than
come back and do training. I basically manage that team that does that work as well as I do some
consulting myself.
You already informed that this role is relatively new to you, but have you been part of the
TeamMate team for a longer period?
It’s been over 6,5 years now. I was a consultant on the professional services team for that time
and kind of lived up the ranks I guess to manager.
So you’re an experienced person towards the TeamMate program?
I’ll say so. Our team is fairly new. Wolters Kluwer purchased TeamMate, which was about 8
years ago. And I joined the team about 6,5 years ago. So at that time there were only about 4
members and I was the 5th to be hired. So we were a small training team as there didn’t exist a
PWC on the product. When Wolters Kluwer bought the product, they built the training team.
Which is our team, consulting / training, whatever you want to call it.
So you have a lot of communication with the customers / with actually users?
Yes, that’s the biggest part of my job.
Have you had worked with other programs like TeamMate?
I have not. I started in audit 2002 as an auditor and they had TeamMate in place already, so I
only used as electronic product was TeamMate. I worked with some clients that have used
AutoAudit, but I personally have not used any other product.
At least you have experience both from a user side as well as the consulting side towards
TeamMate.
50
Right.
That sounds like a broad experience.
It is.
TeamMate is quite a broad package as far as I can see it. With which parts of TeamMate have
you had experience?
Well, there’s three products now that TeamMate offers. My experience has been in TeamMate
Audit Management, AM. There’s TeamMate CM, Controls Management and then there’s TMA,
TeamMate Analytics. I have only worked with teammate AM. TeamMate CM has only been in
place for the last few years and I personally have not had any exposure to it. It’s been kind of a
slow rollout to our clients so only a select few of our consultants have had exposure to CM.
TeamMate Analytics was just put into place last fall. So that’s all very new to all of us.
But the AM part is already containing a lot of area’s of expertice. What I find particularly
interesting is the Risk Assessment part. Can you tell me a bit about the Risk Assessment?
Sure. I can go all different directions with that question, I guess. I’ll just take the approach of
what I generally see working with different clients. TeamRisk is designed to be the annual risk
assessment tool to help departments assess the high level risk of their organization and to
determine where the high and moderate risks lies to justify their audit plans, to justify where they
are planning to spend their time. So it helps them to justify the audit that they’re doing during a
year. TeamRisk is designed so that it can. It’s a 360 product and so basically what I mean by that,
that if auditors find detailed risks associated with a process, so it’s possible for those detailed
risks to then be carried into their audit and than that risk assessment be tied to the work
programs that they conduct to the part of the audit. That’s possible. I will say that’s not what I
see most organizations do, because the annual assessment process is already quite attached
anyway. And if you think about the detailed level of risk that each audit could get into. I mean,
there might be 50 different inherent risks that are identified with a single unit or a single process
or single auditable entity. And if that’s carried into the annual risk assessment than you can
imagine trying to score 50 risks times each auditable area will take quite a while. TeamRisk and
EWP, the formation between those two models is designed to do that, but what most
organizations will do that they will have 5 to 15 core high level risk categories, such as financial
reporting, people risk, HR risk, operational risk, so different high level types of risk. And then
score at that level.
Are those risks comparable to the COSO model?
They are. And that’s a huge important key phase and a big push to last year has been for
organizations to tie their risks to the COSO principles. I’m not sure if that’s true globally, but it’s
51
been in the US essentially. At some high level there’s always ensured that the COSO risk
categories are included in the risk assessment. But what I was more referring to is more related
to risk that are identified as part of the audit, so at a more detailed risk level. What most
organizations will do is put those detailed risks into one of their categories that feed into one of
the high level risks. So they can show that what they’ve identified in their annual risk assessment
is what they’re testing in their audit. If that makes any sense.
For sure. Do you think that the TeamRisk part of TeamMate that is the part where auditors are
looking at most of their time to look at their what type of assessments need to be done, what
type of tests need to be done at the different departments?
I see it used that way. I think I understand what you’re saying. To help to determine the scope of
the audit. When they do the audit. It can do that. If an audit department has broken it down that
way, so if their risks are broken down to sub process and they have their detailed risks there, they
can use TeamRisk for that purpose. I will say that in most cases that’s not the case. Usually their
just from their risk assessment what area they are going to audit. So what department, what
process of what area or to how will they define their audit universe. And then when they actually
start to conduct the audit, they will go to a planning phase. That’s where they will start
identifying or at least looking at, maybe historical identified inherent risks and doing some sort of
risk assessment as part of the audit. TeamRisk is generally more high level. The risks assessment
to determine the scope of the audit is typically done with EWP (Electronic Work Papers).
OK. So how would an auditor start then? Would they first look at TeamRisk and then continue
with TeamSchedule?
Generally speaking the management team, the audit management, are the only ones that use
TeamRisk. And they would determine their audits that they will do for the year and based on
their resources, or resource hours, they will determine how many audits they will be able to do
that year. That will generally go to an audit committee or governing body to get approval. And
then once that happened they release the project to the schedule in TeamSchedule. In
TeamSchedule is basically where they would take all of those audits and put them on a calendar.
So they start planning dates when these audits are going to be performed. And then they start to
assign staff to the audits. So this is sort of a preliminary schedule. A lot of organizations will do
that on a quarterly basis as opposed to the full annual plan, because they may have an annual
plan setup, but they probably only schedule their quarters worth of audit set of time. Because we
don’t know what things happen and they don’t go as planned and then they get pushed out or
moved up and they will have to change the schedule anyway. So I typically see people work
schedules say they have ten projects for the year, they might only schedule two or three they
52
know they’re going to start in the first quarter. So TeamSchedule I would summize my clients to
set truly as a planning tool, a budgeting tool. It’s there to determine when you’re going to do the
audit and then how long it’s going to take, estimate how long it’s going to take. The next module,
Tech, is time an expense capture. It’s the most simple of all the modules and the purpose is to
capture actuals. So it’s essential a glorified timesheet and the auditors are plugging in the hours
that they spend on the audit. But the beauty of it is that you now got TeamSchedule where your
scheduling the audit and budgeting the number of hours and then now Tech, for the auditors to
record hours which they are actually spending, you have a comparison. So it truly gives you that
information that you’re looking for monitoring of how well we’re doing to meet our budget. And
then planning for next year, I guess with that information they can see how much time we’ve
spend in the audit last year. Those kind of things. As we talk about the different components or
modules, they kind of feed that way. So it feeds from TeamRisk to TeamSchedule to TeamTech.
If that makes any sense. So in TeamTech I can see what we planned to do in TeamSchedule. The
start and end date and the budgeted hours. And then I can compare that to my actual hours.
How do you think that an audit manager is actually… What are his main concerns in an audit?
You’re asking me what are those items what they typically track from an audit management
perspective? The two big things that people are looking at is: Are we meeting Schedule? Are we
meeting the planned budget? And then what issues result from the audit. Those are two big
pieces what they’re looking at. Really all that information, that I mentioned so far, feeds into…
Of course we have EWP, Electronic Work Papers, that’s the component that all the auditors are
going to use. That’s where they’re going to put in all their documentation. All they’re work
papers. Write their findings. Write reports out of that component. But all this different
information from TeamRisk, TeamSchedule, TeamTeach and now EWP feeded into a core
reporting tool called TeamCentral. And it’s that tool that audit management generally uses to run
that kind of data out of the system. To see what are the findings that result from the issue, from
the audit. And then there’s also a component within TeamCentral that allows the auditors to do
follow up work. Or allow the contact to be, auditees I’ll say, access to TeamCentral can provide
updates to the status of the issues. So it gives management not only what issues were reported
out of the audit, but also what’s the status of the implementation that correct the issues that were
noted out of the audit. So it’s giving them that full circle that gives the assurance that not only
we’re reporting findings as auditors, but also that audit plans are taking a serious plan to
implement corrective actions that address the issues. And that’s what typically rolls back to the
audit committee or governing board or governing body to say this is our status. This is the issues
we’ve reported and were the auditees have addressed the issues as well as seriously that audit,
53
that other component I talked about, where this is what we originally proposed to you to do this
year, here’s our plan audit, here’s what we have planned to do, here’s what we have budgeted and
this is how we’re comparing to actual time. So you’re getting to real measures to audit
management feeding up to audit committee or board.
Do you think that there are some concerns about what type of system is being used which is
being audited? Like it’s a highly integrated system like an ERP or if it’s a system with multiple
data entry. Do you think there’s an impact to the concerns of an audit?
I’m trying to understand what you’re asking. Are you saying that from an audit management
perspective are they more concerned if data if they’re not using an integrated system?
Exactly. Does that have an impact on the concerns of an audit? Does it change the audit
planning?
I think I follow you. So if I was doing an audit and I was looking at whatever system they are
using that is not highly integrated would that impact my risk concern?
Yes, if there is an highly integrated system, does that change the way you look at your audit? Or
the concerns of an audit?
Sure. As an auditor if I was looking at an area that did not have an integrated system there’s
obviously more room for user error, because there’s not that automated feed from… To use
TeamMate as an example, there’s not that automated feed from EWP to TeamCentral. We do
have some organizations that just come to us and say we only want to use EWP. And that’s
perfectly fine, because they may have a spreadsheet that once they’re done with the audit they
copy paste or they somehow feed those issues that come out of the audit into some other form
that they’re tracking. That would be the same as to auditing any area. I would be concerned with
that. Than you’re concern with all the data what was captured into EWP was transferred to that
secondary tool, whatever is used, say it’s just that spreadsheet. Was all that transferred?
Complete, accurately. You know, it would be that kind of things we would look at as an auditor
and that’s the same thing we think about when TeamMate was developed. TeamMate was
designed by auditors. So we’re always thinking of those things. And in our testing we do testing
of completeness and accuracy in the transferred data. The same one would do as an auditor. So
integrated systems if they have a proven integration, that’s always better than anything that’s say
manual or that requires some user interface.
It’s more or less what I would expect. That’s basically what I’m trying to gather the information
what you guys know and what you guys see. So can you tell me anything about how customer
setups are impacted by this type of variable?
54
I think the reason TeamMate… It’s no secret that TeamMate is the most expensive audit
management tool out there. I think once clients see how integrated the product is, that’s what
sells them. I see that’s the reason why our clients decide for TeamMate, because they’re so much
of… Really the complete audit process is integrated within our product. And they have that
reliability that the data that they put in, is then reportable and transferable amongst the different
tools. And we build to the stadium that clients that buy the entire product and they say “Well, I
only want to implement EWP”. And that’s fine. We will work with them to implement that
piece, but it’s our job also to tell them that there’s so many more benefits and you can integrate
your existing process you do outside of TeamMate and to TeamMate. I think once they realize all
the functions that TeamMate can perform, they integrate more. Meaning that, over my time at
TeamMate in the last 6,5 years. In the beginning I did a lot of EWP implementation. Now it’s
incredible rare that we implement just EWP. Most clients are implementing all of the 5 modules.
Even the small departments they will implement EWP, TeamCentral and a lot of times
TeamRisk. Because we all have to, are required to do a manual assessment and you might as well
do that in a product that feeds back data into your Electronic Work Papers. So then we’ve got
TeamRisk and our Electronic Work Papers, EWP, but then we’ll also need to report on that and
track the findings that we’ve reported in our audit and that’s TeamCentral. The modules that are
least used… Please let me know if I’m not answering what you are looking for.
Well, it’s not completely what I was asking, but it’s giving a better understanding of how the
integration within TeamMate is working. Because that also explains a little bit more to me or
actually it’s a step up of how the main setups are performed at customer sites. So please proceed.
I’ll continue with that and I’ll move onto how we conduct implementation. I would say that the
two modules that are least used, one particularly is TeamSchedule. It’s really TeamSchedule that
gives you a visual of the calendar that just say were are my people and are they well utilized or are
I planning to for them to well utilized. And it gives you that nice picture, that visual on your
calendar of were all the projects are and where are the people assigned to the projects. Most
people just want to simplify “Can I plug in the start mandate and then the number of hours
utilized?”. Sure, you can do that without TeamSchedule. So most, especially smaller departments,
do not use the TeamSchedule tool. That’s the least used component. TeamTech, I personally
don’t quite understand it, because if I was a manager of an audit department, I would want to
know actual time auditors are spending on audits, but some organizations aren’t so driven by
hours. They’re more driven by start mandate of an audit. So if that’s the case they would not use
Tech. EWP, TeamRisk and TeamCentral are the most used components. From an
implementation standpoint, generally speaking our sales team will discuss all that TeamMate can
55
do. Discuss each module and explain to the client what each module can do. So we, as an
consulting team, we already know ahead of time, what the expectation is. So we know what we’re
going to be implementing. However we, no matter the approach, every client implements EWP.
Because the purpose of buying TeamMate is to use Electronic Work Papers. So when we
implement we as a lot of questions. We do a lot of whiteboarding. What I mean by that is that
we get documentation of their audit report, we get their auditing manual, we get audit committee
type reports. Because our goal is, that if we’re setting up TeamMate, or configuring it in such a
way that you can run those type of reports automated out of the system. Before we can ensure
that’s going to be a possibility, obviously they have to have a place to capture that type of data
that’s captured in your reports. Like for example, most audit reports will have brief introductions
that include a background of the area that you’re auditing. If you want that to be pulled out in
your auditing report, we have to have a field in TeamMate that captures the background. That
the auditors would populate when they’re gathering that information from the audits. So we’re
learning their process. We’re learning their requirements, reporting requirements. We’re learning
their methodology. And as we’re doing that, we’re giving them advise on how to setup
TeamMate. Not only advise on what their audit universe looks like, you got to have an entire…
TeamMate calls it a global organization hierarchy. It’s essentially the audit universe; here’s all of
the auditable areas that you can possibly audit. And we start with that and we say here’s what we
would audit. And then what are all the fields you need to capture in all of those areas. So these
are the fields that we would need to attach to what TeamMate calls your terminology. And then
if there are dropping options, that if there are different categories as like audit types. You know a
lot of auditors will do a traditional financial audit or an operational audit or a compliance audit.
Even though CM is designed for SOX audits, we do have some audit shops that use TeamRisk
and EWP to do their SOX testing within TeamMate. So that might be another type. So if we’re
to categories those audits for the one on running reporting later, I could say, show me the list of
all the SOX audits that we’ve done this year. Or show me a list of all the financial audits that we
did this year. Does that make sense? We have to do that no matter what modules they’re using.
Because we have to have that core foundation of our audit universe, our fields and the setup of
all of our fields. Then as we’re learning their methodology, they’re telling us “well, homing
management can do certain tasks.” And then we help them to setup their policies, their user
submissions to ensure that that’s the case. Because we build controls within the system that will
prevent certain users for getting certain functions based on their methodology. So we take that
approach and we’re getting the implementation and guiding them what policies they need to set
and what user access they need to be giving.
56
To each individual user, I suppose?
Right. Another thing that, you know TeamMate involved a lot of… When I first started there
wasn’t an option to import historical data. And not necessarily historical data, but say existing
audit programs that they have. Now TeamMate allows for that. So we can take if it’s an existing
audit department that has predefined audit programs that at least they can leverage, it probably
would tweak exchange based on risk assessment. That at least they have a starting point, we can
import that kind of data into a tool called TeamStore. That once they start an audit they can go
look in the store and look if we have programs for accounts payable, if I’m auditing accounts
payable. And then pull that into their audit and then change it, tweak it based on their risk
assessment. It’s not new, it’s new for the last four years, but not something we always offered,
but clients love that. We have a standard set of programs that we can give them. We had a
gentleman on our team that stratege effort to work with different organizations that specialized
in compliance driven organizations, like zipup or health care. He basically setup a standard set of
programs for different requirements. So that is something which we published to our users called
the content theme store and we can show how to download that database. And then connect to
it and get the information from that database to give them a good starting point if they don’t
have a standard program for typically defined straightened areas, which they are required to
audit.
So it’s more then just the TeamMate package.
Right, you’re getting the content with it. It was a huge effort to get that up and running. Clients
don’t want to start from a blank state. They want something to start with and we can help them
with that.
To continue in a slightly different direction. If a customer has an ERP system installed or in
place, we already briefly discussed about that it will impact some TeamMate settings. Can you
explain a little bit more about that?
Like they have a previous installed audit system in place?
No, not the audit system, but the accounting system, that it’s fully integrated systems with…
Like the entire purchase to pay is fully integrated. Opposed to that each department has it’s own
database and has it’s own point of entry for data. How does that impact the settings for
TeamMate?
It really does not have a whole lot of impact, because TeamMate is designed to handle work
papers. It doesn’t allow for direct feed of financial data for reporting purposes. TeamMate
Analytics now would impact that. TeamMate Analytics is a way to analyze generally. I didn’t have
a lot of exposure to it to TeamMate Analytics, but interviewee #5 is kind of our expert in the
57
team and can give you more information. I did see that recently in the audit industry that more
and more auditors are conducting data analytics from an accounting system to get a dot of
accounts payable transactions or expense trade actions, whatever it is, to be able to run analytics
against that data and look for certain criteria. We used to have a company requirement at Wolters
Kluwer that if you had meals less than 25 dollars, then you didn’t have to present a receipt. So
we might look for a lot of meals that fall just in that 24 dollar range and run data analytics to see
if that’s something which we’re abusing. That’s one kind of example we might use that tool.
There are all different types of tests that you might run against that data. But as far as the
TeamMate AM products, really the only thing that it would impact is the fact that you could
export that data from that accounting tool and import it in as a supportive work paper as part of
the audit record. We can import it in any data format and directly integrates in Excel. But as far
as the actual planning tools, it would impact more the TeamMate Analytics as opposed to AM.
I recognize that from your explanation. Let’s say that if you’re an auditor, put TeamMate
completely aside for a minute, and you’re auditing an ERP system or a non-ERP system, does
that raise concerns for you? Or which concerns?
As an auditor what I would typically do is to get a datadump of transactions for a period of time
and test it. It does raise concerns for me, because we will place a lot of reliance on that system.
We can’t just say it’s an automated system that has been tested and we’re happy with it. We’re
going to test it. To make sure it’s transferring data completely. I was not an IT auditor, but a lot
of those standardized programs do come into play then. I’m going to be testing for transfer data
completeness and accuracy. It raises concern and we test it, but generally speaking if it’s n
automated system is more reliable, because all of those things have been tested in the past and
we can place reliance on it. But that’s where data analytics will come into place. We would do
those types of tests. So more broad testing instead of detailed testing.
With an ERP system it’s usually data gets entered only once, but usually not by accountants or
bookkeeping type of people. Do think that raises concerns about the quality of the data?
It does. Again where anything manual is more risky than anything automated, so if they had
some sort of scanning tool to scan barcodes or something like that, than there would be less risk
than manual user entry. And from an auditor perspective, we would only test manual type entry
much more detailed than an automated process. But anything manual is more risky. That’s why
we had that segregation of duties to whoever is entering data for an account is not also balance
or reconcile where the accounting department would come in. I’m not sure if I’m answering your
question.
58
Yes, you are. Basically you’re saying there’s a shift from responsibilities. Especially the
accounting department is more reconciling instead of data entry type of work.
Right. You generally have different people responsible with those type of transaction with
different departments. And you would looking at that as auditors as a control to see that
segregation of duties. So that if you have a person to do the entry, you have a second person to
review that work.
That’s completely in line with all the literature which I’ve been reading so far. I’m thinking a little
bit how to get more information from your side. Because you already explained me a lot about
TeamMate and especially the AM part. As you no doubt have guessed my research is more or
less about the impact that the use of an ERP system has on audit planning. And I’m trying to
make my research a bit more specific. Because now I’ve put both area’s as a very wide area. I
mean, audit planning is quite big and ERP systems is quite big as well. In order to make it more
specific as far as the ERP side is concerned I’ve got quite an idea about that I would like to
research how much the decentralization of data entry has an impact on audit planning. But I’m
not sure yet, which part of audit planning is mostly impacted. Do you understand what I’m
trying to say?
I think, as an auditor, this is all going to be part of your audit risk assessment. The first thing I’m
going to do as an auditor is having one meeting after another for learning the process I’m
auditing. If there’s a lot of manual entry, into any software. Ideally it’s centralized. If it’s
decentralized it’s even that much more risky. So these are things I’m asking and I’m kind of
doing a risk assessment. Well frankly, how it’s going to effect my audit planning is. I’m now have
got to do testing. Identify whether it is decentralized, it’s manual entry, I’m going to have to do
more and more testing. And they have to do on-sight locations and wherever it is that the
manual entry. Let’s say it’s at a bank and it’s done by the teller at each branch or location, I’m
probably going to get a sample of transaction from each location. And or do data analytics to test
a broad sample from multiple locations. Because you’re looking at greater risk because of the fact
that the process is not centralized, you don’t have control over… Obviously the fewer people
who control a process the less risk there is and when you got more people involved the more
risky it gets. You’re looking for consistency of policies among the different locations of how the
entry is done. And testing and monitoring for completeness and accuracy of the entries. From
the TeamMate perspective it doesn’t have a whole lot of impact. Those are those risks that need
define all or planning our audits. And then that’s how we determine what tests we’re performing.
But again if I identified the process where that’s going to be centralized or decentralized, I’m
going to do a lot of testing around it.
59
So in a way that does have impact on TeamMate as well. Not the system itself, but for the
settings. If I understand the story correctly.
It would just be a matter of those are those risks that form a content and that’s where the
content store, as I mentioned earlier, is so helpful because, especially to a new auditor you don’t
always think about all these little new answers that might occur. So think about all those inherent
risks associated with that manual entry process is extremely difficult to have an open mind.
That’s where like process mapping helps. Along with the narrative that move into different
people that understand the process and I guess are part of the process. Current at TeamStore
helps because it brings light to some of those risks that you may not think about. So you may say
I’m looking at the process that manual entry tool an accounting system. If I can look at content
at TeamStore that might give me some ideas of risks that I may not have thought of. And I can
ask some questions to determine if there is a control in place to ensure that the entries are done
completely and accurately.
But how does that work then? Is that result coming out of the TeamRisk module?
It’s actually part of EWP. There’s a planning tool with the electronic work papers that allows the
auditor when they’re planning their audit to go look in the store of content. To see what risks are
associated with this process. And they can pull that into their Electronic Work Papers. And to
determine controls and test that to procedures that’s going to be performed.
So you expect that it have a big impact on the working papers?
It’s all going to have to be captured in their planning. And within EWP it’s generally designed in
the recommended console model. You do planning, field work and wrap up. So within the
planning you do a risk assessment. So there’s a component to capture all your risks and your
control that you have identified. And then determine the weight of that control to test, that
there’s a climate of control that is going to test. All that is part of the planning, before the field
work commences.
That’s giving me some more thoughts about which part of the audit planning I would like to
focus my research on. What would you think would be the area to do my research on, as far as
the audit planning part is concerned?
One thing I noticed has changed in the last, I don’t know how long I’ve been. It’s been almost
14 years. In my time of auditing or when I first started, the whole concept of risk assessment was
very new. It was very cutting edge 14 years ago. Auditors were more checklist driven. What
works out right now is being truly risk focused. So understanding the process, identifying the
risk. What I’m seeing now, how internal audit has evolved, is that true focus on risk. I would say
that probably 90% of the clients I work with do a true risk assessment as part of their audit
60
planning. Not necessarily the annual planning, that we’re doing with TeamRisk, but within EWP.
Where there only to each set process of the area that they’re auditing can conducting identifying
the inherent risk. Identifying the medicating control. And then determining where they are going
to test. I think as far as research goes, I think that trend is very interesting. Because, I’m not sure
whether this is coming from if this is an overall push from our external examiners. That they say
this is what we need to move and that’s what I’m hearing when I’m out on the field. But I find it
interesting that more and more and more audit departments are more truly risk based as opposed
to compliance or checklist driven. And that’s where we need to be. I think we’re going to miss
those risks and that’s where fraud comes into play and different issues that we see on the news.
That we are not looking at the profit and personally identifying what could go wrong. And
thinking out of the box and thinking about you’ve mentioned that a lot of the risk comes from
the fact that there’s user error as part of manual entry or you got decentralized process and
possibly no consistency in policies. Maybe not monitoring. All of these things are something that
I would need to better understand that process to ensure that I’m capturing the risk, controls
and then testing around that thoroughly. I would probably focus on the risk assessment aspect of
it. I think that was your intention.
For sure it is. The first time I saw something of TeamMate the risk assessment part took my
attention and I never really got away from it anymore so. I saw quite some parts of TeamMate,
but the risk assessment part that is really drawing a lot of my attention. My intention is to see if
processes are decentralized or if the data entries are done by non-accounting personnel how
much impact does that have on the risk assessment. As I understood from you, you more or less
already answered that, that your concerns would be mainly ensuring that the policies are basically
dealt with at the various sites or branches. And that the volume of manual data entry, that that
triggers how much detailed checks need to be done.
Right.
So I’m happy you’re more or less in the same direction as I want to research. That basically
means I’m on the right track. I think that’s more or less it what I wanted to ask you for now. As
no doubt you might have understood you’re my first victim of interview.
My research is starting up a bit broad. So it might be that I’m going to ask you for a second
interview. This will then be a lot shorter and then I will have more specific questions. Directly
towards my research. Then the questions won’t be as broad as now. Are there any other
comments which you would like to state?
I can’t think of anything at this time.
You can always drop me a mail for additional information.
61
8.4 Appendix IV: Interview #2
I’ve got a list of questions. Which is not a tick box but more a guidance for me to get a better
understanding of audit planning. And in the end of our conversation it will go a bit more into the
direction of the research, which I’m doing.
But what are you looking for? Do you look for annual planning, how we make the audit plan? Or
more an individual audit?
Anything. It’s very broad, but if we start out very broad area and then bit by bit we will get more
in depth into the direction of my research. It might be that in the end my research will go into a
slightly different direction, because of the knowledge which I’m gaining through these
interviews. But in the end you will know what my research is about. I don’t want to start up with
it, because you will already develop a position. I’d rather give you room to share your thoughts
and opinions. Let’s get started. My first goal is to get a better understanding of audit planning.
Let’s start with you in the organization. What can you tell me about your role in the
organization?
I’m an internal auditor. So as part of the team I do audits of different business units and
processes or operating companies within Wolters Kluwer. My view on internal audit is that it’s
really looking at how risks are managed. I look at it less from a compliance point of view, but
more from a risk management point of view. There are risks within Wolters Kluwer. There are
risks in how we operate. And then we need to control them and then you need to make sure that
the controls which are in place are adequate and working correctly. So my role in the
organization; I’m part of a team that brings the EB (Executive Board) and the advisory board
comfort and information on the way risks are managed within the company. At a high level and
at a granular level. That’s the way I see my job.
And within the team, what’s your role?
Within the team, we’re not that specialized. We are a team of eight, four in the Netherlands and
four in the US, in Chicago, and those eight two are really IT specialized. I’m more into
operational processes and more into financial processes. My background is that I didn’t study at
all to become an auditor. I did my MBA and started auditing in a bank in Canada. I worked eight
years there. You get that acute awareness of risks, because the raw material in banks is money. So
everybody can touch this raw material and you need to make sure that you have proper control
around that. And then I worked a few months for ABN-Amro and there they had a more
compliance mind set, which I didn’t like. I’m at Wolters Kluwer now.
How long have you been in this team?
62
It’s seven years now.
So you have a good understanding of how internal audit works here at Wolters Kluwer.
Yes. So if we talk about audit planning. In my opinion there are two levels of audit planning.
How you plan the audits you’re going to do and how to plan an audit. Because, if you look at it,
you’re in a company, 3,5 billion euros, several entities. Where do I go? Where do I need to go?
Where do I need to focus to assess the risks? So you have different ways to do it. You can say:
“I’m going here, here and there”. That was a method that was used before. You can say: “Listing
my entities and I’m going there based on rotation”. Or you can say: “I’ll try to focus on the
riskier area”.
Do you mean rotation of the business unit or of the people?
Of the business unit. You can say I will go in each business unit in five years. And for each
business unit you can see who is due now and you go there.
Based on what do you say, this business unit is due?
Based on the last time you’ve been there. You can say: “it’s been five years”. Then the following
year you can look who is due, because it’s not been audited for five years. The approach which
we have here is really different. It is risk based. It’s something which we developed the last years.
What we do is, we try to list all of the entities. And by entities we mean legal entities, projects,
because a project can be an entity which has risks, processes, we do processes when they are
relevant, like accounts payables is a relevant process at corporate, because there is a lot of
purchases going through that. So what we do is we try to list all those entities and processes,
create an audit universe and then we have defined risk criteria and we rate all the entities based
on those criteria. So we have defined those risks criteria and we have defined how we rate those
criteria. And then based on the outcome of that we have the riskier entities and those are the
ones we should be focusing on. And putting on the plan for the following year. So that’s the
approach that we have for the audit planning for what you will be doing the next twelve months.
That means that some of the business units have basically a higher risk level.
Yes, and then you go more often. Some of the business units have very low risk profile and we
go less.
And what are some of the determinants to say that this entity or this process has a higher risk?
We have a presentation about that. I can send it to do. And there you have all the criteria’s,
definitions and how we rate them. But there are several factors, like are we talking about a full
entity that has full processes, or are we talking about the sales center? Like many businesses in FS
for example, they have many offices in Asia. But those are sales centers. So what they do they
have a few people who visit customers and all the back office is processed elsewhere. So those
63
are very low risk. We don’t need to go there. It depends on how many electronic revenue you
have. And one of the criteria is how is the progress in electronic revenue? Because when you
have a shift of ten percent of the revenue from print to electronic, then that means that probably
you’re changing your processes also. Because you don’t sell or process electronic and paper the
same way. So if you are changing processes, the risk that something is not completely adjusted is
higher.
Is that then more towards the Wolters Kluwer policies?
No, it’s our own definition, based on Wolters Kluwer business and based on what we saw in the
past. I would say there’s no absolute guideline on that. You can take a theoretical model, but
then that might not be adapted for Wolters Kluwer. What we try to do is to start with something
and refine it all the time. There are some criteria that we changed or that we added, because we
felt that they were more adapted to Wolters Kluwer. From that we get the raw planning. We say,
based on this analysis, those are the entities we should go to in the next twelve months. And
then based on that we have meetings with the CEO’s, CFO’s and internal control officers of the
division. Asking: “Do you think we are right in the things we are picking up or not?”, “Are there
other area’s that you see that there are higher risks that we should include?” We have this
discussion and then first we make adjustments to that plan and then we discuss it with the board,
with the CEO and the CFO. We ask them “What do you think?”, “Are there area’s where you
have more concerns?”. Then there’s a second run of modifications and after that it is presented
to the audit committee. To say here is our plan for the next twelve months and here’s what we
think for the next three years ahead.
So even a three-year plan?
It’s more flexible. At least it gives us an idea of where we should put our effort.
And do you find some specific area’s where you are checking the numbers or checking the
processes more often? Like the purchase to pay area or what I know from Tech BV that there
are a lot of assets created.
Yes, there are. As a standard we go to every new acquisition. There are some kind of patterns we
see in new acquisitions and the way they are handled and the way that we do what we are
supposed to do and things like that. The other area of focus is the shift in the company. Because
this company is shifting in a sense that we move from paper to electronic, we move from paper
to software, online and we move from local to central. With Tech BV we centralize a little bit.
With GPO we have some centralization. With GSS we have some centralization. So we tend and
we want to focus more on those processes and how is the governance and how are things
managed. How are we managing these changes in fact?
64
The transitions basically?
Yes. This and also what is going to be more and more a trend in the future is then software
controls or software development. You don’t sell software the same way you sold a book. Or
the same way you access to an online plateform. So the way you sell it, the way you organize to
support, the way you organize the licenses, is the way you organize invoicing, because you have
different ways to invoice software. So there are also different ways for us to audit. So there’s this
and then comes the move to the cloud and the fact that then you handle customer data on
servers and you have privacy risk. And all those applications that are access based. What do we
run in terms of privacy risks or hacking and things like that. And that’s also an area which I think
is going to be more and more prevalent in our audit. We did a risk assessment in T&A last year.
It was really penetration testing and things like that. It was done in corporation with KPMG. Just
to see how are we organized around that and do we have the right controls around that. Imagine
somebody is hacking into the T&A application on tax season. Tax is confidential information
and then the reputation of Wolters Kluwer is at stake.
You just mentioned centralization and decentralization. But that’s more of the backoffice
processes. Or do you recognize a broader centralization?
For GSS and GPO yes. But backoffices stay local or with different governance. But if you look
at products like Kleos, then you have centralization of development of a product for different
countries and then you need to look how this is governed also. We address the needs of all those
countries. How do we handle support? How do we handle product road map? It’s not the same
when it’s centralized and when it’s done locally, because it’s really local market and everything is
handled locally. Then you need different mechanisms when it’s centralized or when it’s done for
several countries.
What would you say is more complex for internal audit? To have it centralized or decentralized.
Or do you think it doesn’t impact that much on audit planning?
You have more persons involved when it’s centralized and that makes it more complex, because
you have more stakeholders.
And from an accounting point of view, because you have experience in financial audits… Some
organizations have ERP systems in place, with highly integrated software, so data entry is done
only once. And other organizations with the complete opposite, that the same information is
entered in several databases, several times by different people. From the financial reporting point
of view the data is either entered somewhere in the system by a non-finance person and on the
other hand it’s always done by a finance person. Do you think there is a concern there?
65
If you look at integrated systems, to companies who have a full ERP. It’s true that if they don’t
have it, the risk of error is higher. Then you need to make sure they have good reconciliations
processes. And that they have good completeness controls also. To make sure that everything
balances. But it’s not because you have an ERP or different systems that need to be integrated
that you have less or more risk in one area or the other. Yes, when it’s scattered with more
systems, you will have more risk of errors. But you still have risks in ERP also. Because risks are
some times outside the system. In a sense that you still have to make provisions, manual entries
and those also create risks. You can still reverse transactions in an ERP and you can still modify
some of the entries. So yes, there’s one part that you don’t have interfaces and reconciliations,
but there are still risks to look around there.
If you’re auditing an ERP system. What are your first main concerns?
Access management. Because if access is not managed correctly you have segregation of duties
issues. Second thing is how the process flows in the system. If you look at order to cash for
instance, how all is entered, where are new orders processed, based on what type of customer
confirmation, and how rigid is the master data. Can you play with the base price? Can you play
with discounts? How is it going to make an output after that in terms of revenue recognition and
things like that? Even then how can it be modified and what adjustments can happen after that?
But I would say the first thing is to understand in an ERP when is the ERP used. Is it covering
the whole process or is it only part of it?
I think the ERP should cover the whole process, otherwise you don’t make use of the ERP
properly.
It depends, because most of the time you have an ERP and you have a CRM. So your customer
interaction and your order entries have been in the CRM and from the CRM it will go into the
ERP.
So there’s an interface anyway?
There’s an interface or there’s re-entry. And even if there is an interface; who has access here?
And who has access there? Is it compatible? Sometimes maybe it’s locked here, but if it’s fully
open here, you’re not better. You want to make sure that the order is entered by somebody
independent from sales. Especially if they have sales commissions. You want to make sure that
somebody is controlling discounting and this is somebody different from sales. You want to
make sure that somebody is monitoring credit notes. And that access is also managed in a
different way. So when is credit notes happening? Is it here or is it there? That’s also the type of
questions you need to ask, to see how well is that whole flow.
66
How do you see that those roles, which team or person, is the ones more controlling and who is
more responsible for the data entry?
Data entry is happening at several places. It depends what type of data. If you’re talking about
master data. Before you create a purchase order, you need to have master data of suppliers and
the type of orders you create and products that you can create. Then typically it would be the
purchasing department or it would be finance that would control the master data. Those should
be independent from the ones that are ordering. There should be a segregation of duties between
adjusting the master data and initiating payments. But that is between finance and purchasing.
And do you think that’s different outside an ERP system?
Outside an ERP system what you will see is that there is a manual circulation of documents. So
instead of having a workflow that is managed by the ERP, you will have a workflow that is paper
based. It’s an order that’s going to be signed. It’s send to the dedicated invoice. The invoice is
signed and is getting to accounts payables. Payables look at it and book it. There’s another
person who makes the payment.
But then you have that finance, outside and ERP system, all the data is entered and as well the
payment is processed.
Yes, but it can be scattered, because you can have one part of the data that entered into an
accounting system. One part of the data is entered into your banking system. And one part of
the data is still based on paper, because you don’t get detail on the articles that are purchased or
when was the purchase date. It’s not depending on the systems you’re using.
Coming back a little bit on the question which I raised before, if the data quality within or
outside an ERP system. Do you see different types of concerns for the data quality?
Yes, if you open the possibility to modify the data to everybody, then you create problems. Then
you have duplicate customers. Then you have duplicate vendors. Then you don’t have consistent
products. Then you don’t have consistent base prices. And then you have difficulty to integrate
it.
And if you take a look at the purchase order. If that that person is actually creating a purchase
order, but a purchasing person doesn’t necessarily have any finance background. Do you think
that the quality of the financial data is impacted?
No, not necessarily. As long as you have the right control on the fields that need to be filled and
you have good instructions on how to fill them. If you need to fill the base price, the entities, the
VAT and everything, and if the system forces you to do so and if you have good understanding
of what you’re going to enter in each of them, then I don’t think it can impact.
But that’s really about a proper setup of your ERP system.
67
Yes.
So then is that part of your main concerns in your audit, looking at the setups?
Main concern no, but it’s part of the entire audit. It’s clear that if you see that everything can be
modified and there’s no system control, then you have a problem, because then you have this
consistency issue. Then you have different control issues. If you don’t have a rigid process at one
end, how can you reconcile an invoice against a PO?
Do you actually use TeamMate for your audit planning?
For the audit planning, no. For the audit documenting, yes. There is a planning module in
TeamMate, but we don’t use it. We do our planning based on Excel. With planning I’m talking
about the annual planning. The next phase is the planning of an individual audit. So then you
need to know what is going to be the main focus. When am I going to do it? What should I take
more attention to? For that we have several ways to do it. We generally plan a conference call
before or meeting with the business owners, the CFO, CEO or the main stakeholders. Just to
understand what they are doing, how they are doing that at a higher level, what they think are the
risk area. What we do also is that we have developed a KPI tool, so we have defined criteria and
we pull the data from Hyperion and we apply that financial data to our KPI’s. Like is there a
deterioration in their receivables? How is the revenue for one year compared to the other? How
is the electronic revenue compared from one year to the other? That also helps us drive those
discussions and also discussions during the audit.
And then more discussions about where is the risk.
Yes, it’s identify areas of risk and from there have discussions whether it really is a risk or is it
something that shows on the financials, but has a explanations to it. We pull those numbers for
existing entities. We have these conference calls or meetings. And then we start asking for
documentation and prepare. And then we go on-site.
So if I hear you correctly the planning of an audit is really depending on the risks which are in
the part which is being audited?
Yes, we never do the same thing. We always adjust our focus to what the business is doing and
to why this entity came on the planning that year.
About the risk assessment. You already explained that you have the KPI’s and on top of that the
discussion with the CEO, CFO. And when that package is done, you go to the board.
No, then we prepare a planning letter and that explains what is going to be our scope. When
we’re going to go there. And then we send it to the business entity, with a copy to the division,
the CFO and people that need to be aware of what we’re going to do. No, at that stage we don’t
68
go back to the audit committee or the board. We really make our planning letter. Then what is
going to be our scope.
What type of data entry is driving the risks you think?
What type of risks?
You as an auditor, what is driving you to say “I want to investigate this area?”
Let’s say that in a normal operating company. If they have projects which are big, then we have
to look at them. If we see increasing collection time to see what is going on. If we see reduction
of revenue in one area what is going on. If you see high turnover of employees, what is going on.
In different entities, let’s say an acquisition, it depends on why we acquired it. What is the story
that management tells you and where they are compared to dealbook. Did we acquire an asset?
Generally really simple. Did we acquire a competitor and do we want to integrate systems? That’s
a big risk if you want to integrate systems. Then you want to look on how this is done. Did we
plan to integrate them in the back office? That drives us to say, this is risky area. We need to look
at the integration of the back office. How the products offering. If we talk about a process like
accounts payables, the big risks are in the master data, in payment handling, in approval. That’s
kind of standard. Let me think about something we did recently. Some times there’s a change in
online platform. So you want to understand where are you in the project, how are you going to
handle it, how you’re going to handle the transition, how is it going to be connected to the back
office. It really depends on what are the highlights of what is happening at a certain moment.
To deliberate a bit more what my research is about. You already gave a lot of information in the
direction which I want to go for. What I’m researching is what are the challenges in audit
planning and how does the use of an ERP system or the data quality in the use of an ERP
system, have an impact on the audit planning. Of course you gave a lot more information, but
that’s good for me to get more understanding of audit planning and of quality.
The difficulty in Wolters Kluwer is that we don’t have a unity of systems. We don’t have a unity
of processes. So our difficulty in audit planning is to understand what the entity which we’re
going to audit is doing. What system they’re using to do that and each time we need to adapt.
Each time we need to learn again what the processes are. You need to learn the processes from
scratch each time, because they use different ERP’s and things like that. When I was working in
banking, we didn’t use an ERP, but if you talk about personal banking, it’s one system. Already
from that central system you can pull data and you can already start making analysis. You can
almost start monitoring audit, because if you have a good central system with everybody using
that central system, you pull data and you create your KPI’s and you can see where something is
going wrong. If in one branch you see a higher default rate, then there’s a problem there. There’s
69
a problem in how they handle credit. There are different types of analysis you can do on the data.
We would love to do it here, but we can’t. The only system that is common is HFM. And so the
only KPI’s we could build are the ones around HFM. But I cannot ask every time, give me a list
of your open PO’s. In some cases they use a PO system in others they don’t. In some cases open
PO’s matter, in others it doesn’t. And we don’t always do the same things the same way. If you
look at the European companies; you have now L&R and T&A. Some of them in software,
some of them are still highly in books, some of them are selling online, some of them everything
is in SAP. For others some of their business is in that system or in another.
That comparison is actually quite interesting for my research. How much does your audit
planning change if there’s one centralized system or if it’s scattered?
If it’s scattered then you know it’s going to take you more time. Then you need to understand
how much of the revenues is coming from each of them, to decide do I need to leave some of
them aside. How old are they? Some processes are there for a long time, so they are established.
In some cases it’s a new system. Then you can say that you need to pay more attention to that.
Then it will impact the planning, because you first will need to talk to this and this person,
because they are more in that process. In certain cases, what we’re going to do is standard work
programs. We look at order to cash or purchase to pay or both. In terms of what we want to do
is the same on how we do it, it’s different as we will get different control descriptions. If you take
Italy; everything is in SAP. For order to cash, purchase to pay, all the products, the order entry,
the invoicing, everything is done in SAP. So what you need to do if there are different product
lines, is try to understand what is the difference in terms of order entry. But if you want to
retrieve the information it’s only in one place. If you want to retrieve access, then it’s only in one
place. Generally the controls are similar in terms of revenue recognition and everything. If you
go to France; they used to have one system for book orders, another system for advertisement,
they had another one for software. And each of them was related to different processes and
different teams. So you know it’s going to be different each time you need to assess how does it
work here and how does it work here. So it affects your planning, because you know you’re
going to talk to different people. If you know that advertising is 10% of the business, then
maybe I will skip advertising this time. But that affects your planning, because the type of
questions you want to ask or the way you’re going to schedule your audit is different.
Do you see similarities? You mentioned Italy is using one centralized system, SAP. Do you see
other entities which have similar type of environment with everything in one system?
There are.
Do you see any big differences in your audit planning?
70
No, but again we still need to understand the whole process. You need to see how they use it,
because they probably don’t use it the same way.
And the products or product mix might be different.
Yes. And then because the products are different, the way you process your sales can be
different.
I think I really heard a lot of things, which I can and will use. Also if I hear your correctly, using
an ERP system you will have a lot more focus on who has access to which part of the system.
Whereas a non ERP environment, you will have more focus on the data entry between the
various systems.
And the flow between them.
You also mentioned that when starting an audit, your main concern in audit planning is coming
from risk. Really about getting to know the process.
On the planning phase, getting to know the business. What are their products? Where is the
revenue coming from? And from that I can imagine there are area’s which are more risky then
others.
Just by the nature of the product.
Yes, the nature of what we sell. Some of the business units sell something completely different
from one and each other. FS, for instance, they sell software. But with that software they sell
implementation consultants and they sell projects. So in a total sale of 100.000, there is 50.000 on
software and the rest is professional services. You need to have proper controls on those
professional services. When you sold it for 80.000, does it cost 80.000? How can you monitor
the work of those people? Do you have the right controls on time sheets? Do you have the right
control on the utilization of those people? When I know there’s a big area of professional
services, I know that I will need to spend to time looking at that. Besides from the normal sales
recording process. But this is a big area. We had businesses which have sold professional services
at fixed price. And then when you start looking into it, you realize they’re loosing money. Then
the next time you really want to understand how you monitor that. How do you make sure that
you’re not loosing money? First they explain you what they are doing. Some times it’s the first
time we go there. We need to understand what they’re doing and from what they’re doing we
think where can be more risks.
Like I said, I’m getting to know a lot of things. This is still a very wide interview for me to find
my specific research.
I will send you our annual planning methodology, the criteria and I will send you also
information about our KPI’s and why we created them and what we want to do with them.
71
For sure I can use that as well. Are there other things which you want to mention, now that you
know what my research direction is?
For every company I worked for, I worked for several companies as internal audit. Planning is a
critical area, because what you don’t want is to come at the end and when you write a report and
you find that maybe I didn’t address the right risky area of that business. By planning an by
getting a good understanding you make sure that you don’t get that.
So risk assessment is really a big part of the audit planning.
Yes, actually at Wolters Kluwer, we don’t spend that much time on planning, but when I was
working for the bank, we were doing two weeks of planning. Like we were getting in the entity
for two days, getting an understanding of the processes. Then you come back and create the
work program and then you apply that work program. To make sure that everybody understands
what they have to do. What needs to be covered and how it needs to be covered. And also to
make sure that you cover all the risks. Here we do it more informally.
Because the teams are smaller?
The teams are smaller and more senior and more autonomous. The entities are smaller. We’re
not in a regulated business. We do not have the central bank coming over to check. We’re also
less triggered on documentation and when you work for the big 4 or for a bank, than all the
work papers need to be reviewed and approved. We don’t have that, we don’t have the team for
that.
Have you ever worked with other audit planning tools then Excel or TeamMate?
The first company I worked for, we used auto-audit. And there you had to document your work
plan. In the sense that for each audit you have to create a work plan with all the area’s you’re
auditing, description of the risks and the controls, the remaining risks.
Yesterday I had a discussion about TeamMate Analytics. Are you familiar with that?
No.
It’s basically a program that based on your risk assessment you can point out which transactions
you actually want to look into.
I think it’s an audit dream, a fantasy, to have tools to help you get data, crunches that data and
shows you, here is the problem. Everybody wants to have that, but it’s not workable. You can
only use tools, when you know what are the processes. So if you’re in a very centralized company
and everybody uses that same system, then data is standard and you can have that. But when
each time it’s different, it doesn’t make any sense.
So how do you guys do that?
72
We get the data locally. We use excel. We look at the process and think about what can go
wrong. Like do we have different suppliers with the same bank account. Do we have suppliers
which have changes in bank account many times.
So any logic which you guys have you apply in Excel?
Yes. Let’s say we investigate the information of credit notes. What is really important to do with
the data is how much credit notes have been created in a given year for invoices which were
raised in the previous year. Because if there’s a big proportion, there could be a problem.
Somebody played with the revenues. I think it’s better to have this kind of logic, this kind of
area’s which you can look at. Because the tool does not look at the situation.
Then I think that’s it for now and I want to thank you for this time.
73
8.5 Appendix V: Interview #3
Please note that our conversation will be recorded. I have a list of questions, but that’s more of a
guide and not a tick box. I’m not looking for specific audit information, but more on the audit
planning. That’s both the annual part and the specific audit planning. The first interview will be
more to get an idea which direction my research will go, before I get into any specifics. At a later
stage, while I’m learning what is happening in audit planning, I can specify my research. Then a
second round of interviews might take place.
Can you tell me about your role in the organization?
I’m a senior internal auditor within WK internal audit. I’ve been here about 6 years. We’re all
actually senior internal auditors. We all have comparable experience, so basically in the senior
internal audit roll, what we all do, is take part in the annual planning process. We all contribute
heavily on that and interviewee #2 maybe a bit more. And we’re also all responsible for
executing audits, realizing audits. Interviewee #8 gives us quite a bit of lead way to get things
done. So that’s with regards to the senior role. I have the add-on responsibility of being, it’s not
really a title, but sort of being the quality or compliance officer. As an internal audit department
we need to be sure that if we say something that it is 100% correct. If we’re going to the CEO or
the CFO and say that something is going wrong or if something needs to be done to fix it, they
need to be able to trust us. So in order to ensure that we never get it wrong, we then have a
quality program that’s setup in place, to basically validate the information that we’re producing to
ensure that our processes, that we cover everything that we need to cover, that we don’t make
mistakes, that we’re looking at the right scope and that we’re executing that work in a way that
leads to correct conclusions and results. That’s also that interviewee #8, who is the head of the
department, is an RA, I’m a CA, which is the Australian version of an RA, and further in the
department we have people who are CPA’s, CIA’s, which is certified internal auditor. And with
all those professional bodies they also require that we have some sort of compliance function in
place, some sort of compliance program. And they set standards of how we need to do our work
and we need to be able to not only work just to those standards, but demonstrate that we work
to those standards. So the quality program in the way that it is setup that’s one of the goals of the
program. Perhaps the second important goal. The most important goal is of course to ensure
that we’re adding value and work as an internal audit department should. So in that role as
compliance officer, I think, these days, processes and systems have become a lot more
intermeshed with each other. From the face of it, I’m responsible for the process, to ensure that
our process has been setup in a logical way. That it will achieve all the goals, that it’s supposed to
achieve. But that also means that I’m responsible for the systems. Because the process is to be
74
setup in TeamMate itself. They’re so highly intermeshed, so the system can assist in ensuring that
we follow the process if it’s setup in the right way. So I also have that hat on my head. And as a
third thing I’m also responsible for reporting to the audit committee on a quarterly basis.
Preparing all the reports as in theory I see everything.
It’s a small team.
It’s a small team, but the issue that we always faced, is that we’re one small team, but we’re really
two small teams, separated by an ocean. And to ensure that we have consistency in what we do,
in the work in Chicago and here in Alphen, is perhaps being the biggest struggle. Also the US
mentality and mindset of what internal audit really is, as opposed to the European mindset is
very different. I think here in Europe, certainly interviewee #8’s view, what I see in speaking to
other heads of internal audits in European companies, all the people I’ve worked with in internal
audit in Europe, generally they are a little bit more senior, they have got a little bit more
experience. You never really have a situation that someone qualifies with a bachelors degree
from a university in banks suddenly are in internal audit. Not in the industry. In big 4 yes, but
they have then very rigorous levels of oversight and training and so forth. We’re really not large
enough to actually do so. Internal audit within Europe will have five years of experience before
they work in internal audit, maybe as an external auditor or maybe in some other sort of finance
role; financial analyst or business analyst, or something like that. Or coming from a completely
different direction like they might have been an engineer or they might have been computer
programmer or something. In America you get a lot more junior internal auditors, even in
industry itself. Because audit itself has a little bit more green pin, red pin checklist approach. And
that’s a lot easier for someone who has very little experience. Like here’s a checklist, do it. We
don’t like operating just purely on checklists, because that can also create the problem that it puts
blinkers on you and you don’t see what is actually happening. Folks getting the checklist
completed. Obviously there’s a checklist component in there, in order what needs to happen.
Because that’s how you ensure that you don’t miss anything. But that’s not all that audit should
be. That’s sort of the mentality that I think we’ve struggled with in the past between European
internal audit side and the US internal audit side. That said, all the auditors we have in the US
also have a lot of experience. And I think if you work in a job for longer, then obviously you
want to grow yourself, you want to challenge yourself. Instead of having checklists and doing the
same thing again, again and again. That’s boring, so the US team has grown as well and they are a
lot less traditional monkey work order directed now. Now they’re thinking a lot more and not
just ticking.
Do you think that has an impact on their audit planning?
75
Yes, it does. What it does, I think, it opens up the whole audit planning process. When you
you’re going through planning what you’re actually going to do, you’re no longer limited. If you
approach it in the order of wider mindset, maybe global mindset is a better way to describe it.
When you’re doing the planning you say: “Well, what are the real risks in a specific entity or a
company as a whole?” if we’re talking about the annual plan. And then you open yourself to look
at other areas that maybe are more risky, but you don’t have a checklist for. Where as you’re just
in the checklist mindset, like what can we do, we do this or this. Or say, we’ll do that one, that
one and that one, because they’re the highest risks out of this list, but you don’t necessarily look
at other areas, which could be riskier. You might not have had experience in the past or you
might not have a work program or a checklist in place. I think it improves the audit outcome,
but it really starts from the planning phase, because in the planning phase you say what are we
actually going to do.
And in audit planning: once you’re doing an audit and you have your list of risks, based on which
you perform your audit, and you see new things, which are not on your risk list or risk
assessment, how do you deal with that?
We are fairly lucky. For most audit shops within the industry, we’re not charged out to the
business. So the only costs to the person we’re auditing is the time we’re taking up, while we’re
actually there doing the audit. And potentially political, if we drop a bomb or something. That’s
really the only cost to them. There’s no financial costs. So that gives us the flexibility if we’re
doing an audit that we’re planned that we want to do. We do this and this and this and we go out
there and see there’s something else that’s a huge risk, usually we can add that on, but still cover
we said that we were going to do. Because we have a bit of flexibility in that regard. And
obviously that’s something we do if in terms for our own time constrains, we’re trying to finish
off an audit and we have a hard stop. That means that we can extend the scope into this new risk
area. Probably we just drop something that’s in the existing risk area.
That depends on how high the risk is compared to other items?
Yes. Absolutely. When we’re talking about areas of scope that we’re looking at and usually it’s
sort of a whole audit world that’s reflecting standard business processes. You split up areas of
scope into revenues and receivables, procure to pay, closing of the books, HR, ITGC, perhaps
other IT applications controls or simply the application and that’s pretty much how we break up
our scope as well. I think you would find if we have limited time and we see a risky area that’s in
the scope, but not really part of the intended work program originally or we see something that’s
completely outer left field and totally not covered in the areas of scope, we visit it rather than
drop something entirely. We probably do just a little bit less work in a specific area, to free up
76
time to focus on the new risk area. So if we talk about revenues and receivables, you can break
that down into many subsections, like maintaining a price list, discounts policy and how that’s
handled, order confirmation with clients, etc. Sort of the logical process. Maybe we will drop just
one of those areas entirely if we don’t have time to do it and we need to do something else
instead. Or if we’re talking sample sizes, we might reduce sample size. So instead of looking at 30
sales orders where the price deviates from the price list, we might look at 20 instead. That’s
probably what takes most of our time. That’s the more boring part of the job. Although it needs
to be done.
Currently you’re making use of TeamMate.
Yes.
Did you have experience with other planning tools?
We don’t use TeamMate for the planning. TeamMate is a core product, which is called EWP,
Electronic Work Papers. And there are a lot of build on products, which you get when you buy
the product, in our case we get it for free. You get the whole thing, but you can select the
modules you want to use. We use electronic work papers, which is the core module. That’s really
to document what we’re doing during an audit and a lot of the planning information for that
specific audit will go in there as well. But the tool is not used for the planning. It’s used to retain
the output of the plan. From an annual planning point of view they then have a module called
TeamRisk, which is really intended for annual planning. We don’t use it. We do all our annual
planning in Excel. We could use the software and that’s something we toyed around with. We’re
in the situation at the moment, we do annual planning once a year and it sneaks up on us every
single year, meaning we don’t have the time to do the planning at all, so to put it into a system
and use the system. So it’s something we might do in the future. Probably not this year, because
we’re trying to upgrade TeamMate, because we’re not running two versions behind the current
version and we have some difficulties in finding out how to do that, because GSS host it for us.
Then it’s hosted by T-Systems and will move to Atos or maybe not. So nobody is really sure at
the moment. Until we figure that out and get the upgrade, we don’t want to add complexity to
what we’re doing.
Most likely the Shared Services won’t allow it anyway to have both adjustments at the same time.
Well, it shouldn’t be too much of a problem, because we already have TeamRisk. We can open it.
When you get the TeamMate, then you get the entire package, you just decide which modules to
use. It’s really just about changing the process here. The struggle with that is when you want to
bring in a new system, there’s always going to be a bit of push back if you want to change
anything. And the mindset here is not necessarily going to… if we use the current process and
77
use the system instead of Excel, then it’s not really adding value. And it might be subtracting a
bit of value, because people having to learn how to use the system. If we would change the
process, then the system could be used to make it very efficient, because you have the ability to
send out questionnaires, if you develop some sort of email recipient, than the results can be
automatically gathered back. But at the moment our annual process for performing the risk
assessment device in annual plan, has three phases. The first phase is divining or validating the
audit universe. So we have every WK entity in a big list. And we need to make sure that
everything is in there. That business units that have been sold, have been removed. Most
importantly that acquired business units are added. Sometimes we get merged business units,
that’s happing at Health at the moment. It’s going from three business units to two. So we need
to have that reflected in the audit universe. And then we read the VSP’s, any information we can
find from acquisitions. We read the acquisition proposals and dealbooks, etc. And then we rate
all of those entities, I think it’s not 13 risk factors, we call them risk criteria. We rate all the
entities against those 13 factors. And then the output is a list that you can sort based on how
risky an entity is. Anything that scores over a certain amount in terms of risk, we say that needs
to be on the audit plan for next year. Then there’s a bit of manual adjusting. That’s basically how
the process works now. But all the rating of entities comes from us. We speak to the external
auditor and we speak to the internal control organization within WK, to get their feedback and
opinions, but that’s at a very high level. What we’re not doing is performing any kind of self
assessment. Or asking the business to perform a self assessment and potentially that could be an
improvement, if we had a questionnaire that we could send out to all the CFO’s of every single
business. They know a lot more about their business in terms of detail than we can possibly
know by reading 100 VSP’s. They’re really difficult documents to get through. In a one month
period, when we’re rushing against a deadline the audit committee is coming up and we’re
coming up with a plan for them. So that might be an improvement in the process and then the
system could easily help us. TeamRisk would be fantastic for that. But we’re not quite there yet.
But you already have a process in place for your team, for planning. And the risk assessment is
one of the first steps then?
Yes. That’s in terms of the annual planning the first step and then that will basically be our
rolling three year plan, with a focus on the coming year. So we have then all the entities in the
risk universe. They’ve all been rated, based on the various risk criteria. And everything is being
scored. And then everything that scores above this, needs to be audited next year, and the next
will be in 2016 and the next scores will be in 2017, for example. And that’s what we submit to
the audit committee and they sign off on that in the Q3 audit committee meeting. So they’ll sign
78
off on the plan in Q3 and then it’s our job to actually execute against that plan. We schedule all
the audits very tentatively, so these will be in Q1, Q2, Q3 and Q4. And then we’ll start the
planning process of a specific audit. So maybe in December we say what are we doing in
January? Let’s plan that order. We will start by setting up a call with local management where we
talk about the business. What are the risks? What keeps you up at night? How does this process
work? What systems do you have in place? Etc. etc. And we also have the output of the annual
risk assessment; we know why that entity was scheduled to be audited in this coming year. What
are those reasons? Obviously those risk factors that have caused this entity to be rated very
highly will then play into how we will divide our scope. That information coming from the
annual plan. We have the information coming from speaking to management. We also have
perhaps the last audit report. That’s also some information. Although we probably audit big
entities every 3 to 5 years, so a report that’s 5 years old, is 4,5 years out of date.
And big being with high sales numbers?
Yes, about 50 to 100 million of revenue. That is very big. Of course we also have a lot of entities
that have no revenue. GPO has no revenue. GBS has no revenue. They’re very important and
there can be risk there, so that needs to be factored in there as well. So those will be the biggest
inputs to devising a scope.
How much do you consider the company policies when you’re doing an audit?
It depends. We’re a very fragmented company. We’ve grown by acquisitions. That means, when
it comes to company wide policies, there aren’t really any. There are some, but they’re not treated
that way. You find a big schism between business that are owned by the US and business that are
owned not by the US. There’s almost standardization in the US, because they have shared
services, who do things like payroll. So any US entity, whether you work for Health, whether you
work within CCH, if you work for FCS, Law & Business, certain aspects of those audits will be
the same. And so we don’t need to re-audit payroll for every single business we go to in the US.
We can just do one big payroll audit of shared services and then we’ve covered every single
business in the US. In Europe and also to some extend in Asiapac, everything is a lot more
fragmented, because you have the local legislation. France is run as one country with it’s own
little implant. Germany is run as it’s own little implant. And there you will find particularly with,
English is not the business language, you’ll find from a policy perspective that generally they do
their own thing. You also have a situation that for some policies that in theory might be
company wide, that they are implausible, because particularly in France, local legislation says
certain things of how payroll can be run out of privacy issues. From my perspective, I treat every
audit a new experience. I’d like to go in as much as I can, but I don’t like to go on pre-conceived
79
notions. My assumption is certainly, that if I go to France and say what policies do you have?
They’ll give me a list of policies. Some of them will be translations of a global policy. Some of
them will be complete France only. Some will be Europe only. From an HR perspective, the
European HR are fairly strong in pushing stuff down within Europe. A lot of IT stuff can be
fairly standardized, like acceptable used policies are probably pretty much the same everywhere
in the world, with the exception of a new acquisition, who hasn’t had time to push to WK
acceptable used policy. IT is very important. HR is with the salaries, they’re happy and you don’t
want to knock down any type of privacy legislation or any of that kind of stuff. But that’s not
quite as important as the big thing, which is making money and spending money. The revenue
process and the purchase process that are really the important ones. And there you will find no
standardization at all.
But only in the US with the use of shared services?
In the US even less so. On the purchasing side. For example expense reporting, when they
travel, they submit expense reports. In the US that’s all very standardized and have got a concur
system in place and everything goes through there. And there are system controls that say if you
want hookers and tequila, that’s not an acceptable option. Whereas in Europe it is a lot more
paper based and you can squeeze the hookers and tequila in your expense reports along with
something else. So there’s standardization there. There will be standardization within the CCH
portion of the Tax & Accounting business in the US, because of one being one business with
one CEO and it’s really run one way. But if you were to go to Law & Business side in the US,
that’s a separate business with a separate CEO, not at all the same as what they’re doing in CCH,
with the exception of some times where there have been initiatives to go to a single system.
There you might find similarities. In a lot of cases the system defines the policy and not the other
way around.
That’s what I read about ERP systems, that’s they’re not really guiding, but pushing
organizations into certain processes.
Particularly in the more monolithic ones. If you talk about SAP, it’s easier to change a process
then to change the system.
Talking about ERP’s. If there’s use of an ERP system, how does that impact your audit
planning?
It makes a very big impact. There will be an ERP system everywhere. Nobody is using paper and
whatsoever. We have so many different ERP’s within WK. Depending on what the ERP system
is, depending on how important that is, that’s always going to appear in the scope. We will
usually do an IT general controls review. Which will then cover the platform on which the ERP
80
is running on. Depending on whether it is custom build, which we don’t have in a lot of places,
but we may make it. So often we’re using our own product as an ERP system. Or if it’s highly
configured of the shelve product. Or if we have developed our own modules. Then there’s a lot
more risk there. Instead of buying an of the shelve product, install it and using it. So we will
probably look at those areas a lot more closely. If we have developed an order entry system that
will interface with an ERP, then we will look very closely at the interface, because that’s probably
where the problem can happen, if a problem is going to happen. So pretty much it’s always going
to be in scope. We are not really at the phase as a company where we’ve got a good enough
handle on IT general controls to really enable us to take the next step in the back office systems
area, where we’re really look at our application controls.
Like who has access to which part of the system?
That’s sort of it. Application control is sort of very specific within an application and it’s really
the processing and output controls within a system. So you’ll have hash totals for file transfers
and that sort of stuff. And if you really look into those controls, it’s field controls so you can’t
enter alphabetical characters in numeric fields. That you can’t enter a date which is a hundred
years in the future. If you create an invoice for a hundred billion dollars, that’s going to raise that
it’s probably a mistake. It’s logic within the application. Be is SAP, Navision, anything we’re
using. In order to, from an audit perspective, place reliance on a system, in the old days of audit,
you would have to take a sample of 100, now with such a system in place, you can take a sample
of 1. And if the system works, and the sample of 1 worked, then you can assume it has worked
for all the millions of that went through. In order to do that you would basically validate the
application controls, but you don’t even look at the application controls if you can’t validate the
general controls, which is then the layer beneath the applications. So it’s really the whole
platform. So it’s about change management for the software itself. It’s about segregation in terms
of access to the source code, access to the production environment. Are developers making
changes in the production environment version or do they have a separate test environment? It’s
about backing up of data and storing of data. It’s basically all of the controls that the system
relies on. And if those controls don’t work, then you can’t really trust a system at all. If those
controls work, then you can look in the system itself. Do those controls work? And if they do,
you saved yourself a hell of a lot of time with everything else. But as a company general control
level, they would never really take the step to say “everything is perfect here in the general
controls so now we ‘re going to look at the application”. So generally what we do is a little bit of
everything, because we want to see the general controls improve, such that we can then test
81
application controls and then we can stop all the manual monkey work that we would otherwise
would have to do to ensure that everything is working as it should.
If I’m not mistaken, there are some entities which are not using ERP systems.
Depends on your definition of ERP systems.
If there are more databases used, for each part of a process for example.
I don’t think there are many entities, which have one system that does everything.
Which is a full ERP. But let’s say that the level of integration of systems and databases, might be
of how much you use of one ERP system. If the level of integration is quite low, would that
impact your audit planning a lot?
It would, because the more interfaces you have, the more comfort you need to get that the
interfaces are working. If they’re not working, that’s where the problem is going to be. So if you
have one system, I can’t think of any entity having one system that does everything, what you’ll
find that there will always be bookkeeping system, generally the purchasing side will flow
through that bookkeeping system as well. Although if depends on the business. From business
that have a history in old school publishing, where they really buying paper and buying books.
Then they might have a completely separate system, sort of a materials or purchasing system for
the production side of things. And then purchasing in terms of services which aren’t production
related and all the other types of purchasing, will go through the bookkeeping system. So on the
order entry side it’s very common that there is a completely separate order entry system. And
that becomes very important, both from a real financial audit point of view and from an
operational audit point of view. Because you don’t want to have orders placed, that are not going
to get fulfilled or orders in place that do get fulfilled, but never get billed. And so the real testing
of the interfaces, the end to end testing of a process will be affected by how monolithic the
systems are. Payroll is almost always in a separate system. Obviously when we’re planning we’re
looking to see how are the reconciliations done between the two systems, because there won’t be
necessarily interface checks between a payroll system and a bookkeeping system, when once a
month they process payroll and sort of upload the figures. We can do the reconciliation, but a
company should be able to exist, without internal audit existing. It’s not about me doing the
reconciliation, but about you doing the reconciliation and I’m checking that you’ve done it. And
if you’re not doing that, then that’s an issue which we’ll be raising in the report, because you
should be doing it. That’s sort of how the number of systems will effect the audit planning.
Financial numbers are traditionally entered by accountants, who have specific knowledge about
what they’re doing.
82
It depends. The overall numbers will come from people who aren’t accountants, because we
have the systems setup. So you’ll have an order entry clerk or a sales person directly going into
the system, that number flows through the system, but nobody rekeyed it. It’s the sales person
who entered that number, that’s now ending up in the accounting system.
How do you think that impacts the quality of the financial data?
That’s a tough one. In one way it improves it, because the people who are really inputting the
data are the experts. A sales person knows when he has made a sale. An accounting sitting in an
office who is maybe talking to that sales person by email or whatever, doesn’t really know what’s
going on in the field. So in that way it would improve the numbers a little bit. There will always
be mistakes in numbers but when you get a big enough population, a lot of those mistakes cancel
each other out. When you have one person in accounting doing all the entries and that person
happens to have a bad day and is making the same mistake again, again and again, then the end
number is going to be completely wrong. Which means it could be identified, because maybe it’s
so wrong it sets something all right here. But when you have a hundred sales people entering
those numbers, a lot of the small mistakes cancel each other out. So you get maybe an improved
accuracy in the overall number, by having a lot of little mistakes. To some extend. And in theory
you should have less mistakes in that regard. The big problem really would be in the interfaces,
because a sales person is doing what he’s doing. And there’s an assumption that everything is
going to flow through correctly. The accountant looks at this month’s numbers and last month’s
numbers and says it’s about the same, it’s what we expected. That’s great, but if there is a mistake
in the interface, then maybe nobody has really going to pick that up. Because nobody also knows
the whole process. If you have separate systems and everyone is sort of looking down and doing
the thing that they’ve always done and nobody says “Hang on, is the whole process from end to
end really working?” I think that’s where you get a lot of mistakes. This person is changing the
way they’re doing things and has a project to make things more efficient and all of a sudden the
output of their process within their system now changes. This person doesn’t notice it. Makes it
sort of falling off the table and nobody realizes it. Which to some extend we’re there to notice.
So if I understand correctly that it impacts on your risk assessment.
Absolutely. The numbers of systems that are in place, how long the system has been in place, to
what extend it’s been customized, specifically build, or if it’s more off the shelve with a more
simple implementation, that will effect what is in scope and how long that scope will take to
execute.
This is giving me a lot of ideas for which direction to go for my research. There’s really plenty of
material for that. Also prior interviews gave me more or less the same impression and have given
83
me a lot of thoughts. So that’s something I have to look into. You mentioned the use of ERP
systems, that if someone in a part of the process is adjusting the way of recording, it might not
be picked up on the output side, which can be in a different area. You also mentioned that
basically the accountants are more or less controlling the process, or the output.
Usually they don’t have too much say in the process itself. Different countries hold accountants
to lower or higher esteem as well.
What area would you be more checking? More into the control side, the accountant side, or
more to source side?
We try to go to the source as much as possible. Usually our chief contact is going to be the CFO,
rather then the CEO of a business. And then the person who is going to be helping us the most
is probably the head of controlling or accounting. And then it’s very easy to go to an audit and
talk to the accountants and the CFO the entire time. Based on that you could write your report
and say everything is correct. But if you’ve spoken to a sales person, if you’ve spoken to
someone at marketing, if you’ve spoken to a secretary who’s inputting purchase orders for a
department, that’s where you can actually get a lot of value adding recommendations, coming out
of an audit. So we really try to speak to everybody. Or someone from every area. When it comes
to data analysis we’re getting into a situation where we use a lot more analytics software instead
of doing sample testing look at everything. So if we can get a data file and really do some analysis
on that data file then we can find all the problems I suppose. So to that extend you will find a lot
of the details in the data, which is going to be in the source system and not in the accounting
system. So we very much try to get information that is coming out of the source system. Some of
the really nice things to do is to get a customer master file extract and then we can do all this
wonderful stuff where we get an extract of the employee master file and a vendor master file,
one is coming from either a purchasing system or from the financial bookkeeping system where
purchasing is going through there, the other one is coming from a completely separated HR
system and we can look for duplicate bank accounts and then we can say “haha, we found a
person who set himself up as a vendor and is paying himself when they really shouldn’t be” by
comparing date from various systems. We wouldn’t be able to do that if we didn’t have an
understanding of both the source system as well as the accounting system. If we hadn’t spoken
to both someone who is administrating this system and someone who is administrating that
system. And then of course the other thing what I would like to do a lot more of, is getting into
a lot more statistics and run regressions on data, because we’re getting to a point that there is so
much data now that you can really get meaningful out put from that. I was at a training seminar
two weeks ago and this guy was demonstrating examples of this. He was an auditor for a
84
company that owned dozens of amusement parks in the US. And they have them everywhere.
They did this very interesting thing where they did just a simple regression between the revenue
per day, so the tickets, how much money was collected from tickets and the tons amount of
garbage which was removed every single day from an amusement park. By doing that you’re able
to see the park which is outlying where there is a lot more garbage per unit of revenue. Then
they send people there to see what was going on and apparently a lot of the ticketing people
letting their friends in for free or were taking the money themselves. Looking for those sorts of
patterns where there is not system which is comparing the revenues and the garbage by park, but
if you get information from two different systems and put that together, you can find very
interesting patterns. And if they found there was nothing wrong, if they found that no particular
park was an outlier or no specific park on any day was an outlier from every other day, then you
can say, probably there’s nothing going wrong here. Or there is something going wrong, that’s
going evenly wrong. It’s that sort of stuff that would be really interesting. In user systems, we
have ACL licenses, it’s data analytics software, which can read about every file type and when
you can find the information when it’s coming in, validate it and play around with it and then
you can run all sorts of statistical comparisons between the various information. And when you
get a full file, with the whole year worth of transactions, or a master file at a point of time, then
this is the universe for this business. Then you can do a lot of interesting stuff with that. And
you can even do it in Excel.
Did you ever see anything about TeamMate Analytics?
Yes, I actually tested it, before we bought it. I gave my recommendation to TeamMate whether it
was good or not. It’s quite nice. Part of the reasons why I’m the compliance or quality guy is,
because I’m really good with Excel. There’s nothing that TeamMate Analytics can do that I
couldn’t do before. But it does make some things easier, faster to do.
It might make it easier also for non-Excel specialists.
Exactly. Out of the department… If something needs something done, well, then interviewee
#3, can you do this? And as a result I’m always the one doing it and then they’re not learning it.
If we had TeamMate Analytics, which we’ll probably get in the next upgrade, then they can do it
all themselves. It also does benefits analysis automatically, which is really fascinating. That’s
something we do when we’re looking at accounts payables and expenses as well, then we might
do benefits analysis by cost center and then you can sort of see if maybe someone is faking
something.
Most likely I won’t have it about TeamMate Analytics, but I just was wondering what you
thought of that part.
85
The only thing is, that TeamMate Analytics, we bought the either the company or the licenses to
sell it, and this is something that existed for a very long time and the product itself only had a
different name and now TeamMate bought it. Partially because some customers were saying “I
need TeamMate and some analytical tool”. To provide a full package. Although there’s no full
integration yet. So TeamMate Analytics is just an Excel add in. If you have TeamMate, then you
already have a TeamMate add in, which enables you to open an Excel file within TeamMate and
then save it. So now you have two add ins and then you can do all your analytics in Excel and
save it in TeamMate, but there’s no real integration. That said, I can’t see too much benefit of
having anything seamless. But it’s basically two products which they sell as one. You buy
TeamMate and you can also buy for the price the analytics, but they’re not integrated and I don’t
think they will be and I don’t think that they need to be, actually.
What I’m thinking about for m research so far is to take a look of an ERP system, how the
quality of data is actually changing and that part from the ERP side, how much does that impact
your scheduling of the audit and audit planning? As I hear from you and form other interviews,
the risk assessment is a big part of your audit planning. If you go to an entity and you know there
is an ERP system or not, what is your first main concern?
I would say, maybe to take one step back from that. One of the big things that would trigger for
a specific entity a higher ranking in the overall risk in the annual planning, if it changed systems.
When something is business as usual, you can have a little bit more comfort that everything is
running ok and you can assume that they are setup ok. And over time, if things haven’t changed
and the output is sort of the same, and is in line with what management is expecting, then you
make the assumption that it has been setup correctly. But assuming that’s the case, then
everything is ok. There’s a lot more risk in an entity that is going to roll out a new system, to
completely replaces an old system. So that would cause an entity to be rated a lot more risky.
And maybe an audit would then specifically be on the implementation of the new system. Either
as a go live readiness review, before they actually really go live to look into the testing, you look
into a lot of configuration settings. The entity should be doing they’re own testing to see if
they’re ready to go live and then you look at that to see if they have missed anything. So that’s
one thing we would do, or we might do a post go live assessment. Some times we focus a little
bit more on the project management aspect of it, but more often it will actually focus on is the
system doing what we think it’s doing and what it should be doing. So we’re a lot more likely to
schedule an audit based on the change in the system, particularly for the more largely entities that
is to take place. For an existing system, generally I wouldn’t say it particularly impacts the
planning in a sense that we know that regardless of whether there is a monolithic system or
86
multiple systems in place, we will still be looking at the same scope areas if we go to an entity.
But what we look at in that scope area, will then differ. If you have a fully integrated sales order
entry and bookkeeping system and fulfillment system. If that is all in one, we will then don’t
need to spend quite as much time looking at that, because you know if the order was entered
right and if it’s been fulfilled, then in theory everything in between went well. We might focus
more on change in processes, systems, discount procedures, credit notes. If a lot of credit notes
have been raised, then something has gone wrong. Then we’ll probably be a bit more substantive
in what we’re looking at. We will look at discounts from a system point of view and see if it’s
actually working following the process, that has been setup in the system as well as policies. If
there’s a policy in place. If there are separate systems, then we will probably… It depends on the
system and every entity is different. We will probably look for reconciliations done between the
systems, to ensure that everything is working. And if they don’t exist we might do it ourselves.
And then you focus on that a little bit more then for example the discounting process. That’s
really how it affects what we’re actually doing. When we go out to do an audit. In terms of
timing… When we do a regular audit, based on the size of the entity, we know sort of roughly
how long it’s going to take. So if we’re looking at Germany, France or the UK, that’s going to be
two weeks of field work. If we’re looking at a 20 million business, then it’s probably going to be
one week. If we’re looking at businesses that have just been acquired, then we have a very
different scope and we might be two or three days at a business.
Might that also be impacted if the point of entry is really scattered as well?
Absolutely. You’ve got scattering in terms of systems. You’ve got scattering in terms of
geography. We did Germany last year and we spent three weeks on it, because there are a lot of
locations. They’ve got different systems for different products. So twice the number of order
entry systems that you would expect, when you look at CCH where there is only one order entry
system. It’s huge, but it’s one. Germany has two or three smaller ones. And then you need to do
work in each one. That results in taking more time. We normally know this up front, because it’s
something that has been established during the planning course, before you define a scope, we
know roughly how much time we have available. We know what we should be doing and then
we ask the question if we can be doing what we should be doing. In this time frame. Can we
extend the time frame? Can we reduce what we’re doing? Can we get help from other sources?
Are there other points, which you say that might be interesting for your research?
I’m asking myself the question, what would I do if I were to do a research? Were my attention
goes. Where I focus what goes well in what we do and I know what goes wrong in what we do.
It’s in the area of what goes wrong. I’ve had bad relationships, because things were 99% perfect
87
and 1% wrong and then I focus on the 1% wrong, because I want it to be 100% perfect. And
then people tell me I’m too negative.
If you would be writing a research, what would you choose as topic?
Personally, where things go wrong within internal audit, and it doesn’t mean that the output is
bad, but what I see and try to fight is that there is a lot of dogmatism. Audit is very often… It’s
sort of the common joke that if you go to do an audit, then you take last years file and you roll it
over and you do it again. You’re not going to find anything new, because you’re not looking at
anything new. And what I find, particularly in terms of standardize the process at both sides of
the ocean within internal audit and define the process and make sure that people are actually
following the process, is that some times we follow the letter of the process without following
the soul of the process. All forms are filled in the way they should be filled out, but the scope
was actually defined. Because you already know what you wanted to do and you didn’t even have
the planning call yet. And then you have the planning call, but you have already decided what
you’re going to do, regardless of what will happen in the planning call. And I see that happening
some times. And it’s not just in regard of planning, but it can be in all phases of the audit. And
it’s not just auditors, but everyone is like this. Where we do things, because we want to do them
and afterwards we justify why we did them. As supposed to what should I be doing? Let’s figure
out the best thing to do and then I recap my justification, before I even say this is what I’m
going to do. It’s pretty easy to repeat the things that you’re doing. Then it quickly gets very
boring. Especially if you go to places where you have already been and then doing the same
thing. If you’re doing that you’re not adding value and you’re certainly not doing yourself any
favors. You’re not doing the auditee any favors either. So that’s the story what I’m fighting. If I
were doing research, I would be more interested from a more sociological point of view in how
to measure that, how to ensure that it doesn’t happen.
You raised a very interesting questions. In terms of the planning for the process to be real, you
have to really ask yourself what are the risks here and gather as much information as possible,
before you say this is what we’re going to do. By definition, we have 500 entities in our risk
universe. At the very start of the process, when we do the annual planning. We have 500 entities
and we’re 8 people and we need to score each of those on 13 different levels. Some of those
levels are really easy, because we use revenues and that just comes out of HFM. And then some
times you have entities, which don’t have revenues, but should have. And that’s just technical,
making sure that people cover the work correctly and checking that the final revenue is what the
final revenue should be. That’s really simple. Some of our identifiers are change in senior
management. If you have a situation where the CEO, CFO and CTO have all left within a period
88
of six months, there’s a reason for that probably, possibly. Worth looking into. Some of them are
fairly easy and some of them are a lot more… You read a lot of information and use your own
knowledge. Maybe it’s best to call the people. There are 500 entities, 13 different measures.
That’s 5,000 costs, that need to be yes / no or given a number or a rating between 1 and 5, etc.
etc. It can be easy to rush through that or to just put in an answer, which you think is the answer,
but it might not really be the answer. So at the highest level, the overall annual planning is only
going to be so accurate. There could be an entity that is very risky that you had as moderately
risky and for that reason doesn’t make it on to the plan. That’s the third phase of the planning.
We do so many things to try to avoid that. We want to make sure that the things we’re doing, the
places where we’re going, those are the riskiest ones. So we have this wonderful technical system
in place, which can have errors. We have the soft side of interviewee #8 and Xxxxx, who is the
director of internal audit, do a road show and go and speak to all the divisional CFO’s and also
corporate people who report to either the CFO or the CEO or both. And once we have come
up with the first version of the plan, then they speak to those people and ask “Have we missed
anything to your knowledge?” It’s in that phase that some might dropped in importance or come
in. We don’t know about every project that’s going on necessarily until we speak to the people.
That’s where the real risk is. It’s not going to Germany once every two or three years and say
show me your invoices. We do as much as possible to ensure that the deliverable is a good
deliverable and we do as much as possible to ensure that have an audit plan, we execute the audit
plan and the question that we know what to do in each audit. Information comes from the
overall planning process and information comes from the call. Are we necessarily looking at the
right things when we go and do an audit? Where there is the slight tendency to say we’re going to
audit something that we know. We’re very lucky to have Deep, he’s our IT auditor. He’s a
programmer, who’s wanted to become and auditor to get a different perspective on things. The
risk in Wolters Kluwer, of course we have to cover all back office systems and ERP’s to that
extend, is the reputational risk. If something is going wrong in a customer facing product…
We’re selling legal software. That’s online software for legal practioners to put their case files in.
If that got hacked… We’re moving to one brand. Wolters Kluwer is becoming the name. CCH is
going. It’s all going to be Wolters Kluwer. If our legal software gets hacked and that ends up on
the front pages, that’s the risk. It’s not that on a sight they have a mistake in the revenue. That’s
the concern of KPMG, soon to be Deloitte. It’s our concern a little bit, but we sort of leave it up
to them. The real risk is in front office applications, in customer applications. It’s in producing
software that maybe there’s a disconnect between the development and the strategy. Where the
developers are making this great software, but they’re not making it that it can be used. Those
89
kind of risks is where internal audit needs to be making a better presence. What Deep is doing
very well. In terms of planning you need to know about those sorts of activities. In the past two
years we’ve been auditing development processes a lot more. Developmentshops. We’ve been
auditing bigger projects overall. We’ve been going into Tech BV and GSS and sort of looking at
those things, because those are the projects, which are critical for WK’s entire success. Where we
are in five years time, is not going to be how the order entry system in France is interfaced with
SAP. It’s important, but that’s not vital. I don’t know if you ever read the classical IIA definition
for internal auditing, but the whole reason for being here is to ensure that processes, procedures
and controls are set up to enable a company to meet it’s goals. Regardless of what those goals
are. And that’s why we’re moving into that way a little bit more. Still covering the traditional
auditing stuff, because that’s the other reason for being us, I suppose.
I got a lot of information from you. My research might shift a bit more from data quality into
risk assessment. With the use of an ERP system, how the risk assessment is impacted and then
who the audit planning is adjusted.
The answer is quite easy there. It really just changes what we’re doing and it does for the up time.
Because in theory in the past we might have taken 50 invoices and look if the price is correct, is
the discount approved and all that sort of stuff. When you have a system in place and setup to
do that, you can look at one or two and say it works. Now what am I going to do with the rest of
my time. And that’s where you can actually look, have we missed anything? Is there something
not in scope? Or slightly out of scope or completely in a different scope?
Whether it’s in an ERP system or interfaced, it doesn’t change that much, other than that you
have a lot more interfaces to check. If there is more multiple data entry is where you have more
monkey work, basically.
Yes, absolutely.
Then I want to thank you for your time and we’ll talk soon.
8.6 Appendix VI: Interview #4

I’m in the university and I’m trying to finalize my master thesis and this research is the final part
which I need. I’m very much into ERP systems and that type of integration of systems and my
boss advised me to look into TeamMate. Now I’m looking into how the controls within
TeamMate, or audit planning, are impacted by the use of an ERP system. I got intrigued by the
TeamMate systems. Not only is the site very clear and is giving me a lot of information, but also
the people who are working on TeamMate are so helpful and sharing information in a great way.
That is also giving me a lot of energy for my research.
90
I did my masters in the university of Texas at Dallas. I got a concentration in internal audit. I
also worked for the university in the internal auditing education partnership program. So that
graduate program is, I believe it’s called an institutor program of excellence for the IIA, so it’s
the number 1 graduate program for internal auditors in the world.
So you have been teaching at the university?
No, when I was there doing my graduate work, for my masters, I worked there as a teaching
assistant for the internal auditing program there. I worked with the director of the internal
auditing program at the university as his teaching assistant. I was also vice president of the
program, the student chapter there for internal auditing. That was years ago, but I still keep in
contact and they invite me. I sit in on some of the lectures. Usually once a year, because of my
travelling schedule being a little bit crazy. They usually invite me in as a panel discussions for
when current students in the internal auditing class present back to do an audit and create a
company and do this whole big project. So they’ll invite me, as well as others, to be in a panel to
just judge and give feedback to the students in how they setup their fake internal audit. It’s all
about scheduling and how it’s going to be in their audit plan. It’s all fake, but they have to base it
of an industry. It’s interesting and I try to get in about once a year. My professor is also the co-
author of the internal auditing text book that even I used. I used the first edition and the third
edition came out last year. I helped incorporate TeamMate curriculum into the text book. So we
have about five chapters about encompass TeamMate and I helped to write a curriculum for
that. The students get a part of TeamMate that’s exclusive to students and it incorporates
standards into the work. They can get one of the modules for free as long as they purchase the
book to go along the text book curriculum.
So I’m actually talking to an author of books about internal audit?
I wouldn’t go that far. In the acknowledgements TeamMate was mentioned and there were
several people that have helped out within our group. It was great, because it’s a passion of mine
to get TeamMate in the classroom. Students can put it on their resume and when they are
looking for jobs, they can say that they already had exposure to TeamMate. That helps us and is
also a charitable contribution that we did, because we did it all for free. This helps us to serve
academic relations and to promote TeamMate in the IIA and helps students, because there is no
other audit management software that is cooperated in the text book and that travels around the
world. Also after the sessions when I’m in the panel, the professor will ask me to stay to explain
a little more about TeamMate. It was a collaboration of our marketing helping out, our IT side,
they did a development package. Module specific, which we’ve never done before. It was a full
91
team effort. I’m looking forward for the next edition, when TeamMate is updated and so the
curriculum needs to be updated as well.
And as I understand there has been a lot going on in TeamMate in the last few years.
Yes, so there will be a lot of enhancements. There will be updates, but the foundation of what
they will get will be relatively the same. So it shouldn’t be that much of a learning curve for the
professors.
What I’ve learned of TeamMate so far, is that it’s really a tool of containing the work that you’re
doing and really helping an auditor out. So the work of the auditor is not really changing, but this
is a really powerful tool for ensuring that the audits are done complete and also to go back in
history to see what has been done before.
Yes, there is a lot of historical reporting and it’s very customizable. Not only can documentation
remain within TeamMate, but over time you can begin reporting on it and see maybe trends and
certain areas where findings have occurred. So it does serve a great historical purpose as well. It
also allows you to change and modify if different management comes in over time and starts
requesting different kinds of reports. Reporting mechanisms are taking information out of the
system. It’s fairly easy to change and update that. It’s a pretty robust audit management system.
It helps out in the full realm and not just the documentation of the work of the individual
auditor, but also from a management standpoint it’s a great management tool for risk
assessment, annual planning, capacity planning, scheduling, resource availability, resource
realization for your plan. For not only to document your audit as an end user, but you can also
do a lot of things within TeamMate before the end user even begins to work on their individual
audit.
And how long have you been part of the TeamMate team?
It will be fours year this June 13th. But I’ve been a user of TeamMate since 2008. I started using
TeamMate as an end user.
And what type of role did you have then?
I was an internal auditor. Previous in the healthcare industry and previous to that I was an audit
analyst when I was finishing up my undergrad. I was an audit analyst for the government. I
worked for a local city. That’s where I first got exposed to TeamMate as an end user. When I
moved to my graduate years, when I moved to the healthcare as an internal auditor that’s when I
became more of an administrator of TeamMate and so I learned a lot of the ins and outs and at
the back end of how to set things up. To get it customized.
So you really grew into an expert kind of role?
92
Yes, with the administrative actions on it. Then I wanted to join the TeamMate family. I had a
lot of fun setting it up and I thought I can do this every week. And I’ve been doing this now for
almost four years.
Have you ever used some other auditing tool?
No, TeamMate is the one I ever used as far as an audit management system. I did use ACL and
ID for my data analysis. And now of course we have TeamMate Analytics. When I was in the
field, that wasn’t available for me at the time. But TeamMate has been the only audit
management tool which I used as an auditor.
What is your role at TeamMate at the moment?
I am a senior consultant for the US side professional services team. There’s about 13 of us in the
US and then we have a team in Canada and then a team in the UK. And there are a few teams in
Asiapac including Australia. And then we have a smaller group starting up in Latin America. So
we have kind of people everywhere that have their own team. So we work together.
Obviously TeamMate is US based, and do you also help or instruct other teams how to deal with
customers?
Yes, not so much as dealing with customers, but even though our main office is out of the US,
we have teams all around the world, that help implement, configure and support TeamMate to
their local clients. I had the privilege to go to Tokyo when one of our newer modules, Control
Management, was introduced. So I helped on boarding, learning maps from an internal aspect.
Last year I went to Australia and assisted there with another Australian client for a newer
module. I had a consultant in Australia shadow me, to learn the tools in that way. So if we have
newer launches, come out around the world, we have some times, even though the newer tool
was released months ago it’s their first time to really train or implement it for one of their local
regional clients. And then we help and if we can we will go out there, but if not, they have the
resources to do remote testing or anything like that.
So you’re not only showing TeamMate in the classroom, but also around the world basically?
Yes, and there are a lot of people that go out and assist in other regions should they need our
help.
Which parts of TeamMate are you particularly exposed to?
We’re actually exposed to all of it. We’re experts for the AM, CM and the analytics tool. There’s
not one that we do more than the other. We’re all equally trained. Some people have more
exposure to CM and analytics, because those are newer modules, but all of us are capable of
delivering. We need to know all the products that need to get in time to put on our schedule of
clients, that is implementing CM or AM or Analytics or trainings.
93
Do you visit a lot of customers?
Oh yes, I travel 90% of the time. So every week I’m out. Some times more than one client in a
week. This week I had two clients. Next week I have two clients.
So you have a good knowledge of which settings they require?
Yes. Each client is unique. Setting it up, from a general standpoint, is all relatively the same, but
then you have to know their process. Usually when I do an implementation, I get a clear
understanding of their annual plan processing, from beginning to end. So that way you know
what kind of reporting they want out of it. What does their final audit report look like? You have
to do some times some reverse engineering. Especially with risk assessments and annual
planning for their received audit plan. It’s that to dive into how they have accomplished their
tasks. Then take their final product and get it into the system. So that way going forward they
don’t have to use Excel or other forms of documentation and actually use the system to input
their data and then get the reporting out of it in an easier and much more efficient manner.
When you first start with a customer, how do you start up? What do you begin with?
Well, we usually are going over what their final audit report looks like. What is some of the data
that they want, what are the controls they report on. Were there risks. Any testing procedures.
Do they send surveys. Do they report on time. How do they do that currently. We take a look at
how they schedule their department and their resources. So you usually gather data first, then
analyze it and work with them. The foundation of the software is based on their structure, their
audit universe. And then we begin diving into creating an audit universe for them based on how
they currently assess and how they document their engagements throughout the year. We usually
start by dissecting their data and what they do and how they do it and their main processes. And
then dive into their audit universe.
Can you explain me a bit more what an audit universe is?
An audit universe is a structure. It could be a combination of departments, regions, processes
and depending on their industry. Like for health organizations it could be clinics or hospitals.
For a CPA firm it could be clients. If you are breaking down by department or function. It’s
basically the starting point for how they do their annual planning and their risk assessment. And
if we can build a structure or their audit universe, the areas in which they assess on an ongoing
basis. We can build that for them, so in that way that structure is in the software and will help
them to do their annual risk assessment and will also help them to report on trend based on
different functions or departments. Usually we start with the overall organizational structure of
the company or the client. And then we look at how they’re currently doing their risk
assessment. How they’re currently building out their annual plan. And then we kind of backtrack
94
into that and really make it audit owned to help them on how they do their assessments. In that
way we make sure we’re not missing anything throughout the year. We don’t want the structure
only to include the things they know they audit. We want it to be a representation of their full
universe of their entire company. Structure in a way that will help them perform a full risk
assessment and then be able to narrow down the areas of high risks. So that way they can
concentrate their plan around those high risks areas.
What would you say is triggering most of the times a high risk?
It really depends on the industry. Diving into certain areas of certain companies in an industry. If
you take health care, safety, contamination, privacy acts. Things that require regulation. If you
have government funding. If you’re non-profit. Or if you’re a government. If you loose your
funding, that could be the source of your financial structure. So it really just depends on the
industry and the company and what drives them. And also what their corporate strategies are and
what they want to accomplish. And if anything goes astray with the corporate strategy. I usually
think that those which are financial impacts, are usually with higher risks. But it really depends
on the particular client and their particular industry.
Especially the risk assessment part is really interesting to me.
Yes, it’s really interesting because there’s no set standard on how to do a risk assessment. You
just need to do one. So it’s very open ended. It would be very interesting to create some sort of
mathematical or a series in regard to how risk assessment in a best practice of risk assessment.
We offer a standard of impact and likelihood. But actually developing the risks. Every industry or
client will do it differently. There have been some clients who currently don’t do a risk
assessment and they ask assistance in how to start one. I usually say, you’ve never done one
before, but what are your corporate or company strategies, what are your goals for this year. And
that’s always documented. We usually obtain that document that was usually published internally
or within the company. And we take all those goals and strategies and we turn those into risks.
What are the main areas in the company. They usually have an accounting department, they have
human resources, their IT department, etc. And then we start listing out high level company
ending drastic risks as they relate to the goals, the strategies for the company that year. And then
we develop risks and we put them into buckets. And as we start brainstorming and gathering
these risks, that soon becomes our risk library as starting point for our risk assessment.
And do you ever refer to the COSO model?
Yes, we have the COSO framework and help them to put that into the system. So that way they
can classify risks based on the COSO framework. So that way they can tie the risk and document
the control a little bit.
95
Would you say that the majority of the clients you work with, that they refer to the COSO
model?
Yes, I would say a significant part. Some times they do and some times they don’t. They should
be. We offer these as options for structuring the risks so that way, if they want to classify risks,
that control environment or their internal controls, etc. that they can. And if they want to break
it down even further to even the point of focus, they can as well. Usually we recommend, when
we’re setting up a new client, to put those in there. If they’re not already mapping them, that at
least they have the option now, so going forward if they want to map them, they’re in there. It’s
really up to the client.
What do you think is the main goal for a risk assessment?
The main goal for a risk assessment, is to really to be able to stand back and from a very high
level to be able to focus in on areas that are of higher risk. So that way we can then perform and
audit during that particular year that will further assess those risks. In that audit, ultimately, we
want to be able to provide value and add value to the organization. Because we are finding
information that could potentially impact the company and we’re finding things before they
actually happen or become even worse. That way we can recommend ways to enhance if it’s a
process, operational or if we found misstatements on the financial or something of that nature.
They can correct it. Or make their process better or more efficient. I think the overall goal of the
risk assessment is really just to be able to on a global level look at your company and to be able
to say we need to perform these audits this year, because of these high level risks. Need to make
sure that controls are in place to mitigate these risks. We’re going to perform testing to ensure
that those controls are in place. And we will recommend to management and whoever is
responsible for those controls to remediate what we found or make it better or put in another
process or add a new control to help. So that way that risk, the likelihood of that risk actually
occurring could be significantly reduced.
Risk assessment is essential, especially in the audit planning. Do you think there could be another
way to ensure that controls are in place? Do you think you can do that without a risk
assessment?
Even if they’re not doing an annual risk assessment, you still need to be doing a risk assessment
at the project level. There are two elements to a risk assessment; you have your audit planning,
annual planning risk assessment, but then you also have once you perform the audit, before you
even develop where you need to test and what controls you need to test, you would then do a
more detailed assessment. So if a client is not doing an annual risk assessment, they should be
doing a project level risk assessment. In that way they will know at least what they’re testing and
96
they know the controls which should be there and they know the risks and they can actually test
and perform their audit based on that specific assessment of that area on which they’re about to
perform an audit. If you don’t do an assessment, you could be wasting your time. Maybe not
always, but you could be. Are there people out there who do an audit without a risk assessment?
For sure there are. Are they missing things that should be audited? Most likely. A risk
assessment, even at the project level… The whole foundation of internal auditing is to add value
to your organization and if you’re not doing some form of assessment, how do you know that
you are testing and you are ensuring that the most impacting or critical controls are in place. And
how are you identifying what controls should be in place? There are some clients that don’t
document per se, but they have been with the company for decades, and they can do an
assessment in their heads and know it. They just know because of the knowledge that they store.
So maybe they’re not documenting an assessment, but that doesn’t mean they’re not doing one.
They just form the procedure in a way that they were testing a control and making sure it’s in
place. And when they’re documenting a finding, then they’re documenting a risk associated to
what if management does not fix this. So even though you’re not doing a full assessment, some
people just document it, within their testing set. And then they document what the risk is and
control failure at their finding level.
How does a risk assessment for a process work? Or can you give me an example?
After audit management performed a high level annual risk assessment, and they have done
interviews with executive management, they found out what keeps them up at night. They know
what corporate strategies are for that year. They’ve identified high and impactful risks. And
they’ve assessed the likelihood of risk happening. They could bring in controls at that time or
not. But even if they didn’t and you just have the risk and they determine we need to do an audit
on X, Y and Z this year. So if an individual as an auditor is working on audit X, based on the
high level risk assessment, really before they even perform any of the testing or develop what
they’re going to test or what controls they’re going to test, they need to dig a little bit deeper.
Because the annual risk assessment looks from a high level. When they get down to their audit,
audit X, they need to understand they’re going to do an audit of payroll. I have these high level
high impactful risks, that my audit management has assessed and scored, based on interviews
with the VP of payroll. So when I get to a particular project, I want to home into a scope. We
cannot assure and test absolutely everything. One: there’s time constraints and we have to scope,
in this particular case, the payroll function. So we would start by gathering background. Obtain
policies and procedures that relate to that function. Interviewing management. Interviewing
payroll processors. Doing walk trough’s of the payroll process. And documenting all that along
97
the way. And finding out from a granular level what potential risks are there in regard to the
detailed walk through in process of payroll and we document those risks. Then during our
interviews, we would determine what controls are in place and document those. And we would
also document which controls we think should be in place. That maybe we just don’t see or not
know yet. And once you have that assessment complete for that individual payroll process, than
those risks and controls are much more granular then the risks and controls that audit
management assessed from a very high level. So once you have that, you can determine which of
these controls based on these risks, should we test. Then you build your audit program from
that. Your procedures should be structured in a way that this is what I’m testing; A, B and C as it
relates to this control. I’m going to test to make sure that this control is in place. That could be a
combination of if the policy is there, check, is the policy accurate, is it up to date. Then you can
go down to granular, samples of 25 payroll employees. Are we paying out to people who no
longer work here? Because we forgot to deactivate them. Are they marked as not employed
anymore. Those test app would help us ensuring and test on the controls that should be in place
to mitigate that risk. And if we find anything along the way, we can then document a finding and
produce it to management. And say during our testing of A, B and C we have found the
following and we recommend that you do this. Then we are required that we follow up on that.
Even if they say they will start doing it three months from now. We need to come back three
months later and test to make sure that they did in fact implement some kind of control or fixed
what we found, all to add value to the company and mitigate the risk to ensure something
doesn’t go drastically wrong. Some findings are much bigger than others. All in all, we should be
adding value, not only from a grand scale, but also on a small process. Anything that can help
make the company doing things from an operational point view more efficient to financials, to
safety and security, to IT and addressing general controls over software and databases and
applications that people can have access to. There’s risk in anything you touch and do in your
job as an employee within your business.
That’s a good description on how the risk assessment on detail level rolls forward from the
annual planning. Do you think that the type of system a company uses has an impact on the risk
assessment?
I think so. Using your system to document your risk assessment is definitely much more efficient
then manually documenting. TeamMate offers not only the utility to do a risk assessment, you
can also receive the input of people outside audit. So if you don’t know or if you want further
information you want the VP of human resources to be able to assess the risks that you have
identified, or allow him or her to identify other risks. You can publish assessments to those
98
individuals. So that way they can assess their area and provide feedback and that can help and aid
in the audit planning process. The results of those different assessments that people outside of
the audit department submitted could be directly input into the current plan audit assessment
that audit management is working on. We also have within TeamMate a survey functionality, so
if you wanted to publish a survey to the executives, about what keeps them up at night, or based
on what are their strategies or goals of different areas of this year, you could create a survey and
send it out every year. And therefore you can adjust your risks each year based on the survey
results. All of this could be done and is much more efficient and can give you a lot more accurate
information and less time consuming. If you were to do all that on paper or using Excel, then
you’re going to have to setup interviews. You should do a face to face with executives and a risk
assessment anyway, but if you wanted to do also surveys or maybe the next level for senior
management, surveys would be a great option. Also self assessments for your audit assessment to
help to get that feedback directly without running around and gathering it and planning a bunch
of meetings. They can do that on their own time and they can complete it. That makes it much
more time efficient. And you’re going to get the results much quicker. And you will be able to
get look forward out of the system. That reports and heat maps that you’re going to have to do
manually, using Excel, whereas a system such as TeamMate would be able to provide those
reports and heat maps by a click on a button.
How do you think the risk assessment or the audit planning is impacted by the level of
integration of the audit subject? So if you are auditing an organization using an ERP system or an
organization has multiple data entry, how does that impact the audit planning or the risk
assessment?
If you’re working with a duplicate effort for ERM or the risk management group or the
compliance group, most of the time you’re building your assessment to encompass those. So you
could send self assessments out to those particular areas and have them identify their risks and
put them into the system. I have some times seen share the system, so duplicate work isn’t done.
Using some sort of integration is important, so that way there isn’t inefficiencies you can
collaborate and work together. I think that it is important.
So using TeamMate you can ensure that the risk assessment is still done efficiently because the
information is shared?
It also depends on who has access to the database and how you set it up. So if you integrate with
another system, you can run reports, like for example CM. So if the compliance team is using the
CM module and the audit department is using AM, you can run reports on both and you would
be able to see what compliance found or what audit found.
99
Let me rephrase the question a little bit. If for example you have a purchase to pay process in an
organization and if it’s a highly integrated environment, like using an ERP system, then the
person who is actually placing the purchase order, then nobody in accounting has to do that
much. Only when the invoice is coming in, they have to ensure that it is signed off properly.
Whereas with lower integration of system, then someone might place a purchase order, but
someone in accounting still has to enter a purchase order in the accounting system or record the
invoice in an accounting system. If you take a look at the process with a single point of data
entry or multiple points of data entry. How do you think that impacts the risk assessment and
the audit planning?
So you’re saying that basically having different departments speak to each other and ensure that
there’s not duplicate work and how that would impact the risk assessment that audit performs?
Either there’s a single point of data entry or multiple points of data entry.
I don’t know from a risk assessment standpoint. Maybe I’m not looking at it entirely deep
enough. From a risk assessment stand point, auditing is supposed to be independent, so
independent evaluation of what is going on. So while it’s helpful to do interviews and ask what is
going on, I don’t know if you want it to take into a direct integration of different areas per se
from a detailed level. You really want audit to offer that. Things maybe spoken differently, I
mean, if I’m an audit manager and I’m performing a risk assessment and I want to meet with the
VP of let’s say payroll. And I want to ask them some questions or send them a self assessment,
with high level risks that I’ve identified, they may be more apt to respond to me in a way that
could be differently than if they’d already done a risk assessment or something like this internally
with another department that documented things differently. Because maybe that impacted their
position or job. As when an audit comes, it’s meant to be an independent evaluation of what is
going on in your company. And they may be more apt to explain or document or provide
information to you that differently. I don’t think that you should rely on what has already been
done in other systems, but you should also not solely rely on what you have found. There should
be at some point some communication. I don’t know how to do that. A best method to do that.
From a system standpoint, it would be nice to go in and see things that have been done, but
that’s part of your evaluation for your risk assessment. Most auditors have access to every system
in their company, so they would be able to go in and run those reports and take in that
information to see what is going on. But I don’t know if you would want a fully integrated. It
would need some kind of communication, but a fully integrated tool from an independent
standpoint, audit software should not be integrated with others, but they should be able to get
information from other systems. And they all should have access. I think this is a very grey area,
100
which is currently being explored from the software realm. How much is too much? And where
do we cross the line with independence? But I think you should definitely take into consideration
that has been done by other departments and using that, but not solely rely on that. There should
be at least a one on one communication or self assessment, to know what is going on. And that
should be re-evaluated every year on an annual basis. But it really is the auditors responsibility to
identify risks, no matter how they required it.
To explain a little more about my research: what I’m researching is how much integrated
systems, such as ERP systems, have an impact on audit planning. And if the centralization or
decentralization of data entry how that has an impact on the risk assessment. So for example if at
a bank, which has various branches, and everybody works on the same system. So if someone in
a bank in New York is making a data entry for a loan for example and the head office might be
in Texas, then how would that impact the risk assessment and the audit planning. Do you want
to comment on that?
A good research topic. Like I said, from an auditing standpoint, we need to have independence.
And it comes to data entry, what you’re saying. An entry from an employee. For example the
loan entry; to somehow flag if a loan entry is above a certain amount, should that trigger and
audit to make sure that proper protocol was handled and so forth. How would audit know if a
loan was just approved, if there is a threshold of that approval of the loan, how can audit be
aware at real time, rather than waiting till the audit is going to take place or the assessment and
then determining and sampling. Because the testing will come where you could sample loans that
were approved over a certain amount and were protocol. But it will be a sample, we cannot give
a 100% assurance. So to have something like that integrated, where would you draw the line
though. This is a more aggressive way for audit to ensure that all the risks are being mitigated.
Currently I’m more thinking into the direction if you go to one of the branches as an auditor, to
do an annual audit or an audit because the last audit was five years ago and is due, how would
your detailed risk assessment look like? How would it be impacted by this type of integration?
But no the audit system, but the systems within the financial reporting.
I don’t know and maybe this is the wrong answer, but maybe not the risk assessment would be
impact, but more the procedures steps and the efficiency. Because in the risk assessment, you’re
identifying the risks. And the risks are the risks. From branch to branch most likely they all have
the same risks, unless there are certain aspects of it. Maybe one has an ATM and the other
doesn’t. Maybe one has a safety deposit box that holds more accounts then another. So certain
risks might not be at that branch, but are at others. Ultimately identifying the risks, I don’t know
if having an integrated system would impact that as much as it would how you test it and the
101
data you get to perform your testing. I think that is were the integration would become more
beneficial. When you start to identify your controls to mitigate those risks, you’re then going to
be performing an identifying procedures to test those controls. And when you start testing, if
you can easily report on how many different loans for a branch are approved over a certain
amount that not have two signatures or something like that. If you could have some sort of
integrated system that would help you identify your sample and dump it immediately into a
spreadsheet and capture the work paper into the audit management tool, I think that is where
the efficiency is going to be from a time perspective. As for really performing your risk
assessment, and having some kind of integrated system. I would see more efficiency on the
testing and on the data getting from some sort of integrated system.
Like you said before, where you draw the line, is a very good point. Because as an auditor you
cannot be on top of the processes all the time.
As far as the responsibility on to identify risk, we cannot assure that 100% of those risks all the
time are mitigated. We can only come in and identify the risk and then to perform selected
sampling to ensure that that sample is working by design and operating effectively and efficient.
It’s making good sense what you’re saying. I really hear a lot of things which I can and will use in
my research. As I’m still pinpointing my research, it might be that I will have additional
questions. Would it be ok to contact again?
Yes, just send me a mail and I’ll be more than happy to help you.
102
8.7 Appendix VII: Interview #5
I already had interviews with interviewee #1 and interviewee #4. You work with them, correct?
Yes, I’m in the same team. I’m a senior consultant with the TeamMate group.
They mentioned your name and told me that you could explain me a bit more about TeamMate
Analytics.
I do have some experience as far as using data analytics and since we acquired the Analytics, then
I’ve been one of the individuals that have been kind of the lead on, not so much on the
deployment, but how we’re going to provide that consultation service to clients. Now we’re into
the procedure that the application we do have as well as the theory behind using various proxies.
You just mentioned that you are a senior consultant of TeamMate. Can you explain a little bit
more about your role in the organization?
Being a senior consultant, it’s pretty much once an assignment has been given, a client has
procured our application and desired to have training and configuration, so part of my role is to
basically deliver services. It’s kind of a wide spectrum of duties. Sometimes it will just have to do
with straight training. Sometimes they want us to guide them on how to use the application. It
really depends on how well developed the client environment is before we can determine the
service level that we will need to give. Being a senior consultant we wear many different hats and
we have to rely on a ton of experience that we’ve had in the past in order to, not only get the
client on the best pack, but also to show them best practices. Just a wide range of duties there.
So you are also exposed to teammate settings as well, that customers might or should use?
Yes, initially it starts with configuration and really try to find out what they want and how it
would be applied within the application.
How long have you been at the organization?
I’ve been with TeamMate a little over two years now.
And what did you do before?
I was in practice. I have over 15 years of audit experience. Some of that has been with state and
local government. Some of that has been as employed by Deloitte. And I also worked for a
regional accounting firm here in the States. Most of my background has to deal with auditing and
there’s a bit of internal audit in there as well. I started with state and local government, financial
statement driven and then went into external audit and in the latter part went into internal audit.
Do you see a big difference between internal and external audit?
In my opinion internal audit is only strong as management allows them to be. If internal audit is
seen as a necessary evil in the organization, it will not get the support that is needed. With
external audit there’s communication, but it’s more or less, I don’t want to say a one way street,
103
but it’s a little bit more focused with internal has a lot of collaboration. I think the ability to have
soft skills. Being able to communicate that information and communicated in the right tone. So
there are some differences that I see there.
For my research both internal and external audit can be used. But now I can set a bit of what the
differences are. Have you worked with other audit planning programs such as TeamMate?
Within public accounting, working for Deloitte, we had an internally developed application,
which was all electronic, except for the reporting feature. Since then, coming out of public and
getting into state and local government, that’s when I started to research what the best electronic
work paper solution would be and I decided to recommend to an agency that they use
TeamMate. So most of my experience has either been, when I started off of course was kind of a
manual working paper, electronic work papers and then moved into an internally developed
application. When I was at a regional firm, I think they used an application that was similar to
case wear. And of course I used TeamMate. So I have had four different types of usage there.
Do you see any big differences between the various programs?
TeamMate seems to be the most robust, trying to encompass the entire audit process. The other
items seem to be right at one that they don’t have the ability to encompass for example the risk
assessment process and things like that. Doing the tracking. So there are some differences that I
see. It also making in on the focus. Some of the other items that I used, were strictly financial
statement focused. TeamMate is primarily brewed into the internal audit profession or
procedure. TeamMate accommodates the financial statement features as well.
So TeamMate has a broader scope than others?
Yes.
Now I’d like to move a bit more into the direction of auditing. What do you think is the main
goal of an audit?
By the very definition of audit… It’s depending on the type. We have a financial statement audit,
that has the definition by itself that the financial statements are free of material misstatements. If
you’re dealing with a performance audit, we’re not really dealing with materiality or anything like
that, we’re making sure that things are performing right as intended. When you’re dealing with
compliance, that’s kind of self-explanatory, we need to review a subset of information and
making sure that things are progressing within the compliance of a particular agreement. So the
term audit in general is that things are on the up and up or functioning correctly. But when you
get into the various activities about what you’re trying to accomplish. Financial statement audits
are ensuring free of material misstatement. People want to say that all the numbers are correct
and that’s not necessarily the case. It’s saying that they are materially correct. You can have a very
104
large balance sheet or income statement and that can be off by 50 thousand dollars or something
like that and that can be seen as a huge discrepancy. In general terms audit means to make sure
that things are in agreement, making sure that things are functioning correctly, but you deal with
the various types of audit. That’s where you can see a little bit of a difference in mindset.
Does that also mean that the audit process is different for the different types?
For the audit process the spectrum remains the same. Having an initial planning stage and there
may be some risk assessment areas and then you jump into possibly field work or substandard
procedures and then you get the reporting and wrap up. There are three main areas and possibly
a fourth, depending on the departments and how they see things. They might break that first
stage into two different parts.
So in general the audit process itself at a high level is the same, no matter what type of audit you
do, but if you go to a specific audit, then the audit type does have an impact.
Yes.
What can you tell me about the audit planning process?
What I’ve seen throughout my experience is that the planning process is meant in order to make
sure that your audit is meeting its objectives. There’s a consideration of your time budget, the
team members, communication among the team members, very low level things. And then it
gets into things like acquiring narratives, reviewing prior year work papers, doing research,
conducting fraud discretions. It’s necessary and it happens with the corporate accounting
scandals that took place in the early 2000’s. With the release of SAS 99, which had to do with the
consideration of fraud within a financial statement audit. This agreement started of where it was
most appropriate that those procedures had infiltrated or integrated into internal auditors, where
they could use as consideration of fraud. Doing a fraud brain storming session. The planning
process starts off with the very high level steps, pretty straight forward steps and then you get
into the consideration of fraud. Then just plan the process going forward to make sure that it’s
the most efficient audit and also lowers the audit risk.
What do you mean by audit risk?
As far as your overall audit risk, it has a couple of components. You have your sampling risk.
You have your inherent risk. And things like that. Any time you’re not looking at 100% of the
items, because an audit is meant to save at things or performing well or as intended. An audit
may be taking a look at a snapshot. It’s not looking at 100% of the function. So because of that
there is a risk. A risk that errors would not be detected. There’s a risk that if you’ve giving an
opinion on it, for the audit as a whole, there’s a risk that that this opinion comes to an incorrect
conclusion. As a result, the planning process is meant to adequately plan for all of the possible
105
scenario’s, all of the items that are potentially high risk, and identifying those and making sure
that those have been adequately tested and included all. So the planning process plays a very
major role in trying to identify all of those things. If you have a substandard planning process,
there’s a possibility that you could overlook a risk that is deemed to be very high that is not
tested and not had an adequate scope of procedures to accommodate that risk. The planning
process is mid to ensure that we have everything, make sure that you cover everything in your
procedures.
In what part of the audit planning, you make sure that you’re not missing out on any of those
risks? Or how do you do that in your planning?
You have your research. If this is a performance audit, which is internal by nature. You do all of
your research. First your research has to do with obtaining policies and procedures. And then
you proceed with if there are any documents associated with that particular process that you’re
looking at. Then it also has to do with inquiry and observation. You talk to individuals that are
responsible for a process or for a program or something like that. Then you may obtain a
narrative that clearly documents everything that they’re doing. And then you may map it to a
control questionnaire. So there may be some things that there’s tools or that you have either
through accounting research manager, which is a very good tool that it’s a Wolters Kluwer site,
that gives you guidance and a more rotated pronouncements that help you look at various
processes in auditing techniques and you may use those as well. There is to map what should be
covered of a particular process. That help you with use correct the timing process or help to plan
those high risk guidance and making sure that you’re not missing something.
So what you just described are the steps before performing a risk assessment, right?
Yes, pretty much. Making sure that you have covered all of your risks and then making sure that
there is a control in place to mitigate that risk. And if there is not a control in place to mitigate
that risk, then you ask to management about it. It may be that they accept the risk here.
Because they expect a minimal impact, or something like that?
Yes.
And this type of risk assessment, is that performed at high level or more at detailed level or
both?
There’s both. For an organization there’s usually a high level risk assessment done and usually
those are tied into enterprise risk management. Those same high level risks, could be applicable
at a project level risk assessment as well. Those project level risks are more granular in nature and
very specific to what you’re auditing. Most of the times I would say yes, hyponisations are
conducting into two types of assessments.
106
So one at a high level, maybe at an annual basis, and on detailed level, which is more on the
auditable subject itself?
Yes.
You already explained a little bit about the impact of a risk assessment on the audit planning.
Can you elaborate a bit more on that?
As far as the risk assessment and what affect it will have?
Yes, or how it is used in audit planning.
If you go to your risk assessment and you identify all of the applicable risks and you find out that
controls are in place to mitigate all of those risks. You pretty much say that the control
environment of an organization is in a very good place. It will cause you to feel a little bit better
about the level of substandard procedures that you have to perform. If it’s a first time you’re
going to test it. You’re going to document the controls and are going to test them. Based on the
results you will evaluate whether or not a control is functioning correctly and whether you can
rely on that control. If it is an audit that has been performed before. When you’re going to do
that risk assessment, you know that those controls are in place. In the prior year those controls
worked wonderfully, you may have the ability to not just test as much. So the amount of work
itself can change based on the result of that risk assessment. If that risk assessment is performed
and you deem the environment to be very high risk and that is going to change the scope of your
substandard procedures.
I notice that you use terms like control environment and risk assessments, which I recognize
from the COSO model. Do you actually use the COSO model?
Yes, it’s a very good place to start. Most organizations are in the process of rolling out the new
framework and dealing with the 17 principles and the 87 points of focus. That is a very good way
to identify your control environment or identifying the things that is applicable within your
organization. I think that is a very good start. In the past, when I was in practice, we talked about
COSO and things like that, it was something that was not used very widely. What I’ve seen is
that there has been a huge shift within the last five years, I’d say. Where organizations have been
really using that in order to kind of question or kind of trying to make sure that they have things
in place that all of those items that have been presented within COSO. That’s probably one of
the biggest changes that I’ve seen within the past seven years. I thought it was kind of like a state
or local government type of thing. Now it’s really kicking off and it’s pleasant to see.
And then tools like TeamMate can help out a lot, because to link a company to COSO, what I’ve
seen in TeamMate is actually very helpful.
107
Yes. I just worked with a client a few months ago and they wanted the 17 principles at least,
being able to identify which controls, which principles they fell… They had one time that the
actually wanted to go as granular as the 87 points of focus. And they’re using that as a category,
an option and making sure to map that. Then at the end of the year they can run reports in our
reporting module to see if all of those points of focus had been covered or touched throughout
the organization. It’s really good to see that our tool is being used to facilitate that.
Coming back a little bit more to the risk assessment; how does it work exactly? Do you work
with risk levels of certain items?
Pretty much so. As far as you identify risk. Let’s look at this from a financial statement audit.
The financial statement audit you deal with cycles, like accounts payables. You identify the risks
for accounts payable. At accounts payable one risk could be that failure to improper payments to
vendors. That’s your risk. Another risk is improper recognition in the correct period.
And how would that lead to a high level of risk?
Some of it may be when you valuate the control. Whether or not that control is present. Some of
it may be based on the account balance. Like when it has a very material balance. Also like the
overall structure of the account. If there’s improper segregation of duties, which would probably
be a risk as well. There are certain factors that come into play there, whether or not it’s deemed
to be high risk. For instance on a financial statement audit, cash is seen as a very high risk area.
And the reason why is, because it’s very liquid. There are certain controls that need to be in place
there as far as proper segregation of duties and things like that. It kind of pins upon the nature of
the area that you’re looking at. Whether or not it’s going to be high risk there. If things are going
to be high risks even though controls are in place. Things that you will have to look at,
regardless, because it’s just the very nature of that area. Some of them, you can actually look at
them and say what is present. If it’s deemed to be high, moderate or low. Some of them are
going to high, regardless, just because of the nature of the area.
So basically you analyze the various processes and take a look what controls are in place?
Yes.
And based on that you can say whether it has a higher or lower level of risk involved?
Correct.
And with a higher level, there’s more reason to put it in the audit planning?
Yes.
And there are some areas which you need to audit anyway?
Yes, correct.
108
To get into a completely different direction: the other part of my research is about ERP systems.
What do you know about ERP systems?
I’m probably a bit less familiar with that.
Let me rephrase it a little bit then. My research is about the level of integrated systems, like an
ERP system, how that is impacting risk assessment for the audit planning.
Let’s look at this from a manufacturing. Those types of systems, as far as just in time, related to
product management, those kind of things when it comes to audit, what you’re considering
there. That’s probably one component, one that relates to audit. You do have a consideration of
IT that’s part of the planning process. Of course you would evaluate whether or not that system
is giving or providing reliable information. At that point, dealing with those types of things you
may contract an IT professional to perform various tests, just to make sure that you can rely on
the information that is within that particular application. I take it strictly from an audit
perspective and how that will affect audit or any risk assessment module. It would come pretty
much under the same scope of saying reviews as if anything else in the organization when
performing an audit.
That’s very much in line with what other people have told me. I’m just thinking about how to
rephrase the question to get it from a slightly different angle. I’m not looking for different
answers, I’m just making sure that I get the information which I get for my research. If systems
are highly integrated, then multiple data entry is being eliminated. Because a data entry is done at
one point, but that might be done by non-financials and still whatever they put in will have an
impact on the financial statement in the end. How do you see that from and auditor point of
view?
From an auditors point of view it’s kind of looking at that. First of all you want to make sure that
you have the expertise to evaluation what’s going on. Making sure that you’re documenting how
things are going. Because the input and things like that you have it entered in at one point. You
probably want to look at that, but you also want to follow it all the way through. That’s where
that walk through of information, kind of doing it from start to finish. Being able to understand
of how that information is being processed. It’s just make it where you change your focus. It
makes the walk through portion very critical.
So basically you go through the process itself? You take a look at the steps of the process.
Yes.
What do you think is the impact on the data quality, because it’s entered decentralized, so by
different parts of the entity and that it’s entered by non-financials?
109
I think there is a potential for degradation of the quality. Application controls become
paramount. If you can put in information that’s not correct or not in conformity with what is
expected, then that’s pretty much your system weakness. It’s just makes it where you would
consider all of those active continues, you would consider those to be risks and how those risks
are mitigated.
Then you come back to your risk assessment basically? You make a risk assessment on the
process.
Yes.
Let’s say there’s another entity which does not have a highly integrated system, but uses
interfaces between different systems and databases. How would you deal with that from an
auditor point of view?
Again, kind of going back to where consider in things and evaluate whether or not the proper
controls are in place. Because that would be considered a risk as well. All of those things need to
be evaluated. If there is anticipated or if there are any vulnerabilities identified, then those need
to be considered throughout the course of your work.
Do you think there’s a difference in the evaluation of the data in an ERP environment or a non-
ERP environment? So if the data is correct or complete.
I think if it’s highly integrated it will change the level. That evaluation would be changing
depending on what you would see there.
Let’s take a look at it from a different point of view. If you use TeamMate Analytics on an area
which is highly integrated, would that be different from using TeamMate Analytics on an area
that’s not highly integrated?
If it’s highly integrated then the possibility of you getting information from what source,
increased tremendously. Because if it’s not integrated then you will have to go to different
sources to get that information. I think the ease to get information in the form that you want. I
think an integrated system would be preferred, because there’s the least amount of data
preparation involved. I guess it’s more of a convenience for the individuals who will have to do
the testing or the review matter, than the actual entity itself. The systems could be fine and
working accordingly, it’s just that you have the segregation of your information. That’s where
expertise comes into play. Being able to say exactly what you’re wanting to get out of the system
or systems and moving forward from there.
What you’re saying is that because all the information is coming from one system, with one data
entry point, that makes it easier from an auditor point of view, because you don’t have to
compare the different data, which is the same, because it’s only one point of data.
110
Right.
That would mean that data being entered by non-financials, would then be not that important,
provided that the process is setup properly in the integrated system.
Yes. There are certain protocols to ensure that information is entered in such a way that it’s kind
of goof proof. That kind of helps the process. It’s coming back to what controls are in place to
mitigate that risk. You look at the access point and evaluate who has access to those points and
who can enter it in and evaluate the individuals as well and then go from there. Saying, we’ve
evaluated this and we know that this is in place to mitigate this and you can go from there. You
may see some variation in response to when it comes to this particular issue. My experience is to
look at the overall system, looking at the access points and then evaluating the type of
information that could be put in there. And how does that translate into the overall information
that I’m trying to get out of the system. There are probably various ways in how to approach
that.
And to come back to audit planning. Also thinking about the various physical locations of data
entry not being centralized. How would that impact the audit planning? In the example of
inventory, the data entry might be done in a different location as the accounting is being done.
How would that impact the audit planning?
Pretty much going back to the same thing. If something is being done away from accounting,
you first evaluate how the information is translated. What system is being used for that? You’re
going to evaluate the system and then you’re also going to make sure that you gain an
understanding for that process. If you order something, how is it that you put information in
there? Then you look in how that information translates all the way through to accounting. Like
if you put in a quantity of one item that you need in the warehouse. That should translate to a
purchase order number, than follow it all the way through. Once you have that purchase order
number you track it to the purchasing department to see if that purchase order number was
obtained. Going through how they procured it. Then take a look how that item is received. How
it is logged into inventory. How all of that translates. One of those items actually hits, from a
financial statement perspective, when you receive the invoice it will hit accounts payables. When
the actual invoice is paid. When the item is actually received, it will hit the inventory account. I’ll
walk through on how those various items are triggered throughout the organization.
Like you described before, it’s going through the process. And how would that be different in
and organization which does not have an integrated system?
If it’s not an integrated system, there may be a few steps extra. It would be manual. As far as
quantities there would be a request and the request would go to another individual. You would
111
be looking for an approval. Then the approval goes to procurement. It’s just that in a different
way things would get processed. It will be more manual in nature. And with that you would
expect more a difference in time frame on how quickly that item is processed. With integrated
solutions where they push the quantity and they get the item within two days. If it’s not
integrated or centralized then it has a lot of bureaucracy in place that is really meant for controls.
It would be clearly documented as you would see with a centralized environment.
So would that higher the level of risk?
Not necessarily it would be considered a higher risk. There are just more points for you to look
at. I think it would probably have the same risk level. It’s just that you would have to account for
a bit of extra time to actually verify the process.
So the audit planning will be a bit wider as far as the time frame is concerned?
Yes. Risk does not necessarily translate to… It’s more the nature of the process or the nature of
the environment. That doesn’t necessarily translate to risk. We just need to account for that.
We have covered a great deal of the questions which I invented up front of the interview. Are
there any things you say that I should think about as well?
There might be a bunch of things. You’re now looking at risk assessment and audit planning.
When you’re looking at phases in the time frame. In the past it has been where your planning
face would account for 20 – 25% of your budget for the entire audit. And then maybe 60 – 65 %
would be addressed for field work. And 15% related to reporting and wrap up. I’ve always had
the mindset that now with the way that the environment is, with all the tools and all the extra
applications, I would expect that planning would probably consist of 40% of your budget and
your actual field work or testing work would be less than your planning phase. And reporting
and wrap up would be the same. That’s a huge shift of mind set change for some of your
traditional audit shops. It kind of goes to the premises that a well thought out, a well-planned
audit, should reduce the amount of testing if done properly. A lot of clients say that’s what I
would like to get to. That would be ideal. If the planning is going to be 40% and the actual
testing is going to be equal to or less to the planning phase. It’s not so much saying that you’re
going to put a significant amount of time more into planning and risk assessment. It’s not saying
that. It’s saying that if you put more time into adequately plan, it’s going to reduce the amount of
time that you spend, that you’re saving doing actual testing by that much that it almost equals
then the time that you spend in planning. It’s huge. This kind of occurred from an internal audit
perspective, because there are so many budget leaks. When it comes to internal audit there are so
many things that are not really identified until you actually start field work. So many surprises
112
that happen throughout the course of the work. That’s kind of occurring with the way things are
now. Having the actual tools to accommodate a well-planned audit could help out tremendously.
That’s really interesting. You’re saying that the planning is not really getting bigger, but the actual
testing is getting more limited. Why is that?
It’s not more limited. It’s more that if you take the time to adequately plan and you do your
testing. In the past most auditors became like forensic auditors. They were testing almost 100%
of the population. When you’re dealing with analytical procedures and being able to rely on those
results, you’ll be able to test a 100%. Using an analytical process. I’m not talking about
comparison from this year to last year, but digging much deeper. Looking for information, using
statistical methods. Like if there is a normal distribution to evaluate information. Being able to
look at outliers. Things like that. That was something that was done say about 7 years ago. When
I was at Deloitte, we did perform statistical procedures. In internal audit people thought it was all
theoretical and it provided no value whatsoever, but that’s changing. People are becoming more
aware of analytical procedures and being able to administer and then save time and being able to
look at more things. And just become more efficient in your process. It’s not so much as
becoming limited, it’s more that you will be able to cover more, because you’re changing up the
procedures. You’re not being as traditional as it was maybe a few years ago. It’s just a normal
maturation of the auditor mind set. Being comfortable with the results that have been provided.
For example benford analysis. Are you familiar with benford at all?
No, I’m not.
Well, benford analysis, is a law. It was created by an individual, Frank Benford. He was an
engineer at GE in the early 1900’s. During his work he determined that numbers in a naturally
occurring environment they have a probability of a certain percentage of occurring. The numbers
like 1 or 2 are occurring most within a natural population. The numbers like 8 and 9 they occur
the least amount of times. Every time you see a natural population of numbers and it would have
to be a very large set of numbers, most of the time it will adhere to benfords law. I think there
was a professor at West Virginia University, his name was Marc McGreed, he took this particular
law and said it could be used for auditing techniques to determine fraud. If you’re looking at
fraud, most of the time it will have to be in that triangle of opportunity, rationalization and I
forgot what the third one is. Most of the time when an individual makes a change to something,
they have a certain set of numbers that they always use. Benfords law it basically it’s basically
taking, if you have a general ledger, 9 times out of 10 if there are no boundaries for procurement
or something like that, most of the time a general ledger will adhere to benfords law. And if it
does not, you would think there is a lot of manipulation. This is a test which I’ve used countless
113
times. That’s saving time in itself. In the past there was a lot of manual review in order to say
let’s look at things which are kind of suspect. And now it’s allowing them to see within a matter
of seconds.
You really gave me a lot of information which I can and will use in my research. Probably I will
learn a lot more in the upcoming interviews, which may result in additional questions. Would it
be ok to come back to you, if I have additional questions?
Yes, absolutely.
Then I want to thank you for now.
114
8.8 Appendix VIII: Interview #6
As the invitation mentioned, I’m doing a research about audit planning and as TeamMate is
about THE tool for audit planning, is how I came to the idea to interview TeamMate developers,
consultants and some users as well. What can you tell me about your role in the organization?
I’m the director of product management for TeamMate. It’s a role which I’m in for 4 years now.
I was a senior business analyst. I’ve been with TeamMate now for 15 years. I started out as a
sales training implementation capacity and very quickly moved into business analysis. So that’s
what I’ve done at TeamMate. Prior to that I did work as a senior manager for Price Waterhouse
Coopers in their internal audit function for 4 years. My career started as an internal auditor.
Working my way through as financial auditor to operational auditing to IT auditing.
So you have abroad experience in internal audit?
Yes, I kind of live eat and breathe it, since the beginning of my career. You might have talked to
other people who have a different opinion of how internal audit should work and how it works
in their organization, but probably the most interesting part of the profession for me is that
we’re all supposed to follow the same set of standards, but because those standards are
somewhat loosely written, they’re very much open for interpretation. We’re servicing so many
customers internationally, you think we would see the differences in how the rules are
interpreted and what customers want to do or are prepared to do. We also sell TeamMate to
government agencies as well, so their interpretation of an audit plan is very much dictated. So
they treat it more as a monitoring tool and not so much as a planning tool. If you go to a
corporation, audit planning or audit plans in general start out as a planning tool. When they
didn’t monitor these, they would go to the content, but they put more emphasis on the planning
and less on the end like the reporting part. Government is the complete opposite.
Why do you think it’s completely different for a government?
I certainly know that in both Canada and the US and then some of the stronger English speaking
countries, like the UK, Australia, New Zealand, there is a body whether it’s called congress, or
some other body, that very much dictates, the entities to be audited or reviewed and make up the
audit cycle for them. That generally makes up their annual federal budget time. And they follow
that. They’re told what to go audit and they just have to keep on reporting back on that quarterly
or semiannual basis to that same body on the progress. What have we accomplished? And what
have we learned? And the work that has been completed. There is a small movement in a few
agencies where part of their audit plan is a little bit more risk based auditing. That’s still very
experimental.
Do you think that corporate organizations are a lot more risk assessment driven?
115
At least theoretically. Years ago, probably about 10 years ago, when the COSO framework really
took off, and it was sponsored by all sorts of public accounting firms, big organizations had a
way to better understand business objectives and risks and controls in companies. Identify if the
controls are effective for the risks that are in place. Because business objective may say that we’re
doing things that we shouldn’t do. It was presented as something that everybody has to do. And
being in the internal audit business as I have, you see something I call the hypehype cycle on
some of these things. And there was certainly a hypehype cycle on the COSO framework. We
built our first version of our risk assessment tool, which was closely aligned with the COSO
framework. What we have found though in reality is that most internal audit departments use the
risk assessment. They very seldom ever want to measure or monitor or even capture anything
about business objective. To them that’s something that they will look at when they get to
actually auditing an entity and when they’re doing their detailed engagement planning, but at a
higher level, when they’re doing their risk assessment, they very seldom include that information.
We had to create workarounds in our tools to try to help to accommodate that. And they tend to
assess what we call strategic risks. Not detailed risks. So if you follow the true COSO framework,
for each business objective you would have identified detailed risks that could prevent you from
achieving that objective or risks in if someone tries to overachieve that objective, it’s the positive
and negative sides of those, and from that identify controls and you put risk weightings onto
those specific detailed risks. But if you take the objectives out of that equation, you kind of find
what audit departments do. It’s risk assess the same set of what we call strategic risks across the
entire organization. So as an example they will have a risk they call strategic risks, operational
risks, financial risks, financial reporting risks, compliance risks. Generally there are between 5 or
10 of these. They will find each auditable entity, parts of the organization which they feel they
can audit in one go, and risk at that level. And based on that risk assessment they will decide in
the company where they will go and review. And it’s not until they get into the engagement
planning phase. Then they will look at that financial reporting risk and try to determine the
detailed risks in that business unit. So they take that out of the higher level audit planning
process and push that down to the engagement planning. Corporately they will tell you that they
follow the COSO framework. They’re monitoring risks and measuring them and they will
identify controls. But if you take a look at what the COSO framework was supposed to be for,
you realize they don’t really follow it. In some cases it may be that they don’t understand the
purpose well enough, but I think what happens is that most internal audit departments see that
risk assessments is that pre-activities either to define what it is they will audit this year and they
feel that if you take it too big and you take a look at this and this objective, it’s very timely to go
116
through that entire process and they are generally not staffed for it, and few other say that
Enterprise Risk Management does that so there’s no point in redoing that work. Not that they
necessarily take the work that ERM groups do and create an audit plan of that either, but
sometimes they do try to compare to see what their risk assessment looks like compared to the
enterprise risk management. Although it’s difficult because different scoring points or something
else, so the math’s never works out the same. That’s more or less what I see happening between
the government side versus the corporate side.
That’s a lot of information in about two minutes.
We are more or less obsessing over it, because we’re in the process of… TeamMate has been
around in the market for over 20 years, so we’re in the process of creating a new platform that
we’re migrating functionality over to. So part of this process has been spending a lot of time
analyzing what is really used in our products and what’s not used. And the things which are
under-utilized, try to answer the question why is it under-utilized. Is it too complicated? Does it
no longer meet the market needs? Or do customers don’t want to put certain information in a
system or in too much structure in that process? And this forces us to go back to our customer
basis, spending a lot of time asking questions around this stuff. What risk methodology do you
really follow? Does it even have a name? Especially a corporate world is concept of just assessing
strategic risks and then you get into detailed risks. Most common in our user base is to find
about 5 to 10 strategic risks, but that’s not really a known or prescribed risk assessment
methodology in internal audit. If you read the IIA professional practicing framework, all it ever
says is that you must do a risk assessment and that your audit plans must be based on that risk
assessment information and you must be able to demonstrate that what was assessed in the risk
assessment, is then actually audited when you get into the engagement. But that’s kind of it.
There’s no prescribed formula or no prescribed methodology which is necessary to follow. So
what happens in a lot of corporate departments is, even if where they did try on the COSO
framework is it comes to the amount of time that’s really allocated to the groups, or activities
that are not strictly doing an audit in a business unit and providing in sort of a report an opinion
of how well things in the organization are working. What we do see across the board is
underfunding and understaffing of some of those activities that would provide some more
insight into the corporate as a full. Whether that training on monitoring on the back end of the
process, or giving them more time to do a better or more detailed job at the risk assessment up
front. That’s probably why I have so much to say about the topic.
That’s good, because my research is about risk assessment and which role that has within the
audit planning process. As I understand there are two levels of risk assessments, which you
117
already described a bit. There is the high level risk assessment, which is offered in TeamRisk if
I’m not mistaken. Where you would perform your annual planning for the auditable subjects.
You were saying that the risk assessment you divide into different areas.
Essentially, yes. The high level risk assessment is done and then when they go to start an audit,
they do a planning at the beginning of that process as well. And part of that planning process is
getting a better understanding of that business unit does specifically and what changes have
occurred since the last time that they’ve been audited as well as doing a detailed risk assessment.
So that’s much more likely when they go and sit down with that particular manager of a business
unit and ask questions about what are your objectives for this period? What risks do you see in
the process? Can you describe to be the workflow that happens within this business unit? And in
that process identify controls and potential risks. And then they’ll create an audit plan, a testing
plan, based on that. And that could be as much as 30 to 40% some times of the audit work. It’s
all the planning that they do and then they actually go out and execute some tests. From those
tests they will form an opinion, write up issues and create a report. So they do definitely break it
into two distinct phases. And part of the reason that they do that is… An audit risk assessment is
very much a point in time, it’s a snapshot of the risk. So if you think about that the original risk
assessment is maybe one year old, as the first risk assessment is done like September or October
of the prior year, as it needs to be presented and approved by the auditing committee. Then the
audit starts with reviewing if that risk assessment is still valid. Have any changes occurred in the
meantime? Have any new systems been implemented? Has management changed within that
particular business unit? That’s the kind of questions they will ask in the beginning of the
engagement to see if that risk assessment still makes sense. In the process of doing all that, they
start to identify the detailed risks. Then they want to make sure that they test it for the existence
of that risk or a higher possibility of that risk occurring.
What about those things you mention, if a process has changed, if senior management has
changed, in what way do they have an impact on the risk assessment?
Where you see that there are major system changes or work process changes, quite often when
they’re making those changes, they’re thinking about efficiency of a process and not necessarily
the effectiveness of the controls. It’s something that they’re supposed to consider, but the
effectiveness of the controls is likely the first thing that is left out. Everybody is focused on
getting more done in a faster time period and so they naturally go that way. So if you find a shift
in a business unit where one of those have occurred, it almost forces you to take a step back as
an independent set of eyes on the process. You see if the design of these controls is efficient and
effective. Are they actually being conducted? You might design them to work one way, but are
118
people actually doing that work the way it was designed? Was the implementation of those
changes effective and efficient? And sometimes it really comes down to: does everybody know
what the control is supposed to do or why it is here? And that affects the overall risk assessment.
If you start down with senior members of the team, where you know that process or systems
have changed, and they walk you through their work-flow, and you’re thinking about what could
go wrong in this process. That’s most likely where you will find some gaps. Where they did
things in the old system or the old process, not because they were understanding why they were
doing that, just because of the way it’s always been done and in the redesign stage, because
nobody understood why it was done that way, they changed the processes and you miss some
controls. Generally no key controls, but sometimes secondary controls that expose a weakness in
a shape or form in the overall process.
So the risk assessment is really a lot based on the judgement of the auditor?
Yes, absolutely. For the most part they try to do some supplementary things. So some groups
will just sit down and think about what they have learned in the last year and where there are
changes in the organization and they will do targeted interviews or surveys. So as they know if
there has been implemented a new accounting system as an example, they will probably go out
and interview managers in those areas where this new system has taken effect. If they know there
has been some sort of major process change in some part of the organization, they will target
that area of the organization to have a conversation with. Is the risk assessment from the past,
that we’re still relying on, is it accurate or better to change it. Some of the more progressive
groups will send out a self assessment to every single manager of the organization and have
themselves assess the risks. What they are really looking for is a comparison of what was
management’s view of last year to this year. Because if there has been no change and then you’re
probably ok to rely on that going forward. But if you see changes in risk assessments, then ask
the question why are there these changes. And usually in a self assessment process is an
opportunity for managers to write a comment on why they are rating particular risks the way
they are. Some times that explanation comes through in that self assessment. Generally there will
be a follow up or a change in the risk source year over year and that can make that organization a
more auditable project on an audit plan.
So in a self assessment, someone from senior management will identify the risks within their
processes?
Audit has usually predefined this. They start from the strategic perspective and then I usually see
those self assessments go out. And it’s not necessarily senior management. They often drill down
to the managers of the particular business unit, people who are much closer to the process as a
119
whole. To understand how they think of risk. Audit departments all have different approaches
on how they share this information. Sometimes they share last years risk assessment value and
they give them the opportunity to change it and supply reasons of why they would change it.
Sometimes they see the same list of strategic risks with no rating and then the businesses need to
rate it. That’s probably the most common. You rate yourself this year and you rated yourself last
year. Then compare them to the ratings of last year and take a look at the differences and say
why did you rate yourself differently? And then there are some internal audit departments that
will provide the standard list of risk and they will also be open for managers that show additional
risks. Some times they don’t really promote additional risks, they use their main risk assessments
and they will try to get more insights by trying to find out what is this manager thinking about?
What kind of risk is he perceiving and is trying to share? Some of the more progressive
organizations who are trying to management to own this process, have policies in place where if
you share with internal audit what are really the risks in your part of the organization and they
audit you and yes, some are risks and need to be better controlled, they go a bit easier on you,
because you self identified those up front. And it’s potentially a risk in the organization and it
will probably will get you audited, but you probably won’t get slapped on the wrist quite so hard,
because you made that confession up front.
I’ve rarely seen managers who were anxious to get audited. So I’m thinking a bit how it would
work. In generally in a company the managers see internal audit as a truly additional value, then it
would work. If that’s not the case. If internal audit is seen as the internal grumpy bear, then it
wouldn’t work. Or how do you see that?
You’re absolutely right. I would say that in the companies that we’ve got list, maybe 20% of
them are truly considered to be an advisor in the organization. 80% of the time, not so much.
And the difference as been seen as an advisor, there’s a huge culture shift to cross over that
barrier. And there are a lot more things that internal audit can do in perspective of interacting
with the business. They get other feedback from the business. So you might have a manager that
says that whole mea culpa that it’s a mess over here, I admit that. Audit may come in and review,
but they don’t necessarily grade them. Part of the reason why internal audit has such a bad rep, is
because they are giving grades. There are some business units that are not doing so well from a
revenue perspective. So I’m meeting all my business objectives, but internal audit isn’t working.
So I don’t really value their opinion that much. So internal audit departments stop grading
departments. They still write reports and find issues, but they stop assigning that opinion thing.
It’s amazing what kind of shift that gives to people. Then they say, ok, this is something I need
to fix. Internal audit is also asked to help on the implementation of systems. We don’t perform
120
the implementation, but we were assisting in evaluating control design and system
implementation, because internal audit can give it’s opinion up front and that’s a whole lot easier
then to correct course than afterwards. I see the same thing in the risk assessment and audit
planning side; they’re more seen as a advisor so much more flexibility they have in the kind of
activities they do around creating that risk assessment. Management participation in that process.
Either that or other general surveys. From free form questions like what keeps you up at night as
to the standard set of questions, asking the things we’ve already talked about. Significant
management change are more then turnover in your staff, or process change or system change,
to the sense of what could be interesting in the normal flow of business.
How would you describe the main goal for internal audit?
They’re the last line of defense. They’re the once who are supposed to evaluate after the business
has evaluated and other times it’s compliance, trying to help keep everything on course. It does
evaluate if the organization is well controlled. Are they anticipating the right types of risks
occurring and do they have the right strategies in place either to prevent them or more
appropriately actioned if something like that happened. And as you provide that feedback, not
only to the business unit, but also up the chain of command, to ensure that you’re not going to
have another Enron, or other disaster within an organization. Their main purpose is to provide
an independent review of the organization. Traditionally that has been an independent opinion,
but the term independent review of how the business is doing tend to be more powerful across
the organization. The other thing that I’ve seen in a wide range of internal audit departments, is
also listing of best practices within an organization. You say you need to fix some of these
things, but other things you’re doing really well. You understand the purpose of this control.
You’re making sure it’s being well maintained. Other parts of the organization can learn from
this. Some times there is a learning program that they’ve got running or having a monthly
meeting reminding that certain things are under review or particular key performance indicators
from departments. But stuff that they think that is helping the management to get into control of
things. If they start to do those kind of things and transition into that advisor role and there is
that debate whether or not internal audit should be an advisor in the organization, because then
it’s not clear if you’re being independent. But there are ways to manage that. Like the
implementation of a new process, the internal audit can review the process up front, while still
being independent, before the actual implementation of that process. To make sure that they’re
headed down the right path. Some times it’s evaluating all kinds of programs to make sure that,
even if they’re doing an application or an awareness thing. If you want to do business in a
country that you’ve never done business before, here are some risks that you should start
121
thinking about. And do you have a plan in place to prevent those risks from occurring. Early
detecting programs, that if a process is heading into that direction that you can correct for it. Or
are the right people involved? How do you transfer money from one country to another? Things
like that internal audit can advise on up front to make sure that people are thinking about
controls. As long as that they’re not designing those controls, but rather advising, educating and
evaluating what is being designed, then you have that advisor role without impacting on your
independence.
If you would be auditing Wolters Kluwer, what would be your main concern? What would be
popping out in your risk assessment?
As far as I know the organization, one of the major risks I think that WK has, that they have a
lot of products, so some times you’re in competition and in some cases it’s just the market. To
see what product is suitable for what thing. From a risk assessment perspective it would be
understanding what are the key performance indicators on any given software product or service
that WK offers. To understand is this product performing appropriately and is it similar to
another product. Would money be better spent in one product or the other. It seems that there
are multiple products from the same organization which are essentially taking revenues away
from each other.
Do you think that’s a high risk in companies which are acquired?
I some times wonder whether there is a… If WK people go out they only know some part of the
organization. Without having a full understanding, I know there are a lot of products under the
WK umbrella, but not everything that is offered. They run the risk that they’re inavertedly buying
something that essentially does the same thing that another part of the organization already does.
You can do that to try to consolidate the part in the market place and you’re buying out a
company that does something similar. That’s fine, but I don’t see that happening very often.
What I tend to see is something like ’X;, that seems like almost the same process or solution that
we have.
You mentioned that TeamMate is using elements from the COSO framework. Companies also
have their policies. How much of the policies are still coming through in the audit planning or
risk assessment?
It depends on the specific organization. When it comes to risk assessment, what they do in terms
of policies, is probably more aligned with are they used to calculate risk. The example which I
gave was with the impact and likelihood. The simplest calculation you can come up with. Some
organizations will use the size of the business unit on top of that. That can be based on the
122
number of employees, the size of the revenues or something like that. This is from a scoring
perspective. Then the weight can be different or the math can be different.
Are you aware of how Wolters Kluwer internal audit is doing that?
Somewhat aware of how they are doing it. They have audited us on a couple of occasions. They
follow a more simple formula to do risk assessment. They don’t use TeamRisk, they use an excel
spreadsheet. I know that they take a look at change and how a business unit stands out. Like
we’ve been audited because we’ve had double digit growth as one of the fewer entities. And with
that revenue growth, we’re also growing in the number of people. And do the new people
understand the general controls, are they being followed, have the control systems grown in
order to accommodate new processes and new people. So we end up on the radar regularly.
Being audited is kind of funny, since we’re all have internal audit experience, we’re a group of
people with internal audit experience. Auditing can become a bit more efficient. First you get the
compliance audit, then the internal audit and then the external audit, etc. and being the third or
fourth auditor, the manager might get grumpy for answering the same question. Maybe you guys
should talk. Maybe the auditors should look more at each others work. Audit should come in
first and then compliance and risk management should come in and fill in the gap. Not reinvent
the wheel all the time.
Is that where electronic work papers can help out?
Yes, in the new platform there is a compliance tool and one of the things that we want to do is
give organizations the opportunity to be able to share some of the information across division
lines. If compliance has already audited the risks and the controls, then audit will at least be
aware of it. If you add it to your audit engagement. Compliance has done this control in this
business unit and based on the time updates. Then you can send a notification to compliance
saying, we’re going to audit the same thing, can we look at the work papers? Because with those
notifications you can make more intelligent decisions about what do I really need to audit. What
do I need to ask somebody about? From there it makes it a lot easier to limit scope of work,
understand what others have done and share results. You can reference work of others in your
work. It requires groups to be open.
You really explained a lot about audit planning. Can you give me a brief overview of how the
auditing process works?
The auditing process as far as doing an audit or higher level like all the things the internal audit is
responsible for?
The last one. The first one is basically a result from your annual audit planning.
123
Right. As far as what the global internal audit department is supposed to do and all the activities
that accomplish that for them. At the start of the process is to do a risk assessment. Based on
that risk assessment and the understanding of what the most significant risks are in the
organization, create an audit plan and make sure that the this is explained to the audit committee.
For example they may notify that there are far more risk points then that they have staff to do
the work for it. Some times they will come with the request for additional staffing to get the
work done. Some times they would say that if we have to cover all these risks, then we need to
change the scope of the audit. And we will audit only these kind of risks and these parts of the
organization. So there are three different approaches on how they try to fulfil that audit plan.
With the resources that they have. Then get approval for that audit plan and then scheduling the
audit work throughout the year. And once an audit completes, then issue an audit report. The
issues should find their way to some sort of a system to be tracked until it’s resolved. As audit
work is progressing, there will be things that have resolution dates. Typically what happens is
that internal audit in most organizations on a quarterly basis report to the management and audit
committee about the status of the outstanding issues or some of the new issues that have been
reported this quarter. We have x number of issues outstanding and this number of issues are
closed. And they may report on which are outstanding and are way overdue or where
management offers extensions. So there is this reporting to the audit committee with and follow
up on the outstanding items. In most organizations the result of the audit work also feeds into
the risk assessment of the following year. So what they’ve learned during the course of doing
their work in the twelve-month cycle, typically impacts the risk assessment of the next cycle. For
example if they see that the risk is high, but the controls are in place and hardly ever fail, then
that can be part of the risk assessment of the following year and might change into the medium
risk category. Based on the audit work they report back to senior management and the audit
committee to let them know where they see risk exposure at the cost of the organization and
how well those risk exposures to be handled.
To get to a completely different topic, which is the other part of my research. That’s about ERP
systems. What do you know about ERP systems in general?
ERP systems are typically owned by the business and it’s supposed to be a much more dynamic
ongoing risk assessment versus the annual snapshot one point of time, that internal audit does.
There might be all kinds of risks in an organization, but what is the organization willing to accept
or tolerate? Where you have risks that are above that tolerance level, they are putting action plans
in place to make sure that those are being addressed. I’ve seen implementations of Enterprise
Risk Management. You could see month over month or week over week even some shifts in
124
risks across entities. The purpose of that is senior management can keep the thumb on the pulse
of what is going on from a business perspective and risk across the organization.
You were thinking about Enterprise Risk systems, I was referring to Enterprise Resource
Planning systems. Like SAP, Oracle.
As far as the resource systems, I’ve had very little interaction with them. I know we use them all
the time. But my knowledge of that is somewhat limited. The aspect of them is that the financial
aspect of those are usually well managed and reported. And when it comes to the
implementation of those systems it’s whether or not they expand from the financial reporting
into the resource, like HR type of activities, or not.
One of the main characteristics of an ERP system is that everything is integrated. All the
different data is integrated. Actually data is being entered in a database only once, while non-
ERP systems they actually use various databases or the same information is entered in various
databases multiple times.
It’s the great myth. It sounds fantastic in theory, but I very seldom see this in practice. In
example, in many years the market analysts talked about a single system that was going to
manage a business. I have yet to see a successful implementation of one. What I tend to find is
that each one of those groups that have a stake in a system that’s being used, have very specific
requirements around how to most effectively and efficiently run their part of the business. And
seldom are those principles in line with other groups across the organization. Or if you find a
system that does everything, then it doesn’t do it very well. It only scratches the surface in a lot
of the functions and it doesn’t have the main knowledge to get to the level of depth that these
departments need in order to be able to be successful in doing their job. Then it also rises the
question who is going to be responsible of that the data comes across in the right format. Even
from an ERP system perspective, I think about SAP, Oracle, some of those big guys, they have
got the accounting systems, they have go the HR systems and that all works on a single platform
and that feeds into other parts of the organization, so if you’re running analytics on that data,
you want to have continuous auditing built into the tool. Again, I have not seen any
organizations successfully implement all of that. Usually they don’t have a champion who cares
of having all of that in one system. Or people across the organization have different and better
tools they would rather use, because they think that they can get more out of it.
You definitely have thought about ERP systems. So having everything in one system is very
challenging and there will always be the use of various systems, including multiple data entry.
There are ways to better integrate some pieces of it. I think it ultimately concerns a lot of CIO’s,
what keeps them up at night is if we keep all our eggs in one basket, and all our data is in one
125
system, what happens if there’s a breach. What happens if my one system gets attacked? And
suddenly all parts of the system is down? They will segregate access to data. If internal audit is
gathering data for their samples, they will use scripts to retrieve that data. And if they want to run
those scripts themselves, then you can say you can’t just go into my database and run this script.
What will it do to my data? I need to know that it’s not malicious in some fashion. Therefore I
will need to review it and our policy is that nobody, except for a few, can run anything against
the database. So there’s this debate about who has the right to do what. If you put everything in
one database, that would tighten the concern over access to data. You will have more and more
administrators that will have access to the data and partly segregate what they can or cannot
access. Because typically in a database structure, if you have access to a database, you have access
to everything in that database. And sure you can have loggings of what you have done. In
separate systems I might have access to one database and the guy who sits next to me might have
access to two other databases. When you put things together, then it starts to break down the
segregation of duties.
If there is a single point of data entry, some of the data entry will be done by non-financials,
while still resulting in financial reports. Would you see that as a risk?
It could be. It depends on whether the people who are doing this data entry understand the
implications of everything they do. If the people who enter the data understand what the
information is used for, then it will be ok to have them enter the information. But if they don’t
understand the purpose of it, therefore they don’t think they need to be 100% accurate on
things, it will impact all the way down the chain.
And could system settings help out for that?
It’s possible that system settings could help with that. But what I have found if they find a way to
work around it and it’s simpler, then they will do that work around. It’s what you’re asking
people to do in their day to day job. And they adequately properly compensated to do the right
thing.
How do think the role of company accountants is changing by the use of an ERP system?
I’m not sure if the use of new systems is having their role change, but I’ve certainly seen that
they are expected to do more then just crunch numbers. I’ve seen their role change a lot in the
last couple of years. They’re not just there to do the reporting month over month the numbers,
but rather participating in the process to help to understand how can we manage our business
better. And not just from the perspective of how are we going to cut stuff. Like if they say you’re
travel expenses are up. Rather from the point to predict and analyze.
So not only internal audit, but also the company accountant is getting a more advisory role?
126
Yes.
So now we get to the impact on ERP systems on audit planning. I’ve already heard you say a
couple of points where you would be concerned if the entity you’re auditing is using an ERP
system. If you have in mind whether the auditable subject is using an ERP system or integrated
system, or not, where would your main concerns be or your major risks when you’re auditing
this entity?
My major concerns would be over external access to the data, because you put all eggs in one
basket. So if you don’t have strong security over data and access to data, whether that’s through
the application or through firewalls, that would be my first concern. And my second concern is
internal security. Do you have things partition in a way that it makes it hard for a single person in
IT to access that and that information to do something, not necessarily above board, that could
be anything from insider trading or destruction of data. Some companies don’t think about the
concept of insider trading. When systems are segregated, executives are aware of major business
decisions, but lots of times they have these discussions behind closed doors. The assumption is
that because it’s behind closed doors, nobody else knows about this. The security exchange
commission are watching them much more closely, but they’re probably not watching the
database administrator. And as everything is in one system, they probably have a better picture of
what is going on. They usually don’t think about that. They usually think more in the direction of
destruction of data. It’s extremely tricky to monitor that and that’s why putting everything in one
system leaves you open to those kind of risks. Corporations need to think about how would I
monitor that. How would I protect myself against that?
How would the use of interfaces be impacting an audit?
It probably comes in aside the segregation of duties, if you’ve got everything in one system it’s
more monitoring access to that user interface perspective. You will be looking at if there are
people in the organization who have access to far too much. Do we have adequate controls over
data entry to make sure that if changeshappen or need to happen they are appropriately tracked.
It probably has been discussed with internal audit when they’ve looked at control design. For
example I open up a screen and it’s going to ask me for several date fields. Are there parameters
under which these date fields relate to each other and make sense? Or is it something like pick a
date? Making sure that there is data integrity in that people interface to make sure that dates are
entered appropriately. And making sure that people cannot back date.
So some sort of logical controls?
Yes.
And what about interfaces between databases or systems?
127
Internal audit has been aware of those for a while. They will monitor those. They will do a data
exchange. Is it an update of data or is it additive? Is there a potential to destroy data in the
system that you’re not aware of? Do the data types match? Do they make sense? If system A is
updating system B and I open system B, do I understand what the updates came from? Can I see
some sort of data history or an edit log? So that I understand how that data got altered. Those
are the kind of things they should be looking at. Another thing we need to be looking at is data
in transit. So when it’s in flight, could it have been altered? So something came out of system A,
somebody grabbed that information in transit, altered it and different information have gotten
into system B. Then you hope there’s some sort of reconciliation that system A and B agree in a
point of the process. I see that in large organizations that’s absolutely that happens. In smaller
companies you see that they didn’t think it would be a problem.
Then you have a point of reference. You have two different databases, or two points of entry,
which you can compare. But if you’re using highly integrated systems, like an ERP, you have a
single point of data entry, then you should assume that this entry is correct. Does that raise
concerns for an audit?
Yes, which is why they would go back to and audit transactions to make sure that those
documentation and what would end up in the system record. So they do that end to end kind of
testing for sure. At that point of entry you have the greatest chance of error. Whether it’s on
purpose or by accident, that doesn’t really matter. When internal audit departments come in and
test controls, the first thing they would do is look at the source documentation and what has
come into the system to make sure that there is some sort of control process in place to make
sure that it is reasonable accurate. There should be manual processes as well in a business unit, to
make sure that the information is accurate as well. Although that doesn’t always happen 100% of
the time. So that’s one of the things that audit check.
How would that part be assessed in the risk assessment?
If you’re looking at the strategic risks of the financial controls of a particular business unit and
the knowledge is that in this business unit there is a huge manual user interface between
information that is on a piece of paper and getting it into the system for the first time. That’s
where they would assess the impact and what is the likelihood of an error occurring. And that
would be the first thing at the engagement planning level, like what is the likelihood of error on
that. What happens traditionally? Does management have a threshold as to what is acceptable? Is
there a secondary process to check if that information is correct? Does management itself have a
thought tracker or some sort of control in addition to something else to make sure that the
128
information is accurate? Those kind of things get assessed. If there is more manual work then
the risk goes up, as there is more chance of error.
That’s the area of where my research is heading for. I’m really looking into with he use of
integrated systems or non integrated systems, how that is going to be impacting the risk
assessment and from the risk assessment also the audit planning. Are there additional things that
you think I should be aware of?
The only thing from an audit planning perspective, it’s a challenge that internal audit
departments have, but it’s not which is discussed a whole lot, it’s the concept of capacity.
Capacity planning of an audit plan. It’s really about how much work can an internal audit
department take on in a year. As they do a lot more then just audit, like follow up on issues. So if
you plan the audits for a year, it’s based on the best guess on how long one single audit is going
to take. It’s not an exact science, but a best guess. Then you take a look at the capacity of the
team and compare that to the annual audit plan. Then the question is how do I cover off with
the resources I have.
So there is a potential risk that not everything gets audited, which should get audited.
Yes, there is a risk. A lot of internal audit managers will say that it is a calculated risk in low risk
areas. That does make the assumption that your risk assessment value up front was correct. And
if it has been assessed with low risk, that doesn’t mean it’s no risk. Maybe it’s not a huge financial
risk, but it might be a reputational issue.
I want to come to an end of this interview. The next step is that I will transcribe our discussion
and send it over to you for review.
129
8.9 Appendix IX: Interview #7
What can you tell me about your role in the organization?
I’m in the same group as Coleen and Andy in the product management group. My role is kind of
hybrid. Most of it is providing the features that we need to the development team.
Understanding what auditors must do, part of the audit process, and how they use an existing
system. Really looking into the day to day task of an internal auditor and I’m going to translate
that into features and requirements that should be built into our product.
Do you have a lot of communication with the customers as well?
Yes, not as much as we would like, but big part of this role is really to communicate with the
users and clients and understand what they do. We’re currently in a transition phase; we currently
have a twenty year old product and we started building our next generation audit management
tool. A lot of it now is to understand the features that they use on the current product and then
how can we make that better of different on the next platform. Now we see how they interact
with the system and see that in their natural habitat. And understand how they are not using the
system.
And do you get a lot of feedback as well as to what they are missing or what they would like to
see?
Yes, sometimes it’s feedback that isn’t telling that they’re missing something, but you can see
that. You maybe see that they’re working on an audit process or something and they have to go
to some Excel spreadsheet to get some information or do something that’s stuck on their wall.
So you see little pieces of things that they don’t even realize that they do, but it’s subconscious.
And that is something that the system could provide as well.
But then you need quite some knowledge of auditing yourself, right?
Yes and no. There are details of what goes into an audit and how is it performed the testing and
how are they linking the evidence and asking the questions to auditees. That’s the kind of stuff
that we not necessarily get that much into, but we have to understand it from the procedural
standpoint. What is the process of an audit? What are the pieces of information they might need
to perform the audit? So it depends on what level you mean by knowledge of auditing.
At least you have to get understanding of how they process an audit.
Absolutely.
Have you worked as an auditor before?
I have not, no. My background is in this protocol or business analyst type role.
What do you expect an audit process to look like?
130
There are different parts. There is from the beginning the planning phase of your audit plans at
the beginning of the year. Or based on the previous year you’re kind of determining what audits
or projects you want to take on for the next year. There is a whole process around that. Risk
assessments, talking things over with stakeholders of the business. It could also be external
advise coming from regulatory departments or governments or departments that oversee what
industry you’re in. We don’t have an industry specific solution, so the products that are used
whether you are in a public company or private companies. A third of our client base is
government, so they’re not doing an audit in the true sense of an audit, but they’re doing an
oversight of government agencies and things like that. So there are different types of planning.
First there is to understand among the different industries what is coming out in the processes.
For the risk assessment is there an advisory comment? A lot of this is overrun by the IAA, they
sort of have a guidance on how to do audits and kind of do you planning process, determine
where your areas of high risk are. So there’s a whole kind of guidance around that, but each
industry might have specifics. It’s really understanding all the different pieces. Your next step
would be getting it into the audit, where you take information which you may have identified in
your planning phase and then value more deep with the areas that you work with, the
departments, asking questions, getting documentation from them and obviously identify any
issues or problems that may be occurring.
You mentioned industry based solution. Do you think that would be workable? Do you think
that for audit planning, there could be a set of industry based solutions?
There’s definitely an opportunity. What we’re trying to do is build one solution and then we may
identify some features that a certain industry wants. We may come out with a government
package, because we know some features that government never uses. So we might scale down
the product and say this is a government solution. And we could do that for healthcare. So it
might be industry based. If you take a look at our competitors, problem with many of the
solutions is that there are not a lot of features. There’s a lot of external work that auditors have
to do to meet their requirements. I think we have the opportunity to build something industry
specific modules. We have TeamStore which is a library or a repository for all your risks,
controls and the tests that you do. Sometimes there’s a fairly common set of those for industry
specific. So we can provide that content with the product. We can say you bought TeamMate
and this content for your industry. Then they can build upon that and they don’t have to
manually add everything.
And would you think that such a standard set could be used in the risk assessment?
131
Yes, we provide risks. There’s something that clients can get access to today, we just don’t
package it up. We just sell the product and then the client can go and find the content they need.
And that includes risks and controls and lots of things like that.
Risk assessment is something that is broadly described in the Coso model. How does the Coso
model impact on audit planning?
It’s not necessarily impacting audit planning for internal audit, but more an understanding of
how your controls map back to the framework and the different principles of the Coso model.
That kind of discussions I have with clients. We are selling the CM module and there’s this thing
we call dimensions and that can be from the Coso framework. And we see clients that use that to
map their controls and make sure that they have a control at least, so that every single principle
that they don’t have any gaps there. But it’s mostly from that standpoint, more the coverage. Not
necessarily determine I have a gap here, I maybe need to do an audit. That will come, but not it’s
more of a mapping exercise.
To make sure that your risk assessment is complete?
Yes, right.
How does a risk assessment work? Audit planning has a high level annual planning and the
detailed planning of which tests to perform in an audit. How is risk assessment used in either
one of those levels?
What we see at the planning level you see more high level type risks. So you might take the same
five risks, strategic risks, and how acceptable are at the entities the risks that occur. So that feeds
into the planning process. Once you determine your high risk entities based on that, and then
you may need to audit these, let’s say 40, entities. They might show up with a high impact or a
high likelihood. Then they automatically say that is what I need to focus my audits on and so you
might plan to audit those areas. Once you do begin the auditing and you’re trying to determine
your detail tests and procedures, that’s where you might go deeper into the risks. For example an
organization might be in an earthquake risky environment, and then you take a look what
controls are in place. Most of the detailed risks during an audit are not really assessed, but just
identified. When you go to score them and determine what is the impact and likelihood of the
detail, you’re more looking at the generic strategic level.
Can you give me examples of an entity which is rated with a higher risk or with a lower risk
level?
Obviously financial risks tend to be higher. If the risk of impact is at a financial level. So for
example in an entity the accounts receivable completely disappears, then there’s no money
coming in, so those might be high risk entities. Versus human resources, that might be a little
132
less risky. But that’s where it comes on industry by industry. You’re going to have, depending on
what industry you’re on, certain entities will have a greater impact to the organization. If for
example in healthcare, regulatory impact can be very big. Big fines if you miss certain things,
which could have a huge financial impact to your organization. So you want to make sure that a
lot of your controls are functioning correctly in those areas.
To go into a completely different direction of my research. Are you familiar with ERP systems?
Yes, I am familiar with them.
What would you say is typical or characteristic about an ERP system?
Generally, from a software standpoint, they’re very complicated. They’re very large systems that
impact the entire organization. Usually when you implement an ERP system, you’re going to
have somebody in every different department touching that system. They’re very modular,
generally.
One of the things which is quite typical about an ERP system, is that several departments are
using one database, while with less integrated systems, you will have several databases for each
part.
Yes, that’s the enterprise piece of the system.
Exactly. I just wanted to point it out as it’s important for my research. It also implies that data is
being entered by non-financials and still resulting in the financial reports. Someone who is
working in purchases is not necessarily a finance person, but is still entering purchase orders,
which are ending up in accounts payable in finance. If you have that in mind, would you
consider that an issue?
From an audit standpoint it depends on what controls you have in place. So you have that
person who is entering, but maybe you have a department who is double checking the
information that it is correctly. To make sure that there are controls build into the system to
validate the data. From a software standpoint it depends on how you build that system.
Then you’re talking about which access a user has?
It could be access, but it could also be, they’re filling out a form and they’re putting in financial
data, there could be certain things in place that for example an amount should never exceed a
certain amount. So if they put something above or beyond the threshold, it might not allow them
to. So it’s only an issue if the right controls are not in place to mitigate a lot of the risk of
misinformation from the input.
So then you’re talking about logical controls?
Yes, it could be that. It could be training and different things.
That would be to improve the quality of the data?
133
Exactly. And it’s to get an understanding. The more people who have an overall understanding,
then they have a better idea of what they are inputting.
And do you think that a purchaser might not have the knowledge of what his input does in the
financial reports, that it is an issue?
Yes, I could see that as an issue. If you don’t have the knowledge of the end result, then they
don’t respect the data that’s input. Then there’s no understanding of why they need to put in
correct information. Then they’re not going to check or double check their data entry.
And you might cover that with training and settings of the system, right?
Yes.
If an organization or entity is using an ERP system, what would be your first concern as an
auditor?
The first concern is probably going to be the accuracy of the data. You want to look at the
reporting that is coming out of it. Also be looking at if they will be reconciled, such as financial
information coming out of the purchase information and things like that. Making sure that all the
valid information matches between the different pieces there. Making sure there is a correct
security in place. So making sure that people don’t have access to information they shouldn’t
have.
And how would that impact your audit planning?
Whatever department is responsible for the financial reporting, there will always come up
something to review the different controls, to see if the controls are in place and have the right
oversight. So you will include some aspects of that. For example here in the US we have the
SOX requirements and generally all your financial reporting controls are being tested.
So you have different types of audit, like a SOX audit or a financial statement audit, do you think
that would have an impact on auditing an ERP environment?
For sure. Every different audit is coming from a different direction. In an ERP environment
you’re probably going to do different focused audits on the different pieces. If it’s an IT audit,
you might cross different departments, to make sure there are different access controls. Versus
the financial audit, which is looking more at the end result. In the planning they will split that
out.
Why would you more look at the end result, instead of the data entry?
It’s a starting point and then you work your way backwards. To understand why are things put in
incorrectly.
So you’re reviewing the process?
134
Exactly. The first thing they do is set a priority to the issues. If the data entry is 100% correct
then from an audit point of view you say that passes. Then there might be little less testing on
the process side.
Is that an end to end testing?
Exactly.
That’s in an ERP environment, but if it’s a non-ERP environment, data is either centralized
through interfaces or by multiple entries. How do you think that the use of interfaces is
impacting the audit planning?
So you’re saying there are different systems capturing data and they kind of talk to each other
somehow to get to one final?
Yes.
It definitely increases the risk that’s occurring. The risk of misinformation or incorrect data is
significantly higher. So understanding all the different systems, you might get more IT focused.
So the IT audit will be increasing?
I see clients do it both ways, so they might wrap it up in a financial audit, they might do one
audit and not have different teams assigned. They might say these are the tests which we’re going
to do for the financial processes, the data entry process, and then there might be the IT part. But
I think you need both. There’s more importance of understanding the IT side of things. Because
at a point you probably have multiple administrators and a lot of people who have access to
certain things, so it’s a lot of control.
Because it’s not in one system where one data entry should be enough and then the check is
more on the data entry.
Exactly.
Opposed to the single data entry, there’s the multiple data entry. For the financial reporting for
example, the data will be entered by financial people. How would that impact an audit?
They’re focus is on training a bit more, because you want to make sure that there’s different
systems and there must be an understanding of the financial systems.
But if the entry is done by a financial person, that person already has knowledge of the financial
output. Would that make the audit easier? Or more hectic, because there’s more data to check?
Potentially. There are tools in place nowadays for the scanning of data. There’s not going to be
that many changes, because you will be looking at the output no matter what. So the biggest
concern is that you have different systems and data being entered into different systems.
Actually the use of different systems and multiple data entry also gives a control tool by
comparing the both databases.
135
Yes.
So with the use of an ERP system, you might loose that control tool. Do you see that as an
issue?
It depends. If it’s data being fed from system to system, it will bypass the business rules that are
in place. If you have a business rule in place in a system, it doesn’t necessarily mean that they’re
being honored. So in an ERP system there is no way to bypass the business rules. It’s build in no
matter where the data comes from. Data is always following the business rules. Versus when you
have systems talking to each other, some times that feed from wherever it’s coming from into
another one, might be able to bypass the business rules. So the control might not work.
It’s better covered in an ERP system.
Yes, because it’s all one. So the data being entered is the same, the business rules are the same.
You don’t have to worry about the system does this way in this system.
My research is about the impact of using integrated systems, with the single versus multiple data
entry, on audit planning and specifically the risk assessment. Now that you know more about my
research, do you have additional comments on this topic?
If you look at it from higher buckets, like the people, the process and the technical, there are
three kind of areas you can dive into. Then I think you’ll find the pros and cons on each of the
issues.
If you were doing a research, in which direction would you choose to do a research?
There’s also the perspective of regulatory controls. So maybe talking to internal control
department might be interesting as well.
How long have you been at the company?
Six years.
I will transcribe the conversation and I’ll send it to you for review.
136
8.10 Appendix X: Interview #8
Over audit planning tools, zoals TeamMate:
Een tool is in esssentie hetzelfde als de achterkant van een sigarenkistje, als je het daarop niet kan
lukt het met een tool ook niet. Maar het mooie van een tool is dat het wel enig houvast geeft.
Ook wij moeten achteraf traceerbaar ons werk gedaan hebben. Wij krijgen officiële reviews van
o.a. de IIA en van het NBA, (omdat ik registeraccountant ben), en dan moet ik aantonen dat ik
de kwaliteit handhaaf die ik zeg dat ik handhaaf en daar is die tool goed voor. Veel meer dan dat,
als ondersteuning van onze audit, is het niet. Het is een formeel vastlegging van wat je doet. Het
is van te voren goed bedenken wat je gaat doen en dit vastleggen helpt je, geeft je structuur en
houvast tijdens de audit. Translation: It’s a formal registration of what you’re doing. It is
considering in advance what you're going to do and this recording helps you, gives you structure
and guidance during the audit.
Zodat er geen gaten ontstaan en het zorgt voor de compleetheid.
Precies.
Wat ik van TeamMate mensen voornamelijk hoor is dat het een tool is en dat het uiteindelijk
vooral net zo goed is als je het zelf indeelt en het zorgt er met name voor dat je de audit zo
compleet mogelijke uitvoert.
Maar je moet het er zelf eerst instoppen. Dus in zoverre helpt het je niet als timesaver, tenzij je
heel veel herhaalwerk doet, en dat speelt bij ons minder.
Ik wil graag vastleggen wat je rol is binnen de organisatie en welke ervaring je hebt.
In augustus hoop ik mijn 40ste werkjaar te halen. Min of meer in dit vakgebied. Ik ben ooit
begonnen bij wat nu PWC heet. Daar heb ik 17 jaar in de openbare audit praktijk gewerkt.
Onderweg heb ik ook nog IT werk gedaan. Ik ben in 1992 overgestapt naar KPN. Daar heb ik
eerst bij de interne accountantsdienst de PwC risico analyse benadering geïmplementeerd. Het
was vooral een club die traditioneel testwerk deed, zoals de interne control mensen hier dat
doen. Dat hebben we veel meer geprofessionaliseerd. Toen ben ik overgestapt naar een afdeling
die zich met change management bezig houden. KPN moet je je voorstellen was een bedrijf dat
net naar de beurs was gegaan en nog heel erg archaisch, ambtelijk georganiseerd rondom
districten. En die districten liepen min of meer gelijk met provincies en die hadden alles zelf; hun
eigen processen, hun eigen systeem. Het was een wonder dat je van Groningen naar Friesland
kon bellen bij wijze van spreken. Dus dat bedrijf moest op de helling. Het moest allemaal
gestandaardiseerd. Zoals dat dan gaat met IT, moet je van de 13 systemen die je hebt, moet je de
minst slechte kiezen en zeggen die gaan we landelijk uitrollen en dan heb je een platform en dan
kan je het nog een keer echt gaan doen. Bij die slag ben ik bij vernieuwingsmanagement gaan
137
werken en daar was ik de change controller. Dus dat ging het over de vraag, niet alleen
veranderen we goed, maar is dan de kwaliteit van het proces en zijn de kosten van het proces dat
daaruit komt, zijn die ook concurrentie-proof? En dat was best een lastige vraag, want er was
geen concurrentie. We moesten daar een aanname voor doen. Het systeem waar echt
concurrentie voor ontstond was mobiel. Daar heb ik een voorliefde voor change management
aan overgehouden. Daar heb ik 9 jaar gewerkt in allerlei rollen, niet audit rollen vooral, maar wel
heel veel kijken naar projecten. In 2001 ben ik bij KLM gaan werken als hoofd internal audit.
Daar zochten ze iemand die kan vertellen waar dingen fout gaan en niet achteraf waar ze fout
zijn gegaan. Daar werd ik na het samengaan met Air France, hoofd van de groep internal audit.
Daarna kwam Wolters Kluwer op mijn pad. Een bedrijf in transitie, meer nog dan ik dacht. Ik
ben ik 2008 hier gekomen en doe in essentie hetzelfde als dat ik bij al die andere bedrijven ook
gedaan heb, maar dan toegesneden op WK’s behoeften. Ik zet mijn visie op verandering hier
neer en dat betekent dat je een operational audit team runt, dat vooral vanuit business risico kijkt.
En het business risico binnen dit bedrijf is vooral de verandering zelf. En dat heeft er toe geleid
dat ik ook een soort tweede pet heb gekregen in de loop der jaren, dat heet bij ons quality
assurance, dat is een tweede afdeling die ik trek. En die kijkt alleen naar grootschalige
verandertrajecten. Naar de vraag of dit inderdaad goed gaat landen en binnen tijd en binnen
budget. Daarnaast ben ik sinds 1987 docent bij Nijenrode. Inmiddels ook bij de UvA en heb ik
ook wat gedaan voor de Vrije Universiteit. Daarnaast heb ik nog wat nevenrolletjes gedaan, zoals
bestuurslid bij de IIA.
Wat zie je als het hoofddoel van een audit?
Het voorkomen dat dingen misgaan op basis van een risicoanalyse vanuit het bedrijfsbelang
geredeneerd. Het is het ultieme doel om te voorkomen, dat kan eigenlijk nooit. Er is een
metafoor; in het Confuciaanse China waren er twee broers, een tweeling, en dat waren alle twee
perfecte dokters. De een kon iedereen genezen en de ander kon voorkomen dat je ziek werd. En
die tweede rol is eigenlijk de rol van de internal auditor. Het is een hele ondankbare rol, want a:
je kunt nooit aantonen dat je succes gehad hebt en b: je moet eigenlijk altijd zorgen dat die ander
scoort. Je moet dus altijd een beetje achter de gordijnen blijven en dat is een soort ultieme rol
voor je audit afdeling. Dat is een hele lastige rol, want tegelijkertijd moet je toch aan een raad van
bestuur die meerwaarde aantonen. Dit moeten dat vertrouwen hebben en die moeten ook zien
dat jij daar resultaten boekt. Dat is een lastig proces om te managen, maar dat is het ultieme doel.
Daar kan je allerlei neven- en ondergeschikte doelen aan hangen zoals het geven van assurance
aan partijen over processen, kwaliteit en security en noem maar op.
En hoe vallen internal en external audit samen?
138
Dat is vooral niet overlappend. External audit zijn mensen die assurance geven voor de
verslaggeving van een bedrijf. Die zijn namens allerlei partijen, waaronder de raad van
commissarissen en de aandeelhouders, aan het kijken of de verslaggeving klopt. Wat natuurlijk
maar een kleine sub-set is van waar het bedrijf mee bezig is. Als internal audit moet je je
verdiepen in waar is dat bedrijf mee bezig? Wat zijn de risico’s wat die activiteit met zich
meebrengen? En hoe kan ik op de beste manier een aandeel zijn in het vaststellen en voorkomen
van het risico. Dat is een heel andere scope en een heel andere doelstelling.
External audit stelt dus eigenlijk vast dat het gerapporteerde klopt en internal is zich meer bezig
met de processen van een organisatie.
Ja. Het begint eigenlijk met strategy execution. Wij stellen de strategie niet vast, maar op het
moment dat deze vaststaat, is dat je aangrijpingspunt om te gaan kijken wat hier dan fout kan
gaan. Doen wij nog wel goede dingen? En doen we die dingen dan ook nog goed? Translation: It
actually starts with strategy execution. The company strategy is for us a starting point, from there
on we assess the business risks of WK and see what can go wrong. Are we still doing the right
things? And are we doing those things rigth as well?
Werkt de audit planning dan ook zo? Vanuit corporate strategies?
Absoluut. Je neemt kennis van de strategie en in ons bedrijf doen we een keer in de zoveel jaar
op hoog niveau echt de strategie. Die zijn abstract en de vertaling daarvan waren de BDP’s, dat
zijn nu de VSP’s. Dus wij nemen kennis van de VSP’s, die lezen we en proberen we goed te
snappen. BDP’s en VSP’s zijn slide decks van 100/200 powerpoint slides, waarin het
management van een entiteit of een divisie uitlegt aan de raad van bestuur dit gaan we de
komende jaren doen om deze redenen, dit is het concurrentielandschap, dit is ons
systeemlandschap, dit is onze portfolio en dit zijn de veranderingen die we daarin gaan doen en
dit is wat het kost. De financiële vertaling heeft hier altijd een heel prominente rol, maar wij
proberen om vooral ook de operationele vertaling te zien. Wat betekent dat nou? Wat ga je dan
doen? En hoe concreter dat is, hoe meer je er aan hebt. Hoe vager het is, hoe meer je daarna ook
met business management moet gaan praten om te snappen wat ze nu echt bedoelen. Dus wij
doen een keer per jaar een soort update, op basis van de BDP’s of VSP’s. Dat doen we in
augustus / september. En dan gaan we daarna op basis van de analyse van wat we gelezen
hebben, gaan we met alle managers van de grote entiteiten praten en dan zeggen we dit denken
wij te snappen van hoe jullie de wereld zien, klopt dat. Dit vinden wij wat wij dan als audit partij
moeten gaan doen om jullie zo goed mogelijk te helpen te voorkomen dat er dingen fout gaan.
Zien jullie dat ook zo? Die dialoog heb je dan. Dan gebruik je een trechtermodel omdat er in het
begin meer onderwerpen zijn dan capaciteit om ze uit te voeren. Die trechter is in het begin heel
139
erg gevuld met heel veel ideeën. Langzaam maar zeker ga je die doos met ideeën een beetje
schudden en dan praat je met management, je praat met de raad van bestuur, je praat met de
audit committee, je praat met de corporate afdelingen van treasury, van tax en noem maar op.
Hoe zien jullie dat? Dan wordt die doos met blokken wordt kleiner en steeds minder blokken ga
je door die trechter heen schudden en op een gegeven moment heb je nog een aantal blokken
over en die match je dan met je capaciteit. Kan ik dit aan? Zo niet, dan maak je daarna nog een
keer keuzes in. Die keuzes leg ja dan voor aan je raad van bestuur, dan zeg je is het akkoord?
Vinden jullie dit ook een goed plan? Meestal zeggen die ja. Soms zeggen ze van doe dit niet en
dat wel. Daarna ga je naar de audit committee en dan ga je dat nog een keer doen. En dan zeg je
dit is het plan, zij vonden het goed, vinden jullie het ook goed? En dan heb je nog een keer een
discussie. Dan vragen ze meestal niet waarom dit onderwerp, maar hoe heb je dat selectieproces
nou gedaan? Leg eens uit hoe je hieraan gekomen bent. En dan vertel ik het verhaal nog een keer
met alle slides hoe we dat gedaan hebben. En dat selectieproces gaat ieder jaar weer een beetje
beter, want inmiddels leer je wat er misschien nog meer belangrijk is en dan plak je weer een
extra elementje in.
Dan ben je ook je proces aan het verbeteren?
Absoluut, want het is continue fine-tunen van het proces. Want het proces is niet eenduidig. Het
proces en alles wat wij doen is heel erg toegesneden op wat wij denken dat Wolters Kluwer moet
willen. Als ik hetzelfde bij KLM moet herhalen, zou ik het heel anders aanpakken. Niet qua
essentie van het proces, maar wel qua gebieden die je dan pakt en qua manier waarop je ernaar
kijkt en bepaal je met wie je praat. Want dat is per bedrijf natuurlijk anders.
Als ik het goed begrijp, je pakt dan het meerjarenplan, of eigenlijk de corporate strategy dat zich
vertaald in de VSP’s, daarbij heb je overleg met het management van die entiteit en daar komen
ideeën uit voort van wat eventueel geaudit zou moeten worden.
Ja. De uitkomst van die discussie is die hele grote lijst. En dan zijn er een aantal premissen die we
handteren, die de lijst moeten verkleinen. Risico is er een van. De raad van bestuur heeft gezegd
of je naar grotere acquisities wilt kijken, dan moet je dat eigenlijk tussen 6 en 12 maanden na
acquisitie doen. Dus we willen dat je dat structureel doet. Daar kijken we specifiek naar. Ook de
vraag van grote entiteiten, daar moet je niet te lang wegblijven. Dus daar moet iets van roulatie in
zitten. Daar kan 4, 5 of 6 jaar tussen zitten, maar je moet naar Duitsland 1 keer in de zoveel jaar
moet je daar wel heen. Dat is een groot land.
Is het dan ook omdat de risico’s daar groter zijn?
Ja, omdat het materiele belang groot is en daarmee ook de risico’s. In Roemenië kan het risico
misschien veel groter zijn, maar de exposure is heel klein, want de omzet is klein. Dan is het
140
risico automatisch niet heel groot. Tenzij reputatie een rol gaat spelen. We streven natuurlijk naar
1 brand en als je dan in Roemenië een uitglijer maakt, dan is het wel Wolters Kluwer dat in de
krant staat en niet meer een lokale naam. Daar moet je rekening mee houden en dat proberen we
ook te verdisconteren in die aanpak.
Zit dat dan al in de risico analyse?
Ja.
En uit die risico analyse, dan ga je eigenlijk al fine-tunen naar welke delen je gaat auditen?
Ja, want je moet natuurlijk ook over divisies een beetje spreiden. Daar zit ook nog wel eens het
verzoek van management aan vast. Die zeggen dan, wij voelen ons niet comfortabel bij XYZ,
kan je daar een keer naar kijken? Raad van bestuur die zo zijn eigen prioriteiten heeft, van kijk
hier of daar nog eens naar. Dus in die zin moet je daar een beetje in marchanderen. Maar
uiteindelijk komt er altijd een lijst uit waarvan wij zeggen, ja dat spoort wel met ons beeld van
risico.
En dan heb je een jaarplanning?
Dan heb je een jaarplanning. En die staat voor 90% in beton. En er zijn natuurlijk altijd brandjes
die uitbreken plus additionele verzoeken. Mensen die zeggen, ik snap dat je hier nu wilt komen,
maar we zijn net met een SAP implementatie bezig, misschien is het handig dat je het een paar
maanden uitstelt. In grote lijnen klopt die planning. In detail wilt er nog wel een klein beetje iets
schuiven.
Van de jaarlijkse planning, ga je naar de entiteiten zelf en heb je daar ook nog een bepaalde
planning voor?
Absoluut. We gaan het proces van de engagement letter in. Dat is in feite de bevestiging aan het
lokale management van dit is de scope en daar heb je een hoop voorbereiding voor. Interviewee
#2 heeft met BAC samen een soort van financiële analyse tool gemaakt, waarbij we zeggen van
op basis van de financials zijn dit aandacht vragende punten voor ons. Daar willen we eigenlijk
naar kijken. Dan hebben we ook de risico analyse zelf gedaan, daar kwamen natuurlijk ook een
aantal dingen uit waarom we zeiden dat deze entiteit moet op de lijst. Maar het zou ook goed
kunnen dat je er nooit geweest bent. Dat is ook een reden om te gaan. En dan heb je geen enkele
referentie en dan moet je op een andere manier te werk gaan. Dus wat we altijd doen is met de
belangrijkste stakeholders van zo’n entiteit een gesprek houden. Dat zijn meestal meerdere calls.
Waarbij we zeggen, leg ons je business uit. Vertel waar je staat. Vertel wat je competitive
landscape is. Eigenlijk alles waar je mee bezig bent, wat je bezig houdt. En ook weer vanuit het
VSP kan je heel gericht vragen stellen. En vervolgens vragen we waar blijf jij nou wakker van?
Wat zijn jouw zorgen? En de ene is er heel open over en de ander moet je het eruit trekken, maar
141
dat leer je op den duur ook wel een beetje. En als ze merken dat je hun business snapt, dan
worden ze ook opener. In het begin is er altijd een beetje argwaan…………. De cultuur van een
bedrijf kan bepalend zijn in hoe open men is. Als je een fout maakt en het kan je kop kosten, dan
is men wellicht niet zo open over hun processen. En dat kan ook tegen een bedrijf werken.
Er komen termen naar voren als risk assessment en control environment, hetgeen onderdelen
zijn in het Coso framework. In hoeverre wordt het Coso framework gebruikt in auditing?
Het Coso framework is in ons bedrijf alleen maar van belang voor het framework dat de ICO’s
gebruiken. Eigenlijk is het framework dat hier intern gebruikt wordt, is een Tabaksblad variant
op SOX. PWC is hier in huis gehaald en heeft 10 cycles neergezet waarlangs de Internal Control
Officers hun testing moeten doen. Dat is 10 jaar geleden ongeveer neergezet en 10 jaar lang niet
geupdate. Je kan je voorstellen dat een bedrijf dat zo snel verandert als Wolters Kluwer, dat het
framework niet helemaal meer toegesneden was op de dingen die we doen. Dat heeft de huidige
manager onderkend. Die heeft gezegd dat gaan we aanpassen. Die is dat nu aan het doen en die
worden meteen aan Coso 2013 aangepast. Het speelt wel een rol in ERM, maar zo ver relevant
voor financial disclosures. We hebben als bedrijf niet gekozen voor een centraal ERM functie,
omdat de CEO dat niet wilt. De CEO zegt, ik doe risico management en alles dat mij daarbij kan
helpen is meegenomen, maar niemand anders hoeft dat namens mij nog ergens anders te gaan
doen. Dat betekent dat wij nogal verkokerd naar risico’s kijken, want elke business doet dat voor
zichzelf. De enige die dan wat breder kijken zijn Internal Control, alleen voor die financial
disclosure en wij, voor ons audit universe. En ik denk dat er wel eens wat gaten vallen, omdat
tussen de silo’s in, mensen dingen niet zien. Dat kan gevaarlijk zijn. Maar we moeten niet
overdrijven, omdat er zo veel versnippering is, de risico’s kleiner zijn. Voor de meeste risico’s
geldt dat door de versnippering deze risico’s ook alleen in deze entiteiten te vinden zijn en niet in
de entiteit ernaast. Translation: For most risks one could assume that the dispersion of the
company in so many independent entities in many countries means that the specific risks can
only be found in these entities and likely not in the entity next to it. Er is geen entiteit zo groot,
dat die het hele gebouw kan doen laten kantelen.
Dat is ook een manier van risk management.
Absoluut. Of de raad van bestuur dat ook bewust zo doet, dat weet ik niet, maar ik sluit het niet
uit.
Met een risk assessment als je daadwerkelijk naar een audit gaat. Hoe werkt dat proces?
Dat is gefaseerd. We hebben dus die cyclus gehad, die ik net heb uitgelegd, de jaarlijkse ijking
daarvan. Daar komt een beeld uit wat ertoe geleid heeft dat een entiteit op de shortlist staat van
het plan. En daarna gaan we in de voorbereiding een financiële analyse doen van zo’n entiteit.
142
Dat geeft je een financieel plaatje. Je hebt dan de reden waarom ze op die lijst staan. En die
reden kan of zijn omdat er in het verleden problemen waren, of omdat ze zo groot zijn, dat ze in
de rotatie zitten, of omdat we er nog nooit eerder geweest waren en we vonden dat het nu eens
hoog tijd werd dat we er heen gaan. Afhankelijk van het verleden is je startpositie anders. Of je
moet alles nog uitzoeken. Of je hebt een aantal gerichte follow up zaken waar je voornamelijk
naar gaat kijken. Of het is een standaard follow up vanuit de rotatie en kijk je wat heb ik nou
vorige keer gedaan. Van daaruit, is er nou aanleiding om andere punten aan te pakken? En dan
heb je die calls met management die jouw beeld nog kunnen bijstellen. Van we zijn aan het
shiften in portfolio activiteiten, of onze backoffice systemen zijn veranderd, of we hebben een
nieuw management, noem maar op. Allemaal kunnen dat aanleiding zijn voor je scope en je
risico analyse.
Zijn er in die genoemde zaken een paar dingen waarvan je zegt, daar springen we meteen op?
Backoffice systeem veranderingen. Migratie naar SAP, Oracle of noem maar op. Dat vinden wij
belangrijk. Translation: Backoffice system changes. Migration to SAP, Oracle or any other.
That’s what is important to us. Als het financiële resultaat aantoont dat voor bonus net wel
gehaald hebben of net niet halen. Want dat zijn de momenten dat de creativiteit toeslaat. Dat
soort dingen zijn voor ons een belangrijke trigger.
Dan heb ik een aardig beeld over het audit proces.
Dit is de ingang van het proces. Daarnaast is er de audit zelf. De rapportage. En strikt genomen
zouden we ook moeten evalueren. Dat doen we beperkt. Wat we in ieder geval doen is een
questionnaire naar de geauditeerde sturen zeggen hoe heb je het nou ervaren? Met wat gesloten
vragen waar hij kan scoren en wat open vragen. En we doen zelf file reviews, dat speelt Stans
ook een rol in. Die kijkt of wij TeamMate wel netjes hebben toegepast. Soms, als hij in de
steekproef valt, dan kijkt een collega die niet betrokken was bij de audit naar wat je gedaan hebt
en stelt dan kritische vragen van waarom heb je daarvoor gekozen en waarom heb je niet dit
gedaan. Zo houden we elkaar een beetje scherp en het is de bedoeling dat je er ook weer van
leert.
Het andere deel van mijn onderzoek gaat over ERP systemen. Wat vind je karakteristiek van een
ERP systeem?
Een ERP systeem is een commercieel product dat, net als in accounting diensten, het gevolg is
van enorme clusteringen. Dus je hebt eigenlijk maar een paar grote leveranciers in de wereld
voor de schaal waarop grote bedrijven opereren. Het is eigenlijk een beetje de tragiek van gister.
In de zin dat wat je tegenwoordig zou verwachten is veel meer een best of breed vanuit cloud. Ik
denk dat dat de toekomst is van ERP systemen. Wat je vanuit de historie ziet, is dat een ERP een
143
keurslijf is, waarbinnen fabrikanten zo goed mogelijk proberen verschillende industrieën te
faciliteren door een ERP systeem in modules uiteen te knippen en die modules zo goed mogelijk
te tailoren naar dat deze industrie denkt nodig te hebben. Maar dat heeft enorme beperkingen.
Translation: It's actually a bit the tragedy of yesterday. In the sense that what you would expect
nowadays is very much a best of breed from cloud. I think that that is the future of ERP
systems. What you see in history is that an ERP is a straitjacket, within which manufacturers try
as best as possible to facilitate different industries by an ERP system in modules to cut and
which to tailor to modules that this industry expect to need. But that has massive limitations. En
de essentie van die beperkingen is dat het vooral geschreven is op grootschalige
productiebedrijven. Chemie, auto’s, dat soort bedrijven varen wel bij een ERP systeem. Omdat
het voorspelbare herhaalbare processen zijn. Niet te veel complexiteit. Dan is het hebben van
een centrale repository voor je processen en een centrale repository voor je data ideaal. Het
levert je ook communicatiemogelijkheden met je klanten en met je leveranciers op. Je kunt je
hele logistiek erin kwijt. Voor bedrijven die veel meer complexiteit hebben, die veel kleinere
processen hebben, kleinere volumes en veel meer uitzonderingen is een ERP systeem vaak een
moeizame beslissing en vaak meer een last dan een lust. Los daarvan zijn ze duur. En zie je een
heel scala van bedrijven die toch hebben gekozen voor een ERP oplossing, die daar eigenlijk
ongeschikt voor zijn. Translation: For companies that have a lot more complexity, which have
much smaller processes, smaller volumes and more exceptions, an ERP system is often a difficult
decision and often more of a burden than a benefit. Apart from that, they are expensive. And
you see a whole range of companies that still have opted for an ERP solution, which they are
actually unsuitable for. Wolters Kluwer is daar denk ik een goed voorbeeld van. Wij zijn
ongeschikt voor een ERP systeem. Je kunt zeggen dat het in kleinschalige omgevingen kan
werken, maar voor de backoffice voor ons bedrijf, is geen enkel ERP systeem geschikt. Niet als
geheel. Dat gezegd hebbend, de toekomst van ERP is dus non existant in dit soort bedrijven. In
dit soort bedrijven zal je naar een cloud oplossing moeten, waarbij data standaardisatie voor je
backoffice een veel belangrijker probleem is dat je eerst moet oplossen. Als je dat opgelost hebt,
kun je met cloud oplossingen waar je met een standaard manier inparkeert, veel meer bereiken
door extracties met SAS pakketten.
Betekent dat dan dat er verschillende databases hebt, die ieder apart data in de cloud schieten?
Dat zou kunnen. Op den duur kan het niet anders. Hoe meer gestandaardiseerd je werkt, hoe
meer je gestandaardiseerd naar 1 database hebt kunnen gaan. In feite is naar HFM 1 database,
maar de Chinese interface naar allemaal niet standaard voedende systemen. Als je daar hele
strakke regels op zou kunnen zetten, dat je zegt iedereen doet het nu op deze manier, dan zou je
144
kunnen zeggen van ik neem een echte database en dan ga ik die gegevens er ook echt in zetten.
En ik heb nu ineens een datawarehouse gekregen waar ik wat mee kan. Voordat je zover bent,
ben je al 10 jaar verder en heel veel gesteggel met al die partijen om al die neuzen dezelfde kant
op te krijgen. Maar dat is wel de toekomst. Daar geloof ik heilig in. Omdat wij geen alternatief
hebben.
Een van de punten van een ERP systeem is, dat de data maar op 1 punt in het systeem en
database wordt gezet. Hetgeen per afdeling dan gebeurd. Wat dan allemaal resultaat heeft in de
financial statement. Zie je dat als een risico?
Nee. Het kan een risico zijn, maar dat ligt eraan hoe je het oplost. Maar het feit dat je processen
niet standaard zijn, als je de wokkel in SAP systeem te krijgen, dan krijg je ellende. Dan kan je
data uit het systeem krijgen, die je niet kunt verklaren. Translation: No, it could be a risk, but that
depends how you mitigate it. The fact that your processes aren’t standard and you squeeze it into
SAP, then you’ll get a disaster. You will get data from the system which you cannot explain.
Als je de processen niet goed analyseert?
Eigenlijk dat je de processen niet aanpast, zodat ze in SAP passen. Translation: Actually that you
don’t change the processes, to make them fit into SAP. Dat bedoel ik met dat de systemen die
dicteren als het ware aan jou hoe jij je processen moet indelen. En er zit in die modules enige
flexibiliteit, bijvoorbeeld hoe ze naar bepaalde industrieën dingen hebben aangepast, maar daar
moet je je wel heel strak aan houden. Heel veel bedrijfsonderdelen zien dat niet en zeggen wij
zijn speciaal. Dat moet je bij SAP niet willen. Dat kan niet. Je moet in het keurslijf.
Als je wel een geïntegreerd verhaal hebt en je hebt een inkoopafdeling die purchase orders
genereert, wat uiteindelijk in je financial statement terecht komt. Zo’n inkoper hoeft niet per se
een financieel persoon te zijn.
Maar die moet de processtappen volgen zoals die in zijn vak gelden en dat interacteert dan direct
met het systeem. Dus je hebt de status van een order. Die order heeft een besteller. Die besteller
heeft een purchase order gemaakt. En jij als inkoper gaat die inkoop uitvoeren. De leverancier
gaat het spul leveren. Het magazijn meldt dat het ontvangen is. Er komt een factuur binnen. Dat
zijn allemaal status codes van 1 order die allemaal in dat systeem op volgorde worden gevolgd.
En je kunt precies zien waar je zit. Dat is de kracht van een ERP systeem.
Dus daarmee geef je eigenlijk aan dat als je het ERP systeem goed hebt ingericht, dat het geen
negatief, maar zelfs een positief effect heeft?
Het is enorm krachtig. Ontzettend positief. Het hebben van een centraal ERP systeem is enorm
krachtig. Maar om daar te komen, moet je goed nadenken kan ik dat wel en voldoe ik aan alle
voorwaarden.
145
In die ideale ERP omgeving, in hoeverre verandert de rol van de accountant dan?
Je krijgt dan een team dat alleen maar big data analyse doen. Die halen alle informatie uit de ERP
instantie en die worden gedownload in de servers van internal audit en dan ga je continuous
monitoring en continuous auditing doen. Die hebben dan op een hele andere manier hun risico
analyse proces. Daar heb je bijvoorbeeld fingerprint technologie. Waarbij een bepaald patroon
wordt gevolgd. En als er iets raars in dat patroon zit, dan zeggen ze er is iets aan de hand. Even
kijken hoe het zit.
Zit het audit pakket dan ook geïntegreerd erin?
Dat zijn zelf geschreven pakketten. In wezen is het big data analyse. Je bent dan op een hoog
niveau met statistiek bezig dan. Translation: In fact it’s big data analysis. Then you’re doing
statistics at a high level.
Zie je dat als auditor als ideaal?
Het is heel krachtig, maar het is nooit af daarmee. Dat is alleen maar een signaal functie. Wat je
doet is een vlag zetten op een gegeven en dan zeg je hier is kennelijk iets aan de hand. Hier ga ik
naar kijken. Afhankelijk van het soort vlag en signaal kan je zeggen ik ga hier als auditor naar toe
of ik stuur gewoon een mailtje of een belletje naar de manager van die afdeling en zeg vertel mij
eens even hoe dat zit. Daarmee heb je wel bijna volledigheid van de operationele stromen
bereikt. Terwijl wij met steekproeven hier en daar naar te kijken. Zij zijn in feite integraal aan het
controleren en dat is natuurlijk ontzettend krachtig.
Zie je dat dan nog steeds als een auditor rol of meer als een controllers rol?
Beide. Ik denk dat de controllers continuous monitoring doen en de auditors continuous
auditing doen. En als je dat heel goed aanpakt en heel structureel aanpakt, dan gaat dat bijna in
elkaar over. Dan is de rol van ene om te constateren en te verklaren en de ander te valideren.
Transaltion: I think the controllers do continuous monitoring and the auditors continuous
auditing. And if you do this very good and structured, then that almost melts together. Then the
role of one is to to review and explain and the other to validate.
Continuous monitoring en continuous auditing is wel iets waar een ERP systeem je de
mogelijkheid toe geeft.
In theorie wel. Ik zie het in de praktijk bijna nergens. Translation: In theory yes, but I hardly see
it inpractice.
Eigenlijk doe je ieder jaar een volledige risico analyse van alle entiteiten.
Je kijkt waar dingen nog relevant zijn en waar niet. Met name de verandering daarin is de
drijfveer.
146
Als je geen ERP systeem hebt dan heb je verschillende databases die met elkaar moeten
communiceren. Het gebruik van multiple data entry punten in hoeverre beïnvloed dat de risico
analyse?
Zeer. Want inconsistente data zijn een risico. Wat je als auditor probeert te bekijken is wat is de
kwaliteit van mijn besluitvorming in dit bedrijf. Hoe snel kunnen wij op basis van de juiste
informatie besluiten nemen. Hoe lastiger het is om bedrijfsinformatie boven water te tillen, hoe
moeilijker het wordt om daar een besluit over te nemen en hoe gevaarlijker het is dat dit op basis
van verkeerde informatie gebeurd. Translation: Very, because inconsistant data are a risk. As
auditor you want to review the quality of the decision making. How quick can we make decisions
based on the correct information? If it’s more challenging to retrieve the business information,
then it becomes more challenging to make a decision based on that information and the more
risky it becomes that the decision is based on incorrect information. Dat is bij ons een probleem,
omdat wij bij sommige entiteiten verschillende CRM systemen hebben. Dat genereert risico,
absoluut.
En daar gaat je audit planning dan ook door veranderen.
Absoluut.
Als je vergelijkt tussen en echt ERP systeem en verschillende databases. In hoeverre beïnvloed
dat je risico analyse?
Totaal. Op het moment dat de manager geen overzicht heeft, is de kwaliteit van de interne
controle ook minder. Is het lastiger om vast te stellen of je dingen wel goed doet. Is de kans voor
mensen om misbruik te maken van het systeem ook groter. Is het risico op fraude ook groter.
Maar vooral het risico van fouten. Dus daar ga je op die manier ook mee om.
Dus dan moet je meer samples testen?
Ja, en ook er anders naar kijken en je kan veel minder steunen op het systeem.
En hoe beïnvloed dat het kijken naar de interne controle.
De vraag is wat de interne controle ermee doet. Als het management het probleem onderkent, en
die interne controller die anticipeert op de problemen die de diversiteit van systemen met zich
meebrengt, dan zijn er vaak work around controls. Als je die test, dan kunnen wij daar weer op
steunen. Maar dat is veel meer werk en vaak is die man of vrouw die daar zit, maar in zijn eentje
en kan dat allemaal niet doen. Dus die maakt ook keuzes. En als wij die keuzes snappen en ook
zo gemaakt zouden hebben, helpt ons dat. Als dat keuzes zijn die wij niet zo gemaakt zouden
hebben, omdat ze beïnvloed zijn door de CFO die misschien hele andere prioriteiten heeft, dan
moeten we zelf dingen gaan doen. Dan neem je als bedrijf het risico dat ook wij dingen niet zien.
Bij een geheel geïntegreerd ERP systeem, kan je in twee weken integraal een compleet
147
bedrijfsonderdeel checken, en bij een niet ERP systeem kan je in twee weken een foto maken
van zo’n bedrijfsonderdeel en hopelijk voldoende assurance te verkrijgen om te zeggen het zal
wel goed zijn. Maar zeker weten doe je dat natuurlijk nooit. Hoe complexer het is, hoe lastiger
het is om dat beeld te krijgen. Dat is een andere vorm van risico; dat je geen tijd hebt om vast te
stellen. Translation: With a completely integrated ERP system, you can review a complete entity
within two weeks. At a non ERP system environment you can take a picture of the entity in two
weeks to gain assurance and assume it is correct. But you’re never completely sure. The more
complex it is, the more challenging it is to get that picture. That’s another form of risk: that you
don’t have enough time to ensure that.
Je audit risico eigenlijk? Dat je niet de tijd hebt om alles te zien of dat je op de juiste plek kijkt.
Ja. Dat risico wordt groter naarmate je steekproefsgewijs in een hele complexe huishouding
tekeer moet gaan.
Als je verschillende databases hebt en verschillende punten van data entry, dan heb je ook een
controlemethode door de twee databases met elkaar te vergelijken.
Dat is meestal niet erg effectief. Dat hangt ervan af wat het doel is van je controle. In principe is
het vergelijken van databases niet zo simpel.
Als je een ERP systeem hebt, met 1 database, heb je in ieder geval niet die controle mogelijkheid.
Nee, je steunt dan op het feit dat er maar 1 database is.
Geeft dat dan bepaalde risico’s?
Ja, want je moet eerst vaststellen dat je er überhaupt op mag steunen. De processen die leiden tot
het vullen van die gegevens en de manier waarop dat is geconfigureerd en je access controls, wie
hebben er toegang toe, dat zijn voorwaarden die je moet checken, om vast te stellen dat de data
in dat systeem correct zijn.
Plus eventuele logische controles in het systeem?
Absoluut. Negatieve voorraden dat moet je bijvoorbeeld wel signaleren.
Mijn onderzoek gaat over in hoeverre een ERP systeem, met name data kwaliteit en single point
of entry, in hoeverre dat impact heeft op de risico analyse en de audit planning. Daar heb ik in dit
interview al een behoorlijk beeld van gekregen. Zijn er nog dingen die we niet hebben
aangesneden waarvan je zegt dat kan ik nog wel meegeven?
Single of multiple points of entry voor 1 database of meerdere databases, zijn twee heel
verschillende dingen. In principe heeft een ERP een enkelvoudige database en multiple points of
entry. En als je dat goed configureert dan is dat heel krachtig. Op het moment dat je het niet
goed configureert of je zit met meerder databases, dan heb je dat probleem door en dwars in
huis. Dan heb je geen voordeel van een ERP systeem. Hoe eenduidiger je ERP geconfigureerd is,
148
hoe beter je controls en hoe minder risico’s en hoe minder audit. En andersom als je
complexiteit gaat toenemen, dan neemt dat exponentieel toe. Translation: In fact an ERP has a
single database with multiple points of entry. If you configure that correctly then it’s really
powerful. If you don’t configure that correctly or if you’re using more databases, then you have a
problem. Then you don’t have the full advantage of an ERP system. The more unambiguous
you configure the ERP, the better the controls are and less risk and less audit. And the other way
around, if you increase complexity, then that increases exponantially.
Daar staat tegenover dat de gedetailleerde audit planning kan veranderen, omdat de data entry
gebeurd op verschillende locaties en dat kunnen fysiek heel verschillende locaties zijn.
Absoluut. En de data entry op zich is niet zo’n punt, maar de bevoegdheden bepalen of je wel of
geen zorgen hebt. De bevoegdheden moeten in lijn zijn met de soort functie, maar je ziet vaak in
ERP systemen dat de rechten die aan bepaalde figuren wordt toegekend, veel hoger zijn dan dat
je voor zo’n rol zou mogen verwachten. Waardoor ze veel meer kunnen in het systeem dan dat
ze zouden mogen kunnen.
En dat zijn dan dingen die je gaat controleren?
Absoluut. Terecht dat je zegt, je neemt dingen aan voordat je gaat controleren. Dan ga je dingen
vaststellen en op basis van die vaststelling moet je je koers veranderen of niet. Als jij dingen
tegenkomt die je niet had verwacht en die het risico aanzienlijk verhogen, dan zal je meer
onderzoek moeten doen. Dat zou ertoe kunnen leiden als je toch maar twee weken hebt, dat je
andere dingen laat liggen.
Dan ga je op dat moment eigenlijk weer een risico analyse doen en dan zeg je dit risico is
dusdanig groter dan iets anders?
Ja. Daar heb je ook elkaar voor dat je even moet sparren van ik loop hier nu tegen aan, wat vind
jij? Moet ik hier op door of moeten we dat toch maar pakken.
Berust dat dan niet teveel op de mening van een auditor?
Uiteindelijk heb ik daar dan de eindverantwoordelijkheid voor. De auditor kan mij dan bellen en
vragen: wat vind jij ervan? Wat zal ik doen?
Uit mijn interviews lijkt het dat de risico analyses heel erg geïnitieerd worden door de ideeën van
de internal auditors.
Nou, je probeert het te objectiveren. Maar de beperking van weten en de beperking van niet
weten, is het grote probleem waar je tegenaan loopt. Dus in die zin is het altijd subjectief. Tenzij
je een systeem hebt, waarbij je alles kunt objectiveren. Als je als team de verkeerde kant opkijkt,
dan kan je een risico finaal missen. Translation: Well, you try to objectify it. But the limitation of
knowing and the limitation of not knowing, is the big problem you encounter. So in that sense
149
it's always subjective. Unless you have a system, whereby you can objectify everything. If you as a
team the wrong way, then you can overlook a risk completely.
Denk je dat je met een ideaal ERP systeem dat je veel aan de input kant gaat controleren? Dat
het aantal samples gaat toenemen?
Juist niet. Als het ERP goed geconfigureerd is, dan kan je eigenlijk met een paar proces audits
volstaan. Dan hoef je eigenlijk veel minder waarnemingen te doen, omdat de sampling eigenlijk
een bevestiging moet zijn dat het systeem is goed geconfigureerd. Dus daar kan ik op steunen.
Dan is de aggregeerde informatie die het systeem oplevert, veel waardevoller. Translation: If you
configure an ERP correctly, then a few process audits can suffice. Then you have to perform
fewer checks, because the samples should only confirm that the system has been configured
correctly. So I can rely on that. Then the aggregated information which is system supplies, has
more value.
Je data analyse wordt eigenlijk makkelijker, waardoor je makkelijker de uitzonderingen eruit kan
halen.
Ja, bijna integraal als zij het goed gedaan hebben. Dat is dan ook controllers werk dan. Of
revenue recognition, als je dat met sampling moet vaststellen, kan het zijn dat je net in de
verkeerde sample zit te kijken. Als je dat uit een systeem kan genereren, als het ware overdoen,
dan weet je zeker dat het goed is.
Daar kan je allerlei signalen voor laten maken.
Maar dan moet het systeem dat wel toelaten.
Mocht je de komende dagen nog dingen hebt die je wilt toevoegen, dan hoor ik dat graag. Ik ga
dit uitschrijven en dan stuur ik dit door ter review.
150
8.11 Appendix XI: Interview #9
As you know I’m doing my thesis about audit planning and this information will help me in that.
What can you tell me about your role in the organization?
My role within TeamMate is product manager. So I work as part of the product management
team. Interviewee #6 is the director of product management and I kind of work alongside
Xxxxx, who is the other product manager for TeamMate. My role as product manager of
TeamMate is to work with our development teams to understand the market problems our
clients have. The challenges they have in their day to day work. Where they need a solution.
Where we can make their life easier. We take those ideas and we take them to the development
team to develop with them solutions. That’s how we work. How we evolve the TeamMate
products. So as part of that I’m going out and speak to clients. I’m speaking to prospects.
Listening to them. Asking them question about what they’re doing and basically feeding that
information back to the development team. Coming up with high level ideas about features or
about products. What we think would be successful in the market. I work with interviewee #7,
#6 and also the vice president of development on the long term roadmap for TeamMate. What
the shape of the product would look like over three years. And on a more short term level I walk
through the features that would be in the one or two next development cycles.
That sounds like a really interesting role.
Yes, it is. It’s probably the most exiting job I’ve ever had.
How did you grow into this job?
I started to work with Wolters Kluwer and TeamMate 8 years ago, based in the London office.
Working as an implementation and training consultant. I helped TeamMate clients to implement
the system. I trained their auditors on how to use it. And from there I moved into a role of
business analyst in the development team in Tampa. That was 5 years ago. And from there a
combination of the company growing and me being successful in the role, I was promoted up to
a product management position. Prior to TeamMate my background was actually as a diving
instructor. That’s how I got in training. After 4 years of doing I knew it wasn’t for life. I thought
that a career in IT training might be interesting. That was the transition into IT training. So not a
conventional way to grow into this role. Before joining Wolters Kluwer I worked as a consultant
for a small software house in the UK. More accounting software or Enterprise Resource
Planning software for small to medium size distribution businesses.
Do you have experience with other planning systems such as TeamMate?
Not really, no. Most of my professional experience has been focused on TeamMate.
What would you say is typical about and ERP system?
151
It gives you a variety of tools to manage the business. It could cover a lot of things, because you
could go all the way from software for small to medium sized businesses all the way through to
Oracle, SAP, that kind of level of ERP software. The application I worked with was for much
smaller businesses.
So you’re familiar with several modules to be integrated into one database?
Yes. Not an expert, but familiar with it.
A feature of an ERP system is that there is only one database used. And that the data entry is
done only once opposed to an environment which has several systems which are using various
databases. If you’re taking a look at the difference between using an ERP system or not, so
having the various databases, do you think that it impacts the data quality?
Having multiple databases would increase the potential of having mismatches in the data. Just
because you’re entering the data in different places. There’s an opportunity for human error.
When entering the data in different applications, from a high theoretical level of how they are
being used, because I think there could be cases where you have different applications, but they
have fixed exchanges between them, which would mitigate that risk.
Like using interfaces?
Yes. There might be a way to passing data back and forth. Then it wouldn’t impact the quality of
data.
What you also mentioned is that the data could be entered in various locations. Between
different countries there might be different perceptions of how the data should be recorded in
the database. Do you think that would be an issue using an ERP system?
It could be. If you have people in different locations and entering data into a database there
could be misunderstanding or having different understanding of what data is required or what
the meaning of something is. There are ways to mitigating that risk by configuration of a system.
So setting up required fields and input masks, things like that.
Do you think in an ERP system, while sharing one database, someone who is processing
purchases is not necessarily a finance person, but the work of that person is rolling up into the
financial statement in the end. Do you think that could be an issue?
Potentially it could be an issue. Because there is someone who doesn’t have maybe the same
knowledge or understanding of the downstream use of that data, but those kind of things can be
mitigated by setting the system up to reduce the possibility of user error.
So in the same way as the various locations may cause a problem?
Yes. Normally you would configure a system so that the end user of the application has to do as
little as possible or has to think as little as possible about what they’re doing. So you’re trying to
152
automate the workflow, to setup required fields or next steps and you have different levels of
access to features of different levels of complexity. So maybe where there is more opportunity
for a mistake or the feature is more complex, you restrict that to a smaller group of users.
Usually I start asking questions about audit, but you were starting about ERP systems and so I
continued on that. What is your knowledge about audits?
My knowledge of auditing, more specifically internal auditing, is 100% learned on the job,
working with TeamMate. When I started out at TeamMate I didn’t know what the day to day
tasks of an auditor are, or what their processes are. That was learned initially with some reading.
When I joined and was working with clients and was asking questions.
And most likely you visited and had contact with a lot of clients.
Right. One of the advantages when I was first working at TeamMate, was as an implementation
training consultant. So my day to day work was go out and meeting with clients on sight and
helping them, guiding them through the configuration of the system and training them on how
to use the system. And they would tell me what they needed to do as auditors and I would
translate that into how our application works. So I could teach them and they would have value
from my presence and at the same time I was learning from them. In those years I’ve had the
opportunity to work with lots of different companies ranging from multinationals through to
smaller organizations in the UK. But I also did some work for clients in Europe and in the
Middle East. So I’ve had a real variety of clients in industries and that is where my knowledge of
internal auditing came from.
Interesting. Do you see any difference in the various areas? Like you mentioned you’ve visited
some customers in the Middle East, some in Europe and in the US as well.
Actually not so much difference in how auditors use our applications. There are different degrees
of maturity. Bottom line the processes are the same. So the institute for the internal auditors
what is kind of the governing body for internal auditors. They set the same standards for internal
auditors globally. So where you see big difference is between clients which we have in the
government sector, they follow slightly different standards versus clients in the public sector.
When you go an actually speak to an individual client, the words they use, the auditing they do is
different, so the words that they put into the application, which they type in the text fields. The
framework and the process is fairly consistent. There’s not a huge amount of variety. They will
do some kind of risk assessment to determine an audit plan. They will work out if they will have
resources to complete the audit plan and they will send that for approval. On an individual audit
basis, they will go through a planning phase and determine the scope of the audit and the areas
that they will focus on. They’ll go through the field work phase where they will carry out testing
153
and document results. They’ll do an audit report and there will be a wrap up phase where they
document issues and write a final audit report. They will then go on and monitor the issues or
the recommendations that they’ve made until they have been implemented. And that’s pretty
common. There is not a huge amount of variation on that, because that’s really what is required
by the Institute of Internal Auditors.
Interesting to hear that the main differences are by industry.
I wouldn’t even say that it’s by industry, but as far as for TeamMate, because it’s the first audit
software and it was adopted by government auditors. They are more like external auditors in
terms of the work that they do. So they have slightly different processes. But aside from
government auditors, about every other client that we have no matter what industry follow
almost the same process. So the work might be slightly different and certainly the actual testing
will be different. Like a chemical company would have very different audits as from a bank. And
that’s what I mean by the words that they’re typing. The actuals tests that they do, the content
within our application, will be different, but the overall process and workflow would be fairly
common between a bank and a manufacturing company.
That’s good to hear, because my research is about the planning process and that it can be applied
for all types of industry.
When you say planning, do you mean the planning for an individual engagement? Or is it the
process of determining an audit plan? The term planning could mean both of those things.
Actually it’s a bit on both. What do you see as the main goal of an audit?
I would say the main goal of an audit is that the chief audit executive gains an understanding as
to how certain parts of the business or a certain process of the business works. Ideally to gain
assurance that it’s working or that things are as they should be, but if they are not that they
identify those issues, identify problems that might impact the business. And they work with the
management to put in place a process for remediating them.
As you mentioned the audit process has two parts; the yearly or high level audit planning and the
detailed audit planning with the actual testing.
Yes. That’s a good way for formulate it. Maybe engagement planning versus determining the
audit plan.
How does the process of engagement planning work?
If you’re at the engagement planning stage, there has already been a risk assessment. That risk
assessment has been done over a part of the business. It could be a process. It could be a
business unit. It could be a physical location. There’s an area in the business that needs to be
audited, because there’s deemed to be some kind of risk. Risks in that the business won’t be able
154
to meet it’s objectives. There has already been some work done, mostly by people within the
audit team to determine that an audit is going to take place. So there would be some
understanding of the risks. Maybe some understanding of the controls that exist. The audit team
would then start by doing some initial research on that area of the business. Some might look at
previous audits of the area. It might be looking at information available internally about that part
of the business. It could be looking at information from outside of the business, like what is the
overall market like, what is the economy like, competitors, that kind of general background
information. That would be the initial background work. There would be a notification letter
going out to the auditee to let them know that an audit is planned. There might be some requests
for information there. That might be a request for availability for key personnel to be
interviewed. They would request an opening meeting. They would request for more documents
particularly at that stage. Normally at some point during the planning process, there will be a
control walk through. So the auditor would identify the controls which are in place in that part
of the business. What types of controls they are. Who the control owners are. And together
decide within that part of the business which of those areas are they most concerned about.
There’s a high level risk assessment, which is done to determine whether an audit takes place.
And once it’s been determined that an audit will take place, the audit team would then look at
the more granular risks that might effect that part of the business. And the real controls that are
in place. And from there they then determine what areas of the business do we want to look at.
That’s the process that they’re going through to see what testing is going to be done.
In order to specify which areas contained a high risk to ensure that those risks are mitigated
either through controls or through the test itself?
Yes. Or to find that they’re not mitigated and that there are potential issues. And therefore to
help the business to put processes or controls in place to mitigate them.
So you take a look at the level which is coming out of the risk assessment, right?
I would differentiate between the annual risk assessment and the risk assessment that takes place
during engagement planning. It’s not necessarily the same risk assessment. They’re not
necessarily the same risks. For an internal audit department to exist, the organization needs to be
a large organization by definition. Small businesses don’t have internal audit. And so they tend to
be large complex businesses and so when they’re doing a risk assessment it’s not always possible
always to do a very detailed granular risk assessment, like to pick out specific risks that would
apply to one business unit. So they tend to use high level strategic risks to determine the audit
plan. It also helps to get a apples to apples comparison. If you compare different kind of risks,
then it’s difficult to determine the risk score. This is not always the case, because the risk
155
assessment methodology is probably the one area where there is the most divergence between
different organizations. But largely what we see when we go out and talk to clients about their
annual risk assessment is that they have the same common high level strategic risks that they
consider for each entity and they use the scores for those risks to determine the audit plan. But
when they actually go in to do an audit in a particular area, they’re looking at real business and
they’re looking at what the real risks are. For example business continuity is a high level risk. So
every part of Wolters Kluwer would have some sort of business continuity plan. That would be
considered for every business unit. But if you’re going to do an audit with TeamMate in Tampa,
the specific risks that might occur may simply have to do with the hurricane. There might be
different controls in place to mitigate that risk versus the risk WK FS in Minneapolis where the
risk might be that the offices close because of snow. A more detailed business continuity risk.
With different controls and different risk ratings. At the annual risk assessment level you might
look at those two business units and just consider business continuity risks. But when you might
be doing an audit of those specific areas, you might be looking at different controls, because the
real risks that are faced on the ground are different.
So at a high level you take a look at the risk assessment, taking the same risk assessment items
for each individual entity and then you start comparing them. And once you are inside an entity
and you make a risk assessment there, then you’re going more into detail in to take a look which
areas have the higher risks and you take a look at how it’s mitigated.
Yes. And I should say that there is a difference between audit theory and actually what auditors
do. Audit theory or audit best practice is when you do your annual risk assessment, you do look
at real risks that are faced by that specific part of the business. So in theory at least there
shouldn’t be a difference between the annual risk assessment and the engagement planning risk
assessment. In practice, because of the size of businesses and the complexity, in order to do the
work effectively with the resources at hand, auditors tend to do a more high level strategic risk
assessment for the annual planning. And then a much more detailed real risk assessment for the
engagement planning.
Risk assessment is something I got to know from the Coso model. Is the Coso model used in
audit planning?
Yes, I think it’s now a requirement from the IIA to use Coso to do the risk assessment. We’ve
seen since the last update of the Coso framework it’s been used within organizations for the risk
assessment process.
What I am researching is how the use of an ERP system is impacting on the risk assessment of
an audit planning. How do you think an ERP system is impacting and audit planning?
156
And ERP system, because it has workflow and controls automated, so they don’t require human
intervention work. So if an audit takes place to see if the ERP system is configured correctly, it
gives good assurance that the controls are working correctly and things like fraud and other risks
are well mitigated.
If you don’t have an ERP system, but you’re using interfaces for example, how do you think that
impacts the risk assessment?
If I was an auditor, then I would think that’s another point of potential failure. So maybe that’s
an area where there might be risks and that would be something I would want to spend more
time and resources on.
Do you see any issues in how people access the database?
That’s one of the main control points, I think in an ERP system. It’s segregation of duties and
what level of access a user has to the system. That would be one of the main controls.
If you have multiple databases for example and they’re interfaced or there’s double data entry, at
least you can compare the two databases. You can compare if they have the same values. And if
they don’t you can investigate. So that’s a control method.
Yes, but maybe that’s a more costly control method than an automated one if you have a single
integrated system. But having two databases it adds another point of failure and that might be of
interest when you’re auditing that area.
But opposed to that, if you’re using an ERP system, the data is entered only once and if it’s
incorrect, no one will ever tracks that. There’s no normal control for that.
That is true. I think it would depend on the system and the types of data. There might be internal
controls to help to mitigate that. If you’re entering a value to a single database and that value is
being used in different areas for different reasons and if that data is incorrect then that’s
propagated throughout the system.
How would that impact your risk assessment?
If I were an ERP expert doing an audit, then that would be something I would want to test in
some way. Or ideally find a way of ensuring there is an automated control in place to go against
the possibility that bad data gets into the system.
So using some logical controls?
Yes.
Do you imagine that the samples of the input test would increase?
I think the advantage to this type of testing is that you don’t need to limit yourself to a sample
size. There are tools now that auditors can use where they can use analytics to test an entire data
set. So they don’t necessarily have to rely on small samples of data to gain assurance. Ideally or
157
what is the trend in the industry is towards empowering the business so that they can have their
own controls in place, so towards continuous monitoring. It’s been talked about a lot, but I don’t
know in how far it’s been implemented widely across organizations. Certainly the more forward
thinking audit shops. There has been an interesting presentation on an audit conference, earlier
this year, where they were talking about there’s definitely a push towards more audit shops to
have more people on staff who have data analytic skills and also the tooling to support it.
Essentially they will write scripts to do this kind of testing and to automate it. The audit teams
will work out what kind of questions they want to answer. What are they trying to test? What
data do they need? They’ll setup an automated way for doing that and also set that up and hand
it over to the business. Then the business will take ownership of it. So the business can have
these controls in place to make sure that common things like expenses are within company
policy. The audit team then doesn’t necessarily have to come in and do that testing themselves.
They can rely on the work that has been done by the business unit itself.
How do you think that the risk assessment is used in this type of analytics?
I think the analytics is more used at the testing side of things. You would have done the risk
assessment and identify the risk and presume the controls that are in place to mitigate that risk.
You would use analytics to verify the operations of those controls. So analytics is more on the
testing side then on the risk assessment side.
So now that you know what my research is about; how an ERP system is impacting the risk
assessment as part of the audit planning. Are there additional things you would like to add?
Nothing that comes to mind.
If there will be something in the upcoming weeks, then feel free to drop me a line or a mail. I
will transcribe our discussion and I’ll send it to you for review.
158
8.12 Appendix XII: Interview #10
Ik doe een onderzoek naar audit planning. Aan de hand van de expertise van een aantal interne
auditors en mensen die werken aan een audit tool genaamd TeamMate, denk ik antwoorden te
vinden op mijn onderzoeksvraag. Ik heb eerst wat introductievragen. Wat kan je me vertellen
over jouw rol in the organisatie?
Ik ben verantwoordelijk voor internal audit en voor risico management en de functie van
compliance officer.
En hoe lang zit je al in deze rol?
Internal audit 7 jaar. Risico management ongeveer 4 jaar. Compliance officer 1 jaar.
Is dit allemaal in de zelfde organisatie?
Dit is allemaal binnen USG people. Daarvoor heb ik 7 jaar gewerkt bij KPN, waarvan 4 jaar
audit en 3 jaar in de business. En daarvoor heb ik auditing consultancy werk gedaan bij KPMG.
Daarvoor bij justitie en daarvoor heb ik geen audit werkzaamheden gedaan of consultancy werk,
maar heb ik bij Vluchtelingenwerk gewerkt en daarvoor als market maker op de optiebeurs.
Dat klinkt als een brede ervaring binnen internal audit. In je audit ervaring heb je wel eens
planning programma’s gebruikt zoals TeamMate?
Ja, ik heb dat soort programma’s wel gebruikt. Dat is nuttig als je met grotere aantallen auditors
werkt. En als je niet zo’n grote groep hebt, dan draagt dat wel iets bij, maar niet heel veel.
Dus het is meer een houvast en niet zo zeer een leidend iets dat je door het proces heen helpt?
Ja.
Wat zie je als het hoofddoel van een internal audit?
Het bieden van extra zekerheid aan de raad van bestuur.
Dat ze niet ’s nachts wakker liggen.
Of wel, dat ligt eraan wat je ze gaat vertellen natuurlijk. De raad van bestuur wordt geïnformeerd
door de eerste lijn en de tweede lijn, en die informatie kan verkeerd zijn of onvolledig. Door het
uitvoeren van een audit, door een objectieve onafhankelijke partij vanuit de derde lijn, krijgt de
raad van bestuur een onafhankelijk beeld van bepaalde situaties. En dat vind ik een toegevoegde
waarde van internal audit. Het biedt een extra zekerheid aan de raad van bestuur of aan de audit
commissie, omdat ze onafhankelijk onderzoek doen.
Hoe geven ze dan die extra zekerheid?
Je doet een onderzoek, t bijvoorbeeld naar systeembeveiliging. De staff (1ste en 2de lijn) is
verantwoordelijk dat dat het allemaal goed geregeld is. En die zeggen tegen de raad van bestuur
het ziet er allemaal mooi uit. Maar dat kan die staff well zeggen, want die keurt zijn eigen vlees als
het ware. De internal audit afdeling kijkt dan in welke mate de opzet van het IT
159
beveiligingsbeleid en de uitvoering daarvan ook goed is. En daarmee geven ze de raad van
bestuur extra zekerheid dat het beveiligingsbeleid goed is opgezet en goed wordt uitgevoerd en
nageleefd. Dus in die zin geeft een internal audit een extra zekerheid aan de raad van bestuur
over het onderwerp dat je onderzoekt.
Hoe ziet volgens jou het audit proces er uit?
Globaal bepalen van doel en scope. Vooronderzoek om doel en scope verder af te bakenen.
Informatie verzamelen. Dan veldwerk, daadwerkelijk je onderzoek doen. Het concept rapport is
de eindstap van het veldwerk. In de rapportagefase ga je de audit finaliseren: afstemmen wat er
gebeurd naar aanleiding van je bevindingen en je conclusies. Als laatste stap in deze fase breng je
het rapport uit. Dan is het evalueren met de auditee en de opdrachtgever. En daarna een interne
evaluatie en dan afsluiten van de audit. In grote lijnen.
In het audit proces dat je net beschrijft. Wat denk je dat een essentiële stap is?
Elke stap is essentieel. Voor de kwaliteit van je uiteindelijke rapportage en het effect van je
rapportage. Vaak is het begin het meest belangrijk dat je zeker weet dat je de goede dingen gaat
doen. Dus als je doel en scope niet goed is, onderzoek je het verkeerde. Translation: Often the
beginning the most important that you are sure you are going to do the right things. So if your
goal and scope is not good, you research the wrong things. Dan kan je nog zo’n mooi rapport
maken, maar als het over het verkeerde onderwerp gaat of verkeerde diepgang, dan heeft het
minder effect dan het had kunnen hebben.
En hoe zorg je er dan voor dat je naar het juiste kijkt?
Door echt goed met je opdrachtgever en de auditee en je ervaring, heel goed praten met je
opdrachtgever wat hij wenst. Samen met de auditee de opdrachtomgeving heel goed te bekijken
wat hij er van vindt. En met je jarenlange ervaring en kennis een keuze te maken en dat
voorstellen aan je opdrachtgever om de opdracht zo in te richten.
En wordt het Coso framework daarin gebruikt?
Nou, nee. Het Coso framework, je kan dan de terminologie uit die kubus gebruiken, maar vaak is
het makkelijker om daar eigen woorden te gebruiken. Dat hangt af van de beleving van de
mensen. Je kan wel bij de afdeling marketing het woord control environment noemen en de
focus punten, maar dan moet ik eerst gaan uitleggen wat Coso bedoelt met focus punten. Het
liefst gebruik ik wel de informatie uit Coso, maar gaan we het niet zo expliciet benoemen, want
dan wordt het onduidelijk voor de mensen die daar geen kaas van gegeten hebben.
Een van de aspecten uit het Coso framework is risk assessment. Zou je wel een soort van risk
assessment gebruiken.
160
Ja, er zijn twee dingen: 1 risk assessment voor het vaststellen van je jaarplan, waarin de audits
staan die je dat jaar gaat uitvoeren. Daarna heb je bij het plannen van een individuele audit dat je
gaat kijken naar wat zijn de risico’s waar we naar gaan kijken binnen het doel en scope van deze
audit. Dat je dan je referentiemodel maakt.
Referentiemodel is voor mij nog een nieuwe term.
Als auditor ga je er iets van vinden. Maar om ergens iets van te vinden, moet je een norm
hebben. Een maatstaf en een norm. Anders kan je wel je persoonlijke mening geven, maar dat is
geen audit. Dus als je het theoretisch netjes doet, zorg je dat je een maatstaf en een norm hebt op
basis waarvan je dan kan zeggen het voldoet aan de norm of niet. Door alle normen goed uit te
werken en in een model te voegen vorm je een referentiemodel. Het model waaraan je refereert.
Duidelijk, dank je wel. Hoe werkt de risk assessment voor je jaarplanning?
Vragen aan mensen in de business wat zij belangrijke risico’s vinden. Vragen aan de raad van
bestuur wat zij belangrijke risico’s vinden. En binnen de audit afdeling bespreken wat de
belangrijke risico’s zijn. En de ervaringen van het verleden en de uitkomsten van audits. Die gooi
je allemaal in een hoge hoed en daar maak je een prioriteitenlijst van. Die prioriteitenlijst leggen
we voor aan de raad van bestuur. Die zegt dan dit vinden we een goede risico analyse of dit en
dit willen we anders. En op basis van die risico analyse maken we een audit planning. Die wordt
dan ook goed gekeurd door de raad van bestuur. En vervolgens leggen we de risico analyse plus
het audit jaarplan voor aan de audit commissie en die keurt het dan goed.
Dan ga ik ervan uit dat bij die risico analyse dat de hoogste risico’s dat je die het meest naar
voren laat komen als de hoogste prioriteit om te gaan onderzoeken.
Ja, als internal audit afdeling willen we de belangrijkste risico’s tackelen. Soms is het zo dat de
raad van bestuur of de audit commissie prioriteiten anders leggen voor redenen die vanuit hun
perspectief van belang zijn.
Dan krijg je de goedkeuring voor wat je dat jaar gaat auditen. En dan ga je in overleg met de
entiteiten die je gaat auditen?
Ja, die informeren we dan dat we de audit gaan uitvoeren. Als het goed is dan is het audit plan in
oktober goedgekeurd en als het wat langer duurt in december. Standaard kondigen we dan aan al
die entiteiten aan dat we komen auditen. En een aantal weken voordat we er naartoe gaan nemen
we contact op voor een planning van dan en dan komen we langs. Dan hebben we een gesprek
met de auditee en dat kan fysiek zijn of telefonisch. Dit en dit gaan we doen. Hoe kijk je er
tegenaan? Wat is voor jou belangrijk? Wat moeten we in die audit voor jou meenemen, zodat het
interessant voor jou wordt? En dan maken we de detailplanning en dan gaan we aan de slag.
Zit er in die discussie dan ook een soort van risk assessment?
161
Ja, in het vooronderzoek moet je kijken van wat zijn de belangrijkste risico’s van het audit object.
Zodat je daaraan je referentiemodel opbouwt, zodat je dat goed kan toetsen.
Dan kom je uiteindelijk op de audit zelf aan?
Ja, dit hoort wel allemaal bij een audit, maar dan ga je wat ze noemen het veldwerk doen. Dan ga
je op locatie onderzoek verrichten. Met mensen praten, data analyseren, rapporten lezen op basis
van wat je in dat referentiemodel hebt vastgelegd wat je zou moeten gaan onderzoeken.
Dit is een duidelijk verhaal over het audit proces. Een groot deel komt overeen wat ik eerder heb
gehoord en er zijn een aantal punten die het audit proces mij wat scherper stellen. Daar ben ik
erg blij mee. Dan een ander onderdeel waar mijn onderzoek over gaat zijn ERP systemen. Ik
neem aan dat je bekend bent met ERP systemen?
Ja, wat bedoel je met ERP systemen, want er zijn veel gedachtes over?
Met een ERP systeem is eigenlijk dat zo veel mogelijk onderdelen van een proces geïntegreerd
zijn in 1 systeem en dus gebruik maken van 1 centrale database. Wat daar dan verder typisch van
is, is dat de data maar op 1 punt in het systeem wordt gebracht. En dat daarbij ook de data wordt
ingebracht door verschillende afdelingen op verschillende locaties en dat de communicatie dus
ook steeds meer plaatsvindt via het systeem. Er tegenover staat het geheel niet geïntegreerde
systemen, dan krijg je dat als er iets ontvangen wordt in het warehouse, dan gaat er een briefje
naar accounting en dan toetst accounting dezelfde informatie nog een keer in, maar dan in het
accounting systeem. Daar gaat het andere deel van mijn onderzoek over. Eigenlijk de omslag
tussen niet ERP systemen en ERP systemen. In hoeverre dat impact heeft op de risk assessment
of de audit planning zelf. Heb je daar in eerste instantie al een idee over?
In mijn praktijk heeft dat geen invloed. Translation: It does not have an impact in my experience.
Dus het heeft geen invloed op de risk assessment?
Wij werken in Nederland bijvoorbeeld met SAP. En theoretisch zou het kunnen zijn dat als je
met zo’n systeem werkt dat het risico dan minder is. Alleen ligt het er aan hoe je de risico analyse
doet. Wij kijken meer wat zijn de belangrijkste risico’s en dan kan het zo zijn dat als je een goed
IT systeem hebt, dat het risico wat minder is. Maar dat is maar 1 van de factoren. Translation:
And theoretically could it be that if you work with such a system the risk is less. It is only how
you do with the risk analysis. We look more at what the main risks are and then it may be that if
you have a good IT system, that the risk will be less. But that is only one of the factors. En bij
ons is het IT landschap dusdanig verspreid dat het impact van ERP niet heel groot is. Dus voor
ons speelt dat niet een belangrijke rol.
Dan wil ik graag een paar dingen bij je voorleggen om te zien wat jouw gedachtegang daarbij is.
Ten eerste op de jaarlijkse risk assessment en de jaarlijkse audit planning, zie je daar een impact?
162
Van ERP systemen? Als je een ERP systeem hebt, dan vormt dat een groot onderdeel van je
organisatie en je risico. Dus dat is een onderdeel dat je gaat bekijken. Translation: If you have an
ERP system, then that’s a big part of your organization and risk. So that’s the part that you will
be looking at. Afhankelijk van het risico dat je ziet aan het systeem aan de ene kant. Aan de
andere kant, als het ERP systeem goed werkt, dan kan het zo zijn dat je zegt we hoeven er
minder naar te kijken. Translation: if your ERP system functions well, then it might be that we
have to review it less.
Wat ik me voorstel is dat als een systeem werkt en er wordt niet aan gesleuteld, dan blijft het
goed werken. Dus dan hoef je alleen nog maar een paar samples te testen om te zien dat het nog
steeds zo werkt. Dan zou het impact groot kunnen zijn, maar omdat je ervaring erin hebt, wordt
in het systeem je risico beperkt.
Ja, zeker theoretisch klopt dat.
Daarentegen als je op gedetailleerd niveau, op auditee niveau, gaat kijken, zijn er misschien een
paar punten dat het wel impact kan hebben op je audit planning. Bijvoorbeeld dat de data in het
systeem worden gezet door non financials, maar rolt hun data entry wel op in de financial
statement. Zie je dat als een risico richting de audit planning?
Ja en nee. De veronderstelling was dat je systeem goed was en dat je maar een paar testjes moet
doen, want dan is het goed. Dus dan heeft het er geen effect op. Als je wilt weten of het ERP
systeem betrouwbaar is, de basisregistraties betrouwbaar zijn, dan zal je daar onderzoek naar
moeten doen. En wat is de opzet van de inrichting op operationeel niveau. Dus iemand die
inkopen registreert moet keurig conform de AO/IC regels dat doen en er moet controle zijn op
de invoer of een 4 ogen principe in welke vorm dan ook om te kijken of dat gebeurd. Je zal toch
periodiek als internal audit afdeling moeten kijken of netjes de opzet, het bestaat en werking
functioneert. Want als de registratie niet goed is van de basisgegevens in je ERP systeem, dan kan
je nooit steunen op de informatie die eruit komt. Dus het blijft altijd een aspect van onderzoek.
Vandaar dat ik zei dat in theorie is dat zo, maar je moet kijken in praktijk hoe dat gaat. Alleen al
bijvoorbeeld de rechten die aan mensen worden gegeven die toegang hebben tot het systeem.
Theoretisch kan je dat met toegangstabellen allemaal organiseren, maar in welke mate wordt er
bij het toekennen van rechten strikt de regel gevolgd? Een manager moet autoriseren, maar let hij
daar wel voldoende op dat hij de juiste mensen de juiste rechten geeft en niet te veel of de
verkeerde. Dat zijn een aantal aspecten die wij als auditor er wat van zouden moeten vinden.
Translation: For example the rights that are given to people who have access to the system.
Theoretically you can organize it all through access tables, but to what extent are granting rights
strictly followed? A manager should authorize, but does he pay enough attention to give the right
163
people the appropriate rights and not too much or the wrong ones. Those are some aspects
which we should find some of it as an auditor.
Dan komt de nadruk op de segregation of duties?
Dat is een belangrijk onderwerp.
Denk je dat er dan ook een toename is in het onderzoek waar de data entry plaatsvindt? Of dat
de samples groter worden?
Dat ligt er een beetje aan. Naarmate het systeem groter is en belangrijker, naarmate er meer data
wordt ingevoerd, zal je steekproef groter worden, omdat de populatie groter is. Als er minder
frequent en minder aantallen in te voeren is, dan hoef je minder grote steekproeven te nemen.
Als je dagelijks honderden registraties hebt, dan zal je steekproef ook groter moeten zijn. Dat is
gewoon een kwestie van statistiek. Afhankelijk van de omvang, frequentie en van het risico. Die
drie dingen spelen dan een rol.
Als ik daar tegenover stel dat als je geen ERP systeem hebt, dan heb je verschillende databases en
doordat je de verschillende databases kan vergelijken, heb je ook een controle methode.
Je kan dingen met elkaar vergelijken, maar de vraag is dan wat dan je norm is. Welke database
wordt dan als goed gezien?
Eens, maar als je afwijkingen ziet, dan kan de controller van die entiteit daar induiken. Terwijl als
je een ERP systeem gebruikt, met 1 database, dan heb je die controlemethode niet.
Eens.
Zie je dat als een impact op je risico analyse?
Dan vergelijk je ERP omgevingen met niet ERP omgevingen. Je doet de risico analyse op je
object van onderzoek. Dus als een ERP systeem mijn object van onderzoek is, dan heb ik 1
database en dan moet ik zeker weten dat de invoer goed is. Als mijn subject van onderzoek 2
databases zijn, dan moet ik nog steeds zorgen dat die invoer goed is. Ik moet zeker weten dat wat
er in die databases staan, dat dat klopt. Dus ik moet toch elke database bekijken of die klopt.
Tenminste dat de invoer goed is. Want als die niet goed is, dan heb ik er zo weinig aan.
Begrijp ik het dan goed dat je dan twee databases hebt, waarvan je de data input moet
controleren en daardoor eigenlijk twee samples krijgt van de input die je moet controleren?
Wacht even, want we raken in een spraakverwarring. Je hebt twee databases en deels staat daar
dezelfde informatie in. Deels zou daar dezelfde informatie moeten instaan. En wat is nou precies
de vraag?
Als je twee verschillende databases hebt, dan heb je ook een controlemethode, doordat je de
twee databases met elkaar kan vergelijken. Dan kan je data input controleren. Terwijl als je een
164
ERP systeem hebt, de data entry op 1 punt gebeurd en dat je eigenlijk geen controlemethode
hebt om te controleren of de data er goed in komt.
Maar de ene database staat op getal 8 en de andere staat op getal 9, welke is goed?
Dan weet je in ieder geval dat ze niet gelijk zijn. Dan weet je dat je daarop kan concentreren om
te controleren.
Dat is waar. Daarom is het van belang om te weten wat de kwaliteit is van de basisregistratie.
Dus als je een soort audit trail hebt, van je ERP systeem, dan zou je nog dingen kunnen isoleren.
Nu je weet waar het onderzoek over gaat, heb je misschien een punt waarvan je zegt misschien
moet je daaraan denken?
Wat ik wel moeilijk vind is die combinatie tussen audit planning en ERP systemen. Theoretisch
zou het zo kunnen zijn, dat als je ERP systemen hebt, en die worden getest, dat je daar dan als
internal audit op kan steunen. Maar af en toe zou je dat toch ook moeten testen. Want het ERP
systeem zelf is ook een complex systeem. Daar kan wel het nodige fout in zitten zonder dat
mensen dat merken en dan steunt iedereen op informatie waar ze eigenlijk niet op zouden
moeten steunen. Dus in die zin is het altijd wel een object van onderzoek. Translation:
Theoretically it could be, that if you have ERP systems, and tested, that you as internal audit can
rely on it. But every now and then you will have to test that. Because the ERP system itself is
also a complex system. There can be errors in it without people noticing it and then everyone
relies on information which they actually should not rely on. So in that sense, it is always an
object of research.
Dus dan zou met name de implementatie of verandering in setups in systemen, zou een belletje
laten rinkelen in je risico analyse?
Zeker. Alle wijzigingen op dat gebied. Je hele autorisatie in je ERP systeem is belangrijk. Wie
mag wat? Wie heeft welke rechten. Vaak zijn die systemen dusdanig in omvang dat het… Dat
zijn wel complexe zaken waardoor je gespecialiseerde kennis nodig hebt om dat goed te toetsen.
Dus dan zou je de audit meer laten doen door mensen die gespecialiseerd zijn in ERP systemen?
Kan. Je hebt tooling nodig. SAP daar zitten tienduizenden tabellen in. Als je een goede data
analyse wilt maken, moet je wel de juiste tabel zien te vinden. En je hebt experts nodig die heel
veel verstand hebben van de technische structuur van die database om goed een opzet te kunnen
maken om dat te onderzoeken en ook de tooling te hebben om het te onderzoeken. Niet elke
auditor kan dat. Je zou eigenlijk een IT auditor moeten zijn met uitgebreide kennis om dat goed
te onderzoeken. En dan heb je ook de juiste tooling nodig.
Dat zou eigenlijk het grootste impact zijn op je audit.
Ja, dat komt altijd naar voren in je risico analyse dan.
165
Dat is interessant.
Als je bij grotere organisaties naar binnen kijkt dan moet de audit afdeling ongetwijfeld kijken
naar grote systemen. En dan is het de vraag wat ze daarvan bekijken.
Dat is wel een interessant punt waar ik nog verder naar ga kijken. Bedankt voor dit interview. Ik
ga het uitschrijven en dan stuur ik het je toe ter review. Mogelijke citaten zal ik vertalen naar het
Engels en je ook toesturen ter review.
166
8.13 Appendix XIII: Interview #11
Zoals ik al enigszins in de uitnodiging heb uitgelegd, ben ik een scriptie aan het schrijven over
audit planning. Om te beginnen heb ik wat meer introductievragen en later gaan de vragen meer
over audit planning. Als eerste, kan je me wat vertellen over jouw rol in de organisatie waarvoor
je werkt?
Ja, die is twee weken geleden veranderd. Ik zat bij internal audit en internal audit is onderdeel van
de afdeling audit & security en daarin zitten echt de bedrijfsrechercheurs die op pad gaan bij
fraudegevallen. Daar zit risk management en internal control onder als afdeling en daar zit
intenal audit onder als subafdeling en ik zat dat bij internal audit als senior auditor. Audit planner
en senior auditor. En dan in mijn rol ben ik verantwoordelijk voor bepaalde delen van de
organisatie waar ik dan van op de hoogte moet blijven en daar de informatie vandaan moet
krijgen om zo de kennis binnen de afdeling op dat onderdeel binnenshuis te houden en om het
uit te voeren natuurlijk. En dan gaat het om het plannen van de audit, het opzetten van de audit
en afwikkelen van de audit. Van de openings meeting tot aan de closings meeting aan toe.
En dat is wat je de afgelopen jaren hebt gedaan?
De afgelopen twee maanden ben ik overgestapt naar de controlling hoek. De andere kant van de
lijn zeg maar. Mijn verhaal vertel ik meer vanuit mijn vorige rol.
Hoe lang ben je al onderdeel van de organisatie waar je nu zit?
6 jaar.
Zoals aangegeven heb ik veel informatie geput uit TeamMate en je gaf aan dat je TeamMate ook
kent. Werk je veel met planning programma’s?
Nee. Wij hebben TeamMate en gebruiken dat voor internal audit, maar daar gebruiken we
eigenlijk alleen de database van. Niet de hele planningssectie. TeamSchedule en TeamRisk dat
gebruiken we allemaal niet. We gebruiken meer de working papers, ook om het in de database op
te slaan en om audits gewoon vast te kunnen leggen. Per audit maken we een dossier aan in
TeamMate en in ieder dossier behandelen wij onze bevindingen, onze documenten,
beschrijvingen en dat sluiten we af in de database en dat is hoe we TeamMate gebruiken. Voor
de planning gebruiken we gewoon Excel sheets. Om duidelijk te stellen: in audits heb je twee
soorten planning. 1 is planning wat ga je auditen en waarom ga je iets auditen en wanneer ga je
iets auditen. Wanneer en wie. En het tweede is echt per project planning per audit, waar ga je
heen, wie moet ik spreken, welke onderdelen hebben risico, hoeveel tijd ga ik er aan besteden.
Hoeveel samples neem ik, etc. Op welke spits jij je onderzoek?
167
Daar komen we straks op terug, maar ik ben me ervan bewust dat er twee niveaus zijn van
planning. Je jaarlijkse planning en je engagement planning. Ik ga me niet zozeer op een van de
twee toespitsen op dit moment.
Voor de duidelijkheid: wij doen de engagement planning in TeamMate en de jaarplanning in
Excel.
Wat zie jij als het voornaamste doel van een audit?
Het voornaamste doel is vaststellen of er risico’s zijn die de organisatie niet heeft afgedekt. En
dan zowel financiële risico’s als operationele risico’s.
Hoe zou je het audit proces beschrijven?
Je begint in eerste instantie met, je zou een reden moeten hebben om een audit uit te voeren. Dat
zou gebaseerd moeten zijn op risico analyse. Op het moment dat jij ergens een risico detecteert
of vermoedelijk risico hebt, waar jij wilt vaststellen of er voldoende interne controlemaatregelen
zijn om het risico af te dekken. Dan zou je daar een audit kunnen plannen. Dus het begint bij
risico analyse. Daarna ga je de scope bepalen; wat ga je afdekken. Welke risico’s wil je afdekken
met welke audit werkzaamheden ga je dat doen. Je moet bedenken hoe diep je moet gaan. In het
begin is dit nogal globaal. Op dat moment ga je de audit inplannen. Je gaat je werkprogramma’s
maken die aansluiten met je scope. Vervolgens ga je afspraken maken met de business. Je gaat de
audit uitvoeren, je bevindingen bespreken en erover rapporteren. Dat is globaal hoe ik het audit
proces zie.
Als ik het goed begrijp is het risk assessment het belangrijkste onderdeel?
Ja, dat is de reden waarom je de audit uitvoert inderdaad ja. Als je geen risico’s ergens hebt of
materiele fouten naar voren komen, dan zal je niet heel snel een audit uitvoeren. Dan praat ik wel
over een situatie waarin je de tijd goed kan besteden. Op het moment dat je tijd zat hebt in je
planning, dan ga je ook naar plekken waar het goed loopt, om te kijken of het daadwerkelijk wel
goed loopt. Maar ik neem aan dat niet veel bedrijven tijd over hebben. Dus wil je je tijd goed
besteden, dan doe je dat waar je denkt dat je risico loopt.
En dat is de reden waarom je een risico analyse doet, zodat je aan het pinpointen bent daar zijn
de grootste risico’s en daar moeten we onze aandacht op vestigen.
Ja, je maakt een audit plan op basis van een risico analyse. Een audit jaarplan. Hoe het jaarplan
tot stand komt is dat we afspraken maken met alle FD’s van de bedrijfsonderdelen. Daar
bespreken wij de ontwikkelingen door die er geweest zijn of die er komen en waar zij eventuele
risico’s zien. En met onze kennis daarbij en wat wij zelf hebben gehoord, komt een audit plan tot
stand met de belangrijkste risico’s. Daarmee classificeren wij audits op niveau 1, 2 en 3. Waarbij 3
168
de minder belangrijke, waar de minder grote risico’s zijn. Die zullen ook geschrapt worden als er
verzoeken of nieuwe dingen naar voren komen.
Dus het is een samenspel van de financial directors en het audit team om de grootste risico’s te
bepalen. Dan ga je de grootste risico’s afwegen, die ga je classificeren. En dan pak je degene met
de hoogste risico daar ga je je als eerste op focussen?
Wellicht niet als eerste, maar die komen sowieso aan de beurt. Urgente risico’s gaan altijd voor.
Als wij als internal audit een berichtje krijgen dat er fraude is gepleegd, dan staan we de volgende
dag op de stoep.
Nou is met name risk assessment is een term die ik ken uit het Coso framework. In hoeverre
wordt het Coso framework gebruikt in auditing?
Je zou bijna willen zeggen dat het meeste wel gestoeld is op het Coso framework. Het komt in
heel veel literatuur terug en volgens mij wordt er heel veel naar verwezen. In feite gaat het ook
om welke controls jij in je environment hebt ingericht om risico af te dekken. En daar gaat Coso
ook helemaal over. Translation: You'd say that most of it is based on the Coso framework. It
comes back a lot in literature and I think there is quite a lot of reference to it. In fact it is also a
question of what controls you have in your environment designed to hedge risk. And that’s what
Coso is all about.
Coso geeft ook een bepaalde verplichting om een aantal stappen door te lopen.
Ja, ik denk dat de elementen allemaal terug komen. Het is als het ware zo’n ingeburgerd begrip.
Het is allemaal zo verweven. Veel dingen die in het Coso framework staan, die kom je ook tegen
in je audits.
Een ander deel van mijn onderzoek zijn ERP systemen. Ben je daar bekend mee?
Nauwelijks moet ik eerlijk zeggen. Ik doe er niet veel mee. Je hebt SAP natuurlijk, dat is er eentje.
En dat gebruiken wij in onze organisatie. Ik ben geen expert in SAP.
Mijn onderzoek gaat ook niet over in hoeverre hoe een ERP systeem in elkaar steekt, maar er
zijn een aantal dingen die typisch zijn aan een ERP systeem en dan doel ik met name op dat de
data maar op 1 punt in het systeem wordt gezet, omdat je maar 1 database hebt over de
verschillende onderdelen van een proces heen. Het tegenovergestelde is dat iedere onderdeel van
een proces heeft zijn eigen systeem met zijn eigen database. En dat er tussen de databases ofwel
gecommuniceerd moet worden ofwel dat de data in de twee verschillende systemen ingevoerd
moeten worden. Mijn onderzoek is dan of het gebruik van een ERP systeem, wat voor impact
dat heeft op de risk assessment van de audit planning.
Tuurlijk heeft dat impact. Er komen hele andere risico’s daarbij kijken. Samenvatting van deel
met bedrijfsgevoelige informatie: Er kunnen problemen ontstaan in de master data. Translation:
169
Then you’re dealing with completely different risks. Problems can arise in the master data. Als je
allerlei verschillende systemen hebt, dan wordt het belangrijk om een IT auditor te betrekken bij
de audit om de interfaces te bekijken tussen de systemen. Bij het gebruik van een ERP systeem,
kom je veel meer op het gebied van toegangsrechten, gebruikersrechten, beveiliging, beheer van
je data. Tuurlijk heb je dat ook bij het gebruik van verschillende systemen, maar dat is meer
lokaal geregeld en het risico is kleiner, omdat je maar een klein deel van je data daar hebt staan.
Bij een ERP systeem staat alles centraal en als iemand al die informatie heeft, dan krijgt deze wel
heel veel kennis en macht en gelegenheid tot wat je maar kan bedenken. En daarmee een groot
risico. En bij de audit over losse onderdelen zal de focus gelegd worden op de juistheid van de
interface, input = output, ga zo maar door. Terwijl bij een ERP systeem het risico veel meer ligt
bij gebruikersrechten. Translation: While in an ERP system the risk is much more in user rights.
Bij de implementatie van een ERP systeem wil je als audit ook meekijken, al is dat meer een IT
auditor.
Iets verder kijken naar een omgeving met mutliple points of data entry. Bij een non ERP
omgeving heb je verschillende punten van data entry, in het geval dat er geen interfaces zijn. Je
hebt dus verschillende databases. Daarbij heb je een controlemethode om de verschillende
databases met elkaar te vergelijken. En daar waar het niet overeenkomt, daar kan je op
inzoomen. Dat is een controlemethode. Bij een ERP systeem is er maar 1 punt waar de data
wordt ingevoerd. En heb je deze controlemethode niet. Hoe zie je dat dat impact heeft op de
risk assessment?
Ik heb daar nooit eerder over nagedacht. Als het goed is heb je automated controls in je ERP
systeem zitten die dat voorkomen. Je zou de application controls of database controls moeten
controleren of die daadwerkelijk werken. Translation: Automated controls in your ERP system
should prevent that. You should check the application and database controls if they work
correctly.
Dat zijn controles dat je geen hele gekke getallen invult, bijvoorbeeld?
Ja, zulk soort dingen, maar ook dat twee mensen tegelijk de data kunnen veranderen. Dat heb je
natuurlijk ook. Als het goed is zit daar ook een databasecontrol, automated control op in een
ERP systeem. Dat als op het moment dat iemand in de database is aan het muteren, dat een
ander niet kan muteren. Dan wordt hij on hold gezet. Op het moment dat dat niet het geval is,
dan krijg je hele corrupte data. Je zit hier op de IT auditors hoek naar de risico’s te kijken. Ik
denk dat je risico analyse veel belangrijker is bij een ERP systeem, dat je daar een IT auditor bij
betrekt.
Omdat je risico gewoonweg meer in IT hoek zit.
170
Ja, het zit allemaal onder water. De interne controlemaatregelen zitten allemaal onder water.
Terwijl je elders heb je twee databases tegelijk, dus heb je een soort norm. 1 systeem is de norm
en daar zou het andere systeem aan gelijk moeten zijn. Op het moment dat je dat niet kan
vaststellen omdat er maar 1 database is, dan zal je andere controlemaatregelen moeten hebben.
En zal je de audit en zijn risico’s daarop moeten richten.
Dat is met name dan je segregation of duties en een aantal controles in het systeem zodat je niet
gelijktijdig of dat je niet dezelfde data kan invoeren en zorgen dat niet iedereen overal bij kan.
Ja, maar het is in beide gevallen net zo belangrijk hoor. Ik zou me kunnen voorstellen dat bij een
single entry point je zekerheid (assurance) moet halen uit automated controls. IT controls.
Bij een ERP systeem krijg je dat bijvoorbeeld een inkoper een purchase order in het systeem
schiet en deze is niet per se een financieel persoon, maar zijn input heeft uiteindelijk wel invloed
op de financial statement. Zie je daar een impact op de risico analyse?
In zoverre dat de risico analyse kijkt naar hoe het proces is ingericht. Een purchase order hoeft
niet per se door een financieel persoon ingeklopt te worden. Sales is misschien hetzelfde verhaal.
Die voeren ook sales orders in. Het zijn meer de controles er omheen. Als die purchaser zelf
gemachtigd is om alle purchase orders in te voeren en de facturen worden automatisch betaald
als deze matched met de purchase order, dan zit er een risico in. Dan ga je naar autorisatie levels
kijken. Wie mag tot welk bedrag goedkeuren? En is die purchase order wel goedgekeurd. Dan ga
je naar je reguliere purchase to pay risico’s kijken.
Dat is dus niet anders tussen een ERP omgeving en een non ERP omgeving?
Ik heb zo veel verschillende manieren gezien hoe je tot een betaling van een factuur kan komen,
waarbij de ene uitgaat van het invoeren van een purchase order in het systeem, en op het
moment dat de factuur wordt ingeboekt en er is geen verschil met de purchase order, wordt hij
ook automatisch betaald. Dan moet de purchase order ook voldoende betaalbevoegdheid
autoriteit moet hebben. Op het moment dat jij een purchase order invoert van 1 miljoen, dan zal
iemand moeten tekenen die een betalingsbevoegdheid heeft tot 1 miljoen. Op het moment dat
deze autorisatie pas gegeven wordt bij het invoeren van de factuur, dan wordt een purchase
order al minder belangrijk, omdat de autorisatie pas bij de factuur komt. Natuurlijk voor je
proces moet je nog steeds je purchase order autorisatie hebben. Je wilt natuurlijk niet dat
iedereen in het bedrijf maar gaat bestellen wat ze willen. Maar het wordt wel gesignaleerd door de
juiste personen als de factuur komt. Dan is het risico niet heel groot. Als de factuur betaalbaar
wordt gesteld op basis van de purchase order, dan wordt het risico bij de purchase order wordt
natuurlijk groter. Ik denk dat het niet zozeer afhangt van de persoon die het invoert, maar hoe
het proces is ingericht.
171
Je hebt bijvoorbeeld ook te maken met verschillende locaties. Bij een ERP systeem kan een
invoer in een warehouse kan 100 km van het accounting systeem af zijn.
Ik zou dat niet als een risico zien als dat nou op de begane grond zou zijn of aan de andere kant
van de wereld. Daar zie ik geen risico in. Translation: I wouldn’t see that as a risk. It does not
result in a higher risk if it is entered on the ground floor compared to entering on the other side
of the world.
En ook niet als het om totaal andere culturen gaat?
Daar zou wel een extra risico in kunnen zitten. Als je kijkt naar Afrika, maar daar is je algehele
risico hoger. En dan ga je zelf ook zoeken als auditor naar meer zekerheid. Dan ga je hogere
deelaannemingen nemen. Daar in landen waar omkoping en samenspanning op management een
heel dominante rol heeft, dat zie je sowieso als een hoger risico.
Maar dat is onafhankelijk of je een ERP systeem zou hebben of niet. Dat ligt gewoon aan de
cultuur die er eventueel is.
Ja, klopt. Ik zie dat niet als ERP issue.
Nog heel even terugkomend op mijn algehele vraag van mijn onderzoek: of een ERP systeem
impact heeft op de risk analyse binnen de audit planning. En zelf heb je ook aangegeven dat er
twee niveaus zijn van audit planning en daarmee ook twee niveaus van risico analyse. Bij de
jaarlijkse audit planning en die risico analyse, denk je dat het gebruik van een ERP systeem daar
invloed op zou hebben of niet?
Ja, dat denk ik wel. Ik denk dat je sowieso een veel centralere rol oppakt. Je audit object is 1
database. En 1 set van automated controls, application controls, noem maar op. Bij geen ERP
systeem heb je te maken met losse systemen, losse locaties, losse bedrijfsafdelingen, andere
mensen, waarbij je veel meer tijd kwijt bent vermoedelijk met het auditen van deze onderdelen.
Translation: Your audit object is one database and one set of controls, application controls
amongst other controls. With non ERP systems you’re dealing with separate systems, different
locations, different departments, other people, where you probably need a lot more time in
auditing these parts. Plus het feit dat je bij ERP systemen dat je IT hebt. Dat is dan technisch
vooral. Maar procesmatig, elke locatie heeft zo zijn eigen processen, maar de inrichting van IT is
wat dat betreft, geen interfaces, iedereen heeft dezelfde set of controls onder water zitten. En bij
niet ERP systemen is dat heel verschillend natuurlijk. Dan moet je elke keer opnieuw uitzoeken
hoe dat zit. Dat is misschien heel anders ingericht. Dan kan ik me voorstellen dat je een
roulatiesysteem, waarbij je het ene jaar het ene systeem en het andere jaar het andere systeem
controleert. Want je kan waarschijnlijk niet alles in 1 jaar pakken. En dat kan bij een ERP
systeem waarschijnlijk wel. In die mate kan ik me zeker voorstellen dat het invloed heeft op je
172
audit planning. Translation: But processwise, each location has its own processes, but the design
of IT is, no interfaces, everyone has the same set of controls underneath. And at non ERP
systems that is very different of course. Then you have to investigate every time again. That
might be quite different. Then I can imagine that you have a rotation system, where you check
one year one system and the other year the other system. Because you probably can’t do
everything in one year. And in an ERP system you probably can. To that extent, I can certainly
imagine it affects your audit planning.
Denk je dat als er dus geen ERP systemen gebruikt, je ook meerdere samples zal moeten
controleren, en dat het daarmee meer tijd in beslag zal nemen.
Zeker.
Denk je dat de grootte van de samples dan verschillend is tussen een ERP omgeving en een niet
ERP omgeving?
In totaliteit zeker.
Dan is deze groter bij een niet ERP omgeving.
Ja, want dan ga je bij elk individueel systeem ga jij je samples bepalen. En vaak is het zo als je een
statistische aanpakt pakt om jouw samples te berekenen, heb je boven een bepaalde populatie
wijzigt jouw sample size niet meer. Als jij 200.000 regels hebt of 900.000 regels, je sample size
blijft hetzelfde. Maar als je ze gaat opknippen in 6 verschillende samples van 200.000 stuks, dan
heb je 6 keer die deelwaarnemingen. Dus dan wordt die sample size groter. Translation: Yes,
because then you go to each individual system to get your samples. And often if you take a
statistical approach to calculate your samples to pick, you don’t change your sample above a
certain population size no longer. If you have 200,000 rules or 900.000 rules, your sample size
remains the same. But if you are going to chop it up in 6 different samples of 200,000 pieces,
then you have 6 times that make observations. So then that sample size larger. Dat weet ik 100%
zeker.
En daarmee worden de werkzaamheden groter en dus neemt het meer tijd in beslag.
Een ERP kan nog een voordeel hebben boven allemaal losse systemen; de rapportages die uit
een ERP systeem komen zijn voor iedereen gelijk. Zelfde layout, zelfde soort, mogelijkheden. Je
hebt de data opbouw die ken je op een gegeven moment in je database waarop je de data-analyse
eventueel doet. Dat is allemaal in het voordeel van de auditor. Hij hoeft niet elke keer opnieuw
uit te zoeken wat voor soort rapportages zijn er mogelijk. Translation: An ERP can have another
advantage above all separate systems; the reports that come from an ERP system are the same
for everyone. Same layout, same kind, possibilities. You have the data structure that you know in
your database that you can possibly use for data analysis. That is all in favor of the auditor. He
173
does not need to find out what kind of reports are available every time. Bij het ene systeem is dat
mogelijk, bij het andere systeem is dat mogelijk. Bij de een ziet het er zo uit, bij de ander ziet het
er zo uit. Bij de ene heet het veldje zo, bij de ander zo. Dus je hoeft niet verschillende analyse
tools te maken. Als je 1 tool hebt die analyseert er een negatief bedrag op het veldje of een debet
bedrag voorkomt, dan hoef je geen tweede tool te maken die analyseert of er een negatief bedrag
voorkomt op het veldje debt bedrag. Allemaal zulk soort dingen werkt makkelijker bij een
centraal systeem.
Dat maakt het voor een auditor allemaal makkelijker. Toegankelijker en herkenbaarder.
Herkenbaarder ja. En je weet natuurlijk vanuit de audit van 1 bedrijfsonderdeel, zou je bij een
ander onderdeel kunnen adviseren om ook een bepaald rapport te gebruiken, wat wij
bijvoorbeeld niet weten. Dat zou een voordeel voor je adviesfunctie kunnen hebben. Je hoeft het
jezelf ook niet eigen te maken in meerdere systemen. Je kan zo ook gebruiker worden van het
systeem op leesrechten. Dat is ook efficiënt voor de planning en dan kunnen we ook veel meer
doen. Ik kan eigenlijk alleen maar voordelen, behalve nadelen bedenken. Alleen als het ERP
systeem crasht, ben je als bedrijf wel helemaal lam natuurlijk.
Dat punt, de mogelijke uitval. Is dat iets dat je meeneemt in je risico analyse?
Ja, zeker. Dat zou je bij elke audit überhaupt mee moeten nemen. Je general IT controles neem je
in principe mee bij elke geïntegreerde audit. Dan kijk je wat zijn jullie kritische systemen en hoe
is de back up procedure, recovery procedure en uitwijk ervan geregeld. Wat gebeurt er nou als er
een stroomstoring is en alles ligt helemaal plat? Kost een miljoen per dag, globaal gezegd. Wat
heb je als alternatief? Translation: You take your general IT controls into account at every
integrated audit. You look what are your critical systems and how is the backup procedure,
recovery procedure and alternate location. What happens if there is a power failure and
everything is down? Costs could be millions a day, globally said. What do you have as an
alternative?
Dat is ook een aardig punt om mee te nemen in mijn bevindingen. Ik ben eigenlijk door mijn
vragen heen. Zijn er nog andere punten voor mijn onderzoek, waarvan je zegt misschien moet je
daar eens aan denken?
Voor spits je het onderzoek echt naar ERP systemen.
Ik ga dit interview dan uitwerken, dat zal grotendeels letterlijk zijn, en dat stuur ik naar je toe ter
review.
174

Jas Peter 10681078 MSC ACC

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Jas Peter 10681078 MSC ACC

Uploaded by

Copyright:

Available Formats

Amsterdam

The Impact of ERP Systems on Internal Audit Planning: a

Name: Peter Jas

2 Research Method ................................................................................................................................... 8

2.1 Semi-structured Interviews....................................................................................................... 8

2.2 Document Review ..................................................................................................................... 9

3 Literature Review ................................................................................................................................ 10

3.1 General information about the internal audit process ....................................................... 10

3.2 Main concerns in ERP auditing ............................................................................................. 12

3.3 ERP impact on high level audit planning ............................................................................ 14

3.4 ERP impact on granular level audit planning ...................................................................... 15

3.5 Other findings .......................................................................................................................... 17

4.1 Wolters Kluwer Financial Services........................................................................................ 18

4.2 TeamMate ................................................................................................................................. 19

4.3 Why TeamMate ........................................................................................................................ 19

5.1 General information internal audit planning process ........................................................ 21

5.2 Main concerns in ERP systems ............................................................................................. 25

5.3 ERP impact on high level audit planning ............................................................................ 26

5.4 ERP impact on granular level audit planning ...................................................................... 28

5.5 Other findings .......................................................................................................................... 32

6.1 General information internal audit process ......................................................................... 35

6.2 Main concerns ERP systems .................................................................................................. 37

6.3 ERP impact on high level audit planning ............................................................................ 38

6.4 ERP impact on granular level audit planning ...................................................................... 39

6.5 Other findings .......................................................................................................................... 41

8.1 Appendix I: Mind map to specify research topic................................................................ 48

8.2 Appendix II: Thesis structure ................................................................................................ 49

8.3 Appendix III: Interview #1 ................................................................................................... 50

8.4 Appendix IV: Interview #2.................................................................................................... 62

8.5 Appendix V: Interview #3 ..................................................................................................... 74

8.6 Appendix VI: Interview #4.................................................................................................... 90

8.7 Appendix VII: Interview #5 ................................................................................................ 103

8.8 Appendix VIII: Interview #6 .............................................................................................. 115

8.9 Appendix IX: Interview #7.................................................................................................. 130

8.10 Appendix X: Interview #8 ................................................................................................... 137

8.11 Appendix XI: Interview #9.................................................................................................. 151

8.12 Appendix XII: Interview #10 .............................................................................................. 159

8.13 Appendix XIII: Interview #11 ............................................................................................ 167

This leads to the research question:

a. What are the main concerns of risk in an ERP system?

2.1 Semi-structured Interviews

3.1 General information about the internal audit process

3.2 Main concerns in ERP auditing

3.3 ERP impact on high level audit planning

ERP implementation or adjustments

3.5 Other findings

Future of ERP systems

4.1 Wolters Kluwer Financial Services

- TeamMate Audit Management

4.3 Why TeamMate

5.1 General information internal audit planning process

High level audit planning

Granular level audit planning

5.3 ERP impact on high level audit planning

ERP implementation or adjustments

5.4 ERP impact on granular level audit planning

5.5 Other findings

Future of ERP systems:

Conclusions of this paragraph:

6.1 General information internal audit process

High level audit planning & Granular level audit planning

6.3 ERP impact on high level audit planning

ERP implementation or adjustment