2.2.2.

SLR Design
SLR generally consists of three main phases, i.e. planning, conducting and analyzing, and
reporting (Kitchenham and Charters, 2007; Morioka and de Carvalho, 2016; Petticrew and
Roberts, 2006; Snyder, 2019; Thomé et al., 2016; Wahono, 2015). The planning phase is crucial
as it defines and formulates the SLR’s research questions and review protocol to guide the entire
review. This includes setting explicit criteria to select the appropriate literature and developing a
methodology to analyze the selected literature. The review phase is the core phase of SLR. It
performs literature identification and selection, extraction and analysis to answer the SLR’s
question based on the predetermined review strategies. Identification of literature from the
relevant databases, including application of search string and metadata recording, and analysis
based on the developed analysis strategy and evidence synthesis also performs in this phase. The
final facet is to disseminate the result either as a part of the study (e.g., dissertation) or as an
independent academic work (e.g., paper submission). Table 2-2 summarizes the general step and
critical activities in SLR.
No Step Key Activities
Planning phase
1. Formulate SLR question(s) a. Develop RQs
b. (optional, iterative) consult to relevant expert
2. Formulate review strategy a. Define inclusion an exclusion criteria
b. Formulate analysis method
Review phase
3. Identify relevant literature a. Select relevant databases (or journals)
b. Define appropriate search ‘string’
c. Define metadata for literature extraction
4. Conduct literature extraction a. Perform literature search
b. Apply inclusion and exclusion criteria
c. Populate relevant literature
5. Perform analysis a. Perform descriptive analysis on the extracted literature
b. Apply analysis method
c. Synthesize evidence
Dissemination phase
6. Develop SLR report a. Narrate analysis
b. Summarize key findings

Table 2-2. SLR Phases and Key Activities

(synthesized from (Jia et al., 2020; Kitchenham and Charters, 2007; Rouhani et al., 2015; Snyder, 2019;
Wahono, 2015))

2.2.3. SLR Questions

Research on AA in over 40 years span has covered many important aspects, including drivers,
issues, and tools for AA realization (Eulerich and Kalinichenko, 2018). Nevertheless, the lack of
empirical research in this field indicates there are barriers to implementation requires to be
addressed. As a part of the study, this literature review aims to identify the pivotal factors and
main challenges hindering AA implementation, which represents in the following research
questions:
1) SLRQ1: what are the key determinants in the AA realization (by IAF)?
2) SLRQ2: what are the challenges to a successful AA initiation and/or utilization?
3) SLRQ3: what AA models exist?

2.2.4. Search Strategy

The search strategy applied in SLR aims to ensure relevant studies are retrieved to achieve its
objectives (Thomé et al., 2016). It consists of a database as the source of papers extracted, search
strings, and inclusion (and exclusion) criteria, which is visualized in figure 2-2 as follows and
elaborated further afterwards.

Figure 2-2. Literature Search Strategy

1) Database
 This SLR utilizes the Google scholar database to provide a wide range of research and study
results on AA implementation. This approach assists in ensuring the comprehensiveness of
the search results. One might argue that Google scholar deliverable varies in terms of quality,
which may affect the SLR’s. Nevertheless, there will be protocols in this SLR to address the
possible limitation from the database search results, i.e., inclusion and exclusion criteria.
2) Search strings
There are alternatives in initiating a literature search for an SLR. One of those is through
search strings (Jalali and Wohlin, 2012). This SLR uses search strings approach with the
following keywords to extract studies from the Google scholar database:
a) “Audit Analytics” as the umbrella term encompasses all practices including continuous
audit, predictive analytics, and other technology and digital data utilization in internal
audit practices;
b) “Continuous Audit” as the initial term for the use of digital data for real-time or near real-
time audit (Bumgarner and Vasarhelyi, 2018; Chan and Vasarhelyi, 2011; Eulerich and
Kalinichenko, 2018); and
c) “Audit Data Analytics” as the term for emphasizes in the use of technology in audit
practices (Barr- Pulliam et al., 2020).
Each keyword will be applied to itself or combined with “implementation” to capture the
study of AA practices. Moreover, each keyword will also be combined with “factors”,
“barriers”, or “challenges” to represent the objective of this SLR. This approach will help in
improving the relevance of search results (vom Brocke et al., 2015).

3) Inclusion and exclusion criteria

The initial result will be assessed based on inclusion criteria which including:
a) Only papers presented in English to be included;
b) Only including papers from 2000 or newer to reflect current practices; and
c) Analysis of relevance based on abstract to ensure the paper retrieved examine AA
practices (as suggested by (vom Brocke et al., 2015)).

In addition to the inclusion criteria above, literature observed and selected primarily from the
first three pages of the search result as it considers as the most relevant according to Google’s
algorithm.

The sample result of extraction result based on these criteria will be evaluated randomly by
an external researcher or practitioner before finally included in the literature to be reviewed.
The criteria for this evaluation are as follow:

a) Contain examination of AA, both from conceptual and practical (implementation)

perspectives; and
b) Contain challenges in AA utilization.
Literature selection is important as it addresses the need to balance between the
comprehensiveness of coverage and the depth of analysis from the selected papers (Morioka and
de Carvalho, 2016). The strategy as presented in this section aims to ensure the selection of
appropriate literature as this will affect the next steps of the review and its result.
2.2.5. Analysis and Synthesize Strategy
This section elaborates strategies to analyze and synthesize the extracted papers from the
previous phase, which consists of descriptive analysis, quality assessment, and synthesize and
analysis method.
1) Descriptive analysis
An SLR normally performs descriptive analysis to inform the overview of the selected
literature. Descriptive analysis may contain general information such as journals/publishers,
type of publications, year of publications, or authors (Snyder, 2019). This provides
information about the landscape of the literature being reviewed in the study. Moreover, a
descriptive analysis may also assist in obtaining specific information regarding the objective
of the study (or in this regards the SLR) relevant to the SLRQ (for instance as exemplified in
(Jia et al., 2020; Rouhani et al., 2015)). For this purpose, this study extracts following
information from the selected papers for descriptive and substantial analysis.

Extracted Data Description Type

No
1. Literature Identity Unique ID of literature reviewed General
2. Bibliography Authors, literature title, publisher, year of publication General
3. Research Method Method employed in the primary study being General
reviewed (e.g., case study, experiment, etc.)
4. Factors of AA realization Elaboration of factors relevant to AA realization found SLRQ1
(discussed/found) in the or discussed in the selected paper
study
5. Challenges of AA realization Elaboration of challenges hindering AA realization SLRQ2
(discussed/found) in the discussed/found in the study
study
6. AA model(s) Elaboration of AA or technology-based audit SLRQ3
techniques
7. Findings The finding and contribution of the study related to AA General
realization issue
8. Future study Key insight from the literature being reviewed for General
future study in the AA realization field
Table 2-3. Data Extraction Form

(adapted from (Rouhani et al., 2015))

2) Synthesize and analysis activities

Synthesize and detailed analysis to answer this SLR’s questions are performed based on data
and information extracted (from the selected studies) as presented in table 3 in the previous
section. This phase aims to answer (find and discuss) this SLR’s questions using data collected.
In this step, the SLR will elaborate factors and challenges in the AA implementation; including
the emerged common themes, similarities, and differences from each study. The result from
this SLR will be a list of common factors, challenges, and AA model identified from various
literature in the AA-related research as the basis for further examination in this dissertation.
Furthermore, the identified challenges is crucial for the next phase of the research. Therefore,
the list of challenges as the result of analysis presented in tabular format as shown in table 2-
4 below to guide the MICMAC-ISM as elaborated in the next section of this proposal.

Item Description
No
1. ID Unique ID of challenge
2. Category Category of challenge (e.g., technology, human resources,
organizational barrier, etc)
3. Challenge Name of the challenge
4. Description Description of the challenge
5. Cite count Number of literature mention/elaborate the challenge
6. References Identity of the literature mentions/elaborates the challenge
Table 2-4. Tabular form for the List of Challenges

(adapted from (Matheus et al., 2021)