Leveraging ChatGPT for Enhancing the Internal Audit Process – A Real-World

Example from a Large Multinational Company

Scott Emett
Arizona State University
scottemett@asu.edu

Marc Eulerich
University of Duisburg-Essen
marc.eulerich@uni-due.de

Egemen Lipinski
Uniper SE
egemen.lipinski@uniper.energy

Nicolo Prien
Uniper SE
nicolo.prien@uniper.energy

David A. Wood
Brigham Young University
davidwood@byu.edu

Running Head: ChatGPT at Uniper

Scott Emett, Arizona State University, W. P. Carey School of Business, W. P. Carey School of
Accountancy, Tempe, Arizona, USA; Marc Eulerich, University of Duisburg-Essen, Business
Administration, Duisburg, Germany; Egemen Lipinski, Uniper SE; Nicolo Prien, Uniper SE;
David A. Wood, Brigham Young University, Marriott School of Business, School of
Accountancy, Provo, Utah, USA

Electronic copy available at: https://ssrn.com/abstract=4514238

 Leveraging ChatGPT for Enhancing the Internal Audit Process – A Real-World
Example from a Large Multinational Company

Abstract: We discuss how the internal audit function (IAF) of a large multinational energy
company, Uniper, is starting to use ChatGPT to improve automation of internal auditing. We
investigate the benefits and challenges of integrating ChatGPT into internal audit processes and
assess its impact on the efficiency and effectiveness of the IAF. Uniper has implemented
ChatGPT into parts of audit preparation, fieldwork, and audit reporting. On their initial tests,
they estimate efficiency gains ranging from 50 to 80 percent for various processes. We report
key risks and challenges with using ChatGPT in internal auditing as well as the initial key
practices and rules for using ChatGPT. This study should help practitioners to learn how
ChatGPT can aid in auditing and help inform researchers about this fast, emerging technology.

Keywords: ChatGPT; Internal Auditing; Audit Process; Large Language Models

I. INTRODUCTION

The application of advanced language models, such as ChatGPT, within accounting

has sparked interest and extensive discourse since late 2022 (e.g., Gu et al. 2023; Wood et al.

2023). Large language models have distinguished themselves with their capacity to re-create

knowledge and carry out simple tasks, making them accessible to various user bases through

low-code interfaces (Radford et al. 2018a; Bommasani et al. 2021; OpenAI 2023). This paper

examines how the internal audit function (IAF) at Uniper, a multinational energy company,

adopted ChatGPT for internal audit tasks and how they plan to further use this technology in

the future. By harnessing ChatGPT, Uniper internal auditors have enhanced multiple phases

of the auditing process, ranging from risk-based audit planning and audit preparation, to

actual audit procedures and data analysis.

Internal auditing plays an important role in a company’s internal control, risk

management, and governance functions (e.g, Prawitt et al. 2009; Carcello et al. 2020; Ege et

al. 2022). Yet, internal audit often struggles with budgeting and perception issues that hamper

its effectiveness (Anderson et al. 2012; Bartlett et al. 2016, 2017; Eulerich et al. 2021).

Technology holds promise to enhance internal audit effectiveness and efficiency on a limited

budget (Christ et al. 2021; Eulerich et al. 2023). Thus, studying if ChatGPT is able to serve as

a valuable tool for constructing audit steps, extracting insights, drafting working papers,

Electronic copy available at: https://ssrn.com/abstract=4514238

improving report readability, and facilitating data analysis tasks is important (e.g., Sun 2019;

Schreyer et al. 2022; Cheng et al. 2023; Kok 2023; Wood et al. 2023).

This paper offers an in-depth exploration of how ChatGPT can enhance the internal

audit procedures of a large multinational company. The study aims to investigate the potential

benefits and challenges of integrating ChatGPT into internal audit processes and provide early

insight on its impact on the efficiency and effectiveness of the function. We find that by

leveraging the capabilities of ChatGPT, internal auditors can enhance their ability to analyze

large amounts of textual data, automate routine tasks, and generate valuable insights for risk

assessment and mitigation. Additionally, ethical considerations, risks and potential limitations

associated with the use of ChatGPT in the IAF are discussed. The findings of this research

contribute to the growing body of research studying how technology can enhance auditing

practices and provide practical insights for organizations seeking to optimize their internal

audit processes (Cooper et al. 2019, 2022; Pickard et al. 2020; Eulerich et al. 2022).

II. MOTIVATION

Language models, such as ChatGPT, are noted for their ability to generate coherent

and contextually appropriate text responses, making them potentially useful to auditors. Gu et

al. (2023) propose that auditors can use language models as copilots on the audit. Copiloting

involves the collaborative interaction between humans and AI systems to complete tasks. The

collaboration is a continuous and iterative dialogue, allowing the auditor to control the

direction of the audit and its procedures, while relying on the foundation model to function as

an audit-oriented assistant, providing support, insights, and analysis to facilitate well-

informed decision-making. Copiloted auditing offers significant theoretical potential, but has

heretofore not been tested in practice. To this end, we examine how a multinational company

is using ChatGPT as a copilot to enhance internal audit efficiency and effectiveness.

III. METHODOLOGY

Electronic copy available at: https://ssrn.com/abstract=4514238

 We use a case study approach in this paper. The case study involves real-world

applications of ChatGPT in the audit procedures for a multinational company. While case

studies have relatively low external validity, they provide in-depth details on important topics.

Considering the recent introduction of ChatGPT and the fast-moving changes businesses are

making in response, a case study helps researchers understand how practice is developing and

evolving with this emerging technology.

Case Company and Case Description

Uniper is a leading German energy company specialized in power generation, trading,

and energy sales. Established in 2016 as a spin-off from E.ON, it operates globally with a

strong presence in Europe, and other strategic markets. Uniper reported annual revenues of

approximately $287 billion in 2022. Uniper actively engages in energy trading, managing

risks, and maximizing asset value. They also offer energy-related services such as consulting,

engineering, and asset management.

Internal Audit Function at Uniper

The Uniper IAF has 26 internal auditors. The IAF focuses on regulatory compliance,

identification of potential risks, and enhancing operational efficiency. The CAE oversees all

audit activities, reporting to the CEO, communicating directly with the audit and risk

committee, and coordinating with management, which follows the recommendations of the

IIA (IIA, 2023). Audit managers report to the CAE and are responsible for the planning and

execution of individual audit engagements. Their role involves supervising audit teams and

ensuring that audits are completed within the designated timeframe, budget, and quality.

Auditors work under the direction of the managers.

Their responsibilities include testing controls, verifying adherence to policies and

procedures, identifying potential areas of risk, adequate mitigation measures and, drafting the

audit reports. Finally, the Audit Quality team assists the CAE in effectively managing the

entire function by employing key performance indicators. They ensure adherence to the

Electronic copy available at: https://ssrn.com/abstract=4514238

standards set by the Institute of Internal Auditors (IIA) and provide reports to both the board

of management and the supervisory board. The Audit Quality team is also responsible for

performing the yearly audit planning The IAF completes around 50 internal audits per year,

normally with 2 auditors in a team. All audits at Uniper include at least some level of data

analytics or technology.

The internal audit process at Uniper follows the IIA standards, including stages of

preparation, fieldwork, audit reporting, and monitoring. Uniper designed its internal audit

process to be risk-based, iterative, and ongoing, with the goal of continuous improvement. If

the internal auditors learn of a new risk, the annual audit plan can be adapted to immediately

respond to any changes, including technological innovations. The IAF has authority to

determine how they will conduct testing, including any technologies they will adopt to

improve audit efficiency and effectiveness.

IV. Leveraging ChatGPT in the Internal Audit Process

In January 2023, the CAE and one of the audit managers decided to experiment with

ChatGPT to see how it could be used in the audit process. The audit manager and his team

began exploring ChatGPT by identifying potential use cases and evaluating potential risks of

the technology.

The initial choice of internal audit test cases was driven by curiosity, a desire to

explore the capabilities of AI, and desire to apply these technologies to the entire internal

audit process. During early testing of ChatGPT, the internal auditors sought out supportive

colleagues within the IT function and worked with a colleague in the CIO office and IT

strategy. The team self-taught using internet resources on prompt engineering and learned

primarily by experimentation.

The team primarily interacted with ChatGPT through a browser interface, which had

been approved by Uniper’s Information Security department. However, this approval was

conditional, hinging on strict adherence to sufficient risk mitigation practices to ensure secure

Electronic copy available at: https://ssrn.com/abstract=4514238

deployment of the technology. Furthermore, the IAF involved other internal assurance

providers in the consulting process like the Workers Council, Compliance and Data

Protection. With this support, the IAF addressed potential information security, compliance or

data privacy risks.

The integration of ChatGPT into daily operations was initially met with positivity

from the team. The overarching sentiment was one of enthusiasm about incorporating this

novel AI tool into the workflow. However, there were instances of resistance or skepticism, a

common occurrence when introducing new technology into established processes (e.g.,

Salijeni et al. 2021). As ChatGPT proved useful over time, a considerable shift in attitudes

was observed. An increasing number of team members began embracing the AI tool,

demonstrating a positive change in their perspective and acceptance of ChatGPT.

Based on these efforts, the IAF identified the following initial use cases for ChatGPT

in internal auditing:

• Q&A: ChatGPT can help internal auditors to answer specific questions from various
domains, e.g., laws and regulations, accounting, IT, tax or specific technologies.

• Research: ChatGPT can help the internal auditors to create a list of ideas or
recommendations or a list of relevant topics for an audit engagement.

• Text Analysis: ChatGPT can help internal auditors with different text analysis tasks,
especially to summarize (long) text fragments, find unfavorable clauses and terms,
screen different types of textual information, interpret materials in other languages,
determine the impact of new laws and regulations, or find critical terms in a text.

• Content Creation: ChatGPT can help internal auditors to write, correct, create, or
explain program code, create meeting minutes from bullet points, create individual
emails for different business situations, or create draft audit reports.

It is important to note that during the testing of ChatGPT, no company IP address was

used and no personal or company data was utilized to maintain confidentiality. Additionally,

the auditors were extremely conscious of not providing complete and meaningful texts, opting

instead to use fragments and manually piece together the responses. They took additional

measures by altering data or changing the meaning of statements in certain instances to ensure

Electronic copy available at: https://ssrn.com/abstract=4514238

complete camouflage and prevent any possibility of constructing coherent texts

retrospectively by collecting and piecing together the text fragments that were entered into

ChatGPT. To ensure the prevention of unintentional disclosure of information during testing,

the auditors diligently adhered to the "four-eyes" principle when crafting the prompts for

ChatGPT.1 Based on these categories of potential applications, the team tested the integration

of ChatGPT in a real internal audit engagement. The auditors chose to use ChatGPT in

support of a physical security audit across multiple sites of the organization.

The audit sought to confirm that an adequate Security Management System existed,

including the protection of people (Uniper personnel, contractors, and visitors), sites (power

plants, offices, energy storages), property (equipment, pipelines, infrastructure etc.), and

environment to limit the impact that may result from physical security threats.

The IAF used ChatGPT in various aspects of the audit, as depicted in Figure 1.2 The

figure shows that the IAF attempted to use ChatGPT in three of the four stages of the audit,

not attempting to use it in the monitoring stage in initial testing. We provide detailed

discussion of how ChatGPT was used in each stage and how well it performed.

Audit Preparation Phase

Brainstorming Risks

The IAF started with using ChatGPT to identify potential physical security risks.

Auditors, assisted by ChatGPT, brainstormed various risks relevant to this audit (see Figure

2). The auditors estimate that approximately 80 percent of the total risks identified were

generated by ChatGPT. It should also be mentioned that the risks were accurately described

and this task was completed only in a few seconds by ChatGPT. For validation, the auditors

shared the risk descriptions with the audit client, who found them reasonable with only minor

modifications.

1
This principle is that two people (each with two eyes) are watching what is happening at all times.
2
The figures come from PowerPoint slides prepared by Uniper internal auditors. The slides were prepared to
share the work of the IAF to internal and external stakeholders.
7

Electronic copy available at: https://ssrn.com/abstract=4514238

Defining the Audit Scope

After this initial review of the quality of risks generated, the internal audit team used

ChatGPT to help define the scope of the audit. The internal auditors specifically prompted

ChatGPT to create a scoping document with seven different audit-relevant areas for the

engagement and three sub-items per area (see Figure 3). The internal audit team double-

checked the results with the audit client and followed the suggested scope with only minor

changes. Afterwards, the internal auditors used ChatGPT to define risk areas for the audit

work program.

In sum, in the audit preparation phase, ChatGPT offered valuable insights and

generated preliminary ideas. It assisted auditors in pinpointing potential risks, defining the

audit scope, and proposing relevant audit procedures. Furthermore, it generated customized

audit steps based on specific objectives, refined interview questions, and helped refine audit

programs to align with the unique characteristics of the audit subject.

Audit Fieldwork

Interview Preparation and Summarization

During the audit fieldwork stage, the IAF used ChatGPT to define potential interview

questions for the internal auditor and the initial interviews with the audit client. The internal

auditor prepared a very specified prompt to get the best results out of the model. In this

prompt the auditor defined which role ChatGPT should take (“Internal auditor of an energy

company that operate various power plants in Germany and abroad”), which specific task

should be fulfilled (“What are some key questions to ask the Corporate Head of Physical

Security about minimum security standards for fencing, video surveillance and key lock

systems, implementation processes and the benefits of ISO 270001?”), and which specific

circumstances should be included in ChatGPT’s answer (“Specifically, how can these

standards be defined and monitored to improve the physical security in the company’s power

plant?). Based on the concrete specification of the “role” of ChatGPT, the concrete “task” to

Electronic copy available at: https://ssrn.com/abstract=4514238

fulfill, and additional “circumstances”, ChatGPT produced questions that the internal auditors

used for the actual interview. Figure 4 provides an example of what was entered into ChatGPT

and the result.

Internal auditors conducted the interview of company personnel and recorded their

meetings as they normally do. Most of the interviews were conducted in German. For testing

purposes, the unstructured meeting minutes of one particular interview were summarized,

structured and translated into English by ChatGPT. It should be noted though, that the

auditors camouflaged the meeting minutes before entering them into ChatGPT. (see Figure 5).

ChatGPT excels in this type of data aggregation and structuring.

Based on the original interviews and summarization, the internal auditors wanted to

conduct additional follow-up interviews with the audit clients. Based on the previous

interviews, the internal auditors used ChatGPT to develop additional follow-up questions. The

internal auditors input the information from their testing and asked ChatGPT to generate

additional questions based on the information provided. In every instance, the auditors

consistently and transparently informed their interviewees that they were using ChatGPT as

their source of information. The reactions they received in return ranged from astonishment to

positive recognition.In fact, for some of the interview partners, this experience served as their

motivation to begin using ChatGPT themselves.

Researching Topics

Uniper’s IAF also used ChatGPT to research relevant regulatory guidance and

geopolitical events that were taking place concurrently with the audit. For example, the team

used ChatGPT to better understand how information security standards (ISO 27001) applied

to the specific audit being conducted by the team (Figure 6), allowing them to test adherence

to those standards. Although ChatGPT is not trained on all current information, users can use

ChatGPT to process and understand current events. For instance, Uniper’s IAF team copied

Electronic copy available at: https://ssrn.com/abstract=4514238

and pasted press articles published during the audit into ChatGPT and then had it summarize

and extract relevant information (Figure 7).3

Audit Reporting

Interview Preparation and Summarization

Once all audit activities were finished, the IAF used ChatGPT to improve the

readability of the audit report. The auditors entered short parts of their findings into ChatGPT

while observing strict confidentiality and then asked ChatGPT to produce several alternative

formulations of the findings. The auditors selected which response best represented the

findings in a professional, neutral manner. The internal auditors then drafted the final report.

(see Figure 8).

Identified Benefits, Future Plans, and Risks and Challenges

Identified Benefits

While the IAF applied ChatGPT in various stages of the internal audit process, the

level of improvement varied across different audit tasks. The IAF observed the most

promising results in tasks that involved preparation of audit scope, risk descriptions, interview

preparation, translation and structuring of meeting minutes, research, and report writing. The

highest positive impact the company observed in report writing. ChatGPT was particularly

effective in this aspect of the internal audit process, which is usually very time consuming,

resulting in more accurate, concise, and well-structured reports. Considering that the audit

reports are also intended for the board of management, it is vital to prioritize their readability.

This is where ChatGPT unfolded its highest value for the auditors.

In regard to the tasks mentioned above, it is difficult, if not practically impossible, to

provide solid and measurable evidence regarding time reduction or cost savings based on the

initial tests (Eulerich et al. 2023). Instead, the IAF evaluated the increase in productivity by

3
We note that ChatGPT now has the ability to search online materials, which would make copying and pasting
articles unnecessary.
10

Electronic copy available at: https://ssrn.com/abstract=4514238

conducting a subjective team survey involving four team members. Based on the survey

results, the IAF estimated improvements in time efficiency ranging from 50 to 80 percent for

specific process steps. The IAF strongly believes that ChatGPT has not only improved the

time efficiency but also contributed to more accurate outputs through its text analysis and

generation capabilities. As a result, the overall quality of the audit tasks has been significantly

improved.

Future Plans

Uniper’s internal audit function has charted a comprehensive plan to leverage the

significant advancements made with ChatGPT. The first step in this plan is to finalize the

recruitment of three AI employees with a proven AI background. Their expertise will be

crucial in driving AI initiatives and enhancing workflow automation within the IAF. ChatGPT,

in particular, is expected to automate certain tasks, thereby enhancing efficiency and enabling

auditors to dedicate their attention to other valuable activities, such as engaging with audit

clients and conducting fieldwork.The company also plans to develop AI guidelines and

implement AI training programs. These initiatives aim for responsible AI usage and a process

for employees to certify before using AI. Concurrently, Uniper will maintain a focus on

identifying new AI use cases, especially those involving data analysis and video generating

platforms. The goal is to expand the application of ChatGPT into areas where it can excel in

analyzing large volumes of data, thereby driving further value for the organization.

Finally, Uniper intends to foster knowledge and acceptance of AI within the

organization through ongoing AI road shows, promoting a greater understanding and

acceptance of AI technologies. This will be complemented by collaborations with other

functions within the company to implement AI solutions company-wide. The company also

plans to share its AI experiences with peer companies and at conferences, facilitating mutual

learning and showcasing their AI journey.

11

Electronic copy available at: https://ssrn.com/abstract=4514238

 Uniper has also identified two additional projects they are testing with ChatGPT. First,

the company is applying ChatGPT to the operational manual for a Uniper power plant. This

will allow for employees to quickly search for answers to questions using plain language

querying. It will also allow for summarizing results to make it easier to train others. Second,

they are considering a project to apply ChatGPT to improve Uniper’s governance framework

by enhancing business policies and guidelines.

Overall, the continued use of ChatGPT and specific plans for additional use suggest

that it is valuable for internal audit, even if the benefits are hard to quantify, which is similar

to other technologies in internal auditing (Eulerich et al. 2023). As stated by an internal audit

manager, “Overall, we see significant potential for generative AI- / ChatGPT- supported

projects across all Uniper functions. Once we manage to streamline the many individual

efforts that are already emerging, we will be able to utilize more impactful use cases.”

Finally, in the course of implementing ChatGPT, Uniper noted various risks and

challenges (see Table 1). They also identified key practices and rules to help mitigate these

risks and challenges (see Table 2). By moving into a dedicated, secured and Uniper only

Azure environment, some of the risks, such as data protection, could already be effectively

mitigated during the implementation phase. Uniper mandates the exclusive usage of this

dedicated environment. These risks, challenges, key practices, and rules should help

practitioners that are implementing ChatGPT while also serving as ideas for future research.

V. CONCLUSION
Uniper is utilizing ChatGPT to perform tasks within all parts of the internal audit

process. The initial tests by the IAF show that ChatGPT is helpful in all aspects of the audit

process and can result in significant efficiency gains. The company plans to continue

implementing this new technology within internal auditing.

Although initial tests have proven successful, auditors need to carefully evaluate the

risks and opportunities associated with using ChatGPT. While ChatGPT presents exciting

12

Electronic copy available at: https://ssrn.com/abstract=4514238

possibilities for internal and external auditors, it is crucial to assess the specific risks and

benefits in alignment with their intended applications. Evaluating the potential risks involves

considering factors such as the accuracy and reliability of the model, potential legal and

ethical implications, data privacy and security concerns, and the potential impact of biased or

inappropriate responses. On the other hand, identifying the opportunities entails

understanding how ChatGPT can enhance customer interactions, streamline processes,

provide personalized support, and improve overall efficiency. By conducting a thorough

evaluation of risks and opportunities, companies can make informed decisions about

incorporating ChatGPT into their operations, ensuring that the benefits outweigh the potential

drawbacks and aligning with their business goals. We encourage future research that helps to

identify the risks and benefits of using ChatGPT in auditing.

13

Electronic copy available at: https://ssrn.com/abstract=4514238

 References

Alareeni, B. 2019. A review of auditors’ GCOs, statistical prediction models and artificial
intelligence technology. International Journal of Business Ethics and Governance 2
(1): 19–31.

Anderson, U. L., M. H. Christ, K. M. Johnstone, and L. E. Rittenberg. 2012. A post-SOX

examination of factors associated with the size of internal audit functions. Accounting
Horizons 26 (2): 167-191.

Bartlett, G. D., J. Kremin, K. Saunders, and D. A. Wood. 2016. Attracting applicants for in-
house and outsourced internal audit positions: Views from external auditors.
Accounting Horizons 30 (1): 143-156.

Bartlett, G. D., J. Kremin, K. Saunders, and D. A. Wood. 2017. Factors influencing

recruitment of non-accounting business professionals into internal auditing.
Behavioral Research in Accounting 29 (1): 119-130.

Bommasani, R. et al. 2021. On the opportunities and risks of foundation models.

https://arxiv.org/abs/2108.07258.

Carcello, J. V., M. Eulerich, A. Masli, and D. A. Wood. 2020. Are internal audits associated
with reductions in perceived risk? Auditing: A Journal of Practice and Theory 39 (3):
55-73.

Cheng, X., et al. 2023. Artificial intelligence’s capabilities, limitations, and impact on
accounting education: Investigating ChatGPT’s performance on educational
accounting cases. Working Paper.

Christ, M. H., S. A. Emett, S. L. Summers, and D. A. Wood. 2021. Prepare for takeoff:
Improving audit efficiency and effectiveness with drone-enabled inventory audit
procedures. Review of Accounting Studies 26: 1323-1343.

Cooper, L. A., D. K. Holderness, T. Sorensen, and D. A. Wood. 2019. Robotic process

automation in public accounting. Accounting Horizons 33 (4): 15-35.

Cooper, L. A., D. K. Holderness, T. Sorensen, and D. A. Wood. 2022. Perceptions of robotic

process automation in Big 4 public accounting firms: Do firm leaders and lower-level
employees agree? Journal of Emerging Technologies in Accounting 19 (1): 33-51.

Ege, M. S., T. A. Seidel, M. Sterin, and D. A. Wood. 2022. The influence of management’s
internal audit experience on earnings management. Contemporary Accounting
Research 39 (3): 1,834-1,870.

Eulerich, M., J. Kremin, K. K. Saunders, and D. A. Wood. 2021. Internal audit stigma
awareness and internal audit outcomes. Corporate Ownership and Control 19 (1):
257-271.

Eulerich, M., J. Pawlowski, N. J. Waddoups, and D. A. Wood. 2022. A framework for using
robotic process automation for audit tasks. Contemporary Accounting Research 39 (1):
691-720.

14

Electronic copy available at: https://ssrn.com/abstract=4514238

Eulerich, M., A. Masli, J. Pickerd, and D. A. Wood. 2023. The impact of audit technology on
audit outcomes: Technology-based audit techniques’ impact on internal auditing.
Contemporary Accounting Research 40 (2): 981-1012.

Eulerich, M., A. Sanatizadeh, H. Vakilzadeh, and D. A. Wood. 2023. Can artificial

intelligence pass accounting certification exams? ChatGPT: CPA, CMA, CIA and
EA? Available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4452175.

Gu, H., M. Schreyer, K. Moffitt, and M. A. Vasarhelyi. 2023. Artificial Intelligence Co-
Piloted Auditing. Available at
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4444763.

IIA. (2013). The International Professional Practices Framework (IPPF).

ISO (2022). ISO/IEC 27001 Information security management systems.

Jan, C. L. 2021. Using deep learning algorithms for CPAs’ going concern prediction.
Information 12 (2): 73.

Kok, Ties de (Apr. 2023). “Generative LLMs and Textual Analysis in Accounting: (Chat)
GPT as Research Assistant?” In: SSRN Electronic Journal. doi:
10.2139/ssrn.4429658. url: https: //ssrn.com/abstract=4429658.

OpenAI. 2023. GPT-4 Technical Report. https://arxiv.org/abs/2303.08774.

Pickard, M. D., R. Schuetzler, J. Valacich, and D. A. Wood. 2020. Innovative accounting

interviewing: A comparison of real and virtual accounting interviewers. The
Accounting Review 95 (6): 339–366.

Prawitt, D. F., J. L. Smith, and D. A. Wood. 2009. Internal audit quality and earnings
management. The Accounting Review 84 (4): 1255-1280.

Radford, A. et al. 2018. Improving Language Understanding by Generative Pre-Training.

Available at https://cdn.openai.com/research-covers/language-
unsupervised/language_understanding_paper.pdf.

Salijeni, G., A. Samsonova-Taddei, and S. Turley. 2021. Understanding how big data
technologies reconfigure the nature and organization of financial statement audits: A
sociomaterial analysis. European Accounting Review 30 (3): 531-555.

Schreyer, M., M. Baumgartner, et al. 2022. Artificial Intelligence in Internal Audit as a

Contribution to Effective Governance. Expert Focus 01.

Sun, T. 2019. Applying Deep Learning to Audit Procedures: An Illustrative Framework.

Accounting Horizons 33 (3): 89–109.

Sun, T. and M. A. Vasarhelyi. 2017. Deep Learning and the Future of Auditing: How an
Evolving Technology Could Transform Analysis and Improve Judgment. CPA
Journal 87 (6).

15

Electronic copy available at: https://ssrn.com/abstract=4514238

Warren Jr, J. D., K. C. Moffitt, and P. Byrnes. 2015. How Big Data will Change Accounting.
Accounting Horizons 29 (2): 397–407.

Wood, D. A. et al. 2023. The ChatGPT Ar6ficial Intelligence Chatbot: How Well Does It Answer
Accoun6ng Assessment Ques6ons? Issues in Accounting Education: 1–28.

16

Electronic copy available at: https://ssrn.com/abstract=4514238

 Figure 1
The Use of ChatGPT in Various Stages of an Audit

17

Electronic copy available at: https://ssrn.com/abstract=4514238

 Figure 2
Example of Input Prompt and Results for using ChatGPT in an Internal Audit – Defining Risk Areas

18

Electronic copy available at: https://ssrn.com/abstract=4514238

 Figure 3
Example of Input Prompt and Results for using ChatGPT in an Internal Audit – Defining the Audit Scope

19

Electronic copy available at: https://ssrn.com/abstract=4514238

 Figure 4
Example of Input Prompt and Results for using ChatGPT in an Internal Audit – Brainstorming Interview Questions

20

Electronic copy available at: https://ssrn.com/abstract=4514238

 Figure 5
Example of Input Prompt and Results for using ChatGPT in an Internal Audit – Summarizing and Translating Fieldwork Notes

21

Electronic copy available at: https://ssrn.com/abstract=4514238

 Figure 6
Example of Input Prompt and Results for using ChatGPT in an Internal Audit – Researching Topics

22

Electronic copy available at: https://ssrn.com/abstract=4514238

 Figure 7
Example of Input Prompt and Results for using ChatGPT in an Internal Audit – Summarizing Relevant Documents

23

Electronic copy available at: https://ssrn.com/abstract=4514238

 Figure 8
Example of Input Prompt and Results for using ChatGPT in an Internal Audit – Reporting Audit Results

24

Electronic copy available at: https://ssrn.com/abstract=4514238

 Table 1
Identified Risks and Challenges by Uniper Internal Auditors

1. Inaccurate or misleading information: ChatGPT generates responses based on patterns in

the data it was trained on, which means it may provide incorrect or misleading
information at times. Users should exercise critical thinking and verify information
independently from reliable sources.

2. Bias in responses: AI models like ChatGPT can inadvertently reflect biases present in the
training data, leading to biased or unfair responses. This can perpetuate stereotypes,
discrimination, or misinformation. It’s important to monitor and address potential bias in
AI-generated content.

3. Lack of empathy and understanding: ChatGPT lacks genuine emotions and

understanding. It may provide responses that are technically accurate but lack empathy or
fail to grasp the emotional nuances of certain situations. Care should be taken when
discussing sensitive or emotional topics.

4. Privacy and security concerns: Sharing personal or sensitive information with ChatGPT
or any AI model can pose privacy and security risks. AI platforms should be treated as
public spaces, and users should avoid sharing confidential information such as
passwords, financial details, or personal identification numbers.

5. Dependence on AI without critical evaluation: Relying heavily on AI-generated responses

without critically evaluating them can lead to over-reliance on potentially flawed or
biased information. It’s important to use AI as a tool and supplement it with human
judgment and expertise. ChatGPT can and should be asked how it came to it conclusions
and what data was used. However, answers to these meta questions can be flawed too.

6. User manipulation and malicious use: AI models can be exploited for malicious purposes,
including generating deceptive or manipulative content. This can be particularly
concerning in the context of misinformation, social engineering, or phishing attempts.
Users should be cautious and vigilant while interacting with AI-generated content.

7. Legal and ethical considerations: Deploying AI systems like ChatGPT may raise legal
and ethical concerns, including intellectual property rights, copyright infringement, data
protection, and compliance with regulations. Organizations should ensure they have the
necessary legal and ethical frameworks in place to address these considerations.

25

Electronic copy available at: https://ssrn.com/abstract=4514238

 Table 2
Identified Key Practices and Rules by Uniper Internal Auditors

1. Verify information independently: Don’t blindly trust the information provided by

ChatGPT or any other AI model. Double-check facts and verify information from reliable
sources.

2. Exercise critical thinking: Remember that AI models generate responses based on

patterns in the data they were trained on. Apply critical thinking and consider the context
and reliability of the information provided.

3. Be cautious with personal and corporate information: Avoid sharing sensitive personal
and/or corporate information with ChatGPT or any AI model that could be harmful. Treat
AI platforms as public spaces and be mindful of privacy.

4. Set boundaries and be cautious with sensitive topics: Avoid discussing highly sensitive,
personal, or emotionally charged topics. ChatGPT is a machine learning model and may
not provide appropriate or empathetic responses in such cases.

5. Stay updated on security measures: Request approval from Corporate IT Security before
engaging with any new AI application. Stay informed about the security measures
implemented by the platform provider. Ensure that the platform encrypts data
transmission and follows best practices to protect user privacy.

6. Understand the limitations: Recognize that AI models have limitations and can
sometimes provide inaccurate or biased information. Use AI tools as aids, but always rely
on human judgment and expertise when making important decisions.

7. Stay informed about AI developments: Keep up with the latest advancements and
research in the field of AI to stay aware of potential risks and understand best practices
for using AI responsibly, for example Microsoft`s „Responsible AI Framework“.

8. Develop and implement a ChatGPT drivers license for users (in progress, working group
of Uniper together with Microsoft established): Implement a system where users need to
pass a verification process or complete an educational module on responsible AI usage
before gaining access to ChatGPT. This can help ensure that users are aware of the risks
and guidelines associated with the platform.

9. Learn from shared experience of implemented use cases by running a "road show" for
other colleagues: Organize internal knowledge-sharing sessions or "road shows" where
colleagues who have implemented ChatGPT or similar AI systems can share their
experiences, best practices, and lessons learned. This can help disseminate knowledge
and promote responsible usage within the organization. As of today, we deployed our
Internal Audit ChatGPT road show to approx. 360 Uniper colleagues and the count
continuous to increase.

26

Electronic copy available at: https://ssrn.com/abstract=4514238

10. Use ChatGPT in a "private" and secured space at the Uniper’s Azure cloud. This helps to
ensure better control over data security, access control, and compliance with Uniper’s
policies and regulations. Use that technical capability for training the model with Uniper
data.

27

Electronic copy available at: https://ssrn.com/abstract=4514238