LangerLanders2021ReviewAIatwork Preprint

See discussions, stats, and author profiles for this publication at: https://www.researchgate.
net/publication/351838348
The future of artiﬁcial intelligence at work: A review on effects of decision

automation and augmentation on workers targeted by algorithms and third-
party observers
Article in Computers in Human Behavior · May 2021

DOI: 10.1016/j.chb.2021.106878
CITATIONS READS
122 5,437
2 authors:
Markus Langer Richard N Landers

Philipps University of Marburg University of Minnesota Twin Cities
77 PUBLICATIONS 1,721 CITATIONS 130 PUBLICATIONS 5,925 CITATIONS
SEE PROFILE SEE PROFILE
All content following this page was uploaded by Markus Langer on 31 May 2021.
The user has requested enhancement of the downloaded file.

DECISION AUTOMATION AND AUGMENTATION 1
The future of artificial intelligence at work: A review on effects of decision automation and
augmentation on workers targeted by algorithms and third-party observers
Markus Langer1 & Richard N. Landers²
This preprint version may not exactly replicate the final version published in Computers in Human
Behavior. Licence CC BY-NC 4.0. Copyright: Markus Langer, Universität des Saarlandes,
Saarbrücken, Germany.
Please cite as: Langer, M., & Landers, R. N. (2021). The future of artificial intelligence at work: A
review on effects of decision automation and augmentation on workers targeted by algorithms and
third-party observers. Computers in Human Behavior.
Work on this paper was funded by the Volkswagen Foundation grant AZ 98513 “Explainable Intelligent
Systems” (EIS)
1
Fachrichtung Psychologie, Universität des Saarlandes, Saarbrücken, Germany
²Department of Psychology, University of Minnesota, Minneapolis, MN
Correspondence concerning this article should be addressed to Markus Langer, Universität des Saarlandes,
Arbeits- & Organisationspsychologie, Campus A1 3, 66123 Saarbrücken, Germany. Phone: +49 681 302
4767. E-mail: markus.langer@uni-saarland.de

Highlights: The future of artificial intelligence at work: A review on effects of decision automation
and augmentation on workers targeted by algorithms and third-party observers
- We review how decision automation and augmentation affects second and third parties
- We explore various moderators of attitudes, perceptions, and behavioural outcomes
- Reactions to decision augmentation differ from reactions to decision automation
- We provide recommendations regarding methodology, terminology, and theory
- System design choices are important but respective studies underrepresented

Abstract
Advances in artificial intelligence are increasingly leading to the automation and augmentation of decision
processes in work contexts. Although research originally generally focused upon decision-makers, the
perspective of those targeted by automated or augmented decisions (whom we call “second parties”) and
parties who observe the effects of such decisions (whom we call “third parties”) is now growing in
importance and attention. We review the expanding literature investigating reactions to automated and
augmented decision-making by second and third parties. Specifically, we explore attitude (e.g., evaluations
of trustworthiness), perception (e.g., fairness perceptions), and behavior (e.g., reverse engineering of
automated decision processes) outcomes of second and third parties. Additionally, we explore how
characteristics of the a) decision-making process, b) system, c) second and third party, d) task, and e) outputs
and outcomes moderate these effects, and provide recommendation for future research. Our review
summarizes the state of the literature in these domains, concluding a) that reactions to automated decisions
differ across situations in which there is remaining human decision control (i.e., augmentation contexts), b)
that system design choices (e.g., transparency) are important but underresearched, and c) that the
generalizability of findings might suffer from excessive reliance on specific research methodologies (e.g.,
vignette studies).
Keywords: automated and augmented decision-making, artificial intelligence, algorithmic decision-
making; perceptions, attitudes, review paper

1. Introduction
For over half a century, research and practice have explored how decision-making automation,
which refers to automating decision-processes without remaining human control, and augmentation, which
refers to the addition of system-support for human decisions, can increase decision quality and efficiency
(Benbasat & Nault, 1990; Meehl, 1954; Parasuraman et al., 2000; Raisch & Krakowski, 2021). In
psychology, those concepts date back until at least Meehl (1954), who argued that it could be possible to
translate decisions made by humans in a subjective and informal way (clinical or holistic decision-making)
into a structured and formal way (mechanical or actuarial decision-making). Nowadays, advances in
artificial intelligence (AI)1 help to realize this structured way of decision-making in many application areas.
For example, AI-based systems increasingly automate or augment aspects of decision-making in medicine
and management (Burton et al., 2020; Longoni et al., 2019).
With good design and adequate testing, decision automation and augmentation systems can often
provide better and more efficient decisions than even the most experienced human experts (Grove et al.,
2000; Kuncel et al., 2013). However, these benefits can be undermined by poor system design, misuse, and
reluctance to adopt systems by first-party users (Dietvorst et al., 2015; Parasuraman & Riley, 1997). We use
the term first party to refer to people who use or interact with the output of such systems to make decisions
that affect other people. First-party users are distinct from developers, who develop systems and then
monitor, maintain, and update them. They are also distinct from upper-level managers, who may be
responsible for implementation in a more abstract way but do not work directly with the systems. Our
definition of first parties refers to people who have at least some direct control over whether and to what
degree an artificial system will alter the decisions they personally make. A prototypical first party at work
is a manager who uses a system to augment aspects of their decision-making process regarding the personnel
they manage. First parties often have the freedom to question the quality of the systems they are employing
1
We use artificial intelligence as an umbrella term, subsuming both classical manifestations like expert systems and
deterministic human-programmed algorithms with more recent ones, like machine learning and deep learning.
and deviate from their recommendations, relying instead or more heavily upon their own judgment
(Benbasat & Nault, 1990; Highhouse, 2008).
To date, the majority of research has investigated the perspectives of first parties (Benbasat & Nault,
1990; Burton et al., 2020; Hoff & Bashir, 2015), despite them being only one part of a complex network of
stakeholders for almost any automation or augmentation system (Jungmann et al., 2020; Langer et al., 2021).
For this paper, two other types of stakeholders are central. We call them second parties and third parties.
Second parties are people whose lives, work, and well-being are directly affected and targeted, often without
their consent or knowledge, by automated and augmented decisions. Second parties cannot choose whether
they want to be affected by systems, their outputs, or decisions based on those outputs unless they exit the
decision-making context entirely, such as by quitting their job. Prototypical second parties are employees
that receive work tasks from automated systems (e.g., Uber drivers; M. K. Lee et al., 2015), employees
whose shifts are automatically determined (Uhde et al., 2020), and job applicants whose application
information is evaluated by first parties supported by automated systems (Langer, König, & Hemsing,
2020). Third parties are people who observe an automated or augmented decision without being directly
affected by that decision. Third parties are not directly affected by a particular decision but may feel that
they could become a second party in the future or are concerned for the well-being, privacy, or some other
characteristic of second parties. For example, prototypical third parties are people reading news articles on
automated personnel selection practices or people who hear from friends who are working in jobs where
they receive performance evaluations by automated systems (e.g., in algorithmic management contexts;
Wesche & Sonderegger, 2019). At higher levels of analysis, the label “third party” can even apply to average
group, cultural, or societal reactions to specific decisions or policies, such as global reactions to Amazon’s
failure to automate resume screening procedures without undue bias (Lecher, 2019).
Understanding the perspectives of second and third parties to automated and augmented decision-
making at work constitutes a crucial emerging line of research as decision automation and augmentation
increasingly determine how work takes place (Kellogg et al., 2020; Murray et al., 2020). Even if decision
automation and augmentation is accepted by first parties, second and third parties can either foster or impair
success in practice (Healy et al., 2020). Specifically, second parties might be influential in improving or
sabotaging first-party trust in the accuracy, efficiency, and consistency of automated and augmented
decisions by providing direct feedback to those first parties (M. K. Lee et al., 2015). Similarly, third parties
could protest or use negative word-of-mouth on social media to attempt to shape public opinion. Such
behavior can discourage first parties from employing automation or augmentation, can affect policy makers
and regulators in a way that influences the application of decision automation and augmentation in practice,
can diminish organizational reputation and even spur litigation. Although previous research has investigated
first-party perspectives on decision automation and augmentation (at least in certain application areas
Benbasat & Nault, 1990; Burton et al., 2020; Hoff & Bashir, 2015; Onnasch et al., 2014), researchers only
began to explore effects on second and third parties in the beginning of the 2000s. This newer line of research
is not as well understood.
We contend that this area of research is now at a stage where a review is crucial to reveal systematic
issues and blind spots that need to be addressed in the future. Consequently, the primary research questions
targeted with this review are: (1) how do automated and augmented decisions affect second and third
parties?, (2) what moderates these effects?, and (3) what are the next crucial steps to advance this research
area? In the following section, we describe our review methodology. Resulting from our review, we present
effects (on attitudes, perceptions, behavior) of decision automation and augmentation on second and third
parties, including moderation by characteristics of the decision-making process, of the system, of the second
and third parties themselves, of the tasks, and of outputs and outcomes. We conclude by highlighting
limitations observed across the literature and by providing recommendations for future research.
2. Review Methodology
Prior to our search we defined the following inclusion criteria for records considered in our review.
Specifically, we did not restrict our search to any specific timeframe and only included records written in
English. Moreover, we determined to only include research referring to the use of systems automating or
augmenting decision processes with the potential to directly affect an identified second party. We also
restricted our review to research that collected or interpreted empirical data, whether qualitative or
quantitative, investigating the effects of decision automation or augmentation on second- or third-party
experiences and behavioral outcomes. Furthermore, we only included research that presented a
comprehensible description of their study procedures and analysis methodology, which was not universal.
Given our research questions, we focused our initial review on research about decisions affecting people in
work-related contexts. However, in the course of that review, we found a variety of papers in the field of
medical decision-making that appeared to be relevant to work-related contexts. Medicine is somewhat
unique in that systems may automate or augment medical employees’ (e.g., physicians, nurses, technicians)
decisions about patients instead of about workers or job applicants. Yet because we observed that research
on augmented and automated medical decision-making is in often more advanced than for managerial
decision-making, we decided to explicitly include medicine in this review. Recognizing potential challenges
in the generalizability of this research to management, we discuss these examples only where necessary to
illustrate trends and theoretical avenues not yet explored in management yet relevant to management theory.
We conducted our primary literature search on the SCOPUS database and followed up with a search
on Google Scholar. On each database, we first conducted a preliminary search in which we identified
relevant sources and search terms, and which revealed that research relevant to this review could be found
in a variety of disciplines with different publishing traditions. For instance, in psychology, scholars
generally value journal publications as scholarly contributions whereas conference proceedings are often
unavailable and conference presentations are minimally valued. In comparison, although computer science
journals exist, the majority of scholarly work is published in conference proceedings. Given our goals for
an interdisciplinary review, we acknowledged these varying publishing traditions and included published or
accepted work in conference proceedings, academic journals, and books. For work found in online
repositories that also cover preprints (e.g., arXiv), the authors discussed whether those articles should be
included based upon their individual quality. Furthermore, we talked to subject matter experts from various
disciplines to ensure that we identified the most relevant outlets and conferences.
We required for inclusion at least one hit in each category among search terms referring to a) the
system, b) where the system is used or who is affected, c) terms referring to reactions or perceptions by
second and third parties, and d) the study methodology. Within each of the following lists of final search
terms, multi-word terms were treated as one search term and logically separated by “or.” For (a), we used:
“automated, automation, algorithm*, artificial intelligence, autonomous machine, computer-assisted,
computer-based, decision support system, expert system, intelligent system, machine learning”. For (b), we
used: “advisor*, employee, individuals, job, management, manager, managerial, office, organization,
physician, patients, workplace.” For (c), we used: “accept*, belief, fairness, perceiv*, perception, react*,
respond*, satis*”. For (d), we used: “applicants, employee, experiment, field study, laboratory study,
participants, subjects, worker.” Among papers deemed relevant, we used a snowball technique (Wohlin,
2014), seeking additional relevant articles by scanning references within already-identified articles, as well
as scanning papers citing the respective paper (via Google Scholar) and repeating this process until relevant
references were exhausted. Figure 1 provides a flowchart outlining search steps taken and the number of
sources at each stage of filtering. Table 1 summarizes the final set of studies that served as the basis of this
review and also provides contextual information regarding those studies.
Virtually all research that we identified on automated and augmented decision-making at work
related to human resource management tasks such as personnel selection or scheduling. We also became
aware of a substantial number of papers that came from the extensive area of algorithmic management.
Duggan et al. (2020 p.119) defined algorithmic management “as a system of control where self-learning
algorithms are given the responsibility for making and executing decisions affecting labour, thereby limiting
human involvement and oversight of the labour process.” Although we sought to include this research where
relevant, it was often unclear whether reported findings referred to participant experiences with automated
decisions or with the general work environment (e.g., being self-employed, not having a contract; Galière,
2020; Möhlmann et al., in press), and we only included research that was unambiguously focused upon
reactions to the augmented or automated decision-making itself per our inclusion criteria. As a result, we
included few algorithmic management papers relative to the overall size of that research literature.
One insight that emerged from our review was the many ways that decision automation and
augmentation could be realized, which might ultimately affect second- and third-party reactions. Most
critically, we found that a continuum between full human and full system control (or automation) can be
conceptualized. To formalize this continuum, Kaber and Endsley (2004) presented a taxonomy defining it
across ten distinct levels, ranging from human control, in which a first party is in complete control of a
decision and all information used to make it, to full system control, in which decisions are fully automated
without human oversight or intervention. Between those extremes, the specific role of both humans and
systems vary widely. At the lower end of system control, systems may support first parties by, for instance,
providing processed or evaluated information (Acikgoz et al., 2020; Suen et al., 2019), leaving the final
decision at the discretion of the first party. As system control increases, there is greater interaction between
system and human (O’Neill et al., 2020). For instance, in human-in-the-loop augmentation, first parties and
systems exchange information before an action is taken, meaning that the system might request information
from the human or vice-versa (van Dongen & van Maanen, 2013). As system control increases further,
human control diminishes, until reaching full system control. Although taxonomies regarding the
automation and augmentation decision-making were developed with first parties in mind (see also Makarius
et al., 2020; Parasuraman et al., 2000), these strategies describing how decisions are made could also affect
reactions by second and third parties and were thus considered important for our review. Specifically, we
anticipated that second- and third-party experiences of automated decisions vary by decision agency as
control shifts from a human to an automated agent. In our review, we ultimately used this framework as an
organizational tool and moderator of interest, as classified in Table 1. Specifically, we categorize the
reviewed papers into papers that investigate automated, system-controlled decision-making (Auto), decision
augmentation, where human and system interact in decision-making (Augm), and human decision-making
(Human).
Records identified through

database search
(n = 12,481)
Additional records identified by Records excluded with respect

snowballing; titles and abstracts Titles and abstracts screened to our inclusion criteria upon
screened title and abstract review
(n = 110)
(n = 114) (n = 133)
Full text articles screened Records excluded with respect

to our inclusion criteria upon
(n = 91)
full text review
(n = 32)
Studies included in review

(n = 59)
Figure 1. Flow diagram of search steps. Dashed lines indicate the paths where identified records were excluded.
Table 1.
Overview of the number of participants, the context, the decision process, methodology, words used to refer to the system and system function in the reviewed studies.
Study Participants Context Auto Augm Human Methodology Words used to refer to the system System functioning
Decision-making at work (general)
Acikgoz et al., 2020 298 US MTurk Personnel selection (x) (x) (x) All AI software using advanced algorithms Screens resumes, conducts interviews, rates
Study 1 quantitative interviews, makes recommendation to manager
vignette
Study 2 225 US students Personnel selection (x) (x) (x) AI software using advanced algorithms Conducts interviews, rates interviews, makes
recommendation to manager
Binns et al., 2018 19 UK, no further Promotion x All Computer system based on a computer Evaluates whether a person should receive a
Study 1 information quantitative model promotion
vignette
Study 2+3 Study 2: 325 Promotion x Computer system, predictive model Same as in Study 1
Study 3: 65
Both UK Prolific
Bigman et al., 2020 2090 American Personnel Selection x All Computer program Reviews resume and interview information,
Pilot Study Trends Panel quantitative gives applicant scores, applicants are hired
participants vignette based on these scores
Study 2A-D A: 122 MTurk Personnel selection x x CompNet an artificial-intelligence-based Hires applicants
B: 241 Mturk computer program
C: 241 Mturk
US, Canada
D: 1499 UK
representative
sample from
Prolific
Study 3 240 MTurk Personnel selection x x Same as in 2A-D Hires applicants
US, Canada
Study 4 964 MTurk Personnel selection x x Same as in 2A-D Hires applicants
US Canada
Study 5 155 Norwegian Personnel selection x x Same as in 2A-D Hires applicants
tech workers
Dineen et al., 2004 76 US students Personnel selection x x Quantitative Automated screening system Reviews applications, informs applicants
vignette whether they were chosen
Gonzalez et al., 192 US MTurk Personnel Selection x x Quantitative AI/ML tool Makes hiring decision
2019 vignette
Höddinghaus et al., 333 German Compensation, x x Quantitative Computer program Allocates and selects workshops and training
2020 workers training, promotion vignette courses
Allocates and determines monthly bonuses and
promotion decisions
Hong et al., 2020 233 US Qualtrics Personnel Selection x Quantitative AI, algorithm, program, system Chats with participants, evaluates interview
vignette questions and resume information
Table 1 continued.
Howard et al., 2020 22 US physicians Scheduling x x Quantitative, Automated approach Makes schedules, assigns people to shifts
reactions to
actual
decisions
Langer, König, & 124 German Personnel selection x x Quantitative Computer Analyzes and evaluates participants interview
Hemsing, 2020 students and responses
participants
record
interviews
Langer, König, & 123 German Personnel selection x x Quantitative Virtual interview tool Evaluates people, provides feedback to people,
Papathanasiou, students and decides whether applicants proceed to next
2019 participants stage
watch a video
Langer, König, 148 German Personnel selection x x Quantitative Virtual interview tool Interviews applicants
Sanchez, et al. 2019 students and and
working individuals participants
watch a video
Langer et al., 2018 120 German Personnel selection x x Quantitative Virtual interview tool Interviews applicants
students and
(psychology and participants
computer science) watch a video
Lee, 2018 228 US MTurk Personnel selection, x x Quantitative Algorithm Assigns tasks
Scheduling, and qualitative Decides who should come to work
Performance vignette Reviews resumes and selects top candidates
evaluation Evaluates employees
Work assignment
Marcinkowski et al., 304 German Selection of (x) x Quantitative AI technology Analyzes applicant data and recommends
2020 students students for vignette approval or rejection of applicants
university
admission
Miroswska, 2020 184 students Personnel selection x x Quantitative AI interview assessment software Reviews interviews
vignette
Nagtegaal 2021, 109 Dutch alumni Reimbursement, x x x Quantitative Computer, using an automated algorithm Decides about travel reimbursement
Study 1 performance and qualitative Evaluated performance of employees
evaluation vignette
Study 2 126 UK Prolific, Pension calculation, x x x Quantitative Computer, using an automated algorithm Calculates pensions
employees of hiring and qualitative Scans CVs, interviews candidates, hiring
governmental vignette
organizations
Table 1 continued.
Newman et al., 199 US MTurk Layoffs and x x All An algorithm (i.e., a computerized Determines who gets promoted or laid off
2020, Study 1 promotions quantitative decision-making tool)
vignette
Study 2 1654 US university Employee x x Human resource algorithm Decisions regarding promotions, layoffs,
employees evaluation raises, pay cuts
Study 3 189 US MTurk Bonus payment x x x An algorithm (a computerized decision- Determines how employee bonuses should be
determination making tool) allocated
Study 4 197 US students Employee x x Algorithm Evaluates employee performance data and
evaluation comes to a final decision on the performance
review
Study 5 213 US students Personnel selection x x Algorithm Evaluates recorded responses, top scorers will
be put on a short list
Nolan et al., 2016 468 US MTurk Personnel selection (x) x Quantitative Computer program that uses a Combines data and calculates overall scores
Study 1 vignette mathematical formula for candidates
Ötting & Maier, 149 German Task allocation x x Quantitative Intelligent computer Decides about task allocation
2018 Study 1 students vignette Humanoid robot
including
pictures
Study 2 145 German Allocation of x x Quantitative Intelligent computer Decides about allocation of vocational training
students vocational training vignette Humanoid robot
including
pictures
Schmoll et al., 2020 144 German Personnel selection x x Quantitative Self-learning algorithm Screens applicants’ social media profiles
students vignette
Suen et al., 2019 180 actual Chinese Personnel selection x x Quantitative, AI algorithm Analyzes interviews, serves as a reference for
interviewees real interviews hiring decisions
for simulated
job
Uhde et al., 2020, 51 German Scheduling (who x x Quantitative System Decides who gets vacation but also encourages
Study 2 healthcare workers gets vacation) and qualitative workers to find a solution for themselves
vignette
Van Esch et al., 293 non-specified Recruitment x Quantitative AI Analyzes applicant information
2019 vignette,
survey
Wang et al., 2020 579 US MTurk Work evaluation x Quantitative Algorithm Processes MTurkers work history, decides
and promotion vignette over who will become a Master worker
Table 1 continued.
Decision-making at work (algorithmic management in gig and platform work)
Anwar et al., 2021 19 Indian workers Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
management interviews manages evaluation process
Bucher et al., 2021 12294 posts on Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
forum management analysis of manages evaluation process
forum posts
Galière, 2020 21 interviews with Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
French workers management interviews manages evaluation process
Griesbach et al., 955 US workers Algorithmic x Quantitative Algorithm Assigns work tasks, provides information,
2019 survey management survey, manages evaluation process
55 US workers qualitative
interviews interviews
Jarrahi & 33 workers Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
Sutherland, 2019 98 threads from management interviews, manages evaluation process
forums documents
Probably all US websites,
forum posts
Jarrahi et al., 2020 20 probably US Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
workers management interviews, manages evaluation process
Forum posts forum posts
Kinder et al., 2019 20 workers Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
19 clients management interviews, manages evaluation process
125 forum forum posts
discussions
Lee et al., 2015 21 US drivers Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
12 US passengers management data from manages evaluation process
128 posts in online semi-
forums structured
132 official blog interviews,
posts analysis of
posts in online
forums,
analysis of
blog posts by
the companies
Möhlmann & 15 informal, 4 Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
Zalmanson, 2017 formal interviews management interviews, manages evaluation process
with US and UK post entries;
workers informal and
formal
interviews and
blog posts
Table 1 continued.
Möhlmann et al., in 15 informal, 19 Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
press formal interviews management interviews manages evaluation process
with US workers
8 formal interviews
with employees and
engineers
Myhill et al., 2020 32 interviews with Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
Scottish workers management interviews manages evaluation process
Ravenelle, 2019 31 US workers Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
from two platforms management interviews manages evaluation process
Wood et al., 2019 107 interviews with Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
workers management interviews, manages evaluation process
679 workers in survey
survey
Southeast Asia and
Sub Saharan
countries
Venn et al., 2020 58 Australian Algorithmic x Qualitative Algorithm Assigns work tasks, provides information,
workers management interviews manages evaluation process
Decision-making in medicine
Arkes et al. 2007, 347 US students All diagnosis (x) x All Computer program Tells the physician whether they should order
Study 1 quantitative an X-ray
vignette
Study 2 128 US students (x) x Computer program Determines risk of diseases, advices treatment
Study 3 74 US patients (x) x Computer program Assigns likelihood of diagnoses
Study 4 131 US medical (x) x Computer program Assigns likelihood of diagnoses
students
Araujo et al., 2020 958 Dutch, Treatment (and x x Quantitative Artificial intelligence, computers, Decides about medical treatment
representative others) vignette computer programs
sample
Bigman & Gray, 240 US MTurk Treatment x x All HealthComp an autonomous statistics- Decides whether to perform a surgery
2018, Study 3 quantitative based computer system
vignette
Study 6 239 US MTurk x x Same as in Study 3 Same as in Study 3
Study 7 100 US MTurk x x x Same as in Study 3 Same as in Study 3
Study 8 240 US MTurk x x Same as in Study 3 Same as in Study 3
Study 9 Within: 201 x x Same as in Study 3 Same as in Study 3
Between: 409
Both US MTurk
Table 1 continued.
Haan et al., 2020 20 US patients Diagnosis (x) x Qualitative AI system, computer Analyzes radiological images and
semi- autonomously evaluates scans
structured
interviews
Hamilton et al., 46 US patients Diagnosis and x Group IBM Watson for oncology Analyzes patient data, gives treatment
2019 treatment interviews, recommendation
participants
watched a
video
Jutzi et al., 2020 298 German Diagnosis x Qualitative AI Analyzes melanoma
patients and survey
Jonmarker et al., 2196 Swedish Diagnosis x x x Qualitative Computer-only Breast cancer diagnosis
2019 women and actual survey
patients
Keel et al. 2018 98 Australian Diagnosis x x Quantitative, Automated system Diagnoses retinopathy
patients reactions to
actual decision
Longoni et al., 2019 228 US students Diagnosis x x All Computer uses an algorithm Stress diagnosis
Study 1 quantitative
vignette
Study 2 103 US MTurk Diagnosis x x Computer capable of artificial Stress diagnosis
intelligence
Study 3 3a: 205, 3b: 227, Diagnosis x x 3a: robotic dermatologist is a computer 3a: skin cancer screening
3c:235 all US program 3b: diagnosis
MTurk 3b: robotic nurse, interactive animated 3c: helps human to conduct surgery
avatar
3c: Robotic surgeon
Study 4 100 US MTurk Diagnosis x x Same as 3a Skin cancer screening
Study 5 286 US MTurk Diagnosis x x Robotic dermatologist Skin cancer screening
Study 6 243 US MTurk Treatment x x Computer Recommendation of bypass operation
Study 7 294 US MTurk Treatment x x Computer program uses an algorithm Recommendation of bypass operation
Study 8 401 US MTurk Diagnosis x x Computer program uses an algorithm Recommendation of bypass operation
Study 9 179 US MTurk Diagnosis x x x AI dermatologist that is an algorithm Skin cancer screening
Study 10 No information Diagnosis Robotic ophthalmologist, computer that Dry eyes diagnosis
uses an algorithm
Study 11 92 US MTurk Various x x Well-trained algorithm Gives advice
Nelson et al. 2020 48 US patients Diagnosis x x x Qualitative AI program, AI tool Automated diagnosis or support of diagnosis
semi-
structured
interviews
Table 1 continued.
Palmeira & 36 US panel University x x x All Computer program Provides a favorability score to rank
Spassova, 2015, admission quantitative candidates
Study 1 vignette
Study 2 Between: 117 US Diagnosis and (x) x Computer program and decision aid Determines risk of diseases advices treatment
panel treatment
Mixed: 75 US
panel
Study 3 Medical: 41 US Diagnosis, x x x Medical software Informs about symptoms/candidate
panel university characteristics and probability of
Admission: 42 US admission illness/success
panel
Palmisciano et al., 20 UK patients, Diagnosis and x Qualitative AI system Analyzes images, works out surgical plan,
2020 qualitative treatment and alarms about risks, supports surgeon, operates
107 UK patients quantitative patient autonomously
and relatives survey with
vignettes
Pezzo & Pezzo 59 US students All diagnosis (x) x All Computer decision aid Combines test results
2006, Study 1 quantitative
vignette
Study 2 166 US medical (x) x Computer decision aid Makes a diagnosis
students and 154
students
Promberger & 68 US panel Diagnosis and x x All Computer program Provides recommendation
Baron, 2006, Study treatment quantitative
1 vignette
Study 2 80 US panel Diagnosis and x x Computer program Provides recommendation or autonomous
treatment decision
Shaffer et al., 2013, 434 US students Treatment (x) x All Decision aid [computer program] Determines whether patient should have an X-
Study 1 quantitative ray
vignette
Study 2 109 US students Treatment (x) x Decision aid [computer program] Determines whether patient should have an X-
ray
Study 3 189 US students Diagnosis (x) x Computer-based diagnostic aid Calculates a score that tells the probability for
appendicitis
Srivastava et al., 100 US MTurk Diagnosis x Quantitative, Data driven algorithm Skin cancer screening
2019 decision
between
different forms
of algorithms
Stai et al., 2020 264 US participants Diagnosis and x x Quantitative AI Analyzes images, gives recommendation on
treatment survey further treatment
Table 1 continued.
Tobia et al., 2021 1356 US Treatment x Quantitative AI Recommends chemotherapy drug dosis
representative vignette
sample
Wolf 2014, Study 1 218 US IT students All diagnosis (x) x All Computer program Assigns likelihood of diagnoses
quantitative
vignette
Study 2 101 US IT students (x) x Computer program Assigns likelihood of diagnoses
Yokoi et al., 2021 272 Japanese Diagnosis and x x Quantitative AI Analyzes symptoms, identifies disease,
college graduates treatment vignette suggests medical treatment
York et al., 2020 216 UK patients Diagnosis and x x Quantitative AI Helps human to analyze X-rays, helps deciding
treatment survey how to manage injuries
Note. Auto = this study investigates automated, system-controlled decision-making; Augm = this study investigates decision augmentation, where humans and systems interact in decision-making; Human = this
study investigates human decision-making. (x) = unclear description of decision-making situation. The column “Words used to refer to the system” consists of quotes of the respective papers. For papers including
vignette studies, this highlights the words used to describe the system to participants within the respective vignette studies.
3. Effects of Decision Automation and Augmentation on Second and Third Parties
Our review of the effects of decision automation and augmentation based on the studies presented
in Table 1 is summarized in Table 2. For clarity, we structured the results of our research, both in that table
and in the remainder of this section, in three components: attitudes, perceptions, and behavior. Considered
from the perspective of the study participant in which these constructs are studied, perceptions refer to how
people immediately feel about and understand a system’s actions, whereas attitudes refer to conscious
evaluations of systems when queried. Behavior is defined as actions taken by second and third parties in
response to their experiences or observations, respectively, of automated or augmented decision-making.
Importantly, all three outcome categories are likely to recursively affect each other (Fazio & Williams,
1986), so none should be considered in isolation.

Table 2
Effects of decision automation and augmentation on second and third parties.
Outcome Current consensus Sample research question Selected sources
Attitudes
Evaluations of People are sometimes unsure about the ability of humans using systems, tend to What are the contexts where people are initially Höddinghaus et al. 2020;
trustworthiness question the ability of automated systems for decisions, but also sometimes expect sceptical about ability of automated systems for Nelson et al., 2020
systems to be equally or more able and efficient as humans. Additionally, people decisions?
believe that systems have higher integrity and are less biased than humans.
Furthermore, systems are perceived as less benevolent, but evidence is limited.
Reductionism Decision automation neglects unique conditions, qualitative information, and Is it possible to alter attitudes regarding Longoni et al., 2019;
decontextualizes as well as quantifies decision processes. reductionism? Newman et al., 2020
Reluctance regarding Evidence is limited but people do not appear to want decision automation for Why are people sceptical about decision Bigman and Gray, 2018
automation of moral decisions with obvious moral components. automation for decisions with moral
decisions components?
Perceptions
Fairness and Justice Mixed results; more studies have found negative fairness perceptions, but some What moderates effects on fairness and justice Howard et al., 2020;
have found positive effects or no effects, this seems to depend on the task at hand. perceptions? Newman et al., 2020;
Decision automation generally leads to negative interpersonal justice perceptions, Ötting & Maier, 2018
to better perceptions of consistency, and to lower perceived controllability. Effects
regarding informational justice are inconclusive.
Organizational Mixed findings regarding organizational attractiveness. How do other perceptions and attitudes Langer, König, &
attractiveness translate to organizational attractiveness? Papathanasiou, 2019;
Newman et al., 2020
Accountability and Research suggests that decision automation and augmentation affect accountability What are practical implications of potentially Pezzo & Pezzo, 2006;
responsibility and perceived responsibility but findings are limited to vignette studies and mostly diffused responsibility between decision- Tobia et al., 2021
stem from the medical domain. makers and automated systems?
Being under system In algorithmic management, there is a tension between being under constant system How does the tension between control and Möhlmann et al, 2017;
control vs. autonomy control but at the same time feeling autonomy because of having no human boss. autonomy affect workers’ job satisfaction? Griesbach et al., 2019
Privacy concerns Automated decision-making appears to lead to greater privacy concerns, but the Why do people perceive privacy concerns for Langer, König, &
literature is small. automated decisions? Papathanasiou, 2019
Behavior
Reverse engineering of Some second parties appear to try to reverse engineer (e.g., through Does reverse engineering improve perceived Lee et al., 2015; Kinder
automated decisions experimentation) the functioning of automated systems. control of automated decisions? et al., 2019
Creating workarounds Some second parties try to use system peculiarities for their own benefit or to How do automated systems and bypassing Möhlmann et al., in
circumvent control by the automated system if the outcome is valued. behavior coevolve? Who seeks workarounds? press; Wood et al., 2019
Collective action Some second parties use online forums to share knowledge, complain, and engage Would this kind of collective action also Möhlmann et al., in press
in collective action against automated systems. translate to employees in organizations?
Embracing the system People try to keep their own ratings within systems high, embrace system Does embracing the system undermine Galière, 2020; Wood et
functionality as being fair and efficient. scrutinizing of system outputs? al., 2019
3.1 Attitudes
We identified three major types of attitudes in relation to decision automation and augmentation:
evaluations of trustworthiness, reductionism, as well as reluctance regarding automation of moral decisions.
3.1.1 Evaluations of Trustworthiness
A number of studies investigate second- and third party evaluations of trustworthiness facets that
relate to the system or the decision-maker receiving system support (J. D. Lee & See, 2004; Mayer et al.,
1995). Specifically, in the terminology of Mayer et al.’s (1995) facets of trustworthiness, the reviewed
papers predominantly investigate evaluations of ability and integrity, and some papers also evaluate
benevolence. In the reviewed studies, ability relates to perceived performance of a system or the abilities of
a decision-maker supported by a system. Integrity refers to the believe that the first party or system is
unbiased in their decisions. Benevolence relates to the evaluation of how much a first party or a system will
consider humans’ needs and how much they “care” about individuals. Second parties will consider the
combination of those trustworthiness facets when they decide whether to trust or rely on the respective
automated or augmented decision (Höddinghaus et al., 2020; J. D. Lee & See, 2004).
Overall, results regarding assessment of the trustworthiness facet ability are inconclusive.
Performance and ability evaluations are central to the trustworthiness of first parties and of automated
systems (J. D. Lee & See, 2004). In medical decision-making, there is research investigating second-party
perceptions of the ability of human experts using systems for decision-making. For instance, research
indicates that second and third parties can ascribe lower ability to first parties that use systems in their
decisions (Arkes et al., 2007; Wolf, 2014). However, there is no consensus whether those reactions are
specific to first parties consulting systems versus consulting other human experts (V. A. Shaffer et al., 2013),
if the magnitude of reactions is dependent upon the level of automation (Palmeira & Spassova, 2015), or on
the behavior of the first party (Pezzo & Pezzo, 2006). When explicitly comparing augmented decisions with
unaided expert judgement, using systems can even result in better perceptions of the abilities of the first
party (Hamilton et al., 2019; Palmeira & Spassova, 2015). In addition, second parties seem to be concerned
about human deskilling, such as by worrying that physicians will become less able to diagnose without
augmentation systems and eventually lose their ability to detect system errors (Hamilton et al., 2019; Jutzi
et al., 2020; Nelson et al., 2020).
Further important with respect to ability, in the case of fully-automated decisions, second parties
seem to be initially unsure about the performance of automated systems in high-stakes decision contexts.
For example, in qualitative studies, several representatives of second parties reported that they expect more
accurate whereas others expect less accurate diagnoses when using decision automation in medicine (Jutzi
et al., 2020; Nelson et al., 2020). People also seem to question the validity of automated decisions, and
patients may even explicitly call for scientific evidence showing that they can trust automated systems (Haan
et al., 2019). Höddinghaus et al. (2020) found no differences regarding people’s evaluation of human versus
system data processing capabilities in different managerial tasks. However, humans were evaluated as more
adaptable to changing circumstances which the authors consider another facet of ability. Since automated
systems might not only aim to increase diagnostic accuracy but also decision-making efficiency, efficiency
perceptions can also contribute to overall evaluations of the ability (i.e., performance) of automated systems.
People’s expectations are split regarding efficiency of automated systems as some studies showed that
people expect increasing diagnostic speed and earlier detection of diseases (Haan et al., 2019; Hamilton et
al., 2019; Jutzi et al., 2020; Nelson et al., 2020) whereas others found no differences in expectations
regarding waiting time expectations (Arkes et al., 2007; V. A. Shaffer et al., 2013; Wolf, 2014). Similarly,
high expectations regarding efficiency were reported in studies investigating workers’ beliefs about
algorithmic management (Galière, 2020).
Regarding the trustworthiness facet integrity, people seem to predominantly believe systems to
possess high integrity. People seem to believe that systems have less discrimination motivation than humans
(Bigman et al., 2020) which seems to be related to beliefs that systems possess less agency (Bigman & Gray,
2018) and with the belief that systems do not have an own agenda they follow (Myhill et al., 2021).
Furthermore, Höddinghaus et al. (2020) also found that people perceive systems to be less biased than
human decision-makers. It is important to stress that attitudes regarding integrity of automated systems are
closely related to what we discuss under perceptions of consistency and objectivity. Specifically, if people
believe the decision agent to have less discrimination motivation, to be less biased, they might also expect
(at least initially) that decision processes will be more consistent and objective. However, this is contingent
on second- and third-party beliefs that systems can lead to lower bias, and this attitude in the population at
large is, given the lack of familiarity with automated decision-making, likely unstable.
As a final facet of trustworthiness, reviewed papers also investigate benevolence. In general, there
are only few papers investigating this facet, but those find that people ascribe lower benevolence to systems
than to human decision-makers. Specifically, people believe that systems do not consider individual needs
and do not care about individuals (Höddinghaus et al., 2020; Yokoi et al., 2020). This might also relate to
what we discuss in the section on reductionism as well as lower perceptions of interpersonal justice.
Evaluations of trustworthiness finally result in effects on trust and reliance. Regarding trust,
automated decisions seem to predominantly result in less trust compared to human decisions and this seems
to apply to decision-making at work (Höddinghaus et al., 2020; M. K. Lee, 2018) and in medicine
(Palmisciano et al., 2020; York et al., 2020). However, this lack of trust might be contingent on a variety of
moderator such as characteristics of the task that is to be automated (e.g., personnel selection vs. scheduling;
Lee, 2018; image analysis vs. medical treatment recommendations; Palmisciano et al., 2020).
In sum, existing research regarding decision automation and augmentation and issues of
trustworthiness as well as trust comes mostly from the field of medical decision-making. Although questions
emerge in decision-making at work (Höddinghaus et al., 2020; Nolan et al., 2016), it has not yet received
the same empirical attention. Given unclear and sometimes contradictory results in the field of medical
decision-making, as well as given trustworthiness evaluations as potential antecedents regarding perceptions
of automated and augmented decisions, this is as an important line of future research in the work context.
3.1.2 Reductionism
Our review suggests that second and third parties believe that decision automation suffers from
reductionism (Newman et al., 2020). Specifically, people seem to believe that systems consider more factors
overall but neglect unique conditions, qualitative information, and decontextualize as well as quantify
decision processes (Longoni et al., 2019; Newman et al., 2020). This might also be a cause of several
negative perceptions by second and third parties as Newman et al. (2020) found negative fairness
perceptions to automated decisions due to the fact that second parties believe that systems do or cannot
adequately use qualitative information (e.g., leadership skills). Similarly, Longoni et al. (2019) found that
people are reluctant to use automated systems for medical diagnoses as they think that those systems neglect
their unique individual conditions (but see Yokoi et al., 2020 who did not find significant differences
regarding uniqueness neglect). Similarly, Hamilton et al. (2019) report that their participants believed that
only human physicians have a holistic view of individual patients which makes systems inadequate for
personalized care. In sum, those findings speak towards the attitude that automated decisions are reductionist
and dehumanizing. Importantly, this is also a reason for resistance to use automated systems from the
perspective of first parties (Dawes et al., 1989; Grove & Meehl, 1996). First, second, and third parties all
appear to assume that automated decisions suffer from reductionism.
3.1.3 Reluctance regarding Automation of Moral Decisions
Although empirical research is limited, both second and third parties do not appear to want decision
automation for decisions with obvious moral components. Specifically, over a various set of studies, Bigman
and Gray (2018) found that people do not want systems to make moral decisions in medicine (e.g., decisions
about treatments) and attributed this to reduced agency and reduced ability to feel and sense.
3.2 Perceptions
We identified five major types of perceptions: fairness and justice, organizational, accountability
and responsibility, supervision vs. autonomy, and privacy invasion.
3.2.1 Fairness and Justice
In line with growing interest regarding algorithmic fairness, accountability, and transparency (Shin
& Park, 2019), fairness and justice perceptions reflect the most commonly investigated outcomes in the
reviewed papers. For perceptions of fairness, observed effects were mixed but predominantly negative.
Some of the studies in personnel selection scenarios based on job interviews found no differences in fairness
perceptions between human and automated decisions (Langer, König, & Hemsing, 2020; Suen et al., 2019).
In a school admission context, Marcinkowski et al. (2020) found stronger fairness perceptions for automated
compared to human decisions. Similarly, Howard et al. (2020) reported stronger fairness perceptions after
the implementation of an automated scheduling system for residents at a hospital compared to scheduling
by human decision-makers. However, most of the reviewed studies found that people perceive automated
decisions as less fair than human decisions across personnel selection, performance evaluation, and
scheduling scenarios (Dineen et al., 2004; Langer, König, & Papathanasiou, 2019; M. K. Lee, 2018;
Newman et al., 2020; Uhde et al., 2020; Wang et al., 2020). Additional qualitative findings indicate that
fairness was a concern in algorithmic management contexts (M. K. Lee et al., 2015; Myhill et al., 2021)
where different design choices resulted in more, or less perceived fairness (Griesbach et al., 2019). Overall,
the mixed findings regarding fairness suggest that there are moderators that have not yet been sufficiently
investigated, a concept we revisit in greater detail later in this review.
Regarding justice, three of the four dimensions of organizational justice theory (Colquitt, 2001)
were commonly studied: interpersonal, procedural, and informational justice. Overall, people seem to expect
both positive and negative effects regarding the single justice facets. This seems to be closely related to
what we have discussed regarding attitudes towards decision automation (i.e., reductionism, evaluations of
trustworthiness) which supports previously found close relation between trustworthiness evaluations and
perceptions of justice (Colquitt & Rodell, 2011).
First, interpersonal justice, which refers to the perception of being treated with dignity and human
warmth in decision processes (Colquitt, 2001), was generally harmed by automation. Studies on automated
job interviews found lower social presence, two-way communication, and less adequate interpersonal
treatment in automated interviews (Acikgoz et al., 2020; Langer, König, & Papathanasiou, 2019; Langer,
König, Sanchez, et al., 2019). Similar findings come from the area of medical decision-making where
patients fear a lack of human touch, less ability to ask questions, and that systems might not provide
potentially negative outcomes (e.g., diagnoses) in a sensitive manner (Haan et al., 2019; Nelson et al., 2020).
Second, procedural justice perceptions were affected positively and negatively. This mixed picture
makes sense since procedural justice refers to different facets where some are more likely to positively
affected by decision automation and augmentation, whereas others are more likely impaired (Nagtegaal,
2021). On the positive side, automated decisions improve perceptions of processes as being based on
accurate information, being performed consistent, as well as free of bias (Colquitt, 2001), for instance in the
context of job interview performance evaluations (Acikgoz et al., 2020; Langer, König, Sanchez, et al.,
2019; Marcinkowski et al., 2020). Even decision augmentation can lead to more perceived consistency in
decision-making (Jutzi et al., 2020; Nelson et al., 2020; Nolan et al., 2016). Moreover, in algorithmic
management contexts, automated evaluation was perceived as an efficient and objective way to evaluate
workers’ performance (Galière, 2020).
On the negative side of procedural justice perceptions, automated decisions seem to impair
perceptions of whether it is possible to express one’s views and feelings about a process, appeal processes
(e.g., appeal negative customer ratings; Griesbach et al., 2019; Möhlmann et al., in press; Myhill et al.,
2021), and more generally control decision processes and outcomes. Specifically, reduced perceived control
of automated decisions was found for people in personnel selection (Langer, König, & Papathanasiou, 2019;
M. K. Lee, 2018), where they seem to be unsure how to affect automated decision processes in a way that
could improve their performance ratings (Acikgoz et al., 2020; Langer, König, Sanchez, et al., 2019). In
algorithmic management processes, perceived lack of controllability was often associated with a lack of
perceived transparency of what contributes to decision outcomes by automated systems (Möhlmann et al.,
in press; Myhill et al., 2021; Veen et al., 2020). In these cases, the lack of transparency was usually described
as an intentional design choice by platform providers in order to better control the workforce and to prevent
people from gaming the system (Galière, 2020; Möhlmann et al., in press). Future research could thus
investigate methods to increase perceived controllability without enabling workers to game respective
systems.
Third, results regarding informational justice, which refers to perceived openness, honesty, and
transparency in decision processes (Colquitt, 2001), were inconclusive. As we have just mentioned,
qualitative results in algorithmic management contexts indicate that people might perceive low
informational justice as they do not understand how algorithmic management decisions work (Griesbach et
al., 2019; M. K. Lee et al., 2015; Möhlmann et al., in press; Myhill et al., 2021; Veen et al., 2020). In
personnel selection, there is tentative evidence that people perceive automated decisions as less open
towards the applicants than human decisions, although there were no differences in perceived information
known about the decision processes (Acikgoz et al., 2020). Whereas those findings indicate that
informational justice might be lower in the case of automated decisions, there is also hope that automated
processes could enhance informational justice (Höddinghaus et al., 2020). Specifically, transparency of
decisions could increase if automated system were designed to be transparent (M. K. Lee et al., 2015) or at
least to provide better explanations of their recommendations than humans do (Jutzi et al., 2020).
3.2.2 Organizational Attractiveness
Organizational attractiveness refers to perceptions of how second and third parties perceive
organizations that sponsor algorithmic decisions. In the case of personnel selection, organizational
attractiveness seemed to be partly driven by fairness and justice perceptions of the algorithmic decision
(Ötting & Maier, 2018) and were predominantly negative. Similarly, Acigkoz et al. (2020) found lower
organizational attractiveness, job pursuit intentions, and stronger litigation intentions when using decision
automation in interviews with mediations principally through decreased chance to perform and decreased
two-way communication quality. Similarly, automated interviews have been associated with reduced
organizational attractiveness compared to videoconference interviews, and social presence and fairness
seem to be the most important mediators in this effect (Langer, König, & Papathanasiou, 2019; Langer,
König, Sanchez, et al., 2019). In contrast to the negative findings from selection contexts in organizations,
in university admissions, the use of systems was associated with better organizational reputation
(Marcinkowski et al., 2020).
This suggests unexplored moderators affecting people’s perceptions of organizational
attractiveness. For instance, we could imagine that interpersonal justice facets are especially important in
the case of job interviews (Langer, König, & Papathanasiou, 2019) such that in this context, automated
decisions impair organizational attractiveness. In contrast, there might be situations where consistency and
objectivity are more important and where the perceived benefits of automated decisions could result in
higher attractiveness (e.g., in resume screening as a situation where there is usually no human contact;
Marcinkowski et al., 2020).
3.2.3 Accountability and Responsibility
Accountability and responsibility are commonly discussed for the use of automated systems for
high-stakes decisions across disciplines (Floridi et al., 2018; Kellogg et al., 2020; Mittelstadt et al., 2016).
A common concern is that there will be an accountability gap when humans rely on automated systems as
there might be situations where it is not clear who is accountable for errors (Raji et al., 2020). Whereas there
is only scarce research in work-related contexts, the question of who is accountable for failures of automated
systems in medicine was raised by participants in the reviewed studies, although developers and first-parties
were commonly believed to be ultimately accountable (Haan et al., 2019; Nelson et al., 2020).
Whereas those results speak to perceptions of actual legal accountability, other studies found that
people attributed different responsibility to decision-makers who use automated systems for decision-
making. For instance, systems can deflect part of the blame for negative medical outcomes but only if
physicians follow system advice (Pezzo & Pezzo, 2006; Tobia et al., 2021). Tobia et al. (2021) conclude
that this may constitute an incentive to follow recommendations by automated systems as this could shield
against legal accusations. On the side of second parties, being affected by a decisions that was shared
between human and system can result in being unsure who to blame for negative outcomes of decision
automation and augmentation (Jutzi et al., 2020; Nolan et al., 2016). Specifically, some second parties
perceived less decrease in their own perceived responsibility when systems provided them with
recommendations about medical treatment in comparison to when humans gave this recommendation
(Longoni et al., 2019; Promberger & Baron, 2006). Authors of the respective papers argued that this might
be attributable to diffused responsibility; the second-party may not know who to blame in the event of an
unfavorable outcome resulting from following and automated recommendation. Future work could
investigate whether those effects are moderated by characteristics of the system (e.g., system accuracy;
Lowe et al. 2002) as previous research was unspecific about conditions that affect perceived responsibility.
3.2.4 Being under System Control vs. Autonomy

The tension between being controlled by an automated system versus perceived autonomy at work
it especially prevalent in algorithmic management contexts. Specifically, workers are under constant
surveillance and controlled by algorithmic management but at the same time perceive flexibility and
autonomy in their work (Galière, 2020; Griesbach et al., 2019; Möhlmann et al., in press). These perceptions
seem to be partly due to the fact that workers do not have a human boss telling them what to do (Anwar et
al., 2021; Griesbach et al., 2019; Möhlmann et al., in press; Möhlmann & Zalmanson, 2017; Wood et al.,
2019). Instead, there is an automated system providing instructions, information, and evaluation. Although
this leads to workers being aware of being under constant evaluation (Bucher et al., 2021; Kinder et al.,
2019; Ravenelle, 2019; Veen et al., 2020), it seems that being under system control can be perceived as
providing more autonomy than comparable supervision by a human boss (Anwar et al., 2021; Griesbach et
al., 2019; Möhlmann et al., in press; Möhlmann & Zalmanson, 2017; Wood et al., 2019). We need to
emphasize that parts of these perceptions could also be affected by the overall work environment and by
platform providers such as Uber or Upwork selling jobs on their platforms as being “entrepreneurial”. This
might contribute to workers’ self-identity as being autonomous and having flexible working hours when
they are actually following strict rules and algorithmic control (Galière, 2020; Jarrahi et al., 2020).
3.2.5 Privacy Invasion
Automated decision-making can lead to perceptions of privacy invasion under certain
circumstances, but this conclusion is based on few studies. Privacy concerns are commonly debated when
using automated decisions, as the technologies enabling decision automation and augmentation often rely
on automatic evaluation of sensitive data provided by second parties. In the context of personnel selection,
privacy concerns may be higher for decision automation (Gonzalez et al., 2019; Langer, König, &
Papathanasiou, 2019; Langer, König, Sanchez, et al., 2019). However, the reviewed literature provided no
empirical evidence as to the reasons why second parties would be more concerned providing private data to
an automated system than to a human decision-maker evaluating the same data. Additionally, in algorithmic
management, there is significant discussion of worker concern with constant surveillance, but privacy
concerns seem to be of lesser importance in this area (Galière, 2020; Griesbach et al., 2019).
3.3 Behavior
We identified four commonly studied classes of behavior in response to automated decision-
making: reverse engineering of automated decisions, creating workarounds, collective action, and
embracing the system. Research that investigates behavioral reactions to decision automation and
augmentation predominantly comes from the area of algorithmic management in gig and platform work,
which focuses on behaviors related to perceived controllability of automated decisions (see Kellogg et al.,
2020; or Möhlmann et al., in press, for overviews on control in relation to algorithmic management). Second
parties try to enhance their understanding of automated management processes through reverse engineering
and knowledge sharing in online forums, and building a personal or shared understanding this way may
increase perceived controllability (Jarrahi et al., 2020; M. K. Lee et al., 2015; Möhlmann et al., in press;
Myhill et al., 2021). Furthermore, based on their understanding of automated processes, second parties may
use peculiarities of automated systems to create workarounds that either avoid penalties or increase their
income (Kinder et al., 2019; Myhill et al., 2021; Veen et al., 2020; Wood et al., 2019), and success with
such methods may also increase perceived controllability. Other attempts to increase perceived
controllability may involve initiating collective action in regard to automated processes that affect millions
of other second parties (M. K. Lee et al., 2015). However, in most studied situations, there was a sufficient
power imbalance so as to force second parties to embrace automated decisions or give up their incomes.
This lack of control sometimes even led workers to describe automated decisions as the fairest possible way
to work assignment and evaluation (Galière, 2020).
3.3.1 Reverse Engineering of Automated Decisions
Second parties often do not understand how algorithmic management systems work and do not
know how they can affect system outcomes favorably (M. K. Lee et al., 2015; Myhill et al., 2021; Veen et
al., 2020). To increase understanding, there are examples of second parties trying to reverse engineer
automated decision-making process through experimentation (Kinder et al., 2019; M. K. Lee et al., 2015;
Möhlmann et al., in press). Workers may change inputs into the system in order to see how outputs change
and build a mental model of system functioning (Jarrahi & Sutherland, 2019). Similarly, workers have
logged in as alternative user types in algorithmically managed systems to see how their changes influence
what potential customers see (Jarrahi & Sutherland, 2019). Reverse engineering seems to serve the purpose
of increasing understanding of the automated management system to increase controllability of the system.
3.3.2 Creating Workarounds
Second parties may try to use bugs and (possibly undocumented) features of algorithmic
management systems to create workarounds. For instance, a person may try to bring the interaction with
their clients away from the influence of the automated system, for instance, by directly interacting with
customers (Jarrahi & Sutherland, 2019; M. K. Lee et al., 2015). Additionally, people try to use technical
findings to their own benefit. For instance, users may log out of a system that is tracking them to receive
better evaluations (e.g., because systems cannot track reckless driving behavior or the actual time a worker
spend on a task; Jarrahi & Sutherland, 2019; M. K. Lee et al., 2015; Möhlmann & Zalmanson, 2017; Wood
et al., 2019). In another example, Upwork workers who inputted five one-hour blocks of work discovered
they would have greater system benefits than if they inputted those same hours as a single five-hour block
(Jarrahi & Sutherland, 2019). These ways of how to game the system are under constant change as platform
providers try to fix respective loopholes which makes it necessary for workers to search for new ways to
create workarounds (Galière, 2020; Möhlmann et al., in press).
3.3.3 Collective Action
Numerous studies provide evidence that second parties are inclined to share knowledge and
evidence with other second parties and to keep up to date about potential changes in functions of the systems
they work under (Jarrahi & Sutherland, 2019; M. K. Lee et al., 2015; Möhlmann & Zalmanson, 2017; Myhill
et al., 2021). This is often done via online forums, which may also provide a feeling of social embeddedness.
Second parties use such forums to onboard new workers, to complain about the system, to form alliances in
protest, and to raise their collective voice against the usually inaccessible layer of developers behind the
system (M. K. Lee et al., 2015). In such forums, second parties also sometimes organize resistance against
the automated management system or to try to collectively game the system (Tassinari & Maccarrone,
2020). For instance, Möhlmann and Zalmanson (2017) describes a case where Uber drivers tried to
collectively reduce work effort in order to increase demand which would in turn increase surge pricing to
the benefit of worker pay. Even more drastic manifestations of collective actions are worker strikes
(Möhlmann et al., in press; Tassinari & Maccarrone, 2020).
3.3.4 Embracing the System
Several studies report that workers in algorithmic management embrace algorithmic management
systems. They try to keep their evaluations within the system high (Bucher et al., 2021; Kinder et al., 2019;
Ravenelle, 2019; Wood et al., 2019). Furthermore, they perceive systems as being efficient and objective
and thus recommend other workers to play along with the rules of the system as this is the fairest possible
evaluation of workers’ performance (Galière, 2020).
4. Moderators of Effects on Second and Third Parties
Table 3 provides an overview of characteristics of the decision-making process, of the system, of
the second- and third parties themselves, of the tasks, and of the outcomes. In each case, we either report
research directly testing these moderators or observed differences between studies adopting different
approaches.
Table 3
Factors affecting the effects of decision automation and augmentation on second- and third parties.
Category Current consensus Sample research question Selected sources
Characteristics of the decision-making
process
System vs. human control People call for human control in high-stake decisions and seem to Why do people prefer human control in certain Nagtegaal, 2021;
perceive augmented as more favorable than automated decisions. tasks? Newman et al., 2020
However, this evidence stems mainly from medical decision-making
and may depend on the focal task.
First-party behavior in decision process Rejecting vs. following advice by automated systems seems to affect What are psychological processes underlying Tobia et al., 2021;
second-party perceptions of the first party. different perceptions of first parties who reject Pezzo & Pezzo, 2006
or follow advice?
System characteristics
System performance All else being equal, humans prefer human control but when system How do different system performance Bigman & Gray,
accuracy becomes better, humans start to prefer system control. measures (e.g., accuracy, bias) affect 2018; Longoni et al.,
perceptions of systems? 2019
Understandability and transparency through Inconclusive findings. What are trade-offs of providing explanations Binns et al., 2018;
information and explanation and transparency to second parties? Langer et al., 2018;
Newman et al., 2020
Information about the developer Information about the developer can affect reactions to the system. Do information about the development process Bigman et al., 2020;
(e.g., about training data) also affect reactions? Wang et al., 2020
Characteristics of second and third parties
Experience, familiarity, education Mixed effects with some studies finding no effects other finding Does experience, familiarity, and education Langer et al., 2018;
better perceptions of automated decisions with more experience, affect perceived control and understanding? Wang et al., 2020
familiarity, education.
Personality and traits Inconclusive effects of studies investigating a range of potentially What are influential personality facets and Longoni et al., 2019
influential traits (e.g., perceived uniqueness, privacy concerns) traits and why are they influential?
Gender There is a tendency that females react less favorable to automated What are the underlying influences that may Araujo et al., 2020;
decisions. contribute to gender differences? York et al., 2020
Task characteristics The respective task affects reactions, but it is unclear what exactly it What are the dimensions of tasks that affect Lee, 2018;
is about tasks that affect second- and third-party reactions. Research second and third parties with respect to Nagtegaal, 2021
indicates that it is a mix of stakes of the task, tasks that require human automated and augmented decisions?
versus mechanical skills, perceived quantifiability, and familiarity
with automated systems for the respective tasks.
Output and outcome characteristics Different kinds of outputs (e.g., standard vs. non-standard) affect What are further characteristics of system Wang et al., 2020;
perceptions. Unfavorable outcomes lead to negative reactions outputs (e.g., output comprehensibility) that Hong et al., 2020
irrespective of human or automated decisions. However, overall affect perceptions?
research on effects of outputs and outcomes is scarce.
4.1 Characteristics of the Decision-Making Process
We identified two process characteristics as potentially influential moderators: system versus
human control and first-party behavior in decision process. Whereas there is only scarce research on work-
related decisions regarding these topics, results from medical decision-making support them (Jutzi et al.,
2020; Nelson et al., 2020). For future research, it will be central to investigate psychological processes
explaining the effects reported in this section. For example, why do people prefer human control in certain
application contexts? An explanation based on findings from the perspective of first parties (Grove & Meehl,
1996) might be that humans fear that machines decide the fates of human beings. Other interpretations in
line with findings regarding attitudes and perceptions might be that human control will improve single case
decision-making, will be less dehumanizing, and will allow the consideration of qualitative factors (i.e.,
there might be hope that humans can mitigate reductionism). Additionally, people might hope that human
decision-makers can be influenced to one’s own benefit (i.e., are more “controllable”). Furthermore, with a
human decision-maker available, second and third parties may perceive someone who is a contact person to
whom they can direct complaints and who is accountable for negative outcomes.
4.1.1 System vs. Human Control
Earlier in this paper, we anticipated the distinction between augmented and automated decision-
making to be of importance. Indeed, people call for human control in high-stake decisions and seem to
predominantly perceive augmented as favorable to automated decisions. For instance, second parties
perceive decisions where first parties only have the option to consult an automated system as fairer
compared to full automation or to decisions where first parties can only slightly change automated decisions
(Newman et al. 2020). Furthermore, Suen et al.’s (2019) finding of no negative reactions to algorithmic
decision-making in personnel selection might be because participants were informed that the algorithmic
evaluation only served as a reference for the human decision-maker. Another perspective on the effects of
human control comes from Nagtegaal (2021). She found in hiring or performance evaluation that only
automating decisions negatively affected perceptions; augmentation had no negative effects. In contrast,
less human control led to stronger fairness perceptions in mathematical tasks (e.g., travel reimbursement
decisions). This suggests that there are also tasks where more system control can be beneficial.
Additional evidence on system versus human control is generally lacking for decisions at work;
however, research in medical decision-making also suggests that humans prefer having a human involved
in decision-making for certain tasks. Second parties often specify that systems should be more of an
informant or second opinion than the final trigger of a decision (Haan et al., 2019; Hamilton et al., 2019;
Jutzi et al., 2020; Palmisciano et al., 2020; Stai et al., 2020) and call for human-system symbiosis (Nelson
et al., 2020). Supporting those findings, Bigman and Gray (Study 8; 2018) found that if systems only advice
human physicians, there is a higher likelihood that people accept decisions. However, many people still
preferred the human expert to decide without automated support (see also York et al., 2020) (for further
support of the positive influence of human control see Jonmarker et al., 2019, Longoni et al., 2019, Palmeira
& Spassova, 2015).
4.1.2 First-Party Behavior in Decision Processes
Following versus rejecting advice by automated systems seems to affect second-party attitudes and
perceptions with regard to the first party. For instance, Arkes et al. (2007) found that rejecting automated
recommendations can lead to lower trustworthiness assessment of the decision-maker. Furthermore, when
augmented decision result in an unfavorable outcome for second parties, people seem to ascribe more fault
to physicians when they reject automated advice compared to when they follow the advice (see also Tobia
et al., 2021). This finding might be grounded in attributions of responsibility that might be lower for when
humans follow automated systems compared to when they ignore the advice (Lowe et al., 2002).
4.2 System Characteristics
System characteristics that affect reactions to automated and augmented decision processes are
system performance, understandability and transparency through information and explanation, as well as
information about the developer. Overall, only a small number of specific system-design features have
received attention. In the context of algorithmic management, research shows possible implications of
different design choices when comparing different algorithmic management platforms (Griesbach et al.,
2019; Ravenelle, 2019). For instance, design choices may affect perceived fairness, perceived autonomy,
job satisfaction, and overall work performance (Galière, 2020). However, as respective papers compare
different platforms, without systematically investigating design options (see Ravenelle, 2019), it is hard to
tell which design differences between the platforms led to different worker reactions. Consequently,
systematically assessing effects of design options for automated systems in decision-making and their
effects on second and third parties could receive more attention (Landers & Marin, 2021).
4.2.1 System Performance
System performance affects second-party reactions to automated decisions. Whereas, accuracy
(e.g., prediction accuracy) is the predominant measure of system performance in the reviewed studies, there
are others (e.g., efficiency, lack of bias) that could also be considered to be performance measures and that
may become increasingly important in new application areas of decision automation and augmentation (e.g.,
in management, medicine) (Srivastava et al., 2019).
All else being equal, humans prefer human control in decisions but when system accuracy becomes
better, humans start to prefer system control (Longoni et al., 2019). For instance, patients would prefer AI-
based medical systems when they are more accurate than a human expert (Haan et al., 2019; Jutzi et al.,
2020). However, if systems are similarly accurate like humans, people indicated that they would prefer
human over automated decisions. For instance, Bigman and Gray (2018, Study 9) showed that it is necessary
to make accuracy advantages of automated systems salient (i.e., in a within-subject comparison to the human
expert) and strongly different (i.e., 75% human vs. 95% system) in order to find preference for system
decisions. Potentially, this could indicate that expectations regarding system performance are unrealistically
high (Merritt et al., 2015). The study by Bigman and Gray even showed that in the case of a clear advantage
of the automated system, 28% of participants indicated that they still would prefer the inferior human
decision. In a similar vein, Longoni et al. (2019) showed that people are less sensitive to accuracy when
deciding between human and automated decisions versus when deciding between two human decision-
makers. Specifically, they found that accuracy differences lead to stronger effects when deciding between
two humans (i.e., participants more strongly favored the human showing higher accuracy) compared to
when deciding between a human and an automated system where the system shows higher accuracy.
4.2.2 Understandability and Transparency through Information and Explanation
Whereas research, practice, regulatory bodies, and ethical guidelines commonly call for
understandability and transparency of systems and their outputs, and assume that this positively affects
outcomes for second parties (e.g., Dineen et al., 2004; Jobin et al., 2019), the reviewed findings in relation
to understandability and transparency are inconclusive. For example, Wang et al. (2020) informed their
participants that there are “information online available” about an automated decision process which led to
participants perceiving biased procedures as even less fair compared to where no such information was
available. In a more direct investigation of the effects of explanations, Binns et al.’s (2018) between-subject
study showed only weak effects of different kind of explanations on justice perceptions. In their within-
subject study, explicitly comparing different kinds of explanations resulted in more negative justice
perceptions for case-based explanations compared to other explanations. On the one hand, this indicates that
without direct comparison between different versions of explanations, their effect on justice perceptions
might be negligible. On the other hand, it is important to test the effects of different kinds of explanations
as some might lead to comparably negative outcomes. Furthermore, Langer et al. (2018) found that
providing information regarding what input variables an automated interview system uses increases
perceived transparency but at the same time decreases perceived organizational attractiveness. Similarly,
Newman et al. (2020) findings indicate that whereas for human decisions high transparency leads to more
fairness, for automated systems this effect is reversed.
A possible explanation for the overall inconclusive effects is that more understandability without
also perceiving more controllability might be negative (Ananny & Crawford, 2018). For instance, research
shows that only providing information about variables the system considers as inputs might be detrimental
if people see that they might not be able to control those inputs (e.g., their voice tone) (Grgić-Hlača et al.,
2018; Langer et al., 2018; Newman et al., 2020). Another possible interpretation is that people might have
different expectations regarding transparency of systems compared to humans as decision-makers. For

instance, people may not even expect explanations from systems whereas explanations to increase
transparency of a decision are something natural in the interaction with human decision-makers (Lombrozo,
2011; Zerilli et al., 2018) – a lack of explanations could be interpreted as intentional distortion in the case
of a human decision-maker (Schlicker et al., 2021). Another possibility might be that people believe they
understand human decision-making processes although those are eventually also black box decisions (Zerilli
et al., 2018). We see potential regarding research on understandability of systems which is also reflected in
the current boom in research on explainable artificial intelligence (XAI) (Miller, 2019). Future research
needs to investigate how to optimally induce understanding through the provision of explanation and
information. In this regard, we need to understand under what conditions, in what contexts, and with what
trade-offs (e.g., gaming respective systems; Jarrahi & Sutherland, 2019; M. K. Lee et al., 2015; Möhlmann
et al., in press) increasing understanding of systems happens (Langer et al., 2021).
4.2.3 Information about the Developer
Information about the system developer might affect second and third parties. For instance, Wang
et al. (2020) showed stronger negative effects of biased outcomes when a system was developed by an
“outsourced team” compared to an in-house developer team. Similarly, Bigman et al. (2020) found third
parties perceive more moral outrage regarding biases by automated systems when the developing
organization was describes as sexist. Finally, information about the prestige of the developer can lead to
better reactions when experts use respective systems to augment their decisions (Arkes et al., 2007).
4.3 Characteristics of Second and Third Parties
We identified experience, familiarity, education, personality and traits as well as gender as
characteristics of second and third parties that moderate effects of automation and augmentation.
4.3.1 Experience, Familiarity, Education
Overall, experience, familiarity, and education are related to understandability or perceived control
as people with more experience and familiarity with computers, programming, and higher education might
believe that they understand how systems work which might give them more confidence that the can control
system processes and outputs (Jonmarker et al., 2019; Langer et al., 2018). Intuitively, this could lead to
better perceptions of automated decisions. However, there are mixed effects as some studies found no effects
whereas others found better perceptions of automated decisions with more experience, familiarity, and
education. For instance, Langer et al. (2018) found no correlation of computer experience and any of their
perceived justice variables. Wolf (2014) showed that even IT students derogate physicians who use
automated systems. In contrast, Wang et al. (2020) showed that computer literacy positively correlates with
fairness perceptions regarding automated decisions. Similarly, Jonmarker et al. (2019) found that better
understanding of new technology is associated with better reactions to decision automation in breast cancer
screening. Furthermore, Gonzalez et al. (2019) showed that familiarity with AI can mitigate negative
reactions to automated decisions. Additionally, Araujo et al. (2020) showed that programming knowledge
is associated with more positive evaluations of usefulness and fairness of automated decisions. Additionally,
Bigman et al. (2020) showed that people with stronger AI knowledge were less morally outraged by
algorithmic bias which could be interpreted as either more positive evaluations of automated decisions or
as a positivistic bias in relation to automated systems as they assume that bias by systems is not a fault of
the system but maybe just reflects actual differences between people. Furthermore, Wang et al. (2020) found
that less educated participants react more strongly to unfavorable outcomes by automated decision
processes. Relatedly, higher education levels seem to be associated with better reactions to automated
decisions in medicine (Jonmarker et al., 2019; York et al., 2020).
4.3.2 Personality and Traits
The reviewed papers investigated a range of traits potentially influential (e.g., perceived uniqueness,
trait privacy concerns, locus of control) with inconclusive effects. For instance, Araujo et al. (2020) showed
that trait privacy concerns were negatively associated with fairness and positively with perceived risk of
using automated decisions. V. A. Shaffer et al. (2013) showed that people with a high internal locus of
control react more negatively to physicians using computerized aid. Furthermore, Longoni et al. (2019)
found that the more unique a person feels the less they want to be assessed by automated systems.
4.3.3 Gender
There is inconclusive evidence but a tendency that females react less favorable to automated
decisions. Whereas Hong et al. (2020) found no differences between male and female participants with
respect to automated decisions involving unfair gender discrimination, Dineen et al. (2004) and York et al.
(2020) showed stronger preferences for human versus automated decisions for female participants. Wang et
al. (2020) showed that females react more strongly towards unfavorable outcomes in automated decisions.
Finally, Araujo et al. (2020) report that females tend to perceive automated decisions as less useful. Overall,
it is likely that unconsidered confounding variables (e.g., familiarity with respective systems) might have
contributed to the reported effects of gender on reactions to automated decisions.
4.4 Task Characteristics
Previous research indicates that it is a mix or the perceived stakes of the task, perceptions of tasks
requiring human versus mechanical skills, perceived quantifiability of task-related information, and
familiarity with automated systems for the respective tasks that affect second- and third-party perceptions.
First, people might react differently to systems used in high versus low-stakes contexts (Langer, König, &
Papathanasiou, 2019). Specifically, third parties reacted more negatively to automated decisions used for
personnel selection (high-stakes) compared to training situations (low-stakes) (Langer, König, &
Papathanasiou 2019). Similarly, Longoni et al. (2019; Study 102, supplementary material) showed more
resistance to automated systems in high-stakes medical decision (see also Palmisciano et al., 2020; York et
al., 2020). In contrast, Araujo et al. (2020) found that automated decisions are perceived as fairer in high-
stakes situations in health and justice. Additionally, Ötting and Maier (2018) found no differences in reaction
to task allocation (low-stakes) and allocation of vocational training (high-stakes) between human and
automated decisions. A different perspective on the influence of the stakes comes from Srivastava et al.
(2019) who showed that in high-stakes contexts, people value accuracy more than unbiasedness of system
decisions.
Second, perceptions might differ in tasks that require human versus mechanical skills (Lee, 2018).
For instance, human skills would be subjective and intuitive judgement as well as emotional capabilities
2
We thank the authors for providing us with additional unpublished results from their study.
whereas mechanical skills would be quick and potentially objective processing of large amounts of data
(Castelo et al., 2019; Lee, 2018). For example, systems seem to be more accepted for image processing
tasks in medicine compared to treatment recommendations (Palmisciano et al., 2020; York et al., 2020).
Furthermore, Lee (2018) found that humans perceive less fairness, less trust, and more negative emotional
reactions when an automated system conducted a task that required human skills (i.e., personnel selection,
work evaluation) compared to tasks that required mechanical skills (i.e., work assignment, scheduling).
Unfortunately, in all above examples (imaging vs. treatment recommendation; selection vs. scheduling) the
respective tasks might not only differ with respect to requiring human skills, but could also be perceived to
differ with respect to the stakes involved.
Third, related to the idea of human vs. mechanical skills, quantifiability may influence how people
react to decision automation (Nagtegaal, 2021). Specifically, if it is difficult to measure task components
(e.g., predictors, criteria) with face valid numbers, people seem to believe that it is a bad idea trying to
automate it through systems using mathematical algorithms. This interpretation is in line with the reviewed
findings (M. K. Lee et al., 2015; Newman et al., 2020; Ötting & Maier, 2018). Especially, Nagtegaal’s
(2021) studies support this distinction; for highly quantifiable, mathematically deterministic tasks (e.g.,
determining travel reimbursement given provided documentation), she showed a decline in perceived
fairness as human control increased. In contrast, for less quantifiable (but also more complex; Nagtegaal,
2021) tasks (e.g., hiring), decision automation was perceived as least fair, and there was no difference in
perceived fairness between human and augmented decision-making, possibly because people did not believe
that adding the system would provide any benefit to such a task. Additional support for this task dimension
comes from research investigating first-party reactions. For first parties, previous research found that they
reject automated decisions in uncertain domains (Dietvorst & Bharti, 2020) but prefer automated decisions
for quantifiable tasks (Castelo et al., 2019).
The complexities involved with task characteristics dimensions are illustrated by Longoni et al.
(2019; Study 11, supplementary material), who found that participants preferred human decisions
throughout all investigated application areas but that the strength of this preference differed across the areas
in the following rank order from the greatest difference between human and automated decisions to the
smallest: legal, health, home, fashion, home décor, restaurants, movies, music, financial decisions. This rank
order reveals that high versus low-stakes is a potentially central distinction that affects human reactions to
automated decisions, yet it is clearly not the only meaningful distinction, considering financial decisions
would usually also be considered high-stakes. Instead, financial decisions may represent a quantifiable task
requiring mechanical skills, one done better by a system than by a human. Familiarity with automated
decisions might be another dimension which could explain the rather favorable reactions to automated
decisions for movies and music. Overall, we conclude that a necessary theoretical advance to move forward
is the development of a taxonomy of task differences relevant to algorithmic decision-making. Without such
a taxonomy, distinctions like these will continue to plague unambiguous conclusions in such experiments.
4.5 System Output and Decision Outcome Characteristics
System outputs and decision outcomes serve as stimuli for second-party perceptions. We distinguish
here between the outputs of a system (i.e., the actual recommendation, decision) and outcomes for
individuals (e.g, favorable vs. unfavorable). Although research on the recursive effects of outputs and
outcomes on attitudes, perceptions, and behaviors remains scarce, future research should investigate such
effects given previous research on first-party perceptions showing that outputs and outcomes affect first-
party attitudes, perceptions, and behavior in relation to systems (e.g., evaluations of trustworthiness, trust,
acceptance, use) (Hoff & Bashir, 2015; J. D. Lee & See, 2004). It is likely that these effects persist in the
second- and third-party context.
Hong et al. (2020) showed that biased outputs by systems can have comparably stronger negative
effects on people’s perceptions of decision automation compared to descriptions referring to system quality.
Additionally, Tobia et al. (2021) showed that following versus rejecting standard versus non-standard
treatment recommendations (i.e., outputs) by automated systems differently affected physicians’ perceived
responsibility. On the side of outcomes for second parties, unfavorable and unjust outcomes lead to negative
reactions irrespective of human or automated decisions (Gonzalez et al., 2019; Hong et al., 2020). For
example, Gonzalez et al. (2019) found no difference in organizational attractiveness in the case of negative
outcomes (i.e., imagine a job application was rejected) but that participants preferred human decisions in
the case of positive outcomes. Even expected outcome favorability seems to have an effect (Wang et al.,
2020). Specifically, Wang et al. (2020) showed that participants who expect to fail an automated evaluation
also perceive the general process to be less fair than the ones who expect to pass.
5. Limitations and Guidelines for Future Research
Overall, our review revealed inconsistencies in findings with respect to several commonly
investigated attitudes and perceptions and highlighted important moderators (see Tables 2 and 3 for related
sample research questions). For instance, findings regarding trustworthiness attitudes (especially for the
facet ability) differed between the studies, perceptions of fairness seem to be moderated by the task at hand,
and there are ambiguous effects of transparency and explanation provision. Furthermore, few researchers
explicitly examine the relationships among attitudes, perceptions, and behavior, generally focusing upon
either perceptions, or attitudes, or behaviors. Moreover, although research indicates that human control in
decision processes alters second- and third-party attitudes and perceptions, the psychological processes
underlying this effect remain open for investigation. In addition to the inconsistencies found in the reviewed
studies that hopefully stimulate future research, the following subsections present five broad observations
regarding limitations of the reviewed studies together with guidelines to advance future research on decision
automation and augmentation.
5.1 Research Design that Allows for Generalizability
Overall, the generalizability of the reviewed studies is unclear. Specifically, most relied on
vignettes, generally asking research participants to imagine being in or observing an automated or
augmented decision process. Vignette studies can be difficult to design in order to draw meaningful
conclusions, and they often do not accurately mirror real-world processes (Atzmüller & Steiner, 2010).
Furthermore, for most studies it remained unclear how familiar people were regarding decision automation
or augmentation. For example, a vignette study may ask for participant reactions to “AI” but if participants
do not know what the term AI means, knowledge of AI, novelty effects, technological anxiety, and a variety
of other constructs are potentially confounded. Moving forward, researchers should prioritize research
designs that allow more generalizable conclusions, especially because specific algorithm design and
implementation details in real-world decision-making may have large effects on relevant outcomes for both
second and third parties (Galière, 2020).
This reframing potentially influences interpretations of the reviewed research in two major ways.
First, imagining how a second party might feel when a decision is made about them by a first party and/or
system may more directly reflect third-party reactions than second-party reactions. A consequence of this
might be that most vignette-based studies show a preference for human over automated medical decisions,
whereas Keel et al.’s (2018) study of authentic second-party reactions found the opposite. Second, systems
are generally developed iteratively, refined over time through hundreds or thousands of versions to
maximize positive outcomes and minimize negative outcomes over a long time period for narrowly defined
subpopulations (Landers & Marin, 2021). Thus, development research conducted by technology companies
and held privately often refines real-world systems for targeted outcome in ways that vignette studies of
such systems cannot replicate. In short, researchers may often be studying automated or augmented
decision-making as researchers imagine it, not as it is realized in authentic organizational contexts. In these
ways, a literature relying entirely upon vignettes risks being ungeneralizable to real-world decision-making.
The most informative designs in the literature that we identified were studies investigating
algorithmic management in gig or platform work and studies directly investigating the implementation of
automated systems for decision (Howard et al., 2020; Keel et al., 2018; Ravenelle, 2019). Yet, such insights
predominantly stem from qualitative interviews with second parties or from the analysis of online forums
where workers exchange their experiences. Future studies might try to use experiments and quantitative
analyses to broaden the insights. For instance, algorithmic management could affect employees in
organizations who have more interaction with their colleagues and supervisors compared to Uber drivers,
who almost always work alone (Wesche & Sonderegger, 2019). Additionally, if people know that automated
systems will evaluate them in application processes or in performance evaluations within organizations, this
might lead to similar reactions like the ones found in studies in our review (e.g., reverse engineer algorithmic
evaluation processes). Since the same automated system could be used in different organizations, it would
be interesting to investigate whether this could lead to collective action or attempts to share knowledge
about the involved systems across organizations. For instance, similar behavior is already common in
automated systems for personnel selection where there are online discussions where people share knowledge
and folk theories about to positively affect evaluation by automated interview systems.3
5.2 Multi-Stakeholder View
The stakes associated with accurately investigating real-world processes are high. Consider that
many problems associated with earlier automated systems at work (e.g., expert systems) might reoccur for
current AI-based systems (Gill, 1995). In fact, despite expert systems sparking significant attention in the
1980s and 1990s, they have not been widely adopted in workplaces. Instead, problems transitioning from
development to implementation, and more problematically the lack of system acceptance by users, hindered
the use of this earlier manifestation of workplace AI (Gill, 1995). Although current AI-based automated
systems may have significant advantages over expert systems in terms of flexibility and accuracy, it is
unclear if the fundamental psychological processes affected by the use of these systems and the design
concerns faced in their development to address such processes are substantially different. Additionally,
litigation against automated decision systems has become a concern that will likely affect attitudes towards
such systems (Harwell, 2019), national ethical and legal guidelines are evolving, and there have been several
high profile examples of public outrage about the use of automated decisions for work-related decisions
(Lecher, 2019). Furthermore, the new generation of AI-based automated systems comes with new technical
challenges, such as the detection and response to biased decision-making (Raghavan et al., 2020), the
opacity of decision-processes (Burrell, 2016), and increased demand for explainability (Floridi et al., 2018).
To understand the effects of automated and augmented decision-making in this context, it is
necessary to take a multi-stakeholder view, considering their perspectives both holistically and individually
when making research design decisions (Jungmann et al., 2020; Langer et al., 2021). As it stands currently,
there is a notable stream of research assessing first-party acceptance (at least in certain areas of application;
Burton et al., 2020; Hoff & Bashir, 2015), and our review summarizes research on second and third parties.
3
https://www.reddit.com/r/recruitinghell/comments/e5eyw5/duke_university_is_preparing_students_for/
In addition to the use of authentic decision-making to enhance our understanding of those parties, a more
complete treatment of stakeholders would include supervisors of people using automated decisions (e.g.,
how to best assess the work of direct reports when those use automated systems; Holstein et al., 2019), team
members (e.g., how does the implementation of an automated decision agent into a team affect team
collaboration; O’Neill et al., 2020), developers (e.g., how developers should be involved in redesign to
improve automated systems once they are implemented; Landers & Marin, in press), and regulatory bodies
(e.g., how policy maker actions influence these networks of relationships; Arrieta et al., 2020).
5.3 Inconsistency in Terminology
Substantial variation in terminology (see Table 1) between and even within studies sometimes
harms the coherence of conclusions. The reviewed studies used terms including AI/ML (Gonzalez et al.,
2019), automated system (Dineen et al., 2004), super computer (Bigman & Gray, 2018), and algorithm
(Newman et al., 2020) to refer to similar concepts. Without explicitly defining terms, construct proliferation
(J. A. Shaffer et al., 2016) becomes a substantial and troubling risk, risking unnecessary splits in the
literature and wasted researcher resources.
This problem especially applies to study designs themselves, when considering the specific prompts
provided to participants in vignette studies. Studies varied greatly in terms of the amount and clarity of
explanatory information concerning the system studied. For instance, Lee (2018) used the term “algorithm”
with participants without providing any additional information, whereas Newman et al’s (2020) participants
read “algorithm (i.e., a computerized decision-making tool)”. Similarly, if participants are more familiar
with the word “computer program,” this might result in different reactions compared to when describing the
same system as an “algorithm” due to their personal familiarity rather than the general idea of automated or
augmented decision-making. Supporting this, Gray and Wegner (2012) show that uneasy feelings about
systems result from ascribing the ability to sense and feel (i.e., experience) to machines but not from
perceived agency of systems. Ascribing sensing and feeling abilities might be more likely for some terms
(e.g., “robot”) compared to others (e.g., “computer program”). As such, we believe that investigating the
consequences of terminological differences when referring to a system that enables decision automation or
augmentation is important for future research, as effects across studies may vary due to such differences.
5.4 Considering Design-Features of Systems and Levels of Automation
A large range of papers that refer to “the system” that augments or automates decisions consider
“the system” to be a monolithic concept. This ignores the large range of design possibilities for such systems
that might influence effects for second and third parties (Landers & Marin, 2021). For instance, automated
systems vary in their development process (e.g., what kind of training data are provided for the automated
system), in their interface design (e.g., feedback affordances), or in their performance (e.g., predictive
accuracy). The studies already available in this domain suggest that system design and system characteristics
affect second- and third-party perceptions of automated and augmented decisions (Griesbach et al., 2019;
Hong et al., 2020; Longoni et al., 2019; Ravenelle, 2019), and as such, studying this issue haphazardly is no
longer informative to theory.
Furthermore, decision processes vary in the human and system control and contribution to the
decision, and the difference between fully-automated and augmented decisions seems to strongly affect
second and third parties. However, only a subset of the various possible configurations for decision
augmentation (Kaber & Endsley, 2004) have received attention, and even less so in the area of work-related
decisions. On the one hand, it could be possible that the exact configuration of how humans and systems
interact might not matter for second and third parties who may have little insight into the specific decision-
making process. On the other hand, this might indicate a neglect of many of the possible configurations
between the extremes human control and full automation. Indeed, the few papers that allow a conclusion
about whether the extent of human control matters indicate that there are psychological consequences that
seem to be associated with either the extent of human control or whether human decision-makers have the
final say over decisions (Newman et al., 2020). Future research should thus explicitly investigate the
implications of different levels of human control on second and third parties and thus contrast their potential
psychological consequences.
5.5 Theoretical Integration

The reviewed research either refrained from referring to a broader theoretical frame or mainly
focused on justice or signalling theory (Bangerter et al., 2012; Colquitt et al., 2001) to derive hypotheses
and to embed into the research landscape. Work characteristics and work design research (Parker & Grote,
2020) might provide another valuable theoretical lens for the implementation of decision automation and
augmentation into work contexts (Langer, König, & Busch, 2020) but is currently underutilized. Because
those theories focus on work processes and not necessarily on system design or decision-process-based
consequences that affect psychological processes, future researchers should focus upon the creation of
interdisciplinary theory. This involves combining the technical and system design insights of computer
science and human-computer interaction with the human processes insights of psychology and the applied
theory insights of management to enable more generalizable work for understanding decision automation
and augmentation (Landers & Marin, 2021).
6. Conclusion
Research in the area of automated and augmented decision-making has focused principally upon
first parties (Burton et al., 2020; Grove & Meehl, 1996; Hoff & Bashir, 2015). This review extends this
research by investigating the effects of decision automation and augmentation on second and third parties.
In our highly connected society, second and third parties can be exceptionally influential on organizations
considering the implementation of systems enabling the automation or augmentation of decisions;
understanding second- and third-party responses has never been more critical as society continues its march
towards automation and augmentation.
At the beginning of this review, we proposed three research questions: (1) how do automated and
augmented decisions affect second and third parties? (2) what moderates these effects? and (3) what are the
next crucial steps to meaningfully advance this research area? With respect to (1) we can conclude that there
are manifold influences of decision automation and augmentation related to second and third parties
regarding topics such as trust, fairness, responsibility, controllability, and autonomy. With regard to (2), we
found that those influences appear to be qualified in numerous underexplored ways, especially by the
characteristics of the decision process (i.e., different effects for automation compared to augmentation
contexts), the system, the second and third parties, the focal task, and outputs/outcome characteristics.
Regarding (3), our review revealed that most of our current understanding of second- and third-party effects
is driven by vignette studies, by studies of independent contractors reporting their experiences with
algorithmic management systems, and by generalizing from the field of medical decision-making. The
literature neglects the manifold system design choices that enable decision automation and augmentation,
making the generalizability of this literature to broader work contexts with more complex decision-making
systems unclear. Thus, it is critical for future research to tackle these issues of generalizability directly, by
more carefully considering their choice of terminology, research design, and methodology, as well as by
integrating interdisciplinary literature. Dramatic and substantial augmentation and automation of work by
“the system” is coming quickly, and researchers must be better prepared.

References
Acikgoz, Y., Davison, K. H., Compagnone, M., & Laske, M. (2020). Justice perceptions of artificial
intelligence in selection. International Journal of Selection and Assessment, Advance Online
Publication, 28(4), 399–416. doi:10.1111/ijsa.12306
Ananny, M., & Crawford, K. (2018). Seeing without knowing: Limitations of the transparency ideal and
its application to algorithmic accountability. New Media & Society, 20(3), 973–989.
doi:10.1177/1461444816676645
Anwar, I. A., Pal, J., & Hui, J. (2021). Watched, but moving: Platformization of beauty work and its
gendered mechanisms of control. Proceedings of the ACM on Human-Computer Interaction, 4, 1–
20. doi:10.1145/3432949
Arkes, H. R., Shaffer, V. A., & Medow, M. A. (2007). Patients derogate physicians who use a computer-
assisted diagnostic aid. Medical Decision Making, 27(2), 189–202.
doi:10.1177/0272989X06297391
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-
Lopez, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020). Explainable artificial
intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI.
Information Fusion, 58, 82–115. doi:10.1016/j.inffus.2019.12.012
Atzmüller, C., & Steiner, P. M. (2010). Experimental vignette studies in survey research. Methodology,
6(3), 128–138. doi:10.1027/1614-2241/a000014
Bangerter, A., Roulin, N., & König, C. J. (2012). Personnel selection as a signaling game. Journal of
Applied Psychology, 97(4), 719–738. doi:10.1037/a0026078
Benbasat, I., & Nault, B. R. (1990). An evaluation of empirical research in managerial support systems.
Decision Support Systems, 6(3), 203–226. doi:10.1016/0167-9236(90)90015-J
Bigman, Y. E., & Gray, K. (2018). People are averse to machines making moral decisions. Cognition,
181, 21–34. doi:10.1016/j.cognition.2018.08.003

Bigman, Y. E., Gray, K., Waytz, A., Arnestad, M., & Wilson, D. (2020). Algorithmic discrimination
causes less moral outrage than human discrimination [Preprint]. PsyArXiv.
doi:10.31234/osf.io/m3nrp
Bucher, E. L., Schou, P. K., & Waldkirch, M. (2021). Pacifying the algorithm – Anticipatory compliance
in the face of algorithmic management in the gig economy. Organization. Advance Online
Publication, 28(1), 44–67. doi:10.1177/1350508420961531
Burrell, J. (2016). How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big
Data & Society, 3(1), 205395171562251. doi:10.1177/2053951715622512
Burton, J. W., Stein, M., & Jensen, T. B. (2020). A systematic review of algorithm aversion in augmented
decision making. Journal of Behavioral Decision Making, 33(2), 220–239. doi:10.1002/bdm.2155
Castelo, N., Bos, M. W., & Lehmann, D. R. (2019). Task-dependent algorithm aversion. Journal of
Marketing Research, 56(5), 809–825. doi:10.1177/0022243719851788
Colquitt, J. A. (2001). On the dimensionality of organizational justice: A construct validation of a
measure. Journal of Applied Psychology, 86(3), 386–400. doi:10.1037/0021-9010.86.3.386
Colquitt, J. A., Conlon, D. E., Wesson, M. J., Porter, C. O. L. H., & Ng, K. Y. (2001). Justice at the
millenium: A meta-analytic review of 25 years of organizational justice research. Journal of
Applied Psychology, 86(3), 425–445. doi:10.1037//0021-9010.86.3.425
Colquitt, J. A., & Rodell, J. B. (2011). Justice, trust, and trustworthiness: A longitudinal analysis
integrating three theoretical perspectives. Academy of Management Journal, 54(6), 1183–1206.
doi:10.5465/amj.2007.0572
Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243(4899),
1668–1674. doi:10.1126/science.2648573
Dietvorst, B. J., & Bharti, S. (2020). People reject algorithms in uncertain decision domains because they
have diminishing sensitivity to forecasting error. Psychological Science, 31(10), 1302–1314.
doi:10.1177/0956797620948841
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid
algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126.
doi:10.1037/xge0000033
Dineen, B. R., Noe, R. A., & Wang, C. (2004). Perceived fairness of web-based applicant screening
procedures: Weighing the rules of justice and the role of individual differences. Human Resource
Management, 43(2–3), 127–145. doi:10.1002/hrm.20011
Duggan, J., Sherman, U., Carbery, R., & McDonnell, A. (2020). Algorithmic management and app‐work
in the gig economy: A research agenda for employment relations and HRM. Human Resource
Management Journal, 30(1), 114–132. doi:10.1111/1748-8583.12258
Fazio, R. H., & Williams, C. J. (1986). Attitude accessibility as a moderator of the attitude–perception and
attitude–behavior relations: An investigation of the 1984 presidential election. Journal of
Personality and Social Psychology, 51(3), 505-514#. doi:10.1037/0022-3514.51.3.505
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R.,
Pagallo, U., Rossi, F., Schafer, B., Valcke, P., & Vayena, E. (2018). AI4People—An ethical
framework for a good AI society: Opportunities, risks, principles, and recommendations. Minds
and Machines, 28(4), 689–707. doi:10.1007/s11023-018-9482-5
Galière, S. (2020). When food‐delivery platform workers consent to algorithmic management: A
Foucauldian perspective. New Technology, Work and Employment, 35(3), 357–370.
doi:10.1111/ntwe.12177
Gill, T. G. (1995). Early expert systems: Where are they now? MIS Quarterly, 19(1), 51.
doi:10.2307/249711
Gonzalez, M. F., Capman, J. F., Oswald, F. L., Theys, E. R., & Tomczak, D. L. (2019). “Where’s the I-
O?” Artificial intelligence and machine learning in talent management systems. Personnel
Assessment and Decisions, 3, 5. doi:10.25035/pad.2019.03.005
Gray, K., & Wegner, D. M. (2012). Feeling robots and human zombies: Mind perception and the uncanny
valley. Cognition, 125(1), 125–130. doi:10.1016/j.cognition.2012.06.007

Grgić-Hlača, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018). Human perceptions of fairness
in algorithmic decision making: A case study of criminal risk prediction. Proceedings of the 2018
World Wide Web Conference on World Wide Web, 903–912. doi:10.1145/3178876.3186138
Griesbach, K., Reich, A., Elliott-Negri, L., & Milkman, R. (2019). Algorithmic control in platform food
delivery work. Socius: Sociological Research for a Dynamic World, 5, 237802311987004.
doi:10.1177/2378023119870041
Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic)
and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy.
Psychology, Public Policy, and Law, 2(2), 293–323. doi:10.1037/1076-8971.2.2.293
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical
prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30. doi:10.1037//1040-
3590.12.1.19
Haan, M., Ongena, Y. P., Hommes, S., Kwee, T. C., & Yakar, D. (2019). A qualitative study to
understand patient perspective on the use of artificial intelligence in radiology. Journal of the
American College of Radiology, 16(10), 1416–1419. doi:10.1016/j.jacr.2018.12.043
Hamilton, J. G., Genoff Garzon, M., Westerman, J. S., Shuk, E., Hay, J. L., Walters, C., Elkin, E.,
Bertelsen, C., Cho, J., Daly, B., Gucalp, A., Seidman, A. D., Zauderer, M. G., Epstein, A. S., &
Kris, M. G. (2019). “A Tool, Not a Crutch”: Patient perspectives about IBM Watson for oncology
trained by memorial sloan kettering. Journal of Oncology Practice, 15(4), e277–e288.
doi:10.1200/JOP.18.00417
Harwell, D. (2019). Rights group files federal complaint against AI-hiring firm HireVue, citing ‘unfair
and deceptive’ practices. The Washington Post.
https://www.washingtonpost.com/technology/2019/11/06/prominent-rights-group-files-federal-
complaint-against-ai-hiring-firm-hirevue-citing-unfair-deceptive-practices/
Healy, J., Pekarek, A., & Vromen, A. (2020). Sceptics or supporters? Consumers’ views of work in the
gig economy. New Technology, Work and Employment, 35(1), 1–19. doi:10.1111/ntwe.12157
Highhouse, S. (2008). Stubborn reliance on intuition and subjectivity in employee selection. Industrial
and Organizational Psychology, 1(3), 333–342. doi:10.1111/j.1754-9434.2008.00058.x
Höddinghaus, M., Sondern, D., & Hertel, G. (2020). The automation of leadership functions: Would
people trust decision algorithms? Computers in Human Behavior, 116, 106635.
doi:10.1016/j.chb.2020.106635
Hoff, K. A., & Bashir, M. (2015). Trust in automation: Integrating empirical evidence on factors that
influence trust. Human Factors, 57(3), 407–434. doi:10.1177/0018720814547570
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019). Improving fairness in
machine learning systems: What do industry practitioners need? Proceedings of the 2019 CHI
Conference on Human Factors in Computing Systems, 1–16. doi:10.1145/3290605.3300830
Hong, J.-W., Choi, S., & Williams, D. (2020). Sexist AI: An experiment integrating CASA and ELM.
International Journal of Human–Computer Interaction, 36(20), 1928–1941.
doi:10.1080/10447318.2020.1801226
Howard, F. M., Gao, C. A., & Sankey, C. (2020). Implementation of an automated scheduling tool
improves schedule quality and resident satisfaction. PLOS ONE, 15(8), e0236952.
doi:10.1371/journal.pone.0236952
Jarrahi, M. H., & Sutherland, W. (2019). Algorithmic management and algorithmic competencies:
Understanding and appropriating algorithms in gig work. In N. G. Taylor, C. Christian-Lamb, M.
H. Martin, & B. Nardi (Eds.), Information in Contemporary Society, Lecture Notes in Computer
Science (11420th ed., pp. 578–589). Springer. doi:10.1007/978-3-030-15742-5_55
Jarrahi, M. H., Sutherland, W., Nelson, S. B., & Sawyer, S. (2020). Platformic management, boundary
resources for gig work, and worker autonomy. Computer Supported Cooperative Work, 29(1–2),
153–189. doi:10.1007/s10606-019-09368-7
Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine
Intelligence, 1(9), 389–399. doi:10.1038/s42256-019-0088-2

Jonmarker, O., Strand, F., Brandberg, Y., & Lindholm, P. (2019). The future of breast cancer screening:
What do participants in a breast cancer screening program think about automation using artificial
intelligence? Acta Radiologica Open, 8(12), 205846011988031. doi:10.1177/2058460119880315
Jungmann, F., Jorg, T., Hahn, F., Pinto dos Santos, D., Jungmann, S. M., Düber, C., Mildenberger, P., &
Kloeckner, R. (2020). Attitudes toward artificial intelligence among radiologists, IT specialists,
and industry. Academic Radiology, S1076633220302038. doi:10.1016/j.acra.2020.04.011
Jutzi, T. B., Krieghoff-Henning, E. I., Holland-Letz, T., Utikal, J. S., Hauschild, A., Schadendorf, D.,
Sondermann, W., Fröhling, S., Hekler, A., Schmitt, M., Maron, R. C., & Brinker, T. J. (2020).
Artificial intelligence in skin cancer diagnostics: The patients’ perspective. Frontiers in Medicine,
7. doi:10.3389/fmed.2020.00233
Kaber, D. B., & Endsley, M. R. (2004). The effects of level of automation and adaptive automation on
human performance, situation awareness and workload in a dynamic control task. Theoretical
Issues in Ergonomics Science, 5(2), 113–153. doi:10.1080/1463922021000054335
Keel, S., Lee, P. Y., Scheetz, J., Li, Z., Kotowicz, M. A., MacIsaac, R. J., & He, M. (2018). Feasibility
and patient acceptability of a novel artificial intelligence-based screening model for diabetic
retinopathy at endocrinology outpatient services: A pilot study. Scientific Reports, 8(1).
doi:10.1038/s41598-018-22612-2
Kellogg, K. C., Valentine, M. A., & Christin, A. (2020). Algorithms at work: The new contested terrain of
control. Academy of Management Annals, 14(1), 366–410. doi:10.5465/annals.2018.0174
Kinder, E., Jarrahi, M. H., & Sutherland, W. (2019). Gig platforms, tensions, alliances and ecosystems:
An actor-network perspective. Proceedings of the ACM on Human-Computer Interaction, 3, 1–26.
doi:10.1145/3359314
Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus clinical data
combination in selection and admissions decisions: A meta-analysis. Journal of Applied
Psychology, 98(6), 1060–1072. doi:10.1037/a0034156

Landers, R. N., & Marin, S. (2021). Theory and technology in organizational psychology: A review of
technology integration paradigms and their effects on the validity of theory. Annual Review of
Organizational Psychology and Organizational Behavior, 8(1), 235–258. doi:10.1146/annurev-
orgpsych-012420-060843
Langer, M., König, C. J., & Busch, V. (2020). Changing the means of managerial work: Effects of
automated decision-support systems on personnel selection tasks. Journal of Business and
Psychology, Advance Online Publication. doi:10.1007/s10869-020-09711-6
Langer, M., König, C. J., & Fitili, A. (2018). Information as a double-edged sword: The role of computer
experience and information on applicant reactions towards novel technologies for personnel
selection. Computers in Human Behavior, 81, 19–30. doi:10.1016/j.chb.2017.11.036
Langer, M., König, C. J., & Hemsing, V. (2020). Is anybody listening? The impact of automatically
evaluated job interviews on impression management and applicant reactions. Journal of
Managerial Psychology, 35(4), 271–284. doi:10.1108/JMP-03-2019-0156
Langer, M., König, C. J., & Papathanasiou, M. (2019). Highly-automated job interviews: Acceptance
under the influence of stakes. International Journal of Selection and Assessment, 27(3), 217–234.
doi:10.1111/ijsa.12246
Langer, M., König, C. J., Sanchez, D. R.-P., & Samadi, S. (2019). Highly automated interviews: Applicant
reactions and the organizational context. Journal of Managerial Psychology, 35(4), 301–314.
doi:10.1108/JMP-09-2018-0402
Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021).
What do we want from explainable artificial intelligence (XAI)? A stakeholder perspective on
XAI and a conceptual model guiding interdisciplinary XAI research. Artificial Intelligence, 296,
103473. doi:10.1016/j.artint.2021.103473
Lecher, C. (2019). How Amazon automatically tracks and fires warehouse workers for ‘productivity.’ The
Verge. https://www.theverge.com/2019/4/25/18516004/amazon-warehouse-fulfillment-centers-
productivity-firing-terminations
Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors,
46(1), 50–80. doi:10.1518/hfes.46.1.50.30392
Lee, M. K. (2018). Understanding perception of algorithmic decisions: Fairness, trust, and emotion in
response to algorithmic management. Big Data & Society, 5(1), 205395171875668.
doi:10.1177/2053951718756684
Lee, M. K., Kusbit, D., Metsky, E., & Dabbish, L. (2015). Working with machines: The impact of
algorithmic and data-driven management on human workers. Proceedings of the 33rd Annual
ACM Conference on Human Factors in Computing Systems - CHI ’15, 1603–1612.
doi:10.1145/2702123.2702548
Lombrozo, T. (2011). The instrumental value of explanations. Philosophy Compass, 6(8), 539–551.
doi:10.1111/j.1747-9991.2011.00413.x
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019). Resistance to medical artificial intelligence.
Journal of Consumer Research, 46(4), 629–650. doi:10.1093/jcr/ucz013
Lowe, D. J., Reckers, P. M. J., & Whitecotton, S. M. (2002). The effects of decision-aid use and reliability
on jurors’ evaluations of auditor liability. Accounting Review, 77(1), 185–202.
doi:10.2308/accr.2002.77.1.185
Makarius, E. E., Mukherjee, D., Fox, J. D., & Fox, A. K. (2020). Rising with the machines: A
sociotechnical framework for bringing artificial intelligence into the organization. Journal of
Business Research, 120, 262–273. doi:10.1016/j.jbusres.2020.07.045
Marcinkowski, F., Kieslich, K., Starke, C., & Lünich, M. (2020). Implications of AI (un-)fairness in
higher education admissions: The effects of perceived AI (un-)fairness on exit, voice and
organizational reputation. Proceedings of the 2020 FAT* Conference on Fairness, Accountability,
and Transparency, 122–130. doi:10.1145/3351095.3372867
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust.
Academy of Management Review, 20(2), 709–726. doi:10.2307/258792

Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the
evidence. University of Minnesota Press.
Merritt, S. M., Unnerstall, J. L., Lee, D., & Huber, K. (2015). Measuring individual differences in the
perfect automation schema. Human Factors, 57(5), 740–753. doi:10.1177/0018720815581247
Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial
Intelligence, 267, 1–38. doi:10.1016/j.artint.2018.07.007
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of algorithms:
Mapping the debate. Big Data & Society, 3(2), 205395171667967.
doi:10.1177/2053951716679679
Möhlmann, M., & Zalmanson, L. (2017). Hands on the Wheel: Navigating algorithmic management and
Uber drivers’ autonomy. Proceedings of the 2017 International Conference on Information
System. https://aisel.aisnet.org/icis2017/DigitalPlatforms/Presentations/3
Möhlmann, M., Zalmanson, L., & Gregory, R. W. (in press). Algorithmic management of work on online
labor platforms: When matching meets control. MIS Quarterly. Advance Online Publication.
Murray, A., Rhymer, J., & Sirmon, D. G. (2020). Humans and technology: Forms of conjoined agency in
organizations. Academy of Management Review. doi:10.5465/amr.2019.0186
Myhill, K., Richards, J., & Sang, K. (2021). Job quality, fair work and gig work: The lived experience of
gig workers. International Journal of Human Resource Management. Advance Online
Publication. doi:10.1080/09585192.2020.1867612
Nagtegaal, R. (2021). The impact of using algorithms for managerial decisions on public employees’
procedural justice. Government Information Quarterly, 38(1), 101536.
doi:10.1016/j.giq.2020.101536
Nelson, C. A., Pérez-Chada, L. M., Creadore, A., Li, S. J., Lo, K., Manjaly, P., Pournamdari, A. B.,
Tkachenko, E., Barbieri, J. S., Ko, J. M., Menon, A. V., Hartman, R. I., & Mostaghimi, A. (2020).
Patient perspectives on the use of artificial intelligence for skin cancer screening: A qualitative
study. JAMA Dermatology, 156(5), 501. doi:10.1001/jamadermatol.2019.5014

Newman, D. T., Fast, N. J., & Harmon, D. J. (2020). When eliminating bias isn’t fair: Algorithmic
reductionism and procedural justice in human resource decisions. Organizational Behavior and
Human Decision Processes, 160, 149–167. doi:10.1016/j.obhdp.2020.03.008
Nolan, K., Carter, N., & Dalal, D. (2016). Threat of technological unemployment: Are hiring managers
discounted for using standardized employee selection practices? Personnel Assessment and
Decisions, 2(1), 4. doi:10.25035/pad.2016.004
O’Neill, T., McNeese, N., Barron, A., & Schelble, B. (2020). Human–autonomy teaming: A review and
analysis of the empirical literature. Human Factors, 001872082096086.
doi:10.1177/0018720820960865
Onnasch, L., Wickens, C. D., Li, H., & Manzey, D. (2014). Human performance consequences of stages
and levels of automation: An integrated meta-analysis. Human Factors, 56(3), 476–488.
doi:10.1177/0018720813501549
Ötting, S. K., & Maier, G. W. (2018). The importance of procedural justice in human-machine-
interactions: Intelligent systems as new decision agents in organizations. Computers in Human
Behavior, 89, 27–39. doi:10.1016/j.chb.2018.07.022
Palmeira, M., & Spassova, G. (2015). Consumer reactions to professionals who use decision aids.
European Journal of Marketing, 49(3/4), 302–326. doi:10.1108/EJM-07-2013-0390
Palmisciano, P., Jamjoom, A. A. B., Taylor, D., Stoyanov, D., & Marcus, H. J. (2020). Attitudes of
patients and their relatives toward Artificial Intelligence in neurosurgery. World Neurosurgery,
138, e627–e633. doi:10.1016/j.wneu.2020.03.029
Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human
Factors, 39(2), 230–253. doi:10.1518/001872097778543886
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human
interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A:
Systems and Humans, 30(3), 286–297. doi:10.1109/3468.844354

Parker, S. K., & Grote, G. (2020). Automation, algorithms, and beyond: Why work design matters more
than ever in a digital world. Applied Psychology, Advance Online Publication.
doi:10.1111/apps.12241
Pezzo, M. V., & Pezzo, S. P. (2006). Physician evaluation after medical errors: Does having a computer
decision aid help or hurt in hindsight? Medical Decision Making, 26(1), 48–56.
doi:10.1177/0272989X05282644
Promberger, M., & Baron, J. (2006). Do patients trust computers? Journal of Behavioral Decision
Making, 19(5), 455–468. doi:10.1002/bdm.542
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020). Mitigating bias in algorithmic hiring:
Evaluating claims and practices. Proceedings of the 2020 FAT* Conference on Fairness,
Accountability, and Transparency. doi:10.1145/3351095.3372828
Raisch, S., & Krakowski, S. (2021). Artificial intelligence and management: The automation-
augmentation paradox. Academy of Management Review. doi:10.5465/amr.2018.0072
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D.,
& Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for
internal algorithmic auditing. Proceedings of the 2020 FAT* Conference on Fairness,
Accountability, and Transparency, 12. doi:10.1145/3351095.3372873
Ravenelle, A. J. (2019). “We’re not uber:” control, autonomy, and entrepreneurship in the gig economy.
Journal of Managerial Psychology, 34(4), 269–285. doi:10.1108/JMP-06-2018-0256
Schlicker, N., Langer, M., Ötting, S. K., König, C. J., Baum, K., & Wallach, D. (2021). What to expect
from opening “Black Boxes”? Comparing perceptions of justice between human and automated
agents. Computers in Human Behavior. Advance Online Publication.
doi:10.1016/j.chb.2021.106837
Shaffer, J. A., DeGeest, D., & Li, A. (2016). Tackling the problem of construct proliferation: A guide to
assessing the discriminant validity of conceptually related constructs. Organizational Research
Methods, 19(1), 80–110. doi:10.1177/1094428115598239

Shaffer, V. A., Probst, C. A., Merkle, E. C., Arkes, H. R., & Medow, M. A. (2013). Why do patients
derogate physicians who use a computer-based diagnostic support system? Medical Decision
Making, 33(1), 108–118. doi:10.1177/0272989X12453501
Shin, D., & Park, Y. J. (2019). Role of fairness, accountability, and transparency in algorithmic
affordance. Computers in Human Behavior, 98, 277–284. doi:10.1016/j.chb.2019.04.019
Srivastava, M., Heidari, H., & Krause, A. (2019). Mathematical notions vs. human perception of fairness:
A descriptive approach to fairness for machine learning. Proceedings of the 2019 ACM SIGKDD
International Conference on Knowledge Discovery & Data Mining, 2459–2468.
doi:10.1145/3292500.3330664
Stai, B., Heller, N., McSweeney, S., Rickman, J., Blake, P., Vasdev, R., Edgerton, Z., Tejpaul, R.,
Peterson, M., Rosenberg, J., Kalapara, A., Regmi, S., Papanikolopoulos, N., & Weight, C. (2020).
Public perceptions of artificial intelligence and robotics in medicine. Journal of Endourology,
34(10), 1041–1048. doi:10.1089/end.2020.0137
Suen, H.-Y., Chen, M. Y.-C., & Lu, S.-H. (2019). Does the use of synchrony and artificial intelligence in
video interviews affect interview ratings and applicant attitudes? Computers in Human Behavior,
98, 93–101. doi:10.1016/j.chb.2019.04.012
Tassinari, A., & Maccarrone, V. (2020). Riders on the storm: Workplace solidarity among gig economy
couriers in Italy and the UK. Work, Employment and Society, 34(1), 35–54.
doi:10.1177/0950017019862954
Tobia, K., Nielsen, A., & Stremitzer, A. (2021). When does physician use of AI increase liability? Journal
of Nuclear Medicine, 62(1), 17–21. doi:10.2967/jnumed.120.256032
Uhde, A., Schlicker, N., Wallach, D. P., & Hassenzahl, M. (2020). Fairness and decision-making in
collaborative shift scheduling systems. Proceedings of the 2020 CHI Conference on Human
Factors in Computing Systems, 1–13. doi:10.1145/3313831.3376656

van Dongen, K., & van Maanen, P.-P. (2013). A framework for explaining reliance on decision aids.
International Journal of Human-Computer Studies, 71(4), 410–424.
doi:10.1016/j.ijhcs.2012.10.018
Veen, A., Barratt, T., & Goods, C. (2020). Platform-capital’s ‘app-etite’ for control: A labour process
analysis of food-delivery work in Australia. Work, Employment and Society, 34(3), 388–406.
doi:10.1177/0950017019836911
Wang, R., Harper, F. M., & Zhu, H. (2020). Factors influencing perceived fairness in algorithmic
decision-making algorithm outcomes, development procedures, and individual differences.
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 14.
doi:10.1145/3313831.3376813
Wesche, J. S., & Sonderegger, A. (2019). When computers take the lead: The automation of leadership.
Computers in Human Behavior, 101, 197–209. doi:10.1016/j.chb.2019.07.027
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software
engineering. Proceedings of the 18th International Conference on Evaluation and Assessment in
Software Engineering - EASE ’14, 1–10. doi:10.1145/2601248.2601268
Wolf, J. R. (2014). Do IT students prefer doctors who use IT? Computers in Human Behavior, 35, 287–
294. doi:10.1016/j.chb.2014.03.020
Wood, A. J., Graham, M., Lehdonvirta, V., & Hjorth, I. (2019). Good gig, bad gig: Autonomy and
algorithmic control in the global gig economy. Work, Employment and Society, 33(1), 56–75.
doi:10.1177/0950017018785616
Yokoi, R., Eguchi, Y., Fujita, T., & Nakayachi, K. (2020). Artificial intelligence is trusted less than a
doctor in medical treatment decisions: Influence of perceived care and value similarity.
International Journal of Human–Computer Interaction, 1–10.
doi:10.1080/10447318.2020.1861763
York, T., Jenney, H., & Jones, G. (2020). Clinician and computer: A study on patient perceptions of
artificial intelligence in skeletal radiography. BMJ Health & Care Informatics, 27(3), e100233.
doi:10.1136/bmjhci-2020-100233
Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2018). Transparency in algorithmic and human
decision-making: Is there a double standard? Philosophy & Technology, 32(4), 661–683.
doi:10.1007/s13347-018-0330-6
View publication stats

LangerLanders2021ReviewAIatwork Preprint

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

LangerLanders2021ReviewAIatwork Preprint

Uploaded by

Copyright:

Available Formats

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

The future of artiﬁcial intelligence at work: A review on effects of decision

Article in Computers in Human Behavior · May 2021

Markus Langer Richard N Landers

SEE PROFILE SEE PROFILE

The user has requested enhancement of the downloaded file.

augmentation on workers targeted by algorithms and third-party observers

Markus Langer1 & Richard N. Landers²

²Department of Psychology, University of Minnesota, Minneapolis, MN

4767. E-mail: markus.langer@uni-saarland.de

and augmentation on workers targeted by algorithms and third-party observers

- We explore various moderators of attitudes, perceptions, and behavioural outcomes

- Reactions to decision augmentation differ from reactions to decision automation

- We provide recommendations regarding methodology, terminology, and theory

- System design choices are important but respective studies underrepresented

Keywords: automated and augmented decision-making, artificial intelligence, algorithmic decision-

making; perceptions, attitudes, review paper

and management (Burton et al., 2020; Longoni et al., 2019).

(Benbasat & Nault, 1990; Highhouse, 2008).

is not as well understood.

quantitative, investigating the effects of decision automation or augmentation on second- or third-party

medical decision-making that appeared to be relevant to work-related contexts. Medicine is somewhat

“automated, automation, algorithm*, artificial intelligence, autonomous machine, computer-assisted,

review and also provides contextual information regarding those studies.

Records identified through

Additional records identified by Records excluded with respect

Full text articles screened Records excluded with respect

Studies included in review

3. Effects of Decision Automation and Augmentation on Second and Third Parties

response to their experiences or observations, respectively, of automated or augmented decision-making.

1986), so none should be considered in isolation.

evaluations of trustworthiness, reductionism, as well as reluctance regarding automation of moral decisions.

3.1.1 Evaluations of Trustworthiness

et al., 2020; Nelson et al., 2020).

algorithmic management (Galière, 2020).

decision-making, as well as given trustworthiness evaluations as potential antecedents regarding perceptions

appear to assume that automated decisions suffer from reductionism.

3.1.3 Reluctance regarding Automation of Moral Decisions

and responsibility, supervision vs. autonomy, and privacy invasion.

3.2.1 Fairness and Justice

investigated, a concept we revisit in greater detail later in this review.

perceptions of justice (Colquitt & Rodell, 2011).

workers’ performance (Galière, 2020).

3.2.2 Organizational Attractiveness

(Marcinkowski et al., 2020).

This suggests unexplored moderators affecting people’s perceptions of organizational

Marcinkowski et al., 2020).

3.2.3 Accountability and Responsibility

3.2.4 Being under System Control vs. Autonomy

3.2.5 Privacy Invasion

Automated decision-making can lead to perceptions of privacy invasion under certain

We identified four commonly studied classes of behavior in response to automated decision-

to work assignment and evaluation (Galière, 2020).

3.3.1 Reverse Engineering of Automated Decisions

3.3.2 Creating Workarounds

create workarounds (Galière, 2020; Möhlmann et al., in press).

3.3.3 Collective Action

(Möhlmann et al., in press; Tassinari & Maccarrone, 2020).

3.3.4 Embracing the System

evaluation of workers’ performance (Galière, 2020).

4. Moderators of Effects on Second and Third Parties

Table 3 provides an overview of characteristics of the decision-making process, of the system, of

4.1 Characteristics of the Decision-Making Process