Subjectivity in Performance Evaluations A Review of The Literature

19113838, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/1911-3838.12273 by UNED, Wiley Online Library on [13/02/2023].
See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Subjectivity in Performance Evaluations:
A Review of the Literature*
SARA WICK, University of Guelph†
ABSTRACT
Subjectivity is an important element of employees’ performance evaluations because its
use can motivate employees and improve their productivity. Given this importance, it has
been a prominent area of research within the management accounting literature. Using a
structured approach, I review articles from 11 highly ranked accounting journals across
12 years with the objective of synthesizing and assessing the research to identify research
gaps and opportunities for future research. I observe that two types of subjectivity are
commonly studied: subjective performance measures and ex post discretion, across a
wide range of settings. Research questions are investigated, drawing on theory from eco-
nomics, psychology, and organizational behavior and using experimental, field study,
survey, archival, analytical, and interview methods. My synthesis of the literature high-
lights many opportunities for future research to further the study of subjective perfor-
mance evaluations. This study contributes to practice and accounting research by
synthesizing and providing insights about the subjectivity literature as well as identifying
opportunities for future research.
Keywords: subjectivity, subjective performance evaluations, discretion, literature
review
LA SUBJECTIVITÉ DANS LES ÉVALUATIONS DE RENDEMENT : ANALYSE

DOCUMENTAIRE

RESUM
E
La subjectivité joue un rôle important dans les évaluations de rendement des employés,
car elle peut être un élément motivateur et accroître la productivité des employés.
Compte tenu de cette importance, elle représente un champ d’études de premier plan
dans la littérature sur la comptabilité de gestion. À l’aide d’une approche structurée,
j’examine des articles tirés de 11 revues comptables bien classées parus sur une période
de 12 ans pour faire une synthèse et une évaluation de la recherche afin d’en cerner les
lacunes et d’établir des pistes aux fins de futures recherches. J’observe que deux types de
subjectivité sont couramment examinés, soit les mesures de rendement subjectives et les
* Accepted by Linda Thorne. This paper is based on part of my dissertation completed at Wilfrid Laurier Uni-
versity. Many thanks to Linda Thorne (associate editor) and two anonymous reviewers for their helpful sug-
gestions. I gratefully acknowledge the guidance and advice of my dissertation committee co-chairs, Leslie
Berger and Lan Guo. I am also thankful to the other members of my dissertation committee, Adam Presslee,
Anne Farrell, and Christian Jordan. I gratefully acknowledge the financial support provided by CPA/Laurier
Centre for the Advancement of Accounting Research and Education.
† Corresponding author.
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021) pages 653–685 © CAAA/ACPC

doi:10.1111/1911-3838.12273
19113838, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/1911-3838.12273 by UNED, Wiley Online Library on [13/02/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
654 ACCOUNTING PERSPECTIVES / PERSPECTIVES COMPTABLES
mesures discrétionnaires ex post, dans un vaste éventail de situations. J’examine les ques-
tions de recherche posées qui s’appuient sur diverses théories économiques, psy-
chologiques et du comportement organisationnel au moyen d’expériences, d’études sur le
terrain, de sondages, d’archives et d’entrevues. Ma synthèse de la littérature dégage
plusieurs pistes d’étude possibles pour faire avancer la recherche sur les évaluations de
rendement subjectives. La présente étude contribue à la pratique et à la recherche com-
ptables en présentant une synthèse de la littérature sur la subjectivité et en fournissant
des perspectives sur le sujet, en plus de cerner des occasions aux fins de futures
recherches.
Mots-clés : subjectivité, évaluations du rendement subjectives, discrétion, analyse
documentaire
1. INTRODUCTION
Subjectivity has long been an important component of employees’ performance evaluations
(Prendergast and Topel 1993) and management accounting research (Bol 2008). Subjectivity
in performance evaluations occurs when it is difficult for managers to engage in performance
evaluations with solely objective measures or when the objective measures available do not
fully reflect the underlying performance of the employee. Given the importance of perfor-
mance evaluations to increase employees’ effort and motivation (Bonner and Sprinkle 2002),
the study of subjectivity in performance evaluations has been a significant and abundant area
of the management accounting research in the past decade because of its practical and theo-
retical importance. Practically speaking, the use of subjectivity in performance evaluation sys-
tems is increasingly prevalent. Organizations operating in the 21st century are considering
knowledge workers and their output to be their most valuable assets (Drucker 1999;
Zimmerman 2015). For example, employees who contribute to research and development are
far more in demand due to the rise of companies such as Facebook, Twitter, Amazon, and
Google (Zimmerman 2015). Evaluating the output of knowledge workers requires elements
of subjectivity because it is often judged on its quality, which can make it difficult to evaluate
using objective measures. At the theoretical level, the subjective performance evaluation liter-
ature has been previously reviewed in the Journal of Accounting Literature in 2008
(Bol 2008). At that time, the extant literature was quite limited and in the early stages of
development, and provided promise that it would continue to grow.
Given that this subjective evaluation literature is of practical importance and has seen
recent growth within the management accounting field, in this paper I provide a review
of the existing research examining subjectivity in performance evaluations that has been
published in highly ranked accounting journals for the years spanning 2009 through
2020. My objectives for this review are twofold: first, to synthesize and critically assess
the existing subjectivity in performance evaluations research, and second, to identify ave-
nues for future research to further explore this important topic. To achieve these objec-
tives, I adopt a structured approach where, after determining the topic and objectives of
the review, I identify relevant articles using a manual and electronic search of 11 highly
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

SUBJECTIVITY IN PERFORMANCE EVALUATIONS 655
ranked accounting journals.1 Then, I synthesize and critically assess the literature by clas-
sifying each article by its topic, setting, theory, research method, and results. This method
of classification allows me to critically assess the existing research to identify themes,
knowledge gaps, and opportunities for future research.
The results of my review provide several important insights. I observe that the majority
of articles examine two types of subjectivity: subjective performance measures
(i.e., indicators of performance that require the evaluators’ judgment (Moers 2005)) and ex
post adjustment and allocation (i.e., discretion that allows managers to make ex post
adjustments to ex ante performance expectations (Gibbs et al. 2004)). My analysis shows
that subjectivity is examined across a wide range of settings, such as banking and finance,
retail, and professional services, and is explained by drawing on theory from economics,
psychology, and organizational behavior. I provide evidence that a diverse set of research
methods are used to examine subjectivity, including experiments, field study, survey, archi-
val, analytical, and interviews. In my synthesis of the subjectivity literature, I identify
important gaps that provide opportunities for future research. Specifically, interesting areas
of future research include employees’ reactions to subjectivity, how to mitigate the costs of
subjectivity, the effect of subjectivity on outcomes other than employees’ performance and
bonuses, understanding how different types of subjectivity can be used to reduce compen-
sation risk, and identifying geographic differences in the benefits and costs.
My literature review makes several important contributions to accounting practice
and research. Subjectivity is an important element of employee performance and contract
design, and the literature provides many examples of when its use is beneficial and when
its use can be costly. My synthesis of the literature identifies insights and conclusions
about subjective performance evaluations that will be useful for managers who are
responsible for evaluating employees’ performance and who are interested in the benefits
and costs of a contract design that allows for subjectivity. This literature review also con-
tributes to the management accounting literature in examining performance evaluations
and incentives. Similar to other literature reviews in management accounting
(e.g., Bonner et al. 2000; Hoque 2014; Shields 1997), my review provides a source for
researchers to learn about existing research in subjectivity in performance evaluations
and to stimulate thought for future research.
The remainder of this paper is organized as follows. In section 2, I provide a detailed
description of the method that my review follows. In section 3, I examine the articles
reviewed, and in section 4, I synthesize and critically evaluate the research to provide
future research opportunities. Finally, in section 5, I provide concluding remarks.
2. METHOD
I adopt a structured approach to organize and synthesize existing research in subjective
performance evaluations. A structured approach allows me to make important insights in
1. Thirteen accounting journals are reviewed; however, 2 journals did not publish any relevant articles
over the review period. Therefore, I discuss the results from the 11 remaining journals.
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

developing future research questions (Massaro et al. 2016).2 I begin the structured
approach with the identification of a general topic and the determination of the review
objectives. The topic I selected is subjectivity in performance evaluations because of its
practical and theoretical importance. After the topic was selected, I then defined the
objectives of the review: to synthesize and critically assess prior research, to identify
research gaps, and to discuss future research opportunities.
The next step of the structured approach is to select journals and years to include in the
review and the search vehicle. I select 13 high-quality accounting journals from the Financial
Times’s Top 50 (FT50) Journal List and the Australian Business Deans Council (ABDC)
Journal Quality List.3 For each journal, I examine the years spanning 2009 through 2020. I
select these years because both journal articles and working papers in the years up to and
including 2008 are discussed in a comprehensive review of subjectivity in compensation con-
tracting in the Journal of Accounting Literature (Bol 2008). Although research examining
subjectivity in performance evaluations exists in journals outside of accounting (e.g., Demeré
et al. 2019; Maas and Torres-Gonzalez 2011), similar to other literature reviews in account-
ing (e.g., Broadbent and Guthrie 1992; Shields 1997), I limit the scope of my review to
accounting journals only. I do so to focus my review on subjectivity literature with a contex-
tual focus of performance evaluations as part of the management controls system.
To identify relevant articles, I follow a two-stage approach established in previous
accounting literature reviews (e.g., Hoque 2014; MacTavish et al. 2018). For all journals
from the review period, I first read the table of contents (which includes titles and
abstracts) and then I perform two keyword searches (using the terms “subjective perfor-
mance evaluation” and “subjectivity”). I evaluate all articles against a single criterion: the
article must consider subjectivity in the evaluation of employees. Two of the journals
selected (AAAJ and AJPT) did not publish any relevant journal articles. The remaining
11 journals are discussed throughout the rest of this review. Once the relevant articles are
selected, I continue the structured approach by developing a systematic method to orga-
nize the relevant articles with an overarching objective of identifying relevant themes and
synthesizing the literature, as discussed below.
3. FINDINGS
Across the 11 journals and 12 years reviewed, I identify 59 published articles. As
reported in Table 1, panel A, the journals publishing the most articles are TAR (27%),
2. The structured approach I followed is similar to other literature review articles in accounting
(e.g., Chenhall and Smith 2011; Hoque 2014; Roussy and Perron 2018; Shields 1997).
3. All six accounting journals from the FT50 Journal List are selected (Accounting, Organizations and
Society (AOS), Contemporary Accounting Research (CAR), Journal of Accounting and Economics
(JAE), Journal of Accounting Research (JAR), Review of Accounting Studies (RAS), and The Account-
ing Review (TAR)). The seven remaining journals are the A* accounting journals from ABDC Journal
Quality List which do not appear on the FT50 (Accounting, Auditing & Accountability Journal
(AAAJ), Auditing: A Journal of Practice & Theory (AJPT), European Accounting Review (EAR), Jour-
nal of Business Finance & Accounting (JBFA), Journal of Management Accounting Research (JMAR),
Management Accounting Research (MAR), and The British Accounting Review (BAR)).
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

TABLE 1
Summary of articles
Panel A: Journal frequency organized by year
Journala 2009–2012 2013–2016 2017–2020 Total Total (%)

b
TAR 10 1 5 16 27
EARc 3 3 3 9 15
MARd 1 4 4 9 15
JMARe 0 2 6 8 14
CARf 1 2 3 6 10
AOSg 4 1 0 5 8
BARh 0 1 1 2 3
JAEi 0 0 1 1 2
JARj 0 1 0 1 2
RASk 0 1 0 1 2
JBFAl 0 0 1 1 2
Total 19 16 24 59 100
Total (%) 32 27 41 100
Panel B: Topic organized by year
Topic 2009–2012 2013–2016 2017–2020 Total Total (%)

m
Subjective measures 9 7 12 28 47
Ex post discretionn 7 6 7 20 34
Ex post weightso 0 2 1 3 5
Otherp 3 1 4 8 14
Total 19 16 24 59 100
Panel C: Research setting organized by year
Research setting 2009–2012 2013–2016 2017–2020 Total Total (%)

Banking or finance 3 1 3 7 12
Retail 3 0 3 6 10
Professional services 1 0 2 3 5
Multiple industriesq 6 4 5 15 26
Other single industriesr 1 1 2 4 7
Generics 3 8 8 19 32
No settingt 2 2 1 5 8
Total 19 16 24 59 100
(The table is continued on the next page.)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

TABLE 1 (continued)
Panel D: Theory organized by year
Theory 2009–2012 2013–2016 2017–2020 Total Total (%)

Economic 6 5 5 16 27
Psychology 4 0 8 12 20
Organizational behavior 2 3 3 8 14
Economic and otheru 3 5 1 9 15
Management 1 1 1 3 5
Prior literature 3 2 6 11 19
Total 19 16 24 59 100
Panel E: Method organized by year
Method 2009–2012 2013–2016 2017–2020 Total Total (%)

Experimental 3 8 10 21 36
Field study (data) 4 1 5 10 17
Field study (survey) 3 0 0 3 5
Survey 3 4 3 10 17
Archival 3 0 4 7 12
Interviews 1 1 1 3 5
Analytical 2 2 1 5 8
Total 19 16 24 59 100
Notes: aAccounting, Auditing & Accountability Journal (AAAJ) and Auditing: A Journal of Practice & The-
ory (AJPT) were also reviewed; however, no relevant articles have been identified. bThe Accounting Review.
c
European Accounting Review. dManagement Accounting Research. eJournal of Management Accounting
Research. fContemporary Accounting Research. gAccounting, Organizations and Society. hThe British
Accounting Review. iJournal of Accounting and Economics. jJournal of Accounting Research. kReview of
Accounting Studies. lJournal of Business Finance & Accounting. m“Subjective measures are superior’s sub-
jective judgments about qualitative performance indicators” (Moers 2005, 68). n“Ex post discretion” refers to
bonus payments that are determined without using a strict formulaic approach (Ederhof 2010). o“Ex post
weights” refers to when managers have discretion over the weights on some (or all) of objective performance
measures (Gibbs et al. 2004). p“Other” includes articles that study (equally) more than one element of subjec-
tive evaluations or that examine a topic that does not relate to subjective performance measures, ex post flexi-
bility in weights, or ex post discretion to make adjustments and allocations. q“Multiple industries” refers to
studies that do not define a single industry. r“Other single industries” use a single industry but are not consid-
ered part of banking or finance, retail, or professional services. s“Generic” refers to when the setting is
abstract, stylized, or simplified (e.g., experimental studies). t“No setting” refers to when a referent industry is
not specified (e.g., analytical studies). u“Economic and other” refers to when economic theory and psychol-
ogy or organizational behavior theory is used.
followed by EAR (15%) and MAR (15%).4 In addition, 32% of the articles are published
between 2009 and 2012, 27% are published between 2013 and 2016, and 41% are publi-
shed between 2017 and 2020. The increase in publications in the last three years of my
review suggests that this area of research remains active. Next, I use a modified version
4. This analysis should be interpreted with caution, as not all journals publish the same number of articles
per year (Chenhall and Smith 2011; Hoque 2014).
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

of the framework adopted by Hoque (2014) to categorize each relevant article by topic,
research setting, primary theory, research method, and results.
In categorizing the articles by topic, I observe that 47% of articles examine the role
of subjective performance measures, 34% of articles examine allowing ex post discretion
to make adjustments and allocations, and 5% of articles examine ex post flexible
weighting of performance measures (see Table 1, panel B).
I identify seven research settings in my review of the articles: banking and finance,
retail, professional services, other single industries, multiple industries, generic, and no
setting.5 The articles examining subjectivity in performance evaluations are examined
across a broad range of these research settings. I find that 26% of the articles are in multi-
ple industries and 32% of the articles are in generic settings. Of the remaining article set-
tings, 12% are in banking and finance, 10% are in retail, 7% are in other single industry
settings, and 5% are in professional services (see Table 1, panel C).
I find that a wide variety of theories are used in the articles. The majority of articles
draw on economic theory (27%), psychology theory (20%), or organizational behavior
theory (14%). Some studies draw on a combination of economic and either psychology
or organizational behavior theory (15%), management theory (5%), or use prior literature,
primarily in management accounting, to address their research questions and formulate
hypotheses (19%) (see Table 1, panel D).
In the exploration of subjective performance evaluations, my analysis shows that the
most common method used is the experimental method (36%). Two types of field studies
are identified: one where the researchers obtain performance evaluation data from the organi-
zation (17%) and one where the researchers survey the organizations’ employees directly
(5%). Surveys with a wider distribution (i.e., using respondents from multiple organizations
and/or industries) are also a common method used (17%). Finally, a smaller number of stud-
ies use either the archival (12%), analytical (8%), or interview (5%) method (see Table 1,
panel E).
Finally, in my review of the subjectivity literature an important theme emerged:
research can be also categorized by the benefits and costs of subjectivity in performance
evaluations. In the next section, I use this theme to synthesize the literature and to iden-
tify opportunities for future research.
4. SUMMARY OF RESULTS
Benefits of Subjectivity
I identify several benefits of subjectivity that include risk reduction, reduction of perceived
unfairness, inducement of adaptive behavior, and mitigation of incentive distortion.
5. Other single industries include various single-industry studies where an industry was considered by only
one study. Multiple industries include more than one industry, and generic settings are most commonly
experimental studies that often define a setting but are abstract, stylized, or simplified (Shields 1997).
Articles included in the no-setting category do not specify a referent industry (e.g., analytical studies).
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Risk Reduction
Subjectivity can benefit employees’ contracts because it can reduce the compensation risk
to employees. When only objective evaluations are used, one compensation risk
employees can face is that objective performance evaluations can be negatively
influenced by uncontrollable events (Bol 2008).6 Uncontrollable events are unknown ex
ante in contract design and therefore cannot be contracted on explicitly. To reduce this
risk to employees, managers can be endowed with ex post discretion. This discretion
allows managers to incorporate information learned during the contracting period
(e.g., information related to the occurrence of an uncontrollable event) that was unknown
ex ante in the contract design phase (Baker et al. 1994; Gibbs et al. 2004). Archival
research provides evidence of ex post adjustments in subjective performance evaluations
to reduce such compensation risk that relates to the occurrence of uncontrollable negative
events. One study of CEO incentive contracts from 579 firms shows that discretionary
bonuses are often included in compensation contracts of CEOs when ex ante performance
measures are likely to be impacted by uncontrollable events (Höppe and Moers 2011).
That said, a second archival study examines the likelihood of ex post adjustments before
and after the introduction of the enhanced SEC compensation disclosures mandated in
the compensation discussion and analysis (CD&A) (Gipper 2021). The study shows that
when companies are required to make such enhanced disclosures, they are less likely to
subsequently make discretionary ex post adjustments and rely on more objective (formu-
laic) measures.
Research from the field also provides evidence supporting the use of subjectivity in
performance evaluations to reduce compensation risk stemming from the occurrence of
negative uncontrollable events. For example, when sales targets are determined centrally
by corporate headquarters, managers are more likely to use their discretion and set more
attainable sales targets for postal stores that face higher (vs. lower) environmental uncer-
tainty (Bol et al. 2010). A second study of a large US retailer finds that when managers
are exposed to environmental uncertainty and face greater noise from objective measures
of performance, supervisors will increase subjective performance evaluations (Anderson
et al. 2020). Finally, a survey of foreign subsidiaries that operate in Belgium shows that
when there is an interdependency between headquarters and a subsidiary, evaluations of
the subsidiary can be negatively affected by the decisions made by headquarters
(i.e., representing an uncontrollable event), and therefore the use of subjectivity in the
evaluation of the subsidiary is more likely (Du et al. 2013).
Several studies broaden our understanding of the use of ex post adjustments to filter
out negative uncontrollable events by examining how employees will react to such
adjustments. For example, an experimental study using 142 undergraduate students pro-
vides evidence that when ex post goal adjustments are available to filter out unforeseen
and uncontrollable events, performance is improved for moderately set goals because of
positive perceptions of procedural justice (Kelly et al. 2015). A second experimental
6. Objective performance measures are used when employees’ output is quantifiable and is verifiable for
contracting purposes (Woods 2012).
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

study also shows how employees react to managers’ use of ex post adjustments. When
work periods are difficult and when ex post adjustments are available to filter out nega-
tive uncontrollable events, superior-subordinate identity moderates the relationship
between the availability of an ex post adjustment and the expectancy of receiving a
reward, such that the expectancy of the reward is higher when superior-subordinate iden-
tity is high compared to when it is low (Burt et al. 2020). Research from the field also
examines how employees’ performance is affected by managers’ use of ex post adjust-
ments. Two studies show that managers are more likely to award larger subjective
bonuses when ex ante performance targets are difficult (Anderson et al. 2020; Aranda
et al. 2019), and these larger subjective bonuses result in improved employee perfor-
mance in the subsequent period (Aranda et al. 2019).
Critique of Existing Literature and Future Research Opportunities

As prior literature (e.g., Bol 2008) suggests that ex post adjustments are useful to fil-
ter out negative uncontrollable events, it is not surprising that the most common type
of subjectivity studied is managers’ use of ex post adjustments and allocations
(Table 2, panel A). Since most of the literature examines how managers use ex post
discretionary adjustments and allocations (Table 2, panel A), little is known about the
different ways other types of subjectivity can be used by managers to reduce compen-
sation risk and how employees react to managers’ use of subjectivity. This creates
opportunities for future research.
First, researchers should identify how different uses of subjectivity can be helpful to
reduce compensation risk. For example, work from the field suggests that when environ-
mental uncertainty is high, managers are likely to use subjective ratings of performance
(i.e., subjective measures) (Anderson et al. 2020). Future research should consider in
what other environments subjective measures might be helpful in reducing compensation
risk and whether elements of subjectivity can be combined to reduce employees’ com-
pensation risk. Second, research should continue to explore how employees react to man-
agers’ efforts to reduce their compensation risk. Although prior research identifies that
managers’ use of subjectivity benefits the compensation contract by reducing compensa-
tion risks (Bol 2008), the true benefit will not be fully known until empirical research
can support how employees themselves react to such subjectivity. While some research
shows that performance can be improved when there is environmental uncertainty and
subjective ratings are provided (Aranda et al. 2019), experimental evidence shows that
contextual factors can hinder the effectiveness of subjectivity on performance (Burt
et al. 2020; Kelly et al. 2015). Additional research in understanding employees’ reactions
to subjective adjustments would broaden our understanding of how contextual variables
may hinder the effectiveness of compensation risk reduction. For example, it would be
interesting for research to consider how employees’ productivity is affected in the short
and long term, what the effect of subjectivity is on the quality of work produced by
employees, how subjectivity impacts employees’ extra-role behaviors, and whether there
are impacts of subjective evaluations on employees’ mental health (such as well-being or
stress level).
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

TABLE 2
662
Articles by cost and benefit
Panel A: Risk reduction
Paper Method Setting Topica Perspective Location

Bol et al. (2010) Field study (data) Retail Other—Discretion in Manager Korea
target setting
Höppe and Moers (2011) Archival Multiple industries Subjective measures Manager USA
Du et al. (2013) Survey Multiple industries Discretionary allocations Manager Belgium
Kelly et al. (2015) Experimental Generic Discretionary allocations Employee Canada
Aranda et al. (2019) Field study (data) Retail Discretionary allocations Both Europe
Gipper (2021) Archival Multiple industries Discretionary allocations Manager USA
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Burt et al. (2020) Experimental Generic Discretionary allocations Employee N/A
Anderson et al. (2020) Field study (data) Retail Subjective measures Manager USA
Panel B: Reduction of perceived unfairness

Hartmann and Field study (survey) Banking Subjective measures Employee Slovenia
ACCOUNTING PERSPECTIVES / PERSPECTIVES COMPTABLES
Slapničar (2009)
Bol et al. (2010) Field study (data) Retail Other—Discretion in Manager Korea
target setting
Maas et al. (2012) Experimental Generic Discretionary allocations Manager The Netherlands
Hartmann and Slapničar (2012) Field study (survey) Banking Subjective measures Employee Slovenia
Liu and Experimental Generic Discretionary allocations Employee N/A
Leitch (2013)
TABLE 2 (continued)
Panel B: Reduction of perceived unfairness

Bellavance Survey Multiple industries Mixed—Subjective Employee Canada
et al. (2013) measures, ex post
weights
Voußem et al. (2016) Survey Multiple industries Subjective measures Employee Germany,
Austria,
Switzerland
Arnold et al. (2018) Experimental Generic Discretionary allocations Both USA
Arnold and Tafkov (2019) Experimental Generic Discretionary allocations Employee USA
Hecht et al. (2019) Experimental Generic Other—Discretion over Both N/A
feedback
Habran and Mouritsen (2020) Interviews Banking Other—Discretion in target Employee France
setting
Panel C: Inducement of adaptive behaviors

Höppe and Moers (2011) Archival Multiple industries Ex post weights Manager USA
Liang and Nan (2014) Analytical N/A Subjective measures Employee NA
Cheng and Coyte (2014) Experimental Generic Ex post weights Employee NA
Bol et al. (2015) Experimental Generic Discretionary allocations Manager USA
Mahlendorf et al. (2018) Survey Multiple industries Subjective measures Employee Europe
Chen et al. (2020) Archival Multiple industries Ex post weights Manager China
Kelly et al. (2020) Interviews and Professional services Subjective measures Employee Canada
experimental
SUBJECTIVITY IN PERFORMANCE EVALUATIONS
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

663
TABLE 2 (continued)
664
Panel D: Incentive distortion mitigation

Woods (2012) Field study (data and Professional services Discretionary allocations Manager N/A
survey)
Ter Bogt and Interviews and survey Other—University Subjective measures Employee The Netherlands
Scapens (2012) and UK
Indjejikian and Survey Multiple industries Subjective measures Manager The Netherlands
Matějka (2012)
Grabner and Field study (data) Banking Subjective measures Manager Multinational
Moers (2013)
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Grabner (2014) Survey Multiple industries Subjective measures Manager Germany,
Austria,
Switzerland
Kunz (2015) Experimental Generic Subjective measures Employee Germany
Mangen (2017) Archival Multiple industries Subjective measures Manager USA
Lachmann et al. (2015) Interviews Other—Hospitals Subjective measures Manager Germany
de Castro (2017) Survey Other—service Other—Did not specify Employee N/A

Bol and Leiby (2018) Experiment Generic Subjective measures Manager USA
Bormann (2020) Survey Other—Manufacturing Mixed—Subjective measures, ex post Manager Germany
weights, discretionary adjustments
Panel E: Intentional inaccurate assessments

Ahn et al. (2010) Archival Multiple industries Subjective measures Employee Korea
Bol et al. (2010) Field study (data) Retail Other—Discretion in target Manager Korea
setting
Bol (2011) Field study (data) Banking Subjective measures Both The Netherlands
TABLE 2 (continued)
Panel E: Intentional inaccurate assessments

Golman and Analytical N/A Subjective measures Manager N/A
Bhatia (2012)
Du et al. (2012) Field study (data) Multiple industries Mixed—Subjective Manager China
and interviews measures,
discretionary allocations
Larmande and Analytical N/A Discretionary allocations Manager N/A
Ponssard (2014)
Bol et al. (2016) Experimental Generic Discretionary allocations Manager USA
de Castro (2017) Survey Other Other—Did not specify Employee N/A
Du et al. (2018) Field study (data) Multiple industries Discretionary allocations Manager China
and interviews
Balakrishnan Analytical N/A Discretionary allocations Employee N/A
et al. (2019)
Hao (2021) Field study (data) Retail Subjective measures Both Taiwan
Grabner et al. (2020) Field study (data) Professional Subjective measures Manager International
services
Panel F: Unintentional inaccurate assessments

Rajan and Analytical N/A Discretionary allocations Manager N/A
Reichelstein (2009)
Ederhof (2010) Analytical and archival Multiple industries Discretionary allocations Manager USA
Bailey et al. (2011) Experimental Generic Discretionary allocations Manager USA
Bol and Smith (2011) Experimental Generic Subjective measures Manager N/A
Long et al. (2015) Experimental Generic Ex post weights Manager USA
SUBJECTIVITY IN PERFORMANCE EVALUATIONS
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

665
TABLE 2 (continued)
666
Panel F: Unintentional inaccurate assessments

Dai et al. (2018) Experimental Generic Subjective measures Manager China
Fehrenbacher et al. (2018) Experimental Generic Subjective measures Manager N/A
Majerczyk and Experimental Generic Discretionary allocations Manager United States
Thomas (2021)
Bloomfield et al. (2021) Experimental Banking Subjective measures Manager N/A
Panel G: Influence activities

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Gipper (2021) Archival Multiple industries Discretionary allocations Manager USA
Bachmann et al. (2020) Archival Multiple industries Subjective measures Employee Australia
Panel H: Uncertainty about measurement criteria

Hartmann et al. (2010) Field study (survey) Multiple industries Subjective measures Employee The Netherlands
Luft et al. (2016) Experimental Generic Subjective measures Both N/A
Notes: aDiscretionary allocations refer to ex post discretionary adjustments and allocations.
Taken together, although this area of research has received attention over the last
12 years, unanswered empirical questions remain. Research should continue to explore
how different types of subjectivity (e.g., subjective measures, ex post weights, or other
types) can reduce employees’ compensation risk and how employees react to managers’
efforts to reduce such risk.
Reduction of Perceived Unfairness

Subjectivity also benefits employees’ contracts because it can be used to reduce per-
ceived unfairness (Bol 2008). Perceived fairness is important in performance evaluations
because it affects employees’ motivation (Colquitt et al. 2001). Four studies directly
examine how subjectivity affects employees’ perceptions of fairness. First, a study of
156 finance professionals finds that employees’ fairness perceptions regarding the use of
subjective measures follow an inverted U-shape. Specifically, when the overall weighting
of subjective measures is low, employees’ fairness perceptions are high; however, when
the weighting of subjective measures is high, perceptions of fairness are low (Voußem
et al. 2016). Two studies using a survey of bank managers examine the formality of per-
formance evaluations (i.e., the level of objectivity) and perceptions of fairness. The first
study shows that as performance evaluations increase in formality, this increases
employees’ perceptions of the quality of feedback, which increases their trust (Hartmann
and Slapničar 2009), whereas the second study shows that perceptions of procedural jus-
tice are increased when performance evaluations are more formal (i.e., they are more
objective), but only when task uncertainty is low (Hartmann and Slapničar 2012).
Finally, a study that uses 317 Canadian Certified Management Accountants shows that
perceptions of procedural justice are negatively influenced by the use of ex post subjec-
tive weight flexibility and are not influenced by the use or weight subjective performance
measures (Bellavance et al. 2013).
When employees work in teams, perceived fairness is of particular importance
because it is often aggregate team output that is used to determine individual rewards.
Four experimental studies examine how subjective evaluations allow managers to
restore perceived fairness in a team setting (i.e., when team members do not contribute
similarly to aggregate team output). First, when making subjective evaluations, man-
agers are more likely to obtain costly information about individuals’ contributions to
team performance when aggregate measures provide a noisy signal about individuals’
contributions (i.e., when aggregate performance is less extreme) (Maas et al. 2012). A
second experimental study examines how to increase perceived fairness of compensa-
tion in a team setting (Arnold et al. 2018). Results of this study show that one way to
reduce the costs of obtaining information about individuals’ contributions to aggregate
team output is to endow employees with the opportunity to provide managers with sub-
jective communication about the efforts of other team members (i.e., report on mutual
monitoring). A third experiment shows that when the team task is not interdependent,
subjective evaluations increase team performance, but when the team task is
interdependent, subjective evaluations decrease team performance (Arnold and
Tafkov 2019). The results indicate that when the team task is interdependent, subjective
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

evaluations have a negative influence on team cohesion, which in turn has a negative
effect on team performance. Finally, one experimental study broadens our understand-
ing of managerial discretion in a team setting by examining managers’ discretion over
the provision of feedback rather than the determination of compensation (Hecht
et al. 2019). This study finds that managers will strategically provide information to
individual team members about their relative contributions to team output depending
on whether the team member is a low- or high-performing employee. Specifically, man-
agers will provide low performers, but not high performers, with relative performance
information about their performance.
In addition to the use of subjectivity to reduce perceived unfairness in teams, subjec-
tivity in the target-setting process also allows managers to restore fairness perceptions
when employees are rewarded based on relative performance evaluations (Habran and
Mouritsen 2020). This is particularly important when the group of employees under eval-
uation have heterogeneous abilities. In this case, research using data from 103 postal
stores in Korea shows that managers sometimes use discretion in the contract design
phase and assign more attainable targets to branches with lower ability to generate profit
in order to restore perceptions of fairness (Bol et al. 2010). In a similar regard, an experi-
mental study using 40 student participants shows that targets determined using discretion
ex post led to higher employer profit compared to targets determined ex ante (Liu and
Leitch 2013).
Critique of Existing Research and Future Research Opportunities

There is not one clear way that subjectivity can be used to restore employees’ fairness
perceptions that is evident in the existing literature as papers examine discretionary
adjustments and allocations, subjective measures, how discretion is used ex ante in the
target setting process, and how discretion can be applied in the provision of employees’
feedback (Table 2, panel B) An interesting opportunity for future research that was iden-
tified in my synthesis is understanding the different impacts that subjectivity has on
employees’ perceptions of fairness.
First, given that the extant literature shows that the use of subjectivity does not
always result in positive fairness perceptions, researchers should consider situations in
which subjectivity can positively and negatively affect employees’ perceptions of fairness
(e.g., Bellavance et al. 2013). Second, researchers should examine the consequences on
employees’ behavior of fairness perceptions, as only limited research considers percep-
tions of fairness as the mechanism (i.e., a mediator) to explain employees’ behaviors
(e.g., Kelly et al. 2015). For example, when subjectivity does influence employees’ fair-
ness perceptions, future research should examine what the impact is on employees’ work
behaviors such as performance, turnover, knowledge sharing, extra-role behavior, and
deviance. Thus, although this area of the subjectivity research has shown significant
growth over the last 12 years, it is important that the research continues to identify con-
textual factors and the ultimate effects on employees’ behavior in order to understand
when the use of subjectivity to restore employees’ fairness perceptions can be beneficial
versus when it is costly.
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Inducement of Adaptive Behaviors

Subjectivity also benefits compensation contracts because it can induce adaptive
behaviors. As the priorities of companies may change during the contracting period,
allowing managerial discretion to change the weighting of objective performances
may encourage employees to adapt their behaviors accordingly (Bol 2008).7 For
example, in an archival study using CEO contracts from SEC proxy statements, sub-
jective weights are more prevalent when there is environmental unpredictability
(Höppe and Moers 2011). Providing subjective weights in these fast-changing envi-
ronments can encourage CEOs to adapt quickly, which better aligns the goals of the
company with the goals of the CEO. Discretionary weights can also be used to ensure
that performance measures provide useful information about the actual tasks being
performed by employees. Specifically, the weights of objective versus subjective per-
formance measures can be tailored to be different for top managers compared to mid-
dle managers. An archival study using data collected from performance-equity plans
from 228 public firms in China finds that as environmental uncertainty, CEO power,
competition intensity, and nonprice competition increase, the likelihood that man-
agers will provide different weightings of objective versus subjective performance
measures for top managers compared to middle managers will also increase (Chen
et al. 2020). That said, the same study shows that when the coordination of top and
middle managers is important (i.e., high organizational stability or high growth
opportunities), the weights of objective versus subjective performance measures
between top and middle managers are less likely to be tailored.
Subjectivity can also be used to encourage employees to engage in behaviors outside
of their performance. Employing an incentive scheme that allows managers to subjec-
tively determine ex post performance weightings compared to a scheme that determines
performance weightings ex ante using a predetermined formula can encourage knowledge
sharing and extra-role behaviors (Cheng and Coyte 2014). Interestingly, one study exam-
ines how subjective evaluations are associated with the likelihood that managers will
engage in unethical pro-organizational behavior. The study surveys 253 finance profes-
sionals and shows no association between unethical pro-organizational behavior and sub-
jective evaluations (Mahlendorf et al. 2018). Another study examines whether
subjectivity will affect how law partners will accede to their clients (Kelly et al. 2020).
The study shows that when pay is not transparent, law partners are more likely to accede
when there is less subjectivity in the determination of their pay.
Managers may also consider how their use of subjectivity will affect employees’
adaptive behavior in future periods. An experimental study using 97 graduate student
participants who are required to decide bonus allocations for two employees examines
whether managers will consider the signal they will send to employees if they choose to
7. An analytical model that examines the relationship between precision of performance measures and
limited employee attention shows that when employees can improve the precision of the measures
related to nonproductive tasks, managers can divert employees’ attention away from such nonproduc-
tive tasks if they do not use such signals of performance (Liang and Nan 2014).
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

incorporate noncontractible subjective information into their subjective bonus allocations

(Bol et al. 2015). The study finds that when subjective noncontractible information is the
result of an uncontrollable event, managers are more likely to incorporate the non-
contractible information into bonus allocation decisions, but only when the likelihood of
the event is low (i.e., is not likely to reoccur) and the bonus pool is not shared among
other employees.
Critique of Existing Research and Future Research Opportunities

The study of subjectivity to encourage adaptative behaviors has been examined using
different types of subjectivity, using various methods, and across two geographic
regions (Table 2, panel C) The research identified in this review provides evidence
that supports managers’ use of subjectivity in performance evaluations to encourage
employees to adapt their behavior; however, more research is required in order to
fully understand whether subjectivity will lead to such beneficial adaptive behaviors
(and not to costly ones).
One opportunity is to examine the use of subjectivity in the field. My synthesis of the
literature highlights that there have not been any field studies examining how subjectivity
can be beneficial by inducing employees’ adaptive behaviors. Research from the field is
important because it allows researchers to better understand practice (e.g., implementation
issues, successes and failures can be observed) (Atkinson et al. 1997). A second opportu-
nity is to understand how employees’ adaptive behaviors are affected by subjectivity.
Although some research is underway in this area (e.g., Bol et al.’s (2015) study provides
an example of when managers will and will not use subjectivity to encourage adaptive
behaviors), it remains an empirical question as to whether employees will in fact engage in
the adaptive behaviors and what those adaptive behaviors might be. In order to fully under-
stand the benefit of subjectivity that induces employees’ adaptive behaviors, more research
is required from the employees’ perspective.
Taken together, as only seven papers examine how managers are using different ele-
ments of subjectivity to encourage adaptive behaviors (Table 2, panel C), it can be said
that this is a relatively unexplored area of the subjectivity literature, and more research is
necessary to continue understanding how subjectivity can be used to influence
employees’ adaptive behaviors.
Incentive Distortion Mitigation

Finally, subjectivity can benefit compensation contracts because it can mitigate incentive
distortions that arise when the compensation contract is incomplete (i.e., not all job
dimensions are observable or measurable) (Baiman and Rajan 1995; Bol 2008; Bushman
et al. 1996; Feltham and Xie 1994; Holmstrom 1979; Holmstrom and Milgrom 1991;
Murphy and Oyer 2003). Incentive distortions arise when difficult-to-measure job dimen-
sions are not incentivized, because this encourages employees to focus their effort on
only incentivized dimensions (Holmstrom and Milgrom 1991). To mitigate this problem,
managers can subjectively increase ratings of performance to incorporate effort that is
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

not easily quantified (Baker et al. 1994; Bol 2008).8 For example, a field study at a retail
bank shows that when objective measures of performance provide little information about
how an employee will perform once promoted, subjective assessments of the employee’s
ability are weighted higher in promotion decisions (Grabner and Moers 2013). An experi-
mental study also examines how subjective cues are interpreted in promotion decisions.
The study finds that when professional employees are eligible for promotion, their
engagement in consultative decision making has a more negative effect on promotion
prospect evaluations compared to when they are not promotion-eligible (Bol and
Leiby 2018). A second field study at an international audit organization finds that when
the organization adopts a new performance evaluation system and managers perceive the
new objective indicators to be insufficient measures of performance, the managers are
likely to use knowledge of their employees’ past performance and make upward adjust-
ments (Woods 2012). Interestingly, this second field study finds that while it is likely that
managers will make upward adjustments to objective performance ratings to improve the
measurement of performance, in some cases, upward adjustments are made for reasons
other than improving the measurement of performance (Woods 2012).
Research using the survey method also provides evidence about when subjectivity is
used in performance evaluations. The first study surveys 151 managers at German
manufacturing firms and finds that when firms follow multiple strategies (e.g., low cost
and differentiation), subjectivity and incentive intensity (i.e., fixed/variable pay) act as
complements where subjectivity will increase the effectiveness of incentives
(Bormann 2020). A second study using a survey shows that when business unit managers
have responsibilities over internal accounting decisions, they are more likely to be evalu-
ated using nonfinancial indicators of performance and subjective evaluations compared to
financial indicators of performance (Indjejikian and Matějka 2012). A third study that
surveys 457 creativity-dependent firms shows that one way to mitigate the problem of
narrow task achievement caused by performance-based pay is to include subjective eval-
uations on non-task-related aspects of employees’ work (Grabner 2014). Finally, a fourth
study that surveys 102 middle managers in the service industry suggests that subjectivity
is a multidimensional construct and when it is rule driven (i.e., it is part of the organiza-
tions management control system), it has a positive influence on employee empowerment
(de Castro 2017).
Archival research also examines when subjectivity might be used by managers in the
evaluation of CEOs. One study examines the likelihood that subjectivity will be used
more frequently in evaluation of CEO performance in the year subsequent to an eco-
nomic shock, as it would be more difficult to observe the true performance of the CEO
(Mangen 2017). The study uses US data from corporate proxy filings from 353 firms and
finds that, indeed, subjective evaluations of CEOs are increased after an economic shock.
8. Although subjectivity can be introduced to encourage employees to work on difficult-to-measure

aspects of their job, an experimental study using graduate student participants shows that the effective-
ness of subjectivity might depend on employees’ level of autonomous motivation such that employees
who are low on autonomous motivation will benefit from objective and precise evaluations
(Kunz 2015).
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Outside of a corporate setting, the role of subjectivity has been examined in the eval-
uation of employees. One research study uses interviews to examine whether the owner-
ship structure of a hospital will affect the use of objective versus subjective measures in
the evaluation of clinical staff. The study shows that when the hospital is not-for-profit,
subjective measures are more likely to be used, whereas if the hospital is public or pri-
vate, objective measures are more likely to be used (Lachmann et al. 2015). A second
study that uses interviews and surveys within academia provides some evidence
supporting how faculty members at two different universities in Europe react to a more
objective system of performance evaluation (Ter Bogt and Scapens 2012). The study
finds that when faculty research is evaluated using more objective output measures, fac-
ulty research output increases; however, the quality of that research is unclear.

Prior research indicates that subjective measures can be used to evaluate aspects of
employees’ jobs that are difficult to evaluate using objective measures (Bol 2008); there-
fore, it is not surprising that the most common use of subjectivity studied is subjective
measures Most of the studies exist outside of North America, provide evidence from the
field, and examine managers’ use of subjectivity (Table 2, panel D). The volume and
diversity of the research reviewed indicates that there has been quite a bit of growth in
this area of the literature; however, my analysis has identified two ways that this research
can continue to grow: expanding into different geographic regions and considering
employees reactions.
First, as prior research suggests, subjectivity is used differently in different countries
(Jansen et al. 2009; Merchant et al. 2011), and the bulk of this research is performed in
Europe; thus, future research should continue to consider when and why results may not
generalize to other cultures. Second, as most of the papers reviewed examine how subjec-
tivity is used by managers, the next step in advancing this area of research is to now
understand how employees’ behavior is affected by the use of subjectivity. For example,
research could examine whether there are contexts or individual characteristics that will
cause employees to behave outside of what managers expect and whether there are out-
comes outside of their productivity such as turnover or well-being that might be affected.
Overall, this area of the subjective performance evaluation literature has been well
examined over the last 12 years; however, there is still research that remains to be
explored. Developing a better understanding of how subjectivity is used in compensation
contracting in different parts of the world and examining when employees’ behavior may
deviate from managers’ expectations are interesting directions for future research.
Costs of Subjectivity
Although there are many benefits of subjective performance evaluations in compensation
contracting, existing research has identified some costs associated with its use. In this
section, I will examine three different costs of subjectivity: inaccurate evaluations, influ-
ence activities, and uncertainty about the measurement criteria.
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Inaccurate Subjective Performance Evaluations

An important cost that has emerged as a theme in the literature is that the use of subjec-
tivity can result in inaccurate performance evaluations (Baker et al. 1988). Inaccurate
subjective performance evaluations can be defined as the “divergence from ‘true’ perfor-
mance” (Bol 2008, 12). In some cases, inaccurate subjective performance evaluations can
be harmful because they reduce the effectiveness of incentives, which negatively affects
productivity at the firm level (Baker et al. 1988) and reduces effort or increases quitting
at the employee level (Bol 2008; Prendergast and Topel 1993). The subjective perfor-
mance evaluation literature has identified two types of inaccurate subjective performance
evaluations: (i) intentional inaccurate subjective performance evaluations and
(ii) unintentional inaccurate subjective performance evaluations. In turn, I will review
research on both types of inaccurate performance evaluations.
Intentional Inaccurate Evaluations. The opportunity for managers to act in their own
self-interest and make intentional inaccurate evaluations is increased when evaluations
are subjective. This is because, in most cases, the manager is responsible for making the
evaluations, but the organization (i.e., not the manager) bears the cost of any inaccurate
evaluations (Bol 2008; Moers 2005; Prendergast 1999; Prendergast and Topel 1993). As
a result, managers may act in their own self-interest and consider personal benefits and
costs in the evaluation of employees (Bol 2008; Prendergast and Topel 1993) and be
influenced by reasons other than improving the compensation contract (Woods 2012).
When managers incorporate these preferences into subjective performance evaluations,
they can provide lenient (i.e., overinflated evaluations for all or certain employees) or
centralized (i.e., performance evaluations with little variation between employees) assess-
ments of performance (Bol 2008; Bol et al. 2010, 2016).9
Building on early empirical research that provides evidence of both the leniency and
centrality biases (Moers 2005), recent archival research provides insights as to why sub-
jective performance evaluations are influenced by the leniency and centrality biases.
Using performance evaluation data of a financial services firm, two managerial prefer-
ences that are antecedents to intentional subjective biases are identified: avoidance of
high information-gathering costs and strong employee-manager relationships (Bol 2011).
In the firm studied, both high information-gathering costs and strong employee-manager
relationships influence both the leniency and the centrality biases, leading to inaccurate
evaluations of subjective performance. Interestingly, the results also show that perfor-
mance evaluations affected by the centrality bias have negative consequences on
employee performance, whereas performance evaluations affected by the leniency bias
do not have negative consequences on employee performance (Bol 2011). An archival
study using performance evaluation data from public enterprises in Korea provides evi-
dence that supports why the centrality bias negatively affects employee performance. The
9. An analytical model shows that even if managers prefer to provide accurate performance evaluations,
the inherent noise and a desire to be altruistic in subjective evaluations will result in both the leniency
and centrality biases (Golman and Bhatia 2012).
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

study finds that evaluations affected by the centrality bias lack discriminability, which
can have negative consequences on employee performance (Ahn et al. 2010).
Two studies, based on archival performance evaluation data from the Chinese gov-
ernment’s evaluation of Chinese state-owned enterprises (Du et al. 2012, 2018), provide
field evidence of managers’ intentional leniency biases. In their first study, the
researchers find that both bottom-up influences (e.g., the level of political connections of
the CFO and the geographic proximity to headquarters) and top-down favoritism
(e.g., the political rank of the firm) result in more favorable evaluations (Du et al. 2012).
In their second study, the researchers examine the likelihood of subjective adjustments
when new performance measures replace old performance measures. They find that when
the new indicator of performance is unfavorable and the old indicator of performance is
favorable, evidence of leniency bias out of concerns of fairness is observed
(Du et al. 2018).
One study examines the centrality bias without consideration of the leniency bias.
This experimental study, using 124 MBA student participants, finds that information
accuracy and transparency jointly influence managers’ intentional centrality bias in sub-
jective evaluations of performance (Bol et al. 2016). Interestingly, information accuracy
leads to less centralized performance scores, but only in settings where information trans-
parency is high (i.e., when employees are aware of each other’s performances).
Although the bulk of the existing literature examines leniency and centrality biases
in ex post evaluations of performance, one study provides evidence that the leniency bias
could also exist ex ante as part of the target-setting process (Bol et al. 2010). In a setting
where decreasing the sales target of one store will increase the sales target of another
store, this study shows that managers are more likely to subjectively set lower targets for
stores whose managers have a higher hierarchal status in order to avoid costly confronta-
tion costs.
In addition to examining the existence of the leniency and centrality biases, recent
research examines how such biases can be reduced. A study using performance evalua-
tion data from a professional services provider finds that both the leniency bias and the
centrality bias can be reduced by calibration committees (i.e., a group of higher-level
managers that review initial subjective evaluations and make adjustments where neces-
sary) (Grabner et al. 2020). This study shows that because calibration committees will
punish (via decreased performance ratings) supervisors who are strategically lenient and
reward (via a higher likelihood of promotion) supervisors who are able to provide less
compressed ratings, calibration committees can mitigate the leniency and centrality
biases. When organizations adopt a forced distribution system (i.e., only a certain number
of employees are permitted to earn each performance rank), it limits the opportunity for
the leniency and centrality biases. That said, a field study examining the performance of
car dealership employees shows that two types of managers’ preferences (the quality of
the relationship between the manager and the employee and the age difference between
the manager and the employee) persist (i.e., have a positive influence on subjective evalu-
ations) under a forced distribution system (Hao 2021). An analytical model provides a
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

way that the leniency bias that is often prevalent in employee fit evaluations
(i.e., retention decisions) can be reduced (Balakrishnan et al. 2019). The study shows that
when the performance measurement system evaluates both employee incentives and
employee fit, the system is likely to be more stringent. This has an indirect positive effect
on employees’ effort, which decreases managers’ need to be lenient.
Finally, one study provides evidence that the inclusion of subjectivity in performance
evaluations may not be a unidimensional construct and its consequences may depend on
either the subjective evaluations as set by the organization or the idiosyncrasies of the evalu-
ating manager (de Castro 2017). The study shows that the conflict that may arise between
the manager and the employee because subjectivity is positively associated with supervisor-
driven subjectivity and negatively associated with organization-driven subjectivity.
Unintentional Inaccurate Evaluations. Unintentional inaccurate assessments result

when managers’ cognitive limitations interfere with subjective evaluations of perfor-
mance. Specifically, a cognitive limitation “prevents managers from fully exploiting all
information on employee performance” (Bol 2008, 12). Empirical research examines dif-
ferent factors which result in cognitive limitations that affect subjective performance
evaluations.
Cognitive distortion can arise when managers evaluate employees using both objec-
tive and subjective indicators of performance (Bol and Smith 2011; Fehrenbacher
et al. 2018). In experimental settings, when managers have knowledge of employees’ rat-
ings on an unrelated objective indictor of performance, this knowledge will spill over into
managers’ assessments on the subjective indicator of performance (Bol and Smith 2011;
Fehrenbacher et al. 2018). This bias is shown to be stronger when managers engage in a
more intuitive decision mode as compared to a deliberate decision mode (Fehrenbacher
et al. 2018). An experimental study that uses 79 professional managers in China offers
some evidence as to why objective measures are weighted more heavily than subjective
measures in the evaluation of employees’ performance. The results indicate that objective
measures are weighted more heavily because managers perceive objective measures to be
more scientific (Dai et al. 2018).
Two studies use an analytical model to examine managers’ use of discretion in bonus
decisions when objective indicators of performance are also available. In a single-agent
setting, the first analytical model shows that in cases where both objective and subjective
performance indicators are available, managers will only incorporate subjective measures
into evaluations when objective measures indicate low levels of employee performance
(Rajan and Reichelstein 2009). A second analytical study shows that discretion in bonus
decisions is most likely to be used when objective indicators are extreme (i.e., either
really high or really low) (Ederhof 2010). Interestingly, this same study provides empiri-
cal support for the analytical model by hand-collecting data from 490 firms and 583 firm
years from SEC Form 8-K filings and proxy statements.
Recent research also examines how another type of external information can influ-
ence subjective evaluations. An experimental study uses 223 student participants who
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

assume the role of either a manager or an employee to examine how non-task-relevant

information can influence subjective bonus allocations when there is disparity in perfor-
mance risk among employees (Majerczyk and Thomas 2021). This study shows that
when managers have information about employees’ performance on an unrelated task,
they will allocate more initial resources to employees who perform better on the
unrelated task and they will allocate a larger bonus to these employees.
Experimental research examines how the anchoring and adjustment heuristic is used
when both objective and subjective information is available in bonus pool allocations
(Bailey et al. 2011). In this study, 170 graduate students adopt the role of the manager
and make bonus allocation decisions for fictitious employees. The results provide evi-
dence that, rather than considering objective and subjective information holistically, man-
agers adopt an anchoring and adjustment approach where managers anchor on either
objective, contractible information or an equal split between employees in the bonus
pool, and then insufficiently adjust for subjective noncontractible information. As a
result, the subjective noncontractible information is not fully incorporated into bonus
pool allocation decisions.
Research also examines how the outcome effect (i.e., knowledge of outcomes) can
bias managers’ subjective performance evaluations when knowledge of the outcome
directionally influences the evaluation. Using an experiment and 119 graduate student
participants who assume the role of manager, one study finds that the outcome effect
influences the extent to which managers incorporate subjective noncontractible informa-
tion when they evaluate performance across multiple measures (Long et al. 2015). More
specifically, when weight allocations of multiple performance measures are specified, and
when subjective information is consistent with the valence of a performance measure out-
come, managers are less likely to incorporate subjective noncontractible information into
evaluations of performance compared to when weight allocations are not specified.
Finally, a recent experimental study examines the role of cognitive categorization
and stereotypes in subjective evaluations (Bloomfield et al. 2021). The study uses 179
professional investors as participants and shows that when financial analysts display
behavior that is nontypical of an analyst, they are less likely to be recommended for a
promotion, but only in cases when the focal analyst is a female analyst (compared to a
male analyst).

Research examining subjectivity in performance evaluations that lead to inaccurate
assessments has grown significantly over the last 12 years (i.e., there are 21 papers that
examine inaccurate assessments). Generally speaking, research uses a variety of different
methods to examine different uses of subjectivity (Table 2, panels E and F). That said,
research that examines managers unintentional inaccurate assessments does so primarily
using an experimental method (Table 2, panel F). This means that the field evidence is
primarily from research that examines managers’ intentional inaccurate assessments and
most commonly in Asia (Table 2, panel E). To continue to advance this area of the
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

literature, research should consider questions that examine how employees react to inac-
curate assessments, identify ways to mitigate such biases, and provide more evidence
from the field.
First, inaccurate assessments of employees’ performance have long been considered
a cost of subjectivity because they deviate from the true performance of employees
(e.g., Prendergast and Topel 1993); however, there is insufficient empirical evidence
from the employees’ side in support of this claim. In fact, one empirical study provides
conflicting evidence by showing that just because subjective performance evaluations
deviate from the true performance of employees (i.e., are affected by managers’ bias),
this does not always result in negative performance outcomes (Bol 2011). As there is
evidence to suggest that not all inaccuracies have a negative effect on employees’ per-
formance, future research should consider in which situations inaccurate assessments
will become costly to organizations. This research could explore different contexts
(e.g., whether it depends on employees working physically onsite or remotely in an
offsite location), individual differences in employees (e.g., personality traits), and out-
come variables outside of performance (e.g., turnover, employee learning and growth,
and deviant behaviors).
Second, this research has identified a number of biases that result in inaccurate
assessments; however, very few studies provide ways to mitigate such biases (exceptions
include calibration committees and forced distributions (Grabner et al. 2020; Hao 2021)).
Research should continue to identify other managerial controls that can be used to both
prevent and detect inaccurate subjective evaluations. For example, research could explore
the role of artificial intelligence (AI) in performance evaluations or the use of cognitive
debiasing tools that prompt managers to exert more effort to collect and process relevant
information during performance evaluations, or identify management control features that
motivate managers to overcome biases.10
Third, the bulk of existing research that examines managers’ unintentional inaccurate
assessments uses the experimental method. This is not surprising as unintentional inaccu-
racies can manifest from cognitive limitations, which are tested easily in an experimental
lab. Evidence from the field would help to further understand the cost of unintentional
inaccurate assessments by showing when and how these inaccuracies exist in practice.
Inaccurate assessments in subjective evaluations have been well examined by
researchers over the 12-year period identified in this review. The discretion required by
managers to make subjective evaluations can lead to inaccuracies, which could under-
mine the effectiveness of subjective evaluations, and therefore it is important that
research continues to examine inaccurate assessments. Specifically, examining how
employees react to inaccurate assessments, identifying ways to mitigate such biases, and
providing more evidence from the field would be interesting areas for future research to
explore.
10. One recent study explores the role of AI in performance evaluation (see Bol et al. 2020).
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Influence Activities
Another cost of subjectivity in performance evaluations is that it provides an opportunity
for employees to engage in activities that would influence their managers to provide
higher ratings of performance and/or compensation (Bol 2008). For example, an archival
study using data from the Australian Stock Exchange shows that the more powerful the
CEOs are, the more likely it is that subjective nonfinancial targets will be used in their
bonus contracts, which are negatively associated with future firm performance
(Bachmann et al. 2020). That said, new disclosure requirements (i.e., CD&A) mandated
by the SEC enhance oversight of payments to employees and, therefore, provide one way
to mitigate such influence activities. However, recent archival research finds that these
new disclosures increase the total pay and therefore may not be effective in the mitigation
of influence activities (Gipper 2021).
Critique and Future Research Opportunities

Examining influence activities that occur when subjectivity is used in performance evalu-
ations has not received a great deal of attention during the period under review (Table 2,
panel G) As this area of the research has not been well examined, it is important to iden-
tify when employees are likely to engage in influence activities and in which situations
they are likely to be successful in influencing their managers’ evaluations. For example,
the current research shows that when employees are powerful, they are most likely to
influence their pay (and succeed) (Bachmann et al. 2020). Future research should con-
sider whether there are other employee characteristics (e.g., confidence, gender, rank, ten-
ure) that will lead to influence activities and what characteristics of the manager and/or
the organization (e.g., confidence, size of the company, overall incentive structure) will
lead to the success of influence activities.
Uncertainty About Measurement Criteria

When subjectivity is introduced into performance evaluations, it can be costly to
employees because they can lose control over how their effort will be evaluated and/or
translated into compensation (Bol 2008). Very limited research examines how uncertainty
of the measurement criteria affects managers’ use of and employees’ reactions to subjec-
tive evaluations. One experimental study that uses student participants who assume the
role of either a manager or an employee shows that coordination failures (between man-
agers and their employees) and negative performance evaluation surprises are more likely
when employees are uncertain about the measurement criteria (Luft et al. 2016). A sec-
ond study provides evidence from 196 managers in the field and shows that relationship
between initiating structure leadership style and goal clarity is mediated by the use of
objective (rather than subjective) performance measures (Hartmann et al. 2010).
Critique and Future Research Opportunities

Research examining uncertainty about the measurement criteria has also not received
much attention over the period under review (Table 2, panel H) As there is not a great
deal of empirical evidence to support why uncertainty about the measurement criteria can
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

be costly, this area of research should continue to explore how managers use subjectivity
when they are unsure about the measurement criteria and how employees’ behavior is
affected when the measurement criteria is uncertain. From the managers’ perspective, an
interesting area of future research would be to examine what factors affect how managers
go about making evaluations of their employees. From the employees’ side, research
could explore what factors will influence how employees’ performance is affected when
the measurement criteria are unclear. Exploring these two areas of research will provide
empirical support regarding how costly uncertainty about the measurement criteria in
subjective evaluations can be to organizations.
5. CONCLUSIONS AND LIMITATIONS

In this paper, I performed a structured literature review of the accounting research that
examines subjectivity in performance evaluations. Across 11 highly ranked accounting
journals and 12 years, I identified 59 articles to examine. I followed a systematic
approach where I classified each article by topic, theory, setting, method, and results.
This approach allowed me to identify several important themes. An overwhelming major-
ity of studies examine two types of subjective performance evaluations: subjective perfor-
mance measures and the use of ex post adjustments and allocations. This research
primarily draws from economic or psychology theory and uses a variety of different
research methods (such as experiments, field studies, and surveys). Field studies take
place across a variety of different settings such as banking and finance, and retail and
professional services; in many cases, multiple industries are used as the setting to exam-
ine research questions.
This review of the subjective performance evaluation literature provides both practi-
cal and theoretical contributions. First, with the rise of knowledge-based firms and
knowledge workers (Drucker 1999; Zimmerman 2015), subjectivity in performance eval-
uations is increasingly important in the assessment of employees. My study provides a
source for practicing accountants to better understand the benefits and costs of
implementing and using different types of subjective evaluations.
Second, this review expands on a previous review of the subjective performance
evaluation literature in accounting (Bol 2008). The previous review theoretically iden-
tifies and empirically supports the benefits and costs of using subjective performance
evaluations in compensation contracting while providing several unanswered questions
for future research to explore (such as distinguishing the type of subjectivity, the timing
of managers’ discretion, the relationship between objective and subjective evaluations,
the costs of subjectivity, and the time horizon) (Bol 2008). My synthesis of the literature
finds that many of these research questions have since been addressed, while some still
remain unanswered. Specifically, research has answered the call to distinguish between
the type of subjectivity (e.g., 28 papers examine subjective measures, 20 examine ex post
adjustments and allocations, and 3 examine ex post weights) and to understand the rela-
tionship between objective and subjective measures. For example, several papers exam-
ine cognitive limitations and the inaccuracies that can present when both types of
performance measures are used (e.g., Bailey et al. 2011; Bol and Smith 2011; Dai
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)
et al. 2018; Ederhof 2010; Fehrenbacher et al. 2018). Research has partially answered the
call to examine the timing of managers’ discretion as studies examine how subjectivity is
used ex ante in target-setting (Bol et al. 2010; Habran and Mouritsen 2020; Liu and
Leitch 2013) and in promotions (Bol and Leiby 2018; Grabner and Moers 2013); how-
ever, it still remains an empirical question as to how subjectivity can be used to deter-
mine different outcomes during the evaluation period (e.g., job assignments and training
opportunities). Research has also partially answered the call to examine two separate
costs of subjectivity: inaccurate assessments (evidence suggests that this call has been
answered, as 21 papers examine different types of inaccuracies) and the benefits and
costs of influence activities (this has not been examined by the existing research and
remains an empirical question). Finally, research has not answered the call to examine
time horizons as no paper under this review considers the long-term effects of subjectiv-
ity in performance evaluations; thus, this still remains an outstanding empirical question.
My review has provided evidence that over the last 12 years not only has research
addressed many gaps in the literature, but it has also significantly advanced knowledge by
providing empirical support for the benefits and costs of subjective performance evalua-
tions. That said, it is important to note that this review is not without limitations. First,
although I perform a two-step approach to search for relevant articles (using both manual
and electronic keywords), there is still a risk that articles examining subjective performance
evaluations were missed in this review. Second, this is a sole-authored literature review in
which I reviewed each article and performed the coding and categorization. As these arti-
cles were not reviewed by a second author, there could be undetected coding errors.
In summary, as publications in this field continue to increase, it is clear that there are
still many unexplored phenomena within this literature. In order to facilitate growth in
this field of study and to help researchers develop their questions, I critically assess each
benefit and cost and provide many possible opportunities for researchers to consider.
REFERENCES
Ahn, T. S., I. Hwang, and M. Kim. 2010. The impact of performance measure discriminability on
ratee incentives. The Accounting Review 85 (2): 389–417. https://doi.org/10.2308/accr.2010.
85.2.389
Anderson, S. W., H. C. Dekker, K. L. Sedatole, and E. Wiersma. 2020. When one size does not
fit all: Using ex post subjective ratings to provide parity in risk-adjusted compensation. Man-
agement Accounting Research 49: 1–12. https://doi.org/10.1016/j.mar.2020.100706
Aranda, C., J. Arellano, and A. Davila. 2019. Subjective bonuses and target setting in budget-
based incentive contracts. Management Accounting Research 43: 45–60. https://doi.org/10.
1016/j.mar.2018.07.003
Arnold, M. C., R. L. Hannan, and I. D. Tafkov. 2018. Team member subjective communication
in homogeneous and heterogeneous teams. The Accounting Review 93 (5): 1–22. https://doi.
org/10.2308/accr-52002
Arnold, M. C., and I. D. Tafkov. 2019. Managerial discretion and task interdependence in teams.
Contemporary Accounting Research 36 (4): 2467–93. https://doi.org/10.1111/1911-3846.
12504
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Atkinson, A. A., R. Balakrishnan, P. Booth, J. M. Cote, T. Groot, T. Malmi, H. Roberts, E.

Uliana, and A. Wu. 1997. New directions in management accounting research. Journal of
Management Accounting Research 9: 79–108.
Bachmann, R. L., A. Loyeung, Z. P. Matolcsy, and H. Spiropoulos. 2020. Powerful CEOs, cash
bonus contracts and firm performance. Journal of Business Finance & Accounting 47 (1–2):
100–31.
Bailey, W. J., G. Hecht, and K. L. Towry. 2011. Dividing the pie: The influence of managerial
discretion extent on bonus pool allocation. Contemporary Accounting Research 28 (5): 1562–
84. https://doi.org/10.1111/j.1911-3846.2011.01073.x
Baiman, S., and M. V. Rajan. 1995. The informational advantages of discretionary bonus
schemes. The Accounting Review 70 (4): 557–79.
Baker, G. P., R. Gibbons, and K. J. Murphy. 1994. Subjective performance measures in optimal
incentive contracts. Quarterly Journal of Economics 109 (4): 1125–56. https://doi.org/10.
2307/2118358
Baker, G. P., M. C. Jensen, and K. J. Murphy. 1988. Compensation and incentives: Practice
vs. theory. Journal of Finance 43 (3): 593–616. https://doi.org/10.1111/j.1540-6261.1988.
tb04593.x
Balakrishnan, R., G. Drymiotes, and K. Sivaramakrishnan. 2019. The effects of biasing perfor-
mance measurement systems on incentives and retention decisions. Contemporary Accounting
Research 36 (2): 1098–121. https://doi.org/10.1111/1911-3846.12461
Bellavance, F., S. Landry, and E. Schiehll. 2013. Procedural justice in managerial performance
evaluation: Effects of subjectivity, relationship quality, and voice opportunity. British
Accounting Review 45 (3): 149–66. https://doi.org/10.1016/j.bar.2013.06.013
Bloomfield, R., K. M. Rennekamp, B. A. Steenhoven, and S. D. Stewart. 2021. Penalties for
unexpected behavior: Double standards for women in finance. The Accounting Review 96 (2):
107–25. https://doi.org/10.2308/tar-2018-0715
Bol, J. C. 2008. Subjectivity in compensation contracting. Journal of Accounting Literature 27:
1–24.
Bol, J. C. 2011. The determinants and performance effects of managers’ performance evaluation
biases. The Accounting Review 86 (5): 1549–75. https://doi.org/10.2308/accr-10099
Bol, J. C., C. Brown, and L. LaViers. 2020. Differing perceptions of and preferences for human-
driven and artificial intelligence-driven evaluation systems. Working paper, Tulane
University.
Bol, J. C., G. Hecht, and S. D. Smith. 2015. Managers’ discretionary adjustments: The influence
of uncontrollable events and compensation interdependence. Contemporary Accounting
Bol, J. C., T. M. Keune, E. M. Matsumura, and J. Y. Shin. 2010. Supervisor discretion in target
setting: An empirical investigation. The Accounting Review 85 (6): 1861–86. https://doi.org/
10.2308/accr.2010.85.6.1861
Bol, J. C., S. Kramer, and V. S. Maas. 2016. How control system design affects performance eval-
uation compression: The role of information accuracy and outcome transparency. Accounting,
Organizations and Society 51 (May): 64–73. https://doi.org/10.1016/j.aos.2016.01.001
Bol, J. C., and J. Leiby. 2018. Subjectivity in professionals’ incentive systems: Differences
between promotion- and performance-based assessments. Contemporary Accounting
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Bol, J. C., and S. D. Smith. 2011. Spillover effects in subjective performance evaluation: Bias and
the asymmetric influence of controllability. The Accounting Review 86 (4): 1213–30. https://
doi.org/10.2308/accr-10038
Bonner, S. E., R. Hastie, G. B. Sprinkle, and S. M. Young. 2000. A review of the effects of financial
incentives on performance in laboratory tasks: Implications for management accounting. Journal
of Management Accounting Research 12 (1): 19–64. https://doi.org/10.2308/jmar.2000.12.1.19
Bonner, S. E., and G. B. Sprinkle. 2002. The effects of monetary incentives on effort and task per-
formance: Theories, evidence, and a framework for research. Accounting, Organizations and
Society 27 (4–5): 303–45. https://doi.org/10.1016/S0361-3682(01)00052-6
Bormann, S. 2020. Strategic priorities and organizational design. Journal of Management
Accounting Research 32 (3): 7–26. https://doi.org/10.2308/jmar-16-137
Broadbent, J., and J. Guthrie. 1992. Changes in the public sector: A review of recent “alternative”
accounting research. Accounting, Auditing & Accountability Journal 5 (2): 3–31. https://doi.
org/10.1108/09513579210011835
Burt, I., T. Libby, and A. Presslee. 2020. The impact of superior-subordinate identity and ex post
discretionary goal adjustment on subordinate expectancy of reward and performance. Journal
of Management Accounting Research 32 (1): 31–49. https://doi.org/10.2308/jmar-52559
Bushman, R. M., R. J. Indjejikian, and A. Smith. 1996. CEO compensation: The role of individual
performance evaluation. Journal of Accounting and Economics 21 (2): 161–93. https://doi.
org/10.1016/0165-4101(95)00416-5
Chen, C. X., Y. Gao, Y. Wang, and S. Xue. 2020. Tailoring the weights on objective versus sub-
jective performance measures between top management and middle managers: Evidence from
performance-based equity incentive plans. Journal of Management Accounting Research 32
(3): 49–70. https://doi.org/10.2308/jmar-18-042
Cheng, M. M., and R. Coyte. 2014. The effects of incentive subjectivity and strategy communica-
tion on knowledge-sharing and extra-role behaviours. Management Accounting Research 25
(2): 119–30. https://doi.org/10.1016/j.mar.2013.07.003
Chenhall, R. H., and D. Smith. 2011. A review of Australian management accounting research: 1980–
2009. Accounting & Finance 51 (1): 173–206. https://doi.org/10.1111/j.1467-629X.2010.00371.x
Colquitt, J. A., D. E. Conlon, M. J. Wesson, C. O. L. H. Porter, and K. Y. Ng. 2001. Justice at
the millennium: A meta-analytic review of 25 years of organizational justice research. Journal
of Applied Psychology 86 (3): 425–45. https://doi.org/10.1037/0021-9010.86.3.425
Dai, N. T., X. Kuang, and G. Tang. 2018. Differential weighting of objective versus subjective
measures in performance evaluation: Experimental evidence. European Accounting Review
27 (1): 129–48.
de Castro, V. B. 2017. Unpacking the notion of subjectivity: Performance evaluation and supervi-
sor discretion. British Accounting Review 49 (6): 532–44.
Demeré, B. W., K. L. Sedatole, and A. Woods. 2019. The role of calibration committees in sub-
jective performance evaluation systems. Management Science 65 (4): 1562–85.
Drucker, P. F. 1999. Knowledge-worker productivity: The biggest challenge. California Manage-
ment Review 41 (2): 79–94.
Du, F., D. H. Erkens, S. M. Young, and G. Tang. 2018. How adopting new performance measures
affects subjective performance evaluations: Evidence from EVA adoption by Chinese state-
owned enterprises. The Accounting Review 93 (1): 161–85. https://doi.org/10.2308/accr-
51763
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Du, F., G. Tang, and S. M. Young. 2012. Influence activities and favoritism in subjective perfor-
mance evaluation: Evidence from Chinese state-owned enterprises. The Accounting Review
87 (5): 1555–88. https://doi.org/10.2308/accr-50196
Du, Y., M. Deloof, and A. Jorissen. 2013. Headquarters—Subsidiary interdependencies and
the design of performance evaluation and reward systems in multinational enterprises.
European Accounting Review 22 (2): 391–424. https://doi.org/10.1080/09638180.2012.
739824
Ederhof, M. 2010. Discretion in bonus plans. The Accounting Review 85 (6): 1921–49. https://doi.
org/10.2308/accr.2010.85.6.1921
Fehrenbacher, D. D., A. K. D. Schulz, and K. Rotaru. 2018. The moderating role of decision
mode in subjective performance evaluation. Management Accounting Research 41: 1–10.
https://doi.org/10.1016/j.mar.2018.03.001
Feltham, G. A., and J. Xie. 1994. Performance measure congruity and diversity in multi-task
principal/agent relations. The Accounting Review 69 (3): 429–53.
Gibbs, M., K. A. Merchant, W. A. Van der Stede, and M. E. Vargus. 2004. Determinants and
effects of subjectivity in incentives. The Accounting Review 79 (2): 409–36. https://doi.org/
10.2308/accr.2004.79.2.409
Gipper, B. 2021. The economic effects of expanded compensation disclosures. Journal of
Accounting and Economics 71 (1): 101338. https://doi.org/10.1016/j.jacceco.2020.101338
Golman, R., and S. Bhatia. 2012. Performance evaluation inflation and compression. Accounting,
Organizations and Society 37 (8): 534–43. https://doi.org/10.1016/j.aos.2012.09.001
Grabner, I. 2014. Incentive system design in creativity-dependent firms. The Accounting Review
89 (5): 1729–50. https://doi.org/10.2308/accr-50756
Grabner, I., J. Künneke, and F. Moers. 2020. How calibration committees can mitigate perfor-
mance evaluation bias: An analysis of implicit incentives. The Accounting Review 95 (6):
213–33. https://doi.org/10.2308/tar-2016-0662
Grabner, I., and F. Moers. 2013. Managers’ choices of performance measures in promotion deci-
sions: An analysis of alternative job assignments. Journal of Accounting Research 51 (5):
1187–220. https://doi.org/10.1111/1475-679X.12027
Habran, Y., and J. Mouritsen. 2020. Making intensity of efforts the same: Commensuration work
in target-setting practices. European Accounting Review, forthcoming. https://doi.org/10.
1080/09638180.2020.1832901
Hao, J. P. 2021. Subjective performance evaluation and forward-looking implications: The role of
supervisor incentives. Journal of Management Accounting Research 33 (2): 109–27. https://
doi.org/10.2308/JMAR-19-023
Hartmann, F., D. Naranjo-Gil, and P. Perego. 2010. The effects of leadership styles and use of
performance measures on managerial work-related attitudes. European Accounting Review 19
(2): 275–310.
Hartmann, F., and S. Slapničar. 2009. How formal performance evaluation affects trust between
superior and subordinate managers. Accounting, Organizations and Society 34 (6–7): 722–37.
https://doi.org/10.1016/j.aos.2008.11.004
Hartmann, F., and S. Slapničar. 2012. The perceived fairness of performance evaluation: The role
of uncertainty. Management Accounting Research 23 (1): 17–33. https://doi.org/10.1016/j.
mar.2011.10.004
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Hecht, G., A. H. Newman, and I. D. Tafkov. 2019. Managers’ strategic use of discretion over rel-
ative performance information provision and implications for team-members’ effort. Manage-
ment Accounting Research 45: 1–19. https://doi.org/10.1016/j.mar.2019.01.001
Holmstrom, B. 1979. Moral hazard and observability. Bell Journal of Economics 10 (1): 74–91.
https://doi.org/10.2307/3003320
Holmstrom, B., and P. Milgrom. 1991. Multitask principal-agent analyses: Incentive contracts,
asset ownership, and job design. Journal of Law, Economics and Organization 7: 24–52.
Höppe, F., and F. Moers. 2011. The choice of different types of subjectivity in CEO annual bonus
contracts. The Accounting Review 86 (6): 2023–46. https://doi.org/10.2308/accr-10132
Hoque, Z. 2014. 20 Years of studies on the balanced scorecard: Trends, accomplishments, gaps
and opportunities for future research. British Accounting Review 46 (1): 33–59. https://doi.
org/10.1016/j.bar.2013.10.003
Indjejikian, R., and M. Matějka. 2012. Accounting decentralization and performance evaluation of
business unit managers. The Accounting Review 87 (1): 261–90. https://doi.org/10.2308/accr-
10168
Jansen, E. P., K. A. Merchant, and W. A. Van der Stede. 2009. National differences in incentive
compensation practices: The differing roles of financial performance measurement in the
United States and The Netherlands. Accounting, Organizations and Society 34 (1): 58–84.
Kelly, K., R. Dinovitzer, H. Gunz, and S. P. Gunz. 2020. The interaction of perceived subjectivity
and pay transparency on professional judgment in a profit pool setting: The case of large law
firms. The Accounting Review 95 (5): 227–46. https://doi.org/10.2308/accr-52612
Kelly, K., R. A. Webb, and T. Vance. 2015. The interactive effects of ex post goal adjustment
and goal difficulty on performance. Journal of Management Accounting Research 27 (1): 1–
25. https://doi.org/10.2308/jmar-50729
Kunz, J. 2015. Objectivity and subjectivity in performance evaluation and autonomous motiva-
tion: An exploratory study. Management Accounting Research 27: 27–46. https://doi.org/10.
1016/j.mar.2015.01.003
Lachmann, M., R. Trapp, and F. L. Wenger. 2015. Performance measurement and compensation
practices in hospitals: An empirical analysis in consideration of ownership types. European
Accounting Review 25 (4): 661–86.
Larmande, F., and J. P. Ponssard. 2014. Fishing for excuses and performance evaluation. Review
of Accounting Studies 19 (2): 988–1008. https://doi.org/10.1007/s11142-013-9268-1
Liang, P. J., and L. Nan. 2014. Endogenous precision of performance measures and limited mana-
gerial attention. European Accounting Review 23 (4): 693–727.
Liu, X. K., and R. A. Leitch. 2013. Performance effects of setting targets and pay–performance
relations before or after operations. Management Accounting Research 24 (1): 64–79. https://
doi.org/10.1016/j.mar.2012.12.002
Long, J. H., L. Mertins, and B. Vansant. 2015. The effects of firm-provided measure weightings
on evaluators’ incorporation of non-contractible information. Journal of Management
Accounting Research 27 (1): 47–62. https://doi.org/10.2308/jmar-50837
Luft, J., M. D. Shields, and T. F. Thomas. 2016. Additional information in accounting reports: Effects
on management decisions and subjective performance evaluations under causal ambiguity. Con-
temporary Accounting Research 33 (2): 526–50. https://doi.org/10.1111/1911-3846.12156
Maas, V. S., and R. Torres-Gonzalez. 2011. Subjective performance evaluation and gender dis-
crimination. Journal of Business Ethics 101 (4): 667–81.
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Maas, V. S., M. van Rinsum, and K. L. Towry. 2012. In search of informed discretion: An experi-
mental investigation of fairness and trust reciprocity. The Accounting Review 87 (2): 617–44.
https://doi.org/10.2308/accr-10205
MacTavish, C., S. McCracken, and R. N. Schmidt. 2018. External auditors’ judgment and deci-
sion making: An audit process task analysis. Accounting Perspectives 17 (3): 387–426.
https://doi.org/10.1111/1911-3838.12182
Mahlendorf, M. D., M. Matějka, and J. Weber. 2018. Determinants of financial managers’ willing-
ness to engage in unethical pro-organizational behavior. Journal of Management Accounting
Research 30 (2): 81–104.
Majerczyk, M., and T. F. Thomas. 2021. Superiors’ discretionary bonus pool allocations when
agents face disparate performance risk. Journal of Management Accounting Research 33 (2):
151–66. https://doi.org/10.2308/JMAR-19-040
Mangen, C. 2017. Implications of economic shocks for CEO performance evaluation. European
Accounting Review 26 (4): 629–50.
Massaro, M., J. Dumay, and J. Guthrie. 2016. On the shoulders of giants: Undertaking a struc-
tured literature review in accounting. Accounting, Auditing & Accountability Journal 29 (5):
767–801. https://doi.org/10.1108/AAAJ-01-2015-1939
Merchant, K. A., W. A. Van der Stede, T. W. Lin, and Z. Yu. 2011. Performance measurement
and incentive compensation: An empirical analysis and comparison of Chinese and Western
firms’ practices. European Accounting Review 20 (4): 639–67.
Moers, F. 2005. Discretion and bias in performance evaluation: The impact of diversity and sub-
jectivity. Accounting, Organizations and Society 30 (1): 67–80. https://doi.org/10.1016/j.aos.
2003.11.001
Murphy, K. J., and P. Oyer. 2003. Discretion in executive incentive compensation. Working
paper, University of Southern California.
Prendergast, C. 1999. The provision of incentives in firms. Journal of Economic Literature 37
(1): 7–63. https://doi.org/10.1257/jel.37.1.7
Prendergast, C., and R. H. Topel. 1993. Discretion and bias in performance evaluation. European
Economic Review 37 (2–3): 355–65. https://doi.org/10.1016/0014-2921(93)90024-5
Rajan, M. V., and S. Reichelstein. 2009. Objective versus subjective indicators of managerial per-
formance. The Accounting Review 84 (1): 209–37. https://doi.org/10.2308/accr.2009.84.1.209
Roussy, M., and A. Perron. 2018. New perspectives in internal audit research: A structured litera-
ture review. Accounting Perspectives 17 (3): 345–85.
Shields, M. D. 1997. Research in management accounting by North Americans in the 1990s.
Journal of Management Accounting Research 9: 3–61.
Ter Bogt, H. J., and R. W. Scapens. 2012. Performance management in universities: Effects of the tran-
sition to more quantitative measurement systems. European Accounting Review 21 (3): 451–97.
Voußem, L., S. Kramer, and U. Schäffer. 2016. Fairness perceptions of annual bonus payments:
The effects of subjective performance measures and the achievement of bonus targets. Man-
agement Accounting Research 30: 32–46. https://doi.org/10.1016/j.mar.2015.10.001
Woods, A. 2012. Subjective adjustments to objective performance measures: The influence of
prior performance. Accounting, Organizations and Society 37 (6): 403–25. https://doi.org/10.
1016/j.aos.2012.06.001
Zimmerman, J. L. 2015. The role of accounting in the twenty-first century firm. Accounting and
Business Research 45 (4): 485–509. https://doi.org/10.1080/00014788.2015.1035549
AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Subjectivity in Performance Evaluations A Review of The Literature

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Subjectivity in Performance Evaluations A Review of The Literature

Uploaded by

Copyright:

Available Formats

19113838, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/1911-3838.12273 by UNED, Wiley Online Library on [13/02/2023].

LA SUBJECTIVITÉ DANS LES ÉVALUATIONS DE RENDEMENT : ANALYSE

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021) pages 653–685 © CAAA/ACPC

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Panel A: Journal frequency organized by year

Journala 2009–2012 2013–2016 2017–2020 Total Total (%)

Panel B: Topic organized by year

Topic 2009–2012 2013–2016 2017–2020 Total Total (%)

Panel C: Research setting organized by year

Research setting 2009–2012 2013–2016 2017–2020 Total Total (%)

(The table is continued on the next page.)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Panel D: Theory organized by year

Theory 2009–2012 2013–2016 2017–2020 Total Total (%)

Panel E: Method organized by year

Method 2009–2012 2013–2016 2017–2020 Total Total (%)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Critique of Existing Literature and Future Research Opportunities

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Articles by cost and benefit

Panel A: Risk reduction

Paper Method Setting Topica Perspective Location

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Panel B: Reduction of perceived unfairness

(The table is continued on the next page.)

Panel B: Reduction of perceived unfairness

Panel C: Inducement of adaptive behaviors

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Panel D: Incentive distortion mitigation

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

de Castro (2017) Survey Other—service Other—Did not specify Employee N/A

Panel E: Intentional inaccurate assessments

(The table is continued on the next page.)

Panel E: Intentional inaccurate assessments

Panel F: Unintentional inaccurate assessments

(The table is continued on the next page.)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Panel F: Unintentional inaccurate assessments

Panel G: Inﬂuence activities

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Panel H: Uncertainty about measurement criteria

Notes: aDiscretionary allocations refer to ex post discretionary adjustments and allocations.

Reduction of Perceived Unfairness

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Critique of Existing Research and Future Research Opportunities

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Inducement of Adaptive Behaviors

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

incorporate noncontractible subjective information into their subjective bonus allocations

Critique of Existing Research and Future Research Opportunities

Incentive Distortion Mitigation

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

8. Although subjectivity can be introduced to encourage employees to work on difﬁcult-to-measure

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Critique of Existing Literature and Future Research Opportunities

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

Inaccurate Subjective Performance Evaluations

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021) pages 653–685 © CAAA/ACPC

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)

AP Vol. 20 No. 4 — PC vol. 20, n 4 (2021)