BEHAVIORAL RESEARCH IN ACCOUNTING American Accounting Association

Vol. 21, No. 2 DOI: 10.2308/bria.2009.21.2.37

2009
pp. 37–56

The Judgmental Effects of Management

Communications and a Fifth Balanced
Scorecard Category on Performance
Evaluation
Steven E. Kaplan
Arizona State University
Priscilla S. Wisner
Montana State University

ABSTRACT: This study extends prior balanced scorecard 共BSC兲 research by examin-
ing management’s choice of whether to include four or five BSC categories when a
specific nontraditional strategic objective is present. The results of the study show that
when management communications about a specific nontraditional strategic objective
are limited and a set of specific strategic performance measures indicate superior per-
formance by one manager over another, evaluators discount their reliance on these
measures in their performance-related judgments when these measures are clustered
into a fifth BSC category relative to when these measures are integrated among the
four traditional BSC categories. That is, adding a fifth BSC category resulted in a de-
emphasis of the relevant information about the specific nontraditional strategic objec-
tive. But, when a fifth BSC category is present, enhancing management communica-
tions about a specific strategic objective resulted in greater emphasis on the relevant
information about the specific nontraditional strategic objective. When a traditional,
four-category BSC is used, enhancing management communications about a specific
strategic objective did not result in greater emphasis on the relevant information about
the specific nontraditional strategic objective. Our results showing that under certain
conditions information presented in a fifth BSC category is discounted by evaluators
runs counter to professional literature and has practical implications for companies
considering the use of additional BSC categories to communicate strategic objectives.

Keywords: balanced scorecard; performance evaluations; nontraditional strategic ob-

jective.

Data Availability: Available upon request from the first author.

We thank Bryan Church 共editor兲, Marc Epstein, Kip Krumwiede, Steve Salterio, Janet Samuels, Naomi Soderstrom, two
anonymous reviewers, and workshop participants at the 2005 AAA Annual Meeting and the 2005 AAA Management
Accounting Section Annual Meeting for their helpful comments. We also thank Steve Salterio and Marlys Lipe for sharing
their experimental materials with us.

Published Online: July 2009

37
38 Kaplan and Wisner

INTRODUCTION
he balanced scorecard 共BSC兲 was developed by Kaplan and Norton 共1992, 1996a, 1996b兲,

T in part, for multidimensional performance measurement. A BSC typically contains a rela-

tively large set of measures that are classified or listed within broad performance catego-
ries. The clustering of subsets of measures as part of different performance categories is an
important aspect of designing a BSC. However, research examining the extent to which or the
conditions under which this structuring feature of the BSC influences performance evaluations is
limited 共Lipe and Salterio 2002; Kaplan et al. 2007兲. These studies provide evidence on the effect
of BSC structuring: that is, measures are presented and classified using the four traditional BSC
categories or are presented without classification. BSC research to date has not examined the
extent to which or the conditions under which adding a fifth category to a BSC influences perfor-
mance evaluations.
In this regard, while the traditional BSC contains four categories 共i.e., financial, customer-
related, internal business processes, and organizational learning and growth activities兲, Kaplan and
Norton 共1996a, 34兲 intended for the number of categories in the BSC to be discretionary, stating
that, “depending on industry circumstances and a business unit’s strategy, one or more additional
perspectives may be added.” Further, Maltz et al. 共2003兲 advocate using a fifth category to high-
light one or more specific strategic imperatives beyond the four traditional BSC categories. For
example, Maltz et al. 共2003兲 report that Best Foods added a “people development” perspective to
its traditional balanced scorecard to highlight the strategic importance of human resource man-
agement. Epstein and Wisner 共2001兲 discuss using a fifth BSC perspective to implement sustain-
ability as a strategic objective. According to Epstein and Wisner 共2001兲, adding a fifth perspective
for sustainability will emphasize the strategic importance of sustainability outcomes to managers
and employees of the organization, helping them to focus on meeting the sustainability objectives
of the organization that are communicated through the BSC measures. While academicians have
recommended that firms use additional BSC categories to highlight nontraditional strategic im-
peratives, and some organizations have expanded their BSCs to incorporate additional perspec-
tives, the contention that attention to performance measures about the nontraditional strategic
imperative will be enhanced by presenting these performance measures in an additional BSC
category has not been examined.
The purpose of the current study is to provide evidence on evaluators’ judgments in a setting
including a nontraditional strategic objective and performance measures related to a nontraditional
strategic objective.1 Our study builds upon the works of Epstein and Wisner 共2001兲 and Maltz et
al. 共2003兲 and creates a set of performance measures related to the specific strategic objective. In
this setting, we examine the effect of the BSC format and management communications on
evaluators’ judgments of two subordinate managers. Regarding BSC format, performance mea-
sures related to the strategic objective are either presented in an integrated fashion among the four
traditional BSC categories or clustered into a separate, fifth BSC category. Regarding management
communications, information about the rationale and importance of the strategic objective is either
relatively limited or enhanced.
We propose and test three hypotheses. We first test if clustering specific strategic performance
measures into a separate, fifth BSC category under a condition of limited management communi-
cations impacts evaluations of managerial performance. We then test the impact of the level of
management communications about a strategy on performance evaluations when specific strategic
performance measures are clustered into a fifth BSC category. Third, we test the impact of the

1
For ease of exposition, we refer to strategic nontraditional performance objectives or imperatives simply as strategic
performance objectives or imperatives.

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
Judgmental Effects of Management Communications and a Fifth Balanced Scorecard Category 39

level of management communications about a strategy on performance evaluations when specific

strategic performance measures are integrated across four BSC categories rather than clustered
into a separate, fifth category.
Understanding the potential impact of creating a fifth BSC category on subjective BSC evalu-
ations is important. Management has a range of discretionary choices related to its overall com-
munication processes which may be used to better align managerial actions with strategy 共Mar-
ginson 2002; Rapert et al. 2002; Simons 1994兲. Kaplan and Norton 共2001, 217兲 describe the BSC
as a means to create strategic awareness, stating that “companies must view the communication of
the scorecard to employees as a strategic campaign,” and further describe BSC feedback from
employees on BSC measures as a means to evaluate organizational alignment. Therefore, an
important area of research is to examine if the effect of management’s choice of a fifth BSC
category on performance evaluation-related judgments is influenced by the extensiveness of man-
agement communications. Furthermore, Kaplan and Norton 共2001兲 emphasize the importance of
linking employee incentive and reward systems to the BSC. To the extent that evaluators’ assess-
ments reflect systematic biases 共e.g., common measures兲 or other potential information processing
tendencies, evaluators, through their evaluations, may be providing subordinates with suboptimal
feedback. By examining evaluators’ judgments in light of management’s communication processes
regarding the BSC, this study is contributing evidence on managerial choices that may lead to or
mitigate misalignment in an organization.
We test our hypotheses using an experiment in which M.B.A. students evaluate the perfor-
mance of two plant managers, given background information about the firm and its strategy and
detailed BSC information about each plant manager. The evaluators’ task is to rate the perfor-
mance of two plant managers and to allocate a bonus based on background case information,
including the targets and actual performance outcomes for each BSC measure. In the experiment,
we use environmental performance as the specific strategic objective and identify four perfor-
mance measures related to this objective. We focus on environmental performance as it is increas-
ingly important to business 共Brown et al. 2005; Clarkson et al. 2004; KPMG International 2002兲;
Wisner et al. 2006兲 and many advocate incorporating environmental considerations into the BSC
共Dias-Sardinha and Reijnders 2005; Epstein and Wisner 2001; Figge et al. 2002; Moller and
Schaltegger 2005; van der Woerd and van den Brink 2004兲.
In our experimental setting, one plant manager’s performance on environmental outcomes—
the specific strategic objective—dominates the other plant manager’s performance. However, the
two plant managers perform similarly on all the other measures. Information about the targets and
actual performance outcomes are held constant in the experiment. This design feature allows us to
infer the relative subjective weights attached to environmental measures in assessing the perfor-
mance of the plant managers. The experiment manipulates management’s communications about
the strategic importance of environmental performance 共i.e., low versus high兲 and the BSC format
共i.e., four categories versus five categories兲 in a fully crossed design.
The rest of the paper is organized as follows. The next section reviews relevant literature and
presents our hypotheses. In the subsequent section we describe our research methods. In the next
two sections, we present the results of our study and then discuss the implications and limitations
of the study.

LITERATURE REVIEW AND DEVELOPMENT OF HYPOTHESES

The BSC generally involves identifying a set of performance measures that are related to and
drive strategy implementation, setting targets for these measures, and subsequently collecting
outcome results for the measures 共Kaplan and Norton 1992, 1996a兲. When explicit weights are not
assigned to each performance measure, evaluators combine the outcomes from the various perfor-

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
40 Kaplan and Wisner

mance measures into an overall judgment about performance using a subjective weighting process.
Several BSC studies have explored this subjective weighting process in managerial performance
evaluation 共Banker et al. 2004; Dilla and Steinbart 2005a, 2005b; Ittner et al. 2003; Kaplan et al.
2007; Libby et al. 2004; Liedtka et al. 2008; Lipe and Salterio 2000, 2002; Roberts et al. 2004;
Tayler 2007兲. These studies provide evidence on the effects of the contents of the BSC 共e.g.,
common versus unique performance measures兲, its structure 共e.g., formatted versus unformatted兲,
and related managerial communications about performance measures 共e.g., strategy maps兲. Gen-
erally, the task in BSC experimental research has been evaluating the performance of two man-
agers based, in part, on BSC information 共e.g., targets and outcomes across the multiple perfor-
mance measures兲.
Much of the experimental BSC research has been grounded in cognitive psychology, where
individuals are portrayed as having a limited information processing capacity. For example, re-
search findings indicate that individuals are able to process only approximately seven to nine
information cues simultaneously 共Miller, 1956兲. A broad range of research supports this view
共Baddeley 1994兲. Lipe and Salterio 共2002兲 applied this view to the BSC setting, concluding that
that the use of a large number of performance measures to evaluate managerial performance
represents a complex task. Specifically, Lipe and Salterio 共2002, 532兲 state that separately pro-
cessing, weighing, and combining a large number of performance measures into an overall evalu-
ation “is, cognitively, a very difficult thing to do.” This conclusion is consistent with research
showing a positive relationship between the number of cues or attributes and task complexity
共Bonner 1994; Wood 1986兲.
Lipe and Salterio 共2002兲 contend that when faced with a complex evaluation task, evaluators
invoke a “divide and conquer” information processing approach and that “the organization of the
BSC lends itself quite naturally to this kind of mental approach” 共Lipe and Salterio 2002, 533兲.
Under a divide and conquer approach, each of the performance outcomes is assigned to one of a
small number of groups. Next, performance is assessed for each group based upon the measures
that have been assigned to the group. Because each group contains a relatively small number of
measures, performance assessment at the group level represents a less cognitively demanding task.
Finally, separate group assessments are combined into an overall performance assessment.
Hence, Lipe and Salterio 共2002兲 contend that the format of the BSC is likely to influence the
perceived relations among performance measures in particular situations, such as when perfor-
mance measures within a category are consistently positive or negative, but not in other situations,
such as when performance measures across categories are consistently positive or negative. Based
on a divide and conquer approach, simultaneously processing a small number of measures that are
grouped together within a category should facilitate evaluators’ abilities to see relations among the
measures. Such perceptions, in turn, will influence the subjective weighting of these measures.
Specifically, the evaluator may weight two measures that are perceived to be related less than two
measures that are perceived to be unrelated, due to the perceived correlation, or nonindependence,
of the measures Lipe and Salterio 共2002兲, 533兲. Thus, when all the measures in a group are
consistently positive, these measures will be weighted less when grouped than when these mea-
sures are not grouped together.
In their Experiment One, Lipe and Salterio 共2002兲 present a scenario where the relative
performance of two managers in terms of targets for various performance measures was similar,
with the exception of measures in the customer-related category. For customer-related measures,
one manager performed substantially better than the targets, whereas the second manager per-
formed substantially worse. As expected, the difference between manager evaluations was signifi-
cantly smaller when the multiple above 共below兲 target measures were contained in a single BSC

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
Judgmental Effects of Management Communications and a Fifth Balanced Scorecard Category 41

category than when the measures were presented in an unformatted list, indicating that organizing
customer-related measures into a single BSC category moderated the effect of performance dif-
ferences on performance evaluation judgments.
In their Experiment Two, Lipe and Salterio 共2002兲 present a scenario where the relative
performance of two managers in terms of targets for various performance measures is similar with
the exception of four measures, one from each of the four BSC categories. These four measures,
which show the performance of one manager dominating the performance of the other manager,
are distributed across the four BSC categories. Since the measures are distributed, the structure of
the BSC is not expected to impact the subjective weighting of these or other measures. As
expected, the difference between manager evaluations was not significantly impacted by the pres-
ence or absence of the BSC format. The results from both experiments highlight the notion that
whether the structure or format of the BSC is expected to impact performance evaluation judg-
ments depends upon the specific pattern of performance outcomes.
More recently, Kaplan et al. 共2007兲 provide further evidence on whether the presence or
absence of the BSC format influences performance evaluation judgments. Specifically, they ex-
amine whether the display of information in a BSC format moderates the influence of subordinate
manager likeability when evaluating the performance of subordinate managers. The results show
that subordinate manager likeability influenced performance evaluations. However, the BSC for-
mat did not moderate the effect of subordinate manager likeability on managers’ performance
evaluations.
In contrast to previous research examining the presence or absence of a BSC format 共Lipe and
Salterio 2002; Kaplan et al. 2007兲, the current study examines whether an alternative BSC format
affects overall performance evaluations and bonus allocations. In the current study, when the BSC
contains five categories, the specific strategic measures are presented in a clustered fashion in a
separate, fifth category. Alternatively, when the BSC contains four categories, the specific strategic
measures are presented in an integrated fashion among the four traditional BSC categories. Fur-
ther, our study considers a specific pattern of outcomes. Similar to Lipe and Salterio 共2002兲
Experiment One, one manager’s performance on measures related to a single area 共the specific
strategic objective兲 dominates the other manager’s performance, but the two managers perform
similarly on all other performance measures. In this current setting, the BSC containing five
dimensions is roughly comparable to Lipe and Salterio 共2002兲’s placement of measures into a BSC
format in Experiment One. That is, when the BSC contains five categories, all the specific strategic
measures are clustered together into a single BSC category. This structure should enhance percep-
tions of the perceived relations among the environmental measures, and consequently, these mea-
sures may be discounted when making performance evaluations through performance ratings and
bonus allocations.
Alternatively, consider a BSC only containing the four traditional categories, such that one of
the specific strategic measures is included along with the other performance-related measures for
each of the four traditional BSC categories. In terms of expectations, this condition parallels Lipe
and Salterio 共2002兲 no format condition, in that the environmental measures are distributed across
the four traditional categories. Under the divide and conquer approach to processing information,
the specific strategic measure assigned to each traditional perspective will be used along with the
other performance-related measures clustered in the category to make a performance assessment
within the category. Because the four specific strategic measures are not clustered, the tendency to
perceive the specific strategic measures as related should be greatly lessened. Thus, the specific
strategic measures are expected to have a greater impact on evaluations of managerial perfor-
mance. That is, the difference between the overall performance evaluations 共and bonus awards兲 of
the two subordinate managers should be greater when the BSC contains four rather than five
categories.

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
42 Kaplan and Wisner

In both of Lipe and Salterio 共2002兲 experiments, management communication about the firm’s
strategic objectives was limited. While the case materials indicated that the specific performance
measures were carefully chosen to link to the firm’s strategy and mission, no attempt was made to
emphasize any particular strategic objective or to elaborate upon the importance of/rationale for
any of the strategic objectives. While Lipe and Salterio 共2002兲 do not explicitly discuss this design
choice, it is consistent with their orientation that evaluators should use a divide and conquer
approach. Our low level of management communications is roughly similar to the information
environment in Lipe and Salterio 共2002兲. This discussion leads to the following hypothesis:
H1: Under conditions of low communication about the importance of a specific strategic
objective and performance measures indicate that one manager dominates the perfor-
mance of the other manager on this specific strategic objective but not others, the extent
to which evaluators rely on performance measures related to the specific strategic ob-
jective will be lessened when the measures are clustered in a fifth BSC category, com-
pared with when the measures are distributed across the other BSC categories.
Communicating management’s vision and strategy to employees is critically important to its
successful implementation 共Kaplan and Norton 1992, 1996a, 1996b, 2001; Simons 1994兲. Kaplan
and Norton 共1996a, 2001兲 advocate communicating strategy throughout the organization. Specifi-
cally, Kaplan and Norton 共1996a, 203兲 state, “In general, we encourage companies to communi-
cate the objectives, measures, and targets embodied in the unit’s Balanced Scorecard … through-
out the organization.” More generally, regarding communications by management, Rapert et al.
共2002, 301兲 state, “A key task for top management is to consistently and accurately communicate
the strategic priority of the organization to functional-level managers.” Similarly, McDermott and
Boyer 共1999兲 contend that the effectiveness of implementing strategic goals will largely depend on
the extent to which middle and operational managers share top management’s understanding of the
strategic goals and priorities for the firm. Malina and Selto 共2001兲 provide initial evidence on the
importance of management’s strategy-related communications on the effectiveness of BSCs.
Kaplan and Norton 共1996a兲 discuss the importance of using communication processes to
create strategic awareness within an organization. Kaplan and Norton 共1996b, 2001兲 advocate
strategy maps as a communication device because the complexity of comprehensive performance
measurement systems such as the BSC makes it difficult for employees to easily understand
cause-and-effect relationships among and between performance measures. Banker et al. 共2004兲
provide evidence that strategy maps improve communications with evaluators by increasing their
subjective weighting of strategically linked performance measures and mitigating their biases to
overweight common measures 共Lipe and Salterio 共2002兲兲. This evidence indicates that strategy
maps effectively communicate the relative importance of specific strategically and nonstrategically
linked performance measures.
When a firm establishes an additional specific strategic objective, its future success depends
on managers appropriately attending to this objective and the performance measures related to the
specific strategic objective. Enhanced management communication may be helpful in implement-
ing management’s specific strategies and objectives as intended. That is, when management’s
strategy contains multiple components, communications may also be used to implicitly or explic-
itly indicate the relative importance of a specific strategic component. Such communications may
be characterized as a form of persuasion, in that management is disseminating information, in-
tending to better align the values and norms between themselves and lower level employees. In
this regard, communications discussing a particular strategic component are consistent with the

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
Judgmental Effects of Management Communications and a Fifth Balanced Scorecard Category 43

institutional perspective, which views organizational communications “as a process by which

norms are both adopted and acted on by organizational members” 共Lammers 2003, 620兲.2
We contend that in situations where measures related to a specific strategic objective might be
relatively underweighted, management should be able to elevate the importance of these measures
through organizational communications. Both professional and academic business communica-
tions literatures indicate that when managers communicate a strong justification for a strategic
objective to organizational members, they are implicitly signaling management’s commitment to
and strong support for the strategic objective. In turn, such purpose-based communications are
expected to garner support and acceptance for a strategic objective among organizational mem-
bers, especially in situations where it might otherwise be relatively weak 共Lewis et al. 2006兲.
Under H1, the impact of specific strategic performance measures clustered in a fifth BSC
category is expected to be lessened when management communications are limited. But, when the
BSC contains five categories, enhanced management communications, providing a clear under-
standing of the organizational purpose and centrality of the specific strategic objective, are ex-
pected to guide evaluators into subjectively assigning more weight to the measures related to the
specific strategic objective. This discussion leads to the following hypothesis:
H2: When the BSC includes a fifth category for a specific strategic objective and the perfor-
mance measures indicate that one manager dominates the performance of the other
manager on the specific strategic objective but not on others, providing information from
management as to the strategic importance of the specific strategic objective will in-
crease the extent to which evaluators rely on measures of the specific strategic objective
presented in the fifth category.
Hypothesis 2 predicts that management communications may be used to increase the relative
weighting of performance measures about a specific strategic objective in a setting where the
measures are expected to be relatively underweighted 共i.e., the BSC includes five categories兲.
Under H1, we do not expect specific strategic measures to be relatively underweighted when the
BSC includes four categories. Because we do not expect these measures to be discounted by the
evaluators, we do not expect that enhanced management communications will affect the relative
weighting of the specific strategic measures in the four-category BSC. This leads to the following
hypothesis:
H3: When the BSC includes four categories and performance measures indicate that one
manager dominates the performance of the other manager on the specific strategic ob-
jective but not on others, enhanced management communication about the strategic
importance of a specific strategic objective will not increase the extent to which evalu-
ators rely on these measures in making performance judgments.

METHOD
Overview and Case Description
Participants received case materials describing Metalfab, Inc. 共Metalfab兲, a manufacturing
firm specializing in industrial parts. The case background described that several years ago, finan-
cial and operational performance had been poor; operational problems involved both quality of
production and fines for environmental violations. The company subsequently developed a BSC to
help implement strategy and manage performance outcomes. The case materials included a brief

2
Alternatively, such communications may be characterized in term of salience 共Haynes and Kachelmeier 1998; Ng and
Tan 2007兲.

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
44 Kaplan and Wisner

description of the BSC concept and its development at Metalfab. Working together, Metalfab’s
management team, plant managers, and customers identified a set of five strategic objectives: cost
control, customer responsiveness, operational efficiency, employee development, and environmen-
tal performance. All case instruments indicated that management considered environmental per-
formance to be one of the five strategic objectives of the company.
We selected environmental performance as the specific strategic objective for this study be-
cause it is becoming increasingly important 共Brown et al. 2005; Clarkson et al. 2004; KPMG
International 2002; Wisner et al. 2006兲, especially among manufacturing firms. Firms devote
substantial resources to managing environmentally related issues, and organizational consequences
from failing to manage environmentally related issues can be significant 共Buysse and Verbeke
2003兲. Thus, many firms have adopted proactive approaches to managing environmentally related
issues 共Berry and Rondinelli 1998; Henriques and Sadorsky 1996兲, such that “a primary concern
of most businesses now is how to manage their environmental impacts effectively and efficiently”
共Berry and Rondinelli 1998, 38兲. As part of this proactive management approach, many advocate
incorporating environmental considerations into the BSC 共Dias-Sardinha and Reijnders 2005;
Epstein and Wisner 2001; Figge et al. 2002; Moller and Schaltegger 2005; van der Woerd and van
den Brink 2004兲.
The case materials also introduced and defined the BSC categories and the specific measures
and related targets used by Metalfab. All case instruments included a BSC containing the same 20
common performance measures, targets, and outcomes. The BSC presented performance data for
both plant managers side-by-side in a single table, organized by BSC category. Within each BSC
category, rows listed the specific measure, the target and actual performance for each plant man-
ager, and the percentage by which the actual performance was better or worse than the target.
Presenting the performance of both managers in a single table was intended to ease information
processing and facilitate comparisons between the two managers 共Dilla and Steinbart 2005b兲.
Based on Lipe and Salterio 共2002兲, the actual performance outcomes among the measures
were selected to provide the strongest chance for the BSC format 共i.e., four versus five categories兲
to affect managerial performance evaluations. Four of the 20 performance measures captured
environmental performance, one each related to the financial, customer, internal business process,
and learning and growth perspectives. On the four environmentally based performance measures,
plant manager 2’s actual performance exceeded the targets by an average of approximately 7.5
percent, and plant manager 1’s actual performance was below the targets by an average of about
4.5 percent. Thus, plant manager 2 dominated plant manager 1 in terms of performance on
environmentally based performance measures. The 16 other performance measures captured non-
environmental performance, four each from a financial, customer, internal business process, and
learning and growth perspective. Within each perspective, the performance against target for each
plant manager was balanced such that plant manager 1 outperformed plant manager 2 on some of
the performance measures and vice versa. The performance of each manager was balanced so that
the average variance within each perspective was equivalent for the two managers. Thus, the
average performance of the two managers was similar on the 16 non-environmental performance
measures.

Design, Independent Variables, and Dependent Variables

The experiment used a two-by-two between-participants design, with BSC format and man-
agement communications as manipulated variables. The BSC format displayed the targets and
outcomes for the 20 performance measures in either four or five categories. The four-category
BSC format contained the four traditional categories; each category included one of the environ-
mental performance measures as well as four other category-related measures. Under the five-

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
Judgmental Effects of Management Communications and a Fifth Balanced Scorecard Category 45

category format, all four environmental performance measures were clustered into a single BSC
category, and each of the four other BSC categories included only the four category-related
measures. Tables 1 and 2 present the BSC instrument under the four- and five-category formats,
respectively.

TABLE 1
The Balanced Scorecard Employed in the Experiment
Four Dimensions of Performance
Four Dimensions
Performance Data Plant 1 Plant 2
Percent Percent
Target Better/ Target Better/
Measures Actual (Worse) Actual (Worse)
Financial Production costs 共percent of 45.0% 45.5% ⫺1.1% 45.0% 44.8% 0.4%
sales兲
Indirect costs 共percent of 27.0% 26.2% 3.0% 27.0% 26.2% 3.0%
sales兲
Penalties and fines for 1.0% 1.03% ⫺3.0% 1.0% 0.95% 5.0%
discharges 共percent of sales兲
Scrap costs 共percent of sales兲 4.0% 3.82% 4.5% 4.0% 3.90% 2.5%
Rework costs 共percent of 2.0% 1.91% 4.5% 2.0% 1.90% 5.0%
sales兲
Customer Days to delivery/lead time 共# 18 17.8 1.1% 18 18.1 ⫺0.6%
days兲
On-time delivery 共percent兲 90.0% 88.4% ⫺1.8% 90.0% 91.9% 2.1%
Customer satisfaction rating 93.0% 94.0% 1.1% 93.0% 95.8% 3.0%
共0–100 percent兲
Packaging waste reduction 25.0% 24.0% ⫺4.0% 25.0% 27.0% 8.0%
共percent兲
Customer returns 共percent of 3.0% 3.0% 1.7% 3.0% 3.1% ⫺2.7%
sales兲
IBP Number of days of inventory 42 40.8 2.9% 42 40.2 4.3%
共days兲
Items in stock versus plan 95.0% 96.0% 1.1% 95.0% 96.0% 1.1%
共percent兲
Hazardous waste generated 15.0% 16.0% ⫺6.7% 15.0% 13.0% 13.3%
共percent兲
Defect rate 共percent good兲 85.0% 84.0% ⫺1.2% 85.0% 82.5% ⫺2.9%
Capacity utilization 共percent兲 84.0% 86.5% 3.0% 84.0% 86.8% 3.3%
L&G Employee retention 共percent兲 75.0% 74.1% ⫺1.2% 0.75 75.4% 0.5%
Training hours per employee 7.5 7.8 4.0% 7.5 7.8% 4.0%
Employee satisfaction survey 90.0% 92.2% 2.4% 90.0% 91.8% 2.0%
共percent兲
Environmental practices score 85.0% 81.0% ⫺4.7% 85.0% 88.0% 3.5%
共0–100 percent兲
Innovation survey score 70.0% 72.5% 3.6% 70.0% 71.5% 2.1%
共0–100 percent兲

a
Target amounts were identical for manager A and manager B.

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
46 Kaplan and Wisner

TABLE 2
The Balanced Scorecard Employed in the Experiment
Five Dimensions of Performance
Performance Data Plant 1 Plant 2
Percent Percent
Better/ Better/
Measures Target Actual (Worse) Target Actual (Worse)
Financial Production costs 共percent of 45.0% 45.5% ⫺1.1% 45.0% 44.8% 0.4%
sales兲
Indirect costs 共percent of 27.0% 26.2% 3.0% 27.0% 26.2% 3.0%
sales兲
Scrap costs 共percent of sales兲 4.0% 3.82% 4.5% 4.0% 3.90% 5.0%
Rework costs 共percent of 2.0% 1.91% 4.5% 2.0% 1.90% 5.0%
sales兲
Customer Days to delivery/lead time 共# 18 17.8 1.1% 18 18.1 ⫺0.6%
days兲
On-time delivery 共percent兲 90.0% 88.4% ⫺1.8% 90.0% 91.9% 2.1%
Customer satisfaction rating 93.0% 94.0% 1.1% 93.0% 95.8% 3.0%
共0–100 percent兲
Customer returns 共percent of 3.0% 3.0% 1.7% 3.0% 3.1% ⫺2.7%
seals兲
IBP Number of days of inventory 42 40.8 2.9% 42 40.2 4.3%
共days兲
Items in stock versus plan 95.0% 96.0% 1.1% 95.0% 96.0% 1.1%
共percent兲
Defect rate 共percent good兲 85.0% 84.0% ⫺1.2% 85.0% 82.5% ⫺2.9%
Capacity utilization 共percent兲 84.0% 86.5% 3.0% 84.0% 86.8% 3.3%
L&G Employee retention 共percent兲 75.0% 74.1% ⫺1.2% 75.0% 75.4% 0.5%
Training hours per employee 7.5 7.8 4.0% 7.5 7.8 4.0%
Employee satisfaction survey 90.0% 92.2% 2.4% 90.0% 91.8% 2.0%
共percent兲
Innovation survey score 70.0% 72.5% 3.6% 70.0% 71.5% 2.1%
共0–100 percent兲
Environmental Penalties and fines for 1.0% 1.03% ⫺3.0% 1.0% 0.95% 5.0%
discharges 共percent of sales兲
Packaging waste reduction 25.0% 24.0% ⫺4.0% 25.0% 27.0% 8.0%
共percent兲
Hazardous waste generated 15.0% 16.0% ⫺6.7% 15.0% 13.0% 13.3%
共percent兲
Environmental practices score 85.0% 81.0% ⫺4.7% 85.0% 88.0% 3.5%
共0–100 percent兲

a
Target amounts were identical for manager A and manager B.

Management communications were assigned either a high or a low level. Under the low
condition, communication by management about the relative importance of environmental perfor-
mance was limited. The case materials described the company’s previous environmental problems,
which had resulted in fines from governmental regulators and negative news stories. Under the
high condition, additional information conveyed that management placed a high importance on

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
Judgmental Effects of Management Communications and a Fifth Balanced Scorecard Category 47

environmental performance and also discussed the rationale for its importance. In addition to the
description of previous environmental problems included under the low condition, the high con-
dition case materials included the following:
Management was surprised by these events, as they are concerned with environmental perfor-
mance. They realized that they had not communicated the importance of environmental perfor-
mance throughout the company. Top management made environmental performance a key strategic
objective and made sure that all plant managers were informed. Further, plant managers were
required to attend training related to environmental issues … As these problems became public,
their largest automotive customers announced that environmental performance would be one of the
qualifying factors in awarding future purchasing contracts. That is, future contracts would be
awarded, in part, by how effectively suppliers managed their environmental performance, in addi-
tion to other concerns.
Following the case information, participants were instructed to evaluate the performance of
two plant managers; each plant was characterized as a cost center. Participants rated each manag-
er’s performance by providing an evaluation of each manager’s performance for each of the five
strategic objectives, as well as an overall performance rating. We included performance ratings for
each strategic objective because it is consistent with and facilitates use of a divide and conquer
approach to information processing. The order of these ratings was held constant. The rating of the
environmental objective was listed fifth, after the four traditional BSC categories.
For each evaluation, participants rated the manager’s performance by placing an X on a scale
that ranged from 0 共Reassign: Sufficient Improvement Unlikely兲 to 100 共Excellent: Far Beyond
Expectations兲. After evaluating both managers, participants allocated a $50,000 bonus between the
two plant managers. The inclusion of a bonus measure is consistent with several recent experi-
mental BSC studies 共Roberts et al. 2004; Dilla and Steinbart 2005a, 2005b兲. A bonus allocation
provides some evidence of the economic consequences in addition to the performance rating
measure. Therefore, while the performance rating captures judgments of performance, the bonus
allocation reflects the evaluators’ decisions regarding performance differences 共Maines et al.
2006兲.
Performance ratings and bonus allocations for the two plant managers, respectively, were used
for hypothesis testing. Specifically, the performance ratings dependent measure was constructed
for each participant by subtracting the rating assigned to plant manager 1 from the rating assigned
to plant manager 2. We refer to this measure as the performance rating difference score. A second
dependent measure, bonus allocation to plant manager 2, was based on the amount of the $50,000
bonus pool assigned to plant manager 2.

Participants and Experiment Procedures

M.B.A. students at a private university participated in the study. The case instrument was
randomly distributed to 210 students enrolled in a managerial accounting course. The instrument
was administered in class, subsequent to a two-class module on the BSC. Information about the
participants was collected from the last section of the instrument. In this section, participants
completed a debriefing questionnaire that collected demographic and other perceptual information
about the task. A small amount of participation credit 共5 bonus points out of a total of 500 class
points兲 was given for the assignment.
Three instruments were not sufficiently completed and were dropped. The remaining re-
sponses were screened to ensure that participants had a minimal level of understanding of the case.
One of the debriefing statements asked participants to indicate their level of agreement with the
following statement, “The two divisions used the same performance measures.” Because the case
emphasized the adoption of a common set of BSC measures 共e.g., performance data for the two
managers were presented side-by-side on a single table兲, 30 participants who did not agree with

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
48 Kaplan and Wisner

the statement to some extent were assumed to have an insufficient understanding of the case or did
not vigilantly process information. Consequently, these participants were also dropped.3 The final
sample contained 177 participants 共more than 84 percent of the initial sample兲.
Background information about these participants is presented in Table 3.4 As shown, the
majority of participants were male, from outside the United States, and had evaluation experience.
The mean age of participants was approximately 28.6 years, the mean years of work experience
was approximately 5 years, and the mean number of accounting classes was approximately 3.7.

TABLE 3
Participant Demographic Data

Gender Male Female

124 53
70% 30%
United
Country of Origin States Other
74 103
42% 58%
Evaluation Experience Yes No
116 61
66% 34%
Mean Std. Deviation
Age 28.6 4.0
Years of Work Experience 5.0 3.4
Number of Accounting 3.7 2.0
Courses
Agreement with Debriefing
Statements:a Mean Std. Deviation
Overall, I am quite familiar 5.0 1.2
with the concept of the
balanced scorecard.
I feel that I have a good 5.4 0.9
understanding of the
structure of the balanced
scorecard.

a
1 ⫽ completely disagree, 7 ⫽ completely agree.

3
We examined several issues related to participants dropped from the analysis. The proportion dropped was not signifi-
cantly associated with either independent variable or the interaction between the two independent variables. None of the
demographic variables 共e.g., gender, age, years of work experience, country of origin 关U.S. versus non-U.S.兴, or
evaluation experience兲 was significantly associated with whether a participant was dropped from the analysis.
4
We examined whether the assignment of participants to treatments was associated with the demographic variables 共e.g.,
gender, country of origin, age, years of work experience, evaluation experience兲. None of these demographic variables
was significantly associated with either independent variable or with the interaction between the two independent
variables.

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
Judgmental Effects of Management Communications and a Fifth Balanced Scorecard Category 49

Participants also indicated a reasonably high level of familiarity and understanding of the BSC.
Responses to the item about one’s understanding of the BSC did not significantly differ across
conditions, suggesting that randomization was effective.

RESULTS
Manipulation Checks
Two of the debriefing items relate to evaluators’ perceptions of the importance of strategic
environmental performance to the organization. One item stated, “Environmental performance is
an important strategic objective for Metalfab” 共environmental importance兲. A second item stated,
“The company has a strong commitment to environmental performance” 共environmental commit-
ment兲. Table 4, Panel A presents descriptive statistics for these two measures.
If the manipulation of management communications was interpreted as intended, then the
mean scores for environmental importance and commitment should be significantly higher under
a high level of management communications. As shown in Table 4, Panel A, the means are higher
under a high level of management communications 共6.0 and 4.9, respectively兲 than under a low
level of management communications 共5.2 and 4.4, respectively兲. Further, the results from analysis
of variance 共ANOVA兲 indicate that management communications is significant 共F ⫽ 14.50, p ⬍
0.01兲 and marginally significant 共F ⫽ 3.4, p ⬍ 0.06兲, respectively, for the environmental impor-
tance and commitment items.5
An additional debriefing item captured perceived emphasis on environmental performance
measures. The item stated, “Environmental performance measures were emphasized in this case.”
Descriptive statistics for this item are reported in Table 4, Panel B. The mean score for this item
was higher when the BSC contained five 共5.4兲 rather than four 共4.8兲 dimensions. ANOVA results
using this item as the dependent measure indicate that BSC categories is significant 共F ⫽ 6.02, p
⬍ 0.02兲 and that neither management communications nor the BSC categories by management
communications interaction is significant.
Overall, the results of the manipulation checks indicate that the manipulated variables were
attended to. The manipulation of management communications was associated with differential
perceptions about environmental importance and the firm’s environmental commitment. The ma-
nipulation of the number of BSC categories was associated with differential perceptions about the
emphasis placed on environmental performance measures.

Tests of Hypotheses
Table 5 reports descriptive statistics for participants’ performance ratings for plant manager 1,
plant manager 2, the rating difference score, and the bonus allocated to plant manager 2. All
hypotheses are tested using ANOVA.
In the context of the study, H1 predicts that when management communications are low, the
extent to which evaluators rely on environmental performance measures for performance
evaluation-related judgments will be lessened when these measures are clustered into a separate,
fifth category compared with when these measures are distributed across the four other BSC
categories. As stated, H1 is restricted to cells 1 and 2 shown in Table 5. BSC format is the
independent variable, and the rating difference score and bonus allocations to plant manager 2 are
the dependent measures. Using the rating difference score as the dependent measure, the ANOVA
results show a significant main effect for BSC category 共F ⫽ 10.7, p ⬍ 0.01兲. Using the bonus

5
In addition, the results of the ANOVAs indicate that neither BSC format nor the management communication by BSC
format interaction is significantly associated with either the environmental importance or the environmental commitment
dependent measures.

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
50 Kaplan and Wisner

TABLE 4
Descriptive Statistics on Manipulation Check Measures
Mean and (Standard Deviation)
Panel A: Descriptive Statistics on Environmental Importance and Environmental Commitment
Management BSC Environmental Environmental
Cell Communications Categories Importancea Commitmentb
1 Low Four 5.3 4.4
共n ⫽ 45兲 共1.5兲 共1.6兲
2 Low Five 5.0 4.4
共n ⫽ 44兲 共1.6兲 共1.7兲

Total Low 5.2 4.4

共1.6兲 共1.7兲
3 High Four 5.9 4.5
共n ⫽ 47兲 共1.5兲 共1.8兲
4 High Five 6.0 5.2
共n ⫽ 41兲 共1.1兲 共1.4兲

Total High 6.0 4.9

共1.4兲 共1.6兲

Panel B: Descriptive Statistics on Emphasis on Environmental Measures

Emphasis on
BSC Management Environmental
Cell Categories Communications Measuresc
1 Four Low 4.8
共n ⫽ 45兲 共1.7兲
3 Four High 4.8
共n ⫽ 47兲 共1.8兲

Total Four 4.8

共1.8兲
2 Five Low 5.2
共n ⫽ 44兲 共1.5兲
4 Five High 5.6
共n ⫽ 41兲 共1.4兲

Total Five 5.4

共1.5兲

a
Environmental performance is an important strategic objective for Metalfab.
b
This company has a strong commitment to environmental performance.
c
The environmental performance measures were emphasized in this case.
Response scale for each item: 1 ⫽ completely disagree and 7 ⫽ completely agree.

assigned to plant manager 2, the ANOVA results also show a significant main effect for BSC
category 共F ⫽ 6.16, p ⬍ 0.02兲. Further, consistent with H1, the rating difference score and the

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
Judgmental Effects of Management Communications and a Fifth Balanced Scorecard Category 51

TABLE 5
Means (Standard Deviations) for Performance Ratings and Bonus Allocations by Cell
Bonus
Plant Allocated
Hypothesis Manager Rating to Plant
Management BSC Difference Manager
1 2 3 Cell Communication Categories 1 2 Score 2
冑 冑 1 Low Four 83.7 59.8 23.8 $33,108
共n ⫽ 45兲 共52.3兲 共38.6兲 共21.0兲 共$8,383兲
冑 冑 2 Low Five 75.8 62.9 12.8 $29,416
共n ⫽ 44兲 共8.7兲 共9.3兲 共7.7兲 共$5,267兲
冑 3 High Four 76.6 54.8 21.8 $32,328
共n ⫽ 47兲 共11.3兲 共11.0兲 共14.8兲 共$6,886兲
冑 4 High Five 77.1 56.5 20.6 $31,951
共n ⫽ 41兲 共8.1兲 共12.4兲 共9.8兲 共$7,871兲

bonus assigned to plant manager 2 were each smaller in the low-communications and five-BSC-
category cell 共12.8 and $29,416, respectively兲 than in the low-communications and four-BSC-
category cell 共23.8 and $33,108, respectively兲.
In the context of the study, H2 predicts that when the BSC includes five categories, the extent
to which evaluators rely on environmental performance measures for performance evaluation-
related judgments will be strengthened by providing information about the strategic importance of
environmental performance compared with when this information is not provided. As stated, H2 is
restricted to cells 2 and 4 shown in Table 5. Management communications is the independent
variable, and rating difference scores and bonus allocations to plant manager 2 are the dependent
measures. Using the rating difference scores as the dependent measure, the ANOVA results show
a significant main effect for management communications 共F ⫽ 16.84, p ⬍ 0.01兲. Using the bonus
assigned to plant manager 2, the ANOVA results also show a marginally significant main effect for
management communications 共F ⫽ 3.09, p ⬍ 0.09兲. Also, consistent with H2, the rating difference
score and the bonus assigned to plant manager 2 were lower in the low-communications and
five-BSC-category cell 共12.8 and $29,416, respectively兲 than in the high-communications and
five-BSC-category cell 共20.6 and $31,951, respectively兲.
In the context of the experiment, H3 predicts that when the BSC includes four categories, the
extent to which evaluators rely on environmental performance measures for performance
evaluation-related judgments will not be strengthened by providing information about the strategic
importance of environmental performance. As stated, H3 is restricted to cells 1 and 3 shown in
Table 5. Using the rating difference scores as the dependent measure, the ANOVA results show an
insignificant main effect for management communications 共F ⫽ 0.30, p ⬎ 0.55兲. Using the bonus
assigned to plant manager 2, the ANOVA results also show an insignificant main effect for
management communications 共F ⫽ 0.24, p ⬎ 0.60兲. Also, consistent with H3, the rating difference
score means and the bonus means shown in Table 5 are very similar in cells 1 and 3.

DISCUSSION
Kaplan and Norton 共1992兲 introduced the BSC, in part, for multidimensional performance
measurement and evaluation. While their descriptions and discussions of the BSC primarily are in
the context of four categories, Kaplan and Norton 共1996a, 1996b兲 view the number of BSC

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
52 Kaplan and Wisner

categories as discretionary and that one or more additional categories may be added to the BSC to
incorporate specific strategic objectives. While researchers have examined a variety of behavioral
issues related to evaluators’ performance evaluations when BSCs are used, research has not ex-
plored the effect of the number of BSC categories when management supplements the traditional
four objectives with one or more additional specific strategic objectives. This study demonstrates
that, under certain conditions, the use of an additional BSC category can actually result in a
de-emphasis of relevant information about the additional strategic objective. However, when a
fifth BSC category is used, enhanced management communications are shown to be useful in
focusing evaluators’ attention to relevant information about the additional strategic objective.
Evidence on this issue should be helpful to managers when developing BSC systems within
settings where the firm’s strategic objectives and related measures extend beyond those normally
contained in the four BSC categories.
Before discussing the results of the study, several limitations should be noted. First, the study
focused on a setting in which the performance of one plant manager dominated the performance of
a second plant manager on environmental performance measures, the specific strategic objective
examined in this study. Based upon the work of Lipe and Salterio 共2002兲, we felt this was the most
appropriate setting for testing our hypotheses; however, we recognize that the results are not
generalizable to settings in which two plant managers perform similarly on strategic performance
measures. We encourage further research to examine alternative performance scenarios.
Second, the evaluation processes used to evaluate each plant manager’s performance in the
study may have influenced evaluators’ judgments. In evaluating the performance of each plant
manager, participants were asked to assess performance for each of the five strategic objectives
prior to making an overall performance rating. Generally, prior BSC research has not included
ratings for individual strategic objectives. We included these five initial ratings to be consistent
with and facilitate use of a divide and conquer approach to information processing 共Lipe and
Salterio 2002兲. While these five ratings aligned with the five-category BSC instrument, they
provided an additional rating category for the four-category BSC instrument. We included perfor-
mance ratings for each strategic objective in order to facilitate a divide and conquer approach to
information processing. This formatting decision may have also facilitated support for H1. It is
possible that evaluators’ overall performance ratings and bonus allocations might be different had
they not initially rated performance for each strategic objective, or if we had formatted the
instruments in an alternative manner. Relatedly, all instruments listed the specific strategic objec-
tive category last on the evaluators’ rating sheet, which also may have influenced evaluators to
discount its importance. Again, we encourage further research to examine these issues.
Third, the study focused on the effect of differences in environmental performance measures
on performance evaluation judgments. In order to present a BSC that could be formatted using
four or five categories, we needed to identify a particular “specific strategic objective” and related
performance measures. We selected environmental performance measures because performance in
this area is becoming increasingly important for many companies. However, firms may have
“specific strategic objectives” other than environmental performance, such as “people develop-
ment” 共Maltz et al. 2003兲. While we have no reason to expect that our results would not generalize
to other “specific strategic objectives,” we encourage further research on this issue.
Fourth, building on the work of Lipe and Salterio 共2002兲, H1 is grounded in an information
processing perspective. Similar to Lipe and Salterio 共2002兲, our study did not incorporate process
measures to verify this perspective. Our results are consistent with this information processing
perspective; however, we cannot rule out the possibility that our results are driven by some
competing explanation. Incorporating process measures would help to distinguish among various
potential explanations. Consequently, we believe such process measures would be informative,
and we encourage future research to explore this issue.

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
Judgmental Effects of Management Communications and a Fifth Balanced Scorecard Category 53

Fifth, participants in the study were M.B.A. students. Although participants generally had
substantial professional work experience and the majority had previously evaluated the work of
others, they may have limited experience using the BSC as an evaluation tool. Also, users with
more limited BSC experience may tend to rely more on the traditional four BSC categories, with
which they would be more familiar. As in previous BSC studies 共Banker et al. 2004; Libby et al.
2004; Lipe and Salterio 2000, 2002; Roberts et al. 2004兲, our participants were college students
with roughly equivalent backgrounds. However, more than half of our student participants were
from countries outside of the U.S. Although all are fluent in English and many were educated in
U.S. undergraduate institutions, these students may not have been as comfortable with terminol-
ogy in the case instrument or with the BSC format. This may have contributed to the 16 percent
drop rate, which was primarily due to students who did not sufficiently understand the case. It is
possible that our results may not generalize to evaluators who possess greater experience with and
knowledge about the BSC. In this regard, Dilla and Steinbart’s 共2005a兲 findings suggest that
knowledge about the BSC tended to reduce the common measures bias. We encourage further
research to examine the extent to which our findings hold among evaluators with greater experi-
ence and familiarity with the BSC.
Turning to a discussion of our results, for H1, the results indicate that when management
communications about the strategic importance of environmental issues are limited and managers
perform differently with respect to environmental issues, environmentally related performance
measures were subjectively weighted less when clustered into a separate, fifth BSC category
compared with when these measures were integrated among the four other BSC categories. The
findings for H1 are consistent with and extend the research initiated by Lipe and Salterio 共2002兲.
Where Lipe and Salterio 共2002兲 examined the effect of presence or absence of a BSC format on
evaluation judgments, the current study examines the effect of different BSC formats on evalua-
tion 共and bonus兲 judgments. The findings for H1 suggest that when managerial communications
are limited, use of a fifth BSC category may lessen the impact of the related performance measures
on key judgments such as performance evaluations and bonus allocations.
For H2, the results indicate that when the BSC contains five categories, environmentally
related performance measures were subjectively weighted more when communications about the
strategic importance of environmental issues were elevated compared with when such communi-
cations were limited. The findings for H2 indicate that when firms decide to include separate
categories in their BSCs for additional specific strategic objectives, managers may use communi-
cations about these objectives to influence and increase the subjective weighting of the related
performance measures on key judgments such as performance evaluations and bonus allocations.
Last, the results from H3 indicate that when the BSC includes four categories, the extent of
management communications about the strategic importance of environmental issues did not sig-
nificantly influence the subjective weighting for environmentally related performance measures
with regard to either performance evaluations or bonus allocations.
Overall, the results of this study show that the effects of the number of BSC categories on
performance evaluation-related judgments tend to depend, in part, on the level of management
communications. From a theoretical orientation, our results provide further evidence on the im-
portance of information processing considerations in understanding performance evaluation-
related judgments. However, our results also indicate that management communications represent
a boundary condition related to the effects of information processing considerations. Specifically,
our results indicate that when management communications were more extensive, the number of
BSC categories did not significantly affect performance-related judgments, as they did when
management communications were limited. This suggests that findings from previous BSC studies
in which management’s communications were limited may not generalize to settings where man-
agement communications are more extensive. Thus, further experimental research on the BSC

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
54 Kaplan and Wisner

should carefully consider the amount and type of management communications to include. Addi-
tionally, our results may be used to suggest that further research should be directed at gaining a
better understanding of the form and extent of actual management communication that takes place
in BSC firms.
From a practice orientation, our results suggest that when companies are implementing or
using a BSC, they must carefully consider not only which performance measures to use, but also
how these measures will be formatted within a BSC and within the context of management
communications about the strategic objectives and initiatives of the organization. Also, our finding
that use of a fifth BSC category to cluster information about a nontraditional strategic objective
may result in de-emphasis of relevant information is counter to existing professional literature
共Epstein and Wisner 2001; Maltz et al. 2003兲. Although the structure of the BSC allows for the
addition of additional categories, and although companies using the BSC currently modify the
BSC structure by adding additional categories of performance, our results demonstrate that under
the conditions of our experimental setting, performance measures clustered into a separate, fifth
BSC category were discounted when evaluators made performance-related judgments. Our results
suggest that managers may help organizational employees better understand and attend to the
relative importance of specific strategic objectives through carefully managing their communica-
tions about specific strategic objectives and the format of the BSC.

REFERENCES
Baddeley, A. 1994. The magical number seven: Still magic after all these years? Psychological Review
共April兲: 353–356.
Banker, R. D., H. Chang, and M. J. Pizzini. 2004. The balanced scorecard: Judgmental effects of performance
measures linked to strategy. The Accounting Review 79 共1兲: 1–23.
Berry, M. A., and D. A. Rondinelli. 1998. Proactive corporate environment management: A new industrial
revolution. Academy of Management Executive 12 共2兲: 38–50.
Bonner, S. E. 1994. A model of the effects of audit task complexity. Accounting, Organizations and Society
19: 213–234.
Brown, D. L., J. F. Dillard, and R. S. Marshall. 2005. Strategically informed, environmentally conscious
information requirements for accounting information systems. Journal of Information Systems 19 共2兲:
79–103.
Buysse, K., and A. Verbeke. 2003. Proactive environmental strategies: A stakeholder management perspec-
tive. Strategic Management Journal 24: 453–470.
Clarkson, P. M., Y. Li, and G. D. Richardson. 2004. The market valuation of environmental capital expen-
ditures by pulp and paper companies. The Accounting Review 79 共2兲: 329–353.
Dias-Sardinha, I., and L. Reijnders. 2005. Evaluating environmental and social performance of large Portu-
guese companies: A balanced scorecard approach. Business Strategy and the Environment 14: 73–91.
Dilla, W. N., and P. J. Steinbart. 2005a. Relative weighting of common and unique balanced scorecard
measures by knowledgeable decision makers. Behavioral Research in Accounting 17: 43–53.
——–, and ——–. 2005b. The effects of alternative supplementary display formats on balanced scorecard
judgments. International Journal of Accounting Information Systems 6: 159–176.
Epstein, M. J., and P. S. Wisner. 2001. Using a balanced scorecard to implement sustainability. Environmen-
tal Quality Management 11 共Winter兲: 1–10.
Figge, F., T. Hahn, S. Schaltegger, and M. Wagner. 2002. The sustainability balanced scorecard—Linking
sustainability management to business strategy. Business Strategy and the Environment 11: 269–284.
Haynes, C. M., and S. J. Kachelmeier. 1998. The effects of accounting contexts on accounting decisions: A
synthesis of cognitive and economic perspectives in accounting experimentation. Journal of Account-
ing Literature 17: 97–136.
Henriques, I., and P. Sadorsky. 1996. The determinants of an environmentally responsive firm: An empirical
approach. Journal of Environmental Economics and Management 30: 381–395.

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
Judgmental Effects of Management Communications and a Fifth Balanced Scorecard Category 55

Ittner, C. D., D. F. Larcker, and M. W. Meyer. 2003. Subjectivity and weighting of performance measures:
Evidence from a balanced scorecard. The Accounting Review 78: 725–758.
Kaplan, R., and D. Norton. 1992. The balanced scorecard—Measures that drive performance. Harvard
Business Review 70 共1兲: 71–79.
——–, and ——–. 1996a. The Balanced Scorecard. Boston, MA: Harvard Business School Press.
——–, and ——–. 1996b. Using the balanced scorecard as a strategic management system. Harvard Business
Review 共January-February兲: 75–85.
——–, and ——–. 2001. Transforming the balanced scorecard from performance measurement to strategic
management: Part II. Accounting Horizons 共June兲: 147–160.
Kaplan, S. E., M. Petersen, and J. A. Samuels. 2007. Effects of subordinate likeability and balanced scorecard
format on performance-related judgments. Advances in Accounting 23: 85–111.
KPMG International. 2002. KPMG International Survey of Corporate Sustainability Reporting 2002. De
Meern, The Netherlands: KPMG Global Sustainability Services.
Lammers, J. C. 2003. An institutional perspective on communicating corporate responsibility. Management
Communication Quarterly 16 共4兲: 618–624.
Lewis, L. K., A. M. Schmisseur, K. K. Stephens, and K. E. Weir. 2006. Advice on communicating during
organizational change: The content of popular press books. Journal of Business Communication 43
共2兲: 113–137.
Libby, T., S. Salterio, and A. Webb. 2004. The balanced scorecard: The effects of assurance and process
accountability on managerial judgment. The Accounting Review 79 共4兲: 1075–1094.
Liedtka, S. L., B. K. Church, and M. R. Ray. 2008. Performance variability, ambiguity intolerance, and
balanced scorecard-based performance assessments. Behavioral Research in Accounting 20 共2兲: 73–
88.
Lipe, M. G., and S. Salterio. 2000. The balanced scorecard: Judgmental effects of common and unique
performance measures. The Accounting Review 75 共3兲: 283–298.
——–, and ——–. 2002. A note on the judgmental effects of the balanced scorecard’s information organiza-
tion. Accounting, Organizations and Society 27: 531–540.
Maines, L. A., G. L. Salamon, and G. B. Sprinkle. 2006. An information economic perspective on experi-
mental research in accounting. Behavioral Research in Accounting 18: 85–103.
Malina, M. A., and F. Selto. 2001. Communicating and controlling strategy: An empirical study of the
effectiveness of the balanced scorecard. Journal of Management Accounting Research 13: 47–90.
Maltz, A. C., A. J. Shenhar, and R. Reilly. 2003. Beyond the balanced scorecard: Refining the search for
organizational success measures. Long Range Planning 36: 187–204.
Marginson, D. E. W. 2002. Management control systems and their effects on strategy formation at middle-
management levels: Evidence from a U.K. organization. Strategic Management Journal 23 共11兲:
1019–1031.
McDermott, C., and K. K. Boyer. 1999. Strategic consensus: Marching to the beat of a different drummer.
Business Horizons 42 共4兲: 21–28.
Miller, G. 1956. The magical number seven, plus or minus two: Some limits on our capacity for information
processing. Psychological Review 共March兲: 81–96.
Moller, A., and S. Schaltegger. 2005. The sustainability balanced scorecard as a framework for eco-efficiency
analysis. Journal of Industrial Ecology 9 共4兲: 73–83.
Ng, T. B-P., and H-T. Tan. 2007. Effects of qualitative factor salience, expressed client concern, and quali-
tative materiality thresholds on auditors’ audit adjustment decisions. Contemporary Accounting Re-
search 24 共4兲: 1171–1192.
Rapert, M. I., A. Velliquette, and J. A. Garretson. 2002. The strategic implementation process: Evoking
strategic consensus through communication. Journal of Business Research 55: 301–310.
Roberts, M. L., T. L. Albright, and A. R. Hibbets. 2004. Debiasing balanced scorecard evaluations. Behav-
ioral Research in Accounting 16: 75–88.
Simons, R. 1994. How top managers use control systems as levers of strategic renewal. Strategic Manage-
ment Journal 12: 49–62.
Tayler, W. B. 2007. The balanced scorecard as a strategy-evaluation tool: The effects of responsibility and
causal-chain focus. Working paper, Emory University.

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association
56 Kaplan and Wisner

van der Woerd, F., and T. van den Brink. 2004. Feasibility of a responsive business scorecard—A pilot study.
Journal of Business Ethics 55: 173–186.
Wisner, P. S., M. J. Esptein, and R. Bagozzi. 2006. Organizational antecedents and consequences of envi-
ronmental performance. In Advances in Environmental Accounting and Management, Vol. 3, edited by
M. Freedman and B. Jaggi, 146–167. Bingley, U.K.: Emerald.
Wood, R. E. 1986. Task complexity: Definition of the construct. Organizational Behavior and Human De-
cision Processes 37: 60–82.

Behavioral Research In Accounting, Volume 21, Number 2, 2009

American Accounting Association