Comunicare Organizationala

On the Road to Improved Performance: Changing Organizational Communication through
Performance Management
Author(s): Julia Melkers
Source: Public Performance & Management Review, Vol. 30, No. 1 (Sep., 2006), pp. 73-95
Published by: Taylor & Francis, Ltd.
Stable URL: https://www.jstor.org/stable/20447617
Accessed: 12-11-2018 13:32 UTC
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms
Taylor & Francis, Ltd. is collaborating with JSTOR to digitize, preserve and extend access to
Public Performance & Management Review
This content downloaded from 37.120.250.134 on Mon, 12 Nov 2018 13:32:55 UTC
All use subject to https://about.jstor.org/terms
ON THE ROAD TO
IMPROVED PERFORMANCE
Changing Organizational Communication
Through Performance Management
JULIA MELKERS
University of Illinois at Chicago
ABSTRACT: The rationale behind performance measurement requirements

at all levels of government has to do with changing the way budget decisions
are made. Supporters of performance measurement requirements argue that
the use of performance measures will enhance, or even change, the way that
budget discussions and subsequent decisions are made. This research addresses
the following questions: How does the implementation and use of performance
measures change the way that actors in the budget process communicate?
Do performance measurement activities change traditional relations and
communication patterns? Further, does communication with the public also
change? This paper presents datafrom a recent national survey of state and local
governments. Findings show that the implementation of performance measures
and the use of performance data in the budget process change the perceptions
and actual communication patterns of policymakers, budgeters, and agency
and department staff as well as the media and public. Comparisons are offered
among levels of government, as well as across agency types and budget actors.
An explanatory model is presented addressing the relation between organizational
factors and communication outcomes. This research is important in furthering our
understanding of how budget decision processes are affected by the implementation
of performance measurement requirements and processes.
KEYWORDS: budget process, organizational communication, performance

data, performance measurement
More than a decade has passed since the widespread implementation of perfor
mance measurement requirements across all levels of government in the United
States-from the Government Performance and Results Act of 1993 at the federal
level to a range of similar requirements in most state and many local governments
(Berman & Wang, 2000; Melkers & Willoughby, 1998; Poister & Streib, 1999).
Public Performance & Management Review, Vol. 30, No. 1, September 2006, pp. 73-95.
?) 2006 M.E. Sharpe, Inc. All rights reserved.
1530-9576/2006 $9.50 + 0.00.
DOI 10.2753/PMR1530-9576300104 73
74 PPMR / September 2006
But, to what end? Research shows that performance-based budgeting initiatives

in the states have had some success over the last few years, particularly in the
management of programs but less dramatically in changing actual appropriations
(Broom & McGuire, 1995; Joyce, 1997; Willoughby & Melkers, 2000). Although it
seems intuitive that aspects of performance will be included in decisions about the
allocation of resources, we know that this is often not the case. Although initially
it may be troublesome that performance measurement use has been limited, ad
ditional findings on the impact of performance measurement use in governments
is, in fact, more in line with the larger rationale of today's public management
reforms. Performance measurement has emerged as a communications tool in
government, demonstrating a change in the culture of organizations and aspects of
organizational learning through the adoption of performance measurement reform.
This cultural shift lies at the heart of what is necessary to succeed in this reform
(Hennessey, 1998). Yet, the implementation of such a comprehensive reform that
requires widespread systemic change takes time (Berman, 1998). Observing the
changes in policymaker and agency staff communication behavior and perceptions
of communication can provide evidence of system maturation.
This research addresses the following questions: How does the implementa
tion and use of performance measures change the way that actors in the budget
process communicate? Do performance measurement activities change traditional
relations and communication patterns? Further, does communication with the
public also change? To address these questions, this article presents data from a
recent national survey of state and local government budget actors and agency
and department administrators. The results help to extend our understanding of
the effects of performance measurement within the governmental setting.
Performance Measurement and Communication
Since the era of government reinvention, the emphasis in public management re

form efforts has been not only on efficiency and improved spending and allocation
decisions but also on quality improvement and innovative approaches to manag
ing programs and organizations (Carnevale & Carnevale, 1993; DuPont-Morales
& Harris, 1994). Performance measurement does not stand alone. Instead, it is
linked to larger organizational processes, such as managing for results or strate
gic planning-both comprehensive processes for focusing an organization on its
mission, goals, and objectives and related outcomes. An important cornerstone
of these processes involves coordination and communication both within and
The author thanks the Alfred P. Sloan Foundation and the Government Accounting
Standards Board for their support of this work. She also thanks Katherine Willoughby
and the anonymous reviewers of this work.
Melkers / ON THE ROAD TO IMPROVED PERFORMANCE 75
across organizations. Here, the improvement of results is accomplished through

the development of performance measures but also through innovative manage
ment practices, enhanced communication, and participation of administrators and
staff and organizational change. This communication effect also demonstrates the
needed cultural shift necessary to move the performance measurement forward
(Berman, 2002).
Critiques of performance measurement question its value as a communication
enhancement without corresponding concrete changes in allocations and program
matic components. Yet, do these communication benefits make a large enough
difference? Scholars of organizational communication processes acknowledge
costs and benefits to information exchange:
The question is not whether knowledge can be transferred, but at what cost it can
be transferred, and whether it is worth it to do so. Transfers yield benefits when the
additional knowledge enables the decision-maker to make better choices. The issue
is whether decisions will be improved enough to warrant the transfer costs. (Jensen
and Meckling, 1992, p. 258)
The question in regard to performance measurement, then, is whether budget

and other decisions warrant the costs of performance measurement implemen
tation. For example, Hatry (1999) emphasized the critical role of performance
measurement in any managing-for-results process and also pointed to the use of
performance measures as going beyond budget justification and cost issues to
management improvement and monitoring of program success or problems. As
more states and local governments adopt and use performance measures in man
agement and budget processes, traditional relations and communication patterns
adapt to this new system. Kettl noted, "Performance management and reporting
is a process of 'communication' and not simply measurement that 'occurs within
a broader political process,' in which the players have a wide array of different
incentives" (1994, p. 45).
Certainly, the rationale behind performance measurement requirements at all
levels of government has to do with changing the way budget and other manage
ment decisions are made. Supporters of performance measurement requirements
argue that the use of performance measures will enhance, or even change, the
way that budget discussions and subsequent decisions are made. The expectation
is that performance-based data will contribute more rational and, one hopes, bet
ter information, to provide policy and budget actors with the tools they need to
make better performance-based decisions. Here, performance measurement data
are only a benefit due to their content. This argument is consistent with studies
of content-related knowledge in which a key role of organizations is to collect,
maintain, and disseminate data and information over time (Cohen, 1998; Knock
& McQueen, 1998; Zack, 1999).
However, a focus only on the content (and related quality) of the informa
tion provided in performance measurement systems ignores the dynamics of the

conversation in which the information is presented. For example, a recent U.S.
Government Accountability Office (GAO) report described the use of performance
measurement information as not directly linked to budget decisions, but "instead,
it helped to inform budget deliberations by highlighting problems, supporting
claims, or enriching the debate" (2005, p. 10). It is this debate and discussion
that suggests that performance measurement is maturing and integrating itself
into organizational processes. Performance measurement seems to be changing
the nature of the discussion and the way and substance in which policymakers
and agency staff communicate. Reports like this one from the GAO point out that
performance measurement improves communications internally among employees
to include discussions of outcomes and results, as well as externally between the
organization and customers and stakeholders.
Communication with the public also changes as public expectations for govern
ment accountability shift. Overall, the quality of information and the way that it
is communicated within and between organizations affects their function (Graber,
1992.) Although communication steers the activities of all organizations, public
organizations face particular challenges:
Concern about public sector communication is especially necessary because com
municating has become increasingly difficult in all types of organizations, private
as well as public. Public agency insiders-as well as outsiders concerned about
bureaucracies-are asking more questions and are more willing and often more
eager to express dissenting views. (Graber, 1992, p. xvi)
Does performance measurement change the way that public administrators

communicate? If so, do enhanced communication patterns allow managers and
policymakers to make better, more informed decisions? Viewing communication
effects as a legitimate outcome of performance measurement is similar to views of
program evaluation used by policy researchers. Early critics of program evaluation
and policy research bemoaned their use by policymakers. At the time, program
evaluation scholars moved from the historical view of program evaluation as a
tool for determining the value of a program (Guba & Lincoln, 1981) to recogniz
ing the importance of going through the evaluation process itself as providing
important benefits for organizations (Patton, 1986; Patton, Grimes, Guthrie et
al., 1977). In particular, researchers noted that evaluation in itself was a prob
lem-solving process or a process that provided information for decision making
(Thompson, Brown, & Furgason, 1981). These works recognized evaluation as a
process, not an event, which had the potential to yield benefits to the organization
continually through that process.
Figure 1 shows the implementation of performance measurement using one of
its own tools-a logic model showing the sequence of outcomes. Logic models
are often used in performance measurement and strategic planning to illustrate the
Long-term Improved program quality, responsiveness,

outcomes and effectiveness
Intermediate Better management of programs and public organizations;

outcomes more informed fiscal allocation decisions
Initial Changed/improved communication among program!

outcomes agency staff and with external stakeholders
and policymakers
Outputs Communication of performance information about

programs and activities
Activities Performance measurement development and integration

into existing and new decision processes and systems
Figure 1. Communication as an Early Outcome of Performance

Measurement Implementation
relation between an organization's activities and its early, mid-term, and eventual
desired outcomes. Although the long-term goal and intended outcome of perfor
mance measurement implementation is improved program quality, responsiveness,
and effectiveness, earlier impacts of this implementation may be demonstrated in
the form of changed and improved communication among program and agency
staff about programmatic components and outcomes. As Berman (1998, 2002)
noted, the implementation of managerial reforms, including performance mea
surement, takes time before benefits are realized. The communication benefits of
performance measurement implementation may, therefore, be an indication of
its maturation process. Governments then may be experiencing the intermediate
outcomes of this implementation in the form of communication benefits, leading
in turn to improved organizational processes.
Current performance measurement efforts differ significantly from similar

efforts in the past, in fact, because of their integration with other public manage
ment initiatives. The current trend toward a more holistic view of performance
measurement, rather than simply an emphasis on changing appropriation levels,
is evident in the language governing current performance-based efforts, as well
as in the attitudes of working practitioners (Melkers & Willoughby, 1998). In
the action of integrating performance measurement across an organization and
across other management processes, information that is of use to management
and stakeholders may be gleaned. As a result, we may expect that other impacts
or uses of performance measurement are evident in their impact on public or
ganizations. The communication changes that occur as a result of performance
measurement activities demonstrate the maturation of these activities. They reflect
the cultural shift (or lack thereof in cases in which communication patterns have
not changed) required for the full implementation of performance management.
Measures and their supporting systems are developed concurrently with new and
changing communication patterns and channels. Joyce (1993) noted, in fact, that
performance measurement in the budgeting process has had its greatest success
as a management and not budgeting tool. He noted that performance budgeting,
and the information that results from its application, assists managers of public
programs in understanding the activities carried out under their purview and thereby
contributes to their ability to make more successful changes to programs.
Although performance measurement requirements are meant to influence bud
getary decisions, the rationale behind many of today's efforts remains an overall
improvement of management processes. This paper explores the impacts of perfor
mance measurement development from a larger public management perspective by
focusing on the influence of performance measurement on changing communication
patterns between budget actors and in the budget process. Although beyond the
scope of this paper, additional research may link more tightly with current research
on organizational communication to address the subsequent links between improved
communication processes and more effective public programs and organizations.
Research Questions and Methodology
Viewing the performance measurement process through its own lens shows that
communication impacts and changing the quality and substance of discussions
between budget actors is an important intermediate outcome of developing a
comprehensive performance measurement system. In turn, enhanced communica
tion patterns can have important benefits on program articulation, delivery, and
subsequent outcomes. The research reported in this paper focuses on questions
developed around the following two themes within the context of state and local
budgetary decision processes:
* Administrator perceptions of performance measurement impacts on the quality

and substance of budget-related communication with stakeholders and budget
actors; and
* Factors important in determining the extent of positive communication effects
of performance measurement in state and local governments.
First, research on performance measurement implementation has pointed to

managerial uses of performance measurement (Joyce, 1997; Joyce & Tomkins,
2002; Melkers & Willoughby, 2001). Given these uses, how do administrators in
state and local governments who have developed performance measures perceive
changes in communication with other budget actors within their same government
as a result of this implementation? For example, has performance measurement
changed the way that agency or department personnel interact and communicate
with their central budget office or other policymakers? Further, has the actual sub
stance or tone of these discussions changed? Do the traditional relationships and
communication patterns between agency and budget staff change with the imple
mentation of performance measurement? Second, if improved communication and
relationships between agency and budget actors is an important step in successful
performance measurement, what factors determine the extent of communication
effects? For example, effective leadership is generally pointed to as critical to the
success of current administrative reform (Hennessey 1998; Ingraham 2005). How
important has leadership support for and use of performance measurement been
in state and local governments? Finally, for each of these questions, are any dif
ferences found between state and local government or administrator position?
To address these questions, the data reported in this research are based on an
extensive survey and case study process focusing on performance measurement
use in state and local governments. The study was part of a multiyear effort by
the Government Accounting Standards Board (GASB) to extend their ongoing
work in performance measurement. Funded by the Alfred P. Sloan Foundation,
26 case studies (1999-2001) and a major mail survey (2000) were conducted by
the GASB Performance Measurement Research Team. The team was comprised
of GASB researchers, faculty from the Andrew Young School of Policy Studies
at Georgia State University, a government agency consultant, and two graduate
students. Overall, the project focused on the depth and breadth of the actual use of
performance measures by state and local governments for budgeting, management,
and reporting. The combination of a mail survey that included both close-ended
items as well as some open-ended questions resulted in a large and rich dataset
useful for exploring the nature of performance measurement implementation in
state and local governments. The research reported here focuses on the portion
of the survey that addressed communication-related issues.
For the mail survey, three separate survey instruments were developed, tar
geted at (a) state budget officers in the legislative and executive budget offices,
80 PPMR ' September 2006
Table 1. Survey Response Rates
Response rate Number

comuiteof states!
communities
Survey type % n represented
State budget officers 51 60 36 states
State agency staff 35 152 48 states
Local government 37 277 200 cities, 53 counties
Total 37 489
(b) administrators with perf

in state agencies, and (c) budge
governments. As shown in Tab
budgeters and 48 states for s
were sent to 121 officers in th
states and 434 questionnaires w
ing Corrections, Education, W
The 212 responses include th
budgeters contacted respond
gram directors/staff from 48
percent response rate) and re
35 states. At the local level, 2
200 municipalities and 53 cou
Separate survey instrumen
ing appropriate for the level o
For example, questions partic
contained in the state budget
allow for comparability acros
Therefore, the majority of q
wording.
The nature of the population
sampling. Instead, careful iden
staff with performance measu
respondents were identified th
as the National Association of
ment of this list. At the local l
GASB survey of local admini
from the International City
respondents were promised co
The mail survey included ques
use in state governments, the
and public management proces
ties used in state and local governments, and administrator understanding of the
purposes of generating and using performance data. The survey also addresses
details of where performance data appear in budget and financial reports, when
during the budget cycle performance information is most helpful, and current
reward and sanction systems used to reward agencies for reaching or missing
performance goals. Finally, the survey addressed administrator perceptions of
the relative success of the performance measurement process and its impact on
cost savings, efficiency, effectiveness and program results, enhanced communi
cation, and better understanding among government officials and with citizens.
Most relevant to the research presented here were the questions that addressed
the communication aspects of performance measurement implementation. The
survey addressed a range of questions about the effects of performance measure
ment development and use on specific communication and information needs.
Respondents were asked, "In your opinion, how effective has the development
and use of performance measures been in your agency regarding":
* Improving communication between departments and programs?
* Improving communication with the executive budget office?
* Improving communication with the legislature and legislative staff?
* Changing the substance or tone of discussion among legislators about agency
budgets?
* Changing the substance or tone of discussion among legislators about oversight
of agencies?
* Changing the questions legislators or their staff ask government managers or
executives?
* Changing appropriation levels?
* Communicating with the public about performance?
Respondents were also asked the extent to which they agreed or disagreed with
the following statements:
* Communication between agency personnel and the executive budget office has
improved with the implementation of performance measures.
* Communication between agency personnel and legislators has improved with
the implementation of performance measures.
* Because of the implementation of performance measures, the substance or tone
of budget discussions among legislators has changed to focus more on results.
Finally, to enhance the survey data and allow for fuller analytical options, ad
ditional data were later merged with the survey data. Specifically, community-level
statistics, including population and the form of government, were obtained from
the ICMA Municipal Yearbook (2002) and the U.S. Bureau of Census.
The case study process was a parallel data collection process that allowed for
extensive qualitative data. In 1999, the GASB research team interviewed 10 to 15
individuals each in 26 state and local governments. Interviewees included legisla
tors and legislative staff, state budget staff, and agency performance measurement
contacts, as well as key managers in each selected agency, in the media, and in
citizen groups (when available) to determine the following:
* The depth and breadth of actual use of performance measures by these govern
ments for budgeting, management, and reporting,
* The effect of their use, and
* The extent to which governments are ensuring the relevance and reliability of
performance measures.
A semistructured interview instrument was developed along the same themes

noted for the mail survey. State and local governments were identified through prior
studies of performance measurement in the United States and selected as entities
that were sufficiently engaged in performance measurement to provide detail on
performance measurement use. Together, the survey and case studies provide one
of the most extensive data sources on performance measurement use in U.S. state
and local governments.1 The results here rely primarily on the survey results, using
evidence from the case studies as illustration when appropriate.
Research Findings
Have state and local government administrators and public budgeters changed
the way in which they communicate regarding the outcomes of public programs?
Based on this research, the answer is yes in some circumstances. This research
shows that administrators in state and local governments perceive some impact
of performance measurement implementation on budget actor relationships and
communication patterns and content. First, findings on administrator perceptions
on how performance measurement development and use affects communication
between budgetary actors are presented, including comparisons across levels of
government and administrator position. Second, if improved communication
between policy and budget actors is, in fact, a sign of performance measure
ment maturity and development, then what determines whether communication
benefits are realized? Do barriers exist to improving and changing traditional
communication patters? Here, the results of a series of explanatory models are
presented, highlighting the key factors in communication effects of performance
measurement implementation.
Performance Measurement Use and Changing Communication

Patterns in State and Local Governments
This survey, as well as other studies of performance measurement at all levels

of government, typically shows that performance measurement has done little
(thus far) to change actual appropriation levels or significantly affect the budget
decision process (Broom & McGuire, 1995; Joyce, 1997; Melkers & Willoughby,
2001). Among other things, budget decisions are fraught with politics and other
factors that will not change immediately with the introduction of new processes.
In spite of some of the cynicism about these effects (or lack thereof) on the budget
process, administrators acknowledge that the development and implementation
of performance measurement has resulted in some spillover effects on commu
nication during the budgeting process. However, this perspective dramatically
de-emphasizes the real value of these communication benefits. In fact, consider
ing the implementation of a performance measurement system thorough its own
lens, communication and information effects should be viewed as interim or initial
outcomes of the process itself. Further, communication effects may be the required
missing link between performance measurement implementation and real use. If
communication patterns and substance do not change with the use of performance
measures, then any actual benefits of those performance measurement systems
will not be realized.
As performance data expand beyond internal or budget documents, they have
the potential for reaching (and, therefore, communicating) with a broader set of
stakeholders. Several individuals interviewed for the case studies described the
role of performance measurement in budget and other documents as forging new
communication patterns. For example, the State of Iowa has made an important
commitment to include citizens in its performance measurement process, which is, in
turn, reflected in the expectations for the performance system. One official noted:
Agencies are using budgets for results information for communicating results to
the legislature and to citizens, and for telling their stories differently than they have
in the past. They are also using information to foster internal accountability and to
develop a common language with which to communicate.
Yet, difficulties exist in effectively reaching the intended audiences. Communi

cation of performance results requires multiple formats and language, depending on
audience interest as well as capabilities. One administrator from Illinois noted:
According to the media, the public is interested in program results and costs. Yet,
often the information government provides for public consumption is difficult to un
derstand. Such information is often provided without comparative statistics, making
it difficult for citizens to determine how well a government agency is performing....
And there are other problems with such information-lack of communication in a
timely manner, and often communicated with a "political spin" that further muddies
the water regarding government performance.
These experiences suggest not only that the value of performance measure
ment in changing communication patterns and substance is important, but also
that administrators are viewing it as a component of the overall success of these
systems. Packaging the performance data to communicate effectively requires
knowledge of the audience as well as cooperation of media.
Although communication with external stakeholders is important, how has the
implementation of performance measurement affected the substance and nature

of communication within state and local governments? Overall, performance
measurement implementation has been most marked in increasing awareness of
results and the factors that affect performance results.2 This sort of learning and
awareness can have important effects on discussions among budget actors. But
do they? Comparatively speaking, respondents report higher communication ef
fects with the executive budget office and other internal players and less so with
the general public.
Of the three main groups surveyed (state budget officers, state agency staff,
and local administrators), state agency staff are slightly more positive in their
views about the effects of performance measurement on communication issues.
They are also slightly more positive than either local government respondents
or state budgeters about whether performance measurement has resulted in any
real appropriation changes. This positive attitude is particularly true when asked
whether performance measurement has "increased awareness of and focus on
results." State agency respondents also note more changes in the substance or
tone of discussion among policymakers as a result of performance measurement.
It was critical to this research effort that respondents who were familiar with and
engaged in some level of performance measurement activity respond to questions
about performance measurement effects and other questions about performance
measurement use and implementation.3 Overall, responses indicate that some form
of performance measurement activity is pervasive in the governments surveyed.
Of these, respondents from state agencies agreed more often that performance
measures are focused on results, whereas state budget office respondents were
more similar to local government respondents. Further, this opinion is strongest
for state agency respondents from Departments of Social Services, Education, and
Transportation and weakest for Departments of Correction. At the local level, more
county-level respondents agreed or strongly agreed that performance measurement
efforts are focused more on program results than on straight workload measures.
And, as at the state level, city and county budget officers were less likely to agree
that the use of outcome measures was more prevalent.
From the survey data, state budgeters are the least enthusiastic about commu
nication effects. Overall, budgeters are almost evenly split when asked whether
the implementation of performance measures had improved communication
between agency personnel and the budget office and between agency personnel
and legislators. The incremental and staged aspects of performance measurement
implementation may in part explain this split, as one administrator from Oregon
noted:
The Budget and Management Division of the Department of Administrative Services

is struggling to give legislators information that is useful to them. Performance
measurement is, at first, a tool for communicating goals internally. Then, when that
has been accomplished, it can be used externally-for example, in budget presenta

tions to the legislature.
Overall, budget officers from states where performance budgeting is legislated

tended to feel more strongly about improvements in communication efficacy (Wil
loughby & Melkers, 2001). Importantly, they also note the change in substance or
tone of budget discussions among legislators to a stronger focus on results. When
asked to react to a number of statements about potential problems that have oc
cuffed that may have impacted the effective development and use of performance
measurement in their state, state budgeters point to "a lack of interest of leader
ship in using performance measures," more so than state agency administrators.
The leadership role in supporting and participating in the communication of
performance results are often pointed to as critical in the success of the overall
system. Further, that leadership extends across government. The responsibility
for supporting performance measurement and communicating it across agencies
to budget offices and policymakers is an issue of coordination and cooperation.
As an administrator from Texas noted:
The cooperation between branches of government has enhanced the government's
ability to focus on outcomes. This cooperation is found between the executive and
legislative budget offices and the involvement and participation between members
of the legislature, the governor, and the agency staff.
At the local level, local budget staff were similarly less enthusiastic in their
responses than line department staff or local executives (representatives of mayoral
or county council offices). Although numbers were not sufficient for any indi
vidual line department (such as Fire, Roads, etc.) to allow for comparisons across
service areas, the responses of local line departments overall show much higher
assessments of the changes in communication due to performance measurement.
Further, respondents from local line departments overall were distinctly more
enthusiastic in their agreement with the statement,"Our government is better off
since we began using performance measures."
These findings provide useful descriptive findings on trends that deserve fuller
attention in the continuing examination and research on performance measurement
effects. The most important differences occurred when comparing across important
qualitative aspects of the performance measurement efforts rather than across organi
zation lines. Overall, respondents who strongly agreed or agreed that "performance
measures in (our government) are focused more on program results than straight
workload measures" report far greater positive communication effects that those who
disagreed or strongly disagreed. Here, respondents who reported a greater emphasis
on results indicated not only improved communication overall but also more strongly
agreed that the substance of those discussions had changed. Conversely, respondents
who indicated that the reporting emphasis was not on results more strongly agreed
that leadership lacked interest in using performance measures.
These results strengthen arguments that not all performance measurement efforts
yield equivalent results. In fact, these findings suggest that organizations that do
not mature through the performance measurement process are less likely to reap
any useful benefits at all, even less concrete ones such as improved communication
and understanding among stakeholders. Further, the findings point to real benefits
of performance measurement in the form of improved communication among key
actors and stakeholders, which, importantly, includes both budgetary actors as well
as general communication about programs and results within agencies.
What Affects the Communication Outcomes of

Performance Measurement?
Descriptive analysis of the survey data point to important evidence of communica

tion effects of performance measurement implementation for the state and local
governments in this study. Further, consistent with findings at the federal level
(Joyce & Tompkins, 2002), these benefits appear to be stronger than other antici
pated uses of performance measures in specific budget and allocation decisions.
Yet, the data also show differences across governments. If changing traditional
communication patterns are an early indicator that performance management is
on track to achieving its goals, which factors are most important? To take this
analysis one step further and develop a more detailed understanding of the factors
that are important in the communication outcomes of implementing performance
measurement, regression analysis is used. This explanatory modeling takes the
analysis beyond a purely descriptive context to a more complete picture of the
relation among various governmental, organizational, and personal characteristics
with performance measurement use and related communication effects.
Communication Effects = Bo + BiState Characteristics + B2Respondent

Characteristics + B3Performance Management Culture + B4Performance
Measurement Characteristics + error
In this model, an index of communication effects was constructed to create the

dependent variable. This summative index (Cronbach's alpha - .88-.90 depend
ing on the group of respondents) reflects variables such as respondent reports of
enhanced communication with legislators, executive budget officers, and other
stakeholders. The specific survey items and corresponding Cronbach alpha values
used to construct this dependent variable are noted in Table 2. Consistent with
the descriptive data presented earlier in this paper, the model was applied to
three groups within the dataset: state budget staff respondents only, state agency
respondents only, and local government respondents only (including budget and
department staff). Each model contained a core set of independent variables that
represented questions that were asked of each population. Basic question content
Table 2. Dependent Variable Index: Components and Reliability Statistics

Index Components
Index Cronbach's Alpha (Created Summatively)
Communication effects: State budget staff In your opinion, how effective has
Reported impacts respondents = .90 the development and use of perfor
of performance measure- mance measures been in your agency
ment on communication State agency staff (very effective, effective, somewhat
between various policy respondents = .88 effective, not effective) regarding:
and administrative actors * Improving communications between
Local government departments and programs?
respondents = .88 * Improving communication with the
executive budget office?
* Improving communication with the
legislature and legislative staff?
* Changing the substance or tone of
discussion among legislators about
agency budgets?
* Changing the substance or tone of
discussion among legislators about
oversight of agencies?
* Changing the questions legislators
or their staff ask govemment
managers or executives?
is presented with the results in Table 3. As previously noted, additional variables

reflecting state and local characteristics were compiled and merged with the sur
data, which allowed for more comprehensive models at the state and local levels.
Further, two items (as outlined in the following discussion) were only relevant to
state budgeters and were only reflected in the model applied to that group.
When addressing communication effects among state budgeters, the constructs
of transparency and density were created from survey items whereby transpare
measures the extent to which performance measures exist in a range of budg
documentation across organizational units in a government, including departmen
budget requests, the executive budget itself, operating budget documents, and othe
quarterly and annual reports. Transparency increases with the presence of perfor
mance measures in each report or document. Density measures the extent to whic
respondents claim use of performance measures at various stages of the budg
process, from budget development through audit-both internal to departme
and government-wide. Measurement density increases as respondents indicate tha
more departments use measures across more stages of the typical budget cyc
Density and transparency are only used in the models tested for budget and local
level respondents because state administrators were not asked detailed questi
about the integration of performance measures in budget documentation.
To what extent does the nature of a government affect the extent to wh
t *
sl CN N N "Z C) s r- CI cr) C) 0 CD t Oo
o o t
Xc~~~~~~r 1@144 1 ct l
o;C_o
, |I I
tI ,
II
V o n U X m m o. o o o . o.,C m c o
8 n = t >o ctn >
v > a~~~~~~~~~~~~~~~~~~~
40 4-1~~e~0 Cl -
6666 4 c 66 66666
^ o yf p5 -:9, ,_
> C~~~~~~~~)
P 4 4-i (U Cn C)6)
-4S a
0 -"lq?Xsa
C~~) 7~
JUSpUadaPUI
C'l 04-, 0 a)5 2 z
o ~~~~ ~ ~~~~
- ~~~~ M
i 4 ) -0 C) cU4i)N --
S ci C) 0
a)~ + +-M
m X
+Z 0 ; 0v0I?Z v:C)
0 " 0 -a)a) t *
4i) u i4t- C a) -
SdlqvjVA U- addpu
positive communication benefits are observed in the performance measurement

development process? In the model, state government characteristics include
variables such as the government size in terms of population as well as general
revenues. Drawing from prior research, the model also includes a dummy variable
for the two state-level models indicating whether performance measurement was
legislated or administratively required in that state (Melkers & Willoughby, 1998).
Building on recent work addressing governmental capacities and performance,
additional variables were drawn from the Government Performance Project
(Barrett & Greene, 2001), using grades developed for the states managing for
results processes, as well as average grades for state performance. To reflect the
less tangible factors affecting communication, organizational culture is defined
through a number of variables: Leadership support is a summative index created
from survey items in which respondents indicate problems with leadership sup
port for performance management. Another survey question provides a second
organizational culture variable; this item asked respondents if their government
employed an explicit managing-for-results policy. The final three organizational
culture variables are made up of survey items that measure technology problems
related to the management of performance data and organizational inclusion of
external entities within state government and citizens in the government's perfor
mance measurement development and verification process.
Measurement maturity is created by indexing survey items in which respondents
indicate the extent to which their government uses performance measurement for
a range of public management practices, including strategic planning, budgeting,
contract management and personnel decisions, and extending to benchmarking
with other governments, indicating a reasonably well-developed measurement
system.
Finally, respondent experience in government, as well as familiarity with per
formance measures, may be important in the adoption and use of performance
measures in government. Thus, the model includes variables that reflect respondent
experiences, including reported years of experience in their state agency, as well
as reported years of experience in working with performance measurement.4
Which factors are important in changing the traditional communication patterns
in state and local governments to focus on results? What matters most? Overall,
the regression models point to respondent experience, leadership, the process of
measurement development, and the inclusion of performance measures in docu
mentation as significant in explaining communication effects.
As shown in Table 3, for state agency and local government respondents, the
respondent's own experience matters. Interestingly, administrators with the most
years employed in government were most negative in their views of the effects
on communication. This finding may represent a jaded view of a new reform and
some aspects of cynicism among long-term employees. It also points to the impor
tance of taking seriously the cultural change aspects of performance measurement

implementation. Similarly, lack of leadership support was also detrimental to com
munication effects for these respondents. This result underscores the critical role
that leadership plays in implementing new initiatives. Interestingly, the inclusion
of other organizations in the performance measurement development process was
only significant for state agency respondents.
From the regression results, various aspects of the measures themselves clearly
matter. As noted earlier, the mere presence of performance measures does not
equal use. The findings support this-transparency (i.e., presence of performance
measures in budget documentation) of measures is not significant in any of the
models. However, findings show that measurement use in various stages of the
budget process (density) has a positive and significant influence on communication
effects for state budgeters and local government respondents. Further, the extent to
which output and outcome measures are used in a range of management decisions
and activities (measurement maturity) is positively and significantly related to com
munication effects, but only for state budgeters. These results suggest that as more
evidence of active use and serious consideration of performance measures occur,
shifts in culture, as demonstrated through changing communication patterns, may
take place. As measures are used more actively in decision making, communication
patterns and content change. Finally, the characteristics of the government itself
mattered, but only for local government respondents. Here, city respondents are
somewhat less likely than country-level respondents to realize communication
effects, as are respondents from major metropolitan areas. Population, revenues,
or even the managing-for-results grade for governments were not significant in
explaining communication effects.
Conclusion
To date, little evidence exists that performance measurement has had any real
impact on fiscal outcomes and appropriations. However, this lack of evidence does
not doom these efforts to failure. In fact, this research shows that real progress
has been made in terms of changing organizational cultures in state and local gov
ernments to make use of performance measurement. Performance measurement
seems positioned to provide increasingly useful information to decision makers,
while at the same time changing and improving discussions about programmatic
results among stakeholders.
As an administrator from Oregon observed, "Agencies are making more effort to
communicate performance, strategic plans, and goals." Taking the long-range view
of performance management implementation, we should expect to see changes
in the way in which administrators communicate internally as well as externally
with budget staff and elected officials. Certainly, when routinized, performance
measurement improves communication within and across branches, advances more

informed discussions about the results of government activities and services, and
enlightens budgeting decisions through the provision of additional and relevant
information (Chackerian & Mavima, 2001; Grizzle & Pettijohn, 2002; Ingraham
& Moynihan, 2001; Pettijohn & Grizzle, 1997; Willoughby & Melkers 2000).
Evidence from research on performance measurement use in the states shows that
performance measurement can have important and influential impacts on manage
ment of government as it changes the focus of discussions and even the nature of
communication and relationships among budget actors in state and local govern
ments. But, important circular relationships exist in the process-communication
about policy and priorities affects the quality of performance measurement. As
one administrator from Texas noted:
Performance measures have helped communication with the state govemment by
focusing on relevant issues. On the other hand, without communicating priorities,
it would have been impossible to formulate meaningful performance measures that
would improve the government. Communication is also a component of the leader
ship factor. Communication about the vision, purpose, and objectives occurs from
the top of the government and must be clear and follow up on the objectives.
Another administrator from Oregon also pointed to the change in culture and
behavior within government. He noted, "Internally, performance measurement
has created a climate that values accountability and has given the government a
language to communicate about it."
How do we move performance measurement forward to realize these com
munication effects? Performance management systems continue to evolve, yet
there will always be room for improvement, particularly in the areas of imple
mentation details and the integration of performance measures in political and
organizational contexts (Feller, 2002). Findings from this study point to leadership
support, participative processes in the development of performance measures, and
changing the attitudes of (particularly) long-term administrators about the value
of performance measurement.
Certainly, performance measurement is unlikely to ever become a flawless pro
cess. Problems abound in the nature of the measures themselves within an inherently
political system. As Ammons (2002) noted, performance measurement systems must
include measures that promote managerial thinking. Yet, organizational barriers
and lack of leadership support remain real problems for the successful implemen
tation of performance measurement. Issues of organizational capacity are evident
where the real barriers exist in the "imperfect state of knowledge about what these
measures should be, how to construct them, and the administrative feasibility and
cost-effectiveness of data collection and analysis" (Feller, 2002, p. 449). Lee and
Burns (2000) suggested that states may not be moving forward as much as has been
expected in regard to performance-based budgeting due to lapses in institutional
memory and caution based on past experience that results in administrators' being
unwilling or unable to move to immediate and widespread adoption. Evidence from
this research points to problems of long-term staff attitudes about performance
measurement. Yet, as we move forward, "a more realistic perception may exist
in the states about the effort required in revising budget systems and the likely
outcome of the reforms" (Lee & Burns 2000, p. 53). In other words, practitioners
recognize the existence of a learning curve associated with the implementation of
performance measures in budgetary decision making. Along the way, the recogni
tion of the pitfalls or resulting dysfunctions of performance measurement can then
hopefully be avoided or minimized (Grizzle, 2002; Ridgway, 1959).
The issue of semantics is a reality in discussions of performance measurement
practice. In this study, survey respondents were supplied with explicit definitions
of performance measurement as well as specific types of performance measures
to consider in their responses. Prior studies clearly show that many governments
are in the early portion of the learning curve in terms of the sophistication of
their performance measurement system. Unfortunately, too many report actively
using performance measures, when in reality what they are doing is exclusively
using workload and some output measures but few, if any, outcome measures. The
results of this research show that without an emphasis on outcomes, performance
measurement systems are likely to do little in terms of enhancing or changing
communication patterns with policymakers and budget actors. Earlier analysis of
performance-based efforts points to encouraging signs that performance-based
efforts are moving along the adoption curve (Broom & McGuire, 1995; Melkers
& Willoughby, 2001). These previous studies have shown a commitment to con
tinued development of performance measures and a willingness to revise the use
of performance measurement-based systems given their place within the larger
efforts of strategic planning and benchmarking activities.
This study shows that administrators are experiencing important impacts of
performance measurement activity that have implications for organizations. As
Graber (1992) noted, effective communication is critical to organizational function.
Most important, this study adds more evidence supporting the importance of the
distinction between measuring outputs and outcomes. In this study, respondents
from governments that report more developed systems of performance measure
ment, where measures reflect more outcomes than activity-based measures, also
report much stronger improved communication effects. This finding suggests that,
although challenging to develop, sound outcome measures can provide important
and useful information for public administrators.
Although performance measurement is not the information "silver bullet" that
some hope it may be, it has contributed to important changes in some governments
in which leadership and administrators work together to create a results-based
culture. Some have noted, as evident in anecdotes reported here, the effect of us
ing performance measurement has been to make department staff more aware of
and serious about performance information and to improve internal department
communication and understanding of performance. This research shows that real
interim outcomes are occurring with the implementation of performance measure
ment requirements in government. Communication between agency and budget
actors is changing-for the better. Although performance measurement efforts
in state and local governments are uneven in the quality and sophistication of
performance measurement systems, some governments have been able to move
forward and benefit from the hard work and resources invested in performance
measurement implementation.
Notes
1. All case study quotations are drawn from the cases available for download at www.
seago v. org/sea_gasb_proj ect/case_studies. shtml.
2. See Melkers and Willoughby (2001) for detailed data on budgeter responses in the
states.
3. Detailed research findings on the extent of performance measurement activities and other
uses of performance measures beyond the communication aspects reported in this article may
be found in Willoughby and Melkers (2000) and Melkers and Willoughby (2001).
4. Tests for multicollinearity using the variance inflation factor revealed no such problems,
consistent with acceptable ranges under 10 for any one of the variables (with most near or under
5 [Belsley, Kuh, & Welsch, 2004]).
References
Arrimons, D. (2002). Performance measurement and managerial thinking. Public Perfor

mance and Management Review, 25(4), 344-347.
Barrett, K., & Greene, R. (2001). Grading the states 2001: A management report card.
Governing Magazine. Available at www.governing.com/gpp/2001/gplintro.htm, ac
cessed August 1, 2006.
Belsley, D.A., Kuh, E., & Welsch, R.E. (2004). Regression diagnostics: Identifying influ
ential data and sources of collinearity. New York: Wiley.
Berman, E. (1998). Productivity in public and nonprofit organizations. Thousand Oaks,
CA. Sage.
Berman, E. (2002). How useful is performance measurement? Public Performance and
Management Review, 25(4), 348-351.
Berman, E., & Wang, X.-H. (2000). Performance measurement in U.S. counties. Public
Administration Review, 60(5), 409-420.
Broom, CA., & McGuire, L.A. (1995). Performance-based government models: Building
a track record. Public Budgeting and Finance, 15(4), 3-17.
Carnevale, A.P., & Carnevale, D.G. (1993). Public administration and the evolving world
of work. Public Productivity and Management Review, 17(1), 1-14.
Chackerian, R., & Mavima, P. (2001). Comprehensive administration reform implementa
tion: Moving beyond single issue implementation research. Journal of Public Admin
istration Research and Theory, 11(3), 353-377'.
Cohen, D. (1998). Towards a knowledge context: Report on the First Annual U.C. Berkeley
Forum on Knowledge and the Firm. California Management Review, 40(3), 22-39.
DuPont-Morales, M.A., & Harris, J.E. (1994). Strengthening accountability: Incorporating

strategic planning and performance measurement into budgeting. Public Productivity
and Management Review, 17(3), 231-340.
Feller, I. (2002). Performance measurement redux. American Journal of Evaluation,
23(4), 435-452.
Government Accounting Standards Board. (2000). State and local government case studies:
The use and the effects of using performance measures for budgeting, management,
and reporting. Available at www.seagov.org/sea_gasb_project/local_studies.shtml,
accessed August 1, 2006.
Graber, D.A. (1992). Public sector communication: How organizations manage informa
tion. Washington, DC: Congressional Quarterly.
Grizzle, G.A. (2002). Implementing performance-based budgeting: A systems-dynamics
perspective. Public Administration Review, 62(1), 51-62.
Grizzle, G.A., & Pettijohn, CD. (2002). Implementing performance-based program budget
ing: A system-dynamics perspective. Public Administration Review, 62(1), 51-62.
Guba, E.G., & Lincoln, Y. (1981). Effective evaluation. San Francisco: Jossey-Bass.
Hatry, H. (1999). Performance measurement: Getting results. Washington, DC: Urban
Institute Press.
Hennessey, J.T. (1998)."Reinventing" government: Does leadership make a difference?
Public Administration Review, 58(6), 522-532.
Ingraham, P. (2005). Performance: Promises to keep and miles to go. The Donald Stone
Lecture. Public Administration Review, 65(4), 390-395.
Ingraham, P.W, & Moynihan, D.P. (2001). Beyond measurement: Managing for results in
state government. In D.W. Forsythe (Ed.), Quicker, better, cheaper: Managing performance
in American government (pp. 309-334). Albany, NY: Rockefeller Institute Press.
International City/County Management Association. (2002). The municipal yearbook.
Washington, DC.
Jensen, M.C., & Meckling, W.H. (1992). Specific and general knowledge, and organiza
tional structure. In L. Werin & H. Wijkander (Eds.), Contract economics (pp. 251-274).
Oxford: Blackwell.
Joyce, P.G. (1993). Using performance measures for federal budgeting: Proposals and
prospects. Public Budgeting and Finance, 13(4), 3-17.
Joyce, P.G. (1997). Using performance measures for budgeting: A new beat, or is it the
same old tune? In K. Newcomer (Ed.), Using performance measurement to improve
public and nonprofit programs (pp. 45-61). San Francisco: Jossey-Bass.
Joyce, P.G., & Tompkins, S.S. (2002). Using performance information for budgeting:
Clarifying the framework and investigating recent state experience. In K. Newcomer
et al. (Ed.), Meeting the challenge of performance oriented government (pp. 61-96).
Washington, DC: American Society for Public Administration.
Kettl, D.F. (1994). Reinventing government: Appraising the National Performance Review.
Washington DC: Brookings Institution.
Knock, N., & McQueen, R. (1998). Knowledge and information communication within
organizations: An analysis of core, support and improvement process. Knowledge and
Process Management, 5(1), 29-40.
Lee, R.D., & Burns, R.C. (2000). Performance measurement in state budgeting: Advancement
and backsliding from 1990 to 1995. Public Budgeting and Finance, 20(1), 38-54.
Melkers, J.E., & Willoughby, K.G. (1998). The state of the states: Performance-based
budgeting in 47 out of 50. Public Administration Review, 58(1), 66-73.
Melkers, J.E., & Willoughby, K.G. (2001). Budgeters' views of state performance budgeting
systems: Distinctions across branches. Public Administration Review, 67(1), 54-64.
Patton, M.Q. (1986). Utilization focused evaluation. Beverly Hills, CA: Sage.
Patton, M.Q., Grimes, P.S., Guthrie, K.W, et al. (1977). In search of impact: An analysis of
the utilization of federal health evaluation research. In C.H. Weiss (Ed.), Using social
research in policy making (pp. 141-164). Lexington, MA: Lexington Books.
Pettijohn, CD., & Grizzle, G.A. (1997). Structural budget reform: Does it affect budget
deliberations? Journal of Public Budgeting, Accounting and Financial Management,
9(1), 26-45.
Poister, T.H., & Streib, G. (1999). Performance measurement in municipal government:
Assessing the state of the practice. Public Administration Review, 59(4), 325-335.
Ridgway, V.F. (1959). Dysfunctional consequences of performance measurement. Admin
istrative Science Quarterly, 1(2), 240-247'.
Thompson, P.A., Brown, R.D., & Furgason, J. (1981). Jargon and data do make a difference:
The impact of report styles on lay and professional evaluation audiences. Evaluation
Review, 5(2), 269-279.
U.S. Government Accountability Office. (2005). Performance budgeting: States' experi
ences can inform federal efforts. Report no. GAO-05-215. Washington, DC.
Willoughby, K.G., & Melkers, J. (2000). Implementing PBB: Conflicting views of success.
Public Budgeting and Finance, 20(1), 105-120.
Zack, M.H. (1999). Managing codified knowledge. Sloan Management Review, 40(A),
45-57.
Julia Melkers is associate professor of public administration at the University of

Illinois at Chicago. Her teaching and research interests are in the areas of the
development and use of performance measurement, public management, and sci
ence and technology in public organizations. In conducting her research, she has
worked with public organizations at all levels of government in the United States
as well as internationally.

Comunicare Organizationala

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Comunicare Organizationala

Uploaded by

Copyright:

Available Formats

On the Road to Improved Performance: Changing Organizational Communication through

ABSTRACT: The rationale behind performance measurement requirements

KEYWORDS: budget process, organizational communication, performance

But, to what end? Research shows that performance-based budgeting initiatives

Performance Measurement and Communication

Since the era of government reinvention, the emphasis in public management re

across organizations. Here, the improvement of results is accomplished through

The question in regard to performance measurement, then, is whether budget

tion provided in performance measurement systems ignores the dynamics of the

Does performance measurement change the way that public administrators

Long-term Improved program quality, responsiveness,

Intermediate Better management of programs and public organizations;

Initial Changed/improved communication among program!

Outputs Communication of performance information about

Activities Performance measurement development and integration

Figure 1. Communication as an Early Outcome of Performance

Current performance measurement efforts differ significantly from similar

Research Questions and Methodology

* Administrator perceptions of performance measurement impacts on the quality

First, research on performance measurement implementation has pointed to

Table 1. Survey Response Rates

Response rate Number

(b) administrators with perf

A semistructured interview instrument was developed along the same themes

Performance Measurement Use and Changing Communication

This survey, as well as other studies of performance measurement at all levels

Yet, difficulties exist in effectively reaching the intended audiences. Communi

implementation of performance measurement affected the substance and nature

The Budget and Management Division of the Department of Administrative Services

has been accomplished, it can be used externally-for example, in budget presenta

Overall, budget officers from states where performance budgeting is legislated

What Affects the Communication Outcomes of

Descriptive analysis of the survey data point to important evidence of communica

Communication Effects = Bo + BiState Characteristics + B2Respondent

In this model, an index of communication effects was constructed to create the

Table 2. Dependent Variable Index: Components and Reliability Statistics

is presented with the results in Table 3. As previously noted, additional variables

8 n = t >o ctn >

positive communication benefits are observed in the performance measurement

tance of taking seriously the cultural change aspects of performance measurement

measurement improves communication within and across branches, advances more

Arrimons, D. (2002). Performance measurement and managerial thinking. Public Perfor

DuPont-Morales, M.A., & Harris, J.E. (1994). Strengthening accountability: Incorporating

Julia Melkers is associate professor of public administration at the University of

You might also like