The Role of Leadership in

Performance Management

251
 Have you encountered?

• Strategic planning
• Performance measures
• Performance contracts
• Pay for performance

252
 The role of leadership

• During my 20 years in the private sector as a CEO and advisor to CEOs, I

found that leadership, measurement, and a motivated workforce create the
foundation for good performance. I am confident that the same is true in
government
• Jeff Zients – Chief Performance Officer, 2009

253
 Outline

• Defining terms
• Era of governance by performance management
• From Bush to Obama
• How do we use performance systems?
• What fosters use of performance data?
• Summary points
254
 Performance management

• A system that generates performance information through

strategic planning and performance measurement routines, and
connects this information to decision venues,

255
 Performance regimes

• Performance tools create unprecedented pressure on public

actors to perform, in a context where performance is defined by
quantitative indicators

256
 Purposes of Performance Information

• Promote: How can I convince political actors, stakeholders and

the public that my agency is doing a good job?
• Celebrate: What accomplishments are worthy of the important
ritual of celebrating success?
• Learn: What is what working or not working?
• Improve: What exactly should who do differently to improve
performance?
257
 Purposes of Performance Information

• Evaluate: how well is my agency performing?

• Control: how can I ensure that my subordinates are doing the
right thing?
• Budget: on what program’s, people, or projects should my
agency spend the public’s money?
• Motivate: how can I motivate employees and collaborators to
improve performance?

258
 ERA of governance by
performance management

259
 Era of Governance
by Performance Management

• The rise of a doctrine

• Not new, but more influential than before
• Must justify actions in terms of outputs and outcomes
• Basis for holding new structural forms accountable

260
Doctrinal logic for change

261
 Government Performance
and Results Act 1993

• Mandated:
• 5 year strategic plans, updated every 3 years
• Specific goals and objectives
• Annual performance reviews and plans

262
From Bush to Obama

263
 Bush approach

• Presidents Management Agenda

“everyone agrees that scarce federal resources should be
allocated to programs that deliver results”
• Wanted to integrate performance data into budget process

264
 Congressional Justifications

• Center around performance goals

• Pushback from Appropriations Committees
• Veteran’s Administration told; “to refrain from incorporating
‘performance-based’ budget documents”; later told: “If the Department
wishes to continue the wasteful practice of submitting a budget structure
that will not serve the needs of the Congress, the Congress has little
choice but to reject that structure and continue providing appropriations
that serve its purposes.”
• Two budgets required
265
 Congressional Justifications

 Department of Transportation told: “agencies are directed to refrain from

including substantial amounts of performance data within the budget
justifications themselves, and to instead revert to the traditional funding
information previously provided. Performance-related information may
be submitted under separate cover.”
 Negative consequences were promised for agencies that ignored this
directive: “If the Office of Management and Budget or individual
agencies do not heed the Committee’s direction, the Committee will
assume that individual budget offices have excess resources that can be
applied to other, more critical missions.”
266
 Program Assessment Rating Tool (PART)

• 5 year summary by OMB of evidence on program performance

for 1016 programs
• 18 percent are Effective
• 31 percent are Moderately Effective
• 29 percent are Adequate
• 3 percent are Ineffective
• 19 percent are Results Not Demonstrated
267
 PART as Evidence-based Dialogue
• Third-party program review with a clear opinion
• Greater emphasis on performance
• The standard of proof for program performance can only
be satisfied by positive evidence of results
• The burden of proof for performance rests on agencies
• Entire programs are evaluated on a regular basis
• The routine nature of PART creates an incentive to engage
268
 Obama: A Pragmatic approach

“The question we ask today is not whether our government is

too big or too small, but whether it works -- whether it helps
families find jobs at a decent wage, care they can afford, a
retirement that is dignified. Where the answer is yes, we intend
to move forward. Where the answer is no, programs will end.
And those of us who manage the public's dollars will be held to
account, to spend wisely, reform bad habits, and do our business
in the light of day, because only then can we restore the vital
trust between a people and their government”
269
 Example: Pedometer challenge!

• Voluntary
• Belief that transparent performance numbers will change
behavior, create a sense of competition and raise performance

270
 Early evidence on Obama

• Performance measurement will be important

• “The President is creating a focused team within the White House that
will work with agency leaders and the OMB to improve the results and
outcomes for Federal Government programs while eliminating waste and
inefficiency”
• Chief performance officer
• Continue to maintain agency level performance positions

271
 What happens to PART?

• Not clear
• Criticized as ideological, as too broad, as a data collection
exercise
• Analysis remains in place, but new PARTs have not started
• OMB have offered agencies funds for better evaluations

272
 New emphasis on leadership

• Focusing leaders on what matters – key goals

• Accelerating results – Performance Improvement Council; data
driven meetings
• Style: focused collaboration

273
 New focus on information use

• Will be a central aspect of the Obama administration’s

performance initiatives
• Jeff Zients: “The ultimate test of our performance management
efforts is whether or not the information is used”
• Shelly Metzenbaum: “the key performance management
challenge facing the Obama administration is to use—not just
produce—performance goals and measures”

274
How do we use performance
systems?

275
 Why care about use?
• For reforms to succeed, implies that data is used
• Provides a tractable means of studying the impact of
results-based reform
• Public organizations have devoted significant time and
resources into creating routines to collect and
disseminate data
• Almost no attention to creating routines of use
• How do you use performance data? 276
 Types of responses: 4 Ps

• Passive
• Perverse
• Political
• Purposeful

277
 Passive use of data

• Passive:
• Do the minimum to comply with requirements
• Do not actually use data
• Correlated with cynicism about reforms

278
 Perverse use of data

Effort Substitution: Reducing effort on non-measured dimensions

Cherry picking/Cream-skimming: Focusing effort on subgroups of
clients most likely to provide greatest impact on performance
measures while effectively denying services to others.
Measure selection: Selecting metrics or data to measure that will
offer the most favorable portrayal of a service
Hiding numbers: Declining to present performance measures that
may exist
279
 Perverse use of data

• Output distortion: Manipulating measurement processes to

improve measured performance.
• Ratchet effects: Curbing productivity in one time period to avoid
the setting of more challenging targets in another.
• Churning: Frequently adopting different targets or measures to
prevent comparison across time.
• Cheating: Simply making up numbers, though rare, does occur.
280
 Responding to perversity

• Add new/additional measures

• Change existing measures

• Rely/cultivate intrinsic norms to limit misbehavior

• Avoid high-powered incentives 281

 Political uses of data

• Process of selecting measures means shaping a program

narrative
• “Understand that measuring policy is not a science. It is an art.
It is words, and pictures and numbers. And you create
impressions, beliefs, understandings and persuasions.”

282
 Political uses of data

• Data tells us what happened

• Program officials still need to interpret and explain:
• why performance did or did not occur;
• the context of performance;
• how implementation occurred;
• an understanding of outside influences on performance; and
• how to choose which program measure is a priority.
• Exploit ambiguity and subjectivity of data 283
 Political: Ambiguity of data

• Examine same programs, but disagree on data

• Agree on data, but disagree on meaning
• Agree on meaning, but not on next action steps/resources

284
 Political: Subjectivity of data

• Actors will select and interpret performance information

consistent with institutional values and purposes

285
 Evidence of Ambiguity in PART

• Ambiguity of terms:
• E.g.: Program purpose, quality evaluation, ambitious, having made progress
• How to interpret results? Multiple logics from experiment:
• Argue that ratings are unreliable
• Cut poorly managed programs
• Raise funding for programs with positive assessments
• Parity: Raise funding because program with similiar assessment received more
• Delay cuts because progress being made
• Clear relationship between resources, need and program delivery
• Stakeholder and congressional views
286
 Evidence of Subjectivity with PART

• OMB using PART to expand influence in performance

management/policy
• OMB can define programs, goals, measures, agency responsibility
• Disagreement with agencies/Congress on meaning/relevance of
PART
• Experimental evidence:
• UW students significantly more likely to disagree with OMB, and to
argue for higher assessments and resources
287
 Implications for Decisionmaking

• Performance information use reflects political process, does not

replace it
• Performance information use does not lead to clarity
• Ability to structure dialogue tied to power

288
 Purposeful use of data

• Use data to improve program performance

• Goal-based learning
• efficiency improvements
• better targeting of resources
• more informed strategic decisions,
• tying indicators to rewards/sanctions in contract arrangements

289
 Purposeful use of data

• Use of performance information for problem-solving more

likely to occur in intra-institutional settings
• Reduces competing interpretations
• Problem of neglect
• rarely do anything with information

290
What fosters performance information
use?

291
 The Right Context

• Simple function that is easy to measure

• Clear link between measures of actions, and measures of
outcomes
• One-dimensional – relatively few measures that do not conflict
with one another
• Stakeholder support – clear agreement about purpose

292
 Other factors

• Learning forums
• Mission-based culture/supportive culture
• Resources
• Administrative stability
• Administrative capacity

293
 Quantitative approach

• 3 studies using survey-based data

• Self-reported performance information use
• Results from Moynihan and Pandey (in press) and Moynihan,
Wright and Pandey (2009; 2010)

294
 Study 1: Ordinal regression of reported performance information use for decisions
Variable Hypothesized direction Result
Individual beliefs
Public service motivation Positive ***

Job attributes
Reward expectation Positive --
Generalist leader Negative ***
Task-specific experience Positive --

Organizational factors
Information availability Positive ***
Developmental culture Positive ***
Flexibility Positive *
Budget staff take adversarial role Positive/negative --

External factors
Citizen participation Positive/negative +
Professional influence Positive +
*** = significant at .001; ** = .01; * = .05 +=.10 (two tailed tests) 295

Controls: region, income per capita, government size, population size, population homogeniety
Intrinsic vs. extrinsic motivation

• Sense of public service motivation mattered

• Possibility of extrinsic reward did not create
an incentive to use data
• Implication: performance information use as
extra role behavior

296
 Organizational factors

• Information availability
• Supply-side approach
• Use increases with better information, and when information is tied to
management systems

297
 Organizational factors

• Demand side approach

• Culture matters
• Previous work focuses on whether culture welcomed performance
management reforms
• What about broader measures of culture?
• Developmental culture (adaptability, readiness, growth)
• Flexibility – unlikely to use data if cannot apply insights

298
 Specialist vs. generalist leaders

• Task-specific knowledge provides context in which to interpret

and apply data
• Leadership role
• Task-specific leaders more likely to use data than generalist leaders

299
 Other evidence of leadership

• Support/commitment
• Provision of resources
• Participation

• What other ways can leadership matter?

300
 Study 2: Transformational leadership

Approach to leadership consistent with performance:

 Articulate an appealing vision of the organization’s
mission and future
 Model behavior consistent with vision, inspiring role
model
 Challenge old assumptions
301
 Propositions

• Transformational leadership behaviors will have an indirect,

positive effect on performance information use through its
influence on goal clarity
• Transformational leadership behaviors will have an indirect,
positive effect on performance information use through its
influence on organizational culture.

302
 Key measures
Transformational leadership
 Asked department heads/assistant city managers on extent to which city
manager demonstrates transformational leadership:
 articulates his/her vision of the future.
 leads by setting a good example
 challenges me to think about old problems in new ways
 says things that make employees proud to be part of the organization.
 as a clear sense of where our organization should be in five years.
• Aggregated responses by organization
303
 Structural Equation Model

Goal
0.66*
Transformational Clarity
Leadership 0.60*
0.57 0.26*
E Performance
0.32* Information
0.33* Use
Performance
Information
Availability 0.63
0.52 0.13*
E E

0.17*

Developmental
Culture
0.89 All paths reported as standardized coefficient
E *p < 0.05 304
 Implications

• Leadership and management

• Indirect effects are important
• “Setting the table” as long-term leadership strategy

305
 Study 3: Perceived social impact

• Individuals who see their work as helping others more likely to

use performance information
• Some evidence that individuals who perceive greater social
impact are more motivated
• Why should it relate to performance information use?

306
 Key measures

• Perceived social impact

• I feel that my work makes a positive difference in other people’s lives.
• I am very aware of the ways in which my work is benefiting others.
• I am very conscious of the positive impact my work has on others.
• I have a positive impact on others in my work on a regular basis

307
 Purposeful and political use

Purposeful
 I regularly use performance information to make decisions.
 I use performance information to think of new approaches for doing old things.
 I use performance information to set priorities.
 I use performance information to identify problems that need attention.

Political
 I use performance information to communicate program successes to stakeholders.
 I use performance information to advocate for resources to support program needs.
 I use performance information to explain the value of the program to the public.
308
 Table 2: OLS Regression of Performance Information Use

Independent variables Purposeful Use Political

Perceived social impact .43 (.12)*** .52 (.11)***

Goal clarity .19 (.10)* .25 (.09)**

Centralization .05 (.07) .10 (.08)

Political support .14 (.07)* .06 (.07)

Red tape .04 (.03) .02 (.02)

Years in position .01 (.007) .006 (.008)

Supervisory level .03 (.05) .02 (.04)

Educational level .05 (.07) .09 (.06)

Sector -.06 (.15) .08 (.14)

309

*** = .001;**= .01; *=.05 one-tailed test N=184; R2= .25 N= 186; R2= .31
 Study 4: Experimental approach

• How does performance information matter to decisions?

• How does the framing of performance information affect decisions?
• Respondents given surveys with scenario – make budget
recommendations
• Series of vignettes for different programs
• Half vignettes are control, half are treatment

310
 Theoretical background

• Research on decision frames from psychology and behavioral

economics
• Performance information is strategically selected and presented
– does this work?

311
 Does the Addition of Performance Data
Matter?
• Control: no data; treatment: addition of data without
clear correlation to resources
• The Department of Land and Water Resources is responsible for monitoring
and maintaining the water quality of lakes in the county, including two major
lakes that are popular for swimming and other water sports during the
summer. Estimates of water quality are based on pH levels, pesticides, nitrates
and other chemicals in the water.
2007 2008 2009

Program funding 174,000 179,000 182,000

Number of days water quality of major lakes 12 14 11

deemed unsafe
312
 Does the Addition of Performance Data Matter?

Control: no data; treatment: addition of data with clear

relationship to resources

 The Department of Social Services delivers a program called the Home

Downpayment Initiative. Using a mix of federal, state, and local resources, the
program seeks to increase the homeownership rates among low-income and
minority initiatives. To do so, it provides financial assistance to first-time
homebuyers for downpayment and closing costs.
2007 2008 2009

Home Downpayment funding 723,000 747,000 769,000

Number of families purchasing homes 36 38 46

313
 Is outcome data more powerful than output?
• Control: output data; treatment: outcome data
• The Department of Health Services offers a program called Health Check,
which is a preventive health check-up program made available for anyone
under the age of 21 who is currently enrolled in Medicaid. Health Check
provides a head-to-toe medical exam, immunizations, eye exam, lab tests,
growth and development check, hearing check, nutrition check, and teen
pregnancy services. The goal of the program is to prevent the incidence of
more serious and more expensive health situations.
2007 2008 2009
Health Check funding 232,000 244,000 269,000
Clients treated 1232 1401 1325

Estimated savings due to preventive care 383,000 402,000 389,000

314
 Threshold effects
Treatment: performance data pass a memorable threshold (200)
The County Tourism Board seeks to increase visits from those who live
outside the county, and to increase the use of recreational and cultural
opportunities by both locals and outsiders. It collects data from local hotels,
restaurants, and other businesses that depend on tourists. In the last number
of years, the number of tourists visiting the county has stayed relatively flat at
about 100,000, and the Board has focused its marketing budget on “quality,
not quantity,” by increasing the dollar amount that each tourist spends.

2007 2008 2009

Program budget 120,898 124,490 131,289
Average daily dollar amount spent by tourists 178 184 195

2007 2008 2009

Program budget 124,294 127,196 135,329
Average daily dollar amount spent by tourists 183 188 201 315