American Journal of Evaluation

31(4) 588-596
ª The Author(s) 2010
Book Review Reprints and permission:
sagepub.com/journalsPermissions.nav
http://aje.sagepub.com

Patton, Michael Quinn. (2008). Utilization-Focused Evaluation. Thousand Oaks: SAGE. 688 pp., $78.
ISBN 978-1-4129-5861-5

Reviewed by: Karen E. Kirkhart, Syracuse University, NY, USA

DOI: 10.1177/1098214010373646

The fourth edition of Utilization-Focused Evaluation (2008) updates us on Patton’s recent and con-
current intellectual projects—notably developmental evaluation, international evaluation, and sys-
tems thinking and complexity science. He expands and elaborates utilization-focused evaluation
and raises important questions for future debate, discussion, and reflection. At 582 text pages (plus
an additional 85 pages of references and indices), it is an impressive body of work.
Patton’s writing is lively and personal, filled with stories and examples from his practice. The style
remains engaging and conversational. One can envision the situations he encounters; stories take the
reader into Patton’s varied practice experiences, culling nuggets of wisdom and offering practical
advice. And as any conversation, it jumps a bit from one topic to another, following the storyteller’s
weaving of experience into meaning. Conversations include those with the reader (often written in the
first person), with Halcolm (Patton’s trusty ‘‘internal philosophical alter ego and muse,’’ depicted in
the fourth edition as a court jester), and with some of the 54 evaluators, decision makers, and project
officers of federal health evaluations interviewed in 1976 in the groundbreaking research (Patton
et al., 1977) that stimulated the first edition of Utilization-Focused Evaluation. The tone of fourth edi-
tion is accurately captured by Patton’s description of utilization-focused evaluation itself:

Utilization-focused evaluation combines style and substance, activism and science, personal perspective
and systematic information. I have tried to capture the complexity and potential of utilization-focused
evaluation with scenarios, case examples, findings from our study of federal evaluation use, Sufi
parables, and children’s stories. In the end, this approach to evaluation must also be judged by its
usefulness. (p. 574)

The criterion of usefulness is also relevant to this volume. Congruent with Patton’s focus on intended
use by intended users, this review considers the usefulness of the book for three different reader
audiences: evaluation practitioners; evaluation instructors and students; and evaluation theorists.
Certainly, a given reader may occupy more than one role, but the volume may be read differently
from these three perspectives.

Usefulness in Practice
A clear strength of this book is its practical utility. Through his storytelling, this book provides
glimpses of Patton in action, working with small and large groups. Concrete suggestions and check-
lists abound, accompanied by practice illustrations in shaded boxes set off from the text. The text is

588
Book Review 589

rich with maps and matrices to support evaluation practice. Congruent with his belief in multiple
possibilities, practice guidelines are presented as menus, not narrow prescriptions. Cultural
resources have been added, such as the work of Symonette on culturally competent and responsive
evaluators, and the Evaltalk discussion thread on gathering background data on race, ethnicity, and
other demographics (Exhibit 11.6).
Some practice-rich content is new, some retained, and some expanded or elaborated upon from
the third edition. Chapter 1 expands the discussion of use as a critical social issue while providing a
practice application focused on three simple questions addressed by evaluation: What? So what?
Now what? Potential misuse of evaluation through manipulation or abuse is introduced early,
accompanied by summaries of both the Guiding Principles and the Program Evaluation Standards.
The new content on the international scope of evaluation practice helps evaluators to think globally
about their work.
Chapter 2 lays out the vision and intent of utilization-focused evaluation. The section on gener-
ating meaningful questions, while not new, assumes even greater importance when working in mul-
ticultural contexts. Patton’s illustration of stating what one would like to know as a declarative
sentence rather than a question is valuable practice advice for contexts in which direct questioning
may be culturally inappropriate or even offensive. Chapter 3 emphasizes the importance of both con-
text and individuals (the personal factor). The content on practice implications of the personal factor
is particularly strengthened, both in the narrative text and in an expanded and revised menu (Menu
3.2). Readers of The Tipping Point (Gladwell, 2002) will appreciate Patton’s strategy of identifying
connectors as primary intended users to get evaluation findings out to a broad range of people.
Chapters 4 and 5 address findings use and process use, respectively. In Chapter 4, the menu of
primary uses of evaluation findings based on six alternate purposes has doubled in length from the
prior edition. Exhibit 4.10 provides specific questions to establish an evaluation’s intended influence
on forthcoming decisions, a valuable practice tool particularly useful in evaluability assessment. On
the process side, Chapter 5 refines the definition of process use and provides a menu of six uses
(expanded from four in the third edition), accompanied by examples and illustrations to support
practice. Patton includes a corresponding list of concerns, changes, and caveats to promote critical
thinking about the uses one is trying to encourage.
Situational analysis (Chapter 6) is more focused in the fourth edition, with situational evaluation
becoming the overarching framework for discussing contingency thinking—what Patton terms being
active–reactive–interactive–adaptive. Exhibit 6.2 builds creatively upon the Program Evaluation
Standards to identify corresponding utilization-focused questions for situational analysis. As discussed
below, situational analysis also becomes the vehicle for addressing cultural context.
In Chapter 7, the distinction between service-focused goals and outcome-focused goals is espe-
cially useful because real-world mission statements often appear in either wording. I particularly
appreciate the illustration of how outcome statements evolve through multiple drafts and variations
of focus (Exhibit 7.6). Patton’s discussion of goal manipulation and possible misuse of performance
indicators keeps the practitioner from seeing goal-setting as a straightforward, linear task. This dis-
cussion continues in Chapter 8, anchored in the core point that different types of evaluation ask dif-
ferent questions and focus on different purposes. The multipage summary table listing 79 illustrative
‘‘types’’ familiarizes the practitioner with terminology attached to various evaluation approaches,
but it may also place undue emphasis on categorical labels for overlapping strategies, despite
Patton’s cautionary note about language. Certainly, it illustrates the diversity of practice in our field.
This chapter features considerable new content on developmental evaluation and its roots in
complexity science. Patton connects developmental evaluation to reflective practice and provides
a helpful checklist to identify situations that favor a developmental approach (Exhibit 8.4).
Chapter 9 addresses the important issue of program implementation, but the discussion of adap-
tation versus fidelity as a criterion of excellence is insufficiently linked to stages of program

589
590 American Journal of Evaluation 31(4)

development, a key context concern. Patton guides the practice of implementation evaluation by pre-
senting a slightly expanded menu of implementation questions appropriate to different evaluation
purposes (Menu 9.1), but there is a disconnect between the five purposes used to organize the menu
of questions and the five types of implementation evaluation described in the following section.
Chapter 10 retains the strong discussion of program theory and causal assumptions and visually
depicts the pairing of levels of change with levels of evidence. Supporting critical thinking, Patton
includes cautions against overemphasizing program theory/theories of change, building on concerns
voiced by Scriven about getting distracted from the task of judging merit or worth. Sample logic
models in a variety of formats are included, and a substantial new section on systems frameworks
provides useful antidotes to overly narrow, linear theories of change. The key practice lesson here
is that ‘‘the modeling framework and evaluation approach should be congruent with the nature of the
practice intervention’’ (p. 375).
Chapter 11 carries the theme of situational responsiveness forward into methods decisions, retain-
ing a clearly written enumeration of reasons why primary users should be involved in methods deci-
sions. The Million Man March example is augmented by a practice box on the inexact science of
crowd counting. The discussion of racism, politics, and reporting attendance at the Million Man
March is particularly useful because it is followed by a description of an approach to that scenario
that would be congruent with utilization-focused evaluation. The chapter touches lightly on scale
construction and includes basic content on measurement reliability and validity, accompanied by
Trochim’s accessible ‘‘target’’ illustration (Exhibit 11.4). The natural language expression of design
parameters in terms of ‘‘What? Why? When? How? Where? Who?’’ is a useful device retained from
third edition. Although I do not find Patton’s particular partitioning of validity helpful, the range of
issues he raises supports critical thinking about evaluation practice in general. Patton ties it back to
utilization-focused evaluation by proposing threats to utility that intersect validity threats.
Chapter 12 is more theoretical than practical in the sense of offering menus of practice choices,
but the importance of methodological pluralism and appropriateness—and the corresponding limita-
tions of paradigm orthodoxy—are important foundational understandings for any evaluation prac-
tice. The title change of Chapter 13 (from ‘‘Deciphering Data and Reporting Results’’ to ‘‘The
Meanings and Reporting of Evaluation Findings’’) marks a shift in emphasis toward the construction
of meaning, with important new content on interpretive frameworks, claims, and evidence, presented
with accessible examples and exhibits. The discussion of interpretive frameworks precedes and sets
the stage for content on analysis, interpretation, judgment, and recommendation. Patton makes an
important distinction between simplicity and simplemindedness, which can well be applied to other
aspects of evaluation beyond data analysis. The discussion of recommendations has been updated and
the discussion of utilization-focused reporting principles, including the evaluation reporting menu
(Menu 13.2), remains a valuable practice contribution.
Chapter 14 reminds the reader of the ubiquity of power and politics across all evaluation contexts,
a message that will certainly ring true to practitioners. Patton gives detailed advice on effective com-
position and management of task forces, though other formats of stakeholder engagement are not
similarly addressed. Core elements of this chapter are the discussions of social betterment and ethi-
cal issues of power and responsibility. Although the ethical issue of misuse is well woven throughout
practice guidelines of the book, the topic of social betterment enters late in the conversation. The
section on ethical concerns specific to utilization-focused evaluation is extremely important; the
question of who an evaluator serves is especially critical to the practice role. Chapter 15 presents
two practice tools, one unchanged and one new. The Utilization-Focused Evaluation Flow Chart
appears unchanged from the third edition, a reminder of the core activities and decision points.
A new resource is the carefully detailed Utilization-Focused Evaluation Checklist, a 12-part articu-
lation of tasks, premises, and facilitation challenges that supports the thoughtful practice of
utilization-focused evaluation.

590
Book Review 591

Usefulness in Instruction
The breadth of topics in the fourth edition offers much food for thought for instructors and stu-
dents alike. Large quantities of information are compressed into exhibits, authored by Patton or
adapted with credit from other sources. These resources translate well into presentation slides for
instructional purposes, where discussion and detail can be built around them in classroom interac-
tion. Lengthier summaries such as the six-page menu of Alternative Ways of Focusing Evaluation
(Menu 8.1) or the Utilization-Focused Evaluation (U-FE) Checklist itself (Exhibit 15.3) translate
well into handouts that preserve the level of detail and provide a reference document that may be
cited throughout the course. A simple matrix such as Mapping Stakeholder Stakes (Exhibit 14.1)
could be used as a worksheet to accompany a scenario-based class exercise.
Selected quotes from the text also make excellent slides—witty and engaging—to stimulate class
discussion. My personal favorite is Halcolm’s description of initial reactions to evaluation:

Human propensities in the face of evaluation; feline curiosity; stultifying fear; beguiling distortion
of reality; ingratiating public acclamation; inscrutable selective perception; profuse rationalization;
and apocalyptic anticipation. In other words, the usual run-of-the-mill human reactions to uncertainty.
(p. 197)

There is a high degree of transparency in Patton’s writing—one can see where he is getting ideas,
what he is reading, where he is traveling, and who he is working with. This makes his writing per-
sonal, particularly engaging students who may be meeting him for the first time through this book.
Unlike a traditional textbook, it sets a tone like sitting down for a long chat—another communicative
format in which Patton excels. It is highly accessible, a decided plus for students new to the topic and
quite possibly intimidated by it. Patton puts students at ease by drawing upon both popular and
academic literature. Pairing this book with his personal story (Patton, 2004) also gives students
an excellent opportunity to reflect on how the cultural identifications and experiences of a theorist
shape an evaluation theory.
A new feature of the fourth edition is the presentation of follow-up exercises at the end of each chap-
ter. These are creative activities, designed to stimulate critical thinking and engagement with the chapter
content. Some of these are quite ambitious. As an instructor, I think of an ‘‘exercise’’ as something I can
incorporate into a class session, taking somewhere between 15 and 45 minutes. Although this describes
some of Patton’s follow-up exercises, many pose complex questions requiring more thoughtful reflec-
tion that are best suited for major assignments with instructor guidance and feedback. For example, in
Chapter 1, Exercise 5 begins by suggesting students read the full text of the Program Evaluation Stan-
dards and ‘‘conduct a cultural assumptions analysis.’’ It moves on from there.
Despite its approachable writing style, the usefulness of Utilization-Focused Evaluation as a
textbook is limited by its size. It is lengthy even for a primary text and to pair it with a more tradi-
tional introductory text makes a prohibitive reading load. If one had the luxury of teaching a grad-
uate evaluation seminar on this particular approach or even on a subset of evaluation theories, this
would be an invaluable resource. For general instruction, a primer-length version that could be
paired with other readings would be more useful than this full volume. Happily, this is Patton’s cur-
rent writing project, with an anticipated publication date of 2011 for Essentials of Utilization-
Focused Evaluation.

Usefulness for Theory

Patton takes a pragmatic approach to theory, interweaving theories with practice wisdom to sup-
port his focus on what will affect use in a given situation. Although the practice content of the book

591
592 American Journal of Evaluation 31(4)

is immediately accessible, the reader has to work harder to harvest theory. Not that the theory con-
tent is absent, but it is interspersed with practice wisdom in ways that do not entirely support the
coherence of the theoretical argument. As a theorist, Patton seems a bit impatient, eager to make his
next point rather than carefully and deliberately spelling out logical links. The dedicated theorist
may be frustrated by the leap to an exhibit just as a pithy discussion of a fine point was getting inter-
esting. The result is theory that has more of a ‘‘build-it-yourself’’ quality, engaging the reader in
assembling pieces of the argument. Nevertheless, the text advances conversations on utilization-
focused evaluation’s theoretical foundations and both a theory of use/influence.

Theory of Utilization-Focused Evaluation

Patton’s stance on theory is somewhat fluid. On one hand, he sees the fourth edition as an opportu-
nity to strengthen the theoretical foundation of utilization-focused evaluation, but on the other, he
asserts that utilization-focused evaluation is a process for ‘‘helping primary intended users select the
most appropriate content, model, methods, theory and uses for this particular situation’’ rather than a
theory itself (p. 37). That said, the 15 fundamental premises of utilization-focused evaluation in
Chapter 15 come close to an integrated theory of practice. Together, they speak to the motivations,
principles, practice elements, and cautions deemed essential to utilization-focused evaluation. Inten-
tion remains at the core, as both the driving force (Premise 1) and the focus (Premise 5) of evalua-
tion. Interestingly, Patton’s concern that the point of greatest vulnerability for utilization-focused
evaluation is turnover of primary intended users assumes that the ‘‘right’’ stakeholders were selected
at the outset. The notion of intended users being preordinately selected through careful stakeholder
analysis (Premise 4) versus evolving or emerging over time illustrates an inherent tension within
utilization-focused evaluation. On the one hand, utilization-focused evaluation embraces a goal-
based conceptualization: ‘‘Specifying intended uses is evaluation’s equivalent of program goal
setting’’ (p. 98). On the other hand, utilization-focused evaluation emphasizes situation-specific
responsiveness (Premise 7), which is placed in opposition to goal-based evaluation in Chapter 7. The
practice of utilization-focused evaluation moves fluidly between the two framings; the theory of
utilization-focused evaluation leaves the paradox unresolved.
The personal factor (Premise 3) figures prominently in utilization-focused evaluation, both
historically and currently. The interests and commitments of key individuals shape evaluation
through their active involvement in the process (Premise 8) and the quality of their participation
(Premise 9). Although the time required to conduct utilization-focused evaluation is acknowledged
to be ‘‘far from trivial’’ (Premise 14), the ways in which time requirements constrain the scope of
inclusion are understated. For example, Patton is explicit in stating that a minimum commitment of
eight hours from busy stakeholders is needed to implement utilization-focused evaluation, but the
opportunity costs of such a commitment for a client working two jobs to support a family may preclude
participation. Positioning stakeholder involvement within a task force structure may also limit cultural
diversity of members; situational analysis might suggest other, more culturally appropriate vehicles.
Interestingly, the discussion of deliberate and thoughtful choices among alternate purposes
(Premise 6) does not include social betterment. It is not absent from the book—Patton explicitly
addresses the importance of considering longer term impacts on society in Chapter 14—but it does
not rise to the level of a fundamental premise.
The role of the evaluator in utilization-focused evaluation is powerful—‘‘active’’ in the language
of Premise 11—yet tempered by interaction, respect, and adaptation to incorporate others’ interests
along with those of the evaluator. Toward this end, utilization-focused evaluators must be skilled
group facilitators (Premise 9). Central to the utilization-focused role, evaluators also have a clear
responsibility to educate and train users in both evaluation processes and use of findings (Premise
12). This capacity-building role is wholly consistent with a vision of greater use over time. In the

592
Book Review 593

fourth edition, Premise 15 now explicitly addresses the need to follow up evaluations to find out how
they were used, reflexively improving the use of evaluation by evaluators.
Validity and utility are cast as interdependent in Premise 10, but the book stops short of a thor-
ough discussion of how validity is argued under utilization-focused evaluation. Measurement valid-
ity is addressed, as is validity in terms of causal assumptions, and Patton distinguishes what he terms
‘‘overall evaluation validity . . . from the usual, more narrow conception of validity in scientific
research’’ (p. 397). His discussion of validity includes both traditional and nontraditional elements,
but the presentation is a bit fragmented, ending with the disclaimer: ‘‘This is not a design and mea-
surement text’’ (p. 413). Nevertheless, Patton’s discussion of face validity of both measurement and
design is well worth revisiting and the juxtaposition of validity and utility raises important, unre-
solved theoretical issues. Granted, a comprehensive volume such as this may not be the best venue
for the validity conversation, but it opens the door for a focused article on validity theory and
utilization-focused evaluation that could be a major contribution to the field.

Culture and Utilization-Focused Evaluation

In any evaluation theory, it is important to reflect on how culture is framed and incorporated (Kirkhart,
2005). Patton’s culturally diverse experiences infuse his work; therefore, utilization-focused evalua-
tion illustrates the ubiquity of culture even within theories that are not explicitly culturally posi-
tioned. Utilization-focused evaluation emphasizes complexity and fluidity, which are core
characteristics of culture. The salience of any particular dimension of culture is fluid, shifting with
context. As a problem-solving approach that calls for creative adaptation to changed and changing
conditions, utilization-focused evaluation creates space for cultural reflection and cultural congru-
ence (Kirkhart, 2010).
Utilization-focused evaluation’s emphasis on situational analysis and situational awareness
permits the evaluation to be responsive to context, including but not limited to cultural context.
Situational analysis is generic, with questions organized around four aspects of context: program,
stakeholders, evaluation history, and decision/action context. Cultural factors are nested within
some of the questions, but they are not the main focus. Cultural sensitivity and cultural competence
are presumed to be among the logic and values that undergird evaluation practice.
Utilization-focused evaluation addresses both individual and collective perspectives on culture.
From an individual and interpersonal perspective, respecting and honoring culture is a significant
dimension of the personal factor; involvement of stakeholders and primary intended users has to
be adapted to cultural and contextual factors. Inequalities in power, status, and education are
addressed as factors affecting stakeholder participation and engagement, along with background
variables such as race, ethnicity, gender, disability, and social class. From a collective perspective,
utilization-focused evaluation repeatedly addresses organizational culture and climate in building
an organization’s capacity to think evaluatively. In this vein, Patton notes that evaluation itself
constitutes a culture, making all evaluation practice ‘‘cross-cultural.’’
Culture must also be understood within the exercise of power and privilege. Power is attached in
varying ways and degrees to different dimensions of culture across contexts. Chapter 14, ‘‘Power,
Politics and Ethics,’’ connects use to a theory of power and explores selected intersections of culture
and power, citing examples in which race, gender, and social class are clearly visible. For example,
the quote from Patton’s research study (Patton et al., 1977) in which the evaluator reported tensions
between ‘‘the doctors and more middle class people’’ and ‘‘the blacks and some of the—you might
call them militant liberals’’ (p. 530) is discussed in relation to class, though the intersection with
racism is not named.
Although it is viable to bring culture into theory through a broad framework such as utilization-
focused evaluation, a concern is that one can be situationally responsive and politically sensitive

593
594 American Journal of Evaluation 31(4)

without necessarily stepping outside a majority framework or demonstrating cultural competence.

As Patton continues to write on the theoretical underpinnings of utilization-focused evaluation, more
explicit attention to culture and power will be a welcome addition to his broader vision of a ‘‘shared-
power world.’’ The resources that Patton cites, such as Crosby and Bryson (2005) regarding shared
power, can also deepen the reflection on culture.

Theory of Influence
Interwoven with the practice wisdom of the text and the theory surrounding utilization-focused
evaluation itself are the assumptions and understandings that comprise a theory of use/influence.
Here, there is much we can harvest in consensual agreement and also points of tension that remain
active in discourse and debate.
First, there appears to be considerable agreement that use is to be construed as a process, not a single
report or event. Indeed, the need to consider use from the inception of an evaluation (Premise 2) and
the distinction between use and reporting or dissemination (Premise 13) are deemed fundamental
underpinnings of utilization-focused evaluation. Second, Patton’s early research showing use to be
indirect bears the test of time, a point reiterated in subsequent definitions of influence (Henry & Mark,
2003; Kirkhart, 2000). Use/influence moves through pathways or generative mechanisms (Mark &
Henry, 2004). The practice of utilization-focused evaluation strengthens these pathways, increasing
the likelihood that evaluation will make a difference. Third, there is general agreement that influence
flows from either the process of engaging in evaluation or the findings that result. These sources of
influence may work singly or in tandem. Fourth, there is a general consensus that time is an important
dimension and that some influence is more immediate than others.
Against the backdrop of these considerable areas of agreement, there remain areas of growth and
debate. Intention is a dimension of influence that marks contested space, and Patton has much to
contribute to this conversation, given the centrality of intention to utilization-focused evaluation.
Exhibit 13.13 makes clear the ‘‘gray area’’ between intended and unintended use/influence as it
applies to both findings and process. Yet, the fluidity of the utilization-focused evaluation process
makes it difficult to pinpoint positive examples of unintended process use, because positive organi-
zational change is always framed as intentional, falling within the broad vision of utilization-focused
evaluation even if not represented by a specific leadership development or capacity-building activ-
ity. For all the nuanced discussion of intention, one must not lose sight of its very pragmatic origins
within utilization-focused evaluation. Intention was adapted as a focusing mechanism—intended
users as an alternative to ‘‘multiple audiences, each conceptualized in vague and general terms’’
(p. 518). Theoretical definitions of use/influence are bounded by what the evaluator controls under
utilization-focused evaluation. On this point, Patton is explicit: ‘‘That is the use for which I take
responsibility: intended use by intended users’’ (p. 518). An alternate approach would be to separate
the conversation about what constitutes use/influence from the issue of who is responsible.
The fourth edition adds both practice and theory content on misuse of evaluation. Misuse is cast
as situational and relevant to either results or process: ‘‘Users manipulate the evaluation in ways that
distort the findings or corrupt the inquiry’’ (p. 105). Although he carefully distinguishes misuse from
misevaluation, in the case of process misuse, the distinction is harder to maintain. Corrupting the
inquiry would surely involve failure to adhere to standards and principles, which Patton defines
as misevaluation. Misuse can be intentional or unintentional, and Patton is clear that a continuum
of misuse is separate from a continuum of use. He envisions a continuum of use going from appro-
priate nonuse to appropriate use, while a continuum of misuse goes from inappropriate nonuse to
intentional misuse. This seems to conflate the magnitude of use with appropriateness and intent, but
his discussion of both nonuse and overuse brings additional dimensions to the table that are most
welcome, even if still in their formative stages.

594
Book Review 595

Despite concerted efforts to balance process use and findings use, findings use is still generally
privileged. For example, in laying out definitional distinctions in Chapter 4, process use gets lost
amid an array of fine distinctions among types of findings use. This is not just a statement about the
fourth edition, which carries some historical artifacts from earlier editions, but a reflection of the
current state of our profession. Findings use is assumed unless otherwise noted.
In addition to major themes, the book is full of comments and sidebars that could lead to new
conversations about theory. For example, in discussing the pragmatics of reporting evaluation find-
ings, Patton raises an interesting conceptual point: ‘‘It’s worth remembering, philosophically, that
the positive or negative nature of evaluation findings can never be established with any absolute cer-
tainty’’ (p. 516). It follows that the positive or negative valence of evaluation findings may be of
limited value as a parameter in a theory of use/influence. Might the same uncertainty hold true of
process use? There is much to be gained by continuing to explore similarities and differences
between the two constructs.

Conclusion
Limitations
There are limitations to the book. Some chapters (e.g., ‘‘Implementation Evaluation’’) are not as suc-
cessful as others. Sometimes dated material seems to receive undue emphasis. Not surprisingly in a
text of this length, there are a few noteworthy inconsistencies. For example, the term ‘‘black box’’ is
pointed out in Chapter 2 as an example of metaphors with possible racist and sexist connotations to
be avoided, but that term is then used without comment in Chapter 9 in the title of a practice box and
again in Chapter 12 to name a common criticism of experimental design.
Sometimes Patton’s buoyant enthusiasm leads to overstatement and straw person arguments—for
example, positioning utilization-focused evaluation as the resolution of prior debates between ‘‘para-
digm orthodoxies’’ or presenting a simplistic representation of summative evaluation. Consistent with
his own advice, Patton occasionally lapses into sermonettes in which utilization-focused evaluation is
constructive and alternate approaches are useless and alienating. Sarcasm does not always work as well
in print as in the spoken word.
There is a fine line between being informed and being inundated. Sometimes, the fourth edition
crosses that line with its almost frenetic pastiche of visual displays. Generally, I am a big fan of sum-
mary tables, but some of the tables in this book seem overarticulated, verging on information over-
load. These limitations are overshadowed by the book’s considerable strengths.

Strengths
This book is a good read, with high overall usefulness for the evaluation profession. Although the
book addresses both practice and theory, the practice pieces are somewhat easier to follow, the the-
ories treated more as annotations to his practice wisdom. Classroom application as a required text is
limited by its length, but the book offers a myriad of teaching resources to enrich instruction. The
tendency of bulleted, numbered guidelines to be read as prescriptions is offset by the emphasis on
critical thinking, introducing caveats, concerns, and challenges as well as practice tips.
Usefulness would be enhanced if the fourth edition were available on an electronic reader such as
Kindle or nook so that one could search, highlight, and follow a theme of interest woven all the way
through. This would especially support the use of the book as a source of theory. For all three audi-
ences, the fourth edition may be read at many levels. A practitioner may pick it up to consult a menu
of options or to reflect deeply on ethical issues in his or her evaluator role. An instructor may grab a
quick teaching tool or assign entire chapters to engage students in an examination of evaluation and
change. A theorist may scrutinize the language of a particular definition or study the source materials

595
596 American Journal of Evaluation 31(4)

underlying a logic argument. Whether or not you subscribe to utilization-focused evaluation as an

approach, there is a wealth of experience and insight in this book.
As with any fourth edition, this book contains a mix of old and new, and sometimes the juxtapo-
sition does not sit easily nor the transitions flow smoothly. I raise this not as a point of editorial cri-
tique but to suggest that some of these apparent inconsistencies reflect evolution of the field and of
Patton’s own thinking that offer productive invitations into future conversations. For example, why
does social betterment emerge so late in the volume? Why does meta-evaluation appear stuck in a
very traditional, external, episodic incarnation that seems out of step with the fluidity and reflexivity
of his current thinking on situational analysis? Why, despite concerted efforts at balance, does
findings-based use still dominate process-based use? This book invites the reader to take up these
and other points of discussion, which is a major contribution to our profession. Perhaps surprisingly,
given the length of the book, Patton also achieves his intent of leaving the user/reader wanting more.
For example, I would love to see him revisit the 1990 conversation with House on who an evaluation
serves, given the developments in our profession over the past two decades. Being reminded of this
conversation in Chapter 14 only whetted my appetite.
Finally, I applaud Patton’s attention to making evaluators ‘‘astute and rigorous evaluators of eva-
luation,’’ but the bigger issue may be how to make such evaluations utilization focused. Time to read
the book again through a meta-evaluative lens.

References
Crosby, B. C., & Bryson, J. M. (2005). Leadership for the common good: Tackling public problems in a
shared-power world (2nd ed.). San Francisco, CA: Jossey-Bass.
Gladwell, M. (2002). The tipping point: How little things can make a big difference. Boston, MA: Little Brown.
Henry, G. T., & Mark, M. M. (2003). Beyond use: Understanding evaluation’s influence on attitudes and
actions. American Journal of Evaluation, 24, 293-314.
Kirkhart, K. E. (2000). Reconceptualizing evaluation use: An integrated theory of influence. In V. J. Caracelli
& H. Preskill (Eds.), The expanding scope of evaluation use. New Directions for Evaluation (No. 88,
pp. 5–23). San Francisco, CA: Jossey-Bass.
Kirkhart, K. E. (2005). Through a cultural lens: Reflections on validity and theory in evaluation. In S. Hood,
R. Hopson, & H. Frierson (Eds.), The role of culture and cultural context: A mandate for inclusion, the dis-
covery of truth, and understanding in evaluative theory and practice (pp. 21-39). Greenwich, CT: Informa-
tion Age Publishing.
Kirkhart, K. E. (2010). Eyes on the prize: Multicultural validity and evaluation theory. American Journal of
Evaluation, 31(3), 400-413.
Mark, M. M., & Henry, G. T. (2004). The mechanisms and outcomes of evaluation influence. Evaluation, 10,
35-57.
Patton, M. Q. (2004). The roots of Utilization-Focused Evaluation. In M. Alkin (Ed.), Roots: Tracing theorists’
views and influences (pp. 276-292). Thousand Oaks, CA: SAGE.
Patton, M. Q., Grimes, P. S., Guthrie, K. M., Brennan, N. J., French, B. D., & Blyth, D. A. (1977). In search of
impact: An analysis of the utilization of Federal health evaluation research. In C. H. Weiss (Ed.), Using
social research in public policy making (pp. 141-163). Lexington, MA: D. C. Heath and Company.

596