Barbara Lee

Theories of Evaluation

I was visiting a research facility recently, which has achieved a modest national reputa-
tion in the United States. It receives millions of dollars each year in grants and contracts
to do mental health evaluation. Two years ago, several senior faculty decided to de-
velop a program designed to offer graduate students at the University where they were
co-located, a certificate program in evaluation.
As I walked through the main lobby, I overheard the chairman of the curriculum
committee for the certificate program talking with avisitor. The visitor was wondering
out loud what evaluation really was, and what it would mean for someone to be .. a cer-
tified evaluator" , when they completed the certificate program.
The chairman responded to the visitor , smiling. .. You can do anything you want
with it. No one really knows what evaluation is anyway."

1. Introduction

It is one of the interesting realities of evaluation, that even after over 30 years
of discussion and thought ab out this emerging discipline, most of the people
doing evaluation still come from training other than as evaluators. Further-
more, the well-known writers in the field of evaluation theory did not receive
their formal education in a program designed to produce professional evalu-
ators, although these have been in existence since the late 1970s. Some of
this, of course, can be attributed to the relative newness of the profession.
Many of the people who shaped the emerging discipline entered it in early or
mid-career, mostly from the social sciences, and their interest in evaluation
has been long-lived. Their ideas have, for the most part, continued to mature
and be elaborated following the energizing debates that have characterized
emergence of the fieId. Yet, it is still curious, that some of the people with
well-established careers in evaluation, can comfortably admit that they don 't
know how to define just what it iso
On the positive side, this has resulted in a wonderful level of creativity
and diversity in evaluation methods and thinking. Those educated in sOciology,

R. Stockmann (ed.), Evaluationsforschung

© Leske + Budrich, Opladen 2000
128 Barbara Lee

psychology, education, economics, business, and philosophy have brought

their particular skills, methods, and professional perspectives to bear on the
need to measure and judge value of social programs, where they are tested in
the arena of harsh reality. Also, many evaluators have moved to this profes-
sion after years of work with the kind of programs within which they are now
conducting evaluations. The value of this experiential expertise about what is
being evaluated has brought about a general assumption that this is a neces-
sary resource that must be present in any evaluation team. The down side is
that relatively few people think inclusively about what evaluation theory iso
Even those who are very experienced in doing evaluations easily oversim-
plify what the field is, apply research methods borrowed from other fields
naively, or make evaluation practice (and therefore theory) seem undefined,
as the committee chairman did in the example above.
This has been a persistent issue over time. Guba (1969) identified lack of
definition of the field and lack of evaluation theory, as two of seven impor-
tant gaps in the discipline of evaluation. In 1980, House still characterized the
state of evaluation practice as marked by vitality and disorder, and Cronbach
(1980) also spoke about the need for an evaluation profession that was clear
ab out its role in society and about the nature of its work. Cronbach saw the
social importance of evaluation as enormous, but saw its self-understanding
as relatively minute. Eleven years later, Shadish (1992) responded to a re-
view of his book on evaluation theory (ShadishlCookILeviton 1991), stating
that there was even then, not much good theory in evaluation. As late as in
1997, questions were raised about the distinction, if there was one, between
research and evaluation. The extended discussion of this topic on EV AL-
TALK. an on-line discussion list sponsored by the American Evaluation As-
sociation, demonstrated once again that there is not yet fuH consensus on how
to define the discipline of evaluation.
Nevertheless, there has been a great deal of elaboration in the past two
decades around the early definitions of evaluation. Also, from Cronbach's
(1980) simple definition of evaluation as the process by which a society
leams about itself, through many variations by other theorists, several com-
mon elements have emerged. These will be considered using Shadish, Cook,
& Leviton's (1991) organization of evaluation theory into five components;
social programming, knowledge, value, use, and practice. We note that the
apparent content of three of these is certainlr sufficient in itself to distinguish
evaluation from its kissing cousin, research, which addresses only construc-
tion of knowledge and research practice. However, the defmition which is
most straightforward, while including the most common and persistent ele-

There is a saying in self-help organizations that is often referred to as the .. KISS" prin-
ciple - keep it simple, stupid! The aHusion here is to the relative simplicity of defining
research as systematic inquiry, compared to evaluation, although evaluation professio-
nals need to have a reasonably simple description of what the discipline is, as weH.