Clarifying the Evaluation

Request and Responsibilities

Lisa Dwi Ningtyas
9912819001
 Orienting Questions
Suppose you received a telephone call from a potential client asking if you would do an evaluation

What are some of the first questions you would ask?

are there times you would decline a request for evaluation? If so, what conditions?

How can an evaluability assessment help determine whether an evaluation will be productive?

what are some advantages and disadvantage in having an evaluation conducted by an external evaluator? By an internal evaluation?

What criteria would you use to select an external evaluator?

We discuss how evaluator can better understand
the origin of a proposed evaluation and judge
whether or not the study would appropriate.
Sponsors

Clients

Stakeholders

Audiences
 sponsor
• is the agency or individual that either requests the evaluation or
provides necessary fiscal resources for its conduct, or both.
• Sponsors may or may not actually select the evaluator or be involved
in shaping the study but they often define the purposes of the
evaluation and may specify particular areas that the evaluation should
address or ways in which data should be collected.
• in other cases, the sponsor may delegate that authority to the client .
• The sponsor may be a funding agency or a federal or state
department that oversees or regulates the activities of the
organization that delivers the program.
 Clients
• The client is the specific agency or individual who requests the evaluation.
• That is , the client seeks an evaluator - internal or external - to conduct the
evaluation and typically meets frequently with that evaluator as the
evaluation proceeds.
• In some instances, the sponsor and client are the same, but not always.
• For example, In an evaluation of a domestic violence treatment program
operated by a nonprofit agency, the agency (client) requests and arranges
for the study, but the requirement and the funding may both originate with
a foundation that funds the program and is, therefore, the sponsor.
• In contrast, the sponsor and the client are the same if the program to be
evaluated is a drop - out prevention program for district high schools that is
funded by the school district, and the person requesting the evaluation is a
central office administrator who oversees secondary programs .
 stakeholders
• stakeholders consist of many groups, but essentially include anyone who
has a stake in the program to be evaluated or in the evaluation's results.
• Sponsors and clients are both stakeholders, but so are program managers
and staff the recipients of program services and their families, other
agencies affiliated with the program, interest groups concerned with the
program, elected officials, and the public at large.
• It is wise to consider all the potential stakeholders in a program when
planning the evaluation.
• Each group may have a different picture of the program and different
expectations of the program and the evaluation.
 Audience
• Audiences include individuals, groups, and agencies who have an
interest in the evaluation and receive its results.
• Sponsors and clients are usually the primary audiences and
occasionally are the only audiences.
• Generally, though, an evaluation's audiences will include many, if not
all, stakeholders.
• Audiences can also extend beyond stakeholders .
• They can include people or agencies who fund or manage similar
programs in other places or who serve similar populations and are
looking for effective programs .
What are some of the first questions you
would ask?
Purpose

Users and Use

The Program
Program logic or theory
Resources and
timeframe

Relevant contextual
issues
 Why is this evaluation being requested?
Purpose

What is its purpose?

What questions will it answer?

Users and Use
what use will the evaluation findings be put?

By whom?

What others should be informed of the

evaluation results?
 What is to be evaluated?

What does it include?

The Program What does it exclude?

When and where does it operate?

Who is the intended client for the program?

What are the goals and objectives of the program?

What problem or issue is the program intended to address?

Program logic or theory
What are the essential program activities?

How do they lead to the intended goals and objectives?

What is the program theory or logic model?

What have different stakeholders observed happening as a result of the

program?
Resources and timeframe How much time and money are available for the evaluation?

Who is available to help with it?

What is the timeframe for it

When is final information needed?

Are there requirements that must be met for interim reports?

Relevant contextual issues What is the political climate and context surrounding the evaluation?

Who are the most concerned stakeholders?

What individuals of groups might benefit from a positive evaluation?

Who might benefit from a negative one?

Will any political factors and forces preclude a meaningful and fair
evaluation?
Are there times you would decline a request
for evaluation? If so, what conditions?

Smith (1998)

when the evaluation when it would fail to

could harm the field support the social
of evaluation good.
 Evaluation Would Produce Trivial
Information

Evaluation Results Will Not Be Used

Evaluation Cannot Yield Useful, Valid

In this book
Information

The Type of Evaluation Is Premature

for the Stage of the Program

Propriety of Evaluation Is Doubtful.

How can an evaluability assessment help
determine whether an evaluation will be
productive?
Evaluator can educate stakeholders
what are some advantages and disadvantage
in having an evaluation conducted by an
external evaluator? By an internal
evaluation?
 Advantages of External Evaluations
1. The external evaluation is likely to be viewed as more impartial and
objective
2. The external evaluation is likely to be more credible to outside
audiences, especially if the program is high profile and controversial.
3. External evaluation enables an agency to draw on evaluation expertise
beyond that possessed by agency staff.
4. External evaluators bring with them a new, outside perspective.
5. External evaluators can feel more comfortable than internal
evaluators in presenting unpopular information, advocating program
changes, and working to disclose findings broadly.
 Advantages of Internal Evaluations
1.