Unit IV

Public Policy Evaluation

Introduction

Policy evaluation is the process of assessing whether a policy is achieving its intended
outcomes.

'Evaluation' is a more systematic and scientific attempt with emphasis on impacts and
efficiency, effectiveness, relevance, reliability and , sustainability.

Rossi and Freeman (1993) specify it as a systematic application of social research

procedures for assessing the conceptualization, design, implementation, and utility of social
Intervention programmes.

 The goal of public policy evaluation is to determine what the consequences of public
policy are and what hasn’t been achieved.
 The goal of evaluating an implemented program is to determine whether it is
fulfilling its intended function.
 An unfavorable evaluation of a policy’s impact will have a significant impact on the
implementation and life cycle, whereas favorable evaluations will have a significant
impact on the life cycle.
 A policy evaluation is the process of cataloging the goals of a public program and
measuring how well the goals have been met.

Policy evaluation can be more easily defined as the process of determining general goals,
program effectiveness, impact, and cost.

Policy evaluation, in general, refers to the evaluation of public policy’s impact on the real world.
It takes into account both long-term and short-term effects as well as the costs and benefits of
such evaluations.

GRITERIA FOR EVALUATION

Equity: Primarily, equity relates to fairness, uprightness and equality. Individuals and groups
have different values, objectives, and perceptions, - What satisfies one person or group may
not satisfy another,

Efficiency: it is the amount of outputs created and their quality in. relation to the resources
(capital and personnel) invested. It is then a measure of how productively the resources (as
converted into inputs) have been used. Efficiency is generally equated with economic
rationality.
Efficiency is a condition in which goods are produced at the lowest possible cost and in
accordance with the highest preference of consumers.

Effectiveness: focus on the effects, especially on the outcome side

Adequacy: Adequacy refers to whether a given level of effectiveness results in the

satisfaction of needs or, values. Systematic studies based on rigorous methodology are
required to identify the various dimensions of adequacy of certain policies.

Public Interest: Public is interested in the end product or the real outcome of a policy. It has
been rightly pointed out that the public is interested in law, not in the laws, in the methods of
law, not in the substance; in the sanctity of a contract, not in a particular contract; an
understanding based on custom, not in this custom or that.

POLICY EVALUATION: TYPES

Different scholars have identified different types and approaches of policy evaluation,
besides, different methodologies for the exercise.

i ) Process Evaluation

In an exercise of evaluation, policy analysts are concerned with the two questions. Firstly,
whether specific policy has been implemented in accordance with the policy guidelines or
not.

It focuses on two points: whether or not the policy has been aimed and directed at the
appropriate and specific target group or target area; and whether or not the different
practices and intervention efforts based on strategies have been taken up as specified in the
policy design or taken from the principles explicated in such a design.

ii) Impact Evaluation

Impact evaluation attempts to evaluate the changes, both positive and negative in terms of
goals attained. The conditions prevailing before the implementation of the policy and after
are reviewed in order to bring to the fore the impact of policy.

iii) Comprehensive Evaluation

 It is the combination of process and impact evaluation – comparison.

 However, from the point of public policy, it is comprehensive evaluation which is
more useful.
 POLICY EVALUATION: METHODS

Public policy tools are the means by which government officials attempt to influence or
change citizen behavior. The three main types of public policy tools are: 1) legislation and
regulation; 2) taxation and spending; and 3) diplomacy and persuasion.

Evaluation: In another words, it is an Act of ascertaining or fixing the value or worth of

calculation of the numerical value of an expression

i) Cost Benefit Analysis

Cost Benefit or Benefit-cost analysis is the principal analytical framework, which is used to
evaluate public expenditure decisions. Basically benefit-cost analysis requires systematic
enumeration of all benefits and all costs, tangible and in-tangible, readily quantifiable or
difficult to measure, which will accrue if a particular project is adopted.

With all this information at hand, the analyst should be able to subtract the total cost of each
alternative from the total sum of its benefits and identify the net gain in each case.

In most cases, the costs may be fairly realistically quantified. Rossi and Freeinan (1993)
intention, five means of monetizing benefits:

 Direct measurement
 Market valuation
 Economic estimation (indirect quantification based on explicitly stated assumptions)
 Hypothetical questions (asking target people, how they consider basically non-
monetary benefits to be worth in monetary terms).
 Observing political choices (transforming observed political prioritization into some
judged indicated monetary value).

ij) Cost-Effectiveness Analysis

Cost-effectiveness is considered to be a simplified version of benefit-cost analysis. It is a

technique for evaluating various alternatives in terms of tile degree to which they efficiently
lead to the fulfillment of stated objectives. The alterative adjudged most preferable is the one
which produces maximum effectiveness for a given level of cost or the minimum cost for a
fixed level of effectiveness.

Effectiveness Method

 Meaning: Complete the job in particular time, no matter whatever the cost
 Objective: To achieve end result
 Main Consideration: Doing the right task
 Efficiency

 Meaning: Complete the job with minimal Resource (with minimum cost)
 Objective: To conduct cost benefit analysis
 Main Consideration: Doing the task in right away

iii) Statistical Surveys

 Survey method is quite common in social research. In policy evaluation sphere,

surveys are quite useful.
 Information (data) is collected and analysed by applying statistical tools especially
with the help of sampling techniques, questionnaires and interviews supported by
observation method and secondary data.
 Suitable hypotheses may be developed after identifying appropriate variables.
 Groups from the population are selected on the basis of certain characteristics, such
as, the level of education, income, size of landholding, age, sex, social backgrounds,
etc.
 Subsequently, changes are sought relating to programme or policy interventions and
association between degrees of success of the policy different characteristics of the
samples are tested statistically.
 A number of authors also distinguish between three types of studies of change in one
population that is trend design, cohort design and panel design. Such designs arc used
to study the changes from different angles over different time periods.

Evaluating Agencies

i) Evaluation by Specialized Agencies :

A policy at its various stages needs to be looked into by a specialised agency preferably by
outside or independent experts. Certain research organisations, such as, the Centre for Policy
Research, Programme Evaluation Organisation (PEO),and universities may fit into this
category.

ii) Evaluation by Internal Units: Evaluation by internal agencies/units is a common

practice in some countries. Such internal agencies have the full basic knowledge about
the nature and objectives of a given policy. Evaluation by internal agencies is preferred
because it avoids clash with the policy-making agencies. However, internal evaluation
suffers from certain shortcomings.
Firstly, the administrative agencies may lack the skill and capability of investigation or
research. Policy evaluation being aspecialised task is required to be looked into by
experts and not by routine administrators.
 Secondly, the element of bias is always there. As the adage goes, no one should be a
Judge in his own case. In this context, how one can expect objective evaluation by those
who are involved in the implementation of their own policies and programme.
Thirdly, the administrators always resist economic analysis and data based reports.

iii) Evaluation by Legislative Committees; A common practice in democratic countries is

the involvement of legislative bodies in policy evaluation. In India, for instance, we have
a number of such committees, viz., Public Accounts Committee; Committee on Public
Undertakings; Committee on Women Welfare; Committee on the Welfare of SC/STs,
etc.
iv) Evaluation by Special Committees: Appointment of certain commissions or
committees for policy evaluation is a common practice by governments in many
countries. They are usually appointed to conduct a thorough examination of certain
policy measures. Some of the well known commissions in the Indian context are:
National Commission on Agriculture; Law Commission; Education Commission;
Administrative Reforms Commission; Agricultural Prices Commission, Commission on
Welfare of Women and Children; National Health Commission; Mandal Commission;

Challenge or Problems in Policy evaluation

There are many problems that can arise during the process of policy evaluation. One of the most
common problems is that evaluators may not have access to all of the relevant data. This can
make it difficult to accurately assess the effectiveness of a policy. Additionally, evaluators may
not be familiar with the specific context in which the policy is being implemented, which can
lead to errors in their analysis. Finally, political pressure can often influence the evaluation
process, leading to biased or inaccurate results.

1: Poor Planning

Poor planning can lead to not having the right amount of time needed to conduct your evaluation,
a lack of direction in what outcomes you’re hoping to achieve and poor planning can lead to not
having enough resources (i.e. funding, personnel, space, etc.) for your evaluation.

Poor planning can also result in implementation fidelity issues (i.e. how well a program or
intervention is being adhered to) which negatively impacts the integrity of the evaluation and
leads to unintended consequences.

2: Lack of Readiness

If an evaluation isn’t seen as a priority there can be a lack of buy-in from staff and stakeholders
in the evaluation process, which can result in limited resources, uncooperative staff, and an
absence of understanding of why the evaluation is even needed or valuable.
3: Ineffective Approaches

If evaluators don’t use the right data collection methods, you don’t understand how to properly
and correctly identify data, so there will never be an effective or positive evaluation experience.

4: Bad Questions

Asking the wrong questions can derail a project. So, just what are ‘bad’ questions? Questions
that are unclear, that use too much jargon, that don’t take into account the audience, that are
biased in any way, and that don’t have a clear and understandable method for participants to
respond are all problems that will upend the evaluation process.

5: Bad Data

If we ask bad questions, you’ll get bad responses – it’s as simple as that. In addition, if we don’t
properly and cleanly input the data you do get, if there is missing, messy or unorganized data,
then the results will also be messy and unorganized and, ultimately, not useful.

6: Too Much Data

When it comes to collecting data, quality beats quantity in most instances. More data does not
necessarily equate with better data. In fact, the opposite is often true. Large data will takes time
and resources that many programs just don’t have.

Why Is Policy Evaluation Important In Public Policy?

Policy evaluation is important in public policy because it allows policymakers to assess the
effectiveness of their policies and make necessary adjustments. By understanding how well a
policy is working, policymakers can determine whether it is achieving its desired outcomes and
identify areas where improvements are needed. Additionally, policy evaluation can help to
inform the design of new policies by providing insights into what does and does not work.
Ultimately, policy evaluation is essential for ensuring that public policies are effective and
efficient in meeting the needs of the people they are intended to serve.

1) It creates political awareness and will for change with local concerns and perspectives.
2) It complements institutions (government and privet sector) and improves accountability.
3) It provides information to assist in decision making.
4) It creates public ownership of problem and solutions.