Health Services Management Compiled by: Kiddus Yitbarek, 2022

design, implementation and results. Evaluation

Chapter Seven: Controlling:
deals with questions of cause and effect.
Monitoring and Evaluation
Is typically done on a periodic basis; annually
At the end of this chapter, you should be or at the end of a phase of a project or
able to: program.
 Define monitoring and evaluation Evaluation includes areas of context, input,
 Identify the basic purposes and types of process and impact to assess whether the set
monitoring and evaluation objectives have been achieved. It can be
 Identify approaches of evaluating internal, that is carried out by the
 Differentiate between monitoring and implementers, or external.
evaluation
Evaluation answers:
 Understand indicators
 What have we achieved and how?
Monitoring is the routine tracking and
 How relevant were components of a
reporting of priority information about a
program in addressing societal needs?
project or program.
 What were the reasons behind the
Regular systematic collection and analysis of observed levels of performance?
information in order to track the progress of  What is the contribution of a specific
program implementation against pre-set intervention for observed achievements?
targets and objectives. It focuses in particular
Purposes of monitoring and evaluation
on efficiency, and the use of resources.
Monitoring is the collection and analysis of The essence of evaluation is to determine
information about a program, undertaken while program performance, effectiveness and
the program is ongoing. efficiency. In other words, an evaluation can
be carried out to:
Monitoring answers
 Decide whether an activity was worth
 What resources do we have for the
doing.
program?
 Determine whether the objectives set
 What are we doing?
were achieved.
 How does the situation change over time?
 Determine (formative evaluation) whether
Evaluation is the systematic assessment of activities should be continued or not.
actions in order to improve planning or  Determine whether the project should be
implementation of current and future extended elsewhere, etc.
activities. Program improvement:
It is an objective assessment of an ongoing or – Is performed during the entire planning
completed project, program or policy, its process and program execution.

1|Page
 Health Services Management Compiled by: Kiddus Yitbarek, 2022

– Provide solutions for program Normative evaluation usually performed to

improvement. provide managers or users judgment about a
Answers questions such as: program’s compliance to best practices.
 How can the intervention be modified to
Answers questions such as:
achieve its outputs and outcomes?
 Is the program following recommended
 Are there better solutions compared to
those proposed by the program? guidelines?
 Does the program follow national
 How do the components of this program
guidelines?
relate amongst themselves?
 Does the program comply with prescribed
Knowledge generation
norms?
 Usually may not have specifically identified
C. Summative Evaluation:
users of evaluation findings
 Like other research activities, findings may Usually performed to provide judgment to
be shared with interested ones through managers or users about a program’s worth and
publication or other channels. merit
Accountability:
Answers questions such as:
Performed to provide judgment to managers or
users about a program’s worth and merit  Is the program effective?

Answers questions such as:  Should the program be continued?

 Is the program effective?

 Should the program be continued?

Approaches to Evaluation

A. Formative Evaluation:

Formative evaluation is performed during the

entire planning process and program execution.
Provide solutions for program improvement.

Answers questions such as:

 How can the intervention be modified

to achieve its outputs and outcomes? Program components
 Are there better solutions compared to  Input
those proposed by the program? Inputs are resources used in a program.
 How do the components of this program Include financial, human or material resources
relate amongst themselves?
Examples:
B. Normative Evaluation:

2|Page
 Health Services Management Compiled by: Kiddus Yitbarek, 2022

Health workers, Anti-TB drugs, Laboratory Increase of condom use, Improvement of

reagents, IEC materials quality of healthcare, Reduction of risky
sexual behaviors
 Process
 Impact
Process is program procedures that are
implemented to obtain desired effects Impacts are related to long-term accumulative
effects of programs. They are rarely
Examples:
attributed to a single program or intervention.
• Training health workers for counseling and
Examples:
testing
• Screening patients for opportunistic • Reduction in incidence of HIV infection
infections
• Reduction of HIV/AIDS mortality
• Conducting supervision
• Improvement in quality of life of patients
• Educating women
Types of Evaluation
 Outputs
1. Process evaluation
Are the immediate consequences of the inputs
utilized and program activities conducted. It is usually approached as equivalent to
Measured within the setup of service implementation analysis. It supplements the
provision/program implementation, usually monitoring of inputs and outputs with an
using routine program records explanation dimension, enabling the
understanding of the organizational context
Examples:
that affect the program.
• Number of patients treated
Answers questions such as:
• Number of clients counseled
• Number of condoms distributed  Was the intervention implemented
• Number of HIV tests carried out according to what was expected
(compliance)? What is the implementation
 Outcome
degree of the program?
Outcome is effects of a program upon the
 What program/context/users related
target population that can lead to the intended
factors may explain the observed degree
“ultimate goal” of a program that measured
of implementation?
among targets, which could be health facilities,
 Are the planned actions reaching the
clients, or patients. The effects include
targeted population?
several types and may focus on awareness,
 Do the users have access to the
attitudes, behavior, utilization
intervention? Are there barriers that make
Examples: access difficult or unfeasible?

3|Page
 Health Services Management Compiled by: Kiddus Yitbarek, 2022

Example: Was the Project implemented as be used to monitor program implementation

planned? E.g. the capacity building was and achievement of the goals and objectives.
adequate; inputs were available when needed,
Quantitative and qualitative indicators
etc.
Quantitative indicators are numeric and are
2. Outcome evaluation
presented as numbers or percentages.
Provides explanations about why the program
Qualitative indicators are descriptive
did or did not achieve its results. It gives
observations and can be used to supplement
emphasis to causal relations between
the numbers and percentages provided by
intervention and effect.
quantitative indicators. They complement
Answers question such as: quantitative indicators by adding a richness of
information about the context in which the
Does the intervention explain the observed
program has been operating.
effect on the target population?
Why are indicators important?
Example: Was the observed change in condom
use due to the intervention? Why? How? Indicators provide M&E information crucial
for decision-making at every level and stage of
3. Impact evaluation
program implementation.
Analyze the relationships between disease
Indicators of program inputs measure the
trends, control programs and other associated
specific resources that go into carrying out a
factors.
project or program (for example, amount of
Answers questions such as: How much of the
funds allocated to the health sector annually).
change is due to the program?
Indicators of outputs measure the immediate
Example: How much of the prevalence
results obtained by the program (for example,
reduction was due to the Program?
number of multivitamins distributed or number
Indicators for Evaluation of staff trained).

An indicator is a variable that measures one Indicators of outcomes measure whether the
aspect of a program or project that is directly outcome changed in the desired direction and
related to the program’s objectives. They are whether this change signifies program
used to provide benchmarks for demonstrating “success” (for example, contraceptive
the achievements of a program. One of the prevalence rate or percentage of children 12-
most critical steps in designing monitoring and 23 months who received DPT/pentavalent
evaluation (M&E) system is selecting 3immunization by 12 months of age).
appropriate indicators. The M&E plan should
Characteristics of indicators
include descriptions of the indicators that will
A good indicator should:

4|Page
 Health Services Management Compiled by: Kiddus Yitbarek, 2022

 Produce the same results when used
repeatedly to measure the same condition
or event;
 Measure only the condition or event it is
intended to measure;
 Reflect changes in the state or condition
over time;
 Represent reasonable measurement costs;
and
 Be defined in clear and unambiguous terms.

Differences in monitoring and evaluation

Monitoring Evaluation

Frequency Continuous Episodic

Objective Describing Explaining

Method Follows trends, Compares actual Compares achievements with counterfactual

performance with expected

Performed Mostly internal Internal and External

by

Uses Alerts when to take action Provides detailed information on what types
of actions to take

standards, evaluation dimensions and

Stages in the Evaluation Practice
parameters in advance
1. Get the interested parties involved: 5. Explain conclusions: Clarify step by
People involved or affected by the
step analysis/synthesis, interpretation,
program and primary users of the
judgment and recommendations.
evaluation (stakeholders).
6. Ensure use and share lessons learned:
2. Describe the program: Needs, stage,
Feed-back, follow up and dissemination.
context, resources and logical model
What to evaluate in health services
(articulating expected effects and
organizations?
activities)
3. Focus the evaluation (processes,  Service achievement
outcome, or impact): Stakeholders,  Work progress
purposes, questions, uses, methods,  Use of resources
agreements, possible utilization of  Staff performance
findings.
4. Gather evidences with credibility:
Ensure quality of data, Establish

5|Page