MONITORING AND

EVALUATION
FRAMEWORK
Piloting a Monitoring Framework - A Preliminary
Report

A framework for M&E was developed by the National Steering Committee

for Prevention of Blindness with the assistance of Sightsavers
International. The main aim of this framework is to establish a practical
monitoring and evaluation mechanism, tools and standards that support
the establishment of a national eye health management information system
as an integral part of the National Health Management Information
System.

Pakistan Country Office Team

1/14/2008
 1/14/2008

Contents
Monitoring And Evaluation ..................................................................................................... 2

Introduction ................................................................................................................................ 2

Approach ..................................................................................................................................... 3

Objectives ..................................................................................................................................... 4

Process And Achievements ...................................................................................................... 5

General background ................................................................................................................... 5

Approach adopted....................................................................................................................... 7

Phase of Sensitization ............................................................................................................. 7

Phase of Policy Change ............................................................................................................ 7

What Next ..................................................................................................................................... 9

Activities ................................................................................................................................... 9

Annex 1. Sample of District Reporting Format ................................................................ 11

Annex 2. Mapping of National Programmes (Sri Lanka PEC Meeting 2007) ............ 13

1
 1/14/2008

MONITORING AND EVALUATION

Piloting a Monitoring Framework – A Preliminary Report

Introduction
Monitoring can be defined as: “A continuing function that uses systematic collection
of data on specified indicators to provide management and the main stakeholders of
an ongoing development intervention with indications of the extent of progress and
achievement of objectives and progress in the use of allocated funds”. Thus
monitoring embodies the regular tracking of inputs, activities, outputs, outcomes
and impacts of development activities at the project, program, sector and national
levels.

Evaluation can be defined as “the process of determining the worth or significance of

a development activity, policy or program ….. to determine the relevance of
objectives, the efficacy of design and implementation, the efficiency or resource use,
and the sustainability of results. An evaluation should (enable) the incorporation of
lessons learned into the decision-making process of both partner and donor”.

Monitoring and evaluation are synergistic. Monitoring information is a necessary

but not sufficient input to the conduct of rigorous evaluations. While monitoring
information can be collected and used for ongoing management purposes, reliance on
such information on its own can introduce distortions because it typically covers only
certain dimensions of a project’s or program’s activities, and careful use of this
information is needed to avoid unintended behavioral incentives. In contrast,
evaluation has the potential to provide a more balanced interpretation of
performance. But evaluation is a more detailed and time-consuming activity, and
because of its greater cost it needs to be conducted more sparingly. One approach is
to rely on monitoring information to identify potential problem issues requiring
more detailed investigation via an evaluation.

2
 1/14/2008

Approach
Sightsavers team worked with the National Committee for Prevention of Blindness
(NSCPBL) to identify the strategic components of the Eye Health Management
Information System (EHMIS). Once identified, these would serve as the critical
milestones for strengthening the EHMIS.

3
 1/14/2008

OBJECTIVES
1. To establish a standardized and practical M&E mechanism, tools and standards
to support the establishment of a national eye health management information
system as an integral part of the National Health Management Information
System.

2. To develop a programme specific management information system having

mutually reinforcing linkages with the National HMIS Programme both at the
level of data collection, transmission, analysis and feedback.

3. The development of a standard national database regarding eye health.

4
 1/14/2008

PROCESS AND ACHIEVEMENTS

General background

Institutionalization of an M&E system can be said to have occurred when it

produces monitoring information and evaluation findings which are judged valuable
by key stakeholders, when this information is used to improve government
performance, and when there is sufficient demand for the M&E function to ensure
its funding and its sustainability for the foreseeable future.

There are many potential uses of M&E information by a government and by civil
society. For this potential to be achieved, it is important that the conduct and use of
M&E be made systematic. This ensures the regular flow of M&E information to
support ongoing government processes such as the budget cycle, the management of
government programs and projects, and regular reporting of government
performance to the parliament or congress.

Most governments already conduct a range of M&E activities, such as ministry data
systems. But these are often uncoordinated, with the result that a single ministry
can possess several such systems, each with their own data definitions, data sources,
periodicity of collection, and quality assurance mechanisms (if any). National
statistical offices draw on the data produced by sector ministries, and they also add
to the supply of information through, for example, household surveys, industry
surveys and population censuses. This situation can be described as “too much data,
not enough information”. A diagnosis of data systems and capacities provides a
starting point for rationalizing data collections and improving their quality.

In the Ministry of Health, there are three information streams. Firstly, the Primary
Health Care (PHC) programme collects its own information from Lady Health
Workers (LHWs). The flow of information is from LHWs to Basic Health Units
(BHUs) to district programme implementation units (PIUs) to provincial PIUs to the
Federal PIU. Similarly, the national programme for Health Management
Information System (HMIS) collects data from the First Level Health Facility (e.g.
rural health centre, BHU) and through the district HMIS PIUs to provincial HMIS
PIUs to the Federal HMIS PIU. The national programme for prevention of blindness
has just begun collecting district eye health data from some districts.

5
 1/14/2008

The three national health programmes are presented in the diagram below.

6
 1/14/2008

Thus, there was a pressing need to link these programmes and coordinate eye health
information being provided by these different information streams. Furthermore,
there was no analysis of data from the different implementation units being done at
district or provincial levels that would aid informed planning for eye health.

Approach adopted

Sightsavers approach to programmes has been to institutionalize systems and

processes for long term sustainability and effectiveness. The interventions are
targeted at three levels – policy, planning and implementation.

Phase of Sensitization

The need for an Eye Health Management Information System was first discussed
with the NSCPBL who endorsed the idea and recommended that M&E support
systems be developed. To ensure integration, advocacy was done to facilitate
membership of the national coordinators (or their representatives) of the national
programmes for Health Management Information Systems and Primary Health
Care. Their participation helped highlight the need for integrating information
streams.

Phase of Policy Change

The strategy was initiated by holding a meeting of the NSCPBL to review the
different information needs and based on experience of different comprehensive eye
care programmes develop a reporting format for district eye units (Annex 1). This
was field tested successfully in 4 districts and information is being collected and
collated on a monthly basis.

The second strand of the strategy was to review the information being collected
by the Primary Health Care programme. As a result of successful advocacy,
information being collected by LHWs on eye health at primary level was obtained
from the provincial PIU in Punjab and the Federal PIU. This demonstrated that
LHWs were feeding in information about people examined and treated with eye

7
 1/14/2008

disease. However, this information was not appearing on the formal reports of the
programme that were submitted to the policy makers.

The third stratagem was to invite the provincial Special Secretary for Health of
Punjab province and the provincial coordinator for the PHC programme from the
same province to a SAC Regional Meeting on Primary Eye Care that was held in Sri
Lanka. Their active participation helped map out the programmes (Annex 2),
identify strategic integration points and secure their support for the process. They
have now become advocates for integration.

To lend credence to the advocacy initiatives, two project proposals were prepared
that targeted capacity building of the supervisory chain of Lady Health Workers and
the training of LHWs respectively. These were supported by Irish Aid and Standard
Chartered Bank and served as the fourth element in the strategy to help
leverage policy change.

The fifth component of the strategy was to convince the Federal Secretary for
Health of the need to maximize the use of existing health resources. The information
and evidence available was presented in a way that demonstrated how health
systems strengthening and improved governance could be brought about through
integrating three national health programmes and enhancing their coordination and
planning. The Federal Secretary Health approved the integration of the three
national programmes and issued a notification for inter-departmental collaboration.

In parallel to these initiatives, capacity building of Country Office staff was also
taking place. This was done over two years and achieved the following:

 Orientation about M&E systems in general

 Preparation of an M&E Matrix to assist in conceptualizing a M&E system
 Training in Participatory Rapid Appraisal (PRA) techniques
 Training in use of LogFrame as an M&E tool
 Training in developing reporting indicators
 Development and field testing of the reporting format for the NSCPBL

8
 1/14/2008

WHAT NEXT
Activities

The following activities are planned with the National Coordinators of the Primary
Health Care, Prevention of Blindness and Health Management Information Systems
programmes.

Situation Analysis Phase (3 Months)

o Review of existing data sources on Eye Diseases
o Collection/consolidation and analysis of existing data and development of
a National Report
o National dissemination/result sharing workshop.
o Study visit to National Health Information System - Oman

System Development Phase (3 Months)

o Information system development / consensus building workshops
o Hiring of consultant for development of tools/data collection procedure
and instruction manual
o Finalization of tools/data collection procedure and instruction manual
o Development/linkage of HMIS Software/pilot testing
o Training/pilot testing of integrated system in 10 districts

System Implementation / Operational Phase (2 years)

o Training of Master Trainers on newly designed system (1 workshop)
o Training of Computer staff (District/Hospitals) in software applications (1
workshop)
o Training of District/Hospital staff on newly designed system (5 workshop)
o National wide infrastructure/networking, logistics, supplies and printing

System Consolidation Phase (1 year)

o Development of Reports/Publications
o Implementation review and data sharing meetings

9
1/14/2008

10
 1/14/2008

ANNEX 1. SAMPLE OF DISTRICT REPORTING FORMAT

11
1/14/2008

12
ANNEX 2. MAPPING OF NATIONAL PROGRAMMES (SRI LANKA PEC
MEETING 2007)

Strategic Integration Points identified by Red Circles