You are on page 1of 6

Policy Brief

Strengthening Monitoring and Evaluation System in


Pakistan: The Need of All Development Projects



1. Executive Summary

Government/Development organisations need to know how effective their efforts have been
in order to make their processes efficient and transparent. In order for them to measure their
progress, they need to have strong Monitoring and Evaluation (M&E) systems in place that
are up to the mark with international standards. It is difficult to expect desired outcomes
from implemented policy until effective M&E system is in place. Government of Pakistan
has taken steps to place M&E systems but much potential has to be harnessed yet.
Institutional, structural, cultural, technical, and capacity reforms are required in order to
revitalize the potential of M&E and our projects.


2. Introduction

A government M&E system needs to support effective and cost efficient use of public
funds, including the attainment of government development objectives, and it needs
increasingly to support the policy making process, particularly in regard to public
expenditure. The need for M&E system is in line with the Vision 2025 objective
Institutional Reforms and Governance.

The Government of Pakistan recognises the importance of M&E. For example, the Medium
Term Development Framework (MTDF1) states that: In-depth monitoring and impact
evaluation information is essential to ensure the relevance, efficacy and efficiency of the
resources spent for poverty reduction. GoP has also established the Strengthening of
Projects Wing of P & D Division project in the Planning Commission, which has a budget
of Rs 39.3m. Donor assisted initiatives have also been taken in Pakistan.

However, Governments own assessment of its capacity for in-depth M&E provides a
varied picture. In regard to poverty monitoring, the poverty monitoring framework in the
MTDF focuses on regular tracking of input, output and outcome indicators. However, the
MTDF assesses that: the present capacity for poverty monitoring and reporting progress is
by and large inadequate. The quality and regularity of survey information are weak. The
capacity to analyse statistical information is marginally better. Hence, Achieving poverty
reduction targets of MTDF would require an effective monitoring system that regularly
measures their implementation and impact.

It has been assumed that readers know what functions do a government M&E system needs
to perform. Therefore, the focus of this policy paper would be to present approaches and


results of current M&E system and will provide some recommendations in order to address
those weaknesses discovered through analysis.

3. Approaches and Results in current M&E systems in
Pakistan- evaluation of strengths and weaknesses

GoP has approached to address the problems of M&E in different ways which has resulted
in its strengths. These include:

in the Planning Commission, the newly strengthened Projects Wing, which in 2006/07
undertook implementation monitoring of 500 development projects representing 92% of
the PSDP, and ex-post evaluation of 20 projects
the establishment of M&E Cells in a number of Line Ministries
the PRSP monitoring system, which has a structure of input, intermediate and outcome
indicators, and for most of the life of the PRSP-I published regular monitoring reports on
the internet
the publication of an Medium Term Budgetary Framework (MTBF) blueprint and new
Budget Call Circular which pave the way for output-based budgeting and create the
demand for better outcome monitoring across key Ministries
enthusiasm for managing for development Results (MfDR) and Results based
management (RBM) amongst a group high level officials in the Planning Commission,
Ministry of Finance and EAD, who are part of ADBs MfDR initiative for south Asia


These strengths/past efforts offer the basis for further development of the M&E system.
However, there is need to analyse the Results of these efforts in order to identify
weaknesses so that we can address them later. There are Weaknesses in the Poverty
Reduction Strategy (PRS) monitoring system as well. Apart from the recent hiatus in PRSP
reporting, and the delay in publishing the PRSP II, there are gaps in both approach, data
availability, and institutional environment
2,3
.
Critically, although PRSP monitoring covers an indicative results chain of input,
intermediate indicators and outcome indicators, there is no impact evaluation of the effect
of inputs and progress on intermediate indicators on output indicators. There is a need to
better link inputs to outcomes.
A range of organisations are involved in different poverty monitoring and assessment
aspects of the PRSP, including Ministry of Finance, CRPRID, Federal Bureau of Statistics
and PIDE. However, coordination between them is poor.
The sources of data for some indicators are yet to be established, for example, governance
Progress has been slow, in spite of the Decentralisation Act, to develop the PRSP
monitoring mechanism at the district level.
The PRSP Secretariat has not developed the follow-up mechanisms - this requires that
results of intermediate and outcome indicators are disseminated among stakeholders in
order to have regular feedback to policy makers for appropriate adjustment in policies.
The Federal PRSP Secretariat is challenged in monitoring intermediate input and output
indicators because of issues concerned with availability, timeliness and quality of data.

2
GM Arif (2005). Pakistan: PRSP Monitoring Indicators and Process. Presentation at ADB National Poverty
Reduction Specialists' Exchange, ADB Headquarters, Manila, Philippines: 11-13 October 2005.
3
GoP and UNDP (2007). Strengthening PRS Monitoring, 2008-2012. Project Document.
The management information systems developed by administrative departments in the
provinces are not integrated nationally and there are major data related discrepancies, with
different agencies using different definitions. Other reasons for this poor and delayed
response include: (a) the lack of authority of the PRSP Secretariat over reporting agencies
to demand submission of information on a timely basis; (b) the absence of formal channels
for the flow of information; (c) lack of incentives for staff to make data available; and (d)
the limited processing and editing capacity and the poor quality of technology of the
management information systems (GoP / UNDP, 2007).
Poverty and MDG monitoring is also marred with issues of compatibility
4
, consistency
and insufficient level of disaggregation. These issues of inconsistencies are common in
administrative data versus survey data, posing a greater challenge for effective monitoring
(GoP / UNDP, 2007)..
And there are significant institutional issues, identified by UNDP (GoP/UNDP, 2007):
The existing statistical system is plagued by issues of limited capacity,
fragmentation, weak coordination and reluctance to share data.
To date, limited analytical work has been done or capacities created within the
government to analyze data and evaluate impact of poverty policies and programs.
Problems connected with interagency coordination (between Finance & Planning, for
instance) have also prevented capitalizing on what limited in-house capabilities that
the GOP has at its disposal to analyze data and survey results.
Finally, there is a significant issue, even PRS monitoring systems are improved, of use of
the monitoring information. As the UNDP states: Monitoring should not become an
end itself. The real issue is effective follow-up and implementing remedial measures and
reinforcement of accountability for results. It requires that results of intermediate and
outcome indicators may be disseminated among stakeholders in order to have regular
feedback to policy makers for appropriate adjustment in policies.






















4
E.g. For example, the 1998 Census asks whether respondents did any workin the last year. The Labor Force Survey
(LFS), in contrast, asks whether the respondent did any workduring last week and the Pakistan Social and Living
Standards Measurement Survey (PSLM) asks, during the last monthBecause the relevant reference period is different,
among other things, labour force data from these three sources are not compatible and cannot usefully be integrated into
the same time series to monitor trends over time.
4. Conclusion

Assessment by ADB of the enabling environment for MfDR shows a number of areas for
attention:

Figure3: Enabling Environment Map for Pakistan
5



These areas for development may be summarised as follows; there is a need for:
a joined-up vision of M&E across government, rather than separate pockets being
developed in isolation
clear leadership from within Government on M&E
improved project monitoring, which needs to become more results-focused
redressing the monitoring vs evaluation imbalance to allocate more effort and
importance to evaluation
closing the loop between planning budgeting results i.e. using evaluation and
impact assessment results to inform new cycles of planning and budgeting, thus
moving away from iterative and incremental approaches to planning and budgeting
wide-spread capacity building for both evaluation and results-focused monitoring
stronger accountability

The demand for stronger accountability of government organizations is growing day by day
in Pakistan. GoP need to realize that Projects must be monitored through more efficient
approach (e.g. RMB) in order to take decisions that are most aligned with MDGs and Vision
2025. Efficient use of resources through strengthening the M&E system can be a viable
solution in order to achieve desired targets. Needless to say that all other policies, be it on
poverty eradication, food security, education or health, cannot be implemented with desire of
efficient results until and unless the root mechanism of running project, i.e. M&E, is not
fixed.

5
RETA 6306 Team - McMillan, L., Woods, B., Pokharel, C., Rana, A., Yasmin, F., Rabbani, M., Zia, A.Z., Abeykoon, P.
(2007). Assisting the Implementation of MfDR in Government Organizations - Mainstreaming Managing for Development
Results (MfDR) in Support of Poverty Reduction in South Asia. Discussion Paper (Draft) for RETA 6306 Regional Forum on
Mainstreaming Managing for Development Results (MfDR), 13-14 November 2007, Bangkok.

5. Recommendations

Overall, the recommendations for improving M&E in Pakistan centre around five main
areas:
1. Institutional - providing an overarching enabling environment for RBM and results-
based M&E
This requires leadership from both GoP and donors. Other studies
6
have
identified the lack of leadership for aid effectiveness actions with respect to
the GoP, as well amongst donors.
This is a need for advocates for RBM, and if possible a senior RBM champion
in Government
Initiation of a change management process to accelerate the shift to results-
based management and M&E
The Medium Term Budgetary Framework (MTBF) is the main driver for this
at present.
This range of initiatives will strengthen the demand for better, results-based
M&E

2. Structural - establish high level structures for improved M&E / RBM
These are all significant institutional changes:
The MTBF proposal for a single apex government M&E agency located in the
Planning Commission is supported by this study
A single government results framework also needs to be in place the
perceived in distinction between the MTDF and the PRSP needs to be resolved
The link to the outputs in the MTBF need to thence be established
Donor targets need to link to a unified Government results framework

3. Cultural - central to strengthening M&E in Pakistan is the need progress from
implementation approaches to results approaches.
This is true for both the wider approach to project and programme
management and for M&E specifically
The phrase M&E is very closely equated with the compliance checking
aspects of traditional, implementation-focused project M&E for very many
people in Government. It is equated to project input monitoring, with a
particular bias towards infrastructure investments.
It is recommended that in strengthening M&E, that Government and donors
attempt to signal the new approach by using new terminology it is proposed
that the results-based phraseology is employed to send this signal. Thus
strengthened M&E would be part of an RBM initiative.

4. Technical - improving the basic M&E machinery
This can be done fairly quickly and cost-effectively
Much of this focuses on revision of exist Planning Commission proformae
The key areas of attention would include:
Developing a single glossary of RBM and M&E terminology
Introduction of results frameworks in PC-1

6
Tony Killick and Qaim Shah (2005). Improving Aid Effectiveness in Pakistan. Issues Paper for the Pakistan Development
Forum, Islamabad, 10-11 May 2006. D-PRWG.
Utilising ex-ante results frameworks in project reviews
Improving approaches to evaluation

5. Capacity - Strengthening the supply of professional RBM specialists
Develop a single GoP capacity building package for RBM
Recognise RBM specialism in the GoP cadre (it is not the same as economics)
Explore either establish a Pakistan Evaluation Society (PES), or providing
support to the Pakistan Evaluation Network.

These 57 recommendations should not be seen in isolation of each other and must be
implemented holistically in order to take full advantage. However, there needs to be some
prioritization on the basis of urgency and importance. This needs further input from
Government to indicate what are their priorities.


6. Further Readings

1. PAKISTAN: Strengthening and Harmonising Monitoring and Evaluation (Volume 1,
2) by Julian Barr, Dr M.K. Niazi and Safiya Aftab
2. PARTICIPATORY MONITORING & EVALUATION: LEARNING FROM
CHANGE by Institute of Development Studies, 1998 ISSN 1360-4724



This policy brief is written and edited by Ammar Ali Ayub. The research material was taken from field study,
various reports, inputs from honourable professors of Lahore University of Management Sciences (Dr. Ahsan
Rana among others) and M&E specialists at Higher Education Commission, Pakistan. The author can be
reached through following contact details.
Ammar Ali Ayub
BSc Hons Economics and Public Policy
Lahore University of Management Sciences (LUMS)
Email: ammaraliayub@gmail.com
Mobile: 0343-522-1582