2017

CONCEPT PAPER ON MONITORING AND

EVALUATION
Ministry of FinanceSYSTEM FOR PFM STRATEGY IN
ALBANIA
1
Table of Contents
1. PFM SUMMARY OF KEY POINTS....................................................................................................................... 4
2. INTRODUCTION............................................................................................................................................... 5
3. REVIEW OF THE MANAGEMENT AND MONITORING MECHANISM OF PFM STRATEGY AND ACTION PLAN.........5
3.1. PFM MANAGEMENT, MONITORING AND EVALUATION INSTITUTIONAL SET-UP............................................5
3.2. MONITORING AND REPORTING SYSTEM OF THE PFM STRATEGY IN ALBANIA...............................................6
3.3. POLICY AND REFORMS EVALUATION OF THE PFM STRATEGY IN ALBANIA....................................................9
4. PROPOSALS FOR MANAGEMENT, MONITORING AND EVALUATION OF THE PFM STRATEGY...........................11
4.1. PROPOSALS FOR THE PFM STRATEGY MONITORING AND REPORTING INSTRUCTIONS................................11
4.2. PROPOSALS FOR PFM PROGRESS REPORTS................................................................................................ 19
4.2.1. SEMI-ANNUAL PROGRESS REPORT......................................................................................................... 19
4.2.2. ANNUAL PROGRESS REPORT.................................................................................................................. 19
4.3. PROPOSALS FOR PFM STRATEGY AND ITS ACTION PLAN MID-TERM EVALUATION......................................19
5. Next steps..................................................................................................................................................................20
List of abbreviations

EU European Union
DoPA Department of Public Administration
DPDFA Department of Programming, Development and Foreign Aid
DMRPFM Directorate for Managing the Reforms in the Public Finance Management
HSC High State Control
IPS Integrated Planning System
MoF Ministry of Finance
MTBP Medium Term Budget Program
NGO Non-for-profit Organisations
OPM Office of the Prime Minister
PAF Performance Assessment Framework
PFM Public Finance Management Reform (PFM)
SC Steering Committee
SPU Strategic Planning Unit
SPC Strategic Planning Committee
Concept Paper on Monitoring and Evaluation System 2017
for PFM Strategy in Albania

1. PFM Summary of Key Points

The Public Finance Management Strategy 2014-2020 (PFM) was approved by the Government of Albania in
October 2014. The overall objective of the PFM reform strategy is to achieve a better balanced and
sustainable budget with a reduced debt ratio through stronger financial management and control and audit
processes and where budget execution is properly linked to Government policies.
The main thematic priorities over the medium term can be summarized in the following bullet points:
 Prudent macroeconomic framework and fiscal policy with the objective to decrease the debt/GDP
over the medium term;
 Elimination of arrears and prevention of their reoccurrence;
 Tightened commitment control, control of multiyear commitments and pre-commitments and an
enhanced financial control system;
 A prudent, well-functioning multiyear budget process;
 Strengthened revenue collection, and compliance, with the objective to decrease tax evasion and
the tax gap;
 A well trained and capable internal audit function;
 The High State Control (HSC) auditors will operate in line with international standards for audit
supreme institutions;
 Increased transparency and better accountability mechanisms;
 Efficient public procurement system improving the quality of public spending.
Implementation of the PFM Strategy is an important process therefore a good monitoring system must be
designed at the first stage. This design shall take into consideration the followings:
 The key elements of Public Finance Management Reform (PFM) management structure and
institutional set up have been established in the Public Finance Management Strategy 2014 - 2020
(PFM) and its Action Plan. However, there is a need to review the efficiency of the functioning of
the PFM management structure in practice.
 The key elements of PFM monitoring and reporting have been established in the document of the
PFM Strategy. However, there is a need to define concrete steps for the preparation of the semi-
annual and annual reports; the deadlines for the preparation of the reports and the reporting
formats and templates to be used. The steps, deadlines and reporting templates should be
determined in the Instruction for PFM monitoring and reporting which should be prepared by the
Directorate for Managing the Reforms in the Public Finance at the Ministry of Finance (MOF).
 To facilitate the presentation of PFM reporting information on the progress of reforms to the PFM
Steering Committee it is proposed to define Semi-annual and Annual Progress Report structure in
advance in order to get the required information from ministries, structures and institution
involved in implementation of the actions foreseen in the mid-term plan of the Strategy.
 Based on the type and timing of the evaluation, it is recommended to determine the following
scope of the PFM strategy and the Action Plan evaluation:

4
Concept Paper on Monitoring and Evaluation System 2017
for PFM Strategy in Albania
a. The implementation of Strategy and Action Plan (effectiveness);
b. The adequacy of the interventions to achieve the desired results and objectives (relevance);
c. The adequacy and efficiency of the PFM management structure (efficiency)
The preparatory works for evaluation should start early in 2017.

2. Introduction
As a rule, monitoring and evaluation of public policy in the public sector are the weakest phases of policy
cycle in practice in many countries. Furthermore, to have a monitoring and evaluation system which would
be used practically in technical and political decision making system requires dedicated efforts, expertise,
skills, appreciation and mature management, as well as leadership. However, public policy monitoring and
evaluation system does not grow in a year or even in several years. Very often it starts from administrative
procedures to provide reports on the implementation of different plans or programmes; it then gradually
improves in the quality of information generated in the process and eventually this information is used in
decision making system.
Monitoring of the implementation of the strategies is done from different structures in Albania, both
functional structures and inter-ministerial coordination structures established for specific strategic areas.
The primary responsible unit for monitoring is the Ministry of Finance. Within the Ministry a recently
created Directorate for Managing the Reforms in the Public Finance will be responsible for coordination
and monitoring of the reform. In addition the PFM Strategy is monitored by the Steering Committee who is
the oversight committee with responsibility for directing and monitoring PFM reform activities. A Technical
Committee will be established as well to catalyse the reform activities. Finally, the Strategic Planning Unit
(SPU) under the Department of Programming, Development and Foreign Aid (DPDFA), in the Office of the
Prime Minister (OPM) is responsible for overall monitoring of strategic framework in Albania.

3. Review of the management and monitoring mechanism of PFM Strategy and Action
Plan

3.1. PFM management, monitoring and evaluation institutional set-up

The PFM Strategy has a four-level PFM management and monitoring institutional set-up starting from
political level and ending with the role of the lead institution – the MOF.
The first level consists of the lead institution, the MOF to co-ordinate the PFM across participating
institutions and Contact Persons (CPs) in public administration bodies to co-ordinate PFM implementation
with MOF. The MOF is exercising this function through the recently established Directorate for Managing
the Reforms in the Public Finance (DMRPF). The function of the first level is the operational management of
the PFM strategy on a daily basis.
The second level is the PFM Reform Technical Committee which consists on:
 The Secretary General of MoF
 Pillar and component leaders:
The Secretary General of Ministry of Finance will chair the Committee. The Technical Committee will
propose persons to be responsible for the different components under each pillar.

5
Concept Paper on Monitoring and Evaluation System 2017
for PFM Strategy in Albania
The third level consists of the Steering Committee which is established to ensure the effective
implementation of the reform strategy and to facilitate coordination of efforts. The Steering Committee
was established by the Prime Minister Order no. 202 dated 25.8.2014. The Steering Committee is chaired
by the Minister of Finance, and its members are as follows:
 The deputy Minister of Finance;
 The deputy Minister of Social Welfare and Youth;
 The deputy Minister of the Ministry of European Integration;
 The Minister of State on Local Issues;
 The Head of the Public Administration Department;
 Two representatives from the Department of Programming, Development and Foreign Aid at the
Prime Minister Office;
 The Head of the Procurement Agency;
 The Head of Public Procurement Commission
The fourth level is the Strategic Planning Committee (SPC). The SPC was established in 2006 as part of the
Integrated Planning System (IPS) and is the main high level decision-making committee for strategic
planning process in Albania. The SPC will be informed on the overall process of implementation of PFM
reform through reports that will be submitted by the Steering Committee.

3.2. Monitoring and reporting system of the PFM Strategy in Albania

In order to ensure quality and useful monitoring and reporting it is necessary to establish the following:
 Clear monitoring and reporting process with responsibilities and calendar;
 Templates and tools to be used in monitoring and reporting (including any IT solutions, if possible);
 The ways to use monitoring and reporting information in the decision-making process; and
 Capacity of both lead and participating institutions.
Clear monitoring and reporting process is needed to inform the participants of their responsibilities and to
allow them to get ready for any commitments. Established processes should include the procedures to
complete monitoring and reporting (WHAT), the responsible institutions (units) for completion of
procedures (WHO) and the time to complete these procedures (WHEN).
Templates and tools are necessary to structure the communication between institutions and to ensure that
the presentation of information is compatible and easy to consolidate. Templates on how to present
information have to be elaborated and tested to determine if they produce the required information.
Instructions on how to fill the templates have to be prepared in order to deliver the same message to all
participants. The templates may still not guarantee that everyone will interpret them in the same way;
therefore a briefing may also be needed, especially if many institutions are involved in the monitoring and
reporting process.
The information generated through monitoring and reporting systems can be used internally in the
institution to make decisions, externally to provide an account to wider public, or both.
One of the most important factors in monitoring and reporting process is the use of information in the
decision-making process. This is the factor that directly influences the motivation of the involved parties to
perform better, as well as to provide quality information for monitoring and reporting purposes. Two issues
are important to highlight in order to increase the use of information: i) presentation of information in a

6
Concept Paper on Monitoring and Evaluation System 2017
for PFM Strategy in Albania
friendly and simple way according to the needs of the users and ii) existence of the decision-making
structures to make decisions.
The way performance information is presented to a very great deal influences the use of information. The
more understandable and explicit the information is, the more it will be used. For example, politicians and
managers have limited time to go through the documents. It is therefore important to structure the
information well, so that they spend as little time as possible to absorb it. Report structure has to be
designed to meet the needs of the users.
And finally, capacity of the institutions to produce quality information and to consolidate information is
also very important. It takes an effort and requires skills to produce good reporting information based on
analysis and supported by evidence. The practice in the public sector shows that very often reporting
information is focused on daily, routine actions rather than on analysis of the achievements. The capacity
can be improved through guidance, training and practice. In cross-cutting policies the role of the lead
institution is to ‘set the tone’ and to direct the other institutions.
The basic elements of the monitoring system of the PFM Strategy are defined in the Action Plan.
Frequency: As presented in the figure below the implementation of the PFM will be monitored semi-annual
and annually, with a mid-term evaluation (interim) of the strategy.
Figure 5: M& E System of the PFM Strategy 2014 – 2020

The monitoring of the implementation of the Strategy activities will be conducted on semi-annual basis
based on reports of the respective structures. MOF will draft a semi-annual and an annual report on the
progress of the implementation of the Strategy. Annual reports will present the progress of the strategy
implementation and evaluation based on the of success (outcome) indicators, as well.

7
Concept Paper on Monitoring and Evaluation System 2017
for PFM Strategy in Albania
Based on the mandates of the PFM management bodies and described PFM monitoring and reporting
process, the following key steps can be defined:
Reporting process of implementation of the PFM Strategy

Responsible Units
Submit reports on the
progress of
implementation of the
actions and
respective indicators
MoF
(Directorate for
Managing the PFM
reform)
Assess the progress
made and prepares
the monitoring reports
Technical
Committee
Reviews/approve the
reports. Submits the
brief report on the
overall PFM reform
progress to the SC
SC
Approve the report
and propose changes
as necessary.

SPU
Reviews the reports
in the framework of
the NSDI & sector
monitoring and
prepares an overall
annual monitoring
report for SPC

From the steps it is obvious that the annual reports will be discussed at the Steering Committee level with
the focus both in the progress of implementation of activities and achievements in terms of indicators.

8
Concept Paper on Monitoring and Evaluation System 2017
for PFM Strategy in Albania
The following forms are prepared and presented as integral part of this concept note:
1. The Instruction for monitoring and reporting (including reporting templates and guidelines);
2. The structure of the annual reports.

3.3. Policy and reforms evaluation of the PFM Strategy in Albania

Evaluation is a more elaborate, analytical and independent process of analysing the problems, determining
the possible causes of the problems, and formulating conclusions and recommendations. The purpose of
evaluation is to improve policy implementation and/or design.
Evaluation is a much more resource and skills-intensive process than monitoring. It requires sophisticated
analytical skills and expertise in various data collection and analysis methods. Therefore, evaluation can be
carried out internally by program staff or it can be commissioned to external providers.
Internally undertaken evaluations are implemented gradually, starting with simple questions, such as:
 What are the priorities of the Government/policy and whether the objectives and outcomes/
outputs of the program are aligned with the Government/ policy priorities?
 Are objectives achieved compared to targets set? What are the reasons for failure to achieve
objectives?
 Will the programmed activities and outputs lead to the achievement of outcomes and objective?
 Can program objectives and outcomes be better achieved by implementing a different mix of
activities?
Evaluation can be ex-ante, interim and ex-post. Ex-ante evaluation is done to design a program, an interim
evaluation is done during the implementation and ex-post evaluation is done after completion of the
program.
The scope of evaluation is an issue that program managers and staff have to decide. The scope of
evaluation is decided applying five evaluation criteria (see Table 1 below) and selecting the most relevant
depending on the timing of evaluation, duration and needs.
Table 1: Evaluation criteria
Ex-ante Interim Ex-post
Relevance Is the intended intervention Has the original need changed? Has the intervention reached/
going to solve the problem? affected intended target group?
Does the change require a
To what needs does it corresponding adjustment of the
respond? intervention?

Are the foreseen activities
likely to lead to achievement
Effectiveness of the objectives?
What are the costs? Is there a
way to deliver the program in
a more cost-effective way?
Do the results obtained so far ensure
that the objectives will be achieved?
What are the factors affecting the
effectiveness of part x of the
intervention?
Have objectives been
implemented as planned?
What were the reasons for the
high success/ failure in achieving
objective x?

Efficiency Is the chosen approach the How can the efficiency of the process Was activity x achieved using cost

9
Concept Paper on Monitoring and Evaluation System 2017
for PFM Strategy in Albania
most cost-effective? be increased? – effective approach?

Is the implementation What institutional procedures must How good was the performance
mechanism – structures and be simplified so that implementation of the institution or process?
processes – well justified? becomes more efficient?
What lessons can be learnt?

Impact What is the expected impact
of the intervention?
What are the impact target
groups?
Does the intervention during
implementation bring some
unintended effects?
What is the impact of activity
against expectations?
Why was the impact lower/
higher than expected?

Sustainability Is the impact likely to be long- Are the pre-requisites for What are the chances that results
lasting? sustainability of objectives in place? will last for a long time?

What are the elements that What can be done to ensure that What are anticipated future costs
will ensure this? they are? for the program?

What factors are affecting adversely What are the reasons for low
sustainability of achievements? sustainability of results?

Evaluation has to be selective. It is not always possible to do an evaluation every year and of each policy or
strategy. Decisions on which programs and aspects to evaluate depend on the data generated through the
monitoring process and external sources. For example, it might be useful to undertake an evaluation if
monitoring data shows that certain targets have continuously been underperforming or if service users
have been complaining about the quality of services. Also, programs or interventions with large budgets or
important issues on the political agenda (priorities) can be evaluated.
The evaluations have to be planned in advance to ensure necessary preparations and financial, human or
other resources to complete evaluation. Depending on the tasks and depth of the evaluation, it can last
from three to six or even more months. The preparation of the Terms of Reference for external evaluation
can take up to two months. This means that the preparations for evaluation have to start at least 2-3
months before the start of the evaluation. For evaluation to be useful, it has to be completed in time to use
evaluation results for policy improvement. Therefore, evaluation has to be a well thought-through process
in order to get the best results and to use the resources efficiently.
In Albania, the PFM Strategy will undergo a mid-term evaluation process, expected to be completed by
beginning of 2018. The evaluation will include the period from 2014 to beginning of 2018 and will highlight
the achievements of the first two years of the strategy in relation to the expected outcomes and the level
of achievement of the objectives of the strategy. Mid-term evaluation will serve to define a more precise
action plan for the period of 2018-2020, for an assessment of financial needs for the implementation of the
strategy for the next three-year period by orienting in this way the programming resources in the frame of
the Medium Term Budget Program.
Nevertheless, the mid-term evaluation is also aimed at assessing the relevance (or any substantial change
needs) of the set Objectives of the Strategy and to assess the general “current state”, as of 2017 in order to
reflect to any substantial changes in the environment that requires special, targeted strategic attention that
is not cover under the Strategy properly.

10
Concept Paper on Monitoring and Evaluation System 2017
for PFM Strategy in Albania
A new PFM Strategy will be prepared in 2020 which means that ex-post evaluation should be carried out in
2020, as early as possible.

4. Proposals for management, monitoring and evaluation of the PFM Strategy

Taking the above review of the work carried out so far in relation to PFM Strategy and Action Plan
management, monitoring and evaluation into consideration, the improvements or further developments in
three areas are suggested:

4.1. Proposals for the PFM Strategy monitoring and reporting instructions
To guide the Ministry of Finance in carrying out monitoring and reporting on the implementation of the
PFM Strategy and its Action Plan, instructions for Monitoring and Reporting on the Implementation of the
PFM Strategy (further the Instructions) are suggested in this concept document. The Instructions includes
the following elements:
1. Procedures for Monitoring and Reporting
2. Templates for Monitoring and Reporting
3. Table of content on PFM monitoring report
The proposal for the Instructions is provided in Box 1 below.

11
 Concept Paper on Monitoring and Evaluation System for PFM Strategy in Albania 2017
Box 1: Proposal for the Draft of the Instruction

THE INSTRUCTIONS FOR THE PFM STRATEGY AND THE ACTION PLAN MONITORING AND REPORTING
1. Introduction
These Instructions define the actors and the procedures for monitoring and reporting on the implementation of the PFM Strategy and its Action Plan. The purpose of the
Instructions is to unify monitoring and reporting process by defining the procedures, responsibilities, deadlines and the templates. The primary users of these Instructions are the
Ministry of Finance; state administration bodies and independent institutions that take part in the implementation of the PFM Strategy Action Plan. The secondary users are the
wider public, NGO and other organisations interested in the PFM.
2. PFM monitoring and reporting actors, process, and deadlines
1. The overall monitoring of implementation of objectives, indicators and activities of the PFM Strategy and its Action Plan is carried out by the Department for Management
of the Reform in the Public Financial Management (DMRPFM) area at the MoF through the daily communication with the relevant parties.
2. The monitoring of implementation of separate activities, linked performance indicators and objectives is carried out by the DMRPFM, each department involved at the
MoF and other relevant parties in charge of these activities as indicated in the Action Plan 2015-2017.
3. The reporting is semi-annual and annual. The preparation of the semi-annual and annual reports is co-ordinated by the DMRPFM at the MOF based on the information
received from the various departments at MoF and other relevant parties in charge of respective objectives.
4. The semi-annual report will be prepared in the following way:
a. By the 25th June (the DMRPFM shall send a letter and/or an email to various departments at the MoF and other relevant parties to request to submit semi-annual
report.
b. By 10th of July relevant departments at the MoF, and other relevant parties in charge of the activities as indicated in the Action Plan shall submit a report to the
DMRPFM in the form provided in the instruction. If respective structures, ministries and independent institutions do not provide information in time, the
DMRPFM can inform the Deputy Minister of Finance or take any other actions.
c. The DMRPFM, shall analyse the information received from respective structures/institutions, carry out the assessment by completing the database and compile a
draft Semi-annual Progress Report. The DMRPFM may request additional information from the respective responsible structures if information submitted is not
clear, sufficient or not satisfactory.
d. By the 30th August th the DMRPFM shall finalise the Semi-annual Progress Report and submit it to the Deputy Minister of Finance and the Steering Committee, for
consideration.
5. The annual report will be prepared in the following way:
a. By the 20th December of each year, the DMRPFM shall send a letter and /or email to the respective structures/institutions to remind on the submission of annual
reports on the implementation of PFM Strategy/ Action Plan.
b. By the 20st of January of the next year the institutions in charge of implementation of results shall summarise the information received from partners in
implementation and submit summarised information to the DMRPFM in the format provided by the instructions.
c. The DMRPFM shall analyse the information received from institutions in charge of implementation of results, consolidate the information and prepare a draft
Annual Progress Report. The DMRPFM may request additional information from these structures/institutions if submitted information is not satisfactory or
Concept Paper on Monitoring and Evaluation System for PFM Strategy in Albania 2017
enough.
d. By the 20th March of each year the DMRPFM shall submit the draft Annual Progress Report to the Deputy Minister and to the Steering Committee for
consideration and approval.
6. If needs to be, the DMRPFM can organise workshops to guide the staff of responsible units and help them strengthen the skills needed to prepare PFM-related reports.

Two Tables will be used to provide semi-annual and annual information on the implementation of PFM Strategy and its Action Plan:
a. Semi-annual Report Template
b. Annual Report Template
3. Semi-annual reporting templates and instructions
7. The Format and instruction on how to fill in the semi-annual report Template is provided below. It is used by all responsible units to provide semi-annual information on
the implementation of activities. This Table is in Excel format.
Semi-annual Report Template

Responsible Next steps Expected time of

Nr Activity Deadline Achievements
Unit (milestones) realisation
(Progress in Implementation)

1 2 3 4 5 6 7
Filled in
by each
Filled in by Filled in by each Filled in by Filled in by each Filled in by each
structure Filled in by each structure/institution
DMRPFM structure/institution DMRPFM structure/institution structure/institution
/instituti
on
Filling in instructions
Column 1 In this column the Number of the Action in the Action Plan is indicated. This column is filled in by the DMRPFM.
In this column the activities from the Action Plan are provided. This column is filled in by each structure/institution. In this column the indicators for the
Column 2
realization of results are listed.
Column 3 In this column the responsible unit for reporting is indicated
Column 4 In this column the deadline for implementation of respective activity is provided. This column is filled in by each structure/institution.
Column 5 In this column the description of major achievements in implementing the activity is described. Each responsible structure should provide only strategic
information on the realization of activity which would help to determine the realization of the result. The achievements should be supported by evidence, if
possible (e.g. statistical data, comparisons, etc.). Routine and administrative information (the number of meetings organized, working groups established,
etc.) should be avoided. For example, if the law was prepared implementing the activity, the achievements should explain what will change as a result briefly

13
Concept Paper on Monitoring and Evaluation System for PFM Strategy in Albania 2017
summarizing the main benefits of the law rather than just making a fact that the law has been prepared and when. The information presented under this
column should justify the status of realization.
In this column the next steps to implement the activity should be provided. This column is filled in only for those activities of which the status is “partially
Column 6
implemented” or “not implemented”. This column is filled in by respective structure/institution.
In this column the institution has to indicate the expected new date of the realization of activity based on the next steps. This column is filled in only for
Column 7
those activities of which has not been completed or have not started yet.
Examples:
5.2.2.1. Perform 6 officials trained in 3 General 3 Q 2016 1. During the reporting period the directorate has The next training Q I 2017
professional days Director of organised one training courses of 3 days. The courses are planned
development Regulatory following number of participants attended the to take place on
and training on and training courses, as well as the total % of staff XXXXX, XXX and
the audit Controlling trained until Q2 is presented in the table below. XXX.
methodology Department
and practices for
% of Staff
internal auditors Nr Staff
Ministries Trained until
in the public Trained
Q3-2016
sector
Ministry of
2 33%
Education
Ministry of
2 33%
Transport
Ministry of
1 17%
Economy
Ministry of
1 17%
Agriculture

In total out of X staff, until Q3 are trained XX staff or

XX% of the total staff of central ministries, to be
trained.
4. PFM annual reporting templates and instructions
1. The Format and instructions on how to fill in the Annual Report Template is provided below. Using this format the information on implementation of activities as well as
progress achieved in implementing specific objectives and performance (outcome) indicators is provided.
2. The Annual Report Template is more analytical than the semi-annual report. Apart from information on activity implementation, the Annual Report Template asks to
provide information on the progress towards achievement of objectives and performance indicators. Therefore the Annual Report Template is more elaborate.

14
Concept Paper on Monitoring and Evaluation System for PFM Strategy in Albania 2017
4.1. Annual Report Template
I. Information on progress against objectives (This information is provided in Word format)
In this section the institution in charge of the specific objective provides a brief overview on the implementation of specific objective. This section is prepared based on the
overall progress in implementing activities and on the achievements against performance indicators (both those of specific objective and results). The achievements in this
section should be illustrated using graphs, tables or pictures. The key factors that had an influence on the achievement or failure of results are provided. The assessment of
whether the target values of performance indicators are likely to be achieved should be provided. Also priority actions to improve the performance should be provided.

II. Information on progress against performance indicators and activities (This information is provided in Excel format)
The Template below allows providing full information on the achievement against performance indicators, as well as implementation of activities. To report on the
achievement of performance indicators, the actual achievements for 2 passed years (reporting years and the year preceding the reporting year) is provided, as well as the
target value for respective year. The information on achievement of performance indicator is provided by the institution in charge of performance indicator as indicated in the
Indicator Passport. Respective institutions also have to report on the implementation of activities.
Note: the first annual report will be used for establishing the baseline data. Therefore during the first annual report the column Baseline Data will be filled in by respective
units.
4.2. Performance Assessment Framework (PAF)
Performance Assessment Framework will be used to measure progress of indicators versus targets established in the strategy. The following tables are part of the PAF. The PAF is a
indicator monitoring system introduced within the strategy monitoring system. The PAF is part of monitoring tools introduced in 2016.
Policy Area

1.i. <description of policy area>>

Baseline Data and Indicator Indicator Indicator Indicat

Indicator Indicator Indicator
Objective Objective Source of Data Target FACT Target or Fact
FACT 2016 FACT 2018 FACT 2019
2017 2017 2020 2020

Description of Objective                
1
<<objective>>

Indicator Baseline 2015  

<<description of indicator>>
1.a                
Unit measure: Source of
<<name of the responsible unit for reporting>>
Verification

15
Concept Paper on Monitoring and Evaluation System for PFM Strategy in Albania 2017
Responsible Unit for reporting Department of Programming, Development and Foreign Aid

The table is filled in with information for all objectives and respective indicators for a policy area. The information in all cells is inserted by the MIAP. Only information on level of
indicator reached in a given year ( FACT) will be filled by the respective unit/ministry.

On the implementation of the activities the same template as indicated above for semi-annual reporting will be used.

Example
Pillar no
1.1. Fiscal rules, forecasting and fiscal responsibility framework
Indicator Achievements
Obj.ind. Baseline Level Progress Versus Indicator Target
Policy Objective Achieved Level versus Target Level
number 2013 Baseline Level 2020
2017 %
  Description of Objective          
Objective 1 - Sustainable and prudent fiscal frameworks, based on clearly stated fiscal objectives, underpin the budget
 

1 Indicator          
1 Forecasted nominal GDP used in each budgetary planning phase should
not be higher than the respective forecasted nominal GDP in the World +14.3
Economic Outlook, published by the IMF
Based on the information included in the PAF the progress of achieving the indicators will be assessed by DMRPFM

Monitoring Table - Indicators with Target defined by Trends - "Growing Trends" or "Decreasing Trend"
Policy Area          
4. Administrative Procedures and Oversight
         
Indicator Indicator
Baseline Level Progress Versus Assessment of
Nr Policy Objectives Achieved Level Target Level
2013 Baseline progress
2016 2017
2.3.1.   Description of Objective          
  Objective 2.3.1: Prudent and stringent procedural framework for the          
preparation of the MTBP. MTBP framework that encapsulates the
NSDI, government and sector strategic priorities for the medium

16
Concept Paper on Monitoring and Evaluation System for PFM Strategy in Albania 2017
term delivers desired results within sustainable financial resources
for the three future years.
Indicator          
2.3.1 Ratio between total funds estimated in the sectoral strategies and
. total funding identified for corresponding sectors within the MTBP is
42.0% 0 Growing trend  
90%

    The achievement score is progressing positively  

Legend:   The achievement score has not changed  
 
    The achievement score shows a negative trend  

Based on the information provided by the relevant institutions to the DMRPFM, it shall consolidate the information and prepare draft
semi-annual and annual reports. The structure and contents of the semi-annual and annual reports is provided in the sub-section below.

17
 Concept Paper on Monitoring and Evaluation System 2017
for PFM Strategy in Albania

4.2. Proposals for PFM Progress Reports

The MOF will have a task to draft semi-annual report and annual report. Having in mind that the ultimate
user of the semi-annual and annual reports will be the SC and the EU Delegation it is necessary to take into
account the length of the reports and the way information is presented. Since the SC members have very
limited time, it is necessary to make the reports easy to read. The structure of the semi-annual and annual
reports and guidance on how to present the information are provided in the sub-sections below. The focus
of the Semi-annual Progress Report is on the implementation of activities, with the focus of realisation of
the outputs, while performance indicators are reported on an annual basis.

4.2.1. Semi-annual Progress Report

The Semi-annual Progress Report should give a picture of where progress is satisfactory or unsatisfactory,
what are the reasons for unsatisfactory results and what steps should be taken to improve performance on
implementation of activities. The following is the proposed structure of the Semi-annual Progress Report.
The semi-annual report would be more a working document that will be used for internal purposes, for
review of the progress made within a given year of implementation of the strategy. Therefore a focus on
the delays of implementation and the issues of concerns must be highlighted in this report frequently. The
report is not designed for publication and information for the public.
A proposed template is presented in the Annex 3.

4.2.2. Annual Progress Report

The focus of the Annual Report is to check on the implementation progress achieved against each
objective, using information on performance indicators. The annual report must present information on the
progress on all six areas and will focus on the impact of the activities undertaken in the current reporting
year. It must be emphasised that the first annual report will be used for establishing the baseline data. In
the first annual report there will not be comparative analyses on progress made under each specific
indicator. The other reports will focus on analyses on the progress made towards reaching the targets
initially those of 2017, and after those of 2020. The annual report will have a broader audience than the
semi-annual report, as it is planned to be published. A proposed template is presented in the Annex 3.

4.3. Proposals for PFM Strategy and its Action Plan Mid-Term Evaluation
According to the PFM Strategy Action Plan, the evaluation in 2017 will be mid-term for the PFM Strategy
and ex-post for the Action Plan. Accordingly, it is proposed that the Terms of Reference for the PFM
Strategy and Action Plan Evaluation define objectives and questions to address the following evaluation
criteria:
1. Effectiveness of PFM Strategy and Action Plan
2. Relevance of intervention to achieve objectives
3. Efficiency of PFM management structure
The following structure of the Terms of Reference is proposed:
A. Background
B. Overall and specific objectives of the evaluation
C. Evaluation questions
D. Expected deliverables

18
 Concept Paper on Monitoring and Evaluation System 2017
for PFM Strategy in Albania
E. Approach and methodology
F. Professional qualifications
The main body of the Terms of Reference are provided in Annex 2 to the Concept Paper.

5. Next steps
The calendar of activities presented in the annex 6, presents a set of actions to be undertaken in the up-
coming period which will focus on monitoring of the PFM, preparation of the monitoring reports, as well as
steps related to mid-term evaluation of the PFM in 2017.

19
ANNEX 1: Instruction on Semi-Annual Reporting

1. Purpose of the Reporting

As envisaged in the adopted Public Finance Management Strategy 2014-2020 semi-annual monitoring
report will be produced to review the on-going progress of the strategy. The semi-annual report will be
used to ensure that activities for implementing the planned outputs and reaching the targets are
progressing smoothly. In addition, the semi-annual monitoring report will serve as a tool for the responsible
institutions to flag outstanding issues. Reports are key tools for the decision-makers and in particular the
Minister of Finance to be informed and up-to-date on the progress. The Department for Management of the
Reform in the Public Financial Management (DMRPFM) will coordinate the process of monitoring and
preparation of the semi-annual monitoring report of implementation of the strategy.

2. Reporting format and requirement

The department at the Ministry of Finance and institutions involved in the Strategy should fill in
information only in the column "Reporting H<< number of Semi-annual>> of the Action-Plan matrix
(attached).
 It is compulsory for each responsible structure to provide short information on the status of
implementation of each of the outputs foreseen under respective activity, whether have started or
not. If the activity for implementation of one output has not yet started this must be indicated in
the respective box, providing information on the reasons why it has not started.
 For the activities being implemented the report must provide clear and sufficient information the
status of the process for development of the output.
 If there are activities fully completed clear evidence must be provided that confirms
implementation of the output. If an output refers to approval of a specific document, law or
strategy, (activity completed), the ministries must provide the reference number of the approved
document.
 Statistical data must be provided to inform the level of accomplishment for the planned outputs.
e.g. number of staff trained, number of one-stop offices established etc.
 If for some specific outputs the implementation has not started or is delayed due to specific
problems that have aroused (e.g. lack of funds, delays in the procurement process, problems in
cooperation among institutions etc) these must be presented in the reported.

20
ANNEX 1: Instruction on Semi-Annual Reporting

3. Main steps and timeframe for preparation of Semi-annual <<number>> Report <<year>>

Monitoring and preparation of the report on the progress of implementation of the PFM Strategy 2014-
2020 will be carried out by the staff at the Department for Management of the Reform in the Public Financial
Management (DMRPFM). Preparation of the First Semi-annual Report will go through the following steps and
will be prepared following strictly the deadlines set out accordingly:

Step Deadline Tasks

Step 1 <<25th June>> The respective department/institutions will receive the instructions
for reporting
Step 2 <<10th July>> Each responsible structure that has the primary responsibility (as
indicated in the column "Responsible Institutions" will coordinate
with other participating structures/ institutions to collect
information and prepare the brief inputs on the progress of
implementation of specific outputs in the respective actions.
Step 3 <<date>> DMRPFM will review the reporting’ submissions. In case uncertainties
arise from the reports they will seek additional
information/clarifications from the respective responsible unit.
Step 4 <<date>> After accomplishing the analysis of submissions, the DMRPFM will
carry out the qualitative assessment of the submissions and prepare
the report on the status of implementation of the Strategy 2014-
2020 for the <<number>> semi-annual
Step 5 <<30th August>> The DMRPFM will submit the report to the SC and other stakeholders
for information.

The deadline for each ministry to submit the report to the DMRPFM is <<10th July>> at 4.00 p.m.
The submission of reporting inputs and any question will be delivered in electronic form at the email
address: <<include the address of contact person >>
We very much appreciate your cooperation and submission of the reports on due date. Therefore all
responsible structures are kindly required to strictly respect the date of <<deadline>> for submission of the
reporting information.

21
ANNEX 2: Instruction on Annual Reporting

1. Purpose of the Reporting

As envisaged in the adopted Public Finance Management Strategy 2014-2020 comprehensive annual
reports will be produced to review the on-going progress of the strategy. The annual reports will be used to
assess the progress made in implementation of the activities and planned outputs and reaching the targets
planned though 43 performance (outcome) indicators.

In addition, the annual monitoring report will be used as a tool to inform the high political level, donors and
the public on the progress made towards implementing the Public Finance Management Reform. The
DMRPFM will coordinate the process of monitoring and preparation of the Annual monitoring reports of
implementation of the strategy.
2. Reporting format and requirements
For the purpose of preparation of a comprehensive annual report for the strategy, the inputs of reports
(individual reports from each structures/ institutions) will be focused on the following reporting items:
a. Information on progress against objectives (this information is provided in Word format)
A brief overview on the implementation of each specific objective will be provided. This section will be
prepared based on the overall progress in implementing activities and on the achievements against
performance indicators (both, those of specific objective and results). This section should provide:
 information on the main achievements during the year. Achievements needs to present not
achievements on the implementation of a single activity, but presented in the framework of
the reform undertaken in specific area. Please use the picture, graphs and statistical data to
justify the achievements.
 Information on factors that had an influence on the achievement or failure of results are
provided.
 the assessment of whether the target values of performance indicators are likely to be
achieved should be provided. Also priority actions to improve the performance should be
provided.
b. Information on progress against performance indicators (Performance Assessment Framework)
The information on achievement of performance indicator is provided by the institution in charge of
performance indicator as indicated in the Indicator Passport. The respective institution will fill the cell
on the respective "Indicator Fact" level.
c. Information on progress against activities (Action-Plan 2014-2020)
The responsible units should fill in information only in the following columns in the matrix of Action -
Plan:
i. Achievements (Progress in Implementation)
It is compulsory for each responsible structure to provide sufficient information on the status of
implementation of each of the outputs foreseen under the respective activity. If there are activities
fully completed, clear evidence must be provided that confirms implementation of the output. If an
output refers to approval of a specific document, law or strategy, (activity completed), the ministries
must not only provide the reference number of approval, but also provide information on the
impact that the approved legislation is going to bring. Statistical data must be provided to inform the

22
ANNEX 2: Instruction on Annual Reporting

level of accomplishment for the planned outputs. (e.g. number of staff trained, number of one-stop
offices established etc.)
If, for some specific outputs, the implementation has not started or is delayed due to specific
problems that have aroused (e.g. lack of funds, delays in the procurement process, problems in
cooperation among institutions etc) these must be presented in the report.
For the activities under implementation the report must provide clear and sufficient information the
status of the process for development of the output. Clear information must be indicated in the
column:
ii. Next steps (milestones)
What is expected in terms of milestones to be performed in the up-coming year in order to
implement the activity and produce the planned outputs.
iii. Expected time of realisation
In this column the semi-annual when the full realization of the output must be presented.

d. Main steps and timeframe for preparation of Annual <<number>> Report <<year>>
Monitoring and preparation of the report on the progress of implementation of the PFM Strategy 2014-
2020 will be carried out by the staff of the Department for Management of the Reform in the Public
Financial Management (DMRPFM). Preparation of the First Annual Report will go through the following
steps and will be prepared following strictly the deadlines set out accordingly:

Step Deadline Tasks

Step 1 10 January The respective department/institutions will receive the instructions
for reporting
Step 2 1 February The respective units submit the report to DMRPFM
Step 3 10 February DMRPFM will review the reporting’ submissions. In case
uncertainties arise from the reports they will seek additional
information/clarifications from the respective responsible unit.
Step 4 25 February After accomplishing the analysis of submissions, the DMRPFM will
carry out the qualitative assessment of the submissions and
prepare the annual report
Step 5 20 March The DMRPFM will submit the report to the SC and other
stakeholders for information.
Step 6 31 March The DMRPFM will submit the report to the European Delegation in
Tirana.

The deadline for each ministry to submit the report to the DMRPFM is 1 February at 4.00 p.m.
The submission of reporting inputs and any question will be delivered in electronic form at the email
address: <<include the address of contact person >>
We very much appreciate your cooperation and submission of the reports on due date. Therefore all
responsible structures are kindly required to strictly respect the date of <<deadline>> for submission of the
reporting information.

23
Annex 3 Template of semi-annual and annual reports

Semi-annual Report Template

1. Introduction
< short introduction on the scope of the report >>
2. Overview of implementation of the Action Plan 2014-2020
< brief information on the overall status of implementation, including a graph on the overall number of
activities started, not started , planned to start after Q1 >>
3. Overall progress in implementing the pillars of the Strategy
< brief information on the overall status of implementation of 6 pillars, including a comparative graph for 6
areas >>
A more detailed assessment of the progress made under each of pillars is provided below.
Pillar 1: Sustainable and prudent fiscal framework
Pillar 2: Well integrated and efficient planning of public expenditure
Pillar 3: Efficient execution of the Budget
Pillar 4: Improvement and increase in transparency of Government Financial Reporting
Pillar 5: Strengthening internal control in the public sector
Pillar 6: Effective oversight of public finances
4. Overall performance according to implementing structures
< brief information on the overall status of implementation by responsible units, including a comparative
graph on the overall number of activities started, not started , planned to start after Q1 for each unit>
5. Conclusions and Recommendations
This section must present very shortly and in the bullet points the main issues observed by the DMRPFM
during the assessment. The conclusions must present the most concerning issues which are common for a
large number of actions and that can be linked to issues of both internal and external influencing factors.
On internal factors such issues might be institutional coordination aspects, speed of starting the
procurement process, delays on contracting, not proper functioning of the working groups, etc.. Problems
related to external factors could be also different and they need to be highlighted.
At the part of the conclusions one bullet point must be dedicated to the quality of submissions, as this
would be one of the challenges that DMRPFM staff will face during the monitoring process.
In the end the report must also list some recommendations for the Minister of Finance related to addressing
some specific issues observed, if considered necessary.

ANNEX 1: DETAILED EXPLANATION ON IMPLEMENTATION OF EACH ACTIVITY

This section is prepared using the information received from the relevant departments and institutions.
This information should be supplementary information for those who will want to have a more detailed
picture on the implementation of each individual activity. The information in this Annex is presented using
the Semi-annual Report Template.
Note: All the narrative report should not exceed 4- 5 pages at the maximum.

24
Annex 3 Template of semi-annual and annual reports

Annual Progress Report TEMPLATE

1. Executive Summary
In this section brief information on the progress against each specific objective is provided based on
performance indicators as well as implementation of activities. In this section only achievements
implementing the most important activities should be presented. The importance of activities is decided
based on the scope of the impact of the activity, attention on the political agenda, interest of the media or
civil society. Graphs, tables and pictures can be used to support the achievements.
This section should be used to brief the decision makers, the general public or stakeholders. The following
graphs could be used to illustrate the progress:
Graph 1: The overall progress implementation of activities in 201X (a pie graph to show the overall
ratios full implemented, partially implemented, not implemented, planned to start after 201X)
Graph 2: The overall assessment on reaching the target indicators (the graph to be presented for years
after 201X)
2. Overall progress in implementing the Strategy
This section could have two sub-sections. One sub-section could be dedicated to overall progress using
statistical data (e.g. share of improved indicators, implemented activities). The second sub-section could be
a description of successes or failures in achieving progress under each pillar.
2.1.Overall information on the progress of Strategy by pillars
In this sub-section the information on the progress against performance indicators and activities should be
provided.
Note: the first annual report will be used for establishing the baseline data. In the first annual report there
will not be comparative analyses on progress made under each specific indicator. The other reports will
focus on analyses on the progress made towards reaching the targets initially those of 2017, and after
those of 2020. Even though there are no annual targets defined for performance indicators, it is possible to
analyse and describe how big or small the progress was. The information on improved and not improved
performance indicators should be presented. This information will give a feeling of whether the PFM
Strategy is coming closer to achieving the targets.
Also the overall annual information on the implementation of all activities is provided. It may be useful to
show how well the PFM Action Plan is being implemented by each institution and how well each objective
is being implemented.
Several graphs can be used to support the achievements:
1. A pie chart on the number of well progressed and not progressed performance indicators. This
information can also be presented by each pillar.
2. A pie chart on fully implemented, partially implemented and not implemented activities;
3. A graph on fully implemented, partially implemented and not implemented activities by each of
the six pillars of PFM;
4. A graph on the number of fully implemented, partially implemented and not implemented
activities by each specific pillar
25
Annex 3 Template of semi-annual and annual reports

5. A graph on the number of fully implemented, partially implemented and not implemented
activities by each institution.
2.2.Overall progress of implementation of reforms under the Pillar 1: Sustainable and prudent fiscal
framework
Progress against activities
<brief information on the overall status of implementation of each of components under this pillar
including a comparative graph for all components>
 Graph 4 Overall ratio of implementation (pie graph)
 Graph 5: Implementation of activities by specific component (number of activities)
<for each component short information must be provided on the main achievements in terms of
reforms undertaken. The information would not be simple description of activities, but effects produced
by the implementation of activities>.
Progress against performance indicators
<The progress against objectives will be measured using respective performance indicators under each
pillar>
Next steps for timely implementation
Next steps for timely implementation are recommended:
The same information is analysed for each of the five other pillars.
2.3.Priority actions for next year
This section should summarise the actions that the Government or institutions should implement in order
to overcome the challenges and improve performance along each specific objective. A summary list of
actions should be provided for each objective as well as a list of common actions if applicable. Any capacity
building activities to address the failures and underperforming should also be listed.

26