EUROPEAN COMMISSION

SCR – Common Service for External Relations

Resources, relations with the other institutions, evaluation, and information

(YDOXDWLRQ

SCR/F/5 Evaluation Unit

Brussels, December 2000

EVALUATION GUIDELINES

I: A FORMAT FOR EVALUATION TERMS OF REFERENCE 1

II: EVALUATION METHODOLOGY 3

1 The Logical Framework 3
2 Main Types of Projects or Programmes 3
3 Types of Evaluation 3
4 Methodology: (YDOXDWLRQ&ULWHULD 
5HOHYDQFH 
(IILFLHQF\ 
(IIHFWLYHQHVV 
,PSDFW 
6XVWDLQDELOLW\ 
*HQHUDOLVVXHVUHODWLQJWRHYDOXDWLRQFULWHULDDQGGHVLJQ 

III: FORMAT AND PRESENTATION OF EVALUATION REPORTS 10

Annex I: Key Cross-Cutting Issues for consideration in Evaluations 12

Annex II: Standard DAC Format for Evaluation Report Summaries 14

Rue de la Loi 200, B-1049 Bruxelles/Wetstraat 200, B-1049 Brussel - Belgium - Office: B-28 5/120.
Telephone: direct line (+32-2)299 2729, switchboard 299.11.11. Fax: 299 2912.
Telex: COMEU B 21877. Telegraphic address: COMEUR Brussels.
 3DUW,$)RUPDWIRU(YDOXDWLRQ7HUPVRI5HIHUHQFH1

7KH 6&5 (YDOXDWLRQ 8QLW UHFRPPHQGV WKDW WKH IRUPDW IRU 7HUPV RI 5HIHUHQFH IRU DQ
(YDOXDWLRQ VKRXOG EURDGO\ IROORZ WKH ER[ VFKHPD VHW RXW EHORZ GUDZLQJ RQ WKH
PHWKRGRORJLFDOJXLGDQFHVHWRXWLQ3DUW,,Terms of Reference should be short and to the
point, with background information confined to annexes. The guidance below has been made
flexible to accommodate the many types of project, programme or strategy (for all of which the
word “project” will mainly be used henceforth, for convenience) for which evaluations may be
required. It is assumed that users are familiar with the Logical Framework system (see Part II)

$,QWURGXFWLRQ
Give the WLWOH of the project/programme to be evaluated, and describe in 1-2 short paragraphs
L its main features e.g. FRQWH[W (country, sector, &c), OHYHO (strategy, programme, project),
W\SH (investment, institutional reform); (LL whether the study is being undertaken during
implementation (mid-term evaluation), at completion (final evaluation) or some time after
completion (H[SRVW evaluation); LLL its timetable and estimated budget.

%2EMHFWLYHVRIWKHHYDOXDWLRQ
Indicate here L ZK\ the evaluation is being done (HJ legal requirement, requirement in
financing agreement, &c) or its PDLQSXUSRVH (accountability, lesson-learning for application
to future projects, progress check on current project and possible need for re-orientation,
sometimes with a subsequent phase in view); LL  IRU ZKRP it is primarily intended (the
target audience); LLL the SODQQHGRXWSXWV (e.g. report, presentations, feedback seminars).

&%DFNJURXQG
State in 1-1½ pages the REMHFWLYHV RI WKH SURMHFW to be evaluated, outlining its context and
evolution; its key elements and characteristics; its cost and duration; any significant changes
to the original objectives or plans; and (if an evaluation of an ongoing activity) the current
state of implementation, indicating any notable successes or problems.

',VVXHVWREHVWXGLHG
Here set out in detail L  the main issues which the evaluation should address and the key
questions for which answers are sought, LL  the level of analysis required in each case.
Indicate, if possible, which one or two of the five evaluation criteria (see E below) is/are
most relevant.

1 The finalised Terms of Reference should be no more than 5 pages in length.

1
(0HWKRGRORJLFDODVSHFWV
Here L mention PDLQ UHIHUHQFH GRFXPHQWV (Regulations, project financing agreements,
earlier evaluations &c; LL state the HYDOXDWLRQ FULWHULD - relevance, efficiency,
effectiveness, impact and sustainability - in each case briefly mentioning the specific issues
arising under each (see the examples and explanations in Part II, below); LLL indicate, where
possible, HYDOXDWLRQ WHFKQLTXHV DQG UHVHDUFK PHWKRGV - for example, data collection
methods including questionnaires, sampling, Participatory Rapid Appraisal (PRA) and other
forms of beneficiary contact and involvement; LY include an instruction to the evaluators to
draw up (or revise, as appropriate) the /RJLFDO)UDPHZRUN of the project as it applied both
as its outset and at the time of the evaluation, as a basis for the analysis; Y include any
special instructions on how to approach the key issues to be studied (see D).

)5HSRUWLQJDQGIHHGEDFN
Here specify the UHSRUWVSUHVHQWDWLRQVDQGIHHGEDFN required (e.g. inception report, end-of-
mission DLGHPpPRLUH, debriefing presentations, draft report, final report), with details of
language, date of delivery, and number of copies required. The report format/layout should
be specified (see Part III). The main text of an evaluation report should not exceed 50 pages,
plus Annexes, plus an Executive Summary of no more than 5 pages with fully cross-
referenced findings and recommendations. In addition, a short, separate summary of one
page should be required, to facilitate inclusion of the report in the Commission’s evaluation
databases (see format in Annex II).

*([SHUWLVHUHTXLUHG
Indicate the QXPEHU RI H[SHUWV required, and their key qualifications and experience, with
special attention to the requirements for the position of Team Leader. The use of national
experts should be required wherever possible. It is important that at least one team member
has good experience of conducting evaluations, ideally the Team Leader.

+:RUN3ODQDQG7LPHVFKHGXOH
Here indicate the GXUDWLRQWLPLQJ of the study, including presentations and report
submissions (allowing 3-4 weeks minimum for comments on drafts from Government,
Commission Services and other directly interested parties including local organisations).
Allow also for EULHILQJV for Delegations and local/governmental institutions at the beginning
and end of field missions. Anticipate meetings on the draft report, as well as meetings,
workshops or seminars towards or at the end of the evaluation forming part of the feedback
process.

2
 3DUW,,(YDOXDWLRQ0HWKRGRORJ\

7KH/RJLFDO)UDPHZRUN ³/RJ)UDPH´
It is re-stressed that readers should be conversant with Logical Framework (“LogFrame”)
principles, as set out in the Commission’s Manual on Project Cycle Management. The
LogFrame is basically a structured tabular method of summarising what a project is intended to
achieve, why, how and when. As such, it is as valuable to the evaluator as it is to the project
manager.

The LogFrame, following the Commission’s Manual on Project Cycle Management, should be
used, or drawn up afresh if none was prepared originally, to describe the project as originally
designed at the standard four levels of (i) RYHUDOOREMHFWLYHV, (ii) SXUSRVH, (iii) UHVXOWV and (iv)
DFWLYLWLHV2, each with associated indicators and, at activity level, means (types of resources) and
costs. LogFrames should be annexed to evaluation reports, but should never rigidly determine
their structure.

0DLQ7\SHVRI3URMHFWVRU3URJUDPPHV
While the range of programmes or projects which comprise Commission co-operation with
non-member countries in the various regions (Phare, Tacis, MED, ACP, ALA) is extensive, two
key components of many of them, albeit in varying proportions, are investment and institutional
support. Investments tend to be of a more rigid “blueprint” nature, whereas the nature of
institutional support tends to be an evolutionary, flexible “process”. The evaluation
methodology should be adapted accordingly. Particularly - although not exclusively - in
investment-type projects, the techniques of economic and financial analysis can be valuable to the
extent that time and data allow, especially if sufficient benchmark data was included in the
original project design; in more process-type projects, special attention also needs to be given to
the ability to adapt flexibly and rapidly to changing assumptions, risks and external factors.

The key measure of project success how far its SXUSRVH, or specific objective, was achieved in
terms of the benefits derived from the project UHVXOWV, whether a physical product, trained
resources, draft legislation, etc.. Achievement of purpose (“effectiveness”) is therefore to be
judged more from the side of the beneficiaries’ perceptions of EHQHILWVUHFHLYHG than from the
Commission’s perspective of results delivered. How far these two perceptions converge will
depend in large measure on the degree of prior agreement between donor and beneficiaries on
design, choice of indicators, monitoring procedures, and on the assumptions, risk assessments
and conditions set out in the right-hand column of the LogFrame (especially at UHVXOWV level).

7\SHVRI(YDOXDWLRQ

The guidance below is oriented mainly to PLGWHUP, ILQDO and ³H[SRVW´ evaluations.

³([DQWH´ evaluations at the start of the project cycle (preparatory studies, feasibility studies,
“appraisals”) are mainly concerned with examining needs or problems, identifying performance

2 These are the standard LogFrame terms used by the European Commission. Other terminology in use in the
donor community includes JRDO or ZLGHUREMHFWLYHV for overall objectives; LPPHGLDWHor VSHFLILFREMHFWLYHV
for purpose; RXWSXWV for results; and LQSXWV for activities.

3
indicators, structuring solutions, and assessing likely success, ideally using LogFrame methods;
so the evaluation criteria discussed below need not be considered in a formal sense at that stage.

The terms “IRUPDWLYH” and “VXPPDWLYH” are also sometimes used: a “formative” evaluation is
conducted for the benefit of those managing the project, with their full participation and with
the focus on improving their work; a “summative” evaluation is mainly for the benefit of
external stakeholders not directly involved with project management and emphasises the
independent, external aspects of objectivity and overall judgement, often for reasons of
accountability, applicability to similar future activities, or of optimising the allocation of
budgetary resources.

0HWKRGRORJ\
Most of the basic concepts and approaches (i.e. evaluation methodology) are universal3, and the
following sections are drafted in such a way that any user, tasked with preparing an evaluation
of a project, can easily interpret them and produce clear, focused Terms of Reference within the
framework provided in Part I.

(YDOXDWLRQ&ULWHULD

The diagram below sets out the main linkages between NH\ HYDOXDWLRQ FULWHULD and the NH\
/RJ)UDPHHOHPHQWV, and should be studied alongside the subsequent text

(YDOXDWLRQ&ULWHULD /RJ)UDPH/HYHOV

29(5$//2%-(&7,9(6
RYHUDOOODVWLQJFKDQJHERWKDWWKHOHYHO
RIWKHSURMHFWSURJUDPPHDQGEH\RQGLW
,PSDFWDQG6XVWDLQDELOLW\OLQN
UHVWVHYHQPRUHRQZLGHUULVNVDVVXPSWLRQV
DQGFRQGLWLRQVPDQ\RXWVLGHGLUHFWFRQWURO

385326(
EHQHILWVDFWXDOO\UHFHLYHG
(IIHFWLYHQHVVOLQN GHSHQGVRQULVNV
DVVXPSWLRQVDQGDQ\FRQGLWLRQVWKDWDSSO\
VRPHWLPHVEH\RQGGLUHFWFRQWURO
5(68/76
FRQILUPHGSODQQHGGHOLYHUDEOHV

(IILFLHQF\OLQNVIURPPHDQVWKURXJK
DFWLYLWLHVWRUHVXOWV DQ\DVVXPSWLRQVULVNV
DQGSURJUDPPHFRQGLWLRQDOLW\DUHPRVWO\ $&7,9,7,(6
ZLWKLQGLUHFWGRQRUFRQWURO 
SURFHVVRIFRQYHUWLQJLQSXWVLQWRUHVXOWV


0($16 ,QSXWV
PDWHULDOSHUVRQQHODQGILQDQFLDOUHVRXUFHV

5HOHYDQFH WRWKHLGHQWLILHGSUREOHPV 

'(6,*1 35(3$5$7,21

3 and are closely based on the OECD/DAC criteria for evaluation.

4
RUUHDOQHHGVWREHDGGUHVVHG
5HOHYDQFH

The relevance of a project relates primarily to its design and concerns the extent to which LWV
VWDWHGREMHFWLYHFRUUHFWO\DGGUHVVHVWKHLGHQWLILHGSUREOHPVRUUHDOQHHGV. It needs to be kept
under review throughout the life of the project in case changes occur either in the nature of the
very problems originally identified, or in the circumstances - whether physical, political,
economic, social, environmental, institutional or policy - in which the project takes place,
necessitating a corresponding change of focus. In other words, UHOHYDQFH FRQFHUQV WKH
DSSURSULDWHQHVVRIWKHSURMHFWGHVLJQWRWKHSUREOHPVWREHUHVROYHGDWWZRSRLQWVLQWLPH
ZKHQ WKH SURMHFW ZDV GHVLJQHG DQG DW WKH WLPH RI WKH HYDOXDWLRQ. However, the flexible
KDQGOLQJ of any changes needed to keep a project relevant forms part of the definitions of the
other evaluation criteria, given below.

In an evaluation an analysis of Relevance will therefore focus on the following:

• LGHQWLILFDWLRQRIUHDO as distinct from perceived)SUREOHPVRUQHHGVand of the FRUUHFW

EHQHILFLDULHV;
• quality of assessment of ORFDODEVRUSWLRQFDSDFLWLHV;
• quality of assessment of ORFDOLPSOHPHQWDWLRQFDSDFLWLHV;
• where possible, SUHSDUDWRU\ DFWLYLWLHV undertaken (policy assessments, sector reviews,
[pre-]feasibility studies, planning workshops, &c), by whom, how well the findings were
incorporated into the final project document, and any obvious omissions;
• appropriateness of LQLWLDO FRQVXOWDWLRQV ZLWK, and SDUWLFLSDWLRQ E\, local key
stakeholders including the Delegation, national authorities, intended beneficiaries, and
other donors (the last-mentioned especially on complementarity aspects) before the design
was confirmed and implementation started;
• FRPSOHPHQWDULW\ DQG FRKHUHQFH ZLWK UHODWHG DFWLYLWLHV XQGHUWDNHQ HOVHZKHUH by
government or other donors, at the same level or at a higher level, rather than duplication or
conflict;
• the quality of the entries in the DVVXPSWLRQV ULVNV DQG FRQGLWLRQV column of the
LogFrame at the appropriate levels;
• overall GHVLJQVWUHQJWKVDQGZHDNQHVVHV including:
Πquality of the /RJ)UDPH (or LogFrames if a multi-component programme);
Πclarity and internal consistency of the stated RYHUDOOREMHFWLYHVSXUSRVHDQGUHVXOWV;
Πwhether the objectively-verifiable LQGLFDWRUV of achievement (OVIs) were well-
chosen and widely agreed;
Πrealism in choice and quantity of LQSXWV;
Πoverall GHJUHHRIIOH[LELOLW\and adaptabilityto facilitate rapid responses to changes in
circumstances.
• The quality of ILQDQFLDO DQGRU HFRQRPLF DQDO\VLV done at the design stage (assessing
planned effects on economic growth, salaries and wages, foreign exchange, budget and
comparing with other projects once prices are converted into shadow prices).
5
(IILFLHQF\

The efficiency criterion concerns how well the various DFWLYLWLHV transformed the available
resources into the intended UHVXOWV (sometimes referred to as RXWSXWV), in terms of quantity,
quality and timeliness. A key question it asks is “ZHUH WKLQJV GRQH ULJKW"” and thereby also
addresses value-for-money, that is whether similar results could have been achieved more by
other means at lower cost in the same time.

An analysis of Efficiency will therefore focus on:

• WKHTXDOLW\RIGD\WRGD\PDQDJHPHQW, for example in:
Πmanagement of the budget (including whether an inadequate budget was a factor);
Πmanagement of personnel, information, property, etc.
Πwhether management of ULVN was adequate, i.e. whether flexibility was demonstrated
in response to changes in circumstances;
Πrelations/co-ordination with local authorities, institutions, beneficiaries, other donors;
Πrespect for deadlines;
• FRVWVDQGYDOXHIRUPRQH\: how far the costs of the project were justified by the benefits
- whether or not expressed in monetary terms - that they generated4, in comparison with
similar projects or known alternative approaches, taking account of contextual
differences;
• SDUWQHU FRXQWU\ FRQWULEXWLRQV from local institutions and government (e.g offices,
experts, reports,. tax exemption, as set out in the LogFrame resource schedule), target
beneficiaries and other local parties: were they provided as planned, could re-allocation of
responsibilities have improved performance, were communications good?
• &RPPLVVLRQ+4'HOHJDWLRQLQSXWV (e.g. procurement, training, contracting, either direct
or YLD consultants/bureaux): key questions as for local/government inputs (above);
• WHFKQLFDODVVLVWDQFH: how well did it help to provide appropriate solutions and develop
local capacities to define and produce results?
• TXDOLW\RIPRQLWRULQJ: its existence (or not), accuracy and flexibility, and the use made
of it; adequacy of baseline information;
• whether the chosen LQGLFDWRUV of efficiency were suitable and, if not, whether
management amended them;
• did anyXQSODQQHGUHVXOWV arise from the activities?

(IIHFWLYHQHVV

4 a comparison broadly known as cost-effectiveness analysis, which differs from the narrower cost-benefit
analysis insofar as it covers intangible as well as tangible benefits.

6
The effectiveness criterion, in LogFrame terminology, concerns how far the project’s UHVXOWV
were XVHG or their potential benefits were realised - in other words, whether they achieved the
project SXUSRVH (sometimes referred to as the VSHFLILF REMHFWLYHV). The key question is what
difference the project made in practice, as measured by how far the intended beneficiaries really
benefited from the products or services it made available.

The analysis of Effectiveness will therefore focus on:

• ZKHWKHUWKHSODQQHGEHQHILWVKDYHEHHQGHOLYHUHGDQGUHFHLYHG, as perceived mainly
by the key beneficiaries, but also taking account of the views of donor management, the
responsible national Government authorities, and other concerned parties (NGOs,
business associations &c);
• the appropriateness of the LQGLFDWRUVRIEHQHILW used in the above assessment to measure
achievement of the project purpose (this is also relevant to cost-effectiveness analysis as
referred to in the footnote on the previous page), including a judgement on how far the
project management promptly amended indicators found to be inappropriate;
• in institutional reform projects, ZKHWKHU EHKDYLRXUDO SDWWHUQV KDYH FKDQJHG in the
beneficiary organisations or groups at various levels; and how far the changed
institutional arrangements and characteristics have produced the planned improvements
(e.g. in communications, productivity, ability to generate actions which lead to economic
and social development);
• if the DVVXPSWLRQVDQGULVNDVVHVVPHQWV at results level turned out to be inadequate or
invalid, or XQIRUHVHHQH[WHUQDOIDFWRUVintervened, KRZIOH[LEO\PDQDJHPHQWDGDSWHG
to ensure that the results would still achieve the purpose; and how well it was supported
in this by key stakeholders including Government, Commission (HQ and locally), etc.: in
summary, “ZHUHWKHULJKWWKLQJVGRQH´ to ensure that the potential beneficiaries actually
benefited?
• whether the EDODQFH RI UHVSRQVLELOLWLHV between the various stakeholders was correct,
which accompanying measures were or should have been taken by the partner
authorities, and with what consequences;
• how XQSODQQHGUHVXOWV may have affected the benefits received;
• whether any shortcomings at this level were due to a failure to take account of FURVV
FXWWLQJ RU RYHUDUFKLQJ LVVXHV such as gender, environment and poverty during
implementation (see Annex I).

,PSDFW

The term impact, sometimes referred to as RXWFRPH, denotes the relationship between the
project’s purpose and overall objectives, that is the extent to which the benefits received by the
target beneficiaries had a wider overall effect on larger numbers of people in the sector or
region or in the country as a whole. The analysis, which should be both quantitative and
qualitative wherever possible, will need to take account of the fact that, at this level, the project
will normally be only one of a number of influences contributing to the wider outcome.

At Impact level the analysis examines such aspects as:

7
• WR ZKDW H[WHQW WKH SODQQHG RYHUDOO REMHFWLYHV KDYH EHHQ DFKLHYHG, and how far that
was directly due to the project;
• in institutional reform projects, how far HQKDQFHG HFRQRPLF DQG VRFLDO GHYHORSPHQW
resulted from improved institutional capabilities and communications;
• in infrastructure-type projects, how far did they also enhance economic and social
development EH\RQG the level of their immediate users?
• how flexibly donor management and the Government responded and reacted to
unforeseen external shocks;
• if there were XQSODQQHGLPSDFWV, how they affected the overall impact;
• whether the project’s LogFrame LQGLFDWRUV at this level were suitable and, if not, whether
management amended them;
• where appropriate, all JHQGHUUHODWHGHQYLURQPHQWDODQGSRYHUW\UHODWHGLPSDFWV and
any lack of overall impact resulting from neglect of these issues (see Annex);
• whether overall the desired wider impact could have been better achieved otherwise;
• how the economic effects were spread between economic growth, salaries and wages,
foreign exchange, and budget.

6XVWDLQDELOLW\

The fifth and final - and often most important - criterion, sustainability, relates to ZKHWKHU
WKHSRVLWLYHRXWFRPHVRIWKHSURMHFWDWSXUSRVHOHYHODUHOLNHO\WRFRQWLQXHDIWHUH[WHUQDO
IXQGLQJHQGV, and also whether its longer-term impact on the wider development process can
also be sustained at the level of the sector, region or country

An analysis of sustainability will therefore focus on the aspects below. Their relative importance
will depend on the nature of the project; it is useful to examine how concern for, or neglect of, one
or other of the factors may have affected achievement of a sustainable outcome:

• RZQHUVKLSRIREMHFWLYHVDQGDFKLHYHPHQWV, e.g. how far all stakeholders were consulted

on the objectives from the outset, and whether they agreed with them and remained in
agreement throughout the duration of the project;
• SROLF\ VXSSRUW DQG WKH UHVSRQVLELOLW\ RI WKH EHQHILFLDU\ LQVWLWXWLRQV, e.g. how far
donor policy and national policy corresponded, and the effects of any policy changes;
how far the relevant national, sectoral and budgetary policies and priorities affected the
project positively or adversely; the level of support from governmental, public, business
and civil society organisations; and whether national bodies to provided resources;
• LQVWLWXWLRQDO FDSDFLW\, e.g. the degree of commitment of all parties involved, such as
Government (e.g. through policy and budgetary support) and counterpart institutions; the
extent to which the project is embedded in local institutional structures; if it involved
creating a new institution, how far good relations with existing institutions were
established; whether the institution appears likely to be capable of continuing the flow of
benefits after the project ends (is it well-led, with adequate & trained staff, sufficient
budget and equipment?); whether counterparts were properly prepared for taking over,
technically, financially and managerially;
8
• the DGHTXDF\RIWKHSURMHFWEXGJHW for its purpose;
• VRFLRFXOWXUDOIDFWRUV, e.g. whether the project is in tune withlocal perceptions of needs
and of ways of producing and sharing benefits; whether it respects local power-structures,
status systems and beliefs, and if it seeks to change any of those, how well-accepted are
the changes both by the target group and by others; how well it was based in any event on
an analysis of such factors, including target group/ beneficiary participation in design and
implementation; and the quality of relations between the external project staff and local
communities, notably their leaders;
• ILQDQFLDOVXVWDLQDELOLW\, e.g. whetherthe products or services provided were affordable
for the intended beneficiaries and remained so after funding ended; whether enough funds
were available to cover recurrent operating, running and maintenance costs, and continue
to do so after funding ended; and HFRQRPLF VXVWDLQDELOLW\, i.e. how well the benefits
(returns) compared to those on similar undertakings once market distortions are
eliminated;
• WHFKQLFDO WHFKQRORJ\  LVVXHV, e.g. whether (i) the technology, knowledge, process or
service provided fits in with existing needs, culture, traditions, skills or knowledge;
(ii) alternative technologies were considered, where there was a choice; (iii) the intended
beneficiaries were able to adapt to and maintain the technology acquired without further
assistance; with minimal national maintenance, operating and replacement costs; using
national resources (notably, in creating jobs); and with minimum waste;
• wherever relevant, FURVVFXWWLQJLVVXHV such as JHQGHUEDODQFHHQYLURQPHQWDOLPSDFW
and JRRG JRYHUQDQFH; or more over-arching issues such as SRYHUW\ DOOHYLDWLRQ, all of
which bear on sustainability from the outset of the project: see Annexes.

*HQHUDOLVVXHVUHODWLQJWRHYDOXDWLRQFULWHULDDQGGHVLJQ

The following points should also be borne in mind:

• when drafting Terms of Reference it is necessary to decide the UHODWLYH LPSRUWDQFH of

each of the key evaluation criteria for a given study, as it does not follow that all will
invariably be of equal importance (thus sustainability would not be a significant issue for
very short-term emergency relief actions);
• certain types of PXOWLFRPSRQHQWSURJUDPPH, whether sectoral or otherwise, may often
more helpfully be represented by LQWHUORFNLQJ /RJ)UDPHV than by a single LogFrame
(thus one component’s SXUSRVH or even RYHUDOO REMHFWLYHV might become DFWLYLWLHV or
UHVXOWVfor a higher-level programme, e.g. a sector or country programme);
• the possibility of UHSOLFDWLQJVXFFHVVIXOLPSDFWV should be borne in mind, especially if
an extension to the project is a realistic possibility.

9
 3DUW,,,Format and Presentation of Evaluation Reports

The findings of an evaluation are best presented in accordance with the following 5HSRUW)RUPDW
RU2XWOLQH.

Two introductory comments are pertinent:

• first, while it is important that evaluations are based on existing or re-drawn LogFrames,
which should always be annexed to evaluation reports, they should not be allowed rigidly
to determine the report structure; rather, WKHVWUXFWXUHRIDQHYDOXDWLRQUHSRUWVKRXOGEH
GHWHUPLQHGSULPDULO\E\LWVLQWHQGHGPDLQSXUSRVHDQGWDUJHWDXGLHQFH;

• second, it is most important, when drafting the report, to DFNQRZOHGJH FOHDUO\ ZKHUH
FKDQJHV LQ WKH GHVLUHG GLUHFWLRQ DUH NQRZQ WR EH DOUHDG\ WDNLQJ SODFH, and so avoid
misleading readers and causing unnecessary irritation or offence.

In general, the main sections of an evaluation report are as follows:

,([HFXWLYH6XPPDU\a tightly-drafted, to-the-point and free-standing Executive Summary

is an essential component. It should be short, no more than five pages. It should focus mainly
on the key purpose or issues of the evaluation, outline the main analytical points, and clearly
indicate the main conclusions, lessons learned and specific recommendations. Cross-references
should be made to the corresponding page or paragraph numbers in the main text that follows.

,,0DLQ7H[Wthe main text should start with an introduction describing, first, the project
or programme to be evaluated and, second, the evaluation objectives. The body or core of the
report should follow the five evaluation criteria discussed in Part II, describing the facts and
interpreting or analysing them in accordance with the key questions pertinent to each criterion.

,,,   &RQFOXVLRQV DQG 5HFRPPHQGDWLRQV these should be the subject of a separate final
chapter. Wherever possible, IRU HDFK NH\ FRQFOXVLRQ WKHUH VKRXOG EH D FRUUHVSRQGLQJ
UHFRPPHQGDWLRQ. The key points of the FRQFOXVLRQV will vary in nature but will often cover
aspects of the key evaluation criteria, for example:

L 5HOHYDQFH - whether the design of the project was originally, and still is, sound as regards
targeting the real needs and problems of the right beneficiaries;

LL (IILFLHQF\- whether the same results could have been achieved at lower costs; or whether
there might have been different, more appropriate ways of achieving the same results;

LLL (IIHFWLYHQHVV - whether the planned benefits were in fact received, whether the
beneficiaries’ behavioural patterns changed, whether neglect of cross-cutting issues
affected the achievement of the project purpose;
LY ,PSDFWthe wider outcomes for a larger group of persons or for society as a whole; the
successes and failures in achieving the overall objectives, and the main reasons why;

Y 6XVWDLQDELOLW\ - whether the flow of benefits to the beneficiaries, and to society

generally, is likely to continue or not, and why.

The ultimate value of an evaluation depends on the quality and credibility of the recommendations
offered. 5HFRPPHQGDWLRQV should therefore be as realistic, operational and pragmatic as
10
possible; that is, they should take careful account of the circumstances currently prevailing in the
context of the project, and of the resources available to implement them both locally and in the
Commission. They could concern policy, organisational and operational aspects for both the
national implementing partners and for the Commission; the pre-conditions that might be attached
to decisions on the financing of similar projects; and general issues arising from the evaluation in
relation to, for example, policies, technologies, instruments, institutional development, and
regional, country or sectoral strategies.

Recommendations should be FDUHIXOO\ WDUJHWHG to the appropriate audiences at all levels,

especially within the Commission structure (the project/programme task manager and the
evaluation manager will often be able to advise here).

,9$QQH[HV the report should generally include the following annexes:

1. The Terms of Reference of the evaluation

2. The names of the evaluators and their companies (CVs should be shown, but summarised
and limited to one page per person)
3. Methodology applied for the study (phases, methods of data collection, sampling &c)
4. Logical Framework matrices (original and improved/updated)
5. Map of project area, if relevant
6. List of persons/organisations consulted
7. Literature and documentation consulted
8. Other technical annexes (e.g. statistical analyses)
9. 1-page DAC summary, following the format incorporated in the contract and annexed to
this document.

11
$11(;, .(<&5266&877,1*,668(6)25&216,'(5$7,21,1
(9$/8$7,212)(&$,'352-(&76352*5$00(6

,*(1'(5

Gender aspects should be taken into consideration when carrying out an evaluation of any project in which they
could be of significance. All development actions touch male and female beneficiaries, and very often these two
groups as well as other sub-groups of beneficiaries will have different needs, responsibilities and potential for
benefit from the project. For any project to be as efficient and effective as possible it is important that such
differences are taken into consideration DWDOOSKDVHVRIWKHSURMHFWF\FOH.

7KH³*HQGHU´$SSURDFK

The “gender” approach is not concerned with women per se, but with the social construction of gender and the
assignment of specific roles, responsibilities and expectations to women and to men. A fundamental conceptual
difference between the Women in Development (WID) and Gender approaches is the assumption implied in the
WID approach that men are able to obtain access to all project related services and resources, and special measures
and efforts are necessary to protect and advance the interests of women. Field experience has, however, shown
consistently that this assumption is correct only in the case of better-off men; poor men are disadvantaged vis-à-vis
better-off men and often face many of the problems and constraints similar to those experienced by poor women.
Sensitivity to Gender would detect this. The gender approach does not focus solely on productive or reproductive
aspects of women’s and men’s lives; rather, it analyses the nature of the contribution of every member of society
both inside and outside the household and emphasises the right of everyone to participate in the development
process and to benefit from the results of the process.

6RPH.H\4XHVWLRQVWR&RQVLGHUZKHQ&DUU\LQJRXW(YDOXDWLRQV

Below are listed some questions to be addressed when carrying out evaluations in order to better understand
whether EC aid projects had a significant gender dimension and, if so, whether it was taken into consideration - as
it should according to the Commission’s current gender guidelines - in each phase of the project cycle.

Project Preparation and Design

• Are the beneficiaries clearly identified (sub-groups, age, socio-economic status, etc. “poor” or “women” is
not a homogenous group, so are more details needed)?
• Have these groups been consulted?
• Have their needs, resources and constraints to access the project services been identified?
• Have solutions been sought ?
• Where relevant, how well does the project take account of gender roles in reproduction and raising
children, work, and community management ?
• How well does it address gender-related needs that are (i) practical: access to food, water, shelter; social
services; paid work? (ii) strategic: reducing inequalities in work, domestic & childcare tasks, politics,
rights to land & property, credit, education; preventing male abuse of power and violence ?

Relevance
• Does the project respond to real needs formulated by the intended beneficiary group ?

Efficiency and Effectiveness

• Have appropriate delivery modes for services to reach all beneficiary sub-groups been identified and
implemented?
• Has the traditional division of tasks been taken into consideration?
• Have changes (by the project) to workload been considered?
• Who has access/control of project inputs?
• Is training provided to the right groups, given the project’s objectives?

12
• Do women/other vulnerable groups participate in the different phases of project implementation (the
number of women employed by the project is not necessarily an indication of female beneficiary
participation)?
• Are monitoring and information-gathering gender-differentiated ?

Sustainability
• Are gender aspects in the project mainstreamed or are there specific services for women?
• How can the access of women/other vulnerable groups to services and resources, be ensured?
• Who has access/control of the benefits?
• Has there been capacity building efforts to make local institutions aware of gender issues, capable to carry
out gender analysis and implement projects in a gender sensitive manner?
• Did social-cultural and gender aspects endanger the sustainability of the project during implementation or,
especially, after termination of donor assistance. Did opportunities for men and women to benefit equally
from the project will continue after its implementation (e.g. through women’s and men’s participation in
decision making; the issue of 'ownership' of the project activities by the various beneficiary groups and
implementing agencies should also be discussed)?
• How could better results have been achieved? How could beneficiary participation as between women and
men been improved?

,,(19,5210(17

Environmental aspects should also be taken into consideration when carrying out an evaluation of any project in
which they could be of significance. Many projects impact on the physical environment, some directly, others in
more subtle ways. For any project to be described as truly sustainable it is important that issues of environmental
impact are taken into consideration DWDOOVWDJHVRIWKHSURMHFWF\FOH. The following are some key questions from
which the most appropriate should be selected:

• was an environmental impact assessment made?

• was environmental damage done by, or as a consequence of, the project?
• to what extent did the project respect traditional ways of resource management and production?
• to what extent were environmental risks fully taken into account in the project?
• to what extent can it be expected that such risks will continue to be managed?
• overall, will the environmental effects of the project's activities and results jeopardise the sustainability of the
project itself or reach levels unacceptable for long-term environmental protection and management?

,,,)LQDO1RWH3RYHUW\$OOHYLDWLRQDQGRWKHURYHUDUFKLQJLVVXHV

In many EC programmes, SRYHUW\DOOHYLDWLRQ is the over-arching priority, the key question being how far a planned
project will benefit those in the poorest, most disadvantaged and most vulnerable groups. This requires careful analysis
of the various categories of potential beneficiary during project preparation, and a key element in successful attention
to this issue is an appropriate balance of participation both in the identification of objectives and associated indicators
(OVIs), and in their systematic monitoring during implementation, with responsibilities appropriately divided between
donor and recipient.

Other key cross-cutting issues include questions of JRRGJRYHUQDQFH and KXPDQULJKWV.

These issues are all too extensive and complex to be covered in an Annex of this nature. Evaluation Task Managers
and consultants should refer for more guidance to one or other of the numerous sources available in hard copy and on
the Internet.

13
$11(;,,7KH6WDQGDUG'$&)RUPDWIRU(YDOXDWLRQ5HSRUW6XPPDULHV5

(YDOXDWLRQ7LWOH7LWUHGHO HYDOXDWLRQ (YDOXDWLRQ5HIHUHQFH

$EVWUDFW$EUpJp FHQWUDOOLQHVOLJQHVPD[LPXP

6XEMHFWRIWKHHYDOXDWLRQ$FWLRQpYDOXpH
5 lines/lignes max on the project, organisation, or issue/theme being evaluated / au sujet du projet (&c) évalué

(YDOXDWLRQGHVFULSWLRQ'HVFULSWLRQGHO pYDOXDWLRQ
Purpose / But (3 lines/lignes max)
Methodology / Méthodologie (3 lines/lignes max)

0DLQILQGLQJV&RQFOXVLRQVSULQFLSDOHV
Clearly distinguishing possible successes/obstacles and the like where possible / identifiant si possible les
réussites/échecs et obstacles (25 lines/lignes max)

5HFRPPHQGDWLRQV5HFRPPDQGDWLRQV
25 lines/lignes max

)HHGEDFN5pWURDFWLRQ
To be completed by SCR/F/5  à compléter par l'unité SCR/F/5 (5 lines/lignes max )

Donor / Donateur: Region / Région: 6 DAC sector / Secteur CAD: 7

&RPPLVVLRQ(XURSpHQQH

Evaluation type / Type d’évaluation : 8 Date of report / Date du rapport: Subject of evaluation / Objet de l'évaluation:
9 10
««

Language / Langue : N° vol./pages : 11 Authors / Auteurs :

Programme and budget line concerned / Ligne budgetaire :12

Type of evaluation / Type d’évaluation : ( ) ex ante ( ) intermediate / en cours ( ) ex post

Timing / Dates  Start / Commencement : Completion / Achèvement :

Contact person / Personne responsable :13 Authors / Auteurs:

Cost / Coût:14 Steering group / Groupe de pilotage:

5 Text font should be Times New Roman 10 or equivalent / utilisez la fonte Times New Roman 10 ou l’équivalente.
6 If more than 3 countries but not continent-wide, choose a geographical region / Si plus de 3 pays, mais pas un continent, indiquer la
région géographique.
7 Choose from standard list / A choisir dans la liste standarde.
8 Choose between : relevance, efficiency, effectiveness, impact / Choisir entre : pertinence, efficience, efficacité, impact.
9 Date as on cover page of report / Date indiquée sur page de couverture du rapport.
10 Choose one of : programme/project/sector/country or region/synthesis/thematic/NGO / Choisir entre : programme/projet/secteur/pays
ou région/synthèse/thème/ONG.
11 Indicate number of pages per volume (e.g. 72 pp; 80 pp; 102 pp in case of 3 volumes) / Indiquer n° de pages par tome (ex 72pp; 80 pp;
102 pp dans le cas de 3 tomes)
12 Budget line / Ligne bugétaire (EDF, Tacis, Phare, etc.).
13 Name of the responsible person in SCR/F/5 / Nom de la personne responsable au SCR/F/5.
14 Cost of the evaluation / Coût de l'évaluation

14
15