Presentation GAC RBM May 2017

Results-Based Management in
Global Affairs Canada’s International

Assistance Programming
Short Introduction
(Session for Indonesia Program Partners

May 5, 2017)
The content and descriptions contained in this document have been developed by Result-
Based Management Centre of Excellence (RBMCE) of Program Coherence (PCC) of the
International Assistance Envelope Management Bureau, Strategic Policy Branch, Global
Affairs Canada and is the property of the Government of Canada. This document may not
be edited or modified without prior authorization from RBMCE.
Session Objectives
1. Introduce Results-Based Management

(RBM) as a management approach
2. RBM in GACs international assistance
programming: quick overview of
methodology and main concepts and tools
3. Results-based Reporting
Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

Objective #1
INTRODUCE RBM AS A MANAGEMENT APPROACH

What is Results Based Management?
 RBM is a management approach that asks

practitioners to look beyond activities and outputs
to focus on RESULTS (OUTCOMES)
not only during project design and
planning,
but also during implementation. How?
 By measuring progress on or toward
results, learning, and adapting (taking
corrective action) during implementation,
not after
RBM Illustration: School
 Each grade has clearly

defined expected outcomes
(follows curriculum)
 Each level (of skills,

knowledge and behavior)
is a building block to the
next

RBM Illustration 2
 Would we be happy with

a report card that tells us
the teacher provided 300
hrs. of math lessons to
our child?
 Would we wait to
measure the progress of
our child till the 3rd or 8th
year?
RBM Illustration 3
We manage the development of our children
diligently. We don’t wait till end of each year. We measure
their progress regularly.
How?
Indicators Collection methods

• Marks • Tests
• Behaviour • Conversations
• Observation

RBM Illustration 4
WHY do we do all
this?
For evidence of how our child is doing, so we can:

• Determine progress towards passing the grade
• Identify problems early, take corrective action now
In other words, to
better manage the
development of our
child

Kai’s Story
• Our six year old son Kai just
started Grade 1
• Expected Outcome: “Increased

skills and knowledge in math”
(1st building block for next level
of change)
• Target: “100% completion of

Grade 1 math curriculum by
year end”

Kai’s Story 2
• By mid-term, he is not doing
so well.
• I take corrective action = hire

tutor, thus:
• introducing a new activity
• realigning budget (cannot
buy a car)
BUT we don’t change the expected outcome: “Increased skills and

knowledge in math”, or target: “100% completion of Grade 1
math curriculum by the end of the year”.
The “destination” remains the same.
Slide Source: Global Affairs Canada, PFM, PCC, RBM Center of Excellence.
Kai’s Story 3
• By the end of the year, Kai has not accomplished the Grade 1 math
curriculum.
• An in-depth assessment (evaluation) finds that Kai has a learning
challenge.
• Corrective action = special tutor or school & extend timeline.
BUT we don’t change the expected outcome: “Increased skills and

knowledge in math”.
The “destination” remains the same, but it will take longer.
We adjust our target to “100% completion of Grade 1 math curriculum by

the mid-point of Grade 2” to reflect this significant and unforeseen change in
context

In Short…
RBM is a management
methodology that
… focuses on knowing
where you want to be,
… knowing where you are
as you go, and
… adjusting what you are
doing during your trip to
ensure you reach your
destination.

Objective No. 2
RBM IN INTERNATIONAL ASSISTANCE PROGRAMMING:

QUICK OVERVIEW OF METHODOLOGY AND MAIN CONCEPTS
AND TOOLS
What is a Result?
 Results are the same as outcomes.
“An outcome is a describable or measurable change
that is derived from an initiative’s outputs or lower
level outcomes. Outcomes are qualified as immediate,
intermediate, or ultimate.”
So, what is a
result statement?
Source: 2016 GAC RBM How-to Guide

The Result Statement – Planning Stage
 A result statement Different types of change:
outlines what a policy,  Change in state
program, or project is
 Change in circumstances
expected to achieve
 Change in behaviour
 Change in performance
 A result statement
describes the change we  Change in practices
expect stemming from  Change in attitude
the contribution we aim  Change in skills,
to make through GAC’s knowledge, ability…
projects (in cooperation
with others).
Language of Change
 Change is signified by words such as:
 Improved (heath conditions)
 Increased (financial security for local farmers)
 Strengthened (capacity of local NGOs to advocate)
 Reduced (vulnerability)
 Enhanced* (ability to apply RBM)
 Using change words implies both sustainability and

continuity: in international assistance programming,
we recognize that change is incremental and part of an
ongoing continuum, rather than static or complete.
*"Enhanced" is used to capture both “improved” and “increased” change.

A Strong Expected Outcome Statement
 A strong expected outcome statement starts with a word that
indicates the DIRECTION of the change expected, and tells you
WHAT will change, WHO will experience the change, and WHERE
that change will take place.
 It is formulated in a specific way:

Structure of an outcome statement
Direction What Who Where
in selected
usage of agriculture
Increased by dairy farmers, especially women farmers, communities in rural
extension services
Sampleland
awareness of availability
Increased by adult women and teenage girls
of basic health care in Region X
during pregnancy
Alternative order
Where
Direction What Who What
to develop gender
sensitive economic
Strengthened knowledge and skills of staff in institution X international assistance in county X
policies

Questions to ask when drafting/assessing a
Result Statement
 Is the statement simply worded?
 Does it contain only one idea?
 Does the result statement include an adjective (drawn from a
verb that indicates DIRECTION) and tells you:
 WHAT?
 WHO?
 WHERE?
 Can the result be measured?
 Is the result realistic and achievable?
 Is the result relevant?
 Was the result statement developed in a participatory fashion?
Outcome Quiz
An outcome, or result is a describable or measurable change in
state that is derived from an projects’ outputs or lower level
outcomes.
Is this a strong expected Outcome Statement?
 Quality Basic Education
 Enhanced provision of maternal health care services by local clinics to
women in community A
 Increased financial security for women in community B
 Reduced vulnerability to HIV infection among itinerant male workers in
urban centers of Tanzania
 Peace in Country C
 Reduced child mortality rates among children under 5 in country D

What is a Gender Equality Result?
GE results are changes that explicitly

address a reduction in gender
inequality, or an improvement for
gender equality between women and
men, girls and boys.
and
GE results contribute to one or more of
GAC’s GE corporate results.
20
Is this a GE Result?
1. Improved rural women's and men’s participation in community
decision-making.
2. Increased awareness of the rights of women and girls.
3. Improved inclusiveness in access to training for rural farmers.
4. Reduced gender inequalities in access to and control over the
resources and benefits of community infrastructure decisions.
5. Strengthened knowledge and skills of women and men
cooperative members.
6. Increased influence of women’s organizations in community
consultations with the private sector.
7. Improved women’s and men’s participation in climate change
response initiatives.
8. Strengthened local gov’t programs to equitably respond to
rights violations of women, men, girls and boys in selected
rural communities. 21
What is an output?
 An output is a direct products or services stemming from the activities of an
organization, policy, program, or initiative.
 Outputs are the products and services stemming directly from project activities
implemented by partners Global Affairs Canada funds (held accountable).
Useful questions for wording outputs at planning stage Outputs DO NOT

 Does it include a verb? describe a change;
 Does it say what was delivered/rendered? Outputs ARE NOT
 Does it specify in what subject/area? results (outcomes)
 Does it say to or for whom services are rendered or products are delivered?
 Does the implementer have control over it?
Structure of an Output Statement
What What subject/area Verb To or for Whom
to regional government policy branch
Technical assistance in strategic planning provided
employees
Alternative Order
What Verb What subject/area To or for Whom
on legal instruments (e.g. laws, policies,
Technical assistance provided to personnel in organization Y
legislations, model laws and regulations)

Output Quiz
Is this a strong output?

 Basic literacy and numeracy training provided to marginalized
women in City B
 Reports published
 Capacity of regional government workers to manage services is
built
 National tuberculosis action plan implemented
 Technical assistance for the implementation of a national
malaria action plan provided to selected staff in the Elbonian
Ministry of Health
 Emergency food aid distributed to households in camp A

What is a Results Chain?
A depiction of the logical relationships

that illustrate the links between
inputs, activities, outputs, and the
outcomes of a given policy, program,
or project.
Source: 2016 GAC RBM How-to Guide

What is a Results Chain? - 2
 The results chain addresses practitioners’ need for a concept
that allows them to break complex change down into
manageable building blocks or steps that lead to one another,
making it easier to sequence and identify changes during both
analysis and planning.
 These steps also become the points at which practitioners will

measure whether or not the expected change is actually
occurring throughout project implementation.
 Each organization will have its own results chain, which will
depict and define the number and type of building blocks or
levels it uses. Not all results chains look alike.
Various results chains used around the world
CIDA CIDA, GAC & TBS NORAD DFID

AusAID OECD
(1996-2008) (Since 2008) (2008) (2011)
Impact Ultimate
Outcome
Goal/Impact Impact Impact Impact
(Medium-term)
Outcomes
Intermediate Purpose/
Outcomes Outcome
Outcomes Outcome Outcomes
Immediate Component
Outcomes (Short-term)
Objectives
Outputs
Outputs Outputs Outputs Outputs Outputs
Activities Activities Activities Activities Activities Activities

Global Affairs Canada’s Results Chain
Source: Adapted by GAC from University of Wisconsin-Extension, Enhancing Program and Performance with Logic Models
Theory of Change
 Theory of Change builds on the findings of the
Definition: a set of
situation analysis
assumptions, risks and
 In RBM the “Theory of Change” is the evidence- external factors that
based story of the logical relationships and describes how and why
associated assumptions and risks within a project. the project is intended
to work.
 At GAC, the Theory of Change at the project level is
reflected in the Logic Model and the Outputs and This theory connects the
Activities Matrix, and fully explained in an project’s activities with
accompanying (ToC) narrative. its expected ultimate
outcome.
 Theory of Change is inherent in the project design
and is often based on research, evaluations, best
practices and lessons learned.

TO MANAGE FOR RESULTS, GAC USES THESE
PRIMARY RBM TOOLS:
- Logic Model
- Theory of Change Narrative,
- Outputs/Activities Matrix, and
- Performance Measurement Framework

The Logic Model (LM)
 Is a visual depiction of the main elements of a project’s or
country program’s theory of change, reflecting the series of
changes that are critical to achieving project or program
outcomes
 Should not go over one page
 Depicts the logical connections between the planned outputs
and the expected outcomes the project or program aims to
achieve or contribute to
 The project level LM starts at the ultimate outcome level and
now ends at output level
 The program level LM starts at the ultimate outcome level, with
the activities level being a lists of planned and operational
projects
The Logic Model (LM)
• The LM’s pyramid structure enables practitioners
to illustrate the complex and multifaceted nature
of international assistance programming – the
convergence of different, but complimentary
pathways of change under one ultimate outcome.
• Different intermediate outcomes represent

different “pathways” leading to the same ultimate
outcome.
• Each “pathway” addresses a different aspect of the problematic.
• Keep in mind while the pathways of change flow vertically, in reality there
is a dynamic, complementary, horizontal relationship between the
different pathways within a logic model

The Logic Modelling Process
1. Identify design team and relevant stakeholders

2. Conduct a situation analysis
3. Develop theory of change, including outcomes (ultimate,
intermediate, and immediate), outputs, assumptions and
risks
4. Develop LM to reflect theory of change
5. Write narrative of theory of change to accompany LM
6. Validate all with stakeholders/partners
Steps 2 to 5 should be done in a participatory way

LM Example
ULTIMATE 1000 Improved health of women, men, girls and boys living in community X
OUTCOME
1100 Increased proper usage of safe drinking water by 1200 Improved management of water, waste and
INTERMEDIATE
women, men, girls and boys in community X sanitation infrastructure in community X
OUTCOMES
1110 Increased equitable 1120 Increased knowledge and 1210 Increased 1220 Increased knowledge and
access to safe drinking awareness of the importance ability of women in skills in waste management and
IMMEDIATE water for women, men, and proper usage of safe community X to sanitation among male and
OUTCOMES girls and boys living in drinking water among women, maintain wells female local community
community X men, girls and boys in committee members
community X
1111 Wells built in 1121 Awareness material, 1211 Training 1221 Technical assistance in
community X, in including material appropriate provided to women waste management and
consultation with local for a non-literate audience, in community X on sanitation provided to local
stakeholders, especially developed in consultation with maintenance of wells community committees members,
women as primary water male and female community both women and men
managers in the members
OUTPUTS community
1122 Awareness campaigns
1112 Existing wells conducted on the importance
rehabilitated in community and proper usage of safe
X drinking water for women,
men, girls and boys in
community X

Results Chain vs. Logic Model
 While some practitioners use the terms “results chain” and “logic model ”
interchangeably, GAC differentiates between the two.
 The results chain provides a conceptual model for how a given organization
breaks change down into building blocks or steps.
 It establishes and names the levels that will be used when that organization
undertakes the development of the theory of change as part of project
design.
 Because change, particularly the types of change expected from
international-assistance programming, is complex and multi-faceted, a
theory of change includes several complementary pathways of change that,
in combination, lead to one ultimate outcome.
 The LM pyramid structure is more useful to enable practitioner to illustrate
this complexity, i.e., the convergence of different, but complimentary
pathways of change under one ultimate outcome.

GE results in the Logic Model
based on findings from the

gender analysis
supported by immediate
outcomes, outputs (and activity
matrix)
35
Results Chain example
Ultimate Outcome: Improved health and education of

marginalized people in the region.
Intermediate Outcome: Enhanced involvement of civil society

in social service planning and implementation that meets the different
needs of women, men, girls and boys in the region.
Immediate Outcome: Strengthened skills of local governments

to engage civil society for gender-responsive social services delivery,
particularly to marginalized people in the region.
36
GE Integrated Project example
Ultimate
Improved living conditions for women and men in poor rural areas.
Outcome
Title No. PTL
Country/Region/
Plaisance Budget Duration
Institution
200 Improved organizational

100 Improved access to agriculture
and economic efficiency and
Intermediate inputs/ services and to decision-
gender-responsiveness of
Outcomes making in farm business by
key stakeholders in the spice
producers, particularly women.
value chain.
110 120 Increased 130 210 Increased 220 Increased

Increased knowledge Increased knowledge and capacity of
skills of and skills of awareness skills of stakeholders
agricultural agricultural of men to agricultural to optimize
producers, producers, promote cooperatives in business
particularly particularly decision- the spice value operations.
Immediate women, to women, in making with chain and in
Outcomes optimize leadership women. mechanisms for
agricultural and decision- gender-
inputs. making in responsiveness.
farm business
Avoid the following common problems in a LM
• Process
 The logic model was developed by only one person
 Project team is engaged after the logic model is developed and no effort is
made to validate it with them.
 No local stakeholders were involved in developing the logic model.
 Outcome statements are not realistic/overly ambitious.
• Logic
 Logic model is not linked to any problem or stakeholder analysis.
 Desired changes have been reduced to overly simplistic results statements.
 Gender equality is not integrated to Global Affairs Canada standards.
 There are gender equality activities, but no gender equality outcomes
 Objectives are used at the ultimate outcome or any other level, instead of
outcomes
 Tautologies: saying the same thing with different words. In the logic model
this often manifests as an outcome which summarizes the level below and
does not describe a substantively different change.
Avoid the following common problems in a LM - 2
• Outcome Statements
 Statements that are general and generic.
 The intended change is not clear.
 The logic model has too many intermediate outcomes.
 Statement includes more than one idea or change (“and”).
 ‘Through’, ‘via’, ‘by’, ‘for’, ‘due to’ or ‘in order to’ or other expressions in a
result statement that describe linkages to other levels of the logic model.
 Statement includes targets.
 Logic model contains too many details and is confusing.
 Statements describe changes at the wrong level of the logic model.
 Statements include targets.
• Output Statements
 Output statements include change words like “strengthened.”
 Statement includes targets.
 Statement is too long, vague or wordy to communicate the output being
delivered.
 The output represents an activity that could fall under another output.
 The range of activities presented in the outputs and activities matrix is too
limited to allow for the production of the output.
Avoid Tautology in the LM
 Tautology Definition: Saying the same thing with different words.
 In the LM this often manifests as an outcome which summarizes the
level below and does not describe a substantively different change.
For example: an ultimate outcome that summarizes the changes
described in the intermediate outcomes.
Not at right level – this is

another Intermediate
Ultimate Outcome outcome which
This is Improved use of well managed water, waste summarized the two
Tautology and sanitation infrastructure by women, intermediate outcomes
men, girls and boys in community X below.
A correct ultimate outcome
could be: Improved health
of women, men, girls and
boys living in community X.
Intermediate Outcome Intermediate Outcome Or Reduce vulnerability to
Increased proper usage of safe Improved management of water, water born illness for men,
drinking water by women, men, girls waste and sanitation women, girls and boys.
and boys in community X infrastructure in community X

Logic Model (LM) – Can it be Revised?
• The LM is developed in the planning and design stage
• It is not static - it is an iterative tool.
• As project’s circumstances evolve during project implementation, or, if the
analysis of the data collected on the indicators suggest adjustments are
required to achieve the expected outcomes, changes to the LM may be
required.
• The LM should be revised to reflect the changes.
• Changes must be discussed and agreed amongst all project stakeholders,
including GAC. In addition, such adjustments must be justified,
documented and RESPECT the scope, scale and intent of the project.
• If the changes affect the scope, scale and intent of the project, or imply
significant increases to the resources/funding required, they will need to
be approved by the original approval authority at Global Affairs Canada.
NOTE: Changes to the LM and PMF that trigger the Canadian Environmental Assessment Act (2012) will require that steps be taken in compliance with the
Act. Source: Canadian Environmental Assessment Act, Statutes of Canada 2012, c. 19, s. 52. Retrieved from http://laws-lois.justice.gc.ca/eng/acts/c-
15.21/FullText.html. For additional guidance, please refer to the Results-Based Management for International Assistance Programming How-to Guide, 2nd
edition, 2016 (EDRMS#7433286) and the APP Implementation Guide
Outputs and Activities Matrix (OAM)
• The OAM is a companion to the LM theory of change narrative.
 Together, they capture the project’s “Theory of Change” from the ultimate
outcome to the activities.
• The OAM is a table which displays the activities of the project,

organized by immediate outcome and outputs.
• Outputs and activities are not expected results, but play an important
role in the theory of change of a project.
 The OAM helps GAC staff and its partners ensure that the project has a
design conducive to achieving the expected outcomes
 Once a project begins implementation, the OAM can also serve as the
basis for the annual work plan’s schedules and budgets.

Outputs and Activities Matrix
Outputs and Activities Matrix
Immediate Outcome 1110
Output 1111
Activity 1111.1
Activity 1111.2
Activity 1111.3
Output 1112
Activity 1112.1
Activity 1112.2
Immediate Outcome 1120
Output 1121
Activity 1121.1
Activity 1121.2
Activity 1121.3
Output 1122
Activity 1122.1
Activity 1122.2

The Theory of Change Narrative
 Every LM should be accompanied by a narrative that

describes the theory of change for the project
 The narrative should:
 Be structured by outcomes
 Focus on what is not explicit in the logic model
 Explain the linkages between each level, including assumptions
 Provide reference to the evidence and best practice justification for design
choices made
 Include a brief mention of any key risks and contextual factors that could
influence the achievement of results
 Where applicable, include how environmental sustainability, gender equality
and governance are integrated throughout the Logic Model to the
intermediate-outcome level.

Does an LM give practitioners enough information to
monitor, manage and report on performance?

Performance Measurement Framework (PMF)
 In order to monitor and evaluate your project, it is important to establish a

structured plan for the collection and analysis of performance information.
The PMF:
 Is a plan to collect relevant data on performance indicators to assess
progress made in achieving results
 Is developed in consultation with partners, other donors, local
stakeholders and sometimes beneficiaries
 The PMF is not a paper exercise, like all plans, the PMF has to be used.
 Remember, the data collected on the performance indicators is used:

 To assess progress on the achievement of results
 For ongoing, evidence-based decision making to manage the project for
results
 To demonstrate progress on results through reporting
 To facilitate and inform evaluations
Performance Measurement Framework (PMF)
Title No. Team Leader
Country/Region/ Budget Duration

Institution
DATA
BASELINE
EXPECTED RESULTS1 INDICATORS2 TARGETS3 DATA SOURCES COLLECTION FREQUENCY RESPONSIBILITY
DATA
METHODS
ULTIMATE OUTCOME
INTERMEDIATE OUTCOMES
IMMEDIATE OUTCOMES
OUTPUTS
1. from Logic Model

2. Gender and Environment where possible
3. including time range (where possible)

Indicator (Performance)
• An indicator, also known as a "performance indicator",

is a means of measuring actual outcomes and outputs.
• It can be qualitative or quantitative, and is

composed of a unit of measure, a unit of analysis and
a context.
• Indicators are neutral: they neither indicate a

direction of change, nor embed a target.

Quantitative Indicators
 Are discrete measures, used to measure quantities or

amounts.
 For example:
 # / total of human rights violations,
 ratio of women-to-men in decision-making positions in the

government,
 # / total of small farmers (F/M and region/urban) who have used

extension services in the past year,
 % / total of women-owned businesses represented in trade fairs.

Qualitative Indicators
 Capture experiential information, such as the quality of something, or

beneficiaries’ perception of their situation.
 Can help measure the presence or absence of specific conditions, or an individual

or group’s perception of how a service compares with established standards.
 Can capture contextual information about situations, events and practices.
For example: Note: Qualitative indicators can be

• Level of (1-5 scale), “quantified”; i.e. expressed using a scale
• Quality of, as perceived by… (Level of…) or quantitative units of
• Extent to which (1-4 scale)… measure such as
• Level of congruence with,
“%/total women interviewed who felt
• Etc.
that…..”

Performance Indicators
Performance indicators are composed of 3 elements:
Unit of measure Unit of analysis Context
of girls living within a 1 hour walk of a
#/Total provincially funded public school
(rural/urban)
health institutions providing services to ethnic populations

%/Total in their language of choice
(public/private)
Level of confidence of rural farmers in the security of police-patrolled rural

(1-5 scale) (f/m) roads leading to and from markets
mostly or completely confident (4 or 5 on

a 5 point scale) in the security of police-
%/Total rural farmers (f/m) patrolled roads leading to and from
markets

Hints for Selecting Indicators
• Choose 2-3 indicators for each outcome and 1-2 for each output
• This allows for the collection of different kinds of information
about performance
• Ensure that each adjective in the outcome statement (equitable,
effective, gender sensitive, etc.) is measured by an indicator
• Have a balance of qualitative and quantitative indicators
• Ensure that indicators capture proportion:
 % of total, # out of total
• Ensure there is an indicator that measures the reach at each level
of the LM
 How many people are experiencing the change described?
• Wherever people are the unit of analysis, make sure the
indicator is gender-disaggregated

Performance Indicators vs. Results
 Often, there is confusion between results (outcomes)

and performance indicators.
 Results (outcomes) are statements of the changes to be

achieved, e.g.
 Increased literacy among groups X and Y in select regions
 Improved health of women and girls in village X
 Indicators measure results (outcomes); they are neutral

and do not indicate the direction of change, nor embed a
target:
 % of total literacy rate,
 level of knowledge as perceived by female and male X and Y (using a 4-point
scale where 1 means very low and 4 means high)

Criteria of a Good Performance Indicator
1. Validity: Does it measure the result?
2. Reliability: Is it a consistent measure over time?
3. Sensitivity: When the result changes will the indicator be

sensitive to those changes?

Examples of
Gender Sensitive Indicators
 % women and men in wage employment in the non-
agriculture sector
 % women and men in decision-making positions
 Women and men’s perception of their employment
opportunities
 Level of participation of men in child care
 Men’s level of comfort with the increased mobility of
women
 Percentage of first marriage before age of 15 (f/m)
 % of pregnancies under 16 years old
 Level of awareness of women and men of laws on
domestic violence
Criteria of a Good Performance Indicator
4. Simplicity : Will the information be easy to collect and
analyze?
5. Utility: Will the information be useful for decision-

making and learning?
6. Affordability: Can the program/investment afford to

collect the information?
Note: Indicators need to be disaggregated by gender, age, ethnicity, socio-economic

status, etc… wherever possible if they are to be useful for decision making and
measuring results.
Baseline
 Baseline data provides a specific value for an indicator at the
outset of a project.
 Baseline data is collected at one point in time, and is used as a
point of reference against which progress on or toward the
achievement of results will be measured or assessed.
 A baseline is required in order to establish realistic, achievable
targets.
 A baseline is needed for each performance indicator in the PMF.
Indicator Baseline
% of total women (by 83% of 300 women (200
community) walking to community A and 50
river for water daily. community B)

Targets
 A target specifies a particular value, or range of values, to be reached by a
specific date, for a performance indicator
 In the GAC PMF, the target column must show end-of-project targets, but
shorter-term targets can also be indicated
Strong Targets:
 Disaggregated in the same way as indicators
 Developed using an established baseline
 Realistic and reviewed regularly
 Developed involving beneficiaries and stakeholders to ensure local ownership
and achievability
Indicator Target
% of total women End of Project: 25-30% of 300
(by community) women (from both communities)
walking to river for walk to the river daily to get
water daily. drinking water by 2017.

Remember…
Results (outcomes): are statements of the changes to be achieved
Performance indicators: measure results (outcomes); they are

neutral and do not indicate the direction of change, nor embed a
target:
So what do targets add?
Targets: are projections/estimates... specify a particular value for an
indicator to be accomplished by a specific date in the future:
 Indicator = %Literacy rate/total (f/m)(group X/Group Y)
 Targets =
 fX: 80%/200 by 2015
 mX: 85%/350 by 2015
 fY: 84%/270 by 2015
 mY: 90%/180 by 2015

Data Source
 Data sources are the sources of performance information from
which data about the performance indicators will be obtained
 individuals: beneficiaries (m/f/g/b), officials, reporters, etc.
 organizations: co-op or association, government body, court, CSOs, UN
bodies, communities, other donors, etc.
 or documents: project reports, UN reports, newspaper, research paper,
diary, map, picture, etc.
 To ensure reliability data sources should not change over time
(within reason)
Primary Data: Data collected directly by the implementing organization

Secondary Data: Data that has been collected and recorded by another person
or organization, sometimes for altogether different purposes.

Data Collection Methods
 Data-collection methods represent how data about
indicators is collected
 They can be informal and less structured, or more
formal and more structured
 Choosing a data-collection method depends on:
 A project or an organization’s resources, access, needs, time
constraints, etc.
 The type of indicator
 How the information collected will be used
 How often this information will be gathered
Taken from World Bank, Jody Zall Kusek & Ray C. Rist, Ten Steps to Result-based Monitoring and Evaluation System, p. 85.

Data Collection Methods
Source: Jody Zall Kusek and Ray C. Rist, 2004 Ten Steps to a Results-based Monitoring and Evaluation System, World Bank, page 85

Frequency and Responsibility:
When and by whom is data collected?
Ultimate Every 2-3 years during life of a project & after

Outcome
Continuous
Intermediate Annually
Outcomes
Immediate
Outcomes Quarterly
Outputs Weekly
Daily
By whom: Implementing organization, local partners, beneficiaries, local, regional, national

government, international organization, other donor, GAC, etc.

Elements of a PMF
Title Safe water for rural communities No. A0006857 Team Leader Jane Smith
Country/Region/Institution Ghana Cost $1.2 million Duration FYs 2012-2017
Expected Indicators Baseline Target Data Collection Frequency Responsibility

Outcomes Sources Methods
% of total 83% of 300 Year 1: 65% of • Women • Interviews •Annually • Implementing

Increased use of women (by women (200 300 women • Children • Focus • Semi- Organization
safe drinking community) community A (from both Group annually • External
water by walking to and 50 communities) Monitor
women, men, river for community walk to the river
girls and boys in water daily. B) daily to get
region X. drinking water.
Year 5:
25-30% of 300
women (from
both
communities)

Objective #3
RESULTS-BASED REPORTING
Reporting?
Source: Dilbert by Scott Adams, January 27, 2002, found on the following website:
http://search.dilbert.com/search?p=Q&srid=S3-
USESD01&lbc=dilbert&ts=custom&w=Ad%20Campaign&uid=397008890&method=and&isort=date&view=list&filter=type%3acomic&srt=6

What is Results-Based Reporting?
Reporting on actual outcomes against expected outcomes: comparing

what you expected to achieve with what you actually achieved, and
explaining any variation between the two
This is in addition to financial reporting and

reporting on outputs, activities
 Not only how many schools were built, but also improved access to quality
education and improved learning outcomes of girls and boys
 Not only how many wells were drilled, but also increased use of safe drinking
water by the women, men, girls and boys
Source: Retrieved from Embassy of Canada to Colombia, in

August 2016.
How to Report on Outcomes – Big Picture
Assess what is being achieved (actual outcomes) using actual

data (qualitative and quantitative) collected on indicators
identified during planning (in the PMF or equivalent) &
collected during monitoring and evaluation
Use the output and outcome data as evidence to support

your assessment of progress on the expected outcomes in
your report
Source: Retrieved from UNMAS,, Photo in August 2016.

How to Report on Outcomes - Details
 A partner's report should provide the following for each expected outcome and
output:
Actual Data for every indicator
 For every indicator, provide actual data as per the collection frequency identified in your
monitoring plan or framework (PMF,LFA etc.)
Narrative Assessment of Progress that includes:

• Progress during the reporting period on the expected Outputs & Immediate Outcomes
 Provide an evidence based narrative that describes the progress made during this reporting
period on or toward achieving the planned output or expected outcome using data
collected on the indicators identified in the PMF for this output or outcome.
• Variance and unexpected outcomes
 Explain any variance. If the project has identified annual targets, it is in this section where
you would address instances where the actual and expected targets differ by more than
approximately 10%. Analytical comments on the significance of these variations should be
presented.
• Progress from project inception to date (cumulative) on the expected outputs, and
immediate, intermediate and ultimate outcomes
 Provide an evidence based narrative that describes the progress made (from project
inception to date) on or toward achieving the expected outcome using the data collected
on the indicators identified in the PMF for this outcome.
Example No. 1 – Simple Illustration
How to use indicator data to report on actual results
• Expected Outcomes:
– Improved Access to Information and Privacy request (ATIP) processing by GAC
staff
• Indicators and Actual Data

Indicators Actual Data
Average # of days turn-around time for requests Average 12 days turn-around time
for requests
Level of client satisfaction as defined by timeliness and quality of Data not yet collected
service (1-4 scale where 1 is dissatisfied and 4 is very satisfied)
• Actual Outcomes - Progress to date towards the expected outcomes:

– There has been a modest improvement to date in ATIP processing by staff, as
evidenced by a drop in the average turn-around time, from 15 days to 12.
There is anecdotal evidence that clients are more satisfied with the timeliness
and quality of service, although a survey has not yet been conducted.

Progress “on” and “toward”
• When reporting on outcomes, we can speak about progress “on” or “toward” the
achievement of that outcome. This difference allows us to report on progress
“toward” an outcome even without a significant change in the value of indicators
for that outcome.
– Progress on is defined as actual change in the value of indicators being tracked for the
respective outcome or output. An outcome or output is considered to have been
achieved when its targets have been met.
– Progress toward is defined as actual change in the value of indicators tracked at the next
level down in the logic model (i.e. the intermediate outcomes, or their supporting
immediate outcomes, or their supporting outputs depending on the level in question),
with an explanation of how they are expected to lead to the higher-level outcome.
• When there has been no perceptible change in the actual value of indicators at the
respective outcome level, go to next level down in the logic model.
• For example, if there has been no perceptible change in the actual value of
indicators at the intermediate outcome level, go to the supporting immediate
outcomes and their indicators.

Proportionality and Scope
 When reporting on the expected outcomes, the report should not only
provide evidence of the change, but also proportionality and scope of that
change, context and cross-cutting themes:
• Proportionality and scope of that change, for

example:
 78% of 20,000 women living in rural communities in three
regions out of 15 are now using pre-and post-natal health
services in comparison to 60% at the start of this program;
or
 Out of 1,000 street youth living in six impoverished
neighbourhoods in city X, 20% now have access to youth
drop-in centers in comparison to only 10% at the start of
the program.
Context
• Provide sufficient context, for example:
 Where did the outcome occur (region and distribution in
the region)?
 Who, and how numerous, were the beneficiaries or
intermediaries who experienced the change – women,
men, girls and boys, specific groups, organizations?
 Context that may have had an effect on performance
during reporting season (political, economic, social,
environmental, a risk occurring…)

Example No. 2 – How to use indicator data to report on actual results
(using one outcome and one indicator from a 5-Year Education project)
Planning Stage Reporting Stage

Selected Elements/Columns of a Performance Measurement (3rd year of a 5-year
Framework project)
Expected Indicators Baseline Data End of Project Actual Data
Outcomes Targets
Intermediate %/total of 40% of District Y 70% of District Y 50% of District Y Grade V
Outcome: District Y Grade Grade V girls (out of Grade V girls (out of girls (out of 1,215 girls)
Improved V children 1,123 girls) tested, total number of girls) tested, scored a
proficiency in (girls/boys) scored a minimum tested, scored a minimum 60/100 on a
math by girls tested, who 60/100 on a minimum 60/100 on standardized math test
and boys in score a standardized math a standardized math in 2013
primary minimum test in 2010. test in 2015.
school in 60/100 on a
District Y in standardized 50% of District Y 70% of District Y 60% of District Y Grade V
country X math test. Grade V boys (out of Grade V boys (out of boys (out of 1,312 boys)
1,245 of boys) total number of boys) tested, scored a
tested, scored a tested, scored a minimum 60/100 on a
minimum 60/100 on minimum 60/100 on standardized math test
a standardized math a standardized math in 2013
test in 2010. test in 2015.

Example No. 2 cont. – Actual Outcomes
Using indicator data as evidence of progress on expected outcomes
Expected Intermediate Outcome: Improved proficiency in math by girls and boys in
primary school in District Y in country X
Since the start of the project, 3 years ago, there has been a significant improvement in
math proficiency for both girls and boys in primary school in District Y in Country X. Out of
1,215 female Grade V students tested in 2013, 50% scored a minimum 60/100 in
math. This is an increase of 25% from the baseline of 40% in 2010. Likewise, out of 1,312
male Grade V students tested in 2013, 60% scored a minimum 60/100 in math. This is an
increase of 20% from the baseline of 50%. This means that more Grade V students are
passing the standardized math test than was the case at the beginning of the project. This
also means that there is a greater possibility that students will transition to lower secondary
school after successfully completing primary school.
Note: This is a 5-year project. The improved proficiency in math by the Grade V girls and
boys (intermediate outcome) stems from other results in this project’s logic model, such as
improvement in teachers’ gender-sensitive pedagogical skills (immediate outcome) and
their better use of the new, improved math textbooks and other teaching learning materials
(intermediate outcome). While the girls’ improvement is greater than their male
counterparts (25% versus 20% improvement in the math test pass rate of 60/100), the
project will be challenged to ensure the girls catch up with the boys by the end of the
project.

What GAC looks for in Reports from
Implementers
• GAC looks for overall, a clear “Evidence-based Performance Story”;
more specifically, for:
 Actual data for each indicator identified in the PMF or equivalent,
as per the data collection schedule in the PMF and reporting
frequency
 Analysis of that data, in the form of a narrative on actual outcomes
that describes the progress made on or towards each of the
planned output and expected outcome
 The use of this data as evidence of progress in the narrative
performance story – e.g. Out of 1,215 female Grade V students tested
in 2013, 50% scored a minimum 60/100 in math. (see previous slide)
 An explanation of any variance between expected outcomes and
actual outcomes
 Unexpected results (positive or negative) with an assessment or
explanation
Expected vs. Actual Outcomes
• Expected Results are clearly defined at the
start of any intervention & form the focal
point for any ongoing management decisions Expected
Results
• Implementation must continue to focus on
achievement of expected results and taking
corrective action as needed Inputs,
Corrective Activities,
Actions Outputs
• Reporting focuses on the comparison of the
actual results being achieved against
expected results established in the beginning Actual
of the project: comparing what you expected Results
to achieve with what you actually achieved,
and explaining any variation between the two.

Project Reports - Issues
• Reports only describe complete activities, outputs (products developed,
and services delivered), e.g. “27 Wells Built”
• Reports are organized/structured by activities, not outcomes
• Reports lack evidence to support claims of progress on or towards
outcomes (or only have activity or output level evidence) and lack
examples to illustrate the outcomes achieved
• Although a PMF was created at the planning stage, no outcome level
monitoring (and evaluation) system was established, or the monitoring
system was established but not implemented.
 Thus, outcome level information was not collected on an ongoing
basis, and thus not available for assessment of progress or as
evidence of this progress for reporting.
• Risk – No mention about the status of the risk & if any action was taken to
respond to it.
• Reports are overly positive for fear of losing funding

Reporting - Conclusion
• Reporting is a means of communicating about project’s performance and
achievements (actual outcomes) to a variety of stakeholders & decisions
makers in a way that will enable them to make decisions.
• The foundation of good outcome reporting is a good result-based

monitoring and evaluation system established and diligently maintained
from start to project end.
• Results-based monitoring should yield useful performance information that

can be used by decision makers during project implementation to take
corrective action (make adjustments) in order to maximize achievement of
the expected outcomes (immediate, intermediate and ultimate).
• If data on outcomes is not collected and / or not being used actively, then
RBM is not being applied (you cannot manage for results)!
• Data collected through results-based monitoring, is essential for both:

Reporting and Results-Based Project Management

Results-Based Management is:
1. Defining realistic expected results (outcomes), based on
appropriate analyses;
2. Clearly identifying program beneficiaries and designing programs to
meet their needs;
3. Monitoring progress towards results (outcomes) and resources
utilized, with the use of appropriate indicators;
4. Identifying and managing risks, while bearing in mind expected
results (outcomes) and the necessary resources;
5. Increasing knowledge by learning lessons, integrating them into
decisions during project implementation (taking corrective action)
and after to other similar projects/programs; and
6. Reporting on actual results (outcomes) achieved (supported by
data as evidence) and the resources involved
Global Affairs Canada, PFM, PCC, RBM Center of Excellence. Source: GAC’s RBM Policy 2008
End of Session – Any Questions?

Presentation GAC RBM May 2017

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Presentation GAC RBM May 2017

Uploaded by

Copyright:

Available Formats

Results-Based Management in

Global Affairs Canada’s International

(Session for Indonesia Program Partners

1. Introduce Results-Based Management

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

INTRODUCE RBM AS A MANAGEMENT APPROACH

 RBM is a management approach that asks

 Each grade has clearly

 Each level (of skills,

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

 Would we be happy with

Indicators Collection methods

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

For evidence of how our child is doing, so we can:

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

• Expected Outcome: “Increased

• Target: “100% completion of

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

• I take corrective action = hire

BUT we don’t change the expected outcome: “Increased skills and

The “destination” remains the same.

BUT we don’t change the expected outcome: “Increased skills and

The “destination” remains the same, but it will take longer.

We adjust our target to “100% completion of Grade 1 math curriculum by

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

RBM IN INTERNATIONAL ASSISTANCE PROGRAMMING:

Source: 2016 GAC RBM How-to Guide

 Using change words implies both sustainability and

*"Enhanced" is used to capture both “improved” and “increased” change.

 It is formulated in a specific way:

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

GE results are changes that explicitly

Useful questions for wording outputs at planning stage Outputs DO NOT

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

Is this a strong output?

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

A depiction of the logical relationships

Source: 2016 GAC RBM How-to Guide

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

 These steps also become the points at which practitioners will

CIDA CIDA, GAC & TBS NORAD DFID

Activities Activities Activities Activities Activities Activities

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

• Different intermediate outcomes represent

• Each “pathway” addresses a different aspect of the problematic.

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

1. Identify design team and relevant stakeholders

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

based on findings from the

Ultimate Outcome: Improved health and education of

Intermediate Outcome: Enhanced involvement of civil society

Immediate Outcome: Strengthened skills of local governments

200 Improved organizational

110 120 Increased 130 210 Increased 220 Increased

Not at right level – this is

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.

• The OAM is a table which displays the activities of the project,

Global Affairs Canada, PFM, PCC, RBM Center of Excellence.