Results Chains: A Tool for

Measuring Effectiveness
and Attributing Change to
Conservation Programs

Environmental Evaluators Forum

Washington D.C.
June 12, 2008
This Presentation

1. Background on the Use of Evaluation &

Adaptive Management in the Field of
Biodiversity Conservation
2. What Is a Results Chain
3. How to Develop Results Chains
4. How to Use Results Chains as a
Framework for Attributing Change to
Conservation Programs
5. Build Example Chains
This Presentation

• Background on the Use of Evaluation &

Adaptive Management in the Field of
Biodiversity Conservation
• What Is a Results Chain
• How to Develop Results Chains
• How to Use Results Chains as a
Framework for Attributing Change to
Conservation Programs
• Build Example Chains
Common Questions

• Are we achieving an impact?

• Are we doing the right things?

• Are we doing them well?

To Achieve Success, We Need To:

• Develop sound “theories of change”

• Measure the results of and improve
effectiveness of actions
• Assess impact on ecosystems and species
• Convince managers, donors, and
supporters that results are credible
How Can We Help Practitioners Meet
These Challenges?

Adaptive
Management
What is Adaptive Management?

The integration of design, management, and

monitoring to provide a framework for:
• Testing assumptions
• Adaptation
• Learning
Foundations of Success
Our Mission

To improve the practice of conservation through

adaptive management – working with
practitioners to systematically test assumptions,
adapt, and learn.
Foundations of Success
Our Goals
Our goals involve achieving the three
foundations of success:

• Foundation #1: Define clear goals and

practical measures of success.
• Foundation #2: Determine guiding
principles for using conservation strategies
and tools.
• Foundation #3: Develop and strengthen
practitioners’ ability to do AM
The Conservation Measures Partnership:
Leading Conservation Organizations
Core Members:

Collaborating Members:

Funding Support:
The CMP Open Standards
for the Practice of Conservation
www.ConservationMeasures.org

CMPinfo@ConservationMeasures.org
Many Versions of Adaptive
Management in Practice
CMP Open Standards
Results Chains – A Tool for
Implementing the CMP Open Standards

Results
chains
 How are the CMP Open Standards
Structured?

Steps
Sub-steps
Description

Outputs

References
 Example from the CMP Standards

Step 1: Conceptualize
Sub-step 1: Complete Situation Analysis
Description: This standard asks you to complete the description of
the context within which your project takes place…
Outputs:
• Identification and analysis of indirect threats and opportunities.
• Assessment of stakeholders.
• Initial conceptual model that illustrates cause and effect
relationships among factors operating at your site.
• Ground-truthing and revision of your model.
References
 Interchange Between CMP and Orgs
AWF Heartland Conservation Process

CI AWF
1. Priority 2. Heartland
Setting Selection

Initial Scoping

Learning and Performance

Adaptive And Impact
Management Assessment
Conservation Socio-economic
4. Heartland Strategy Target and Analysis
Implementation, Goal setting

TNC
Evaluation, and
Adaptation
3. Heartland/Landscape-
Implementation level Planning
of Priority
Interventions

Implementation Threats and

Planning Opportunity
Analysis

5. Scale-down Heartland
operations

1. Define
 Initial team
 Scope & vision
 Targets
 Context & stakeholders

5. Share
WWF’s
2. Design
CMP Define the
context
Vision and scope
Threats mapping
Stakeholders

WWF
 Lessons  Action plan: goals,
 Formal products objectives & activities
 Feedback & evaluation Conservation  Monitoring plan
 Learning culture Project/Programme  Operational plan Review Progress and
Revise Approach
Cycle Evaluate impacts Design Approach and
Adapt to changes Measures of Success

WCS
Share lessons learned Select targets
Create conceptual models
Build conservation landscapes
Monitoring frameworks

4. Analyze/Adapt 3. Implement
 Incoming data  Workplans & budgets
 Results & assumptions  Fund raising Implement Actions
 Operational functions  Capacity building and Measure
 Plans & budgets  Partnerships Effectiveness
Work-plans and Budgets
Implement actions
Monitor progress
This Presentation

• Background on the Use of Evaluation &

Adaptive Management in the Field of
Biodiversity Conservation
• What Is a Results Chain
• How to Develop Results Chains
• How to Use Results Chains as a
Framework for Attributing Change to
Conservation Programs
• Build Example Chains
What is a Results Chain?

A tool that clarifies assumptions about how

conservation strategies contribute to
reducing threats and achieving the
conservation of targets
What is a Results Chain?

The Basic Components of a Results Chain:

Strategy
What is a Results Chain?

The Basic Components of a Results Chain:

Strategy

Impact on
Target
Goal
What is a Results Chain?

The Basic Components of a Results Chain:

Strategy

Result Impact on
Result (Direct Threat ) Target
Objective Objective
Goal
 Results Chain Terminology

Logic Model
Inputs Activities Outputs Outcomes Impacts

Results Chain
Inputs Strategy Outputs Outcomes Outcomes
staff, time, set of actions immediate interim results interim results Impacts
money, other undertaken by a products of (objectives) (objectives) desired end goals
achieved by achieved by of the project
resources project project activities outputs outputs

Process Results
What is a Results Chain?

A diagram of a series of “if…then” causal

statements that:
• Defines how a project team thinks a
strategy will contribute to reducing a threat
and conserving a target
• Focuses on the achievement of results –
not the implementation of activities
• Is composed of assumptions that can be
tested
Results Chains

Implicit Assumptions:

Strategy ? Conservation
target improved

?
Achieving Success
SUCCESS!

Accurate used to Well executed leads to

design
Desired results
Results Chain project

THEORY FAILURE

Inaccurate used to Well executed does not

design
Desired results
Results Chain project lead to

PROGRAM FAILURE

Poorly
Accurate used to does not
design
executed Desired results
Results Chain lead to
project

TOTAL FAILURE!!!

Poorly
Inaccurate used to does not
design
executed Desired results
Results Chain lead to
project
This Presentation

• Background on the Use of Evaluation &

Adaptive Management in the Field of
Biodiversity Conservation
• What Is a Results Chain
• How to Develop Results Chains
• How to Use Results Chains as a
Framework for Attributing Change to
Conservation Programs
• Build Example Chains
How to Develop a Results Chain

1. Construct an initial results chain

2. Complete the links in the results chain
3. Verify that your results chain meets
criteria of a good results chain
A Chain From the Model Allows You
to…
Traditions & Scope:
Demand for Volga River
consumer
caviar Harvesting watershed
preferences
for caviar
High price for
caviar Sturgeon
Need to
generate Pollution
income from domestic
sewage
Inadequate
zoning Volga River
regulations & tributaries
Global warming
Limited gov’t
capacity for land
use planning
Dams
Need for Riparian
Rapid electricity forest
urbanization

Gov’t policies Clearing for

favorable to new home
urban dev. construction
Forest
Demand for corridors
Population second homes
growth

Failure of rural
Cheap land
economy
A Chain From the Model Allows You
to…
Traditions & Scope:
Media Demand for Volga River
consumer
campaign caviar Harvesting watershed
preferences
for caviar
High price for
caviar Sturgeon
Need to
generate Pollution
income from domestic
sewage
Inadequate
zoning Volga River
regulations & tributaries
Global warming
Limited gov’t
capacity for land
use planning
Dams
Need for Riparian
Rapid electricity forest
urbanization

Gov’t policies Clearing for

favorable to new home
urban dev. construction
Forest
Demand for corridors
Population second homes
growth

Failure of rural
Cheap land
economy
1. Construct an Initial Results Chain

Healthy
sturgeon
population
1. Construct an Initial Results Chain

Reduction in Healthy
harvesting for sturgeon
caviar population
1. Construct an Initial Results Chain

Reduction in Healthy
Media
campaign ? harvesting for
caviar
sturgeon
population
1. Construct an Initial Results Chain

Consumers Decreased Reduction in Healthy

Media
prefer not to demand for harvesting for sturgeon
campaign
eat caviar caviar caviar population
 2. Complete the Links in the Results
Chain

Consumers Decreased Reduction in Healthy

Media
campaign ? prefer not to
eat caviar
demand for
caviar
harvesting for
caviar
sturgeon
population
 2. Complete the Links in the Results
Chain

Increased
Consumers Decreased Reduction in Healthy
Media
campaign
knowledge of
importance of
sturgeon
? prefer not to
eat caviar
demand for
caviar
harvesting for
caviar
sturgeon
population
 2. Complete the Links in the Results
Chain

Increased Increased
Consumers Decreased Reduction in Healthy
Media knowledge of interest in
prefer not to demand for harvesting for sturgeon
campaign importance of sturgeon
eat caviar caviar caviar population
sturgeon conservation
 3. Review the Criteria for Good Results
Chains

Increased Increased
Consumers Decreased Reduction in Healthy
Media knowledge of interest in
prefer not to demand for harvesting for sturgeon
campaign importance of sturgeon
eat caviar caviar caviar population
sturgeon conservation

• Results oriented: Boxes contain desired results

(e.g., reduction of hunting), and not activities
(e.g., conduct a study).
• Connected in a “causal” manner: There are
clear connections of “if…then” between each pair
of successive boxes.
 3. Review the Criteria for Good Results
Chains

Increased Increased
Consumers Decreased Reduction in Healthy
Media knowledge of interest in
prefer not to demand for harvesting for sturgeon
campaign importance of sturgeon
eat caviar caviar caviar population
sturgeon conservation

• Demonstrates changes: Each box describes

how you hope the relevant factor will change
(e.g., improve, increase, or decrease).
 3. Review the Criteria for Good Results
Chains

Increased Increased
Consumers Decreased Reduction in Healthy
Media knowledge of interest in
prefer not to demand for harvesting for sturgeon
campaign importance of sturgeon
eat caviar caviar caviar population
sturgeon conservation

• Relatively complete: There are sufficient boxes

to construct logical connections but not so many
that the chain becomes overly complex.
• Simple: There is only one result per box.
How to Develop a Results Chain

1. Construct an Initial Results Chain Based

on Your Conceptual Model
2. Complete the Links in the Results Chain
3. Verify that Your Results Chain Meets
Criteria of a Good Results Chain
What is a Results Chain?

A diagram of a series of “if…then”

statements (“causal”) that:
• Defines how a project team thinks a
strategy will contribute to reducing a threat
and conserving a target
• Focuses on the achievement of results –
not the execution of activities
• Is composed of assumptions that can be
tested
What is NOT a Results Chain?

It is not an implementation flow diagram…

Monitor &
Produce Distribute Healthy
Media Identify target evaluate the
educational educational sturgeon
campaign audience campaign’s
materials materials population
effectiveness
 Your Turn: Which of the Following is NOT a
Results Chain?

Lobbying of Identify key Educate Decision Research & Jaguar

A. government for
stronger decision
makers
decision
makers
makers pass
laws
develop
regulations
No wildlife
trade
populations
increased
regulations

Coastal
Increased forests
yields conserved
Increased Less
Promotion of Farmers implement permanence conversion of
B. sustainable
agriculture
sustainable of agricultural forest to
agriculture methods occupation agriculture
More Miombo
permanent woodland
crops conserved

Community Greater More control Less illegal

More illegal Primary
C. capacity building
for forest resource
indigenous
knowledge
of & vigilance
over external
wood
confiscated
selective logging
in indigenous
forest
conserved
management about rights actors communities
This Presentation

• Background on the Use of Evaluation &

Adaptive Management in the Field of
Biodiversity Conservation
• What Is a Results Chain
• How to Develop Results Chains
• How to Use Results Chains as a
Framework for Attributing Change to
Conservation Programs
• Build Example Chains
Steps to Attributing Change to Programs

1. Define the Program’s “Theory of Change”

2. Develop Key Results into Good
Objectives
3. Define Indicators for Objectives and
Goals
4. Measure Indicators
5. Define What Results are Directly vs.
Indirectly Attributable to Program
Example from the Meso-American Reef

Mexico

Belize

Guatemala
Honduras
1. Define the Program’s Theory of Change
1. Define the Program’s Theory of Change

Promote
Coral
Agricultural Best
reefs
Mgmt Practices
(BMPs)

Obj Ag6, Ag7

Man-
Less groves
agrochemical
contamination in
marine waters &
organisms
Sea-
grasses

Littoral
KEY zone

Threat
Strategy Result Reduction Target
Result
1. Define the Program’s Theory of Change

Obj Ag1 Obj Ag2

Obj Ag3
Promote Bioaccumulation Companies agree
Companies Coral
Agricultural Best study completed to participate in Companies
accept BMP reefs
Mgmt Practices & shared with program to reduce sign MOU
plan
(BMPs) agroindustry agrochemicals

Obj Ag6, Ag7

Man-
Less groves
agrochemical
contamination in
marine waters &
organisms
Sea-
grasses

Littoral
KEY zone

Threat
Strategy Result Reduction Target
Result
1. Define the Program’s Theory of Change

Obj Ag1 Obj Ag2

Obj Ag3
Promote Bioaccumulation Companies agree
Companies Coral
Agricultural Best study completed to participate in Companies
accept BMP reefs
Mgmt Practices & shared with program to reduce sign MOU
plan
(BMPs) agroindustry agrochemicals

Obj Ag6, Ag7

Obj Ag4, Ag5 Man-
Less Less toxic Less groves
Companies agrochemical
agrochemical agrochemical
implement contamination in
toxicity in contamination in marine waters &
BMPs
plantations streams & rivers organisms
Sea-
grasses

Littoral
KEY zone

Threat
Strategy Result Reduction Target
Result
1. Define the Program’s Theory of Change

Obj Ag1 Obj Ag2

Obj Ag3
Promote Bioaccumulation Companies agree
Companies Coral
Agricultural Best study completed to participate in Companies
accept BMP reefs
Mgmt Practices & shared with program to reduce sign MOU
plan
(BMPs) agroindustry agrochemicals

Obj Ag6, Ag7

Obj Ag4, Ag5 Man-
Less Less toxic Less groves
Companies agrochemical
agrochemical agrochemical
implement contamination in
toxicity in contamination in marine waters &
BMPs
plantations streams & rivers organisms
Sea-
grasses
Other Industry
companies adopts
accept BMPs BMPs
Littoral
KEY zone

Threat
Strategy Result Reduction Target
Result
 2. Develop Key Results Into Good Objectives
Ag5: By the end of 2012, reduce by 40% the
total pesticide toxicity from fungicides,
insecticides, and herbicides used on
Obj Ag1 Obj Ag2
Obj Ag3

banana,
Promote
citrusBioaccumulation
Agricultural Best and
study sugarcane
completed
Companies agree
inin highCompanies
to participate run-off Companies
accept BMP
Coral
reefs
Mesoamerican
Mgmt Practices
(BMPs) Reef watershed
& shared with
agroindustry
program to reduce
agrochemicals areas
sign MOU
plan

Obj Ag6, Ag7

Obj Ag4, Ag5 Man-
Less Less toxic Less groves
Companies agrochemical
agrochemical agrochemical
implement contamination in
toxicity in contamination in marine waters &
BMPs
plantations streams & rivers organisms
Sea-
grasses
Other Industry
companies adopts
accept BMPs BMPs
Littoral
KEY zone

Threat
Strategy Result Reduction Target
Result
 3. Define Indicators for Objectives and Goals
Ag5: By the end of 2012, reduce by 40% the
total pesticide toxicity from fungicides,
insecticides, and herbicides used on
Obj Ag1 Obj Ag2
Obj Ag3

banana,
Promote
citrusBioaccumulation
Agricultural Best and
study sugarcane
completed
Companies agree
inin highCompanies
to participate run-off Companies
accept BMP
Coral
reefs
Mesoamerican
Mgmt Practices
(BMPs) Reef watershed
& shared with
agroindustry
program to reduce
agrochemicals areas
sign MOU
plan

Obj Ag6, Ag7

Obj Ag4, Ag5 Man-
Less Less toxic Less groves
Companies agrochemical
agrochemical agrochemical
implement contamination in
toxicity in contamination in marine waters &
BMPs
plantations streams & rivers organisms
Sea-
grasses
Other Industry
companies adopts
accept BMPs BMPs
Littoral
KEY zone

Threat
Strategy Result Reduction Target
Result

Indicator: total pesticide toxicity from

fungicides, insecticides, and herbicides used on
banana, citrus and sugarcane
 3. Define
Ag1: By the endIndicators
of 2008, developfor
a listObjectives
of and Goals
agrochemicals to reduce…

Obj Ag1 Obj Ag2

Ag4: By the end of 2012, at least 16
Promote Bioaccumulation Companies agree
companies are implementing BMP
Obj Ag3

Companies Coral
Agricultural Best
Mgmt Practices
study completed
& shared with
to participate in
program to reduce
Companies
sign MOU programs
accept
plan
BMP reefs
(BMPs) agroindustry agrochemicals

Obj Ag6, Ag7

Obj Ag4, Ag5 Man-
Less Less toxic Less groves
Indicator: list Companies
implement
agrochemical agrochemical agrochemical
contamination in
toxicity in contamination in marine waters &
developed BMPs
plantations streams & rivers organisms
Sea-
Goal for CoralOtherReefs: Industry
Goal for Coral Reefs: grasses

Indicator: # of By 2017, ataccept

least
companies
25%adopts
of coral reefs in the
BMPs BMPs
companies implementing MAR (including all reef types) are Littoral
zone
KEY
BMP programs Threat ecologically viable.*
Strategy Result Reduction Target
* ecologically viable = accretion rate >/= X,
Result

live coral coverage of 7-20%, healthy

population of herbivores, “balanced” species
composition, etc.
4. Measure Indicators
4. Measure Indicators

Indicator: total pesticide toxicity on banana

plantations

Possible Monitoring Strategies:

• Pre-test / Post-test
• Time-series
• Comparison with strict control group
• Comparison with comparison group
5. Define What Results are Directly vs. Indirectly
Attributable to Program

Obj Ag1 Obj Ag2

Obj Ag3
Promote Bioaccumulation Companies agree
Companies Coral
Agricultural Best study completed to participate in Companies
accept BMP reefs
Mgmt Practices & shared with program to reduce sign MOU
plan
(BMPs) agroindustry agrochemicals

Obj Ag6, Ag7

Obj Ag4, Ag5 Man-
Less Less toxic Less groves
Companies agrochemical
agrochemical agrochemical
implement contamination in
toxicity in contamination in marine waters &
BMPs
plantations streams & rivers organisms
Results directly Sea-
grasses
attributable to Other
companies
Industry
adopts
WWF accept BMPs BMPs
Littoral
KEY zone

Threat Results
Strategy Result Reduction
Result
Target
indirectly Impossible
attributable to to attribute
WWF results
solely to
WWF
This Presentation

• Background on the Use of Evaluation &

Adaptive Management in the Field of
Biodiversity Conservation
• What Is a Results Chain
• How to Develop Results Chains
• How to Use Results Chains as a
Framework for Attributing Change to
Conservation Programs
• Build Example Chains