Facilitation Protocols

J-PAL SEA Research Resource

Overview

• Why do we need protocols?

• Implementation planning
• Field safety protocols
• Training of facilitators
• Piloting
• Implementation
– Intervention monitoring
– Field visit planning
– Monitoring activities
– Intervention data collection
– Reconciliation
 • To minimize challenges in field coordination
• To maintain the rigor of the study
• Ensuring that we can eventually infer causality
from the evaluation
Why do we need – Differences in outcomes should be attributable
to whether or not study participants receive
protocols? treatment and not due to other factors (e.g.
quality of facilitation)
• To feed into process evaluation
• To ensure the safety of staff, team, and study
participants
J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 3
 Implementation Planning

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 4
 Local context

Budget and
Scope of the study
timelines

Implementation
plan

Key decisions
(e.g.: in-house Permissions
vs. outsourcing)
Understanding local context: checklist

Type Completed ?(Y/N/NA) Type Completed ?(Y/N/NA)

Permissions and Internet connection

authorizations
Terrain Printing option

Electricity Storage

Topography Office space

Holidays and major Training venues

events
Availability of target Gifts/compensation
populations
Transport options Facilitation equipment

Pool of potential Office supplies

facilitators

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 6
Preparing for unexpected events

Anticipating unexpected events and working around them is critical to

ensure that our plan is successful

Buffer
• Time allocation
• Staff allocation

Why?
• There are always unexpected events, for example:
– Natural disasters
– Sample villages are found to not actually exist
– Social turbulence (conflict, strikes)
• Researchers are known to change their minds often, for example:
– New sections added to the facilitation manual

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 7
Field Safety Protocols
 Ensuring the safety of
everyone

• Planning for safety

– Piloting to understand the field challenges
– Ensuring safe transportation &
accomodation for the staff and field team
– Securing intervention materials
– Assessing specific risks in relation with the
projects (e.g. geographical, cultural, etc)

• Creating and enforcing staff safety policies

– Sexual harassment policy
– Road safety and helmet policy
– Safety while in the field policy (dress code,
working hours, etc.)
– Safety and security of equipment (including
compensation when applicable)
– Emergency Card and Second Emergency
Contact
– Insurance for staff and field team

• Identifying and minimizing risks to respondents

– Risks of Participation
– Data Confidentiality
– Informed Consent
J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 9
 Training of Facilitators

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S
Objectives of facilitator training

• To standardize and maintain the quality of interventions

• To identify the strengths and weaknesses of the field team
• To ensure that the field team is able to understand and
deliver standardized interventions

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 11
Before training begins

Task

Make sure that training manual is comprehensive and in accordance with the study
protocols
Make sure that training logistics are well prepared, including location, food,
compensation, and training materials
Make sure that training schedule is efficient and in line with the overall project timeline

Prepare human subject training material

Prepare a child protection agreement (if applicable)

Prepare social media guidelines

Prepare a consensus log

Prepare monitoring forms (spot and back checks)

Create emergency cards for field officers

Collect emergency contact information from field officers

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 12
Training checklist
Task
Have a separate, dedidacted training for supervisors, managers, leaders to ensure better
coordination and smooth implementation of activities
Review training manual comprehensively (section by section)

Make sure that technical training is conducted

Conduct human subject training

Ensure that there is a safety training
Ensure that there is role clarity by clarifying all roles (facilitators, RA, supervisors, etc.)
Thoroughly review the facilitation instrument
Ensure that field officers/facilitators understand the organization (objective, mission, work and issues
we take seriously), policies (sexual harassment, road safety, child protection), project
Ensure that field officers/ facilitators understand data confidentiality policy
Teach adequately about research
Observe training sessions and answer questions that may arise
Walk through, mock facilitation, or role play (supported by accompaniments and feedback
sessions)
Have facilitators sign social media guidelines
Distribute emergency cards
Collect the facilitator assignment sheets (for tracking)
Document all changes and consensus agreed during the training
13
Piloting
Objectives of Piloting

• To test the intervention materials

• To test the procedure (e.g., listing, implementation)
• To understand the logistical requirements (e.g. Transportation,
accommodation)
• To understand the sample population
• To map possible challenges and risks

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 15
What could happen in the field?
CASE 1: Piloting
One part of the project
intervention is inviting women in
a village who are interested in
migrating abroad to attend
community meeting. In the
community meeting, they will be
provided with information about
various official placement
agencies operating in Indonesia
and how their qualities differ
based on reviews from previous
migrants
What would happen if
we do not pilot the
intervention first?
Findings
The participants came with their
Result & Consequences
passport to the meeting
because they thought the
The invitation method might
meeting was part of migrant lead to non-targeted people
worker recruitment. Some
attending the meeting, deliver a
middlemen also showed up to false message, increase
the meeting and confronted the
potential risks and conflicts
organizers because they felt
threatened by ‘a competitor’. It
was also found that international
migration was a socially sensitive
issue in the village.
16
Testing the Materials

• Do the intervention materials appeal targeted study participants?

• Can the intervention materials be understood by the targeted study
participants?
• Do the intervention materials need to be changed?
– If so, what changes are needed?

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 17
Testing the Procedure

• Listing:
– How long does it take to list the study participants? (Depend on the listing
criteria)
– How long does it take to find the potential study participants?
• How long does it takes to deliver the intervention(s)?
• Is the current procedure working well?
• Is the flow of data/intervention working well?
• Is any revision to the procedure needed?
J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 18
 Understanding Logistical
Requirements

• What are the local conditions?

– How are road connectivity and network
quality?
– What are the geographical challenges
(terrain, area size, density, etc.)?
– How to get to the targeted area?
Vehicle rental? Public transportation?
Ojek? How long does it take?
– How to send intervention materials to
the area? How long does it take?
• How to stay in the targeted area?
– Hotel? Hostel? Guesthouse? Village
office? Villager’s house?
– Can you set up a base camp? Where?
• Is any permission letter or authority
letter required?
– How to process the letter?

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 19
Understanding Your
Sample

• How many study participants will you

target?
• How are the study participants
clustered?
• How will you find study participants?
• Do the study participants need to be
interviewed at a particular time of day
or year?
• What type of information do you need to
collect from each study participant?
• Do you expect other special challenges
in interacting with these study
participants? (e.g.: taboo, customary
law, local conventions, language
barrier)

Not for implementation, but important for

data collection:
• Where will you interview study
participants?

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 20
Mapping Possible Challenges and Risks

• What are the possible risks in delivering the intervention(s) that you
have thought during the design phase?
– e.g. sensitivities, potential conflict, trust issues, competing interventions,
noncompliance, spillover, refusal, low take-ups
– Do any of those risks occurs during the pilot?
• Do any other unexpected challenges occur ?
• How to mitigate those challenges and risks?
– possible revision to the procedure or materials

If any revision is
made, how would
that impact the
overall timeline?

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 21
Implementation
 Intervention Monitoring

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 23
Objective of
Intervention Monitoring

Quality Control
To ensure the quality and
standardization of the process of
implementing interventions

1. Monitor treatment
2. Track study participants over
the course of the
experiment

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 24
What could happen in the field?

Case 2: Intervention
Implementation
• The facilitators should do
secondary visit to the small Result &
retailers and provide short Consequence:
guidance and training on
how to do book keeping • The intervention was not
and restocking to the standardized. This might
owner. lead to wrong inference.

Findings:
• From 5 targeted retailers
owner, only 2 were visited
by facilitator B.
What would happen
if the situation keeps
happening
unnoticed?

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 25
Intervention monitoring summary

Activities PLANNING IMPLEMENTATION RECONCILIATION

• Spot
checks/shadow Conduct additional
Field-Based checks checks if required
• Back checks
• Review team
allocation and
scheduling
• Provide hard
• Identify and fix
copies of target
discrepancies
participants to • Compare master
• Update timeline
facilitation teams and facilitator
(and budget), if
• Create tracking tracking sheets
needed
sheets every day
• Refine strategy, if
• Schedule audits
Office-Based needed
(spot
checks/shadow
checks/back
checks)

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 26
 Planning

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 27
 Guidelines for site visits

1. Cover both routine tasks and

pressing issues
2. Use checklists and folders to
organize site visit tasks
3. Keep in mind that treatment
groups should be treated
identically to control groups in
every respect except for the
treatment itself
4. Do not tell people they are in a
treatment or a control group
5. Come up with standard responses
if there are questions about the
program or research and share the
responses with your facilitators
6. Submit a Back-to-Office Report
after your every visit

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 28
 Frequency of site visits

• Quality assurance starts on Day 1

• Typically, the frequency of monitoring

required depends on:
– Complexity of the treatments
– Size of the study
– Partner organization’s willingness and ability
to comply with study protocol (due to
personal interests, work pressures, or other
reasons)

• If you find early on in the process that there

is basically perfect compliance with the
research protocols, you may be able to
increase the intervals between regular
monitoring visits over time.

• If you notice problems or inconsistencies

during monitoring, you should increase both
the frequency and scope of monitoring
activities.

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 29
 Monitoring Activities

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 30
Monitoring intervention:
step by step

• Selecting study participants to monitor

– Subjects should be selected randomly for
monitoring and stratified by implementing field
staff /facilitator well as location if some are more
remote or there are other reasons to believe that
compliance would vary by location.
– You may find it necessary to monitor extra study
participants affected by weak facilitator(s) or
difficult location(s).

• When monitoring involves extensive travel to reach

isolated individuals or groups, you must make a
rough analysis of the costs and benefits associated
with additional monitoring visits.

• Monitoring interviews with study subjects

randomized into control are usually not advised as
they could bias subjects’ behavior and possibly
encourage them to seek offers made to subjects in
treatment
– However, some form of monitoring is necessary to
verify that control groups have no access to
treatments, and ideally have no knowledge of
offers and/or products associated with treatment.

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 31
Making life easy when monitoring: Monitoring form

Create a template spot check/back check form to document findings from

monitoring activities by including:
• Date and time
• Monitor’s name and title
• Facilitator’s name and ID
• Location
• Quality of facilitation
• Think about things you look for when you are doing monitoring (e.g.,
timeliness, key study protocols, completeness of equipment, confidence,
respect, urgent issues, suggestions for facilitator)

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 32
Monitoring activities

• The best methods for monitoring implementation will depend on study details
and the monitoring options available, and will normally include a combination
of field observation and regular collection of administrative data.
• You should monitor implementation as much as possible at the start, in order to
ensure that partner organization staff is complying with the research protocol
and that any issues are immediately corrected

Activity Implementation
Is the implementation is on track and
Tracking (Office-Based)
targeting the intended sample?
Do the facilitators deliver the intervention
Spot Checks (Field-Based)
consistently according to the procedure?
Do the facilitators actually deliver the
Back Checks (Field-Based) intervention to the targeted participants and
do they do it according to the procedure?

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 33
Office-Based
Monitoring: Tracking
• Objectives: to ensure that
assigned tasks are completed
by facilitators in a timely
manner
• Tracking is conducted by
matching process data from
the following:
– Assignment sheets (i.e. details
of how implementations tasks
are assigned to each team of
facilitators)
– Tracking sheets (i.e. list of which
facilitator deliver which
intervention to whom, where,
when)

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 34
How to conduct tracking

Activities PLANNING IMPLEMENTATION RECONCILIATION

• Facilitators and
supervisors fill in
Tracking Sheets
Participant revisits
Field-Based after
implementation is
completed
• Identify missing
participants
• Team allocation is • Compile all
• Update
identified in the Tracking Sheets
productivity
Assignment Sheet • Check for
estimates
• Tracking Sheets discrepancies
• Update timeline
are prepared for between
and budget if
implementation Assignment and
needed
Office-Based Tracking Sheets
• Refine efficiency
strategy if needed

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 35
Assignment sheet

Assignment Sheet

Date Supervisor Village No. of Team ID Team Remarks or

ID targeted composition comments
participants

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 36
 Tracking sheet
Tracking sheet

Name of supervisor

Date

Team ID

Area code
Supervisor District ID Village ID Facilitator Facilitation Facilitation If re- If refused, Remarks or
ID ID date status scheduled, provide comments
provide reason
date

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 37
Field Based Monitoring: Spot-Checks and Back-
Checks
Type: Description:
Spot Checks The RA joins in and observes as the
intervention is being implemented
in the field
Back-checks The RA returns to the study
participants and ask questions
to make sure that the
intervention(s) actually took
place according to the
procedure
J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S
Notes:
Set a target of study
participants to spot check, and
ensure to take notes on key
performance indicators and
offer feedback after the
intervention is completed
Must devise a brief
monitoring instrument to
identify whether the
subjects are correctly
receiving treatment and
briefly ask about what they
had received
38
Spot checks and back checks: what are we
checking for?

• Problems with the intervention

– Individual activity
– Overall structure of the activity

• Errors in implementation
– Adherence to organization and study protocols
– Opportunities for retraining

• Fraud
J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 39
Field-Based Monitoring: Spot-Checks

• Objective: To ensure that facilitators are interacting with study participants correctly
in accordance with the intervention manual
• When to do spot checks?
– Should be commenced straight after facilitation training
– Ideally within the first 2 weeks of implementation, because pressing issues usually
occur during this period (front loading is ideal)
• Need to be planned in advance:
– Check intervention materials and finalize spot-check schedule, starting from weak
facilitators (refer to facilitator training to identify facilitators who are lacking in
experience and potentially need improvement)/difficult study
locations/challenging study participants, etc. (can vary depends on project)
– Leverage assignment and tracking sheets to identify facilitators to be spot -
checked
• Ensure that the facilitator is comfortable
– Every facilitator is either spot-checked or back-checked
• Think about if and how observers need to be introduced to the respondent
– International staff members are potentially disruptive
• Fill in a spot –check form during implementation

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 40
Steps in conducting spot-checks

Join facilitator training, identify

facilitators who still need
Planning and preparation
improvement, pain points of the Finalize spot check schedule by
(understanding the interventions
intervention, ensure they understand taking the insights from facilitator
and key elements, preparing the
the intervention and their tasks, training as consideration
forms needed if necessary,etc)
socialize forms that they need to fill
in

Discuss with implementing partners

on how to best act when some
issues are found during spot-checks Conduct spot-checks, take detailed
(i.e. can we provide feedback Follow minimum must do (MMDs) for notes, provide regular field reports
directly to facilitators or need to be spot-checks, confidentiality to PIs, act to issues as soon as
through their supervisors? What is agreement signed, etc. possible to prevent mistakes being
the communication flow like?). This is repeated
to minimize problems and inefficient
communication

Complete a BTOR when spot-checks

are completed

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 41
 Spot-check sheet

Monitor Name

Date Unit ID Facilitator Type of check Format of check Time of check Overall Remarks
ID rating
Spot Shadow Full Partial Start Finish

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 42
Field-Based Monitoring: Back-Checks

• Objective: To ensure that facilitators are actually delivering the

intervention(s) according to the procedure outlined in ToR
• Back check could also be used to respond to findings from spot-
checks (e.g. facilitator doesn’t deliver the interventions well or at
all)
• Explain the purpose of revisits to participants
– Participants may be wary as to why they are being visited twice
– Ideally conducted within a week of original visit so participants can still
remember what happened, but if the intervention is targeted at
communities, can be done later
• Make sure that the implementation is completed and the field
team has left the area before conducting back-checks. If
interventions happen in multiple locations, alternate between
locations (no need to wait until every single field activity is
concluded)

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 43
Steps in conducting back-checks

Discuss with implementing partners on

Planning and preparation how to best act when some issues are
(understanding the interventions and found during back-checks (i.e. can we
Follow minimum must do (MMDs) for
key elements, preparing the forms provide feedback directly to facilitators
back-checks, confidentiality agreement
needed, formulate questions to ask, or need to be through their supervisors?
signed, etc.
have a list of participants and schedule What is the communication flow like?).
for visits ready) This is to minimize problems and
inefficient communication

Conduct back-checks, take detailed

notes, provide regular field reports to PIs, Complete a BTOR when back-checks
act to issues as soon as possible to are completed
prevent mistakes being repeated

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 44
Back-check sheet

Monitor Name

Date of visit Unit ID According to participants… Remarks

Was unit of Did facilitation Name of Date of
facilitation happen? facilitator who facilitation
found? came

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 45
 Intervention data collection

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 46
Collecting process data: Facilitation company’s
data

• Before implementation, work with partner organization staff involved in the

project to develop a plan about what tools you will use to collect important
data at each stage of the intervention and how that will be executed
– Weigh costs, time investments, data availability, and data accuracy
– Partner organization staff must be actively involved
– All parties should agree on a plan
– Communicate your seriousness for data quality
– Prepare forms and data entry program (coordinate with Data Manager)
– Decide and be clear on who is responsible for entering the data into the program,
then provide training

• In most studies, the partner organization will be asked to deliver a data report at
regularly scheduled intervals
– The variables to be included in these reports should be determined before the
study begins
– Reports should be created and tested early to prevent delays in data reports later

• Cultivating a good relationship with the systems staff at your partner

organization is very important

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 47
Collecting process data: Data collection forms

• Why are forms needed?

– Track take-up of an intervention
– Record feedback of study participants
– Document offers of treatment to the appropriate study sample

• Any forms related to experiments should undergo a pre-test

phase in which partner organization provides feedback
about their design and content and again revised if
necessary after testing in the field

• Forms should always be checked to confirm accuracy (both

errors or forgery)

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 48
Setting up your field
monitoring team
Sometimes, it is possible for you to hire
officers to monitor implementation

• Be clear on numbers (how many

people, how many days) and working
conditions (location, travel
requirements)
• Requirements for team members
– Knowledge of language and culture
(hire locally if possible)
– Attention to detail
– Good communicator
– References
• Monitoring team needs to be trained
on the intervention
materials/facilitation detail, etc.
• Feedback sessions and
accompaniments play an important
role in building the capacity of the
monitoring team
J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 49
 Reconciliation

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 50
General reconciliation strategy

• Review monitoring data throughout the implementation

process

• Always identify and reconcile issues as soon as possible

(ideally within one week of implementation)

• Conduct additional, targeted checks if necessary

• Use results for retraining and case studies—and to correct or

refine your instruments when possible

• Have a strategy for communicating monitoring findings to

the facilitation team
Dealing with problems encountered: reconciliation
in the field

Low take-up is a common problem in implementation

• Talk about it with your partner to come up with potential solutions

• Keep your PI informed of take-up rates regularly

Some problems can be dealt with right away

• Example 1: a certain field officer has been sharing too much information about the experiment with
study subjects
• Example 2: facilitation company has not included certain easily available data in its reports.

Sometimes a problem will be symptomatic of deeper issues with some aspects of the
implementation.
• Example: a loan officer in a certain area has not been implementing the intervention according to study
protocol
• It may mean that you should be extra careful about checking other loan officers’ work
• It could also mean that there are problems with partner organization buy-in to the study as a whole
• It could even mean that some aspect of the implementation as it is currently designed is impractical or
infeasible
• Solution: increase monitoring as needed to determine both the true cause of the trouble and its extent,
in order to be confident that the steps you have taken have really solved the problem.

J - P AL S E A | W O R K I N G WI TH F AC I L I T AT I O N C O M P AN I E S 52
Terima Kasih
www.povertyactionlab.org/southeast-asia
Abdul Latif Jameel Poverty Action Lab (J-PAL)
@JPAL_SEA