1

TERMES OF REFERENCE
DATA COLLECTION FOR THE ELIIP IMPACT EVALUATIONS

1. BACKGROUND

Egypt’s labor market has been sluggish since the 2011 revolution and the economic downturns
that ensued in the revolution’s aftermath. Unemployment rates have increased from 9.6
percent before the revolution to 11.9 percent in September 2011, while poverty rates increased
from 21 percent in 2009 to 25 during the same period. Unemployment rates were further
pushed upwards by the return of migrant workers from neighboring countries that experienced
political distress. Workers with relatively low skills from rural areas appear to be those who
experienced the strongest adverse effects of these economic challenges. As the result, more
Egyptians have fallen into poverty and face chronic food insecurity. According to a World Food
Program (WFP) study, between 2009 and 2011, 15.2 percent of the population moved into
poverty—twice the number that moved out of poverty (7.7 percent)—while another 12.6
percent of the population remained in chronic poverty.

It is against this backdrop that the Government of Egypt is implementing, through Social Fund
for Development (SFD) the Emergency Labor-Intensive Investment Project (ELIIP) financed by
the European Union. Both projects are part of a cash for work program that provides social
safety net to millions of beneficiaries. The program aim is "to contribute to the reduction of the
negative impact of crisis that may lead to food insecurity and unemployment of the poor and
vulnerable in selected areas, and support the protection and building of community assets in
poor communities." It does so by providing short-term employment opportunities for
unemployed unskilled and semi-skilled workers by supporting locally generated subprojects
such as community level infrastructure construction or upgrading or community services to be
proposed by the local government and non-government organizations (NGOs).

Given the strategic importance of ELIIP, the Development Impact Evaluation Unit of the World
Bank’s Research Group is partnering with SFD to conduct an impact evaluation study, which will
provide scientific evidence of the effects of the cash for work activities and infrastructure
created on the socioeconomic welfare of beneficiary households and communities. This study
will present robust evidence to improve policy decisions pertaining to cash for work programs
in Egypt and beyond.

The first step in achieving this impact evaluation is the collection of high quality quantitative
data areas targeted by the program. In view of this, the World Bank is recruiting a consulting

1
 2

professional firm to prepare and conduct data collection, under the technical supervision of an
evaluation team consisting of specialists from the World Bank and partner universities.

2. SCOPE OF WORK

The proposed survey work will focus on roughly 823 villages/communities targeted by the
impact evaluations across 13 governorates. In each, a survey will be conducted with three
respondents: (i) the head of village/community; and (ii) 2 additional community leaders
randomly selected from a set of pre-identified community leaders. (In other words, we expect a
total sample of approximately 2,400 respondents.) The survey will collect information on the
characteristics of the village/community (e.g., the number and quality of socio-economic
infrastructures, the main agricultural products, etc), the Community institutions and socio-
cultural activities that could interact with program activities. The final sample size as well as the
strategy for selecting community leaders to be surveyed will be finalized during negotiations.
Table 1 below shows the distribution of the sample by governorate.

Tableau 1: Sample composition by governorate

Number of Number of surveys to

Governorate villages/communities be carried out
Aswan 70 210
Beheira 153 459
Beni Suief 8 24
Dakahlia 23 69
Giza 100 300
Luxor 54 162
Matroh 5 15
Meniya 40 120
Menoufia 25 75
Qaliobia 146 438
Qena 31 93
Sharkia 29 87
Sohag 139 417

Total 823 2,469

2
 3

3. TASKS AND RESPONSABILITIES

The consulting firm selected will be responsible for collecting and processing data in all targeted
communities and will work under the direct supervision of the evaluation team that will be
responsible for data analysis and drafting of analytical reports. The surveys will be implemented
using electronic tablets enabling the field teams to collect and transmit data from the field to a
cloud-based server. This method allows for real time monitoring of data quality, eliminates the
need to print paper survey forms, reduces enumerator error and eliminates the need for data
entry. The survey firm will be responsible to provide appropriate software and hardware, training
and supervision to guarantee high-quality data recording. Tables purchased using World Bank
funds will be the property of the World Bank. The main tasks and responsibilities of the Survey
Firm are outlined below.

3.1. Survey Preparation

3.1.1. Acquisition of clearance and insurance to conduct the survey.

The Survey Firm is responsible for acquiring all clearances and permissions necessary for
conducting the survey. In particular, the firm will be responsible to submit the questionnaires
and manuals to obtain ethical clearance in the country. The Survey Firm is also responsible for
adhering to local formalities and obtaining any required permits related to survey
implementation.

3.1.2. Adaptation and pre-testing of questionnaire. The Survey Firm will be provided draft
survey instruments in French. In consultation with the evaluation team, the Survey Firm will
pre-test and adapt the questionnaire to the local context. This will include adapting the
phrasing of questions and adaptation of response codes so they are appropriate to the local
context. Before testing the questionnaire instruments in the field, the survey firm will also
ensure survey questionnaires are successfully programed on tablets. The consulting firm will be
responsible for the translation (and back translation) of the questionnaires into local languages
beforehand. Therefore the survey firm should take care to identify in advance the most
common local dialects in the survey areas in which the questionnaires should be translated. The
field tests therefore should also be used to test the quality of the translation.

The pre-test phase will last at least 7 days and will take place in program villages out of the study
sample. Final modalities for the pre-test will be defined in consultation between the survey firm
and the evaluation team.

Based on the pre-test results, the survey firm will finalize the questionnaire. The firm will also
produce a manual for field staff for each questionnaire. The field manual will contain detailed
information on how to conduct an interview based on the questionnaire and should discuss
special cases that may arise and how to deal with them. The deliverable for this step will include
a pre-test report, finalized household and child survey questionnaires, and finalized child and
household survey manuals.

3
 4

3.1.3. Field Procedure Plan and interviewer training curriculum

The field procedure plan should outline in details all aspects of the field work to be conducted by
the Survey Firm. The field procedure plan should discuss the following:

 Composition of teams, expected profiles, tasks and responsibilities of each member of the
team,
 Guidelines and protocols for sampling and survey data collection
 Quality control protocol
 Tracking protocols to identify and interview households and village leaders that may have
been absent during the first visit, or may have relocated.
 Outline of progress report to be shared with the impact evaluation team on a weekly basis
 Detailed calendar of activities/workplan
 Travel logistics
 Anticipated challenges (e.g., connectivity; power outages) and contingency plans
 A training curriculum outlining how the training will be organized and what its objectives will
be, as well as how the pilot in real field conditions will be undertaken.
 Composition of field teams, expected profiles, tasks and responsibilities of each member of
the team, including:

o Composition of coordination team (suggested as follows)

 1 Senior Project Manager/Coordinator
 1 Senior Specialist in IT/ programming and use of tablets

o Composition of field survey teams (suggested as follows)

 4 Supervisors
 24 enumerators
 4 Specialists in programming use of tablets/quality control

The impact evaluation team will provide inputs to the field procedure plan. The draft Field
Procedure Plan should be presented to the Evaluation Team for comments and clearance, and
revised as necessary. The Survey Firm must then implement the survey adhering to the field plan.
If field conditions dictate significant changes to these plans, the Survey Firm is required to inform
the Evaluation Team, in the form of a written report or progress report.

3.1.4. Recruitment and training of field teams

As outlined above, the coordination team will include an adequate qualified enumerators,
supervisors, IT specialists and back-checkers. The survey firm will send the CVs of key personnel
preselected for coordination/supervisory and ICT/programming positions one week before the
start of training. In addition, the survey firm will pre-select potential field staff to participate in
the training. The survey firm will be responsible to pre-select 25-50 percent more than the
number of field staff required for the training.

4
 5

In coordination with the evaluation team, the survey firm will provide training to field teams for
a minimum of 10 days. The date of the training will be agreed with the evaluation team. The
training program should include theoretical training (on questionnaires and field protocols),
classroom practice, small group exercises as well as field exercises and human subjects. After
training, the enumerators will go on the field for a field test that will last at 2-3 days, during
which each interviewer administered at least 10 complete questionnaires. Testing and training
should be used primarily to simulate the administration of the questionnaire in real
circumstances. The pilot phase will also evaluate the organization of the investigation team,
logistics and global strategies for data collection in place. To the extent possible, training of
supervisors and enumerators should occur simultaneously in a central location to ensure the
evaluation team will be able to provide the training. The survey firm will be responsible for
logistics and transportations of prospective candidates.

Following the training, interviewers and supervisors should go through a written evaluation and
selected based on their performance at the test and during the pilot. In its pilot report, the survey
firm will include a full listing of training participants and a full ranking of their performance,
identify applicants selected to proceed with the survey, applicants on a waiting list, and non-
selected applicants. The impact evaluation team will clear this report before the field staff is
formally hired. Wherever possible, the survey firm should put in place a balanced team in terms
of gender and local languages.

3.2. Data collection

3.2.1. Sampling strategy

The set of villages/communities in which data collection will be carried out have been pre-
determined. As noted, in each village/community, 3 respondents will be selected for the
interview. One of the respondent will be the village/community chief, whereas the other two
respondents will be randomly selected from a list of pre-identified community leaders. The
evaluation team will provide specific guidance about the identification and selection of these
community leaders.

3.2.2. Launch of survey data collection and re-training

Once the training has been completed, data collection will start. It is expected that field teams
will be grouped in nearby location for the first week of data collection, to allow thorough quality
control and monitoring. At the end of the first week, a one-day re-training session will be
organized to take stock of difficulties and give consistent instructions to all field team.

After the re-training, and conditional on quality being high, field teams will then be deployed
independently. It is expected that field teams will be concentrated in one commune at a time,
hence finalizing a town/village fully before moving to the next commune. The evaluation team
has an established the order in which data collection will be rolled out to different towns/villages.
The Survey firm should coordinate closely with the evaluation team on the work plan to ensure
that the ordering is respected.

5
 6

3.2.2. Successful rollout of the data collection

During data collection, the survey firm will be responsible to:
 Provide the field staff with the necessary equipment and materials for data collection (e.g.,
tablets, enumerator manual, field protocols, identification cards, etc.).
 Ensure safe transportation of field staff for all field activities as well as their safety and
security in the field.
 Manage the field staff team to collect the data based on the specific requirements described
in these terms of reference and in coordination with the impact evaluation team.
 Ensure implementation of data collection plan and quality control protocols.
 Ensure that only the correct individuals are interviewed.
 Ensure that the questionnaire (and digitized data) records respondents’ refusal to answer any
questions during the interview.
 Ensure that all enumerators perform consistency checks of the questionnaire to ensure that
questionnaires are fully consistent and complete before they leave the respondent’s
household.
 Ensure the security of the data and data collection equipment.

The survey firm will make every effort to contact each household and respondent selected and
complete all interviews. The World Bank expects a minimum response rate of 95 percent.

3.2.3. Survey supervision and weekly progress reports

The survey firm will be responsible to undertake internal quality control procedures and engage
in regular supervision activities to ensure all questionnaires are comprehensive, consistent and
accurate. This will include:
 An electronic check of all the tablets by local supervisors at the end of each work day.
Questionnaires with missing or inconsistent responses will need to be completed in the
household.
 Random quality control by supervisors and sspecialist in programming use of
tablets/quality control during and after the interviews to ensure proper procedures and
protocols are being strictly followed.
 Random quality control by supervisors by re-visiting households and verifying accuracy
of a random sub-sample of questions (e.g. for 10% of questionnaires).
 Random quality control by quality controllers and the DIME team during the interviews
to ensure the interviews are implementing according to established procedures.
 Random quality control by quality controllers by re-visiting households and verifying
accuracy of a random sub-sample of questions (e.g. for 10% of questionnaires).

The survey firm will send a weekly progress report to the impact evaluation team. The progress
report will include the tracking database, a short assessment of the overall progress of fieldwork,
an update on data quality and main quality issues encountered in the field, and name any
relevant deviations from the standard field plan. The survey team will be expected to work in
close coordination with the impact evaluation team, which will also supervise survey
implementation in the field. In particular, the survey firm will:

6
 7

 Provide all necessary documentation to allow the impact evaluation team to verify all
information collected. In particular, the evaluation team should be given access to all
fieldwork operations, and be allowed to perform random checks at all stages of data
collection and data processing.
 Guarantee close collaboration with the impact evaluation team throughout the different
stages of the work.

If field conditions impose changes, the survey firm will be required to send a written notification
to the World Bank immediately and a contingency plan of how the challenges will be addressed
in within a reasonable time.

3.3. Data processing and management

3.3.1. Review and validation of an interview

As noted, surveys will be implemented using electronic tablets enabling the field teams to collect
and transmit data from the field to a cloud-based server via the mobile phone network. This
method allows for real time monitoring of data quality, eliminates the need to print paper survey
forms, reduces enumerator error and eliminates the need for data entry. The survey firm will be
responsible to provide appropriate software and hardware, training and supervision to guarantee
high-quality data recording. To ensure accuracy, the survey firm should ensure:

 Every enumerator has a fully operational table and each field team has at least two
reserve tablets and chargers.
 Every tablet is fully charged and loaded before beginning the work;
 Every supervisor monitors the survey by checking the GPS location of where and when
the interview took place; and
 Every supervisor review and validates interviews at the end of each day before uploading
the data to the central server.

3.3.2. Compile a complete database of collected data.

The survey firm will be responsible for delivering a final dataset, final report and final monitoring
dataset. The data is to be properly organized, with variables named and labeled and appropriate
identifiers that permit seamless merging between databases.

Note: In case of emergency such as tablet malfunctioning, supervisors may authorize the use of
paper survey. In such case, however, collected data should be transfer to the tablet as soon as
feasible and keep the paper surveys for record.

7
 8

4. DELIVERABLES AND PAYMENT SCHEDULE

4.1. deliverables

Preparation

 Pre-test report, final questionnaires and manuals approved by the evaluation team.
 Ethical clearance and a written statement ensuring that all permissions and insurance to
conduct the survey have been acquired.
 A written Field Procedure Plan and interviewer training curricula approved by the
Evaluation Team.
 Report summarizing results of the training of enumerators s and the test/pilot phase /
for household and community questionnaires (final versions of the questionnaires and
field manuals approved by the evaluation team.)
 Delivery of the sampling frame (lists of households in each targeted town/village) by
province and household selection protocols.
 Programing of questionnaires on the tables, using a software/program approved by the
evaluation team.
 Delivery of training for all field team and pilot test report

Data collection, tracking and progress reporting

 Written weekly data collection progress reports during the data collection period
(including i) detailed listing of household surveys completed, partially completed and
pending, ii) detailed, and iii) the tracking dataset.
 Database tracking on a weekly basis.
 Fieldwork report. The survey firm should produce a comprehensive report of field
activities that documents: (i) the timeline and progress; (ii) major incidents that may
affect the quality of the data or performance issues, and lead to the modification of the
data training manual; and (iii) changes to the questionnaire or data collection during the
field work and the implications on the data.
 Final datasets for household, community/chief and infrastructure surveys. The data is to
be properly organized, with variables named and labeled and appropriate identifiers that
permit seamless merging between databases.
 Final data collection report.

4.2. Schedule of deliverables

The table below provides a listing of key deliverables, along with corresponding deadlines.

8
 9

Deliverables Deadline

1. Obtain the administrative authorizations and clearance One week after

human subjects review board (or equivalent) signature of the
contract

2. Recruitment of key project staff a. One week after

a. Assistant coordination/supervisors signature of the
b. Field staff (enumerators, etc.) contract
b. Week 1-2

3. Administrative and logistical planning:

a. Purchase of necessary equipment a. Week 1 – 2
b. Establishment of logistics for training and data b. Week 1 – 2
collection c. Week 2
c. Delivery of provisional sampling frame for targeted
areas and sampling strategy.
4. Adaption/finalization of survey instruments, including
translation/back into local languages and back a. Week 1 – 2
translation: b. Week 2
a. Programming of questionnaires to tablets c. Week 2
b. Pre-testing of survey instruments d. Week 2-3
c. Training of supervisors and first piloting e. Week 3
d. Report of pre-testing, detailing how the tests, the
results, changes in the questionnaire, problems After approval of the
encountered and lessons for the survey finalized survey instruments
implementation by the evaluation team
e. Revise & finalize survey instruments
5. Refinement of field manuals:
a. Refinment of field manuals and other data collection
protocols a. Week 2 - 3
6. Training:
a. Training of enumerators a. Week 3-4
b. Field piloting of the survey b. Week 3-4

7. Detailed workplan for data collection rollout, including

deployment schedule of field teams a. Week 4
a. Development of quality control protocols to plan,
monitor and manage data collection b. Week 4
b. Deployment of field teams

9
 10

8. Data collection:
a. Community sensitization
b. Household listing a. Week 2 – 4
c. Rollout of surveys
d. Delivery of all documentation and equipment related b. Week 4 – 5
to data collection, including (at least) tablets, field
manuals and training materials. c. Week 5 – 10

d. Week 11-12
9. Reports:
a. Field report on the first week of data collection (for a. Week 5-6
more precise adjustment of the implementation
process / procedures)
b. Weekly report on progress of fieldwork b. Every week (5-10)
c. Mid-term Field Report midterm detailing the area
covered, problems and solutions, including the basic c. Week (8)
statistical data entered
d. Final field report, including descriptive statistics of d. Week 11-12
key variables (to be defined with the evaluation
team).

Note: When the implementation is hindered by unexpected difficulties, the survey firm should
submit a written report to the World Bank informing about the cause of the delay, the expected
duration of the delay and proposed options to solve the problem, ensuring that data collection
will continue with a minimum of delay. This report should reach the World Bank not later than
one week after the identification of each problem.

4.2 Payment schedule

The contract period is scheduled from April to May/June 2016. The survey firm will be paid based
on submission of deliverables in the agreed in the timeframe. All data related to this contract and
equipment will remain the property of the World Bank. The use of such data by third parties for
research purposes is not authorized unless written authorization is expressly provided by the
World Bank.

Output/stage Contract amount (%)

Signing of Contract (this is considered a front-payment for data 10%
collection activities)
Report from training and pilot as well as well as delivery of work 40%
plan and deployment schedule
Delivery of complete data for 85% of the sample 35%
Delivery of complete data for 100% of the sample, final report 15%
and World Bank equipment

10
 11

5. REQUIRED QUALIFICATIONS

The selected firm/consortium shall possess the following qualifications:

 A legal status recognized by the Government of Egypt, enabling the organization to
perform the above-mentioned tasks.
 Demonstrated experience in conducting surveys of a scale and complexity similar to
those envisaged by the ToR for the past five years.
 Experience in surveys of households Middle East, Egypt and better if proven experience
in data collection with a low rate of non-response.
 Experience in household surveys in MNA countries, better if in Egypt.
 Strong capacity and experience in planning and organizing survey logistics.
 Good network of enumerators with strong experience in electronic data collection.
 Strong capacity in data management and statistics.
 Strong capacity in electronic data collection programs such as survey CTO, ODK, etc.
 Proof of sufficient funds to cover at least 50% of the budget upfront.

CVs for the core staff are expected to be included in the firm technical proposal.

6. SUPERVISION

The World Bank will be responsible for the overall technical oversight of the work to be carried
out under these TOR. Moreover, the World Bank team will have the following responsibilities:

 Provide to the firm a draft of the quantitative survey instrument and tests for pre-testing.
 Finalize the survey instrument, in collaboration with the firm.
 Assist in training of potential field staff, including supervisors and enumerators.
 Provide to the firm the list of list of towns/villages targeted by the survey.
 Oversee the field work and provide recommendations to ensure that all data collected
meet high quality standard.
 Review all work products generated by the survey firm and provide timely feedback.
 Supervise the overall quality of the information collected.

Appendices:

 Draft survey instruments (village/community-level questionnaire).

11