An Evaluation of Juego y Aprendo

An innovative model to provide preprimary education in rural Honduras

Javier Luque, PhD12

Interamerican Development Bank

1
2

The views expressed in this document do not necessarily represent those of the Interamerican Development Bank or its Board of Governors.
Email: jluque@iadb.org

Introduction
Preprimary education is at the center of education policy debate in most countries around the
world. Important numbers of students, mostly for disadvantaged backgrounds, do not reach
schools ready to engage successfully on first grade learning activities, increasing the risk of
grade repetition, leading eventually to low academic achievement, and low productivity in labor
markets and vicious circles of poverty. A growing number of studies is showing that attending
pre-primary education prepare students to school, decreasing repetition rates and increasing
academic achievement; and that their benefits can be traced far beyond, as students that attended
preprimary education are more likely to attend tertiary education and less likely to engage in
risky behaviors3.
The exact mechanisms for the impact of pre-primary education are likely to be different for kids
under different contexts. Kids in rural areas tend to be exposed to very little cognitive
stimulation, and, for example, they master a very limited vocabulary in comparison to kids from
urban settings, not allowing them to construct higher order thinking. On the other side, kids from
marginal urban areas tend to live in violence-prone households. Neuroscience has clearly
stablished the negative impact of been exposed to violence and the capacities of the brain to
engage in learning activities. Pre-primary education has a great role promoting cognitive
stimulation and nurturing environments for kids to develop.
Given its importance, countries have been expanding aggressively the coverage of pre-primary
education. Nonetheless, this expansion has faced important challenges, as the number of students
to cover is large and the resources, both pedagogical and economic, are scarce. Countries have
invested resources to create learning standards for pre-primary education, though, in many cases;
there are not clear strategies to teach those learning standards at pre-primary schools.
Honduras is looking to increase the attendance to pre-primary education. The Secretary of
Education has, under the mandate of the Ley Fundamental de Educacin, to achieve universal
coverage for pre-primary education for all the five years old. To do so, besides the development
of the right learning standards, the Secretary has to create or expand in all cities and rural
communities in Honduras pre-primary schools. Interventions in rural areas are particularly
challenges given the level of dispersion of the populations and the difficulties to find teachers to
work on those communities.
In order to overcome the difficulties of providing pre-primary education in rural areas, the
government of Honduras in collaboration with the Fundacin Ferema, deployed the Juego y
Aprendo model in rural communities through community centers for pre-basic education
(CCEPREBs). Juego y Aprendo is a scripted model that can be deployed through community
volunteers. Community volunteers are not required to have a degree on pre-school education, but

See Vegas (2012) for a discussion on the impact of pre-primary education on student outcomes.

to participate on regular training sessions. The IADB and the World Bank, among other donors,
financed the program.
In order to assess the impact of the Juego y Aprendo model, an evaluation was proposed. The
design has to deal with the challenging characteristics of rural areas. The main challenge was to
identify communities with 10 kids between five and six year old, the minimum number of kids to
stablish a CCEPREB. In many cases, the walking distance between the CCEPREBs and the kids
household was too long for kids to walk on a daily basis, reducing the potential number of kids.
A large number of communities willing to participate would have allowed a sequential
implementation of the model, with a potential randomization. As an experimental design for the
evaluation was not possible, the team selected a quasi-experimental design, taking in advantage
the great discontinuity in school enrollment patterns that is introduced by the requirement to
attend primary school that kids have to be six year old. Due to this requirement, kids that at the
beginning of the year were 5 years 364 days old will not be able to attend first grade, but would
have to attend pre-primary. On the other side, kids that were 5 years old can also attend preprimary, and those kids will be 5 years 364 days old at the end of the school year. The evaluation
strategy will compare the kids with 5 years 364 days old at the end of the school year
(treatment), with kids that were 5 years 364 days old at the beginning of the school year
(control). The underlying assumption is that a kid, in a context without interventions, that is 5
years 364 days old in January has the same level of achievement of a kid with 5 years 364 old in
December. In the intervened rural communities there were no other interventions that could
invalidate the assumption.
In order to assess child development, the team employed the PRIDI method developed by the
Interamerican Development Bank. The PRIDI assess child development in nine domains of early
child development: cognitive aspects, shapes and forms, use of classroom material, memory,
time management, emotional expression, knowledge of colors, fine and gross motor skills.
The evaluation shows that kids that attended the program increase their learning achievement by
two years in comparison to the age progress of the control group, and students are about ready to
engage in first grade by ministry of education standards.
Honduras challenges to close enrollment gaps in Pre-K
The Government of Honduras has education as one of its key priorities. Despite important
progress in attendance at the primary school level, there are large challenges in attendance preschool and secondary (both lower and upper). The current government administration has as an
objective the achievement of universal pre-primary education for all the 5 years old. The current
enrollment rate for that age group is 70 per cent. The current enrollment for 3 and 4 years old is
low (5 and 20 percent, respectively).

Figure 1
Enrollment in Pre-School
100%
90%
80%
70%
60%
50%

Si

40%

No

30%
20%
10%
0%
3

Age

Source of data: National Households Survey (2012);.

In order to close enrollment gaps, the government needs a strategy to promote the demand and
increase the supply of pre-school services. Demand for education is related to the educational
achievement of the households, households with lower socioeconomic status do not value
education in general, and pre-school in particular. On the supply side, to stablish pre-schools in
rural areas is difficult given the high levels of dispersion of the population. Primary education
has been expanded through the multi-grade model (teachers with students from several grades in
the same classroom), and it is not replicable for the expansion of pre-school.
The government has chosen to have different strategies depending on the education group that
can be stablished. In areas that groups of 15 or more students are achievable, the government will
establish a pre-school with a formal teacher. In areas that can achieve a model larger than 8 but
smaller than 14 students, the community model will be implemented. In other areas, house visits
will be implemented.
The community model is based on the Juego y Aprendo model developed by the Fundacin
Ferema. Rural communities are visited by authorities of the sector, and asked to join the
program. If they do, the community has to provide a place for the pre-school to work, and
provide a local trainer. The local trainers, later on, participate on courses to familiarize them with
the Juego y Aprendo methodology. The community participation creates a local demand for preschool, and helps to create the supply.

Assessing the impact of Juego y Aprendo

Objectives
Given its importance in the overall strategy to expand pre-primary education, it was important to
assess whether the Juego y Aprendo model was helping to "improve the learning of basic
education schools that serve the poorest people in the country" was achieved. In particular, it was
important to assess whether the intervention was "Expanding access of disadvantaged children to
pre-primary level, to improve the conditions of schooling by attending the first cycle of basic
education." To do so, we proposed an strategy to evaluate the impacts on child development of
students who benefited from the provision of pre-school through community model
(CCEPREBs), and specifically, we measure the progress of students in each of the dimensions
included in the instrument of the Regional Program for Early Childhood Development Indicators
(PRIDI). Through a wide range of instruments, PRIDI evaluates nine domains of early child
development: cognitive, shapes and forms, use of classroom material, memory, time
management, emotional expression, knowledge of colors, fine and gross motor skills.

Figure 1. Treatment and control groups design of a quasi-experimental impact evaluation

2012

January 1st 2009

December 31st
2008

2015

PREPRIM

1ST GRADE

PREPRIM

1ST GRADE

2ND GRADE

PREPRIM

1ST GRADE

2ND GRADE

1ST GRADE

2ND GRADE

3RD GRADE

BIRTH DATE

2011

SCHOOL YEAR
2013
2014

January 1st 2008

December 31st
2007

NO PREPRIM

THE STUDENT HAS 5 YEARS AND

364 DAYS AT THE BEGINNING OF
SCHOOL YEAR 2013

THE STUDENT HAS 5 YEARS AND 364

DAYS AT THE END OF SCHOOL YEAR
2013

It is important to notice that the evaluation did not focus on the rate of repetition of first grade, as
from 2014, promotion is automatic in Honduras between the first and second grade of primary
education, affecting the cohort included in the evaluation (students who attended the CCEPREBs
in 2013 and first grade in 2014). A previous paper, by Aguilar and Rapalo (2008) evaluated the
CCEPREBs with that indicator, showing an important reduction in the grade retention.

Methodology and implementation.

While originally the possibility of an experimental evaluation was explored, it was not
considered given the difficulty to find rural communities that met the minimum qualifications
required for enrollment standards by the ministry of education (5 years 8 students per
classroom). Consequently, the evaluation team coordinated with the Ministry of Education
sought alternative impact assessment of the intervention.
According to the laws in force in Honduras in 2013, to engage in pre-school the prerequisite was
to have been born in 2007. Given the distribution of births during the year to January 1, 2013
enrolled students would be in preschool aged 5 years 0 days and 5 years, 364 days. At December
31, 2013, the same students would have between five years, 364 days and 6 years, 363 days old.
In the absence of external intervention, students in five years, 364 days old at the beginning of
2013 and those of six years of age at the end of 2013 would have a similar level of development.
However, the first group would have benefited from participating in pre-school, while the second
did not (see Figure 1). Given that the date of birth is a random variable, we consider that these
two groups it possible to analyze the impact of assistance to CCEPREBS a quasi-experimental.
Of the 2,124 CCEPREBS with support from the IADB, a sample of 200 centers was randomly
selected. The evaluation sought to interview 5 students per CCEPREB, which were also
randomly selected by field staff. A baseline and follow up indicators of child development were
performed using PRIDI. At baseline were interviewed 980 students in 201 CCEPREBS between
late April and early May 2013. At follow-up interviews were conducted with 1,034 students in
203 CCEPREBS between late October and early November. To avoid bias in the data collection,
the same villages were visited. Data collection and construction of the sample was carried out by
ESA Consultores, a company with extensive experience of fieldwork in Honduras. Their
personnel st had previous experience in the methodology PRIDI given an earlier application in
Nicaragua. Table 1presents a summary of the information collected:

Table 1: Demographic variables and indicators of child development (according to PRIDI) on the
baseline and follow up collection.

Baseline

Follow up

Centers

201

203

Studentes

989
Average

1.034
Standard
deviation.

Average

Standard
deviation.

Sex

52,8

51,9

Age

5,14

0,56

5,63

0,57

Cognitive aspects

3,70

3,24

6,06

3,30

Forms and shapes

2,27

1,67

3,32

1,75

Use of clasroom materials

2,77

1,19

3,14

1,10

Memory

3,22

1,09

3,49

0,95

Time management

2,52

1,71

3,43

1,87

Express emotions

4,37

1,88

5,39

1,76

Knowledge of colors

1,84

1,23

2,39

1,02

Gross motor skills

4,08

1,11

4,46

0,89

Fine motor skills

4,90

1,89

5,65

1,82

Data source: Baseline and follow up data collection.

The information collected allows us to compare the results of students before and after the
intervention. However, to be an important trend in the process of acquiring skills in children gets
older (see Figure 2), simple differences before and after over-estimate the results of the
intervention. To avoid the tendency to affect the results, the analysis focuses on the results of
children who were between 5 1/2 and 6 years old when the baseline (pre-program), compared to
children who had 5 1/2 and 6 years at the time of collection of monitoring information (postprogram).

INDICE DESARROLLO AGREGADO NORMALIZADO

-2
-1
0
1
2

Figure 1: Index of Child Development Based on the PRIDI scale

HABILIDADES DE LOS NINOS - ESCALA ENGEL PRIDI
LINEA DE BASE

5
EDAD

Data source: Data source: Baseline and follow up data collection.

Figure 3 shows the age structure of students enrolled in CCEPREBS sampled. It is noted that
children with four years serving the CCEPREBS, contrary to the directives of the Secretariat.

They observe students aged between 5 1/2 and 6 years in both the baseline and follow-up.
Additionally, students are seen with over 6 years follow-up. The group between 5 and 6 years
interviewed in the baseline shall control for purposes of this evaluation group, while the group
between 5 and 6 years interviewed in the follow define the treatment group. Members of the
treatment and control groups will define a restricted sample. 980 of students who belong to the
baseline, 328 belong to the restricted sample, and monitoring 1,034 belonging, 350 belong to the
restricted sample.
Furthermore, the existence of other interventions that were affecting the level of development of
children could skew the results of the evaluation, particularly among the collection of baseline
and monitoring. However, during the execution of the program it was verified that there were no
other interventions in the areas of influence.

Figure 2: Age structure kids in the baseline and follow up

90
80
70
60
50
40
30
20
10
0
0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11
4 aos

5 aos

LINEA DE BASE

6 aos

SEGUIMIENTO

Data source: Baseline and follow up data collection.

A quick look on the baseline, were kids in rural Honduras ready for first grade?

The baseline data collection identified very big developmental challenges of pre-school bound
kids. And in some cases, these kids are almost school bound kids. Figure 4 and Figure 5 show
the developmental challenges among kids with the PRIDI instrument. Figure 4 identifies

challenges on time management. Note that only 35 per cent of students on their path to school do
not master expressions in the past and present tense. Figure 5 presents the capacity of kids to
identify colors. A surprising 30 percent of kids do not identify the color red, and 25 of kids do
not identify the color blue.
Figure 4
Time management
(By age at baseline)

Figure 5
Color identification
(By age at baseline)
0.8

0.7
0.65
0.6
0.55
0.5
0.45
0.4
0.35
0.3
0.25
0.2

0.75
0.7
0.65
0.6
0.55
0.5
5 A I TRIM 5 A II TRIM 5 A III TRIM 5 A IV TRIM

5 A I TRIM 5 A II TRIM 5 A III TRIM 5 A IV TRIM

Pasado

Presente

Futuro

Azul

Data source: Baseline data collection.

Rojo

Data source: Baseline data collection.

Evaluation results
Table 2 shows the results in each of the domains evaluated by PRIDI. The table shows four sets
of results. The first two, called "total sample" compared to all students present in the evaluation,
regardless of age. One excluding the aging effect and another that includes it: To control for the
effect of age on child development, two models are estimated. The analysis of the results of the
assessment focuses on the restricted sample (defined in paragraph 8), which includes only
students between 5 1/2 and 6 years.
Table 2: Results of the evaluation of CCEPREBS under the PRIDI1/
TOTAL SAMPLE
A 2/

RESTRICTED SAMPLE 2/

B 2/

A 2/

B 2/

COGNITIVE ASPECTS

0,72

***

0,44

***

0,46

***

0,40

***

FORMS AND SHAPES

0,62

***

0,47

***

0,51

***

0,50

***

0,31

***

0,19

***

0,26

***

0,24

***

0,24

***

0,06

TIME MANAGEMENT

0,53

***

0,37

***

0,42

***

0,41

***

EXPRESS EMOTIONS

0,55

***

0,40

***

0,37

***

0,36

***

COLORS KNOWLEDGE

0,44

***

0,25

***

0,19

***

0,17

***

GROSS MOTOR SKILLS

0,35

***

0,20

***

0,35

***

0,33

***

FINE MOTOR SKILLS

0,39

***

0,12

**

0,22

***

0,19

**

USE OF CLASSROOM
MATERIALS
MEMORIZATION

0,01

-0,02

Note:
1/ Dependent variable index on PRIDI domain. Standard errors clustered at the school level.
Siginficance levels as follow:
*** P<=1%; ** P<=5%; *
P<=10%.
2/ Models:
A: Dependent variable standardized measure of the test result in the module; independent variable
treatment..
B: Dependent variable standardized measure of the test result in the module; independent variable
treatment and kids age.
3/ Student between 5 a 6 years old at the time of data collection.
Data source: Baseline and follow-updata collection.

Figure 4 shows the results for the various dimensions calculated by the PRIDI, but to facilitate
comparison, the progress expected during a year of life is included. It is noted that the
intervention gains are similar to the progress achieved by students in a year of life in 6 domains
(cognitive, shapes and forms, time management, expressing emotions, coarse engine, fine
motor), which means a year the children moved the equivalent of nearly two years (1.83 years)
than they would have proceeded without assistance to preschool.

Figura 3: Impact of the Pre-School impact evaluation on PRIDIs early childhood

indicators.

Fine motor
skills
Gross motor
skills

Cognitive
aspects
0.60
0.50
0.40
0.30
0.20
0.10
0.00

Colors

Forms and
Shapes
Use of
classroom
material
Memorizing

Express
emotions

Time
management

Data source: Baseline and follow-up data collection.

due to intervention
one year of life

Final recommendation and future actions

The evaluation has shown the positive impact of the CCEPREB model to improve student outcomes and
increase readiness of student towards school. This positive evaluation and other assessments have been a
key element for the expansion of the model, as an cost effective intervention. Current work is focused on
assessing whether there are long lasting effects of the program, and whether the program outcomes
compare to those of other possible interventions.

Abadzi, H. (2006). Efficient Learning for the Poor: Insights from the frontier of Cognitive Neuroscience.
Washington DC: Banco Mundial.
Alderman, H., Behrman, J. R., Lavy, V., y Menon, R. (1997). Child Nutrition, Child Health, and School
Enrollment: A Longitudinal Analysis. Policy Research Working Paper
Berlinski, S., Galiani, S., y Gertler, G. (2006). The Effect of Pre-Primary Education on Primary School
Performance. Working Paper (04/30)
Berlinski, S., Galiani, S., y Manacorda, M. (2007). Giving Children a Better Start: Preschool Attendance
and School-Age Profiles. Londres: University College.
Carneiro, P., y Heckman, J. (2003). Human Capital Policy. In J. Heckman, y A. Krueger, Inequality in
America: What role for Human Capital Policies? Cambridge, Massachussetts: MIT Press.
Cunha, F., Heckman, J. J., Lochner, L., y Masterov, D. V. (2006). Interpreting the Evidence on Life Cycle
Skill Formation. In E. Hanushek, y F. Welch, Handbook of the Economics of Education (pp. 697-812).
Amsterdam: North Holland.
Schady, N. (2012). El desarrollo infantil temprano en Amrica Latina y el Caribe: acceso, resultados y
evidencia longitudinal de Ecuador. In M. Cabrol, y M. Szekely, Educacin para la Transformacin (pp. 5392). Washington DC: Banco Interamericano de Desarrollo.
Vegas, E. (2012). Marco Sectorial de Educacin y Desarrollo Infantil Temprano. Washington DC: Banco
Interamericano de Desarrollo.
Vegas, E., y Santibez, L. (2010). La promesa del desarrollo en la primera infancia en Amrica Latina y el
Caribe. Washington DC: Banco Mundial.
Verdisco, A., y opo, H. (2012). Intervenciones Tempranas y el reto de los recursos humanos. In M.
Cabrol, y M. Szekely, Educacin para la Transformacin (pp. 93- 131). Washington DC: Banco
Interamericano de Desarrollo.