You are on page 1of 45

Education for Children from the Plateau, Eastern DRC

Teacher Training and Community Sensitisation


Programme
(2010-2014)

Implementation: Children in Crisis & Eben-Ezer Ministry International

FINAL EXTERNAL EVALUATION REPORT

Learning games: Tujenge Primary School, Minembwe

Evaluators: Jean-Emmanuel Bui and Franois Van Lierde

June 2014
2

Acronyms

IGA: Income generating activities

CEPAC: Communaut des Eglises Pentectistes au Congo (Community of Pentecostal


Churches in Congo)

CiC: Children in Crisis

CPD: Continuous Professional Development

PTA: Parent Teacher Association

CR: Comic Relief

EMI: Eben-Ezer Ministry International

IPW: In Place of War

NC: Current National Curriculum

MoE: Ministry of Education

SERNAFOR: The MoE service for Continual Professional Development

SECOPE: MoE office for personnel

TENAFEP: End of primary school state exams


3

Table of contents
Executive summary ............................................................................................................. 4
1. Background ..................................................................................................................... 7
2. The project ....................................................................................................................... 8
3. Evaluation objectives and methodology........................................................................ 9
3.1 Objectives of the evaluation ......................................................................................... 9
3.2 Methodology ................................................................................................................ 9
4. Key programme outcomes ............................................................................................. 9
4.1 Capacity building for teachers...................................................................................... 9
4.1.1 The modules, training delivery and consolidation of knowledge .......................... 10
4.1.2 Programme outcomes ......................................................................................... 13
4.1.3 Focus on two non-targeted schools .................................................................... 15
4.1.4 Teacher support networks: description and evaluation of this measure............... 15
4.1.5 EMI's partners ..................................................................................................... 17
4.1.6 The challenges ................................................................................................... 18
4.2 Capacity building for school directors ........................................................................ 18
4.2.1 Core training for school directors ........................................................................ 18
4.2.2 Teacher support networks................................................................................... 19
4.2.3 Programme outcomes ......................................................................................... 19
4.2.4 Challenges facing the basic education sector ..................................................... 21
4.3. Training and capacity building for PTAs .................................................................... 21
4.3.1 Training module .................................................................................................. 21
4.3.2 Implementation and monitoring ........................................................................... 21
4.3.3 Programme outcomes ......................................................................................... 22
4.3.4 The challenges ................................................................................................... 22
4.4 Community awareness raising ................................................................................... 23
4.4.1 The choice of topics and the training module ...................................................... 23
4.4.2 Implementing the training and monitoring outcomes ........................................... 23
4.4.3 The outcomes and the changes observed/reported ............................................ 24
4.5 Research/advocacy ................................................................................................... 26
4.6 Summary in light of the criteria relating to schooling and improving pupils' learning ... 26
5. Project Management ..................................................................................................... 28
5.1 Management.............................................................................................................. 28
5.2 Trainers ..................................................................................................................... 28
5.3 Working with CiC and developing EMI's capacity....................................................... 29
5.4 Monitoring and Evaluation ......................................................................................... 29
5.5 Accounting and financial management ...................................................................... 30
6. Learning Questions: High quality education - the project's contribution towards
social cohesion ................................................................................................................. 30
7. Relevance effectiveness efficiency impact sustainability .............................. 30
8. Recommandations ........................................................................................................ 31
Appendix I: Overview of the programme outcome indicators........................................ 33
Appendix II: Compiling quantitative data ........................................................................ 34
Appendix III: Graphical representation of the recommendation .................................... 39
Appendix IV: List of the people interviewed and the schools visited during the
evaluation mission ............................................................................................................ 40
Appendix V: Evaluation programme ................................................................................ 45
4

Executive Summary
Eben-Ezer Ministry International (EMI) and Children in Crisis (CiC) have been working in the Plateau
region since 2007, one of the most isolated areas of South Kivu, providing support to a crumbling
education system, left in ruins following decades of bad governance and conflict. The latter has led to
ineffective provision of initial teacher training, a shortage of qualified staff, an almost total absence of
state-run support and monitoring services, low enrolment and retention rates, particularly for girls and
the relative disengagement of parents, whose involvement and financial contributions nevertheless
remain key, both in terms of running and developing the region's schools.

An initial programme, aiming both to improve access to and standards of education, was implemented
between 2007 and 2009 and provided training to the school director and three teachers from each of
the 105 target schools. Bolstered by the successful outcomes of this programme, yet aware of the
significant gaps which remain, EMI and CiC rolled out a second phase to the programme (2010-2014)
funded by the Baring and John Ellerman Foundations and Comic Relief (CR), with additional support
from the Turing Foundation, the Bliss Family Trust and other individuals, foundations and trusts.
This second phase consolidated learning from the training organised in the schools involved in phase
1 and scaled-up the programme by targeting 72 new schools. Three main innovations were
introduced: re-establishing Parents Teacher Associations (PTA), implementing Teacher Support
Networks (exit strategy) as well as wide-ranging community awareness-raising activities on a number
of key issues.

The evaluation of this second phase was carried out by two consultants from 10th to 26th May in
Uvira, Minembwe, Marungu and Katobo. The main findings are summarised below.

1. Capacity building for teachers: firstly, the evaluators highlighted the relevance of the types of
training offered as well as the quality of the EMI/CiC teams' instructional design, the capacity to adapt
the various modules year on year and to remodel them (in mid-2013) using a competency-based
approach to professional development. The training strategy used offered quite intense support to
those groups of schools identified annually whose teaching staff were offered training, support and
monitoring over a period of around 10 months, with a total of 176 (1 of the 105 target schools in the
Baring and John Ellerman component ceased to function in 2010) schools supported by the
programme, and 176 school directors and 1,055 teachers trained or given refresher training by the
end of the programme. Two training cycles were organised for each cohort of schools (a total of 33
days), which were supplemented by two one-week follow-up missions in each of the target schools.
The main strengths of these training and monitoring activities included: the high participation rate of
teachers; the fact that training covered all teaching staff (to help reduce the impact of staff turnover
within schools and to make school directors aware of their responsibility); the dynamic and
participatory nature of training sessions as well as the relative consolidation of learning resulting from
the follow-up activities organised by the trainers (one of the project's added-values).
One major limitation was nevertheless observed: in order to target a very high number of schools, EMI
and CiC chose to implement a rolling programme, providing support over short periods (10 months)
after which the schools trained received no further support or monitoring, which, without detracting
from the outcomes and achievements of the programme, may help explain in part the relative decline
in learning observed in some schools.
Either way, in terms of teaching practices and improving the school environment, significant changes
were observed, despite the fact that these varied greatly from one school to another: use of active and
participatory teaching methodology; improved application of the national curriculum and the core
teaching documents; use of textbooks and teaching resources provided; wide use of positive
discipline; an increasing sense of responsibility amongst teaching staff (attendance and punctuality);
the installation of latrines in most schools and an improved capacity to manage and mediate conflict.

To ensure the sustainability of these outcomes and to address the aforementioned limitations, the
programme planned to implement teacher support networks. These networks, bringing together 3 or 4
schools, were set up to take over the reins and ensure initial training was provided independently to
new teachers as well as a degree of continual professional development for all teaching staff. In
theory, this measure was perfectly appropriate, all the more so as discussions are underway within the
MoE to roll it out nationally.
5

In practice, however, it was more problematic. Whilst some of the networks work well and meet
reasonably regularly, others are no longer operational, which leads us to draw the conclusion that
although this model is feasible more intensive support for these networks is required over the long-
term to consolidate the work carried out.

2. Capacity building for school directors: directors of the 176 target schools took part in general
training (core curriculum) as well as an additional 3-day training course on school management. The
module was also reviewed in 2013 (competency-based approach), making it possible to target more
clearly the key areas involved in running a school and in particular the pedagogical dimension, in
which school directors' involvement had been relatively low. Two follow-up visits were organised after
the on-site training offered to school directors during which the EMI trainers were able to offer
individual, targeted support to participants.
The outcomes for this component were marked in terms of the administrative and financial aspects of
running a school, however more mixed results were observed for the pedagogical component, which
varied from one school to another depending on the involvement and active role played by the director
in pedagogical matters. Overall, however, management practices often remained focussed on
administrative aspects, primarily addressing teacher and pupil attendance figures and teaching reports
to be filled out and sent to management. The school directors maintained that they carried out
classroom visits, gave advice and ran CPD meetings (SERNAFOR), however the evaluators were not
able to assess their implementation, scope and quality from the information available. The majority of
the changes observed therefore related to administrative aspects of school management (punctuality
of teachers, changes in the punishment systems, checks on staff, etc.) and financial matters (use of
basic accounting tools, setting school fees in consultation with parents, etc.). By way of a conclusion,
two objectives remain: to ensure more directors are able to ensure minimum standards are adhered
to; to enable some directors to develop a real momentum with regard to their management and
performance.

3. Reviving parent teacher associations (PTA): the aim to provide capacity building for PTAs stemmed
from the overall ambition to involve communities more in educational issues, and more specifically, in
managing schools. Two main problems were highlighted in relation to the training module used by the
EMI teams: the wide range of objectives which were not particularly coherent and the vague definition
of the role of PTAs, which could be improved by referring to their 3 main functions more specifically.
Nevertheless, the training course was delivered well, the school directors' participation was meaningful
and most of the parents were very actively involved. At the end of the programme, 176 PTAs had
been set up and revived, with 979 parents trained including 313 women.
All the PTA members interviewed stated that the training had enabled them to learn about and
understand the actual remit of a parent teacher association, which had been largely misunderstood.
Parents also became more involved, mainly in the administrative and financial aspects of running
schools (checking late arrivals, punctuality, mediating in the event of conflict, monitoring disciplinary
measures, setting fees, getting involved in various infrastructure-related projects, tighter monitoring of
the management of funds, in particular through the implementation of management councils in some
schools).
The involvement of parents in pedagogical aspects, a critical issue, remained very limited, however
two significant changes - better working relations between schools and PTAs and the development of
new accountability practices (increased reporting from school directors) - are evidence of a significant
trend towards increased ownership and involvement amongst PTAs that the programme has managed
to initiate.

4. Awareness-raising in the community:


The choice of awareness-raising activities (education for girls, early marriage, etc.) was deemed
relevant given their direct link to educational issues and the fact that they offer consolidation and
coherence to the overall programme strategy. Key competencies were set for each topic included in
the training module, further details of which can be found in the supporting documentation. Four
training cycles were organised by the EMI trainers in aid of 243 community animators (including 100
women) from 40 different committees and of 153 pastors and local leaders who took part in a one-day
training session.
The involvement of pastors and the part they play both in terms of selecting animators as well as
during the training itself were fully justified given their highly influential role in the community. As with
the teacher training, two follow-up sessions were organised by the EMI trainers which strengthened
the committees and consolidated what they had learnt during the training course. We were impressed
6

by the commitment and motivation of the three awareness-raising committees we met, despite the fact
they were all volunteers, however it is likely that EMI/CiC's aforementioned rolling approach and the
departure of their teams had a negative impact on the sustainability of some of the other committees
which we did not meet. We should also highlight the work carried out in association with the UK-based
project, "In Place of War" (IPW), specialists in developing participatory theatre and performance, with
Minembwe radio station as well as with the Club Umuco, a particularly enthusiastic local community
theatre group. The use of theatre and radio programmes as vehicles for raising awareness was
perfectly appropriate and explains, at least in part, the successful outcomes achieved in this
component of the programme and some of the main changes observed, including: increased
importance placed on the role of girls and their schooling, a more equal distribution of household
chores, almost unanimous condemnation of early marriage, greater awareness amongst parents
about the importance of education and their increased involvement in helping, monitoring and
developing local schools.

Having analysed the various activities carried out, the evaluators went on to analyse their impact using
two different approaches: 1) access to education (monitoring of enrolment rates) and 2) improvements
in the quality of education (monitoring the TENAFEP pass rate).
EMI and CiC worked on enrolment rates by comparing data for the academic year preceding their
support with the data for the academic year during which they organised their support. Significant
1
developments were noted . As for the evaluators, they focussed on changes in the workforce at the
target schools and noted, over periods of 3-4 years, a positive development (up 4.5%), due, at least in
2
part, to the increased appeal of the schools on the programme . In terms of success rates, due to a
3
major fraud issue, it is not possible to rely on the data collected . Consequently, the evaluators hereby
insist on the necessity for EMI and CiC to develop an alternative test model in order to be able to
assess the impact of their future work more accurately.

By way of a conclusion, despite the high number of target schools, the remoteness of the target area
and all the associated logistical issues, and despite the devastating, fatal events at the end of 2011,
EMI/CiC have played a substantial part in reorganising and reviving a deeply dysfunctional primary
education system by raising general standards for teachers, by giving a sense - admittedly as yet only
partial - of responsibility to school directors and by getting parents more involved.
Having said that, several considerable challenges remain and there is a real risk of seeing a
deterioration in the programme's outcomes in the event of a break in support.
Consequently, we recommend that EMI, CiC and their funding bodies launch a 3rd phase quickly in
order to consolidate the progress made during the first two phases, and in particular:

1) by working on initial teacher training in order to provide an organised, long-lasting solution to


the very low qualifications of the new generations of teachers successfully completing the
pedagogy option at the regions secondary schools;
2) by targeting a more limited number of primary schools (i.e. feeder primary schools which are
key to developing initial teacher training) and by contributing towards their development into
reference schools through a greater level of continuous support;
3) by laying down the foundations for these schools to come to prominence in the Plateau region,
mainly through strengthening teacher support networks;
4) by ensuring that the human resources required to run these networks and to organise
targeted, comprehensive and continuous support for future partner schools are permanently in
the field.

1
CR: up 6% (2010-2011), up 5% (2011-2012), up 20% (2012-2013), up 32% (2013-2014). Barings: up 11%
(2011-2012), up 9% (2012-2013), up 14% (2013-2014). .
2 Figure obtained from the data collection carried out by the evaluators in 20 schools. EMI/CiC's internal

monitoring data reports a figure which fluctuates around 10-11% (appendix 1).
3
Cases of fraud linked to various problems, including the requirement to retain a certain number of pupils in order
to benefit from the fees paid by parents, an issue which EMI/CiC have tried to address with interesting results (eg.
exchanging exam results with other schools to ensure pupils who are underperforming do not re-enrol).
7

The High Plateau of Minembwe Women's workshop: Mishashu

Sketch on forced marriage A pupil from Lubemba primary school (not


targeted)

Parents' meeting: Kalingi primary school Grade 3 pupils: Mufariji primary school

Tippy Tap: Lemera Grade 4 class: Kalingi primary school


8

1. Background
Taking the character of the projects location into consideration is key to understanding both
its origins and the inherent implementation difficulties and obstacles faced.
In South Kivu, the Mid and High Plateau is a remote area which is difficult to access by land
without using roads which are in bad condition and which make progress slow and
dangerous. The availability of air transportation (provided through the UN) is limited; EMI has
been fully supported but the reality is that UN personnel have to have priority
On a socio-economic level, the area is inhabited by various tribal groups whose coexistence
has occasionally led to conflict and whose main activity revolves around farming and
livestock. In the past, inter-ethnic clashes as well as the presence of rebel groups and
traffickers have led to population movement and displacement. The region is currently
experiencing relatively peaceful times, but the population has been left scarred, particularly
the children, and a cloud of concern hangs over their future.
On an educational level, the remoteness, the regular conflicts, the withdrawal of public
authorities from the area, the lack of awareness and the poverty of parents have led to a
continual decline in education both in terms of the quality of teaching and learning and the
schooling of children, particularly young girls.

In view of the context, particularly the declining education system in the region, EMI and CiC
decided to implement a project to improve the quality of teaching and learning for primary
school children from 2007, by working primarily on teacher training. Following this first phase,
which took place between 2007 and 2009, the second phase, which is the subject of this
evaluation, continued to work along the same lines. Lessons learnt from the first phase were
integrated by adapting a range of activities linked to the requirements of the target
populations. Improving the overall quality of education remained the main objective of phase
2.

2. The project
The project under evaluation was composed of two main elements:
The first, funded by the Baring and the John Ellerman Foundation, focussed on
capacity building for the 3 additional teachers and the director in the 105 schools
which formed part of the first training phase, on refresher training for the 3 teachers
trained during this 1st phase as well as on work to revive the PTAs.
The second part has been delivered in two halves; i)funded by Comic Relief (CR)
(and others), focussed on the training of 6 teachers and the director in 72 new
schools targeted by the project in the Mid and High Plateau, on reviving the PTAs, on
implementing teacher support networks as well as 40 community animator
committees (awareness-raising component); ii) funded by the Baring and John
Ellerman Foundation (and others), focussed on revisiting the 105 schools (although,
as already explained, this in fact reduced to 104) from phase 1 to train the 3 untrained
teachers, refresher train the previously trained 3 teachers and school director. PTA
training and TSNs were also incorporated.

The project objectives were both qualitative and quantitative:


On a qualitative level, the aim was to improve teaching practices and school
management practices and to encourage parents to be more involved in their
children's schooling.
On a quantitative level, aside from the number of staff trained, the objectives
focussed particularly on increasing the enrolment rate and the completion rate,
increasing the number of pupils taking the TENAFEP and ensuring more women
were involved in the decision-making processes.
9

In addition, the project aimed to contribute towards EMI becoming a reference organisation in
the education arena, both in the project implementation zone but also at a regional and even
a national level.
The project was to be rolled out over a four-year period, from 2010 to 2014.

3. Evaluation objectives and methodology

3.1 Objectives of the evaluation

The evaluation should measure the degree to which the various elements of the project
have been achieved through their associated activities.
The evaluation analyses how well the project was managed and the contributions of the
main stakeholders involved in its implementation.
The evaluation assesses the project in light of standard criteria relating to relevance,
efficiency, effectiveness, impact and sustainability.
Finally, based on the three aforementioned objectives, the evaluation gives a series of
recommendations for the future.
The evaluation process analyses two aspects linked to the projects success (learning
questions): the criteria for high quality education and the project's contribution to social
cohesion.

3.2 Methodology

Qualitative and quantitative data collection methods included:

A review of the documentation: the consultants had access to various documents


relating to the project: internal reports, training documents and funding body reports
amongst others.
Qualitative interviews: a series of interviews was carried out with the project team,
representatives from the decentralised education authorities, representatives from the
NGOs involved in the education sector, representatives from the regional education
administrators.
School visits: several school visits were carried out in the Lemera area and around
Minembwe and Marungu. In total, visits were made to 10 primary schools, two of which
were not targeted by the programme.
We were able to carry out lesson observations during the school visits as well as to meet
the teachers, pupils, representatives from the parents and the school directors.
Focus group bringing together the key stakeholders from the education sector: three
focus groups were set up to broaden our field visit to include schools other than those
selected for a targeted visit.
Quantitative data collection: in light of the project's key indicators of success, the
consultants compiled various categories of quantitative data4.

4. Key programme outcomes


This section presents the main achievements for each of the project's key areas: capacity
building for teachers (4.1), capacity building for school directors (4.2), training PTAs (4.3),
community awareness-raising (4.4) and research/advocacy (4.5). The final section analyses
enrolment indicators and pupils attainment (4.6).

4.1 Capacity building for teachers

4
See data collection tables in appendix
10

4.1.1 The modules, training delivery and consolidation of knowledge

1) The choice of modules

The core curriculum

For teachers and directors from the 176 target schools, training on the core curriculum was
organised into eight main modules5.

The general pedagogy and subject-specific pedagogy modules the two key areas of
the training programme - were designed to address the serious shortcomings in initial
teacher training, offering a means of professionalising and improving the knowledge base of
under-qualified teaching staff, as well as the opportunity to build capacity in teaching
methodology which is generally very weak: lack of familiarity with key teaching pedagogies; a
rough approach to planning, preparing and delivering lessons; lack of awareness of the
national curriculum and its requirements; low-level of pupil participation.

A second set of modules focussed on teacher behaviour, supervising children and also the
issues which have had more or less of a direct impact on access to and the quality of
education. Including this as part of the core curriculum was therefore viewed as coherent
given the programmes' objectives.

1) EMI/CiC deals with the issue of corporal punishment and the application of severe
punishments for pupils "at fault" (e.g. weeding a teacher's field): a reasonably widespread
practice which is ethically wrong but also has a direct impact on pupil motivation, on the
opting out of some pupils and therefore on the high drop-out rates recorded. The "child
rights" and "positive discipline" modules were therefore designed to address this issue
and provide a means to make teachers aware of their responsibilities, to continue to raise
the status and rights of pupils and, ultimately, to increase pupil retention rates within
schools.

2) Another issue: the irregular attendance and lateness of teachers and directors and the
effects of this deeply-rooted, lax approach to running a school on the quality of education
(failure to adhere to schedules and the National Curriculum, low curriculum completion
rate) as well as on the overall negative image parents and the community have of these
schools. In addition to the work to raise teaching staff's awareness of their responsibilities,
presented in point 1, EMI/CiC incorporated a "Professional ethics" module into the core
curriculum.

3) Finally, the "psychosocial" module was designed to build teachers' capacity to 1)


identify struggling pupils and 2) offer these pupils appropriate support. In a region marked
by 20 years of conflict, where violence remains part of everyday life, dealing with trauma
is completely warranted.

Similarly, the last two modules (Peace and health education and the environment) were
developed in order to address the Plateau region's specific contextual issues, offering a
means of improving the classroom environment but also of using education as a vehicle to
raise awareness, to encourage people to get involved and to bring about social change in a
region deeply scarred by inter community conflict and tensions, facing serious health and
environmental problems.

Academic consolidation: French and Maths modules

5General pedagogy, subject-specific pedagogy, professional ethics, child rights, positive discipline, psychosocial,
health education and the environment, peace education
11

The inclusion of these modules was one of the innovative aspects of the second phase of the
programme and an appropriate means of addressing the aforementioned shortcomings in
initial teacher training, both in terms of classroom practice and subject knowledge. Despite
the fact that significant progress also needs to be made in other subject areas (History,
Biology, etc.), the priority given to French and Maths - the two core subjects - is fully justified.

Refresher training for teachers trained during Phase 1 (2007-2009)

Refresher training was carried out in the 105 schools which were part of the Barings project,
where 3 of the 6 teachers as well as the director were offered initial training between 2007
and 2008.
The aim was therefore to bring them up to standard as well as to consolidate and update
their learning. This refresher training was necessary due to the high level of staff turnover in
schools and the departure of a number of the teachers previously trained by EMI/CiC (see
appendix 2, table 4). This enabled new teachers, who had not benefited from training in
2007-8 and who were not included in the core curriculum training for 2010-14, to take part in
general pedagogy and subject-specific training which was admittedly shorter, but
nevertheless key.

2) Composition and content of modules (course design)

Some of the training modules were designed for the launch of phase 1 of the programme,
others during the course of this phase. The EMI team played a key part in developing them
and was supported through 1) several training sessions by the Uvira representative from the
MoE 2) a pedagogical advisor provided by CiC for over a year and 3) organised
consultations with CEPAC and the Congolese Ministry of Education curriculum department.
Over the course of the years, the EMI and the CiC teams met regularly to discuss, review
and develop these modules, as such creating a high-quality training resource tailored to suit
the training requirements identified in the region.

A module review workshop was organised in 2013. Run by an external consultant, this
offered the opportunity to review these modules, a useful and widely appreciated process
both by the EMI trainers and the teachers who took the modules.
The starting point for this review process: putting together professional competency
guidelines for primary teachers in order to identify and set out key competencies for each
area of activity. Having chosen to focus on professionalisation, the EMI team therefore took
on the task of reviewing each of the eight modules, redrafting them and organising them
based around these various competencies, as well as the details, materials and resources
required to develop them.

In terms of course design, without challenging the quality of the worked previously carried
out, this approach seemed both useful and interesting: modules were more in line with
national policy and the pedagogical aspirations of the current national curriculum, the quality
of training was improved thanks to the advantages this approach brought in terms of clarity of
content and training and appropriation objectives as well as the involvement and motivation
of the teachers being trained.

It is however unfortunate that this review took place some time into the programme, and as
such only benefited the fourth and last cohort of teachers and directors trained by the
programme in Bibokoboko and Lemera.

Having said that, the two organisations managed to develop a package of very high quality
training modules covering a wide range of training areas, which were tailored to suit and
12

comply with national policy in the sector and which, over time, could undoubtedly become a
reference for state and non-state stakeholders working in the area of teacher training in
DRC.

3) Implementing modules and monitoring the application of learning

EMI/CiC's training strategy is based on offering relatively intensive support to school groups
(18 for Comic Relief and 35 for Barings) identified annually in each of the project's six zones
of intervention6 and in which the teaching staff was given training, support and monitoring
over a period of roughly ten months by the EMI teams (six trainers for CR and five for
Barings). This support was broken down into four key stages:

1) 23 days of general training organised during August. For Comic Relief, this training
was offered to all of the teachers and directors in each of the 18 target schools each
year, whereas for Barings, three teachers (i.e. those who did not receive training during
phase 1) from each of the 35 target schools took part. They were divided into groups of
30 people and worked with one of the EMI trainers in charge of the training programme
for all eight modules. Further details about the competencies targeted by each module
and their duration can be found in the appendix (table 1).

2) 10 days additional training in French and Mathematics organised for three


teachers per school deemed to be struggling. Alongside this, the Barings team
organised refresher training in each school for the three teachers and directors trained
during phase 1 (six days), whereas the Comic Relief team offered a three-day course
for the director and/or one highly-experienced teacher on setting up and running
teacher support networks (see appendix).

3) Two follow-up and knowledge consolidation missions. This is undoubtedly one of


the approach's key added-values. For each cohort, the EMI teams organised two
month-long follow-up missions (in October and February) during which each trainer
was tasked with monitoring three (Comic Relief) or six-seven (Barings) schools and
carrying out at least six (Comic Relief) or three (Barings) lesson observations for the
trained teachers. This made it possible to evaluate and monitor whether what
participants had learnt in training was being applied in the classroom. In addition to
offering advice to the teachers observed, the trainers were able to keep the targeted
teaching staff well motivated and involved thanks to their regular presence on-site.

STRENGTHS AND WEAKNESSES OF THE IMPLEMENTATION


I. Teacher training
Very few absentees and an attendance rate of
nearly 100% for training courses: evidence of
teachers' interest in the training.
Targeting three teachers/school (Barings),
Targeting six teachers/school (Comic Relief):
assuming that the three other teachers had
to address the shortcomings of phase 1 (low
already been trained, which was not
level of internal feedback)
necessarily the case (due to turn-over).
Targeting three out of six teachers for
The inclusion of directors in the training:
additional training in French and
essential given their responsibilities in terms of
Mathematics when the need was, and
pedagogical supervision.
remains, more wide-spread.
The relatively short duration of the training
Covering a wide range of subject areas, issues
given the number of modules and the very
and competencies.
low initial level of participants.
The quality of the training delivery, the highly- Modules were reviewed and adapted too

6 Minembwe, Itombwe, Marungu, Katobo, Bibokoboko and Lemera.


13

qualified trainers and the use of a competency- late (competency-based approach).


based approach (2013)
The very short time devoted to refresher
The small group size (around 30 people) and
training for teachers who had not been
the systematic use of active and participative
trained for a long time (sometimes four
teaching methodology.
years).
The involvement of several schools,
sometimes from various different communities,
and the diversity this created within the
participating groups.
The fact that training took place on the Plateau
at one of the target schools
II. The two follow-up missions and the lesson observations
In order to target a large number of
The regular presence of trainers in the field: a schools, the programme decided to opt for
must in terms of consolidating learning and a "rolling" approach: intensive support over
keeping the project going. a short period (10 months) and the
departure of the teams after one year.
The high number of lesson observations and The more limited means of the Barings
the personalised and "tailored" approach to team and the reduced number of lesson
monitoring and feedback in this situation. observations (three rather than six).

4.1.2 Programme outcomes

1) Quantitative data

The programme reached its objectives and complied with its forecasts (1,231 people trained
from 176 schools), despite the fatal events of October 2011.
The way in which the EMI team managed to get back on its feet, to reorganise itself and to
stay the course during this extremely difficult time is quite remarkable and demonstrates the
genuine commitment of the EMI teams, without forgetting all the support, both moral and
logistical, offered by CiC which also deserves to be highlighted here.

2) Changes in teaching practices and improvements to the classroom environment

The table below outlines each of the training modules/topics and comments on the key
changes in practices noted by the consultants in the schools they visited and/or discussed
with the teachers and directors during the three discussion workshops organised on site
(Marungu, Katobo and Minembwe).

Training module Key changes observed/reported


Current national curriculums in (almost) all schools and greater use of
the curriculum by teaching staff.
The majority of teachers complete the annual programme of study.
Some teachers prepare detailed lesson plans using EMI's simplified
format (once a week at best).
General and subject- Teachers fill out their pedagogical paperwork better.
Teachers have, for the most part , adopted some or all of the active
7
specific pedagogy
and participative teaching methodology techniques.
The majority of teachers stick to the lesson format (setting out the
learning objective, main body of lesson, etc.)
More text books available to prepare lessons.
Teachers give more homework to pupils.

7
According to EMI internal monitoring: 87% of teachers (Comic Relief) use A&P methodology (compared to 8% at
the beginning of the programme). For the Barings component, the figures are 68% compared to 30% at the
beginning of the programme.
14

The take-up of pedagogical practices varies significantly from one


school to another and we saw a relative decline in the schools which
received support at the beginning of the programme.
Lesson planning is still rare in a considerable number of schools. In
CEPAC schools, planning is based on the (more complex) mandatory
lesson planning template which is distributed by the school governing
body.
Whereas in some schools pupils are very actively involved in lessons,
in others they remain limited to rote learning.
Contextualising lessons remains a problem almost across-the-board
Comments
(problems delivering lessons, being creative and imaginative).
Overall, we noted very little use of reflective approaches amongst
teachers and a poor ability to adapt. Teaching styles and pedagogical
guidance remain very formal.
Writing practice (exercises, lesson content, homework) still remains
too limited in most schools. Generally speaking, pupils' exercise books
are not used effectively and are very rarely marked by teachers.
Given the starting point, teachers' low capacity and the lax approach to
school management, the changes observed remain significant overall,
despite their variation.
Significant reduction (or complete eradication in some schools) of
corporal punishment ("the cane") and use of alternative and "positive"
8
punishment systems (community service tasks).
Significant reduction (or complete eradication in some schools) of
Positive discipline / severe punishments applied by teachers to some pupils (e.g. weeding
Child rights a teacher's field).
Overall: greater consideration and respect for children, a more
collaborative relationship between children and teachers, a change in
mentality (ongoing) and in teachers' perception of pupils and their
rights.
Whilst cases of corporal punishment do still occur, they are almost
unanimously condemned by the teachers, directors and parents met
during the evaluation process.
This is one of the changes people interviewed commented on most
Comments readily and most frequently.
Overall, we observed that children are more interested in school and
happier to go there, which can be explained by the reduction in
corporal punishment as well as by the more active and participative
teaching methodology used.
Teachers arrive on time and keep to lesson times more.
There are fewer unauthorised absences and these are subject to
disciplinary action.
School directors are on site more frequently.
Professional ethics Teachers no longer routinely turn away pupils unable to pay their
school fees.
Overall, teaching staff have become more aware of their
responsibilities, which is in no small measure due to the work carried
out to raise the status of their profession, a result of the support and
training contributed by EMI.
The punctuality and regular attendance of teachers and directors at
the target schools are two of the changes parents interviewed during
the evaluation process commented on very frequently, almost
Comments systematically, and for them it is a significant change.
This has without doubt had an impact on parents perception of the
schools where their children are educated ("before, it was all a mess,
now it is run more professionally") and also, at least in part, on getting

8
According to EMI internal monitoring: 93% of teachers (Comic Relief) use positive discipline (compared to 8% at
the beginning of the programme). For the Barings component, the figures are 77% (compared to 22%).
15

them back on board.


In terms of pupils unable to pay their school fees, non-mechanised
schools still put significant pressure on children and their parents, for
quite obvious reasons.
The wage system for teachers goes some way towards explaining the
repeated absence of school directors. A delegation system (proxy)
exists in some areas and it would be beneficial to extend this
approach.
Some teachers made efforts to approach "difficult" or socially isolated
children to open up dialogue with them and their parents. Very few
Psychosocial cases. This is one of the training components where relatively few
education changes were observed.
Games are organised regularly during break time. It is difficult to judge
the impact of these accurately.
It is a question of changing mentalities, which is more linked to culture
and therefore more complex. This explains to a great extent the limited
impact of the one-day training course organised on the topic.
Wider-reaching awareness-raising activities for parents and the
community would no doubt have had a greater impact on the
Comments anticipated outcome.
A question concerning the level of qualifications of the EMI trainers
and their capacity to lead training sessions, and the need for a debate
about this type of issue more generally.
Another issue: the way teachers and schools deal with academic
failure, which is insufficient and sometimes nonexistent.
A number of latrines were built in most schools, most often with the
support of parents.
Health and Improvements in the general hygiene conditions in schools:
environment cleanliness of site, wearing of uniforms, children's personal hygiene.
Installation of Tippy Taps in all schools.
The module acts as a useful resource document for teachers.
With regard to the environmental component: 1) lack of
Comments contextualisation (in relation to the local environmental issues) and 2)
little or no putting theory into practice (e.g. reforestation initiatives)
Teachers more aware of their responsibilities in terms of mediating
conflict between pupils (previously indifferent).
Teachers try harder to encourage pupils from different tribal groups to
Peace education
work together more through games, group work and seating plans in
class.
A greater diversity amongst teaching staff in some schools.
Comment See point 6 relating to social cohesion.

4.1.3 Focus on two non-targeted schools

Whilst it is difficult to make generalisations based on two school visits, the serious failings
recorded there do highlight the positive changes observed in the project schools. The visits
carried out in these two schools can be summarised as follows: no director present at the
time of the visit; serious issues of punctuality amongst teachers; lax approach to running the
school and supervising children (completely left to their own devices in the absence of
teachers); no copies of the national curriculum, few or no textbooks, few or no up-to-date
teaching documents; systematic use of chalk and talk approach to teaching; very low level of
pupil participation; little or no use of teaching resources; homework rarely given; low level of
subject knowledge; teaching staff generally very demotivated (NB: $5 (3)/month for
teachers in the school visited in the Marungu area).

4.1.4 Teacher support networks: description and evaluation of this measure


16

The implementation of teacher support networks is one of the innovative aspects of this
second phase of the programme. They form the main element of EMI/CiC's exit strategy and
were designed to address the key challenges to ensuring the sustainability of the
programme's outcomes:

The level of staff turn-over at schools and the departure of trained staff (see table 3),
which can be explained by their lack of job security. The challenge: to train newly
recruited teachers and crucially, to raise standards.
The inherent weaknesses in initial teacher training, teachers very low level of
qualifications, the limitations of the training delivered by EMI/CiC (a relatively short
training course in relation to the magnitude of requirements - but nevertheless high
quality training which was in many cases the only training which teachers had
received for many years), and because of this, the need to offer teachers continual
professional development.
Along similar lines, the "rolling" nature of the training and the fact that support was
concentrated over short periods of time (9 months) and then discontinued.
The almost complete lack of state-run educational monitoring and supervisory
services and the withdrawal of most of the school governing bodies in charge of this
type of monitoring9.
The inadequacy of SERNAFOR and its inability to address the challenges of
providing high quality, sustainable continual professional development for teachers.

Bringing together three to four schools within each of the programme's zones of intervention,
the teacher support networks were designed to take the helm, organise regular training and
independently ensure initial training for new teachers as well as continual professional
development for the others, by capitalising on the resources and skills available internally.
This is an appropriate measure. Its application is currently under discussion at the MoE.

1) Analysis and evaluation of the implementation

The training module: the module designed at the beginning of the programme was
subsequently reviewed in 2013 following the workshop on teacher support networks
which the EMI programme officer attended in Kinshasa. Whilst the module was
redesigned using a competency-based approach, determining the key competency
nevertheless proved problematic. It should have been more detailed and more
practical in its composition in order to give members of the networks a clearer idea of
their responsibilities and the operational strategies and approaches to setting up and
running these networks.

Training: at the end of the programme, 47 networks had been set up following the
three-day training courses organised for the director and one teacher from each
school. Without challenging the usefulness of the relatively formal approach to the
training courses (giving information about the role, remit and internal organisation),
they could have benefited from having a more practical focus (developing action
plans, programmes of activity, identifying together the skills/resources available) in
order to lay the foundations and embed these networks in a more concrete fashion.

Follow-up: as a result of being set up during the training session in December, the
networks only benefited from one follow-up visit. During this visit, the trainers' time
was mainly devoted to monitoring teachers and observing lessons, less time was

9
Only the 8th CEPAC has an educational advisory department, but the funding it has so far received for its
schools is not yet enough to put in place a package which could be used to establish an effective educational
support system in these schools (4 advisers for 266 primary and secondary schools) and this is particularly the
case in the Plateau region.
17

allocated to supporting and coaching the networks and those running them, an aspect
which, given the complex nature of launching these collaborative approaches, would
have benefited from greater support. An additional problem: the "rolling" nature of the
programme which once again limited the programme's capacity to ensure the work
was both continued and reinforced.

The resource centres: the programme envisaged the establishment of resource


centres within a network-member school. Concerned about tensions between the
schools, EMI and CiC decided to take a different approach: they used the available
budget to distribute additional textbooks and $150 to those networks deemed to be
operating well to help them launch IGAs. This support is too recent (1st half of 2014)
for us to be able to judge the impact.

2) Conclusion and general observations

Due to the aforementioned weaknesses, but also due to a serious issue with motivation and
local technical capacity, which is still not adequate enough to promote involvement in these
kind of training activities, the overall outcomes of this part of the programme remain broadly
mixed, and vary greatly once again from one zone of intervention to another.
Whilst it is clear that some of the networks are not or no longer operational, others seem to
be working well and continue to meet at more or less regular intervals (an average of five
times a year for the networks which the consultants met), making it possible to maintain a
certain level of continual professional development within the schools in question10, offering if
not an alternative, then at least an addition to the SERNAFOR service which schools have
for the most part abandoned, and highlighting both the potential of this measure as well as
the difficulties in setting it up. We will come back to this point in more detail in the
recommendations.

4.1.5 EMI's partners

Whilst Congolese NGOs often tend to work in isolation, it is interesting to highlight that
throughout the implementation of this programme, EMI, with active support from CiC, has
tried to build collaborative links and coordinate their work with bodies both at a local, district
and national level. We have listed and explained them below:

The Uvira I local education officer took an active role in training the EMI trainers, right
from the beginning of the programme, particularly with regard to developing training
modules.
Training modules presented to the MoE (Uvira I and II) and official recognition
secured.
Training modules presented to the National Ministry in Kinshasa where the approval
process is currently underway.
Involvement of the local education authorities and the Uvira I inspector in the launch
of the two training programmes. They were not, however, directly involved in running
the training sessions.
Active involvement in the Uvira education sub-cluster, co-lead by EMI, which helped
to increase the organisation's exposure.
Meetings with the CEPAC governing body to share information and particularly to
discuss the modules, the training materials and the selection of beneficiary schools.
Despite this, we noticed duplication in their detailed lesson plans which created a
certain amount of confusion within schools supported by both organisations.
10
Some of the topics discussed during these meetings: reviewing the programme of study, training on detailed
lesson planning, subject-specific training (mainly French and Maths) and model lessons delivered by various
members of the networks.
18

4.1.6 The challenges

In point 4.1.4 we listed and explained some of the main challenges faced by the EMI/CiC
programme: inadequate initial teacher training, lack of job security, high level of turnover
(particularly in non-mechanised schools), the near-complete withdrawal or lack of state-run
educational monitoring services and the main school governing bodies.
By way of a conclusion for this "capacity building for teachers" component, we would like to
put forward a further key challenge: standardisation.
During the programme implementation, EMI/CiC was faced with schools which were in very
different situations in terms of 1) their infrastructure (bricks and mortar buildings v schools
built from local materials which were often run down), 2) their status (private schools, non-
mechanised schools or state-run and funded schools), 3) teachers' level of qualifications and
the level of supervision by directors and 4) the teaching and pedagogical resources and
equipment available.
This considerable diversity highlights a major issue: the DRC education system's complete
lack of minimum standards both to regulate how primary schools operate (and the conditions
required for establishing them) and to guide their development.
The consequences of this lack of standards could be seen in the field: 1) a relative
proliferation of non-viable schools whose development prospects remain very low, if not
hypothetical 2) a lack of direction or vision even in the better organised schools, a result of
the absence of clear objectives (and standards) from which to draw up effective professional
development plans and 3) operational challenges for organisations such as EMI and the
targeting of some schools which simply did not meet the necessary conditions to implement
changes.

4.2 Capacity building for school directors

Firstly, this analysis looks at the means implemented to build the capacity of school directors,
and secondly it presents the impact these made within the primary schools.

4.2.1 Core training for school directors

The core training comprises training modules for teachers and a specific module relating to
school management.

Training modules for teachers


Ensuring school directors took the teacher training modules had two key benefits: 1) it paved
the way to establish a common culture for pedagogical and educational practice and 2) it put
directors in the position of being the main source of support and feedback for teachers.

School management module


The training in school management comprises a specific training module and follow-up visits
to assess how well learning is then implemented on-site.

Training Content Main changes made to the


module module between 2010 and 2014
School The module focuses on three competencies In 2013, the school management
management associated with the main areas of school training module was completely
management: 1) the pedagogical management reworked to develop clearly-
of a school, 2) administrative management and targeted professional
3) financial management .The module competencies.
develops cross-disciplinary competencies: how
to plan the various management activities.
19

Comments:
Until 2013, the module was mainly based on imparting knowledge relating to the
administrative management of schools. Since 2013, the key areas of management have
been clearly defined and the training has targeted the development of operational
professional competencies. The aim is for directors to move from the position of an
administrator, to that of a manager, capable of improving the performance of their team and
their school. In terms of pedagogy, the director should focus on the pedagogical supervision
of teachers and the provision of a system to assess pupil performance. It is unfortunate that
this reorientation towards developing competencies only took place at the end of the
programme.
The module was delivered through week-long on-site training sessions. From 2013, the new
version of the module instigated the use of active teaching methodology and several role
plays preparing for real-life management situations.
The on-site training was followed up with two support phases in the field, carried out by the
EMI trainers. The support visits offered an opportunity to see how management practices
had changed, to correct any divergence of practice and to remind directors of the key
notions.

4.2.2 Teacher support networks

Whilst the primary mission of teacher support networks was to train teachers, the
involvement of directors also enabled them to identify best practice and to discuss
management issues with their counterparts in the other schools belonging to the network.

4.2.3 Programme outcomes

Quantitative data: at the end of the project, 176 directors had been trained
Changes in practice observed in the various areas of school management.

Management Changes in management practice


area
Pedagogical and At the pedagogical level, management practices remain focussed on
educational administrative aspects, which mainly cover teacher and pupil attendance, filling
management out and transferring pedagogical reports to the authorities.
i) In terms of supervising teachers, the directors state that they carry out
lesson observations, offer advice and run pedagogical meetings. However
there is little that enables us to assess the reality and quality of this
supervision:
- lesson observations often reveal problems with teachers' pedagogical
paperwork (not updated, no signature or checking by the director, etc.)
- lesson observations also reveal that teachers may take certain liberties when
delivering teaching and learning (lack of learning objective, no practical
application of learning, lack of contextualisation, etc.);
- the interviews carried out confirm what was found during lesson observations,
however they also highlighted irregular, if not an almost systematic absence of
continual professional development sessions.
ii) In terms of setting up a standardised system to assess pupils'
performance in each school, it should be noted that no school was in a
position to do this by the end of the project.
At a more educational level, the directors are much more aware of positive
discipline, respecting child rights and adhering to some of the health and
environmental rules (pupils' personal hygiene - latrines - Tippy Tap - health and
safety teams - cleaning duties etc.)
Administrative According to the directors, this is where the most progress has been made:
management identifying and filling out paperwork and reports for the authorities; punctuality
of teachers and pupils; monitoring and checking staff. By contrast, lesson
observations revealed: school rules were not systematically displayed in
20

classrooms; late and staggered arrival of pupils.


Accounting and Practices did improve in this area, particularly thanks to the growing authority
financial of the PTAs. Henceforth, school fees and operating costs are set in association
management and with the agreement of the PTAs, ensuring greater transparency and
fairness. PTAs also check the use and management of funds.
However, an interview carried out with the representative from the MoE office
for personnel in MINEMBWE revealed that some school directors tend to hide
the state contribution to operating costs from parents and to misappropriate
these funds.
The contribution by the state towards operating costs remains insufficient to
cover schools' requirements and the lack of regular payments makes it hard to
forecast and draw up investment plans. As for the teachers' salaries, parents
contribute to or pay these systematically, whether the school is mechanised or
not. Schools remain predominantly funded by parents.

Comments:
Given the starting point, it seems that the most effective directors today are those who
ensure the minimum rules are adhered to and that their schools are operational: children
come to school - teachers take classes - the facilities and materials are available to ensure
effective teaching and learning - parents are involved in the running of the school. For the
administrative and financial management component, significant progress has been made
which is a first step. In terms of the pedagogical management component however, the
outcomes and changes remain more limited, a fact which needs to be viewed within the
context of the starting point which, in this area, was particularly weak before the project.
Two objectives remain: to ensure more directors are able to ensure minimum rules are
adhered to; to enable some directors to develop a real momentum with regard to their
management and performance.

Focus on non-target schools

Focus of the Comments


observation
Punctuality of In each school, the director did not arrive on time at the beginning of the day. At
the director the school in Minembwe, someone even had to go and get him from his own home
one hour after lessons had theoretically started.
The director's Lesson observations and interviews with teachers revealed the low level of
pedagogical pedagogical guidance offered to teachers which was evidenced by:
leadership - lack of national curriculum;
- lack of pedagogical documents;
- lack of structure to continual professional development sessions;
- no use of active teaching methodology;
- irregular or lack of SERNAFOR sessions;
- children badly presented (dirty, no uniform, etc.).
Administrative - children and teachers did not arrive on time at the beginning of the school day
and financial (between 30 minutes and 1 hour late in Minembwe);
management - classes were chaotic + insufficient furniture and equipment;
of the school - school rules not on display;
- the evaluator observed the persistent use of corporal punishment;
Working with The PTA at the school in Minembwe stated that it was not longer running and that
the PTA it had delegated management to the director.
Other During the interview with the director at the school in Minembwe, the latter
indicated that he had no authority over his teachers.

Comments:
These visits highlighted how the education system probably worked before the project began
and before the specific school management training was given.
21

4.2.4 Challenges facing the basic education sector

As regards capacity building for school directors, there are several challenges:
To move towards applying standards to primary schools, which would cover several
aspects: the status of the school, the infrastructure, the equipment, teachers'
qualifications and pay. Indeed, it was noted that the involvement of directors, as well as
the effective deployment of their competencies, often depended on the status of the
school, its financial resources and the involvement of parents;
To review the status of school directors and in particular the recruitment process, career
path and how these are handled.
To establish initial training for directors which could be implemented as part of the their
pedagogical studies or as a specific management module;
To initiate a change in governance at all levels of the educational authorities. The aim
would be to introduce the ideas of target-based and project-based management and to
instil greater use of evaluations;
To set up or restore monitoring and evaluation of school directors, either through school
inspectors, or through the school governing bodies.
Eventually, directors should become key players in the education system for, in the end, it is
their role to ensure national education policy is implemented. In Congo, where there is no
intermediary administration, this is all the more so as they are solely responsible for ensuring
their schools are well managed.

It is important to highlight that both for the capacity building with teachers and the capacity
building with directors, EMI/CiC consistently involved the decentralised education authorities
and the most established governing bodies, in particular the CEPAC 11 . In addition, they
contacted other NGOs involved directly or indirectly in the education sector.

4.3. Training and capacity building for PTAs

4.3.1 Training module

Training module Objective/content


Involvement of parent teacher The module focuses on 3 competencies: building the capacity of
associations in running PTA members to enable them to get involved in the sound
schools. management of schools; respecting child rights; applying positive
discipline.

Comments
The structure of the module is confusing; it targets a wide range of objectives without any
overall coherence. Indeed, how is it possible to associate a management-specific
competency with two others relating to pedagogical and educational matters?
The competency relating to involving PTAs in school management lacks structure. It should
have been divided into three separate skills along the lines of: getting involved in a school's
pedagogical management - administrative management - and financial management.
Despite this misjudgement in its design, the module offers resources to explain the remit of
PTAs to parents, and to equip them to intervene in the 3 management areas.

4.3.2 Implementation and monitoring

11
Involving and informing partners focussed on developing training modules, the implementation phases on-site
and the outcomes. By working this way, EMI made it clear that they chose to promote national policy in terms of
the curriculum, training and running the education system effectively.
22

The on-site training session, which in some cases also involved the school directors, was
followed by a monitoring phase to assess the level of commitment of the PTAs and to
organise an action plan.

4.3.3 Programme outcomes

Quantitative data: 176 PTAs were trained, namely 979 parents including 313 women.
Changes in behaviour and practice: according to all the PTA members interviewed, the
training enabled them to find out about and understand the actual remit of the PTA, which
was unclear to almost all of them.

Area of Practice/behaviour post training


intervention
Getting involved Parents are involved in checking pupils', teachers' and the director's lateness and
in the school's absence. They lay down standards which are agreed by the PTA and director. In the
administrative end, the school timetable is better adhered to. Parents offer their services to mediate
management any disputes or disagreements which may occur within the school. Parents ensure
that corporal punishment is abolished.
Getting involved Parents and directors decide together on the amount of their different contributions as
in the school's well as how to raise and allocate these between teacher salaries, running the school
financial and various construction and rehabilitation projects. Directors are no longer the
management "almighty" in this area, they have to keep the books and report back. PTAs contribute
towards ensuring schools are better managed.
Getting involved In this area, parents' contribution was mainly administrative.
in the school's Parents are not very involved in the pedagogy implemented, in ensuring the
pedagogical curriculum is adhered to and taught, or in the distribution of work carried out at school
management and at home. Parents are reticent to get involved at a pedagogical level due to a lack
of information and having themselves not had the opportunity to complete their
schooling.
Awareness- For this part of the work with PTAs, our observations highlighted a range of different
raising in the initiatives: awareness-raising about educating children, particularly girls; work carried
community out with parents about school fees and operating costs. In some specific cases, the
PTAs were able to come up with solutions for the poorest families; awareness-raising
initiatives about supervising homework; awareness-raising about reducing the burden
of household chores imposed on young girls. In this area, the evaluators noted a
significant trend towards a fairer distribution of chores.

Comments:
The training had a significant impact on parents and school management.

Focus on a PTA which was not targeted by the project


During a visit to a non-targeted school (in the Minembwe area), the evaluator met a PTA
which had not been given training. The representatives of this PTA stated that, for lack of
information about their remit, they did not get involved in managing the school. In addition,
the representatives made it clear that they wanted more specific information about their role
to be able to get more involved in the various aspects of school management. The parents
interviewed appeared open to the idea of changes in terms of girls' education, abolishing
corporal punishment, optimising the management of grants received by the school to fund its
operations and paying teachers wages. This visit emphasized the significant differences
between trained and untrained PTAs.

4.3.4 The challenges

Several challenges remain in relation to PTAs:


Setting up a system to enable future elected representatives to benefit from the training
given to their predecessors;
23

Bolstering PTAs on a pedagogical level so that they can contribute effectively in this area
and that this involvement helps to improve the quality of education: understanding how
the education system is organised - understanding the curriculum and pedagogical
approaches - understanding how learning is assessed, which will enable parents to
monitor their children's work and to engage in constructive dialogue with teachers and
the school director;
Finding a middle ground between the PTA prerogatives and those of the school director.
PTAs should not interfere too much in the operational management of schools but should
act as a source of advice, support and, in some cases, be called upon to make an
assessment;
The biggest challenge is undoubtedly obtaining financial support from parents, who find
this burden difficult to bear and which remains the true obstacle to children's education.
As such, it would be preferable for teachers to be mechanised, for their status to be
raised and for schools to be granted an operational budget which reflects the reality of
their requirements.

4.4 Community awareness-raising

4.4.1 The choice of topics and the training module

The awareness-raising component was a new development introduced during the second
phase of the programme. At the beginning of this phase, the EMI officer in charge of this
component carried out a baseline evaluation of the various zones of intervention. This
evaluation helped identify the key awareness-raising topics12.
Overall, the choice of topics is deemed appropriate: 1) they address real issues and are
tailored to the region's specific context and 2) most are directly linked to educational issues,
as such the awareness-raising component supports the overall project strategy in a
completely coherent way.
In relation to the training module, it was developed by the EMI team at the beginning of the
programme with the particular benefit of various contributions from those people taking an
active role in this area. Key competencies were agreed for each awareness-raising topic,
which have been set out in detail, presenting several references, in the form of documents as
well as the structure and content of the key awareness-raising messages and arguments
(see table 5 in the appendix).

4.4.2 Implementing the training and monitoring outcomes

Four training cycles were organised by the EMI trainer13, who was supported from 2012 by
an assistant trainer. Despite the trainer being a victim of the attack at the end of 2011 and
having to postpone the training for over six months due to serious health problems, the
awareness-raising component managed to achieve its objectives, which demonstrates once
again the determination of the EMI teams to implement this programme.
Overall, these training sessions targeted 243 community animators (including 100 women)
identified in association with religious leaders, who underwent a four-day course, as well as
153 pastors and local leaders who were given a one-day course in each of the zones of
intervention. The participation of pastors and their involvement in both selecting the
animators and in the training itself was fully justified, essential even, given their role and
influence in the local communities.

12
The role and the importance of education; education for girls; the role of parents in supervising their children's
schoolwork; the prevention of early marriage; involving women in decision-making processes; hygiene, sanitation
and preventing HIV/AIDS
13
In Minembwe, Marungu, Itombwe and Bibokoboko.
24

At the end of these training sessions, 40 community animator committees had been set up.
They were then tasked with running two to three awareness-raising sessions per month,
mainly in churches, on a voluntary basis.
Whilst the EMI trainers brought a certain amount of ethnic and cultural diversity to the
committees - and without detracting from this - it is nevertheless important to note the case of
the Kitoga committee in which 6 members are Bafulira and from the 8th CEPAC, which
seems to have limited their capacity to intervene in different church/tribal communities.
The EMI awareness-raising team should also be more diverse in the future in order to
improve its capacity to mobilise people.
Two follow-up visits were systematically organised by the EMI trainers, which was key in
terms of consolidating learning. On this point, EMI's "rolling" approach nevertheless
presented a certain drawback, as this monitoring and supervision were limited to a relatively
short period (10 months), after which the trainers, having been sent to other areas, no longer
had any direct contact with the committees. Despite the fact that the three committees we
met were highly motivated, and whilst it is difficult to come to a clear-cut conclusion on this
point, it seems likely that this rolling programme had a negative impact on the sustained
commitment of some of the committees trained.

CiC and EMI also worked in association with the UK-based NGO "In Place of War" (IPW),
specialists in participatory theatre, to implement this awareness-raising component. IPW was
tasked with training the EMI teams which would go on to train the animator committees. The
IPW representative also went to Minembwe and to Bibokoboko to work directly with the
committees established in these areas and in particular the Club Umuco in Minembwe, a
particularly dynamic local community theatre group with which EMI and CiC signed a
partnership protocol, thereby entrusting the group to carry out some of the awareness-raising
work and radio broadcasting. Using theatre as a means of raising awareness and this
partnership with the Club Umuco are two factors which contributed significantly to the
sustainability and relevance of this awareness-raising component.

4.4.3 The outcomes and the changes observed/reported

At the end of the programme, records showed 11,280 participations (including 5,992 women)
in the various awareness-raising activities organised by the 40 committees, making an
average of 282 participations per committee, which seems a relatively low figure overall, but
this probably reflects the committees' varying levels of input and pro-activity.14
We should not forget that some of the direct beneficiaries of this awareness-raising
component include the 153 local leaders trained, as well as the thousands of radio
Minembwe listeners, who tuned in to the awareness-raising programmes.15
The table below summarises the main changes observed and also draws on the work carried
out in the focus groups organised in Minembwe, Marungu and Katobo (see table 6).

Changes within each awareness-raising topic


The importance Reviving the PTAs led to parents taking a more active role and being more
of education and involved in monitoring and managing schools.
the role of Parents were more interested in their children's education: more supervision
parents at home of classwork and homework.
The evaluators looked at the fees paid by parents and how these had
changed, deeming an increase in fees as an indicator - admittedly one which
Comments is far from perfect - of their more active role. The findings of the qualitative
study are set out in detail in the appendix (table 7). In summary: 1) a
pronounced increase in Minembwe (up 47%) and 2) a relative stagnation in

14
The figures presented by the EMI trainer do not show the number of individual people who took part in the
various awareness-raising sessions.
15
Radio Minembwe broadcasts over a wide area in this region and covers almost the whole of the Mid and High
Plateau.
25

the Marungu area where average fees are significantly lower


(1,100FC/month (0.71) compared to 2600 (1.68) in Minembwe). This may
be explained by the fact that the population in Marunga is more vulnerable
and that awareness-raising activities only began recently.
The issue of education for girls is widely discussed in the local area (this
topic was brought up at every opportunity by local people we met).
The status of girls seems to be improved / parents understand that a girl who
is educated can also offer considerable support to her family.
Some pastors are also actively engaged in raising awareness.
Education for
girls A great many of the parents we met stated that they had redistributed the
household chores more equally between the girls and boys.
Schools organise lessons in the afternoon for girls who have to stay at home
in the morning to complete household chores (e.g. looking after their younger
siblings in infancy).
Girls speak out more freely in group situations.
Tables 8 and 9 show the results of the quantitative study carried out. A few
observations: 1) the proportion of girls enrolled has increased in Marungu
(from 42 to 45%) and in Minembwe (from 49 to 52.5%), 2) the proportion of
Comments
young girls enrolled in the final two years of primary has increased by a small
degree (42.7% to 43.3%) and the drop-out rate for girls in the final two
grades has fallen considerably, from 13.7 to 8.3%.
Almost unanimous condemnation of early marriage.
People are aware of and, on the whole, abide by the legal age for marriage.
Girls' freedom of choice is more widely recognised.
The prevention Signature of a statement of commitment by the pastors (Marungu, active role
of early played to raise awareness and a certain number of measures taken, for
marriage example temporary excommunication of parents who consent to early
marriage for their daughters).
There is a strong link between this and the topic of education for girls. A
change in mentality observed in young girls (and in some parents): the
priority is now education and completing their schooling.
The issue of marriage by abduction was discussed locally (parents, pastors,
teachers), but remains by and large a taboo subject. Consequently, it is very
difficult to judge this point and/or confirm that this practice has been
Comments eradicated as some of our contacts reported.
Another serious issue: equal inheritance rights for women. Whilst we have
reported cases of some women who have inherited, these remain
exceptional.
Relative improvement in women's representation in the PTAs (25% Barings /
35% Comic Relief) and the awareness-raising committees (around 50%).
Involving women
Female members of the awareness-raising committees take a very active
in decision-
role and often speak out and give their opinion in public (in particular during
making
the theatre sessions in the churches).
processes
In Katobo, 41% of the class prefects are girls (no baseline data available to
16
provide a comparison) .
Whilst women are represented in the PTAs, the degree to which they
genuinely participate in decision-making however seems to remain (very)
limited.
The proportion of female teachers in schools remains extremely low (9.5% in
Comments the Katobo area - and 7% of all the teachers trained by EMI/CiC), which is a
major impediment to improving girls' education and their retention rates at
school.
There were two reported cases of women who had become school directors,
which is a noteworthy development (may act as role models).
Hygiene, Relatively few developments observed within communities, compared to

16 EMI's internal monitoring shows the following results: 43% of class prefects are girls (Comic Relief
schools). They represent 30% of class prefects in Barings schools.
26

sanitation and schools, where changes seem to be more visible (installation of latrines,
preventing Tippy Taps, etc.).
HIV/AIDS Nevertheless, some accounts do suggest changes are being made in terms
of treating milk (i.e. boiling to eliminate the transmission of TB) and the
building of family latrines, to a limited extent.

Some significant changes were observed in this awareness-raising component and these
relate mainly to the importance parents place on education, on the schooling of girls and on
the fight against early marriage, the three topics which were the primary focus of the
community animator committees.

Whilst these changes are particularly apparent in Minembwe, where a critical mass seems to
have been reached to ensure the values and behaviour promoted by the programme
continue to be communicated and reinforced independently, in Marungu - a more isolated
area and one in which EMI is the first organisation to have carried out this type of
awareness-raising campaign - it is not yet the case.
We recommend that EMI continues their awareness-raising work, by capitalising on the
genuine interest developed by the programme, on the information shortfall which it managed
to address as well as on the active involvement and real commitment of the committees we
met. Furthermore, we recommend that they extend and ensure the sustainability of their work
in particular by developing the use of participatory theatre, by extending the "Club Umuco"
model, by offering monitoring in the field and more permanent and in-depth support to the
animator committees. It may also be worth considering offering limited financial support.

4.5 Research/advocacy

Throughout the programme, EMI/CiC successfully carried out advocacy work with the
central government as well as the decentralised education authorities.
o At a national level, the teacher training modules are currently being validated by
the MoE.
o At a regional level, advocacy work produced significant results: the opening of a
local MoE office for personnel in Minembwe, the decentralisation of the TENAFEP
exams and the creation of two distinct educational authorities in Uvira and Fizi.
o At a local level: EMI/CiC worked with schools and communities to ensure the
mechanisation fees ($150 - $200 [90 - 120] per application) were collected and
the applications sent to the authorities, whilst monitoring the situation in Bukavu
and Kinshasa.
In order to build capacity in this area, in April 2013 EMI/CiC recruited a person to take
specific responsibility for research and advocacy. To date, Patrick Rugabirwa has carried
out research into several areas including initial teacher training for primary teachers
(pedagogical training) and the key barriers to educating children in the High and Mid
Plateau region.
This approach is effective and the results are promising with a view to a third phase to
the programme.

4.6 Summary in light of the criteria relating to schooling and improving pupils'
learning

Having completed this analysis of the various activities implemented by the project, the
evaluation team felt that over and above the changes in practice and situation, noted in
various places, these activities had an impact on two points which are strong indicators of
improvements to the education system in the High and Mid Plateau region: the enrolment
rate of children in primary school (4.6.1) and the TENAFEP pass rate for 6th grade pupils
(4.6.2)
27

4.6.1 Access to education

The evaluators focussed on monitoring school enrolment rates and their development in the
target schools, which were positive as the table below shows:
Enrolment (before) Enrolment (after)
1.1 Marungu HP average (3 non-mechanised primary schools) 186.2 147.5
1.2 Marungu HP average (3 mechanised primary schools) 165 204.2
1.3 Katobo MP average (5 mechanised primary schools) 218.3 197.8
1.4 Fizi HP average (8 mechanised primary schools + 1 private
235.94 268.28
primary school)
OVERALL AVERAGE (weighted) 213.4 222.9

Observations and comments:

From a sample of 20 schools (>10%), we can see a 4.5% (10 pupils) increase in
enrolment (which would equate to 1,680 pupils out of the 176 target schools).
We should first highlight that this connection between the increase in pupil numbers
and the hypothetical increase in the overall enrolment rate is impossible to establish.
Whilst it cannot be ruled out that this increase is the sign of higher enrolment rates in
the region, it is also due to the greater appeal of the schools involved in the
programme and therefore the transfer of pupils between schools.
There were significant increases in the mechanised schools in Minembwe and
Marungu (up 14% and 24%), which goes some way towards indicating the effect of
mechanisation on the appeal of a school (this was not confirmed for the Katobo area).
We can also note a considerable reduction in the drop-out rate in the target schools, taking
all year groups as a whole: an average of 15.9% for the two years before the programme
started and an average of 9% for the two years after the programme launch (see table 11).

4.6.2 The quality of education: the TENAFEP pass rates

EMI and CiC suggested using pass rates for the internal exams organised by schools as a
monitoring indicator, however this is not appropriate given the tests which pupils take in each
school are not identical. The end of primary school state exams (TENAFEP) offer a more
appropriate reference point. The table below show the average pass rates recorded for the
TENAFEP before and after EMI's support in a sample of 20 schools.

Av. pass rate (before) Av. pass rate (after)


1.1 Marungu HP (3 non-mechanised primary
86.80% 92.40%
schools)
1.2 Marungu HP (5 mechanised primary schools) 95.00% 81.30%
1.3 Katobo MP (5 mechanised primary schools) 98.90% 88.80%
1.4 Fizi HP (8 mechanised primary schools) 82.40% 91.30%
OVERALL AVERAGE Weighted 88.60% 89.00%

Observations and comments:

The overall increase in the pass rates remains very limited (up 0.4%) and there is
even a decrease in the mechanised schools in Marungu and Katobo. Stranger still: in
Marungu, the pass rates in the non-mechanised schools are higher than those in the
mechanised schools.
When processing the data collected in the schools, it rapidly became apparent that it
would quite simply not be possible to use this data due to a widespread problem of
28

fraud with the TENAFEP, which, according to several contacts, is the work of the
directors and teachers themselves for the reasons highlighted on page six. For
example: eight of the 20 schools involved in the study have pre-project records
showing scores of 100%, sometimes over several years, which is suspect to say the
least.
In light of this, it seems impossible to use these success rates to evaluate the impact
of the programme on the quality of teaching. Consequently, it is essential for EMI and
CiC to develop an alternative, standard test model in the future for all the schools on
the programme in order to provide reliable data with which to assess and report on
the impact of their work.

5. Project management
In this section we assess the project management, whether the various planned activities
were successfully completed, the respective remits of each of the team members and how
they worked together, the monitoring/evaluation process and the involvement of the EMI/CiC
partnership.

5.1 Management

The projects operational management team worked well together, despite the tragic death of
the Education Manager, Eraste Rwatangabo, and the death of three other members of the
EMI team, in October 2011. Jean Paul Rubyagiza (EMIs current Education Manager
previously part of the original training team) and Amy Parker (CiC Programme Manager)
worked constructively together, enabling the activities to be completed successfully and
adapting them in accordance with the circumstances. This close and considerate cooperation
facilitated the exchange of knowledge and competencies.
The Finance Manager, whose involvement went far beyond what would normally be
expected, was heavily involved in ensuring the smooth-running of the project.
Internal management was of high quality, dynamic and participatory: monthly internal
coordination meetings, quarterly activity planning involving all the trainers and general
debriefings at the end of each of the five annual field missions, an important factor which
made it possible to ensure there was enough communication between members of the two
teams and that their respective tools and approaches were coordinated.
The evaluators observed a strong team spirit amongst the EMI teams, positive leadership,
good working relations, high quality discussions and strategic planning carried out on a
regular basis as a group and, overall, a marked drive to capitalise on lessons learnt.
The only comment to make: the executive secretariat could be more directly involved in the
future, particularly in terms of representation and advocacy, whereas its role was
undoubtedly too restricted to general administrative and financial management, something
which EMI and CiC nevertheless worked on throughout the programme.

5.2 Trainers

The project relied on a stable team of course designers/trainers. Several factors:


their solid understanding of the region's socio-economic and educational context.
their core competencies in education and training. Throughout the project, further training
was regularly delivered in competencies such as using the national curriculum, using
textbooks, course design, participatory theatre, managing qualitative and quantitative
data;
their heavy involvement in the process of improving the quality of education in the project
implementation zone. Trainers' involvement often went far beyond the call of duty;
their courage.
29

The main limitations facing trainers related to logistics and security, due to the fact they were
working in a very isolated, sometimes volatile area, on mountainous terrain, where the
distances between schools are often very considerable. They spent around 105 days per
year in the field - in difficult conditions and not always appropriately equipped - where they
organised two training cycles (August and December), two follow-up missions (October and
February) as well as a visit to identify and assess schools (April). In the future, consideration
will undoubtedly need to be given to inverting their time allocations (currently 1/3 in the field -
2/3 in the office). This would enable a greater presence of the teams on-site - despite the
limitations highlighted - as well as continuous monitoring and supervision of the schools and
the staff trained, which is vital to consolidating learning.
The trainers each expressed their wish to continue the work they had started and to receive
further training in order to do so: in data handling software, course design, managing
development projects, the Congolese education system, etc.

5.3 Working with CiC and developing EMI's capacity

Without taking a directly operational role, and leaving its local partner enough independence
to manage and implement the programme, CiC nevertheless carried out monitoring and
offered intense technical, methodological and strategic support, working closely with the EMI
teams throughout the programme (weekly telephone calls and regular field visits, around four
times a year).
This made it possible to build EMI's capacity slowly but surely in terms of planning,
monitoring, evaluating and reporting.

CiC also acted as a driving force behind the various steps taken by EMI with regard to the
education authorities on a provincial and national level.
This substantially reinforced EMI's exposure and credibility in the sector. Today the
organisation is invited to participate in various strategic workshops in Kinshasa and
Bukavu17. This was not previously the case and it is mainly due to CiC's support.

Capacity building activities were also organised by CiC for some members of the EMI team:
personal security and first aid, strategic planning18, administrative and financial management,
gender and child rights, leadership and management. This training, consolidated by coaching
sessions which EMI has been receiving since the beginning of their partnership with CiC in
2007, has strengthened EMI's technical and organisational capacities, the impact of which is
increasingly evident today.
We should however note that over the past few years CiC has remained EMI's main, if not
sole, partner. Whilst this has been conducive to building very strong working relations, it also
raises the questions of EMI's dependence and relative financial vulnerability.

5.4 Monitoring and Evaluation

The monitoring and evaluation tools and system developed by EMI with technical support
from CiC are relatively advanced and comprehensive: 1) pre- and post-training tests were
used during the main training cycles to assess the degree to which the topics delivered had
been understood and taken on board and 2) annual field visits by all the trainers to the
schools which had received training made it possible to collect the data required to report on

17
In particular, EMI took part in the national workshop on teacher support networks, was invited to Goma by the
DRC Education for the East platform and also participated in the annual provincial pedagogical meetings
organised in Bukavu.
18
At the end of this training course EMI developed a long-term strategic plan, the operational implementation of
which to date still remains limited.
30

the key indicators initially determined in a relatively consistent manner (changes to


pedagogical practice, workforce, drop-out rates, etc.).
We should however highlight the short period of time (10 months) between initial data
collection and the follow-up visits which made it difficult to put the changes recorded and
their consolidation into perspective. To compensate for that, at the end of the programme
EMI and CiC carried out an extensive internal evaluation using a sample of the schools
supported during the programme's early years. The results of this evaluation will be
presented in the final report which is currently being drafted.

5.5 Accounting and financial management

To a certain extent, the evaluators were surprised by CiC's exacting standards in terms of
monitoring and financial reporting, but also by the way in which EMI managed to meet these
standards, which were fully understood and generally accepted, and the application of which
reduced the financial risks significantly. Expenditure forecasts, financial reporting, requests
for funds and transfers were all carried out on a monthly basis, which represented a
considerable workload for EMI's financial department, but one which the accountant, with the
support of a bookkeeper, seemed to be able to manage without any major difficulty. No
significant delays were recorded in terms of transfers or payments.
Whilst this system was laborious to set up initially, EMI now seems accustomed to it and fully
capable of meeting the standards imposed by the main active institutional donors in the
region, particularly in terms of risk management, authorising expenditure, the procurement
process, etc19.

6. Learning Questions: High quality education - the project's contribution


towards social cohesion
Alongside the quantitative and qualitative objectives attributed to the project, the aim of the
programme was also to offer answers to the two questions expressed in section 3 of this
report: What classifies as high quality education in the High and Mid Plateau region? Did the
project contribute towards social cohesion in the area where the programme was
implemented?

In answer to the first question, the analysis and interviews carried out by the evaluators
highlighted several distinctive elements of a high quality education in the region20: pupil
attainment; pupil satisfaction; the level of teaching competencies; a decent salary for
teachers; pupil monitoring by teachers; school facilities; teaching resources;
monitoring/evaluation carried out by education authorities; school management.

With regard to the project's contribution to strengthening social cohesion in the region,
the various activities undertaken had a significant impact. This can be evidenced by: 1)
the various tribal groups living side by side and building constructive working relations
within and around the schools, 2) the overwhelming adoption from all parts of the
community of the principle of mandatory, high quality primary education, 3) the
commitment made by the various churches to challenge customary practices such as
early marriage, marriage by abduction, excluding girls from school, etc.

7. Relevance - effectiveness - efficiency - impact - sustainability

19
These standardised procedures are set out in the administrative and financial procedures manual which EMI
developed with technical support from CiC.
20
These two issues were discussed with the various focus groups (parents, directors and teachers) organised by
the evaluators in Marungu, Katobo and Minembwe. The main conclusions to come out of these discussions are
summarised in table 10 in the appendix.
31

Relevance: The project provided solutions which were appropriate to the specific
educational issues in the High and Mid Plateau region:
o The systemic approach enabled the targeting of key stakeholders involved in
education and facilitated active dialogue;
o The training activities were designed to bring about deep-rooted, sustainable
changes in behaviour and practice.

Effectiveness: Apart from difficulties linked to the security context, to the isolated
nature of the project area and to the tragic death of the head of programme,
almost all the activities were successfully completed.

Efficiency: The high level of management involvement combined with the


relevance of activities and the effectiveness of the management system meant
that the use and output of the allocated funds were optimised.

Impact: The impact of the project could be seen at several levels:


o The changes in practice and behaviour of teachers, school directors, PTAs and
members of the wider community;
o The significant increase in enrolment and completion rates, particularly with
regard to young girls;
o The wider dissemination of the primary national curriculum;
o The increased community awareness, amongst churches in particular, of the need
to put a halt to certain practices which are hindering development;

Given the lack of reliable data for the TENAFEP, it is up to EMI/CiC to design an evaluation
mechanism which is able to offer an accurate assessment of pupil learning for those involved
in the programme compared to those in non-targeted schools.

Sustainability: the sustainability of the project is generally weak and we believe it


is essential to implement a new programme in the region in order to consolidate
learning.
o EMI/CiC's support focuses on short periods (ten months per cohort) and we
observed a certain degree of decline/slip in standards in the schools supported at
the beginning of the programme.
o In some schools there is a very high level of turnover, initial training for new
teachers as well as continuous professional development is still key.
o The almost complete lack of state-run pedagogical supervisory services as well
as most school governing authorities and their inability to provide training and
support for teachers and directors.
o The qualifications of the new generation of teachers are very low due to the
collapse of the initial teacher training system.

8. Recommendations
Focus 1: Professionalising initial teacher training for primary teachers.
Focus 2: Setting up feeder primary schools (reference schools) where trainee teachers
can complete teaching practice.
Focus 3: Building up a network of associated schools around each model school.
Focus 4: Consolidating the networks of associated schools, supervised by the trainers.
Focus 5: The EMI trainers should support and train the directors of the associated
schools in order to implement school projects progressively (to begin with, based on
minimum standards).
Focus 6: EMI should identify and draw up the list of minimum standards.
Focus 7: EMI should increase its research and advocacy work with the aim of developing
expertise in the area of teacher training.
32

Focus 8: EMI should help the Club Umuco roll out its awareness-raising activities
throughout the region.
NB: The graphical representation of the model put forward by the evaluators is presented in
appendix 2.
33

Appendix I: Overview of the programme outcome indicators

The table below presents the internal monitoring data collected by the EMI teams for some of
the key indicators. For the more qualitative changes, we would refer readers to the
observations set out by the evaluators earlier in this report.

Overview of the programme outcome indicators


Outcomes and indicators Figures recorded
Outcome 1: Improving the quality of teaching

34,000 pupils 36,809

177 schools 176

1239 teachers and directors trained 1231

75% of teachers prepare their lessons and reports 81% (CR); 78% (Barings)

70% of pupils pass their end of year exams See EMI/CiC internal evaluation

80% of final year pupils take the TENAFEP 80.8% (CR); 89.7% (Barings)
An 80% increase in the use of A&P teaching An increase of: 79% (CR); 38%
methodology (Barings)
50% of schools organise inter-school activities 88.4% (CR); 86.5% (Barings)

80% of teachers use positive discipline 93% (CR); 77% (Barings)


Outcome 2: Access and increasing enrolment
rates
15% increase in pupils enrolled in partner schools +10.5% (CR); +11.2% (Barings)
Increase in the % of pupils completing the school
+30% (CR); +6.2% (Barings)
year
Increase in the number of girls enrolled +8.9% (CR); +13.8% (Barings)

Increase in the proportion of girls enrolled -0.7% (CR); +1% (Barings)


Outcome 3:the involvement of marginalised
groups in decision-making
At least 2 women members of each PTA (33%) 32% (CR and Barings)

50% of class prefects are girls 34.8% (CR); 30.1% (Barings)


Outcome 4: Local bodies support the schools
(exit strategy)
75% of teachers/parents report an improvement in
See evaluation in point 4.3.3
school-community interaction
In 50% of schools: infrastructure projects
72% (CR); 51% (Barings)
supported by parents
Outcome 5: capacity building for EMI
See point 5
(implementation and advocacy)
34

Appendix II: Compiling quantitative data


Table 1: the modules, key competencies and duration of the training

Modules Key competencies No. Days

Making use of the national curriculum


Preparing a teaching and learning session
General pedagogy 5
Delivering a teaching session
Evaluating a teaching and learning session
Subject-specific Preparing, leading and delivering a lesson at primary
5
pedagogy school, according to the year group and class
Positive discipline Implementing positive discipline in the classroom 1

Child rights Ensuring teaching practice is in line with child rights 1.5

Ensuring behaviour is in line with the professional


Professional ethics 0.5
ethics of teaching
Identifying children suffering from psychological
problems
Psychosocial education 1.5
Offering support to children suffering from
psychological problems

Peace education Ensuring teaching practice is in line with child rights 2

Reusing our knowledge to prepare, implement and


evaluate a lesson on health issues at school and in the
Health and environment
community 4
education
Encourage the appropriate attitudes and behaviour to
promote health and the environment

Table 2: teachers, directors and the beneficiary schools

Figures
I. BARINGS Planned
recorded
Number of teachers trained 315 313
Number of teachers given refresher training 315 312
Number of directors trained 105 104
II. COMIC RELIEF

Number of teachers trained 430 430


Number of teachers given refresher training N/A N/A
Number of directors trained 72 72
Number of target schools 72 72
GRAND TOTAL 1 (target schools) 177 176
GRAND TOTAL 2 (teachers and directors given training/refresher
1237 1231
training)

Table 3: schools' financial means and capacities

Annual average budget for schools in $US


35

Operating Fees Average


Salaries
Budget + (paid by TOTAL monthly
(state)
PIRUS. parents) salary
Mechanised primary school
5833 606 2304 8743.3 93.3
budget (Uvira)
Mechanised primary school
5833 606 6581 13020.2 103.3
budget (Fizi)
Non-mechanised primary
0 0 1904.5 1904.5 27.2
school budget (Uvira)
NB1: the amount of fees paid by parents in Fizi (Minembwe) is higher due to the fact that parents are
more aware and take a more active role to support their children's education in this area (which is less
isolated and has benefited from a greater number of programmes and awareness-raising campaigns
over the past few years, etc.).
NB2: PIRUS = support from the World Bank towards running of schools.

Table 4: monitoring turnover in the schools trained by EMI

Turnover within primary schools


Teachers trained Teachers left %
Marungu HP 6 2.9 48%
Katobo MP 6 1.4 23%
Average Uvira MP and HP 6 2.22 37%
Mechanised average 6 1 17%
Non-Mechanised average 6 3.84 64%
Turnover for Barings component (Uvira and Fizi)
2011-2012 259 83 32%
2012-2013 245 79 32%
2013-2014 224 67 30%
Total (Barings) 728 229 31%

Table 5: awareness-raising component (the message and the reasoning)

Ten good reasons to educate girls, the women of the future


1. An educated woman understands her rights and responsibilities.
2. An educated woman offers the best upbringing to her children.
3. An educated woman offers better protection to her children, supervises and advises them in life.
4. An educated woman organises her own life and her home life better.
5. An educated woman takes useful decisions for her family and supports her husband.
6. An educated woman manages the familys assets properly.
7. An educated woman looks after her own health and that of her children.
8. An educated woman is an organised worker who produces good results.
9. An educated woman knows how to choose economical activities that improve her situation and
that of her family.
10. An educated woman is a citizen who takes an active role in the development of her village, her
community and her country.
36

Table 6: Summary of the focus group work (awareness-raising component)

Parents and members of the PTAs (Marungu & Katobo, Uvira area)
Issue 1: Education for girls
Initial situation and obstacles Current situation (changes)
Girls were responsible for all household chores Girls are considered as equals to their brothers
(washing, collecting wood, water, etc.) (schooling and household chores)
The status of girls has improved / parents
understand that an educated girl can offer
Girls had to stay at home in the morning to look considerable support to her family ("after
after the babies completing their studies, girls give more
consideration to their parents and can support
them")
Girls had to stay with their mothers and help Girls speak out more in groups and in the
them work in the fields presence of their parents
They were directed towards marriage very young
so that the family could get the dowry (money
and/or cows)
Whilst there has been a reported case of a
They were not entitled to inherit woman who recently inherited, these remain
exceptional
Issue 2: the fight against early marriage
Initial situation and obstacles Current situation (changes)
People are now aware of and widely abide by the
A girl could be married off from the age of 13 or
legal age to marry (18 years old). Girls' right to
14, and her opinion would not be taken into
choose is more widely recognised: choice of
account
husband and choice of schooling v marriage
Girls were seen as a source of wealth, whatever Signature of a statement of commitment by the
their age ("We didn't consider the age, as long as religious leaders of the various churches on the
we got the money or the cows for the dowry") Uvira High Plateau
Cases of parents being excommunicated for
having married off their children/underage girls
Change in mentality observed in young girls: their
priority is education and completing their
schooling
Cases of pregnant girls who return to school after
having given birth (which did not happen
previously)
Table 7: monitoring the development in fees paid by parents

Monitoring the development in fees


Fees Fees
Difference Change (%)
(before)* (after)
HP & MP average (4 non-mechanised
1183.0 1166.0 -16.7 6%
primary schools)
HP & MP average (8 mechanised
1007 1121 114 11%
primary schools)
Average for Fizi High Plateau 1867.0 2630 763.0 47%

*data was collected over the two years prior to EMI/CiC's programme ("before") as well as over the
two years following the programme ("after").

Table 8: monitoring the proportion of girls enrolled in school


37

Monitoring the proportion of girls enrolled in school (in 20 target schools)


% girls
% girls (after)
(before)
I. Uvira area (Averages for Katobo Plateau & Marungu High Plateau

1.1 Marungu HP average (3 non-mechanised primary schools) 36% 37%

1.2 Marungu HP average (3 mechanised primary schools) 50.0% 49.7%

1.3 Katobo MP average (5 mechanised primary schools) 40.8% 45.9%

Weighted average for the Uvira area (MP & HP) 42.0% 45.0%

II. Fizi area (Minembwe High Plateau)

Minembwe HP average (8 mechanised primary schools + 1


49.1% 52.5%
private primary school)

OVERALL AVERAGE 45.2% 48.2%


*the average figures have been calculated for the two years prior to the programme and the two years
following the programme

Table 9: Monitoring workforce and drop-out rate for girls (final 2 years of primary school)

Monitoring workforce and drop-out rate for girls in 5th and 6th grade
% of 5th Drop-out rate, Drop-out rate,
% of 5th and
and 6th 5th and 6th 5th and 6th
6th grade
grade girls grade girls grade girls
girls (before)
(after) (before) (after)
Uvira HP & MP average (non-
23.5% 22.8%
mechanised)
Uvira MP & HP average
42.1% 46.5% 14.7% 10.6%
(mechanised)
Average for Fizi High Plateau
48.1% 48.0% 12.5% 5.7%
(mechanised)
OVERALL AVERAGE 42.7% 43.3% 13.7% 8.3%

Table 10: Learning Questions (conclusions from the focus groups)

Learning Question 1: The programme's contribution to social cohesion


Observations and significant changes
In the schools, children from different tribal groups play together (games organised during break time)

Mixed marriages between Mufulero and Munyamulenge are increasingly common


Diversity is increasing in the schools targeted by the project, children work together, eat together, sit
next to each other, without making distinctions
Friendships have been formed between the teachers trained by EMI ("now I can spend the night in a
Bafulero village, at my friend's house, in complete confidence")
Banyamulenge teachers work in majority-Bafulero schools and vice-versa
Religious leaders from the various churches on the High Plateau (Uvira) have agreed on a statement
of commitment (fight against early marriage) which they have all signed
The EMI schools are built without any distinction for the benefit of children from all the tribal groups
38

The various tribal groups are represented within the PTAs and awareness-raising committees:
members work constructively and get on well together
Learning Question 2: The necessary conditions for high quality education
The criteria (in order of priority)
Competent, qualified and motivated teachers ("passionate about their vocation")

Sustainable, high quality facilities

Discipline is maintained within the school (positive discipline)

The PTAs and parents are actively involved in their children's education as well as in the
management of schools. Regular dialogue between directors, teachers and parents

Teachers and directors keep to their schedule and arrive on time.

Teachers have the required textbooks and teaching resources

The school environment is hygienic and clean

Teachers give regular homework to pupils


Pupils can express themselves well, quickly understand questions they are asked and get good
results

Table 11: monitoring the number of pupils dropping out (all year groups taken together)

Monitoring the drop-out rate (1st - 6th grade)


Drop-out rate (before) Drop-out rate (after)
Uvira HP & MP average (6 mechanised
16.4% 10.1%
primary schools)
Fizi HP average (8 mechanised primary
15.3% 8.3%
schools)
OVERALL AVERAGE 15.9% 9.0%
39

Appendix III: Graphical representation of the recommendation

IP: Pedagogical secondary school (institute)


EP: Reference primary schools
EP: Associated primary school
R: Teacher support network
40

Appendix IV: List of the people interviewed and the schools visited
during the evaluation mission

Organisation Person interviewed


EMI - all of the trainers
- the programme manager, Jean-Paul
Rubyagiza
- the head of finance, Gratien Sebagabo
- the head of research/advocacy, Patrick
Rubabirwa
- the executive secretary, Pastor Samson
Muvunyi
CIC The programme manager, Amy Parker
AVSI (International NGO the Association The RRMP (Rapid Response to Population
of Volunteers in International Service) Movement) programme manager
MoE - The Uvira territorial inspector, Mr
Musungela
- The head of the SECOPE branch in
Minembwe, Mr Bizimana
- the EPSP Uvira II local education authority
CEPAC - the head of training, Mr Sobanuka

LIST OF SCHOOLS VISITED

AREA SCHOOL VISITED AND INVITED


MINEMBWE Schools visited
- MUFARIJI primary school
- LUBEMBA primary school
- KALINGI primary school
- JUGENGE primary school

Schools involved in the workshop on


17/05/2014
- CHAKALA primary school
- NGOBI primary school
- RUMUNDU primary school
- ILUNDU primary school
- MUZINDA primary school
- MAMBO primary school
- KITAVI primary school
- KITABI primary school

MARUNGU Schools visited


- Murambi primary school
- Kahololo primary school
- Bipimo primary school
- Rudefwe primary school (non-targeted)

Schools involved in the workshop on


17/05/2014

- Marungu primary school


41

- Rudefu primary school


- Murambi primary school
- Kahololo primary school
- Kitembe primary school
- Kikabi primary school
- Kageregere primary school
KATOBO Schools involved in the workshop on
21/05/2014

- Kakuba primary school


- Kivibwe primary school
- Giti primary school
- Katembo primary school
- Mashuba primary school
- Mutumba primary school
LEMERA One primary school visited by both
consultants on 14/05/2014

Participants
Men: 28
Women: 83
Grand total: 111

EXTERNAL EVALUATION WORKSHOP / MARUNGU


PARTICIPANTS: PTA MEMBERS Date: 17/05/2014
No. Surname School Gender
1 Munyamahoro Masenga Kitembe primary school M
2 Rukeba Amos Marungu primary school M
3 Muzingwa Naluja Murambi primary school M
4 Mugendererwa Ezekiel Bipimo primary school M
5 Adele Kangombe Rudefu primary school F
6 Nabeza Nahoza Kitembe primary school F
7 Nabihumbi Mvungano Nalubira primary school F
8 Bukuru Damari Kagerigeri primary school F
9 Rwiyamirira Evariste Kahololo primary school M
10 Nyaganza Ana Kitembe primary school F
11 Kesiya Nandagirwa Marungu primary school F
12 Kineri Jacqueline Kahololo primary school F
13 Sada Narwaka Marungu primary school F
14 Kwangaba Magendane Kitembe primary school F
15 Sephanie Mukulikire Rudefu primary school M
16 Mbindo Mukuze Kagerigeri primary school M
17 Serukiza Joel Kahololo primary school M

PARTICIPANTS: DIRECTORS AND TEACHERS Date: 17/05/2014


No. Surname School Job title Gender
1 Bukuru Sepa Milimba primary school Teacher M
2 Gilbert Munagana Kagerigeri primary school Director M
3 Mapaya Malekuza Rudefu primary school Director M
4 Kishoshe Ngoy Murambi primary school Director M
5 Ntezekumana Mukenga Kahololo primary school Director M
6 Imani Muragara Kigabi II primary school Teacher M
7 Rugondera Thomas Kahololo primary school Teacher M
42

8 Safari Basimike Kagerigeri primary school Teacher M


9 Mushangwa Madaho Rudefu primary school Teacher M
10 Bonne anne Assani Marungu primary school Teacher M
11 Mezake Muganwa Milimba primary school Teacher M
12 Mufariji Muheto Kigabi II primary school Teacher M
13 Byamungu Amon Marungu primary school Director M
14 Byamungu Paul Rudefu primary school Teacher M

EXTERNAL EVALUATION WORKSHOP / GITIGARWA


PARTICIPANTS: PTA MEMBERS Date: 21/05/2014
No. Surname School Gender
1 Mazaro Rugenuza Kakuba primary school M
2 Luwi Musindiro Katembo primary school F
3 Nobe Musholeza Kakuba primary school F
4 Kabobwe Mulindiro Kakuba primary school M
5 Mbwine Rukarira Primary school M
6 Tomasi Mbakanyi Primary school F
7 Tabaro Mutabwe Butumba primary school F
8 Anjela Sangiko Ndegu primary school F
9 Mashozi Anaresa Katembo primary school F
10 Ndabitwa Kivibwe primary school F
11 Oseya Tawimbi Kivibwe primary school M
12 Rugimbana Giti primary school M
13 Rutaha Zebede Giti primary school M
14 Tomaso Malebeka Mashuba primary school M
15 Muhivwa Shadrake Kivibwe primary school M
16 Matyasi Kirebe Butumba primary school M
17 Kaloto Ruzi Mulenge primary school M
18 Nyamagajo Nyangabire Mashuba primary school F
19 Zuwena Njinginya Kivibwe primary school F
20 Bizimana Nyirimuhanga Giti primary school M

PARTICIPANTS: DIRECTORS AND TEACHERS Date: 21/05/2014


No. Surname School Role Gender
1 Kategere Musindiro Katembo primary school Teacher M
2 Aimable Mugomberwa Kivibwe primary school Teacher M
3 Kiruhura Claude Giti primary school Director M
4 Kihingira Barondere Butumba primary school Teacher M
5 Kalala Muguzi Kakuba primary school Teacher M
6 Musesimo Lembusa Kivibwe primary school Director M
7 Nandeya Runegeka Katembo primary school Director M
8 Mwimule Majagura Butumba primary school Director M
9 Ndaluba Marundu Mashuba primary school Director M
10 Byamasu Mukera Mashuba primary school Teacher M
11 Niyibizi Muhoza Giti primary school Teacher M
12 Mbata Ndasakuza Kakuba primary school Teacher M

EXTERNAL EVALUATION WORKSHOP / MINEMBWE


PARTICIPANTS: PTA MEMBERS Date: 17/05/2014
No. Surname Gender School
1 Kahoja Bosco M Kibati
2 Yakobo Sekarori M Chakila
3 Ntwari Rutekereza M Muzinda
43

4 Muzingwa Sabune M Ngobi


5 Sedoho Ruheka M Mambo
6 Mariko Rwambuguza M Chakila
7 Sembari Eraste M Runundu
8 Ntimugura Rwihangana M Mambo
9 Ngendahayo Mukenge M Ngobi
10 Bukuru Kaboyi M Ilundu
11 Ramaraninda Nfanshingabo M Mambo
12 Sadori Buraga M Chakila
13 Nantore Naganza F Muzinda
14 Chakupewa Elizabeth F Ngobi
15 Binja Aimee F Kibati
16 Nantebuka Namuteto F Chakila
17 Nandegba Jeanne F Runundu
18 Sada Nashuramo F Ilundu
19 Esteri Ana F Mambo
20 Ruganza Benoit M Runundu
21 Mazuru Ruhorimbere M Chakila
22 Bukubure Mezake M Muzinda
23 Kamundara Rutikanga M Chakila
24 Madaki Serieux M Kibati
25 Bukuru Semaharo M Ilundu
26 Namahoro Angelique F Kitavi

PARTICIPANTS: DIRECTORS AND TEACHERS Date: 17/05/2014


No. Surname Gender School
1 Ndayibona Ephraim M Chakila primary school
2 Irakiza Gihura M Chakila primary school
3 Gabo Mubangura M Chakila primary school
4 Mugaza Kinyoni M Chakila primary school
5 Nyarugabo Joseph M Chakila primary school
6 Sebatware Charles M Ilundu primary school
7 Ndosimana Felix M Ilundu primary school
8 Bitakwira Namigaba M Ngovi primary school
9 Byichaza Rumara M Ngovi primary school
10 Magambo Mbwete M Ngovi primary school
11 Mushika Ruben M Mambo primary school
12 Runezerwa Pave M Mambo primary school
13 Mihingano Musirikani M Runundu primary school
14 Semuhanuka Ngendahayo M Runundu primary school
15 Kofita Ruhindiza M Kibati primary school
16 Abahenya Mbehumo M Kibati primary school
17 Mutima Gentil M Kitavi primary school
18 Namahoro Soleil F Kitavi primary school
19 Rugazura Mathias M Muzinda primary school
20 Rugizama Ntwari M Muzinda primary school
21 Mutebutsi Dieudonne M Minembwe primary school
22 Nandorimana Nadiama F Minembwe primary school

NUMBER OF PEOPLE MET/INTERVIEWED

1) In the schools visited:


44

Bilateral interview with the director


Group interview with the 6 teachers
3 lesson observations and interview with the pupils in the lessons observed
(between 10 and 30 pupils)

2) With the schools involved in the focus groups

Between 6 and 8 schools involved in each of the 3 workshops


Two representatives per school (the director + one teacher).

3) The focus groups with parents and members of the PTA

Groups of 15-20 people who met during the 3 workshops (around 50% women).
45

Appendix V: Evaluation programme

Date Activity
11/05 Arrival
Journey from Bujumbura Uvira
Planning
12/05 Workshop with the trainers
13/05 AM: Workshop with the trainers
PM: Workshop with the awareness-raising team
14/05 AM: Workshop with the trainers
PM: Interview with research and advocacy
15/05 Team A: Team B:
Journey to Minembwe Journey to Marungu
16/05 Mufariji primary school (CR) (lesson Primary school (Barings)
observations, interviews with director, (idem)
parents, teachers, pupils, PTA)
17/05 Workshop at Mishashu primary school Workshop at Bijojo primary school
(Appendix IV): (Appendix IV):
AM: Teachers, directors AM: Teachers, directors
PM: PTA, leaders PM: PTA, leaders
18/05 Church - awareness-raising in action Church - awareness-raising in action
19/05 Rubemba primary school (non-targeted) Ngogwe primary school (non-targeted)
(lesson observations, interviews with (idem)
director, parents, teachers, pupils, PTA)
20/05 Kalingi primary school (Barings) Kahololo primary school (Barings)
(idem) (idem)
21/05 Evomi/Tujenge primary school (CR) Journey to Gitigarwa
Interview: Samson; SECOPE Workshop (Appendix IV)
Journey to Uvira
22/05 Journey to Uvira Visit to S/PROVED II Uvira
23/05 Interviews: CEPAC, Sebagabo, Rubyagiza, Amy
24/05 AM: Feedback
PM: End