NATIONAL EDUCATIONAL ASSESSMENT AND EXAMINATIONS AGENCY

Policy Framework for National

Large-scale Assessment in Ethiopia

National Educational Assessment and Examinations Agency ©

7/1/2012
Table of Contents
1. Background ............................................................................................................... 2
2. Purposes of the National Large-scale Assessment in Ethiopia ........................ 3
2.1 Main Purposes ................................................................................................... 3
2.2 Other Purposes .................................................................................................. 3
3. Vision and Mission statements for NLSA ............................................................. 3
3.1 Vision ................................................................................................................... 3
3.2 Mission ................................................................................................................ 3
3.3 General Objective .............................................................................................. 4
3.4 Specific objectives ............................................................................................. 4
4. Principles for NLSA .................................................................................................. 4
• Validity..................................................................................................................... 5
• Reliability ................................................................................................................ 5
• Efficiency ................................................................................................................ 5
• Utility ........................................................................................................................ 5
• Defensibility ............................................................................................................ 5
• Explicitness ............................................................................................................ 5
5. Rules/guidelines for NLSA implementation .......................................................... 6
5.1 Who should provide policy guidance?............................................................ 6
5.2 Who should carry out the National Large-scale Assessment? .................. 6
5.3 Grade/population and Subject coverage for NLSA ...................................... 7
5.3.1 Grade/population coverage ...................................................................... 7
5.3.2 Subject/content coverage ......................................................................... 7
5.4 The frequency of NLSAs .................................................................................. 7
5.5 The issue of responsibility for implementation of the findings NLSA ........ 8
5.5.1 Responsibility of policy makers or MOE ................................................. 8
5.5.2 Responsibility of NEAEA ........................................................................... 9
5.5.3 Responsibility of Curriculum developers .............................................. 10
5.5.4 Responsibility of Regional Education Bureaus.................................... 11
5.5.5 Responsibility of Zonal and Woreda Education Bureaus .................. 11
5.5.6 Responsibility of teachers ....................................................................... 12
5.5.7 Responsibility of Schools and School managers ................................ 12
5.5.8 Responsibility for monitoring the implementation ............................... 12
5.5.9 Users of NLSA information ..................................................................... 13

1
 ETHIOPIAN MINISTRY OF EDUCATION
NATIONAL EDUCATIONAL ASSESSMENT AND
EXAMINATIONS AGENCY
National Large-scale Assessment Policy Framework
July 2012
1. Background

The efficiency of any system is measured through a set of performance

indicators or outcomes. There are a number of performance indicators for an
education system. Perhaps the most important is the learning achievements
of a student in the system. Measurement of the learning achievements of
students can be done using various types of assessment instruments, data
from which can be used for different purposes. Depending upon their
purposes, assessments can be designed and conducted locally, regionally,
nationally or even internationally.

There is an increased interest in the role large-scale assessment programs

can play as agents of education reform and accountability. More and more
education systems around the world have adopted national or system-wide
testing programs to gather data that will provide an indication of the overall
“health” of the education system. System-wide testing has become an
important feature of the quality assurance frameworks of education systems
throughout Africa and other countries such as Australia, China, Jordan, the
Philippines, Singapore, the United Arab Emirates, the UK and the US.

In Ethiopia, a large-scale assessment program has been in place since 2000.

Assessment exercises have been conducted at the national level at exit
grades of the general primary and secondary education. The National Large-
scale Assessment (NLSA), as it is known, is administered uniformly to
students across the country and serves as a common metric for determining
students’ achievement across the country.

2
2. Purposes of the National Large-scale Assessment in Ethiopia

2.1 Main Purposes

2.1.1 To raise educational standards;
2.1.2 To provide information that can be used to serve for decisions
about the allocation of resources;
2.1.3 To obtain information that can be used to assign accountability for
student performance;
2.1.4 To compensate for the weak assessment practices of school
teachers;

2.2 Other Purposes

2.2.1 To alter the balance of control in an educational system. It may be
used to ensure what is taught in schools is less dependent on the
professional judgment of teachers, and more dependent on
central authorities that mandated the assessment;
2.2.2 To serve as part of modernization movement;

3. Vision and Mission Statements for NLSA

3.1 Vision
Raising students’ achievement at all levels of general education,
primarily by providing relevant data to policy makers & key stakeholders
as a way to help them make better decisions to enhance the quality of
education.

3.2 Mission
Assessing students’ achievement levels in order to check and help
improve the quality of education.

3
 3.3 General Objective

To provide information on the health of the education system to help

support sound policy decisions for reform, curriculum revision and other
possible interventions.

3.4 Specific Objectives

3.4.1 Providing relevant and factual information to assist policy making,
informed decisions on curricula, resource provision, and teacher
training;
3.4.2 Identifying correlates of achievement;
3.4.3 Providing guidance for teachers to assist them in raising students’
performance levels;
3.4.4 Promoting responsibility at different levels; the findings of the
NLSA should enable policy makers, educators, Regional
Education Bureaus, Zonal Education offices, Woreda Education
offices, schools, teachers, parents, students and other
stakeholders to share accountability and responsibility for
improvement of the quality of education;
3.4.5 Raising public awareness; the timely dissemination of the findings
of the NLSA will increase public awareness of education issues;
3.4.6 Monitoring standards; by collecting regular information on
students’ performance, it should become easier to maintain
standards of learning at each level.

4. Principles for NLSA

The undertaking of the national large-scale assessment exercise in Ethiopia

is guided by the following underpinning principles:

4
• Validity

NLSA tasks will assess what is intended to be assessed and be fit for
purpose. The learning outcomes that emanate from the curriculum
documents cover knowledge/ understanding, cognitive skills, practical
skills and personal/transferable skills. The NLSA must cover all these
objectives, and it must be clear what a given NLSA task covers and that it
is appropriately designed to do so.

• Reliability

NLSA processes and procedures will be accurate, consistent and

trustworthy and their broad outcomes will have the full confidence of all
parties.

• Efficiency

NLSA needs to be efficient, i.e. it needs to be cost effective as well as

manageable.
• Utility

NLSA needs to make a sound impact on students’ achievement via which

to enhance education quality by ensuring validity, dependability,
acceptability, feasibility and cost effectiveness.

• Defensibility
NLSA tasks, processes and procedures as well as its outputs need to be
technically qualified and convincing to all stakeholders.

• Explicitness
NLSA tasks, processes and procedures need to be accessible, clearly
explained to and understood by all stakeholders including prospective and
current students, funding agents, academic and related staff, professional
bodies and employers.

5
5. Rules/Guidelines for NLSA Implementation

5.1 Who should provide policy guidance?

The Ministry of Education (MoE) should appoint a national steering

committee to provide overall guidance to the National Educational
Assessment and Examinations Agency (NEAEA) or other team that will
carry out the assessment. The committee can ensure that the national
assessment has status and those key policy questions of interest to the
ministry and others are addressed. The steering committee can also
resolve serious financial and administrative issues.
The composition of the national steering committee varies from country
to country depending upon the power structure in the education system.
In Ethiopia, in addition to representatives of the Ministry of Education,
the national steering committee might include representatives from
regional government education bureaus, school teachers, colleges of
teacher education, and curriculum personnel.

5.2 Who should carry out the National Large-scale Assessment?

The National Educational Assessment and Examinations Agency is

responsible for conducting national large-scale assessments in Ethiopia.
The Agency should have high-level implementation and operational skills
and is expected to carefully design the assessment process and
facilitate the development of a test blue print, drafting items, and
selection of items after pretesting or piloting.

The Agency is also expected to have statistical expertise and analytical

competences in selecting samples, weighting data, data input and file
preparation, item analysis of test data as well as general statistical
analysis of the overall results. Also, the directorate should have the
capacity to draft and disseminate results.

6
5.3 Grade/population and subject coverage for NLSA

5.3.1 Grade/population coverage

Target grades for national assessment vary from country to country. The
grades or population to be assessed should normally be dictated by the
information needs of the ministry of education. Thus, the responsibility
for determining and selecting the population to be assessed resides with
the MOE or NEAEA. To date, NLSAs have been carried out at the exit
levels of each cycle (i.e. at grades 4, 8, 10 & 12). Alternatively these
grades may be substituted by a student population of certain year-olds
(11, 15, 17 or 19 year-olds).

5.3.2 Subject/content coverage

The subject or content coverage for 4th grade NLSA include Mother
Tongue Reading, English, Mathematics and Environmental Science,
while that of grades 8, 10 and 12 comprises English, Mathematics,
Physics, Chemistry and Biology. Provided that this subject-based nature
of content coverage is clearly justified by the existing curriculum policy
settings, it will be replaced by tests of basic skills (literacy and
numeracy) that would yield useful information on skills fundamental to
learning progress.

5.4 The frequency of NLSAs

The frequency with which a national large-scale assessment is carried

out varies from country to country. It is the purpose of the assessment
which determines how frequently the assessment is to be carried out. In
the Ethiopian context, NLSAs will be carried out every four years for
every cycle. A four-year interval for each cycle provides the chance for
school interventions to reveal their impact as education systems do not
change rapidly, and for the thorough preparation of subsequent NLSA
cycles.

7
5.5 The issue of responsibility for implementing the findings of NLSA

A key factor is the use of the findings of the NLSA for promoting the
quality of general education. If national assessments are to be the main
vehicle for raising standards, it is essential that assessment should be
done well, according to best practice technically, employing open and
transparent practices administratively, and providing to the regions and
schools comprehensive information about learning revealed by the test
data. These essential aspects require acceptance of, and adherence to,
the notion of accountability at all levels–federal, regional, zonal, woredas
and schools. The notion of accountability relates to accepting and
demonstrating responsibility for actions at all levels that are designed to
improve the quality of learning.

Thus, responsibility is shared by different stakeholders in order to make

use of the findings of NLSA. Responsibility is shared by policy makers,
administrators, education managers of schools, curriculum developers,
teacher training institutions, text book developers and publishers, test
developers, assessment technical personnel, teachers, students,
parents, and other stakeholders who have a direct or indirect
involvement in the education systems.

5.5.1 Responsibility of policymakers or MOE

The ministry of education and other key stakeholders should carefully

consider the findings of the NLSA so that they can devise specific
policies and actions to address areas of deficiency in the education
system.

8
5.5.2 Responsibility of NEAEA

The NEAEA shares accountability with respect to the quality of the

assessment instruments as well as the timely dissemination of the
research findings. Standardized tools are of paramount importance for
effective assessment so as to measure the achievement of students.
Test items should have curricular relevance, cognitive complexity,
linguistic appropriateness etc. Thus, a test should not be restricted to
measuring knowledge that requires students to simply recall facts or
information. But rather, it should measure higher level cognitive skills
(involving reasoning, the ability to identify and solve problems). Proper
administration of the tests, developing reports of the NLSA findings as
well as the timely dissemination of the findings to stakeholders at all
levels are important activities which may also be the responsibility of the
NEAEA (in particular NEAD).

With respect to technical quality, it should be the primary responsibility of

the NEAD to investigate and control the quality of the assessment
instruments. To control the quality of the research instruments, test
score validity, reliability and fairness should be carefully investigated.

The following features should be taken into account in developing test

items:
i. A test should contain an adequate number of items. This is to
secure adequate representation of a domain.
ii. A test should assess knowledge and skills at a level that is
appropriate for the students.
iii. A test should provide valid information on students’ knowledge,
skills and attitudes in a particular curriculum domain.
iv. The assessment results should be used to monitor changes
overtime, the assessment instruments must be comparable.

9
The results of the NLSA should be reported and disseminated in a
timely manner as soon as data analysis has been completed. Thus,
NEAEA should allocate adequate budgets for report preparation and
dissemination of the research findings.

It is the responsibility of the NEAD to prepare a draft report for the

ministers, the NEAEA board and senior officials in the MoE. The main
findings of an assessment should be prepared in a concise form and
the possible implications of those findings should be indicated. After
briefing ministers and senior officials, a summary report for a non-
technical readership should be prepared in addition to a main report.

It is of paramount importance to involve key stakeholders in a

discussion and interpretation of the findings of the NLSA. The
implications of the findings should be carefully communicated so that
experts and key stakeholders are able to propose and suggest
appropriate approaches to teaching and learning processes to address
the identified problems. The NEAD should draft a brief report on the
implications of the findings of the assessment for curriculum revision
and discuss it with the curriculum Development and Implementation
Directorate of the MoE.

Technical reports are crucial for any national large-scale assessment

because research and scientific communities require detailed
information about the assessment. Technical reports also act as a
record of the activities involved in the assessment, which is needed to
implement future cycles of an assessment.

5.5.3 Responsibility of curriculum developers

As the findings of a national assessment can provide information

relevant to curriculum implementation or reform, curriculum developers
are required to use the findings of NLSA for curriculum revision.

10
Thus, curriculum developers at federal and regional levels should take
responsibility for developing relevant, reliable, valid, simplified, updated
curricular materials by taking those points of recommendation from
NLSA findings. There must be a regular communication channel
between curriculum developers and the assessment team.

The value of national assessment for curriculum analysis and revision

is likely to be enhanced if curriculum developers are involved in
identifying the precise skills to be assessed. The use of national
assessment findings for curriculum revision will also be facilitated if
curriculum developers are involved in defining proficiency levels and in
specifying the kind of data analysis that would facilitate their task.

5.5.4 Responsibility of Regional Education Bureaus

Regional Education Bureaus share responsibility in facilitating NLSA

programs in collaboration with the NEAEA. The involvement of
Regional Education Bureaus as key stakeholders in NLSA programs
is of paramount importance for effective and timely school intervention
programs.

5.5.5 Responsibility of Zonal and Woreda Education Bureaus

Like Regional Educational Bureaus, Zonal and Woreda educational

offices are also responsible for creating enabling conditions for NLSA
programs to be implemented. The recommendations of NLSA findings
for school intervention programs may be implemented at Zonal,
Woreda and School levels. Thus, Zonal and Woreda education offices
ought to share responsibility in implementing the findings of the NLSA.
Provision of training for teachers on pedagogical aspects, teaching
methodology, allocation of resources to schools, timely monitoring of
the teaching –learning process etc. are some of the responsibilities to
be shared by Zonal and Woreda education offices.

11
 5.5.6 Responsibility of teachers

Teachers and students bear some responsibility for teaching-learning

processes. It would be reasonable that they should be held
responsible only for matters over which they have control. Teachers
are highly responsible for the achievements of their students provided
that all the school’s facilities are fully or mostly fulfilled. Not only
school facilities but also the content of the curriculum, the quality of
teaching materials etc. are of paramount importance for determining
the achievements of students.

5.5.7 Responsibility of schools and school managers

Schools and school managers are key stakeholders to make use of

the findings of the NLSA. Appropriate interventions for promoting the
quality of education begin at the school level. The commitment of
school managers, school teachers, and other members of school
communities for quality education deserves much attention.
Thus, as long as schools share greater responsibility in respect to the
quality of teaching they provide, the findings of the NLSA should be
appropriately communicated to schools so that school communities
are able to act accordingly to raise the achievement of students.

5.5.8 Responsibility for monitoring the implementation

The NEAEA needs to carefully monitor and document the operation

of the system with a view to improvement. The Agency should be
able to investigate whether or not the findings of the NLSA and the
recommendations put forward are appropriately utilized for school
interventions to aid the raising of student achievement.

12
 5.5.9 Users of NLSA information
Policy makers: need to know whether the education system is
providing an appropriate environment for student learning in order
to support those areas in need of improvement.
Public/community: need to know how well the education
system is functioning.
Funding agencies: the information provided from the NLSA
may help funding agencies to make decisions about what to
fund based on the needs and relative effectiveness of
educational programs.
Curriculum developers: curriculum developers also need to
know whether the intended objectives are attained or not.

Table 1. Concerned bodies with primary responsibility for decisions

regarding the National Large-scale Assessment.
Primary responsibility
Ministry National Educational National Regions or
of Assessment & Assessment other
Decisions Education Examinations Agency Team stakeholders
*
Giving policy guidance
Estimating and deciding costs * * *
Carrying out National Large- * *
Scale Assessment
Determining the frequency of * * *
assessment
Deciding on populations to be *
assessed
Deciding what to be assessed * * *
Administering tests and * *
questionnaires
Deciding methods of reporting *
Determining statistical *
procedures
Identifying methods of * * * *
communicating and
disseminating results
Evaluating the extent to which * * *
the results of the NLSA are
utilized in raising students’
achievement

13