UNESCO EMIS Readiness Assessment Tool for Transformation

Guiding Note

I. Global Context

Education sector management refers to the administration of the education system and encompasses the
practical measures to ensure the education system efficiently achieves its development goals and objectives.
Its overall goal is to create and maintain environments within education administration and educational
institutions that promote, support, and sustain teaching and learning effectively and efficiently. Within this
scope, education management involves setting directions, establishing and enhancing standards, ensuring
program implementation, optimizing resource organization (human, time, materials), monitoring
implementation progress, and facilitating reporting procedures.

To provide quality education, ministries need to ensure availability of relevant educational services for learners
on a day-to-day basis. This requires efficient institutional and organizational arrangements for mobilization of
resources such as teachers, educational infrastructures, materials, and equipment, as well as strong capacities
to organize and manage these resources in an optimal and cost-effective manner while ensuring that delivery
of education services translates into quality learning outcomes for learners. Inadequate management capacity
at various levels of ministries of education and in schools can hinder the success of education sector reforms
and plans and the delivery of quality education. Many countries face difficulties in using resources efficiently
and effectively to improve education services because they lack management capacity (GPE, 2019).

For the purpose of ensuring system-wide efficiency and effectiveness in education management, it is
imperative that considerations and decisions are based on precise and timely information which is essential
for the successful implementation, monitoring, and evaluation of administrative and educational processes.
Education Management Information Systems (EMIS) serve as a fundamental component in this regard,
enabling the access of critical information for decision-makers, administrative staff, and education
professionals to effectively carry out their responsibilities (see Box 1.). EMIS is more than just data collection
and storage; it's a dynamic tool that helps in setting directions, improving standards, and making informed
policy decisions. By capturing data on various aspects of the education system, including student enrollment,
teacher qualifications, resource allocation, and learning outcomes, EMIS provides valuable insights and
evidence that inform strategies to enhance education delivery. This system's role is multidimensional,
extending from routine administrative tasks to serving as a foundation for informed policymaking and resource
allocation and empowering educational administration to better understand the strengths and weaknesses of
their education systems, identify areas for improvement, and measure progress towards their goals.

1
 Box 1. EMIS framework for action

EMIS is the coordinated network of individuals, organizations, institutions, technologies, processes

and procedures, rules, and regulations, that produce the fit-for-purpose quality data needed for
informing decision-making processes in operating an educational system. The elements included in
an EMIS focus on the needs of educational planners and managers, from high-level decision-makers
to decentralized level, to perform their responsibilities, regardless of the educational sub-sector,
programme orientations, or administrative entity in which they work. EMIS materializes into
interoperable datasets, for which there are distinct functionalities (or modules) for data collection,
processing, management, storage, archiving, and extraction for analysis (including sometimes
simulations and projections models) and dissemination to different types of audiences. Typically, an
EMIS has components referring to students, schools, human resources (teaching and non-teaching
staff), infrastructure and material resources, and financial resources. The information managed by
the EMIS can be quantitative or qualitative and combine various sources of input. These sources
include the school census questionnaire (as a bare minimum), and can include human resources
management records, inspection reports, test scores and assessment results, financial
management, expenditures, special surveys, and geographic information, to name a few.
Accountable entities (designated under well-defined institutional leadership) are responsible to
create, maintain, and update the EMIS by guaranteeing both the financial sustainability of the
system and the continuous professional development of involved actors. Good governance also
implies leadership in sharing the strategic direction of the EMIS, as well as assessing the relevance
and fitness-for-purpose of the data system (e.g., the relevance with the education sector plan).

Source: (UNESCO – IIEP)

EMIS has the transformative potential to revolutionize education by seamlessly translating national-level
policies into effective classroom practices. In doing so, such a system can identify systemic challenges, and,
most importantly, plays a pivotal role in delivering quality, lifelong learning opportunities for all. A proficiently
mature EMIS offers a more comprehensive understanding of student education experience; therefore, it can
aid policymakers and planners process of evaluating the current state of the education system. Improved
learning outcomes will follow, as well as better working systems for students, teachers, and decision makers.
Ergo, if effectively used by the stakeholders, well-integrated and sophisticated EMIS structures can provide
detailed, real-time information to help planners, managers, and educators make informed decisions about
learning performance.

Although the need for such integrated data systems for education management and planning has long been
discussed, actions to meet this need have been slow. EMISs around the world are at distinct stages of maturity
in design, deployment, and use of data. In many countries, EMISs play a purely monitoring and statistical role
through the collection and reporting of headcounts of students, teachers, infrastructures, and assets. This
approach is often based on school census providing a yearly static snapshot of the education system
characteristics. In some countries, they are becoming a comprehensive, integrated, and dynamic education
management solution where information is captured and used in real time and at a much more granular level
than provided by school census. This approach helps to provide timely insights for daily management and
monitoring of ministry’s operations at distinct levels of education administration. The transactional nature of
such information systems helps to support and inform student learning and resource management. In
between basic and comprehensive EMISs, there are diverse data and information systems (see Figure 1.).

2
 Figure 1. Diversity of EMIS

Source: Authors

EMIS transformation pathways have been evolving rapidly in recent years, with a growing focus on the
digitalization of data collection and management processes. This shift towards digital forms and systems,
when successfully implemented, has enabled more efficient data collection and data processing processes, as
well as greater accuracy and timeliness of information to enhance education management to deliver improved
learning outcomes for students. There has also been a move towards a more transactional approach, which
the annual school census approach is progressively complemented or replaced by real-time data systems
tracking HR resources and learners individually: Traditional approaches that relied on annual school census
data, which only provided a snapshot of the education system at a particular point in time are supplemented
by growing emphasis on using unique identifiers for teachers and students, which enables more accurate
tracking of individuals and their progress throughout their educational journey. Overall, these trends reflect a
growing recognition of the importance of real-time data-driven decision-making in education, investing in
the digitalization of data collection and processing, and the need for effective systems to manage and utilize
this data (see Figure 2.).

3
 Figure 1: Evolution of EMIS worldwide

Source: Authors

Yet, many ministries suffer from a lack of information about their education systems’ inputs and outputs,
leaving them to grapple with demands for information about learning outcomes and the effectiveness and
efficiency of education policies and programs. National education stakeholders and development partners
particularly in low or middle-income countries (LMICs) often lack comprehensive, disaggregated, timely, and
reliable information that can serve as a baseline for identifying the systemic and structural challenges in the
education system, developing education plans and programmes, monitoring the structural capacities
(institutional, organizational, human resources, financial, technical and ICT (Information Communications
Technology) infrastructure) and the outcomes of the education policies. Fewer than half of Global Partnership
of Education (GPE) partner developing countries currently gather, communicate, and utilize sufficient
education data (GPE, 2019). The lack of data is holding back progress in education and jeopardizing the
monitoring progress toward the global education goals, particularly SDG (Sustainable Development Goal) 4.

Considering that many developing countries lack the capacity to produce, process, analyze and utilize data in
education planning, and the critical importance of EMIS in education planning, management, monitoring the
performance of education systems and systemwide accountability, strengthening EMIS should be a top
priority and a central focus of education reform and transformation efforts.

Despite increasing investment and focus on strengthening education management information systems,
many countries still struggle with identifying the structural barriers & limitations in national EMIS, prioritizing
the specific areas for investment and resource mobilization in EMIS, and assessing and benchmarking the
structural capacities of EMIS infrastructure. To map out the existing limitations & barriers, identify the
necessary field of action & investment, tailor the use of EMIS to the stakeholders’ use and ensure system-wide
effectiveness, efficiency & accountability, assessing the current level of the structural capacity and maturity
of EMIS is the first step.

As the global commitments to education are prompting countries to evaluate and monitor learning processes
and outcomes, and ensure system-wide effectiveness and efficiency, several diagnostic tools have been
developed to assess the level of development of national EMIS. Three frameworks acknowledged for their
precise evaluation of countries’ education management information systems are the:

- Systems Approach for Better Education Results Education Management Information System
(SABER-EMIS from the World Bank)

- Education Data Quality Assessment Framework (Ed-DQAF from UIS (UNESCO Institute for Statistics))

4
 - Education Management and Information Systems (SIGED from the Inter-American Development
Bank).

Among these tools, the SABER-EMIS has garnered prominence for its data-driven approach to EMIS
assessment and provides in-depth insights into EMIS functionalities. However, the inherent complexity of its
data-driven assessment process introduces a significant demand for time, financial resources, and skilled
human capital. The comprehensive nature of SABER-EMIS can prove to be challenging, particularly for
countries grappling with resource constraints, limiting their ability to conduct an exhaustive evaluation.
Notably, recent activity report (2018) indicate that the application of this SABER EMIS assessment has been
limited in comparison to the other SABER country reporting. As its application is largely tied to specific project-
based assessments rather than serving as a mechanism for sustained, this limitation could be attributed to the
considerable workload and extensive resource prerequisites involved.

In a parallel context, the Ed-DQAF framework is renowned for its meticulous focus on data quality, structured
around six dimensions and twenty-two sub-dimensions, encompassing a diverse array of indicators. However,
Ed-DQAF's scope remains confined to the assessment of statistical data produced by local offices, omitting the
role EMIS should play to incentivize educational administration and management.

Similarly, the SIGED tool, customized for the Latin American and Caribbean (LAC) context, accentuates
management processes and decentralized education governance. While offering a suite of indicators for
assessing EMIS performance, SIGED's contextual specificity potentially limits its applicability in diverse regions
and educational contexts.

It is noteworthy that despite their commendable attributes, the practical impact of these diagnostic tools has
been constrained due to the factors aforementioned. According to the UIS study (2020), the most commonly
used EMIS-quality assessment is the ED-DQAF. It has been used by 28.2% of countries. In the second place,
3.9% of countries have conducted the World Bank-SABER mainly confined to countries in the Arab States, South
and West Asia and Pacific regions. These tools often require substantial resource allocation, both in terms of
finances and skilled personnel. While they are intended to catalyze positive changes in EMIS and prompt policy
reform, the heavy resource burden they impose may inadvertently hinder their widespread adoption and
implementation.

II. The Tool and Approach

As education systems are incredibly complex, taking a systems approach to education sector analysis is gaining
popularity due to its ability to simplify these complexities and find the most efficient solutions to the
entrenched global learning crisis. Examining the national EMIS structure through the lens of the systems
approach means capturing how all of the components and sub-structures of the EMIS system interact with the
whole, recognizing interconnectedness, causality, and feedback loops. To further explore the systemic
alignment and coherence, system-wide effectiveness, and efficiency for better policies, adequately
investigating or diagnosing all the relationships within the EMIS is necessary.

To this end, to ensure that EMIS improves efficiency in the management and use of the available resources,
and that countries smartly invest in education planning and management of information and data, the
diagnosis of the current EMIS structure, capacity and the interrelationship of all EMIS components is critical.
In this context, there is a need for more focused and actionable assessments that delve deeper into the
maturity and effectiveness of EMIS architectures, as solutions designed to support educational administration,
taking into account the evolving needs of education administration.

5
To address this gap, the UNESCO PLS/EDP - EMIS team, to better identify the capacity, limitations, effectiveness,
and efficiency of the national education administration structures, developed a maturity assessment tool
specifically to evaluate national education administration’s information system capabilities and EMIS to well-
address the countries’ needs to track progress in national education administration and monitoring progress
toward the national education objectives and also SDG 4 indicators.

In this milieu, the proposed maturity assessment tool emerges as a strategic response to address the
limitations of existing diagnostic tools. The tool is designed to strike a balance between comprehensiveness
and resource efficiency. Its distinct advantage lies in its ability to offer a lighter workload, simplified
administration, and expedited evaluation compared to other assessments. By prioritizing efficiency, the tool
aims to facilitate a comprehensive understanding of EMIS maturity within national education systems without
imposing unsustainable resource demands and to provide the necessary information to identify countries that
require tailored assistance in strengthening their national EMIS. This tool has the potential to allow the
identification of the bottlenecks of the EMIS structure and provide detailed knowledge to smartly invest in the
significant, relevant, and necessary sub-structure components of the education administration structure.
Investing in these sub-structure components of EMIS will strategically enhance the capabilities of the national
EMIS approximating to a smart system1 and allow the national education administration and EMIS structures
to ensure efficient, transparent, and robust management and monitoring of the education sector.

Thus, the tool will allow to assess the level of maturity of the national EMIS infrastructure and the structural
capacity and EMIS ecosystem, evaluate the features and attributes of the processes, including the data
collection, processing, storage, analysis, tailoring data for the policy use and dissemination of the data and
proper utilization of feedback information. Ultimately the tool is designed to assist countries in enhancing
their EMIS capabilities to effectively address the education sector information and administration needs.
Besides, with this tool, countries will have a standardized and comparable means of measuring their EMIS
maturity, making it easier to identify gaps and areas for improvement. By identifying the bottlenecks of the
EMIS structure and providing detailed knowledge to smartly invest in significant, relevant, and necessary sub-
structure components of the education administration structure, the tool will enable countries to strategically
enhance the capabilities of their national educational administration. On that basis, the proposed maturity
assessment tool represents a transformative and substantial contribution to education transformation as it
goes beyond merely assessing the current status of EMIS; it also underlines and provides countries with a
roadmap for enhancing their EMIS capabilities and realistic education transformation pathways. By doing so,
strengthened EMIS can improve the overall quality of education data and promote greater policy coherence,
more effectively linking policy and needs.

These pathways encompass two main types: (1) consolidating the current level of maturity by strengthening
alignment under a systems approach, and (2) advancing to the next maturity level by mobilizing context-
specific enabler and catalyst dimensions while addressing any blocker dimension.

In alignment with the aforementioned paradigm shift from traditional to modern frameworks in EMIS, the EMIS
Readiness Assessment Tool for Transformation assumes a role that harmonizes with the tenets of the modern
EMIS framework. The tool's scope is purposefully tailored to address the specific needs of educational
administration, focusing on key governance and strategic frameworks, institutional processes, technical
capacities and gathering reliable data essential for effective education sector management. This targeted
approach planned to enable education ministries to comprehensively evaluate the degree to which their data
systems support critical administration processes.

1
A smart system is one that incorporates functions of sensing, actuation, and control in order to describe and analyse a situation, and
make decisions on the basis of the available data in a predictive or adaptive manner. In the EMIS context, smart means a system that
functions in the same way by contributing to smart actions toward student learning and system improvement.

6
By centering its focus on student-level demographic data, individual health data, access and participation
metrics, teacher and human resource management, school infrastructure planning, learning outcomes data,
financial management, policy compliance, and sector-wide monitoring and evaluation, the tool aligns with the
modern EMIS vision of utilizing data to enhance education management comprehensively.

Figure 3. Elements of an EMIS covering all subsectors of the education sector

Source: UNESCO Bangkok (2016)

The tool will evaluate the national education system management according to the 4 dimensions (key success
factor of an EMIS) defined in the framework notably, (I) governance, (II) management, (III) technical
architecture, and (IV) data and information services, as well as 12 sub-dimensions within the dimensions for
constant improvement (see Figure 4.). The four dimensions were chosen to comprehensively evaluate the
national EMIS and due to their critical function and importance in assuring the efficacy and maturity of an EMIS
in education systems. The multi-layered design enables separate maturity assessments for several discrete
areas, in addition to an overall assessment of EMIS. Ultimately, the results obtained from this layered model
serve to provide a deeper understanding of their relative strengths and weaknesses in the domain and to target
specific improvement strategies thereby enabling more efficient resource allocation.

7
 Figure 4. The EMIS Readiness Assessment Tool Architecture (dimensions and sub-dimensions)

The governance dimension was included as it acknowledges the critical significance of an enabling policy
ecosystem in guiding and governing the implementation and operation of an EMIS. Clear policies, strategies,
and legal frameworks are essential for facilitating data collection, analysis, and utilization and ensuring that the
EMIS is aligned with national education objectives and international standards. This dimension recognizes the
need for a policy environment, strategic policy frameworks, multi-stakeholder cooperation and sectoral
collaboration, and financial resources that facilitates effective information management and decision-making
based on evidence.

The management dimension was selected to emphasize the vital role of organizational structures, processes,
and human resources in education system management. It evaluates the efficiency, effectiveness, and capacity
of the institutions responsible for data collection, processing, and dissemination. This dimension emphasizes
the importance of strong institutional management for the efficient operation and long-term viability of the
EMIS by evaluating aspects such as delineation of organizational structure and functions, institutional
processes, stakeholder engagement, capacity development, and dedicated human resources.

The technical architecture dimension was included to acknowledge the crucial role of technological
infrastructure and systems to effectively functioning data management and it evaluates the availability,
reliability, and accessibility of the technical infrastructure meaning hardware, software, networks, and
databases required for efficient information processing and reporting. This dimension also considers available
digital services, and standards & policies as they play a crucial role in augmenting technical architecture
capabilities.

The data and information services and information services dimension was chosen to emphasize the
criticality of high-quality data for informed decision-making and effective education planning. This dimension
assesses data collection methodologies, validation and verification processes, accuracy, timeliness, and
completeness of data which also considers aspects such as data disaggregation, metadata, data standards, and
data end-use mechanisms. By focusing on data quality, this dimension evaluates that whether the EMIS
generates reliable and usable data for monitoring progress in timely manner, to what extent that the data-
driven culture is integrated into the organizational culture as such, evaluating interventions and evidence-
based decision making.

The methodology underpinning the scoring and grading reflects the multi-dimensional nature of education
administration ecosystems. The tool employs a multi-dimensional and multi-layered approach to evaluate the
8
 maturity of EMIS, recognizing each dimension as an integral component contributing to the holistic system's
optimal functionality. The framework assumes a paradigm wherein all dimensions hold equivalent significance
in the assessment, negating any hierarchical bias and ensuring a nuanced appraisal. Scoring within this
framework is executed through a methodical aggregation of scores obtained across these dimensions. For each
dimension and its constituent sub-dimensions, an arithmetic mean is calculated, encompassing the totality of
scores assigned to each corresponding indicator. The determination of these scores is executed with the utmost
rigor based on the defined scales for each question, ensuring that each indicator within every dimension is
evaluated on a consistent scale globally. The resulting scores are synthesized to render a holistic view of the
EMIS's maturity level, offering a clear and quantifiable reflection of the system's readiness and operational
capacity in forming enabling education administration ecosystem for system management and evidence-based
decision-making. The model continues to be refined through the identification of specific indicators referred
to as "prerequisites" for progressing to a higher level of maturity.

The maturity model can be used as a full assessment with the support of the UNESCO HQ-EMIS team to get
quality assurance regarding the submitted data and get tailored recommendations. This also reinforces
transparency and technical overview of the assessment process. The assessment methodology for the EMIS
Readiness Assessment Tool that UNESCO HQ proposes to the Ministry of Education in each country typically
involves the following assessment activities shared below (see Figure 5.).

Figure 5. The Assessment Methodology

UNESCO

Global EMIS
Transformation
Observatory

The assessment methodology proceeds through a series of interconnected steps, forming a comprehensive
approach to evaluating and advancing the readiness of national EMIS. Before the assessment, the UNESCO
Task Team will share the questionnaire and a list of relevant and necessary documents with the Ministry of
Education (MoE) to ensure their familiarity with the tool. The Ministry of Education is expected to form an
internal task team for the assessment based on the guiding document for task team composition, profiles and
responsibilities shared by the UNESCO Task Team.

9
Step 1 - Initiation: A kick-off meeting will be conducted to convene the UNESCO Task Team with the Ministry
of Education Task Team to outline the assessment's scope and establish the ground for the upcoming sessions.
The meeting will include a brief introduction to the tool, an overview of the assessment methodology, and
address any questions from the Ministry of Education who have reviewed the assessment tool. Following the
kickoff meeting, the Ministry of Education Task Team is expected to share the relevant reference documents
with UNESCO Task Team or the UNESCO consultant, which will be diligently reviewed by the consultant before
the self-assessment.

Step 2 – Self Assessment: The Ministry of Education Task Team undertakes the task of self-assessments and
compiling necessary supporting documents with the technical support of a UNESCO consultant. The sessions
are planned to start with the self-assessments responding to the questionnaire (excel template) to reflect
personal insights and perspectives on the EMIS from within the team, creating a diverse pool of information.
These individual assessments are then collectively reviewed to ensure alignment and coherence on the
answers to the questionnaire. Right after this coherence check is done, the questionnaire and the compiled
supporting documents will be shared with the UNESCO Task Team.

Step 3 - Quality Assurance: Upon receiving the filled questionnaire and compiled supporting documents, the
UNESCO Task Team will review the documents and consequently organize online or on-site working sessions,
depending on the needs. With the Ministry of Education Task Team, four working sessions of approximately
three hours each will be scheduled for each dimension of the maturity model. These sessions will involve semi-
structured discussions, during which the questionnaire will be discussed along with open-ended and follow-up
questions. This step is vital to assess the quality and thoroughness of the information gathered in the step 2.

Step 4 - Feedback: Upon the completion of the quality assurance process, a reporting process ensues, lasting
approximately three weeks. Depending on the country's needs, one of the following two report types, either
a country factsheet or analytical report, will be prepared and subsequently shared by the UNESCO Task Team:

- A country factsheet report: key findings from the assessment for each dimension.
- An analytical country report: more detailed insights and recommendations for transformation
pathways.

Following this stage, a feedback session will be arranged with the key focal points to discuss the report’s
findings, providing an opportunity for stakeholders to validate and interpret the results. If necessary, the
outcomes will be refined to accommodate additional information, feedback, and supporting evidence.

III. Expected Outputs and The Global Vision

Central to this project is the development of a lightweight and generic EMIS diagnostic tool. This tool plays a
vital role in evaluation of the existing system for education management which will allow to diagnose and
analyze current state of national EMIS environment with specific regard to diverse subcomponents and their
interrelationship.

Building upon the insights from this diagnostic tool, the project aims to offer countries and ministries
customized transformation pathways for their EMIS transformation. These pathways are tailored to each
country's specific context and requirements. This approach is grounded in the foundational principle of a
country-led assessment and serves a multifaceted role. Going beyond the provision of customized
recommendations to guide countries in strategically allocating resources and developing essential policies, the
tool is expected to play a pivotal role in fostering a comprehensive process of capacity development. Through
the active engagement of national staff in the assessment and decision-making process, the tool will serve as
a catalyst for substantive policy dialogues involving key ministerial officials which is expected to facilitate in-

10
depth discussions about formulation of aligned policies tailored to the specific requirements of a mature and
effective EMIS system for system-wide transformation.

The project extends beyond country level, emphasizing collaboration and cooperation on regional level, as it
aims to facilitate regional policy dialogues, workshops and exchanges among countries via sharing insights,
challenges, and best practices. The aim is to create a network of knowledge, cooperation, and support across
countries on EMIS and utilizing the existing expertise and knowledge.

Begins at the country level, expanding to regional cooperation, the vision extends to a global perspective. The
project is designed to establish an online platform for global benchmarking and serving as an observatory for
global EMIS readiness for transforming education. Through this platform, the project seeks to provide a
comprehensive, real-time view of EMIS maturity on a global scale which will allow cross-country comparison
and global benchmarking.

Figure 6. Expected Outputs and Global Vision

IV. Conclusion

The UNESCO EMIS team recognizes the critical role of EMIS in improving efficiency, monitoring learning
outcomes, and making informed policy decisions. By diagnosing the current maturity scale, capacity and
limitations of national education administration structures, this work aims to identify systemic needs and
provide tailored support to strengthen EMIS capabilities which will lead to improved data collection, processing,
analysis, reporting, and data utilization for evidence-based education policies.

The approach employed in this work is distinguished by its user-friendly nature, lighter response requirements,
and emphasis on country-specific contexts. The team acknowledges the limitations of existing diagnostic tools
such as SABER-EMIS and DQAF (Data Quality Assessment Framework), which demand significant time, financial,
and human resources. The proposed maturity assessment tool optimizes the assessment process, providing a
more efficient and cost-effective solution for countries to track progress, identify bottlenecks, and strategically
invest in relevant components of their education administration structures.

Furthermore, the project goes beyond assessment by fostering knowledge exchange and peer learning through
regional workshops. These platforms will showcase promising best practices in regions, allowing education
management professionals to share experiences and learn from one another. Additionally, the development

11
of an online platform will provide global benchmarking on the maturity level of information systems, offering
insights into country-specific challenges and contextual information.

The project's commitment to enhancing EMIS capabilities is poised to empower countries in the effective
management and monitoring of their education sectors. This, in turn, will pave the way for a profound
transformation in education management, fostering policy coherence and ushering in an era of evidence-
driven decision-making. Ultimately, such a transformation in education management is expected to lead to a
substantial improvement in learning outcomes.

12