EXECUTIVE SUMMARY

Student achievement in Latin America and the Caribbean

Student achievement in Latin America and the Caribbean
Results of the Second Regional Comparative and Explanatory Study (SERCE)

organización de las naciones unidas para la educación, la ciencia y la cultura united nations educational, scientific and cultural organization organisation des nations unies pour l’éducation, la science et la culture

Regional Bureau for Education in Latin América and the Caribbean

Published by the Regional Bureau for Education in Latin America Latina and the Caribbean OREALC/UNESCO Santiago. LLECE Team Héctor Valdés (Coordinator), Ernesto Treviño, Carmen Gloria Acevedo, Mauricio Castro, Sandra Carrillo, Roy Costilla, Daniel Bogoya, Carlos Pardo. Thematic Areas Beatriz Macedo, Liliana Bronzina and Ana Atorresi. Administrative Staff Silvia Ortiz Our special thanks go to Rosa Blanco and Ana Luiza Machado, a.i. Director and former Director of OREALC/UNESCO Santiago, respectively, and to all LLECE members who collaborated to make SERCE possible. We are particularly indebted to Javier Murillo and Marcela Román for their help in drafting the preliminary versions of this report.

Design and Lay-out Ana María Baraona, Ximena Milosevic, Julia Salazar and Alejandro Urbán English Translation Ernesto Leigh

The members of the working team are responsible for the contents of this report. The opinions expressed herein are theirs alone and not necessarily those of UNESCO. The place names and maps used in this publication do not imply on the part of UNESCO any opinion or position in regard to the legal status of countries, cities, territories, or zones; nor regarding their authorities or the drawing of their borders. This publication may be reproduced in its entirety or in part provided that explicit reference always be made to the source.

ISBN: 978-956-8302-94-8 Santiago, Chile. June, 2008

Table of contents

PRESENTATION SECOND REGIONAL COMPARATIVE AND EXPLANATORY STUDY STUDENT ACHIEVEMENT FACTORS ASSOCIATED WITH ACHIEVEMENT FINAL REFLECTIONS

7 8 12 45 47

UNESCO has been called upon to generate, through its mandated fields of action, conditions that guarantee individuals and communities the benefits of a genuine peace and opportunity for development. Considering that poverty and inequality in the region continue to pose an important threat to the dignity and safety of the population, the international community should adopt a humanised vision of development based on respect for human rights, intercultural dialogue and the pursuit of justice. In the field of education, UNESCO has embraced three major objectives, namely: promoting education as a fundamental human right; furthering educational quality and innovation; and generating knowledge to inform educational policy-making. In recent years, the Latin American and Caribbean countries have made important inroads in the region in terms of expanding compulsory education and increasing the system’s coverage, designing new curricula, improving the provision of didactic materials and strengthening school infrastructure, all of this accompanied by substantial investments in teacher training initiatives. Nevertheless, quality of education and its equitable distribution across social groups still remains an unresolved issue. UNESCO’s Regional Bureau for Education in Latin America and the Caribbean proposes, from a human-rights approach, a concept of quality that integrates five dimensions: relevance, fostering learning that takes into account the developmental needs of individuals and societies; pertinence, the need for education to be meaningful for people of different social and cultural strata; equity, giving to all persons the aid and support that will guarantee equal opportunity to access and complete their education, and fully develop their potential; efficacy, ensuring that relevance, pertinence and equity-related goals translate into concrete actions; and efficiency, the proper assignation and use of resources in the quest of the proposed objectives. One of the central activities of the Regional Bureau is generating and disseminating knowledge to inform decision-making on initiatives that promote educational policies and practices aimed at strengthening the quality of education in the various countries. Within this framework, the Latin American Laboratory for Assessment of the Quality of Education (LLECE), founded in Mexico City in 1994, and under the co-ordination of OREALC/UNESCO Santiago, represents a regional network of education evaluation systems committed to provide technical support to the countries of the region. The LLECE launched its First Regional Comparative and Explanatory Study (PERCE) during the 1995 - 1997 period releasing its results in December 1998. Subsequently, seven countries participated in a qualitative research study of the highperforming schools identified in the First Study. The major findings of SERCE’s Second Regional Comparative and Explanatory Study (SERCE, 2002-2008), are presented in this report. We hope they will facilitate decision-making and foster the implementation of educational policies and practices that make possible a faster and more assertive transition to quality education without exclusion in the region.

Rosa Blanco a.i. Director, Regional Bureau for Education in Latin America and the Caribbean UNESCO Santiago

Executive summary

5

Presentation
Improving the quality of education is still the major challenge confronted by the education systems of Latin America and the Caribbean. Governments strive to implement policies that foster quality education, ensure it is made available to all, and equitably distributed, in an attempt to break away from the social determinism that keeps the lower income sectors –and the minorities within them– at a permanent disadvantage. The information generated by evaluations on educational quality of national education systems has allowed the technical and political authorities to review and analyse what is being taught, how is being taught and, obviously, what are primary school girls and boys learning in Latin American and Caribbean schools. In late 2002, member countries of UNESCO’s Latin American Laboratory for Assessment of the Quality of Education (LLECE) led by the OREALC Santiago Office, launched the Second Regional Comparative and Explanatory Study (SERCE) which, drawing on the experience and lessons learnt in a first such study (PERCE, 1998), took relevant steps aimed at expanding the analysis so as to include a higher number of countries, grades and areas in its evaluations. The main objective of the SERCE is to gather valid, accurate, and reliable data on what are primary students actually learning, as well as relevant information on associated factors. The extent to which these results are discussed and integrated into educational and social actions/policies aimed at enhancing and strengthening the quality of public education in participating countries, will provide a measure of its success. This document summarizes SERCE’s process and application, its findings and results. It offers an outline of its purposes, the conceptual perspective used in evaluating performance in the areas of Primary Education Mathematics, Reading and Science of students who during the period 2005 /20061 attended third and sixth grades, major factors associated with these results, and their implications and recommendations to social and educational policies. The SERCE is the result of the effort and commitment of many teams, organisations and national and regional authorities. We owe a special debt of gratitude to the persons who headed the Organisation during the various stages of this Study, namely, Ana Luiza Machado former OREALC/UNESCO Director and Rosa Blanco a. i. Director of the Organisation; to the World Bank, the Inter-American Development Bank, and Ford Foundation, major donors, for their support in each one of the phases; to National LLECE Coordinators; and to the national delegates and their teams. Our most sincere thanks to the principals, teachers, fathers and mothers, boys and girls of participating schools, without whose collaboration and commitment this research would have not been possible. These are ultimately the main actors and beneficiaries of SERCE’s findings.

1

Based on the school calendar of the surveyed countries.

Executive summary

7

Second Regional Comparative and Explanatory Study
The Second Regional Comparative and Explanatory Study (SERCE) represents the most important and ambitious student performance evaluation project ever launched in Latin America and the Caribbean. Under the direction and coordination of the Laboratory for Assessment of the Quality of Education (LLECE), this Study forms part of UNESCO Regional Bureau for Education in Latin America and the Caribbean (OREALC/UNESCO Santiago) global actions aimed at guaranteeing the right to quality education to all Latin American and Caribbean students. Its objective is to give insight into the learning acquired by Latin American and Caribbean Third and Sixth Grade Primary Students in the areas of Mathematics, Language (Reading and Writing) and Natural Science during their school trajectory.

8

Student achievement in Latin America and the Caribbean

In addition to identifying what girls and boys have learned, the results obtained are analysed and explained keyed to factors related to students, classrooms, and schools, placing special emphasis on those factors that may be changed through the implementation of relevant programmes and policies. SERCE represents a collective effort on the part of participating countries, duly articulated by a central LLECE team, and supported by a Technical Advisory Committee and panels of experts in every area. The Study was launched in February 2004 and its main stages will extend through the second semester of 2008.

GRAph 1

ENTITIES PARTICIPATING IN SERCE

Sixteen countries and the Mexican State of Nuevo Leon are taking part in this survey. Third and Sixth Grade Primary Students of all participating countries were evaluated in Mathematics and Language, while sixth graders of nine countries and the State of Nuevo Leon were evaluated in Natural Science. A total of 3.065 schools – encompassing 4.627
Executive summary

9

Third Grade classrooms and 4.227 Sixth Grade classrooms – were surveyed. This represents a total of 100.752 Third Grade and 95.288 Sixth Grade Primary school students2. This sample is representative of approximately eleven million third graders and ten million sixth graders in the region. In terms of evaluating performance and associated factors, SERCE uses a set of instruments specially designed for this purpose. Each of the evaluated students took the Mathematics, Reading and Science tests on different days and was allotted a time consistent with the nature of the tests. Contextual, socio-demographic, family and personal data, in addition to information associated with school processes and dynamics, were captured through the direct administration of questionnaires to students, teachers, principals, and parents of the sampled schools. The objectives of each of these instruments are summarised in the following table.

TAbLE 1
Actor Students

SYNTHESIS OF SERCE’S DATA COLLECTION INSTRUMENTS
Instrument Student Questionnaire Objective Inquire about the family and socio-cultural environment, classroom dynamics and interaction, degree of satisfaction with the school, classmates and teachers, among other topics. Inquire about socio-demographic aspects, professional training, labour conditions, teaching experience and degree of satisfaction with the school, among other topics. Look into pedagogical practices at the corresponding grade and area, such as time management, availability of educational resources, expectations teacher form of their students, types of activities, curricular implementation, evaluation strategies, among other topics. Capture data relative to personals traits, professional profile and trajectory, management model adopted, expectations, degree of satisfaction with the school and co-workers, in addition to other aspects of school life. Collect information on school location, equipment and infrastructure. Inquire about the socio-demographic characteristics of the family, the availability of services and physical amenities in the home, involvement in and support of the educational process of their children, degree of satisfaction with the school, among other aspects.

Teachers

Teacher Questionnaire Questionnaire on teaching practices

Principals

Questionnaire for Principals

Questionnaire on School Characteristics Parents Family Questionnaire

2

Student sample data correspond to the total number of students who took at least one of the tests. This total differs from the total number of students evaluated in each area.

10

Student achievement in Latin America and the Caribbean

The Study shows student performance results from two different perspectives. • On the one hand, it presents mean scores of students’ and their variability by country, areas and grades. Also, the relationship between average scores, national per capita income and Gini Index, for each country. • On the other, it shows results based on student distribution at each national level of performance. This information gives a clear idea of the percentage of students who have similar performance profiles in each country. The First Report on SERCE’s Results includes a progress report on achievement-associated factors that provides a preliminary insight into the variables that have an impact on student learning.

Executive summary

11

Student achievement

Evaluation of Learning - Approaches
In order to evaluate student performance, SERCE used tests based on curricular elements known to be common to the region, fashioned after the life-skills approach propounded by UNESCO. The creation of a common and consensuated curricular framework for Latin America and the Caribbean implied reviewing, systematising and analysing contents prescribed by the curricula for the different areas to be evaluated in the region, in order to determine

12

Student achievement in Latin America and the Caribbean

which conceptual domains were common to the primary education students of participating countries3. The identification of common contents, the approaches used by participating countries to evaluate their students’ performance, and the organisation of this performance, were the criteria guiding the curricular analysis on which the elaboration of tests was based. For its part, the life-skills approach establishes the abilities, principles, values and attitudes that Latin American students should learn to develop in order to ensure their full and active participation in society, both as actors and citizens. This means, dealing with situations, making decisions based on available information, solving problems, and supportting their points of view, among others. Designing tests that inspired on a common curricular framework also place emphasis on life-skills, challenges education to go beyond academic success by offering students learning spaces that promote and ensure a better quality of personal and social life. The tests administered by SERCE evaluate not only the knowledge acquired by Third and Sixth Grade primary education students, but also how these students use – or are capable of using – such knowledge to understand and interpret the world under various daily-life circumstances and contexts. The questions administered during the tests were distributed into six different booklets, thus ensuring coverage of all domains contained in the test reference framework. The inclusion of open-ended questions, allowed students to construct their own responses and, based on that construction, the strategies used by the students to respond could be inferred. This type of question also provided insight into the degree to which the students have acquired attitudes, values and procedures, and developed their own ways of thinking. The questions asked vary substantially in terms of how the information is presented: • Some questions present information as written texts, whereas in others, the information is contained in tables, narratives, graphs or drawings. • Content is also presented in everyday contexts that are familiar to the students, as a way of highlighting the functionality and usefulness of this learning. In order to determine what do Latin American and Caribbean students know, two dimensions were conceived: conceptual domains or area-specific knowledge, and cognitive processes, understood as the mental operations students use to establish relationships with and among objects, situations and phenomena.

3

Guatemala joined the study after the curricular analysis had been completed. Therefore, there is no guarantee that Guatemala’s curricular content will fully agree with the contents selected for the tests.

Executive summary

13

TAbLE 2
Area Mathematics

CONCEPTUAL DOMAINS AND PROCESSES INVOLVED IN EACH SERCE TEST
Conceptual Domains Numerical Geometrical Measurement-based Information handling skills Variational processes • Recognition of elements and objects • Solution of simple problems • Solution of complex problems

Reading

Length of the tested text Text type and genre

• General processes • Processes related to specific texts • Metalinguistic processes • Concept recognition • Interpretation and application of concepts • Problem solving

Natural science

Health and living beings Earth and environment Matter and energy

The following are some examples of the items used:

14

Student achievement in Latin America and the Caribbean

Level I - 3rd Grade Mathematics
Example 1. Books sold per month

Summary card for example 1 3rd Grade I Information handling skills Recognition of objects and elements Interpreting direct information presented in a bar graph Key A: January Difficulty level 412.02 Percentage of correct responses 75.64% Percentage of responses involving distractors B: 9.01% C: 6.16% D: 6.02% Percentage of non-valid responses 3.17% Grade Performance level Domain Process Required action /task

Executive summary

15

Level IV - 3rd Grade Mathematics
Example 2. Number sequence

Summary card for example 2 3rd Grade IV Variational Solving complex problems Recognise an additive numerical sequence rule by its definition Key C: 300 units were added each time Difficulty level 629.06 Percentage of correct responses 30.45% Percentage of responses involving distractors A: 24.95% B: 21.19% D: 15.98% Percentage of non-valid responses 7.43% Grade Performance level Domain Process Required action /task

16

Student achievement in Latin America and the Caribbean

Level II - 6th Grade Reading
Example 3. The perfect horse

Summary card for example 3 6th Grade II Length: Complete text Type of text and genre: Narrative; introduction-climaxresolution Process General: Identifying secondary information Specific: Identifying “voices” in the narrative Metalinguistic: None Required action /task Recognising a character’s attribute based on the saying of a third party Key B: smart Difficulty level 436.69 Percentage of correct responses 74.95% Percentage of responses involving A: 5.45% distractors C: 6.09% D: 11.28% Percentage of non-valid responses 2.23% Grade Performance level Domain

Executive summary

17

Level IV- 6th Grade Reading
Example 4. Title and passages of a narrative

Summary card for example 4 6th Grade IV Length: A relatively lengthy text. Type of text and genre: Explanatory/narrative: legend Process General: Associating a synthesis with that synthesised Specific: Identifying which part of the narrative text is synthesised in the title Metalinguistic: Knowing the meaning of “title” and the different names of the passages Required action /task Identifying which part of the text is synthesised in the title, distinguishing them from other passages through the use of metalanguage Key B: The conflict Difficulty level 599.623 Percentage of correct responses 35.77% Percentage of responses involving A: 22.99% distractors C: 18.67% D: 18.95% Percentage of non-valid responses 3.62% Grade Performance level Domain

18

Student achievement in Latin America and the Caribbean

Level II - 6th Grade Science
Example 5. Balanced breakfast

Summary card for example 5 6th Grade II Health and living beings Recognising and applying concepts The student should be able to recognise the concepts involved and apply them to a familiar and daily situation Key A: Fruit, milk and bread Difficulty level 495.60 Percentage of correct responses 56.18% Percentage of responses involving B: 31.29% distractors C: 5.73% D: 5.77% Percentage of non-valid responses 1.03% Grade Performance level Domain Process Required action /task

Executive summary

19

Level IV - 6th Grade Science
Example 6. The Moon

Summary card for example 6 6th Grade IV Earth and environment Problem Solving Handling concepts related to the force of gravity and their correct application to solve the problem at hand Key D: there is little gravitional force Difficulty level 822.45 Percentage of correct responses 18.37% Percentage of responses involving A: 30.39% distractors B: 13.93% C: 33.97% Percentage of non-valid responses 3.34% Grade Performance level Domain Process Required action /task

20

Student achievement in Latin America and the Caribbean

presentation of Results Results are presented, by grade and area, as follows: • Average scores and variability for each of the countries, based on an arbitrary scale with a mean of 500 and standard deviation of 100. This is meaningless in terms of approving/not approving grade. • Performance levels which classify students on the basis of what they are capable • • of doing. Comparisons of students in urban and rural contexts, and gender-based analysis. Relationship between learning results, per capita gross domestic product of each country and income distribution, using the Gini Index.

Learning in Third Grade
Mathematics Mathematics results for Third Grade students reveal significant differences among countries. Countries situated at the high and low ends of the performance scale are separated from each other by more than 250 points, equivalent to more than 2.5 standard deviations. However, a comparison between the second and next to last countries reveals a difference of approximately one standard deviation. This implies that there is greater homogeneity among countries occupying mid-positions. Based on a global analysis of results, countries may be classified in five groups according to their difference with the countries’ average: • Countries that exhibit mean scores in Mathematics, significantly higher than the regional average (more than one standard deviation). This, however, is only true • for Cuba. Countries that exhibit average scores higher than the regional average (but less than one standard deviation): Chile, Costa Rica, Mexico and Uruguay, and the Mexican State of Nuevo Leon. Countries matching the regional average, that is, cases where no statistically significant differences are evident. This group is comprised of Argentina, Brazil and Colombia. Countries that exhibit mean scores in Third Grade Mathematics lower than the regional average (less than one standard deviation): Guatemala, Ecuador, El Salvador, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic4 *.

4

*

Significant differences (5% error) based on a t test for median comparison.

Executive summary

21

At the regional level, the difference in performance in Third Grade Mathematics between 10 and 90th percentiles is 241 points, with extreme values of 165 points (Nicaragua) and 341 points (Cuba).
th

GRAph 2

MEAN AND VARIABILITY OF THIRD GRADE MATHEMATICS SCORES IN EACH SURVEYED COUNTRY

LAC total: Latin American and Caribbean countries’ total. CILL: Confidence interval lower limit (a = 0.05). CIUL: Confidence interval upper limit (a = 0.05). Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in each country. That is to say, the far-right segment of each bar represents the scores of students in the 90th percentile and the left those of students in the 10th percentile. The greater the distance between these two points, the greater the students’ performance variability. The white vertical line running through the centre of each bar identifies the mean, while the confidence interval is shown as a dark area around it. The width of this darkened area illustrates its possible values.

Along with Cuba, Paraguay and Brazil exhibit the greatest differences between their 10 and 90th percentiles, with 258 and 245 points, respectively. For their part, Panama, El Salvador and Guatemala exhibit the smallest differences (fluctuating around 180 points) between their 10th and 90th percentiles.
th

22

Student achievement in Latin America and the Caribbean

TAbLE 3

DESCRIPTION OF THIRD GRADE MATHEMATICS PERFORMANCE LEVEL AND PERCENTAGE OF STUDENTS OCCUPYING EACH LEVEL
% Students 11.23% Description • Students recognise a numerical sequence rule and identify it. • Students solve multiplication problems with one unknown or problems which require the use of equivalences between commonly used measures of length. • Students identify an element on a bi-dimensional plane and the properties of the sides of a square or rectangle in order to solve a problem. • Students solve multiplication problems or problems which require the use of an addition equation or two separate operations. • Students solve addition problems involving measurement units and their equivalences or problems which require using common fractions. • Students must identify the graphic or addition numerical sequence rule being used in order to continue it. • Students identify the elements of unusual geometrical shapes and interpret different types of graphs in order to retrieve information and solve problems that involve operating with the data. • Students recognise the organisation of the decimal-positional numeral system and identify the constituent elements of geometrical shapes. • Students identify a trajectory on a plane and the most suitable measurement unit or instrument, in order to measure a known object’s attribute. • Students interpret tables and charts in order to obtain information and compare data. • Students solve addition or multiplication problems involving proportional relationships, using natural numbers. • Students recognise the relationship between natural numbers and common bi-dimensional geometric shapes in simple drawings. • Students locate relative positions of an object in a spatial representation. • Students interpret tables and graphs in order to obtain direct information. • Students at this level have not been able to acquire the abilities required in Level I.

Level
Cut-off score

IV

621.68

III

14.30%

558.54

II

28.26%

489.01

I

36.03%

391.50

Below I

10.19%

As shown in Table 3, 10.2% of all students are not capable of completing the tasks designed for the lowest level. This group of girls and boys – which total over a million for all the countries surveyed – demands urgent and appropriate help given their low levels of learning. Table 4 shows student distribution by performance levels for each participating country. Cuba exhibits the highest performance levels, with 54.36% of its students occupying Level IV. In Chile, Costa Rica, Mexico, Uruguay and Nuevo Leon, over a third of their students occupy Levels III and IV.
Executive summary

23

In Brazil and Argentina one fourth of their students occupy Levels III and IV. In the rest of the countries less than one fourth of the surveyed students placed at these levels. In the case of the Dominican Republic, 41.28% of the country’s students are below Level I, a figure that is considerably lower (between 14% and 16%) for students of Ecuador, Panama, Paraguay and Peru.

TAbLE 4

PERCENTAGE OF THIRD GRADE STUDENTS BY PERFORMANCE LEVEL IN MATHEMATICS IN EACH SURVEYED COUNTRY
Country Argentina Brazil Chile Colombia Costa Rica Cuba Ecuador El Salvador Guatemala Mexico Nicaragua Panama Paraguay Peru Dominican Rep. Uruguay Nuevo Leon Total below I 10.46 10.32 5.10 8.57 2.62 1.09 14.34 10.31 17.34 5.15 12.10 15.98 15.87 15.24 41.28 5.78 2.34 10.19 I 32.77 36.55 27.90 38.60 24.44 10.19 45.48 45.00 50.06 28.85 47.95 49.69 37.88 45.42 49.27 25.95 18.45 36.03 II 31.13 26.74 33.60 33.19 37.00 16.95 28.12 31.80 25.07 30.70 30.50 25.15 25.50 25.95 8.49 30.03 31.69 28.26 III 15.17 14.32 19.37 12.97 22.30 17.41 7.91 9.25 5.46 19.71 7.49 6.42 11.56 8.61 0.84 19.29 24.41 14.30 IV 10.47 12.07 14.02 6.67 13.65 54.36 4.14 3.64 2.08 15.59 1.97 2.75 9.20 4.77 0.13 18.95 23.11 11.23

Note: Below I students are those who cannot attain level I.

School location is also responsible for the differences in student performance levels observed in the region. Table 5 shows that Latin American and Caribbean girls and boys attending rural schools perform at lower levels when compared to their counterparts attending urban schools54. In this sense, Peru, Brazil, and Mexico exhibit the largest urbanrural gaps. Cuba, Nicaragua, and Paraguay do not reveal statistically significant differences in the averages obtained by urban and rural students.

4

The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean represent a rough measure that, given the various definitions of rurality, should be taken with caution.

24

Student achievement in Latin America and the Caribbean

TAbLE 5

AVERAGE SCORE DIFFERENCES BETWEEN URBAN AND RURAL SCHOOLS, AND BY GENDER. THIRD GRADE MATHEMATICS
Country Argentina Brazil Chile Colombia Costa Rica Cuba Ecuador El Salvador Guatemala Mexico Nicaragua Panama Paraguay Peru Dominican Rep. Uruguay Nuevo Leon Total Urban/ Rural Difference 40.09* 62.17* 33.29* 26.29* 29.25* 7.79 20.70* 39.92* 40.07* 43.01* -1.15 22.41* 17.91 69.88* 17.60* 31.72* 28.68* Girl/ boy Difference -1.42 1.91 -13.37* -8.26* -10.80* 4.47 0.55 -10.90* -6.98* 0.09 -12.72* 5.53 2.31 -9.20* 12.66* 0.28 -3.92 -1.25

* Significant at a 5% confidence level.

In terms of gender, at the regional level, median scores for Third Grade Mathematics do not reveal significant differences. However, this overall result conceals important differences among countries: • In Argentina, Brazil, Cuba, Ecuador, Mexico, Panama, Paraguay, Uruguay and the Mexican State of Nuevo Leon, gender-based differences are not significant. • Chile, Colombia, Costa Rica, El Salvador, Guatemala, Nicaragua and Peru exhibit significant differences which would seem to indicate that boys outperform girls in Mathematics. • Exceptionally, in the Dominican Republic the opposite is true. Analyses of the existing relationship between performance and gross national product and income distribution reveal interesting differences among countries. There is a correlation between student average score in Third Grade Mathematics and their national per capita GDP. In fact, this economic indicator accounts for 28.37% of the countries’ average performance variance. The relationship between results and the Gini Index –as income distribution indicator– is equally significant, although inverse. In other words, the greater the inequality

Executive summary

25

the lower the results obtained in Third Grade Mathematics. The Gini Index can account for 17.06% of the countries’ average performance variance in Mathematics. Reading As with Mathematics, important differences are evident among participating countries. Thus, the difference between the countries with highest and lowest performances is 2.3 standard deviations, that is to say, about 230 points. However, the difference between the second and next to last country is 1.15 standard deviations, which reveals a somewhat more homogeneous distribution of results. Based on the high disparity and internal dispersion of national averages, five groups of countries were identified relative to the average performance of their students. • Countries where average performance is markedly higher than the median of SERCE participants by more than one standard deviation. This case is illustrated by Cuba. • Countries where average performance is higher than the average of SERCE participants by less than one standard deviation. This group is comprised of Argentina, Chile, Colombia, Costa Rica, Mexico, Uruguay, and the Mexican State of Nuevo Leon. • Countries where performance exhibits a mean score statistically identical to the regional average: Brazil and El Salvador. • Countries where performance exhibits a lower score than the average of SERCE participants of less than one standard deviation. This is the case of Ecuador, Guatemala, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic6 *.

6 *

Significant differences (5% error) based on a t test for median comparison.

26

Student achievement in Latin America and the Caribbean

GRAph 3

MEAN AND VARIABILITY OF THIRD GRADE READING SCORES IN EACH SURVEYED COUNTRY

LAC total: Latin American and Caribbean countries’ total. CILL: Confidence interval lower limit (a = 0.05). CIUL: Confidence interval upper limit (a = 0.05). Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in each country. That is to say, the far-right segment of each bar represents the scores of students in the 90th percentile and the left those of students in the 10th percentile. The greater the distance between these two points, the greater the students’ performance variability. The white vertical line running through the centre of each bar identifies the mean, while the confidence interval is shown as a dark area around it. The width of this darkened area illustrates its possible values.

In terms of Third Grade Reading tests, performance differences between percentile 10th and percentile 90th students in each country fluctuate between 208 and 242 points, with the exceptions of Cuba and Nicaragua. • Cuba shows the greatest dispersion of results, since the distance between students of the percentiles under comparison is 295 points. However, the lower performing Cuban students obtain scores that are similar to the countries’ average. • For its part, Nicaragua shows a low dispersion of results with differences between students at both extremes that scarcely exceed 183 points. • Guatemala, Peru and El Salvador, show a moderate dispersion with differences between extremes fluctuating in the 208 - 220 point range. • The twelve remaining countries reveal differences between their first and last deciles in the 224 and 241 point range.

Executive summary

27

TAbLE 6
Level
Cohort score

DESCRIPTION OF READING PERFORMANCE LEVELS OF THIRD GRADE STUDENTS
% students 8.41% Description • Integrate and generalise information given in a paragraph or in the verbal codes and graph; • Replace non-explicit information; • Read the text identifying new information; • Translate from one code to another (from numeric to verbal, and verbal to graphic) • Locate information discriminating it from adjacent information; • Interpret reformulations that synthesise several data; • Infer information based on knowledge about the world; • Discriminate, based on the text, the meaning of words that have several other meanings. • Locate information in a brief text that must no be distinguished from other conceptually similar information; • Discriminate words with a single meaning; • Recognise simple sentence reformulations; • Recognise redundancies between graphic and verbal codes • Locate information with a single meaning, in a prominent part of the text, repeated literally or synonymously, and isolated from other information. • Students at this level have not been able to acquire the abilities required in Level I.

IV

637.49

III

21.63%

552.14

II

37.74%

461.32

I
367.36

25.51%

Below I

6.71%

In terms of reading achievement, 6.7% of the total number of Third Grade Primary Education students in the region scored below Level I. This means that students failed to locate information, with a single meaning, which is repeated in the text and isolated from other information. Table 7 shows performance levels by country, and confirms the fact that: • 44.3% of Third Grade Cuban students scored the highest in Reading, followed by students of Nuevo Leon (18.4%), Costa Rica (18.2%), and Chile (17.8%). • 31.4% of the students of Dominican Republic scored below Level I, similarly to more than 14% of the ones of Ecuador and Guatemala, and approximately 11% of those of Panama and Paraguay. In terms of Reading, rural school students participating in SERCE obtained lower scores than their counterparts attending urban schools75. This is shown in Table 8 where the differences in the results obtained by urban school students versus rural school students are described.

5

The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean represent a rough measure that, given the various definitions of rurality, should be taken with caution.

28

Student achievement in Latin America and the Caribbean

TAbLE 7

PERCENTAGE OF THIRD GRADE STUDENTS BY READING PERFORMANCE LEVEL IN EACH SURVEYED COUNTRY
Country Argentina Brazil Chile Colombia Costa Rica Cuba Ecuador El Salvador Guatemala Mexico Nicaragua Panama Paraguay Peru Dominican Rep. Uruguay Nuevo Leon Total below I 6.26 6.29 1.60 4.94 1.46 0.56 14.62 5.34 14.37 3.65 6.95 11.21 11.47 9.24 31.38 4.69 1.70 6.71 I 22.01 25.25 9.97 23.61 10.40 6.48 37.47 29.05 43.18 19.64 37.29 37.24 37.85 36.18 46.73 19.96 12.71 25.51 II 39.73 39.84 34.46 41.78 34.20 21.09 34.20 41.05 32.04 37.09 43.38 35.29 32.27 35.79 18.04 39.02 34.82 37.74 III 23.63 21.54 36.22 21.16 35.73 27.61 11.61 19.15 8.51 27.52 10.69 12.35 12.92 15.13 3.29 24.94 32.40 21.63 IV 8.37 7.07 17.76 8.52 18.22 44.27 2.10 5.40 1.91 12.09 1.70 3.91 5.49 3.65 0.56 11.39 18.38 8.41

Significant differences in Reading results obtained by Third Grade students attending urban and rural schools are evident in Latin America and the Caribbean. • Peru exhibits the greatest differences –over 79 points– in terms of rural versus urban school results. The country is followed by Guatemala, Brazil and Mexico with differences that fluctuate between 62 and 64 points. • Cuba and the Dominican Republic show the smallest differences between rural and urban schools – 16 and 19 points, respectively. Reading results also reveal marked differences in gender. Overall, among SERCE participants, girls obtained the highest Third Grade Reading scores. In fact, girls outperformed boys by an average of 12.7 points. • Argentina, Brazil, Cuba, Mexico, Panama, Paraguay, the Dominican Republic, Uruguay and the Mexican State of Nuevo Leon, show significant differences between boys and girls in terms of Reading scores. • The rest of the countries show no statistically significant differences when making gender-based comparisons. Reading performance of Third Grade Primary Education students shows a direct correlation with the gross national product of each country. In particular, differences in national per capita GDP, account for one third of the variability observed in national performance averages.
29

Executive summary

The greater the income distribution inequality, the lower the average Reading performance observed among Third Grade students. The Gini Index, for its part, accounts for 12.6% of the variability detected in the national performance median.

TAbLE 8

DIFFERENCE IN AVERAGE SCORES BETWEEN URBAN AND RURAL SCHOOLS, BY GENDER. THIRD GRADE READING
Country Argentina Brazil Chile Colombia Costa Rica Cuba Ecuador El Salvador Guatemala Mexico Nicaragua Panama Paraguay Peru Dominican Rep. Uruguay Nuevo Leon Total Urban/ Rural Difference 34.53* 62.67* 34.68* 50.92* 41.24* 15.94* 42.83* 57.29* 64.07* 62.47* 29.42* 54.70* 36.45* 79.30* 19.45* 28.56* 37.24* Girl/ boy Difference 17.74 18.57 2.46 4.64 4.69 13.34 9.02 1.39 1.67 13.20 1.77 14.94 15.69 0.76 13.05 12.73 9.05 12.74

* Stands for 5% confidence level.

30

Student achievement in Latin America and the Caribbean

Learning in Sixth Grade
Mathematics The analysis of Sixth Grade Mathematics mean scores shows marked differences. The difference between the average scores of highest and lowest performing countries (Cuba and the Dominican Republic, respectively) reaches 220 points, that is to say, more than 2 standard deviations. However, the difference between the second and next to last countries is 1.26 standard deviations.

GRAph 4

MEAN AND VARIABILITY OF SIXTH GRADE MATHEMATICS MEDIAN SCORES IN EACH SURVEYED COUNTRY

LAC total: Latin American and Caribbean countries’ total. CILL: Confidence interval lower limit (a = 0.05). CIUL: Confidence interval upper limit (a = 0.05). Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in each country. That is to say, the far-right segment of each bar represents the scores of students in the 90th percentile and the left those of students in the 10th percentile. The greater the distance between these two points, the greater the students’ performance variability. The white vertical line running through the centre of each bar identifies the mean, while the confidence interval is shown as a dark area around it. The width of this darkened area illustrates its possible values.

Executive summary

31

Based on an overall analysis of results, countries may be divided into four groups, relative to their difference with the countries’ average: • Countries where Mathematics Sixth Grade students exhibit a higher average performance than the regional average, at one standard deviation above this average. Cuba, with an average of 637 points, is part of this first group. • Countries that exhibit mean scores above the regional average, but less than one standard deviation. Uruguay, the Mexican State of Nuevo Leon, Argentina, Chile, Costa Rica and Mexico form part of this group. • Countries with average performances equal to the average of all participating countries, in other words, where no statistically significant differences between these two average values are evident. Brazil, Colombia and Peru belong in this group. • Countries that exhibit mean scores below the countries’ average (less than one standard deviation): Ecuador, El Salvador, Guatemala, Nicaragua, Panama, Paraguay and the Dominican Republic8*.

An analysis of student performance variability can shed light on the inequality of education. Within the region, the difference between average scores in the 10th and 90th percentiles represents 242.6 points. Disaggregating by countries, we find differences between the 10th and 90th percentiles that fluctuate between 182 and 385 points. On this basis, four groups of nations can be established: • In the Dominican Republic, Nicaragua, El Salvador, Panama and Guatemala, the range of dispersion between these percentiles is less than 200 points. • In Colombia, Paraguay, Brazil, Costa Rica, Argentina, Ecuador, Chile and the Mexican State of Nuevo Leon, variability between the 10th and 90th percentiles fluctuates between 200 and 250. • Mexico, Peru and Uruguay exhibit a performance dispersion range above 250 points but below 300 points. • Cuba’s internal variability exceeds 300 points.

8 *

Significant differences (5% error) based on a t test for median comparison.

32

Student achievement in Latin America and the Caribbean

TAbLE 9
Level
Cut-off Score

DESCRIPTION OF MATHEMATICS PERFORMANCE LEVELS OF SIXTH GRADE STUDENTS

% students 11.44%

Description • Students find averages and do calculations using the four basic operations in the filed of natural numbers. • Students identify paralleIism and perpendicularity in a real situation and the graphic images of a percentage. • Students solve problems involving properties of angles, triangles and quadrilaterals as part of different shapes, or involving operations with two decimal number • Students solve problems involving fractions. • Students make generalisations in order to continue a complex graphic sequence pattern. • Students compare fractions, use the concept of percentages when analysing information and solving problems that require this type of calculation. • Students identify parallelism and perpendicularity on a plane, as well as bodies and their elements without the benefit of graphic support. • Students solve problems that require interpreting the constituent elements of a division or equivalent measures. • Students recognise central angles and commonly used geometrical shapes, such as circles, and resort to their properties for solving problems. • Students solve problems involving areas and perimeters of triangles and quadrilaterals. • Students make generalisations in order to continue a graphic sequence or find the numerical sequence rule that applies to a relatively complex pattern. • Students analyse and identify the structure of the positional decimal number system, estimate weight (mass) expressing it in units consistent with the attribute being measured. • Students recognise commonly used geometrical shapes and their properties in order to solve problems. • Students interpret, compare and work with information presented through various graphic images. • Students identify the regularity of a simple pattern sequence. • Students solve addition problems in different numerical fields (natural numbers, decimals) including commonly used fractions or equivalent measures. • Students solve multiplication or division problems, or two natural number operations, or operations that include direct proportionality relations. • Students arrange natural numbers (up to 5 digits) and decimals (up to thousands) in sequence. • Students recognise common geometrical shapes and the unit consistent with the attribute being measured. • Students interpret information presented in graphic images in order to compare it and change it to a different form of representation. • Students solve problems involving a single addition using natural numbers. • Students at this level have not been able to acquire the abilities required in Level I.
Executive summary

IV

624.60

III

32.35%

514.41

II

40.82%

413.58

I

13.91%

309.64

Below I

1.48%

33

Table 10, shows that nearly 75% of Cuban and Uruguayan students are placed in Levels III and IV, exhibiting the highest performances for Mathematics achievement. More than 50% of Sixth Grade students in Nuevo Leon, Costa Rica, Mexico and Chile, attained the highest performance levels in Mathematics. On the other hand, between 50% and 60% of all surveyed students in Argentina, Brazil, Peru, Colombia and Paraguay, performed at Levels I and II. This is also true for more than 70% of the surveyed students in Ecuador, El Salvador, Guatemala, Nicaragua, Panama and the Dominican Republic.

TAbLE 10

PERCENTAGE OF SIXTH GRADE STUDENTS BY MATHEMATICS PERFORMANCE LEVEL IN EACH SURVEYED COUNTRY
Country Argentina Brazil Chile Colombia Costa Rica Cuba Ecuador El Salvador Guatemala Mexico Nicaragua Panama Paraguay Peru Dominican Rep. Uruguay Nuevo Leon Total below I 1.53 1.46 1.40 1.02 0.09 0.19 4.24 1.95 2.78 0.51 2.25 3.32 3.85 2.41 5.69 0.67 0.34 1.48 I 11.89 14.00 9.84 13.29 4.55 4.43 24.86 19.18 24.94 8.38 23.88 27.16 21.00 19.58 41.79 4.26 6.29 13.91 II 37.99 44.09 37.85 47.64 32.71 17.93 45.15 51.61 50.80 32.41 52.69 49.55 46.50 39.82 45.43 22.36 29.35 40.82 III 36.26 31.65 37.39 32.60 43.70 26.33 21.41 23.81 19.52 39.10 19.41 17.64 23.91 28.90 6.85 40.41 40.66 32.35 IV 12.34 8.80 13.52 5.46 18.95 51.13 4.34 3.45 1.96 19.60 1.76 2.33 4.74 9.29 0.24 32.31 23.36 11.44

As shown in Table 11, Sixth Grade rural school students participating in SERCE obtain lower scores in Mathematics than their counterparts attending urban schools.

34

Student achievement in Latin America and the Caribbean

TAbLE 11

DIFFERENCE IN AVERAGE SCORES BETWEEN URBAN AND RURAL SCHOOLS, BY GENDER. SIXTH GRADE MATHEMATICS
Country Argentina Brazil Chile Colombia Costa Rica Cuba Ecuador El Salvador Guatemala Mexico Nicaragua Panama Paraguay Peru Dominican Rep. Uruguay Nuevo Leon Total Urban/ Rural Difference 40.21* 42.74* 36.51* 29.03* 23.34* 4.98 42.81* 44.76* 38.39* 51.42* 10.24* 37.33* 31.18* 87.03* 9.01* 52.45* 35.79* Girl/ boy Difference -5.79 -10.02* -6.84* -14.53* -20.67* 8.24* 0.29 -9.48* -6.91* 6.35 -10.16* 2.81 -0.59 -18.94* 0.96 0.18 0.27 -6.17*

* Stands for 5% confidence level

Latin American and Caribbean Sixth Grade students attending urban schools outperform rural school students in Mathematics. • In terms of urban versus rural school results, Peru shows the largest gaps exceeding an 87 point difference, on average. Uruguay and Mexico follow with differences in the neighbourhood of 52 points. • By contrast, Cuba and the Dominican Republic show the smallest differences between rural and urban schools (5 and 9 points, respectively). A SERCE’s gender-based analysis reveals that, at the regional level, boys score some 6 points higher than girls in Sixth Grade Reading tests. Furthermore, based on other important differences detected among countries, three groups can be established: • A first group is comprised of Argentina, Ecuador, Mexico, Panama, Paraguay, the Dominican Republic, Uruguay, and the Mexican State of Nuevo Leon, where no statistically significant differences in terms of performance of girls and boys were detected. • A second group includes Cuba, where girls obtained significantly higher scores than boys.

Executive summary

35

Finally, a group of countries where average performance is skewed in favour of boys. These countries are: Brazil, Chile, Colombia, Costa Rica, El Salvador, Guatemala, Nicaragua and Peru.

On the other hand, national per capita income is strongly associated with student performance in Mathematics. Differences in GDP account for 41% of the average score variability observed in Sixth Grade Mathematics tests. These results reveal that, in any one country, the greater the inequality the lower its average performance. Similarly, differences in the Gini Index among countries account for 32% of the average score variance. Reading An overall analysis of average Sixth Grade Reading scores and their distribution provides insight into the inequalities within and across countries. The difference between countries located at both extremes is 1.75 standard deviations. However, between the second and next to last countries this difference is reduced to only 1.16 standard deviations. Based on students’ average performance countries may be classified in five groups: 1. Countries where students’ scores exceed the average of the countries participating in SERCE (less than one standard deviation). Cuba, Costa Rica, Brazil, Chile, Colombia, Mexico, Uruguay and the Mexican State of Nuevo Leon are part of this group. 2. Countries where students’ mean scores are equal to the regional average. Argentina illustrates the only case. 3. Countries where students’ scores are lower than SERCE’s regional average, and less than one standard deviation: Ecuador, El Salvador, Guatemala, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic9 *.

9

*

Significant differences (5% error) based on a t test for median comparison.

36

Student achievement in Latin America and the Caribbean

GRAph 5

MEAN AND VARIABILITY OF READING SCORES OBTAINED BY SIXTH GRADE STUDENTS IN EACH SURVEYED COUNTRY

LAC total: Latin American and Caribbean countries’ total. CILL: Confidence interval lower limit (a = 0.05). CIUL: Confidence interval upper limit (a = 0.05). Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in each country. That is to say, the far-right segment of each bar represents the scores of students in the 90th percentile and the left those of students in the 10th percentile. The greater the distance between these two points, the greater the students’ performance variability. The white vertical line running through the centre of each bar identifies the mean, while the confidence interval is shown as a dark area around it. The width of this darkened area illustrates its possible values.

Looking at the variability across student performance facilitates an analysis of the learning inequalities that characterise each country. Performance differences between students in the 10th and 90th percentiles of the various countries fluctuate in the 182 to 294 point range (244 points at the regional level). In El Salvador, Nicaragua and the Dominican Republic, students in these percentiles are separated by less than 200 points. This is not the case of Argentina, Brazil, Chile, Colombia, Costa Rica, Ecuador, Guatemala, Mexico, Panama, Paraguay, Peru, Uruguay and the Mexican State of Nuevo Leon, where the difference between the 10th and 90th percentiles fall in the 206 to 259 point range. Lastly, Cuba exhibits the greatest difference in scores, with a 294 point gap between 10th and 90th percentiles.

Executive summary

37

TAbLE 12
Level
Cut-off Score

DESCRIPTION OF READING PERFORMANCE LEVELS OF SIXTH GRADE STUDENTS
% students 20.30% Description • Integrate, rank and generalise information distributed across the text; • Establish equivalences among more than two codes (verbal, numerical and graphic); • Reinstate implicit information associated with the entire text; • Recognise the possible meanings of technical terms or figurative language; • Distinguish various tenses and nuances (certainty, doubt) used in a text • Locate information and separate it from other near-by information; • Interpret reformulations and synthesis; • Integrate data distributed across a paragraph; • Reinstate implicit information in the paragraph; • Re-read in search of specific data; • Identify a single meaning in words that have several meanings; • Recognise the meaning of parts of words (affixes) using the text as a reference • Locate information in the middle of a text that must be distinguished from a different piece of information found in a different segment; • Identify words with a single meaning • Locate information with a single meaning in a prominent or central part of the text (beginning or end), that is repeated literally or synonimously and is isolated from other information. • Students at this level have not been able to acquire the abilities required in Level I.

IV

593.59

III

26.79%

513.66

II

35.46%

424.54

I
299.59

16.51%

Below I

0.93%

Table 13 shows how students are distributed in each of the performance levels, by country. In terms of Reading achievement, 50% of Cuba’s Sixth Grade students can be found at Level IV, followed by Costa Rica with slightly over a third of its students occupying this level. For their part, the percentage of students performing at Level IV in Uruguay, Chile, the Mexican State of Nuevo Leon, Mexico and Brazil, fluctuate between 20% and 30%. At the other extreme, 47.8% of the Dominican Republic’s Sixth Grade students performed at Level I, followed by Ecuador, Guatemala, Panama and Paraguay with slightly over one third of their students at this level.

38

Student achievement in Latin America and the Caribbean

TAbLE 13

PERCENTAGE OF SIXTH GRADE STUDENTS BY READING PERFORMANCE LEVEL IN EACH SURVEYED COUNTRY
Country Argentina Brazil Chile Colombia Costa Rica Cuba Ecuador El Salvador Guatemala Mexico Nicaragua Panama Paraguay Peru Dominican Rep. Uruguay Nuevo Leon Total below I 1.78 0.57 0.30 0.39 0.22 0.30 4.47 0.95 2.86 0.23 1.02 1.95 3.90 2.24 4.08 0.47 0.21 0.93 I 17.93 14.85 8.02 13.17 5.00 5.26 33.69 21.49 33.06 12.23 22.08 28.97 33.46 24.08 47.84 9.60 9.12 16.51 II 35.59 34.65 30.06 38.25 23.45 19.57 39.48 44.02 43.36 33.40 50.58 38.76 36.81 41.65 37.50 30.80 29.99 35.46 III 25.48 27.47 32.37 30.40 36.73 24.20 16.63 23.99 15.73 29.75 21.10 20.77 18.60 22.57 9.19 29.68 32.37 26.79 IV 19.22 22.46 29.26 17.80 34.59 50.68 5.73 9.54 4.99 24.39 5.22 9.55 7.23 9.46 1.39 29.45 28.31 20.30

Data on performance by school type reveal marked differences between the learning 0 acquired by students in urban and rural areas6 1as shown in Table 14. Latin American and Caribbean Sixth Grade students attending urban schools outperform rural school students in Reading. • Cuba is the only country that does not show significant performance differences • • between urban and rural school students. In terms of school location, Nicaragua and the Dominican Republic show the smallest differences - 21 and 24 points, respectively. By contrast, Peru shows the greatest differences –around 80 points– between urban and rural school students, followed by Mexico, Panama and Paraguay with differences approaching 57 points.

6

The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean represent a rough measure that, given the various definitions of rurality, should be taken with caution.

Executive summary

39

TAbLE 14

DIFFERENCE IN AVERAGE SCORES BETWEEN URBAN AND RURAL SCHOOLS, BY GENDER. SIXTH GRADE READING
Country Argentina Brazil Chile Colombia Costa Rica Cuba Ecuador El Salvador Guatemala Mexico Nicaragua Panama Paraguay Peru Dominican Rep. Uruguay Nuevo Leon Total Urban/ Rural Difference 43.55* 49.35* 35.66* 41.74* 34.37* 12.75 46.22* 54.31* 53.75* 57.71* 21.42* 56.67* 56.32* 78.96* 23.75* 49.10* 39.23* Girl/ boy Difference 11.05* 15.69* 6.89* -4.43 -0.75 15.21* 6.39 -0.19 -2.44 13.32* -0.61 15.89* 11.14* -1.87 15.09* 19.64* 7.98 10.44*

* Stands for 5% confidence level

A gender-based analysis reveals that in Latin America and the Caribbean Sixth Grade girls outperform boys in Reading. The regional gap between genders is 10.4 points. Girls also obtain significantly higher scores in Argentina, Brazil, Chile, Cuba, Mexico, Panama, Paraguay, the Dominican Republic and Uruguay. Per capita GDP bears a direct correlation with students’ average learning. Differences in national wealth account for 44.4% of the variation detected in Sixth Grade Reading national averages. The greater the Gini Index the lower the Reading average performance among Sixth Grade students. Differences in the Gini Index account for 11% of the variation observed across national Reading averages. Natural Science The Natural Science test was administered to Sixth grade Primary Education students exclusively, with the participation of only 10 national entities: Argentina, Colombia, Cuba, El Salvador, Panama, Paraguay, Peru, the Dominican Republic, Uruguay, and Nuevo Leon. The difference separating countries located at the upper and lower ends of the performance scale was calculated at 2.35 standard deviations. However, the difference between the second highest and next to last country is only 0.68 standard deviations, which im40
Student achievement in Latin America and the Caribbean

plies that there is greater homogeneity among countries occupying mid-positions in the distribution. Overall, both national averages and distribution of scores show differences in each country. Relative to performance in Science, four groups can be identified: • The first group is made up of countries with mean scores markedly higher than the regional average (more than one standard deviation, that is, over 650 points). • Cuba is the only case. The second group consists of countries with scores higher than the Latin American and Caribbean average (less than one standard deviation): Uruguay and the Mexican State of Nuevo Leon. Colombia is the only country in this third group, characterised by a national mean that does not show significant differences versus the regional media. Countries that exhibit lower scores than the Latin American and Caribbean average (less than one standard deviation) are part of a fourth group. These countries are: Argentina, El Salvador, Panama, Paraguay, Peru and the Dominican Republic1 *.

• •

GRAph 6

MEAN AND VARIABILITY OF SCIENCE SCORES OBTAINED BY SIXTH GRADE STUDENTS IN EACH SURVEYED COUNTRY

LAC total: Latin American and Caribbean countries’ total. CILL: Confidence interval lower limit (a = 0.05). CIUL: Confidence interval upper limit (a = 0.05). Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in each country. That is to say, the far-right segment of each bar represents the scores of students in the 90th percentile and the left those of students in the 10th percentile. The greater the distance between these two points, the greater the students’ performance variability. The white vertical line running through the centre of each bar identifies the mean, while the confidence interval is shown as a dark area around it. The width of this darkened area illustrates its possible values.

1 *

Significant differences (5% error) based on a t test for median comparison.

Executive summary

41

The differences detected in learning results are reflected in the students’ scores dispersion. Three scenarios characterise the region: • In most countries the distance separating the 10th from the 90th percentiles fluctuates between 200 and 230 points. This is the case of Argentina, Colombia, Panama, Paraguay, Peru, Uruguay and the Mexican State of Nuevo Leon. • El Salvador and the Dominican Republic, with less than 200 points separating the 10th and 90th percentiles, exhibit the smallest dispersion of results. • Cuba, in addition to showing the highest average score, also shows the greatest dispersion of results – 386 points between students in the 10th and 90th percentiles.

TAbLE 15
Level
Cut-off score

DESCRIPTION OF SCIENCE PERFORMANCE LEVELS OF SIXTH GRADE STUDENTS
% students 2.46% Description • At this level, students use and transfer scientific knowledge, which requires a high degree of formalisation and abstraction, to diverse types of situations. • Students are capable of identifying the scientific knowledge involved in the problem at hand. These problems are more formally stated and may relate to aspects, dimensions or analyses that may be detached from the immediate setting. • At this level, students explain everyday situations on the basis of scientific evidence; use simple descriptive models to interpret natural phenomena, and draw conclusions from the description of experimental activities. • At this level, students apply school-acquired scientific knowledge: compare, organise and interpret information presented in various formats (tables, charts, graphs, pictures); identify causality relations and classify living beings according to a given criterion. • In connection with Level I, it should be noted that these students are capable of accessing information presented in different formats, which requires the use of much more complex skills. • At this level, students relate scientific knowledge to daily situations that are of common occurrence in their context. • Students are capable of explaining their immediate world based on their own experiences and observations, and establish a simple and lineal relation with previously acquired scientific knowledge. • Students describe concrete and simple events involving cognitive processes such as remembering, evoking and identifying. • Students at this level have not been able to acquire the abilities required in Level I.

IV

704.75

III

11.40%

590.29

II

42.24%

472.06

I

38.72%

351.31

Below I

5.18%

42

Student achievement in Latin America and the Caribbean

Data on Science performance levels provide the grounds for grouping countries around three possible scenarios: • In Cuba, 65% of its students perform at Levels III and IV. • In Colombia, Uruguay and the Mexican State of Nuevo Leon, practically half their students perform at Level II. • In Argentina, El Salvador, Panama, Paraguay, Peru and the Dominican Republic, over 40% of their students perform at Level I or below.

TAbLE 16

PERCENTAGE OF SIXTH GRADE STUDENTS BY SCIENCE PERFORMANCE LEVEL IN EACH SURVEYED COUNTRY
Country Argentina Colombia Cuba El Salvador Panama Paraguay Peru Dominican Rp. Uruguay Nuevo Leon Total below I 5.32 2.62 0.26 3.78 6.34 7.20 6.97 14.29 1.69 2.59 5.18 I 37.73 31.68 8.78 44.73 44.60 46.18 46.93 62.82 22.76 30.98 38.72 II 43.04 51.09 25.92 42.55 39.89 38.11 39.36 21.50 48.47 47.78 42.24 III 12.73 13.59 30.31 8.23 8.40 7.52 6.37 1.37 24.01 16.38 11.40 IV 1.17 1.02 34.73 0.71 0.77 0.99 0.36 0.03 3.06 2.28 2.46

In terms of Science, students attending urban schools outperform their rural school 2 counterparts71 . Peru exhibits the greatest difference –in excess of 57 points– in Science performance between urban and rural schools. El Salvador and Panama follow with an approximate 40 point difference. Located at the opposite extreme is Cuba where no significant performance differences between urban and rural school students are evident. For its part, the Dominican Republic shows minimal differences that fluctuate around 11 points.

7

The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean represent a rough measure that, given the various definitions of rurality, should be taken with caution.

Executive summary

43

TAbLE 17

DIFFERENCE IN AVERAGE SCORES BETWEEN URBAN AND RURAL SCHOOLS, BY GENDER. SIXTH GRADE SCIENCE
Country Argentina Colombia Cuba El Salvador Panama Paraguay Peru Dominican Rep. Uruguay Nuevo Leon Total Urban/ Rural Difference 19.74* 22.83* 11.36 41.91* 38.27* 30.23* 56.18* 11.14* 29.28* 26.65* Girl/boy Difference -5.06 -18.93* 7.41 -10.16* 1.26 1.88 -16.12* -0.65 -4.44 -12.77* -11.52*

* Significant (5% confidence level)

Gender-based comparisons in the region reveal that boys have a marked advantage over girls, obtaining average scores that are 11.5 points higher. • In Colombia, El Salvador, Peru and the Mexican State of Nuevo Leon, boys’ Science scores are significantly higher than girls’. • By contrast, in Argentina, Cuba, Panama, Paraguay, the Dominican Republic and Uruguay, no statistically significant differences between girls and boys were detected. Student performance and the internal production of a country are directly related. National per capita GDP accounts for 11.57% of the variations observed in Science performance. Data seem to indicate that there is an inverse relationship between the learning of Science and income distribution inequalities. In fact, the Gini Index accounts for 30.68% of the national mean variances observed in Science performance.

44

Student achievement in Latin America and the Caribbean

foto pendiente
UNESCO/G.Tealdi

Factors associated with achievement

It will probably come as a ray of hope to all educational systems, that through the study of associated factors, SERCE has been able to corroborate the fact that schools are in a position to contribute importantly to student performance. While the socioeconomic dimension has a strong influence on performance, school-related variables can help significantly to reduce the learning inequalities associated with social inequity. In line with PERCE’s conclusions, the school climate variable was confirmed to have the greatest impact on student performance. It follows that, in order to promote learning among students, it is essential to provide a welcoming and warm environment based on mutual respect.

Executive summary

45

Collectively, the school resources variable also contributes to performance. While it is entirely possible that variables such as school infrastructure, basic services, the number of books in the school library, and the work experience of teachers, can only make modest individual contributions, as a whole, they can help substantially to encourage learning. The clear message behind this assertion is that resources are necessary elements to drive performance. School segregation based on the socioeconomic and/or cultural status of the student is the second most important variable that explains performance. Segregation seems to have a stronger impact on Reading than on Mathematics or Science. And, while this is not an education-related variable per se, any progress in this area will translate into important advances in students’ learning.

46

Student achievement in Latin America and the Caribbean

foto pendiente
UNESCO/D.Roger

Final reflections

Quality education must be seen as a right of all girls and boys. Attaining it represents a solid base for sustainable development, democratic progress and social equality. The SERCE embodies joint efforts undertaken by the Latin American and Caribbean countries and OREALC/UNESCO, aimed at enhancing educational opportunities for all students and, ultimately, promoting development in the region. The evaluations conducted within the framework of this Study, attempt to provide an analysis of what students learn, the inequalities that affect learning, and the factors that determine differential achievement.

Executive summary

47

On Primary Education student learning
In terms of academic performance, quality education is expected to lead to high levels of learning among all students, without exclusions of any kind. From SERCE’s perspective, equity is transversal since it focuses on social conditions that prevent from fully exercising the right to education, and on the way schools ensure a balanced provision of learning opportunities to their students. Significant differences in the quality of student learning are evident in the region. This can be observed across all areas and grades, as reflected by the dispersion of results within countries, and by the gaps in scores detected among participating countries. Thus, in connection with Third Grade education, the differences observed between the highest and lowest performing countries exceed 230 points, both in Reading and Mathematics. In terms of Sixth Grade, the differences although somewhat smaller, still exceed two standard deviations in Science and Mathematics, and rise to 174.5 points in Reading. This diversity affecting quality of learning can also be presented graphically by dividing participating countries into four groups, on the basis of their average test results.

TAbLE 18

COMPARISON OF THIRD GRADE SCHOOL RESULTS
Mathematics Cuba Chile, Costa Rica, Mexico, Uruguay and Nuevo Leon Argentina, Brazil and Colombia Guatemala, Ecuador, El Salvador, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic Cuba Argentina, Chile, Colombia, Costa Rica, Mexico, Uruguay and Nuevo Leon Brazil and El Salvador Ecuador, Guatemala, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic Reading

Difference relative to the regional mean Higher than the mean– more than one standard deviation Higher than the mean– less than one standard deviation Identical to the regional mean Lower than the mean– less than one standard deviation

48

Student achievement in Latin America and the Caribbean

TAbLE 19

COMPARISON OF SIXTH GRADE SCHOOL RESULTS
Matemática Cuba Reading Cuba Science

Difference relative to the regional mean Higher than the mean– more than one standard deviation Higher than the mean– less than one standard deviation Identical to the regional media Lower than the mean– less than one standard deviation

Argentina, Chile, Costa Costa Rica, Cuba, Rica, Mexico, Uruguay Brazil, Chile, Colombia, and Nuevo Leon Mexico, Uruguay and Nuevo Leon Brazil, Colombia and Argentina Peru Ecuador, El Salvador, Ecuador, El Salvador, Guatemala, Nicaragua, Guatemala, Nicaragua, Panama, Paraguay Panama, Paraguay, Peru and the Dominican and the Dominican Republic Republic

Uruguay and Nuevo Leon

Colombia Argentina, El Salvador, Panama, Paraguay, Peru and the Dominican Republic

It should be noted that in countries occupying the second and next to last position in the distribution scale, in practically all cases, mean scores differences are slightly above one standard deviation. This would point to a greater homogeneity among countries occupying mid-positions on the performance scale. Science constitutes a special case, since here standard deviations between the upper and lower extremes rise to 2.35 points, while intermediate results show a standard deviation of 0.68, indicative of greater homogeneity in this segment, and a substantial difference versus the extremes. This diversity within countries is also made evident when comparing differences between students in the 10th and 90th percentiles. On this basis, four country categories may be established, both for Third and Sixth Grade Primary Education students, namely: 1) Countries where the dispersion range between highest and lowest performance levels is less than 200 points; 2) Countries that exhibit variability between 10th and 90th percentiles in the 200 - 250 point range; 3) Countries with a performance dispersion range of more than 250 points but less than 300 points, and 4) Countries that exhibit an internal variability in excess of 300 points In connection with scores obtained by Third Grade students, differences fluctuate between 165 and 341 points in Mathematics, and between 183 and 296 in Reading. Cuba, Uruguay and Paraguay exhibit the highest internal dispersions in Mathematics and Reading, while Nicaragua shows the lowest.

Executive summary

49

TAbLE 20

COMPARISON OF SCHOOL RESULTS DISPERSION FOR THIRD GRADE STUDENTS, BY COUNTRY
Mathematics Colombia, Ecuador, the Dominican Rep., Guatemala, El Salvador, Panama and Nicaragua Brazil, Uruguay, Argentina, Mexico, Chile, Costa Rica, Peru and the Mexican State of Nuevo Leon Nicaragua Reading

Difference between 90th and 10th percentiles Less than 200 points

Between 200 and 250 points

Between 251 and 299 points 300 and over

Paraguay Cuba

Paraguay, Mexico, Uruguay, Argentina, Brazil, the Dominican Rep., Costa Rica, Chile, Colombia, Panama, Ecuador, El Salvador, Peru, Guatemala and the Mexican State of Nuevo Leon Cuba

In connection with Sixth Grade students, average performance differences between students in the 10th and 90th percentiles fluctuate between 182 and 385 points in Mathematics, and 176 and 387 points in the case of Science. Once again, Cuba shows the highest dispersion in all three areas, while the Dominican Republic exhibits the lowest internal dispersion in the aforementioned areas and grade.

TAbLE 21

COMPARISON OF SCHOOL RESULTS DISPERSION FOR SIXTH GRADE STUDENTS, BY COUNTRY
Mathematics The Dominican Rep., Nicaragua, El Salvador, Panama and Guatemala Colombia, Paraguay, Brazil; Costa Rica, Argentina; Ecuador, Chile, and the Mexican State of Nuevo Leon Mexico, Peru and Uruguay Cuba Reading El Salvador, Nicaragua and the Dominican Rep. Uruguay, Mexico, Brazil, Chile, Paraguay, Costa Rica, Peru, Panama, Ecuador, Guatemala, Colombia, and the Mexican State of Nuevo Leon Argentina and Cuba Science The Dominican Rep., and El Salvador Argentina, Colombia, Uruguay and the Mexican State of Nuevo Leon

Difference between 90th and 10th percentiles Less than 200 points

Between 200 and 250 points

Between 251 and 299 points 300 and over

Cuba

While Cuba shows the highest dispersion and the Dominican Republic the lowest, these findings should be carefully interpreted. On the one hand, scores obtained by the lower
50
Student achievement in Latin America and the Caribbean

performing Cuban students are similar to those of the average Latin American and Caribbean students. This fact places lower performing Cuban students much farther ahead than the rest of the region’s student population. On the other hand, the Dominican Republic exhibits the lowest results of the surveyed countries while the minimal score dispersion would seem to indicate that the results obtained by these students are generally low. In short, the results yielded by these two countries illustrate that, on the one hand, high general performance and high variability are not mutually exclusive and, on the other, that there are cases where results may be equally distributed but nevertheless learning levels remain low. In qualitative terms, this diversity affecting the quality of learning of Latin American and Caribbean students is reflected in the distribution of performance levels. Analyses based on performance levels give an in-depth view of what students are capable of doing in each of the surveyed grades and areas. SERCE classifies students’ achievements into four performance levels (I through IV) of increasing complexity. Each level is made up of a set of tasks the students will be tested on. In order to solve these tasks students must master specific contents and apply distinct cognitive process. An ideal distribution would show that most students perform at the higher levels. However, results do not generally conform to this pattern and while over 20% of the students in the region do, in fact, perform at the higher levels in practically all areas and grades (except in Science, where the percentage drops to 13.8%), there is an important number of students who are unable to perform beyond Level I: more than 40% in Third Grade Mathematics and Science; 32% in Third Grade Reading; and more than 15% in Sixth Grade Mathematics and Reading. The implication is that these students can only attempt the tasks that SERCE has defined as having the lowest levels of complexity. For instance, while more than 40% of Cuban students perform at the highest level, in every area and grade, there are other countries where approximately 50% of their students perform at or below Level I, in every area and grade. These results give insight into the region’s learning gaps and underline the importance of going beyond average scores when discussing educational quality, in order to identify and understand what students know and are capable of accomplishing. These findings shed light on the challenges that must be surmounted to enhance the quality of instruction imparted in Primary Education, and highlight the serious learning inequities that persist in the region. The equitable distribution of learning across different social strata remains a pending task a) The economic conditions of countries, particularly income generation and distribution, have a bearing on primary Education student learning. In order to explain this assertion, SERCE has analysed the existing link between student average performance, per capita Gross Domestic Product, and the Gini Index for each

Executive summary

51

country. Due to the unavailability of relevant data, Cuba and the Mexican State of Nuevo Leon have not been included in this analysis. Data confirm the existence of a positive correlation between the average scores of a given country and its per capita GDP. However, many countries obtain results beyond what their internal production would have predicted, which indicates that while resources are important they are not the only factors that determine student performance. Country by country analysis of average performance versus the Gini Index, shows an equally significant but inverse relation. In other words, the higher the income distribution inequality the lower the average student performance exhibited by Latin American and Caribbean students. b) Student gender has an impact on SERCE’s results Consistent with other studies on gender-based student performance, the present Study corroborates differences in most countries favouring girls in Reading and boys in Mathematics. Exceptions can be found in the Dominican Republic and Cuba where girls outperform boys in Third Grade and Sixth Grade Mathematics. In terms of Science, four participating countries show differences skewed in favour of boys, while in the remaining six countries no significant gender-based differences are evident. c) School location influences student achievement Within the region, the location of schools is also responsible for generating differences in student performance. In Latin America and the Caribbean, rural school boys and girls show lower levels of performance when compared to their urban school counterparts. These inequalities become sharper in some countries. The greatest differences in performance favoring urban school students –in both areas and grades surveyed– can be found in Peru, while the smallest differences attributable to the geographic location of schools were evident in the Dominican Republic and Cuba. In terms of Science, the greatest inequalities related to location are found in El Salvador and Panama. Conversely, Peru and the Dominican Republic, show the smallest differences. An analysis of student distribution by performance levels corroborates the existence of these gaps. There are clear differences, both at the regional level and within the countries, relative to the percentage of students occupying each of these performance levels that depend almost exclusively on whether the student is attending an urban or rural school. Moreover, in urban schools, performance distribution seems to have shifted to the next upper level, vis-à-vis rural schools. As a result, the percentage of students performing at Levels II, II and IV is systematically higher in urban schools, while at the lower levels (I and below I) there is a larger percentage of rural students represented.

52

Student achievement in Latin America and the Caribbean

The school does make a difference
In what undoubtedly constitutes an encouraging message to all education systems, SERCE has been able to corroborate through its study of associated factors that schools can, in fact, contribute importantly to student performance. While socio-economic factors are known to have a significant effect on performance, school-related variables can have a substantial impact on reducing the learning inequalities associated with social disparities. In line with PERCE’s findings, school climate was found to be the single most important variable conditioning student performance. Hence, generating a friendly and positive environment based on mutual respect becomes an essential strategy to foster student learning. As a whole, school resources variables also contribute positively to student performance. While the contributions made by school infrastructure, availability of basic services, number of books comprising the school library, and the teaching experience of educators is, at best, modest when taken individually, collectively these variables represent a valuable help to student learning. The key message derived from this finding is that resources are indeed necessary to drive performance. School segregation based on the students’ socio-economic and cultural status is the second most important performance conditioning variable. Segregation has been shown to have a stronger impact on Reading as opposed to Mathematics and Science, and although this is not an educational variable per se, any efforts aimed at reducing it will greatly influence student learning and achievement. Clearly, the Second Regional Comparative and Explanatory Study conducted by the Latin American Laboratory for Assessment of the Quality of Education, has contributed important information and knowledge to inform the decision-making process in matters concerning social and educational policies in Latin America and the Caribbean. Each of the countries participating in the Study must now retrieve the main lessons derived from this important inquiry.

Executive summary

53

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.