Comision Simce Anexo2 Forster

A NEXOS
LA EXPERIENCIA INTERNACIONAL
EN S ISTEMAS DE M EDICIÓN:
ESTUDIO DE CASOS
M ARGARET F ORSTER
G ILBERT A. V ALVERDE
Comisión para el Desarrollo y Uso del Sistema

de Medición de la Calidad de la Educación
D ICIEMBRE 2003
La Experiencia Internacional en Sistemas de Medición:
Estudio de Casos
Comisión para el Desarrollo y Uso del Sistema de Medición
de la Calidad de la Educación
ISBN 956-292-081-X
Ministerio de Educación, República de Chile
Alameda 1371, Santiago
www.mineduc.cl
Diciembre 2003
PRESENTACIÓN
La Comisión para el Desarrollo y Uso del Sistema de Medición de la Calidad de la Educación

recibió entre sus encargos recopilar insumos que permitan conocer los modelos de sistemas
de medición que se están utilizando en diferentes países, y cuáles son los beneficios y proble-
mas que se asocian a dichos modelos. Ello, con el fin de rescatar aquellos aspectos que se
consideren adecuados para ser incorporados de acuerdo a la realidad de nuestro país.
Nace así la iniciativa de contar con dos consultores internacionales expertos en el área de
medición a nivel de sistemas educativos. Ellos serían los encargados de entregar información
para que las recomendaciones de la Comisión tuvieran en cuenta los inconvenientes y solu-
ciones que se han encontrado en otros países del mundo.
Para cumplir con esta tarea, la Comisión contó con el apoyo de Margaret Forster –investiga-
dora del Consejo Australiano para la Investigación Educativa (Australian Council for
Educational Research, ACER)– y de Gilbert Valverde –investigador de Albany, Universidad
Estatal de Nueva York–.
El trabajo realizado por estos expertos fue dado a conocer en el Seminario Internacional sobre
Sistemas de Medición y Uso de Resultados1, en el que participaron como expositores. Los
principales planteamientos presentados en dicho seminario fueron completados y
profundizados en documentos elaborados especialmente para la Comisión. Estos documen-
tos son Assessment systems: two case studies, de Margaret Forster y La política en evaluación
y currículo ante el desafío de la calidad, de Gilbert Valverde.
A continuación, se presentan dichos documentos, ellos constituyen los anexos que acompa-
ñan al documento Evaluación de Aprendizajes para una Educación de Calidad, elaborado por
la Comisión.
ÍNDICE
P RESENTACIÓN 05
LA POLÍTICA EN EVALUACIÓN Y CURRÍCULO ANTE EL DESAFÍO DE LA CALIDAD 09

G. V ALVERDE
Introducción 11
T ENDENCIAS MUNDIALES EN LA OPERACIONALIZACIÓN DE CRITERIOS DE CALIDAD 13
Fijando criterios para la evaluación de la calidad 15
E STUDIO DE CASOS 19
Países Bajos 21
Una prueba longitudinal del aprendizaje para monitorear 22
el impacto de políticas
Un modelo mixto y voluntario de pruebas de egreso en 23
educación primaria
Un modelo mixto y obligatorio de pruebas de egreso de 23
educación secundaria
Estado actual de la discusión 24
Dos casos en los Estados Unidos 26
Nueva York 29
Perfil general del sistema de evaluación actual 29
Sistema actual de pruebas de egreso 31
Sistema de responsabilización 32
Tennessee 34
Perfil general del sistema de evaluación actual 34
El sistema de evaluación del valor agregado 35
Algunas lecciones ilustradas por los casos 37
Obras Consultadas 39
A SSESSMENT SYSTEMS : TWO CASE STUDIES 45

M. F ORSTER
Executive summary 47
Background 51
Introduction 52
C ASE S TUDY 1: W ESTERN A USTRALIA (WALNA & MSE) 55
Contextual features 57
Geographic, demographic, economic, political and religious context 57
Responsibility for schooling in Australia 58
Provision of schooling in Australia 59
Funding for schooling in Australia 60
Common and agreed national goals for schooling in Australia 65
The Provision of schooling in Western Australia 66
Student intake 67
Funding 68
Systemwide assessment 69
Different programs for different purposes 69
WALNA 70
MSE 73
Educational impact: positives 75
System level monitoring 75
School level: use of results and acceptance of program 77
Educational impact: concerns 88
C ASE S TUDY 2: E NGLAND (N ATIONAL C URRICULUM A SSESSMENT ) 93
Contextual features 95
Geographic, demographic, economic, political and religious context 95
Responsibility for schooling in England 95
The provision of schooling in England 96
Funding for schooling in England 96
Agreed goals for schooling in England 98
Changed processes of school management 99
Target setting 100
Student intake 100
Funding 105
Systemwide assessment 107
Educational impact: positives 109
System level monitoring of standards and initiatives 109
School level use of results 109
What evidence is there that these strategies have improved student learning? 115
Educational impact: concerns 117
Final reflections 121
References 122
Useful websites 123
ASSESSMENT SYSTEMS:
TWO CASE STUDIES
M ARGARET F ORSTER
A USTRALIAN C OUNCIL FOR E DUCATIONAL R ESEARCH
T HIS PAPER HAS BEEN PREPARED BY THE A USTRALIAN C OUNCIL FOR E DUCATIONAL R ESEARCH (ACER)
FOR THE C HILEAN M INISTRY OF E DUCATION TO INFORM DISCUSSION WITH THE N ATIONAL
C OMMITTEE FOR THE DEVELOPMENT AND BETTER USE OF THE N ATIONAL A SSESSMENT S YSTEM .
EXECUTIVE SUMMARY
The management of an education system is a complex and expensive operation, and effective
management demands dependable information on educational outputs. Systemwide (state
or national) assessment programs provide this information through the systematic and regu-
lar measurement of student learning. These programs are designed primarily to investigate
and monitor the “health” of the system and to improve student learning by providing
information to stakeholders at different levels of the system: policy makers, schools, teachers,
parents, students and the general community.
Of increasing interest internationally is the role that systemwide assessment programs can
play as agents of reform and accountability–to provide both direction and motivation to
schools, teachers, parents and students. In some countries, student achievement data collected
through state assessment programs are used as a measure of schools’ contributions to student
learning. In some, schools are “rewarded” or “punished” on the basis of students’ results. In
others, public comparisons of schools’ achievements in the form of “league tables” are made.
Using assessment programs to drive change directly brings some challenges. First, there can
be unintended negative consequences of assessment driven reform. Second, using assessment
programs as both the instrument of change and as the measure of change makes procedures
open to scrutiny from two perspectives and takes them into the arena of public debate both
about their effectiveness and about their reliability and validity as measures.
The two case studies explored here–Western Australia’s state Literacy and Numeracy Assessment
(WALNA) and Monitoring Standards in Education (MSE), programs; and England’s National
Curriculum Assessment–provide two different approaches to the collection and use of student
achievement data, in the context of reform and accountability demands.
Western Australia has two parallel assessment programs. The WALNA, a full cohort (census) program
administered annually to students in years 3, 5 and 7, provides information on students’ knowledge,
skills and understandings in numeracy, reading, spelling and writing. The purpose of WALNA is to
provide parents and caregivers with information about the performance of their children in relation
to nationally agreed “benchmarks”, and in relation to that of other Year 3, 5 or 7 students across
Western Australia; teachers with information on the performance of individual students and class
groups, and on the level of their students’ literacy and numeracy development in relation to the
performance of other Year 3, 5 or 7 students; and schools with summary information of their overall
performance.
The MSE program is administered annually to a representative sample of students in each of

Years 3, 7 and 10 in one or two of the eight curriculum learning areas. (Aspects of learning areas
48 A SSESSMENT SYSTEMS : TWO CASE STUDIES
that are assessed through the WALNA testing program are not assessed through the MSE
program.) The MSE assessments are part of the Department’s public accountability procedures
and the results provide a basis for initiating developments that will further improve the outcomes
of education in government schools. The purpose of the MSE program is to provide information
to stakeholders about the progress students make in their learning; assist the Department and
schools to monitor and report student performance; and assist the Department and schools to
plan for improvement in student performance.
England has a single systemwide assessment program. The National Curriculum Assessment, a
full cohort (census) program, is administered annually to students at the end of keys stages 1, 2
and 3; that is, at ages seven, 11 and 14. The tests provide information on students’ knowledge,
skills and understandings in mathematics, reading, writing and spelling and except at key stage 1
in science. The purpose of the tests is to give teachers information about their students’ attainment,
for use in planning teaching and learning; to provide information to the Government, to local
education authorities (LEAs) and to the public in general about standards in education nationally
and locally; and to encourage school accountability.
There are many similarities between the Western Australian and the English assessment systems,
in particular the intention to address a number of purposes for assessment and to provide
information of different kinds to a range of stakeholders. Both systems also express a key
intention to impact directly on the work of the classroom teacher.
Both systems:
• assess aspects of English (reading, writing, and spelling), and aspects of mathematics at
key stages of schooling;
• address explicit learning outcomes in the development of their tests;
• monitor performance against expectations detailed as described levels of achievement; and
• require teachers and schools to use systemwide achievement data in their planning.
The reform and accountability demands addressed directly by the assessment systems are
different, however: The stakes are significantly higher in England than in Western Australia.
The level of expectation is higher in England. When the standards were established in England,
63 per cent of students were above the standard, compared with 85 per cent in Australia.
Target setting has been introduced in England as a management and motivational device.
Schools are required to set targets for individual students stating what they are to learn next.
They must also set targets for the school as a whole. One of these targets has to be in terms of
the proportion of students who will achieve level 4 in the National Curriculum tests in
English.
The test results are high stakes for politicians and their opponents as well as schools, teachers
and students. In 1997 the Government set an overall target that 80 per cent of 11-year old
children should achieve level 4 or higher in English by the year 2002 (an increase of 17 per
cent over five years). The politician responsible, the Secretary of State for Education and
Employment promised to resign if the target was not met.
The underlying model for the national assessment system is one of competition between
schools with the publication of unadjusted results (“league tables”). The model assumes that
pressure from parents will assist the government to raise standards.
In other words the central difference between the Western Australian and English systems is
the degree to which the programs are driven by direct accountability pressures. In Western
Australia, although schools need to use the systemwide assessment data in their planning, the
stakes are not high and there are no public school-by-school comparisons. Rather, the
government has moved quietly forward to influence teacher practice by sending the message
that all areas of the curriculum are valued, that assessment materials will be made available as
a model for teachers, and that support will be provided to assist schools to use their data more
effectively. The use of two parallel and complementary assessment programs has also assisted
this process. And although this is partly historical (MSE was in existence before the National
agreement to undertake full cohort testing), it is still a strategy worth contemplating.
The difference in reform and accountability pressures also has impacted on the debates
surrounding the assessment systems. In England debate has focused on the validity of
the tests for formative assessment purposes, the distortion of the curriculum and the
misleading use of league tables leading to calls for the publication of value-added measures.
This is understandable given the pressure of national targets, and the public nature of
comparisons. Western Australia has so far avoided a heated debate about curriculum
fidelity and is instead exploring issues of equity, and ways to better support teachers in
their classroom practice.
Looking at the changes in school practice in the last couple of years in both countries one
might ask whether the debates are ones now held by primarily by academics and researchers,
rather than by teachers and the community more generally. There is some evidence that in
both countries teachers and schools have accepted the assessment programs and are now
getting on with the job of using data more effectively.
Looking forward and continuing to compare the two systems one might reasonably ask in a
few years time whether the accountability pressures have made enough of a difference to be
worth the initial upheaval. Or whether a less demanding set of expectations can achieve the
same goal of improving teacher practice and student learning outcomes.
And taking the big view, as noted by Whetton et al (2000), the experience in England illustrates
that the setting and maintenance of standards is a social and societal process. Assessment is a
social activity that we can understand only by taking account of the social, cultural, economic
and political contexts in which it operates. While the setting of standards, for example, relies
on empirical information, the information must be interpreted, and the integrity and expertness
of those making the judgements must be accepted publicly and politically (Gipps, 1999).
BACKGROUND
The Chilean Ministry of Education has appointed a National Committee to revise the National
Assessment System so that it is better aligned with the purpose of improving education. This
National Committee is providing advice to the Minister of Education in particular about
ways to improve the communication of the assessment system’s results to different stakeholders.
To achieve its objectives, the Committee is gathering information about assessment models
used worldwide and considering the benefits and problems of each model with a view to
improving the Chilean assessment system.
This paper contributes to the Committee’s deliberations by providing two detailed case studies:
England’s National Curriculum Assessment, and the Western Australia’s (WA) state Monitoring
in Education (MSE), and Literacy and Numeracy Assessment (WALNA) programs. The
paper provides a complete description of these assessment systems, including the social and
educational contexts in which they operate, and evaluates the contribution that each makes
to the education system. In addition detailed bibliographic references and possible sources of
information about the systems and tests are provided.
The paper draws on information from websites, research papers, public documents, discussions,
and previous research undertaken in preparation for the publication of ACER’s Policy maker’s
guide to systemwide assessment programs (Forster, 2001) and Principal’s guide to the use of
systemwide assessment data (in final draft form).
INTRODUCTION
The management of an education system is a complex and expensive operation, and effective
management demands dependable information on educational outputs. Systemwide (state
or national) assessment programs provide this information through the systematic and regu-
lar measurement of student learning. These programs are designed to investigate and moni-
tor the “health” of the system and to improve student learning by providing information to
stakeholders at different levels of the system:
• policy makers with information to monitor standards over time, to monitor the impact
of particular programs, and to make decisions about resource allocation; and
• schools (principals, councils) and teachers with information about whole school, class
and individual pupil performance that they can use to make decisions about resource
allocation and to support learning in the classroom.
Full-cohort (census) programs also provide:

• parents with information about their child”s progress to assist them to make decisions
about the best ways to support their child; and
• students with information about their progress to assist them to take an active role in
their own learning.
Of increasing interest internationally is the role that systemwide assessment programs can
play as agents of reform and accountability–to provide both direction and motivation to
schools, teachers, parents and students. In some countries, student achievement data collected
through state assessment programs are used as a measure of schools’ contributions to student
learning. In some, schools are “rewarded” or “punished” on the basis of students’ results. In
others, public comparisons of schools’ achievements in the form of “league tables” are made.
The theoretical appeal of this intention is easy to understand. The influence of assessment
programs in motivating educational change is well known. When results matter, (when the
“stakes are high”) assessment programs influence what is taught by reinforcing aspects of the
curriculum; and they engage schools, teachers, and the community in educational debate.
The political appeal is understandable also:
Compared with reforms such as targeting instructional time, professional development for
teachers, and reducing class sizes, state assessment programs are relatively inexpensive. The
assessments also can be mandated (unlike changes in classroom practice), can be rapidly
implemented, and have a public visibility (Linn, 1995).
However, using assessment programs to change the education system more generally brings
some challenges. First, there can be unintended negative consequences of assessment driven
reform. For example, the impact of the minimum competency testing movement in the United
States is well known (Marion & Sheinker,1999). Second, using assessment programs as both
the instrument of change and as the measure of change makes procedures open to scrutiny from
two perspectives and “takes them into the arena of public debate both about their effectiveness
and also their reliability and validity as measures” (Whetton et al, 2000, p.5).
This paper explores two systemwide assessment systems: Western Australia’s (WA) state
Monitoring in Education (MSE), and Literacy and Numeracy Assessment (WALNA)
programs; and England’s National Curriculum Assessment. Despite their different political,
social and educational contexts, there are many similarities in the Western Australian and the
English assessment systems, in particular the intention to address a number of purposes for
assessment and to provide information of different kinds to a range of stakeholders. Both
systems also express a key intention to impact directly on the work of the classroom teacher.
The reform and accountability demands addressed directly by the assessment systems are
different, however with the stakes significantly higher in England than in Western Australia.
C ASE S TUDY 1
WESTERN AUSTRALIA
(WALNA & MSE)
CONTEXTUAL FEATURES1
Geographic, demographic, economic, political

and religious context of Australian schooling
Australia is a large island continent located close to south east Asia and the South Pacific. The
land mass totals 7.7 million square kilometres and much of the interior is extremely arid. The
vast majority of people live in the major cities on the coasts.
The total population in 1996 was approximately 18,311,000, with an annual population
growth rate of 1.2 per cent. Around 80 per cent of the population live on narrow coastal
strips covering less than four per cent of the total land area. As a result, Australia has a high
level of urbanisation. Aboriginal people and Torres Strait Islander people, the indigenous
population, account for around 1.6 per cent of the total population and are spread throughout
urban and rural areas. There is considerable diversity of cultures within both Aboriginal and
Torres Strait Islander communities and particular efforts are made to provide for their special
educational needs and to assist in cultural maintenance.
In recent years, Australia has had one of the highest levels of immigration of any developed
country. By 1996, for example, approximately 24 per cent of the population had been born
overseas. Since the early 1970s, as the population has grown, the proportion of the population
of school age has decreased. The International Labour Organisation unemployment rate for
Australia (persons aged 15 years and over) for 1999 was 7.2 per cent (7.3 per cent for men;
7.1 per cent for women). Australia is secular, having a wide variety of religious affiliations,
including Christians (mainly Catholics and Anglicans), Buddhists, Muslims, Hindus and
Jews. The Catholic church is the largest provider of school education, other than the State.
The Australian political system includes an elected federal government and elected governments
of the six States (New South Wales, Victoria, Queensland, Western Australia, South Austra-
lia, Tasmania) and two self-governing Territories (Northern Territory and the Australian Ca-
pital Territory). Voting is compulsory.
1 This section of the paper draws directly from two texts:

• the 2000 National Report on Schooling in Australia which can be found on the Ministerial Council on Education, Employment, Training and Youth Affairs
(MCEETYA) website http://www.curriculum.edu.au and
• the Archive of the International Review of Curriculum and Assessment Frameworks Project (INCA) http://www.inca.org.uk/
Responsibility for schooling in Australia
The Constitution of Australia allocates responsibility for school education to State and Territory
governments, all of whom provide and manage government schools as well as supporting
non-government schools. Government schools operate under the direct responsibility of the
relevant State or Territory Minister, while non-government schools are established and operate
under conditions determined by State or Territory government registration authorities. Many
non-government schools have some religious affiliation, most commonly with the Catholic
Church; 19.9 per cent of all students and 63.6 per cent of non-government students were
enrolled in Catholic schools in 2001.
The Ministers of Education in the States and Territories have responsibility for primary and
secondary schooling, student enrolment policies, curriculum content, course accreditation
and certification procedures, and student assessment. Each of the State/Territory Governments
is responsible for determining teacher qualifications, establishing and paying teachers’ salaries,
recruiting and appointing teachers in government schools and supplying buildings, equipment
and materials.
In cooperation with the States and Territories, the Federal government through the
Commonwealth Department of Education, Science and Training (DEST) plays a significant
role in addressing equity and quality issues through general recurrent, capital and specific
purpose programs. In addition, it has specific responsibilities for Aboriginal people and
migrants and is responsible for international relations in education.
The key national body is the Ministerial Council for Education, Employment, Training and
Youth Affairs (MCEETYA) which comprises the Commonwealth, State, Territory and New Zealand
Ministers with responsibility for the portfolios of education, employment, training and youth
affairs, with Papua New Guinea having observer status. MCEETYA meets on a biannual basis to
discuss issues of mutual interest and to coordinate collaborative policies and approaches and generally
to facilitate the exchange of information on education in Australia and overseas. MCEETYA is
supported by taskforces that are convened as needed for particular tasks. These taskforces have
prescribed timeframes and reporting arrangements and are reviewed annually.
MCEETYA has established a range of consultative mechanisms at Federal and State levels to
obtain the views of the non-government school sector and, whenever possible, extends membership
of its working parties and committees to the major national bodies representing the non-government
employing authorities: the National Catholic Education Commission and the National Council
of Independent Schools Associations. Regular consultation also takes place with the principal
national bodies representing parents teachers and the business sector. Two national research and
development companies also facilitate cooperative initiatives in schooling: the Australian Council
for Educational Research (ACER), an independent research body, and the Curriculum Corporation
which is wholly funded by Commonwealth and State Ministers of Education.
Provision of schooling in Australia
Schooling in Australia is compulsory for children aged 6 to 15 (16 in the state of Tasmania)
years of age. However, the majority of children start school before they are six and remain at
school beyond the age of 15. In most States, children start full-time schooling at around the
age of five, when they enrol in a kindergarten or preparatory year. Commonly, the majority
of these students will have already had some part-time school or preschool experience. After
the preparatory year, primary education lasts for either six or seven years, and secondary
schooling for five or six years, depending on the State. Students usually commence secondary
school at about age 12.
In 2001, there were approximately 1.9 million primary school students and 1.36 million
secondary school students in Australian schools. Approximately 72.4 per cent of primary
students were enrolled in government schools, and 63.7 per cent of secondary students in
government schools. Most government schools are coeducational, but a significant number
of non-government schools are single-sex schools.
Some features of the structure of Australian schooling in 2001 were as follows:
• There were 9596 schools in Australia.
• There were 3.268 million full-time students in Australian schools.
• The proportion of students enrolled in non-government schools continued to rise in

all States and Territories. In 2001, 31.2 per cent of students were enrolled in non-
government schools compared to 30.8 per cent in 2000.
• There were 115,465 Indigenous students enrolled in Australian schools in 2001. This
represented 3.53 per cent of the total school population and was a significant increase
from 2.13 per cent in 1990.
• In 2001, 49.9 per cent of Australia’s 15-19-year-olds were full-time students at school.
• The largest year cohort of primary school students was in Year 1 (138,576 students)
and the largest year cohort of secondary students was in Year 8 (131,507 students).
• There were 221,927 teaching staff (in full-time equivalents) employed in Australian
schools, which resulted in an average of 14.8 students per teacher in government schools,
15.9 per teacher in Catholic schools and 12.8 per teacher in independent schools.
• Females comprised 78.7 per cent of the teaching staff in primary schools and 54.8 per cent
in secondary schools, compared with 78.3 per cent and 54.4 per cent respectively in 2000.
Funding for schooling in Australia
Within each State and Territory, ministers, departments, statutory authorities and individual
schools (particularly in the case of non-government schools) variously determine policies and
practices and provide resources for infrastructure in areas such as curriculum, course
accreditation, student assessment and certification, resource allocation and utilisation, and
teacher employment and professional development.
Through DEST, the Commonwealth provides funding to both government and non-government
school authorities to support agreed priorities and strategies. The overall result is that government
schools receive the majority of their government funding from State and Territory governments
and less from the Commonwealth, while non-government schools receive the majority of their
government funding from the Commonwealth, and less from the relevant State or Territory.
Funding is provided also from fees, charges and levies paid by students and their parents or
caregivers and private donations and income, including that derived from fundraising.
Figure 1 Public funding for school education

Source: 2000 National Report on Schooling in Australia Amounts in AUD
In 2000, the Commonwealth provided around AUD $4.96 billion for Australian schools
and students. This comprised:
• $3.91 billion for general recurrent grants (representing 79 per cent of specific-purpose
Commonwealth funding for schools for the calendar year 2000)
• $454 million for targeted programs (9 per cent), including $217 million in grants for
literacy and $103 million for special education
• $309.7 million for capital grants (6.3 per cent)
• $192 million for Indigenous programs (4 per cent), including IESIP and the Indigenous
Education Direct Assistance Programme
• $91.7 million for student support through Assistance for Isolated Children (AIC) and
ABSTUDY (2 per cent). See Figure 2 below.
In total, Commonwealth capital expenditure for schools in Western Australia was AUD $30.6
million.
Source: 2000 National Report on Schooling in Australia Amounts in AUD

In the financial year 1999-2000 the total per capita expenditure on government schools was
$AUD 6 358 ($5,687 for primary students; $7,416 for secondary students.)
Non-government schools derive their income from fees and donations, and State and
Commonwealth government grants. Per capita expenditure on non-government schools in
the financial year 1999-2000 is shown below in Table 1.
Source: Derived from data provided by Commonwealth DETYA , 2000 National Report on Schooling in Australia
Of the $4.78 billion of grants for schools spent under the States Grants (Primary and Secondary
Education Assistance) Act 1996 and the Indigenous Education (Supplementary Assistance) Act
1989, the government sector received $1.77 billion and the non-government sector $3 billion.
A further $10 million was spent on joint national programs benefiting both sectors.
Table 2 below shows how funding from the States Grants (Primary and Secondary Education
Assistance) Act and the Indigenous Education Supplementary Assistance Act was distributed
between government and non-government authorities, by State.
Note: Expenditure with respect to a certain program year may continue in relation to that year in future years. Source: 2000
National Report on Schooling in Australia, Commonwealth DETYA (2000 program year cash expenditure as at 30 June 2000)
Common and agreed national goals for schooling in Australia
In 1989 the Education ministers from the States and Territories and the national
(Commonwealth) Government published a set of “Common and Agreed (National) Goals
for Schooling in Australia”. Known as “the Hobart Declaration”, these goals provided a structure
for cooperation between schools, States, Territories and the Commonwealth, named eight
Key Learning Areas (the arts; English; health and physical education (HPE); languages other
than English (LOTE); mathematics; science; studies of society and environment (SOSE);
and technology) and described ten national goals for schooling.
In 1999 the State, Territory and Commonwealth Ministers for Education endorsed a revised
set of National Goals, “the Adelaide Declaration2 See and affirmed their commitment to
national reporting on comparable educational outcomes in literacy; numeracy; student
participation, retention and completion; vocational education and training in schools; science;
and information technology. (The Ministers also noted the need to develop performance
indicators for civics and citizenship education, and enterprise education.)
The literacy and numeracy goal is described as follow: “students should have attained the
skills of numeracy and English literacy; such that, every student should be numerate, able to
read, write, spell and communicate at an appropriate level”. To achieve this goal Ministers
agreed to support:
• assessment of all students by their teachers as early as possible in the first years of
schooling;
• early intervention strategies for those students identified as having difficulty;
• the development of agreed “benchmarks” for years 3, 5, 7 and 9, against which all
student’ achievement in these years could be measured;
• the measurement of students’ progress against these benchmarks using rigorous State-
based assessment procedures;
• progress towards national reporting on student achievement against the benchmarks,
with reporting commencing in 1999 within the framework of the annual National
Report on Schooling in Australia; and
• professional development for teachers to support the key elements of the plan.
By 2000 performance standards for literacy and numeracy at years 3, 5 and 7 were developed
and the Ministers agreed that benchmarking for years 9 or 10 be postponed pending findings
from the OECD Program for International Student Assessment (PISA) project. The
benchmarks represent “the minimum acceptable standard of numeracy without which a student
will have difficulty making sufficient progress at school”. They were developed with reference
to levels of achievement as demonstrated in national surveys and State assessment programs
through an extensive consultation process with stakeholders and with experts in the areas of
numeracy and educational measurement and trial testing in classrooms in all States and
Territories. Existing State-based programs are used for assessment against the national
2 See http://www.detya.gov.au/schools/adelaide/index.htm
benchmarks and a nationally agreed procedure to equate State and Territory tests ensures that
reporting of student achievement data is comparable by States and Territories.
Professional development for teachers is an integral part of the National Literacy and Numeracy
Plan, as it is recognised that the classroom teacher is the major determinant of the literacy
learning of students. During 2000, there was considerable professional development related to
the assessment and intervention programs described above. In particular, teachers were assisted
to interpret data from assessment programs and devise programs based on their findings.
Many teachers were involved in professional development programs associated with the
introduction of particular intervention programs. As many schools had instituted literacy
teams, a number of the professional development initiatives were directed towards the team
leaders. A train-the-trainer approach was common in such programs.
Another professional development strategy was the establishment of literacy networks among
teachers. The increasing use of modern communication technology contributed significantly
to the extent and effectiveness of such strategies. Many authorities also provided opportunities
for teachers to engage in postgraduate, accredited study in literacy teaching and learning.
The provision of schooling in Western Australia
In Western Australia under the provisions of the School Education Act 1999, every person
living permanently in Western Australia and aged between six and 15 years must be enrolled
at a government school or a non-government school, or receive approved home tuition.
Government schools provide schooling during the period of compulsory attendance to any
person who wishes to enrol. In addition, access to optional schooling is available for four-
year-old and five-year-old children and persons aged over 15.
The Education Department is the largest employer in Western Australia employing the
equivalent of 25 000 fulltime staff and accounting for a quarter of the State budget. The
Department is responsible for the education of 258 000 students at 771 schools across the
state from metropolitan Perth (the capital) to rural and remote centres. Over 1300 students
receive approved home education.
In the Perth metropolitan area and major rural and remote centres, government primary
schools provide sessional kindergarten, optional full-time pre-primary and full-time primary
education (Kindergarten to Year 7 or Primary to Year 7), and high, middle and senior high
schools and senior campuses full-time secondary (7-12, 8-10, 8-12 or 11-12) education. A
small number of early childhood education centres and junior primary schools offer Kinder-
garten to Year 2, or Kindergarten to Year 3 education respectively.
Outside the major rural and remote centres, smaller primary schools, district high schools
and remote community schools are provided. Kindergarten to Year 10 (and, in some cases,
Kindergarten to Year 12) schooling is delivered by district high schools supported by
“telematics” learning systems which integrate the use of personal computers, interactive
television, the Internet, fax and telephone. The remote community schools deliver Kinder-
garten to Year 10 education with the support of communications technology to Aboriginal
children, for many of whom English is a second language or second dialect. A number of
campuses deliver specialised services to male youths experiencing social or behavioural
difficulties. Education support schools and centres cater for students with disabilities for
whom specialised school environments are most appropriate. Students with mild to severe
intellectual disabilities attending rural or remote schools are supported by the provision of
equipment and learning materials and, for their teachers, professional development and
assistance with the design of individualised education programs. Delivery of schooling to
geographically-isolated students involves the Schools of Isolated and Distance Education
(SIDE); small Kindergarten to Year 7 and Kindergarten to Year 10 schools in remote areas;
and a combination of face-to-face, interactive and traditional distance education teaching
and learning.
Student intake
Government schools in Western Australia cannot exclude students from enrolment within
their intake area. The School Education Act 1999 1/Jan/2001 states under section 78
“Enrolment of children of compulsory school age at local-intake school”.
• A child of compulsory school age is entitled to be enrolled at a local-intake school if his

or her usual place of residence is in the intake area for that school; and an appropriate
educational programme is available for the child at that school.
• A child of compulsory school age whose usual place of residence is not in the intake
area for a local-intake school is entitled to be enrolled at that school if there is available
for the child at that school an appropriate educational programme and classroom
accommodation, and the enrolment would conform with any other criteria prescribed
by the regulations for the purposes of the subsection of the Act.
Schools must give priority to students who are within their intake area. If there are spaces left,
they can take students from outside the school intake boundaries. Schools can also apply to
have “special programs” recognised and funded by the State (for example, a program for
gifted and talented students). Schools delivering these programs can set aside a number of
places that can be filled from within or outside of the school intake boundaries.
Funding
Schools receive funding primarily on the basis of the number of pupils enrolled but also on
the basis of location (metropolitan, rural, remote) school type (for example, primary, secondary,
education support) and need (for example, low achieving schools in the monitoring program
receive additional funding).
Funding is transferred to schools through the School Grant at the beginning of each semester in
February and July on the basis of census returns. The second (July) payment includes all remaining
allocations based on staffing and enrolment figures given in the second census return.
Additional “gateways” have been included in April and October to allow transfers for special
program funding to schools throughout the year.
The School Grant Operation Manual provides all funding details, including special resource
funding such as “site characteristic” payments (specific site management costs); “school
characteristic” payments (costs associated with the school’s location and community, repair
and replacement of major equipment); and “student characteristic” payments (including
Aboriginal and Islanders Education Officers professional development, education support,
education support transport, wheelchair modifications).
Also addressed in this operation manual are access to the school “development” grant and
other “specific professional development” funds. These include, for example, professional
development for Aboriginal and Islander education officers, and remote community school
professional development.
SYSTEMWIDE ASSESSMENT3
Different programs different purposes
The Western Australian Department of Education conducts two systemwide assessment

programs:
• the Western Australian Literacy and Numeracy Assessment (WALNA) program; and
• the Monitoring Standards in Education (MSE) program.
The WALNA is a full cohort (census) program administered annually to students in years 3,
5 and 7. The assessments address explicit curriculum learning outcomes in numeracy, reading,
spelling and writing.
The purpose of WALNA is to provide:

• parents and caregivers with information about the performance of their children in
relation to nationally agreed benchmarks in numeracy, reading, spelling and writing,
and in relation to that of other Year 3, 5 or 7 students across Western Australia;
• teachers with information, not only on the performance of individual students, but also on
that of class groups, and on the level of their students’ literacy and numeracy development
in relation to the performance of other Year 3, 5 or 7 students. This information can be used
in conjunction with their own class records in future teaching and learning programs;
• schools with a report showing summary information of their overall school performance.
This data is useful additional information in the schools’ identification of students’ needs.
Through the, schools are also provided with assistance in interpreting their results for
school improvement and accountability purposes.
• The MSE program is administered annually to a representative sample of students in
each of Years 3, 7 and 10 are tested in one or two of the eight learning areas. Aspects of
learning areas that are assessed through the WALNA testing program are not assessed
through the Random Sample assessment program. The assessments are part of the
Department”s public accountability procedures and the results provide a basis for initiating
developments that will further improve the outcomes of education in government schools.
The purpose of the MSE program is to:

• provide information to stakeholders about the progress students make in their learning;
• assist the Department and schools to monitor and report student performance4; and
• assist the Department and schools to plan for improvement in student performance.
3 This section of the paper draws information primarily from three sources
• the Western Australian Education Department Annual Report 2001-2202 http://www.eddept.wa.edu.au/AnnualReport/index.htm
• information provided on the Western Australian Department of Education websites for the two system-level assessment programs http://www.eddept.wa.edu.au/
walna/ and http://www.eddept.wa.edu.au/mse/
• A paper evaluating the 2002 system level assessment program prepared for School and System Performance Directorate, Department of Education, Western
Australia by Murdoch University and Estill and Associates (Murdoch University, Estill and Associates, 2003).
4 The Department also produces “School Assessment Materials” for the key learning areas. These materials assist schools to fulfill their accountability requirements
as well as provide diagnostic information to teachers on the strengths and weaknesses in individual student learning. This information can help teachers plan for
student progress through the curriculum. Some of these documents are available in PDF on the Department website: http://www.eddept.wa.edu.au/mse/materials.html
WALNA
The WALNA full cohort (census) assessment program commenced in 1998 with the assessment
of Year 3 students in literacy (defined as reading, writing and spelling) and expanded in 1999
to include Year 5 students and the assessment of numeracy. In 2001 the program was further
extended to include Year 7 students. In 2002 almost all students in Government schools in
Year 3, Year 5 and Year 7 (approximately 61,500 students) undertook the WALNA tests in
literacy and numeracy5. In addition, around 22 000 students from Catholic and independent
schools now also participate in the testing program. The annual budget for WALNA is $AUD
1.3 million. Around 40% of the budget is spent in the hand-marking operation of the writing,
the open ended reading and numeracy questions and the hand marking of spelling.
The WALNA program was developed in response to the National Literacy and Numeracy
Plan. The tests, which address both outcomes in the WA Outcome and Standards Framework
(1998)6 and the National Benchmarks, are administered annually in August (the school year
runs from February to December).
The timeline for the 2002 WALNA program for all WA schools involved –government and
private– is provided in Table 3.
(Source: Murdoch University and Estill & Associates, 2003).
5 Parents/caregivers can exercise their right to withdraw their child or children from the testing program. This can be carried out through the completion of an
exemption form, available from schools prior to testing. Exemptions are also granted, at the school level, on the grounds of disability or impairment. Decisions to
exempt students are made by the principal, after discussion with the classroom teacher and school-based specialists, and with the signed agreement of the parents/
caregivers. Some English as Second Language (ESL) students in mainstream classes, as well as those in Intensive Language Centres, may also be granted exemptions.
In 2002, less than 1% of exemptions were lodged on the grounds of temporary or permanent disability, and 0.2% of children were withdrawn from the testing
program by their parents.
6 All Department of Education schools are required to plan and structure their learning and teaching programs using this Curriculum Framework.
Extensive consultation with classroom teachers is a feature of the development of the assessment
materials and the tests are written to cater for the diverse range of students in Western Australian
schools to ensure that there is no systematic bias associated with such factors as gender,
culture or geographic location. The numeracy and reading components of the test take between
30 and 60 minutes, and the writing component takes 65 minutes (though children only
spend up to 35 minutes writing). The spelling component takes about 20 minutes. In most
cases, the regular teacher conducts the assessment tasks as part of the normal school program.
Samples of the WALNA literacy and numeracy tests can be found on the WALNA website
at:http://www.eddept.wa.edu.au/walna/pdfs/sample%20magazine.pdf and
http://www.eddept.wa.edu.au/walna/pdfs/sample%20test.pdf
A team of experienced teachers completes all marking and to ensure reliability, markers undergo
extensive training. To further enhance reliability, all marking is conducted at a central location.
Markers work in groups under the direction of a team leader, and reliability scripts are administered
at each session. Each group leader monitors the marking, and provides advice as necessary.
The tests are designed to:

• provide an indication of how students are performing in relation to the National
Benchmarks,
• measure student performance in relation to the Outcomes and Standards Framework and
• provide comparative information on student performance in relation to State perfor-
mance.
Data are analysed using the Rasch Measurement Model. This model allows the abilities of
students and the difficulty of the assessment tasks they have completed in an area of learning
to be brought onto a single scale. Each scale provides two pieces of information: a continuum
of skills and understandings in the area of learning, based on the assessment items, arranged
in order of increasing difficulty, and the distribution of students in relation to the tasks.
This method of scaling allows inferences to be made about the probability of the success of
any student on all individual items. Each item can also be mapped to the levels of achievement
described in the WA standards framework (the Student Outcome Statements) making it
possible to identify areas of the sale that relate to each level. In turn, this makes it possible to
draw inferences about the level of achievement of groups of students. (Appendix 2 provides
an explanation of this process.)
Three types of report are provided:
• Individual student reports for parents and caregivers

These reports are sent from the school as a supplement to normal semester two reporting
procedures. They are a confidential communication between the school and home.
• Class reports for teachers

Each class report provides a total score for the test (eg reading) and a detailed analysis of
student performance on each test question. Schools are provided with one copy of each
class report. The reports contain significant diagnostic information and assist teachers
to identify students who would benefit from extension, as well as students not meeting
the minimum expected standard.
• School reports for principals

These reports provide school performance summaries and shows trends in student per-
formance in reading, writing, spelling and numeracy. They also show the results for all
Year 3, 5 and 7 students. (For examples of the reports see Appendix 3.) These data
provide information for school planning.
Students who are exempted are shown as being below the benchmark in the school (and
system) reports. While displayed on school reports, exempted students are not attributed a
score and therefore are not included in the calculation of the school or state mean (average).
Students who are withdrawn by parents are reported as a parent withdrawal in the class
report. These children would generally not have disabilities of a temporary or permanent
nature that would have prevented them from demonstrating benchmark performance, therefore
these children are not designated as being below benchmark. Students who are absent for
part of the testing program receive a report detailing performance on only those components
completed. Students who are absent for the entire testing program are not included in the
reporting process7.
Results are provided electronically to schools. Schools receive both the individual school and
the individual student results and are assisted through the Data Club to interpret and use
their results (see Education Impact: positives). The key findings for the State as a whole and
by district are available on the Department website
7 In 2001, for the first time, schools were required to register the name of students who were absent from school for the entire testing program. Fewer than 3% of
students were registered as absent during the WALNA assessment week. In 2002, 3.2% of students were registered as absent during the WALNA assessment week..
MSE
The MSE program began in 1990. Each year a random sample of approximately ten percent of
students in each of years 3, 7 and 10 in government schools participate in the program8. Schools
are notified of their involvement in the weeks prior to the assessments. Schools decide the days
and times to administer the assessments during the specified week (week 8 of term 3).
Assessments occur in all key learning areas, as well as in the social outcomes of schooling, and
address the significant learning outcomes (knowledge, understandings, skills and processes)
that students are expected to develop as they move through the compulsory years of schooling.
These learning outcomes are described in the Outcome and Standards Framework. In most
cases, all sequenced strands of a learning area are assessed. For example, in the 1997 Random
Sample assessment of Science, Year 3, 7 and 10 students were assessed in Life and Living,
Energy and Change, Natural and Processed Materials, Earth and Beyond and Investigating
Scientifically. Some learning areas, such as Mathematics and English, are assessed more regularly
than others so that trends in performance to be closely monitored. For example, aspects of
the English learning area have been tested in 1992, 1995, 1997, 1999 and 2001.
The annual budget for the Random Sample program is almost $AUD 1 million. This budget
covers the development or administration of three learning areas (two being assessed and at
least one under development), and the production and on-going provision of school release
to schools.
The MSE program uses rigorous procedures to train markers to make consistent judgements
of student performances. Markers are trained to use standardised marking guides, in
conjunction with student work samples, to judge student work.
As with the WALNA program, MSE data are analysed using the Rasch Measurement Model
and single vertical scales from Year 3 through to Year 10 provide the basis for reporting.
Student performance is reported using the eight levels of achievement described in the Student
Outcome Statements. Aggregated information on the performance of students in Years 3, 7
and 10 is reported. Within these year groups, information on boys, girls, Aboriginal and
Torres Strait Islander (ATSI) students and Language Backgrounds Other Than English
(LBOTE) students is also reported. For each of these groups, the mean performance and the
range of performance across the state are described in terms of the levels of the Outcomes
Framework.
MSE produces public reports for the Western Australian government education system . The
reports present information descriptively, in table form and through graphs. In most reports,
samples of students’ work from the assessments are included, to exemplify level achievement.
Examples of MSE reports are included in Appendix 2.
8 Year 3 marks the end of the early childhood phase of schooling, Year 7 the end of the middle childhood phase and Year 10 the end of the compulsory years of
schooling, and the end of the early adolescent phase.
MSE releases a selection of the random samples materials free of charge to WA Government
schools. The intention is that the materials will provide models of good assessment tasks, assist
teachers to set their own assessment criteria, and develop teachers’ capacity to make judgements
about students’ work using the Student Outcome Statements. Depending on the nature of the
learning area, the released materials consist of stimulus materials (print/audio/video), student
test booklets, teacher administration instructions, marking keys with moderation instructions,
guidelines for analysing results, and performance profiles or progress maps.
At the classroom level, teachers use the results to:

• diagnose individual students’ strengths and weaknesses in terms of actual skills and
understandings that the student does or does not demonstrate on the assessment;
• determine the strengths and weaknesses of the class as a whole;
• determine the strengths and weaknesses in the teaching of aspects of the curriculum;
• obtain an indication of level achievement in the substrands described in the Curriculum
Framework
At the whole school level, school administrators use the results to:
• infer level achievement in the substrands described in the Curriculum Framework;
• make comparisons with the state mean;
• make comparisons with the state range;
• track year cohorts longitudinally;
• track groups of students over time;
• report to the school community;
• inform school development plans; and
• provide continuity for students between schools (primary/primary and primary/high)
School administrators and teachers are reminded that the MSE assessments do not cover the
entire curriculum and that the results provide only one source of information. They are also
reminded that the administration instructions need to be followed carefully in order to maximise
standardisation and that the materials cannot be used to compare teachers and schools.
EDUCATIONAL IMPACT: POSITIVES9
System level monitoring
Since 1990, MSE data have facilitated the monitoring of student achievement at a system level.
Now, the combined data from WALNA and MSE allow for the monitoring against an expected
standard. Data from WALNA have also provided a check on the random sample findings.
The WA Department reports two major findings in relation to the standards:
1. Some minor fluctuations are evident in the data from year to year and in the different
strands but the overall trend emerging from the data is one of slight improvement.
2. The percentages of students achieving the standards in reading, writing and spelling
vary from 1998 to 2002. Variations of this sort appear in all large scale testing programs,
and the apparent improvements or declines should be interpreted cautiously. The
observed trend of slight improvement over the years of testing is encouraging.
The Department also reports the following findings in relation to previous MSE findings:
1. Girls continue to do better than boys in literacy at all years. The difference in perfor-
mance between boys and girls increases from Year 3 to Year 5 and from Year 5 to Year 7
and is most evident in writing.
2. Girls’ and boys’ numeracy results are comparable.
3. Aboriginal students’ literacy performance has shown a trend of improvement since

1998. While Aboriginal students’ performance is still below that of the total population,
there are indications that their rate of improvement in reading and writing is slightly
greater than that of the general population.
9 This section of the paper draws information and direct text primarily from the following sources
• the Western Australian Education Department Annual Report 2001-2202 http://www.eddept.wa.edu.au/AnnualReport/index.htm
• information provided on the Western Australian Department of Education website http://www.eddept.wa.edu.au/
• A paper evaluating the 2002 system level assessment program prepared for School and System Performance Directorate, Department of Education, Western
Australia by Murdoch University and Estill and Associates (Murdoch University, Estill and Associates, 2003).
• information provided electronically by Helen Wildy, Associate Professor of Educational Leadership, School of Education, Murdoch University
• an article written by Helen Wildy reviewing the work of the Data Club (Wildy, 2003)
• the Western Australian School Accountability Framework, 2002 http://www.eddept.wa.edu.au/regframe/index.cfm
• discussions with Jocelyn Cook, Manager Educational Measurement, Standards and Accountability, Education Department Western Australia
The following three tables show the percentage of students reaching the benchmark standards
at Year 3, Year 5 and Year 7.
School level: use of results and acceptance of the program
At the school level, evidence of the impact of WALNA data is confirmed by the work of the Data
Club, by an independent review, and by the requirements of the school accountability framework
The Data Club
The Data Club assists schools to use their WALNA information through an annual half-day
workshop program that the principal and one other person with curriculum responsibility
from each participating school attend. The Club began in 2000 as a pilot project with two
hundred volunteer schools and grew over the following three years until in 2003, 508 schools
(of a total of 612 government schools) were participating10.
In advance of the workshops WALNA data are re-analysed to provide participants with box
and whisker representations of school level performance on each of the four WALNA “strands”:
Reading, Writing, Spelling and Numeracy. Comparisons are made over time, as the data are
accumulated, and regression analyses are used to show expected growth or “value added” in
school performance in a given strand over time. Standardised residuals are used to indicate
the relative amount of change.
Workshop participants are introduced first to the box and whisker method of representing
data as well as to regression analysis, standardized residuals, confidence limits and examples
of valid and invalid inference. In the second part of the workshop participants are given
envelopes containing their processed data and are supported as they work through tasks
designed to explore their data.
10 The project began as a partnership between Edith Cowan University and the Western Australian Department of Education and Training, supported by the
Commonwealth Department of Education, Science and Training to develop schools’ capacity to make performance judgements in relation to the National
Literacy and Numeracy Plan. From 2003 the project was retained, managed and funded entirely within the WA Department.
Schools are encouraged to keep data from year to year and to keep records detailing the
particular conditions under which the assessment are conducted and other factors likely to
help interpret the data (for example, who supervised the tests, what changes had occurred in
classroom arrangements, teaching programs, and staffing allocations). Participants are
challenged to think about the data in relation to what is happening in their school. Most
importantly, they are urged to be cautious when interpreting data, as most variation is chance
variation rather than real upward or downward change.
The Club recognises that schools need support to understand the data they receive and generate,
and that with the turnover of leadership in schools there is a need for ongoing support. It also
recognises that the best support for interpreting data is a combination of information about
data analysis and representation strategies together with application of this information to
participants’ own data. The experience of the club confirms that schools need guidance about
how to use the information they receive particularly for whole-school decision-making in
relation to planning, resource allocation and performance reporting. There are dangers when
systems place pressure on schools to report against targets before schools have demonstrated
that the data used to set targets is even modestly understood. Without a reasonable proficiency
in interpreting data, and distinguishing between fair and valid accounts and whose which are
not fair and valid, efforts to generate school-level improvement through a focus on meeting
targets may be compromised.
An evaluation of the Data Club by an independent consulting group reports that educators
in schools are not only examining literacy and numeracy test data far more thoroughly than
before but less defensively. Principals are using their greater understanding of the data primarily
to inform whole-school planning11. This outcome illustrates the enormous impact of the
Club. In 2000 when schools nominated for the Data Club were asked to send their WALNA
data so it could be represented in ways that might help them to make use of it, the typical
response was “What data?” In 2000 not only were schools not using their data, they were not
even keeping it.
According to the managers of the Club the success of the project can be attributed to four
principles:
• school’s own data are the focus of the workshop;
• participation is voluntary;
• more than one school leader attends the workshop; and
• presenters are specialists and independent of the employer.
11 The report concludes that it is rare to find an educational initiative that has been as warmly received and as effective in changing practice in schools as the Data
Club. (Jane Figgis, Director of AAJ Consulting Group)
Independent review of WALNA
In 1999, in response to criticism of the WALNA test, an evaluation survey was conducted by
the Education Department. The survey report indicated that teachers’ criticisms of the length
and difficulty of the test were not supported by parents who regarded the test information as
an appropriate mechanism to help raise literacy standards. Four years later there was anecdotal
evidence that the WALNA program had become more accepted by teachers as the longitudinal
benefits to parents, teachers and schools become apparent. In light of this anecdotal evidence,
the Department of Education commissioned an independent evaluation study of the 2002
WALNA (Murdoch University & Estill and Associates, 2003).
The study addressed two main questions: What are the attitudes of Year 3, 5 & 7 teachers
and parents to WALNA five years after its commencement, and how have they changed since
1999? How is the information from WALNA used by schools?
To answer the first question, the evaluation re-administered a revised form of the teacher and
parent questionnaires from the 1999 WALA evaluation to allow comparison with the data
from that year. The second question was addressed by developing a new questionnaire for
principals, and by supplementing the teacher questionnaire with additional items.
In all, 458 principals, 1458 teachers and 1414 parents responded to the survey. Principals’ responses
were analysed as a total group and according to school type and size. Teachers’ responses were
analysed as a total group and according to school type and size, year level taught, and the number
of years teaching experience. Parents’ responses were analysed as a total group and according to
school type and year level of their child.
Overall, principals reported that they made considerable use of the WALNA information, found
it useful for teachers and parents, and had adopted it into their school’s performance measurement
and planning activities. However, principals in Remote Community Schools and Education
Support Schools did not have the same positive perceptions of the WALNA information.
Generally, teachers had positive perceptions about the WALNA program. The majority
understood the purposes of the program, felt they received sufficient information and support
to administer the tests and analyse the results, and were not overly concerned that the tests
were too stressful or too long for most students. The exception to this is teachers in Remote
Community Schools and Education Support Centres, who had consistently less positive
perceptions. Compared to teachers” perceptions in the 1999 survey, the current survey shows
a general improvement in teacher perceptions.
The perceptions of parents and caregivers of children who undertook the 2003 WALNA tests
were generally positive and had improved since the 1999 survey in nearly all areas.
Nearly all the parents agreed that the purpose of the test was to provide them with information
about the achievements of their children (90%) and to assist schools to provide better literacy
and numeracy teaching for all students (95%). The large majority also agreed that it provided
information about student weaknesses (81%). The majority (60%) thought that the test was
used to compare different educational systems; however, they disagreed (57%) that the purpose
of the test was to judge the quality of the teachers. They were evenly divided over whether a
purpose of the test was to judge the quality of schools. Nearly all parents (96%) agreed that the
report gave them information about the achievement of their child in relation to the benchmark
and that the report was clear and understandable (89%). About half the respondents (52%)
agreed that the school had provided additional information which helped them understand the
report. The large majority agreed that the report confirmed what their child’s regular school
reports were saying (74%) but thought that the report gave them additional information (73%).
The large majority of parents/caregivers did not believe that their children were unusually stressed
in the lead-up to the test (87%) or when completing the test (90%). Neither did they think that
the test was too difficult (80%) or too long (83%) for them. Rather they believed that their
child enjoyed participating in the test (74%) and were pleased with their test results (82%).
Only a few (12%) thought that their child was upset with their result. While they were divided
as to whether the test results had encouraged their child to work harder (54% agreed with this
statement), almost all (95%) disagreed that it had had a discouraging effect on them.
Both in 1999 and 2003 parents/caregivers and teachers generally agreed that the purpose of
the test was to provide parents with information about their children, and highlight student
weaknesses.
Role of the test in assisting schools to

provide better literacy teaching for all students
The 1999 report pointed out that nearly all parents/caregivers (92%) believed that the purpose
of the test is to help provide better literacy teaching and that the information from the test
program will help to raise literacy standards across the State. In contrast, a small majority of
teachers (58%) felt the results would be useful in this way. This difference was reduced
dramatically in 2003: as shown below, 73% of teachers valued the test information more as a
source of diagnostic information about their students, and used it to identify students at risk
and who required extension.
QUESTION: ASSIST SCHOOLS TO PROVIDE BETTER LITERACY TEACHING FOR ALL STUDENTS
The quality of the information provided
In 1999, the majority of parents believed that the reports confirmed regular school reports
but nonetheless supplied them with additional information. On the contrary, teachers surveyed
in 1999 agreed that the results were in line with school-based tests but only a minority of
teachers (37%) agreed that they provided valuable diagnostic information about their students.
This pattern has changed over the intervening period to the point where the majority of
teachers (62%) now value the test information, using it to identify students at risk or requiring
extension. A greater percentage of parents also believe the tests provide useful additional
information (62% to 73%).
QUESTION: THE REPORT GAVE ME ADDITIONAL INFORMATION QUESTION: THE TEST RESULTS PROVIDED ME WITH VALUABLE
NOT AVAILABLE IN THE REGULAR SCHOOL REPORT DIAGNOSTIC INFORMATION ABOUT MY STUDENTS
Reaction of students to the test
In 1999, considerably more teachers than parents/caregivers were of the opinion that students
were stressed before and during the test. In 2002, the proportion of teachers with this opinion
was reduced, though still considerably greater than that of parents/caregivers.
QUESTION: IN THE LEAD-UP TO THE TEST MY CHILD WAS QUESTION: IN THE LEAD-UP TO THE TEST SOME OF MY
UNUSUALLY STRESSED ABOUT PARTICIPATING IN THE TEST STUDENTS WERE UNUSUALLY STRESSED ABOUT PARTICIPATING
IN THE TEST
Confidentiality of results
In 1999, a large majority of teachers (70%) were concerned about the confidentiality of results,
a view not shared by many parents (23%). This concern was shown to be considerably reduced
in 2003, dramatically for teachers (decline to 38%) but also for parents (decline to 14%).
QUESTION: I AM CONCERNED THAT MY CHILD’S RESULTS QUESTION: I HAVE CONCERNS ABOUT THE
WILL NOT BE KEPT CONFIDENTIAL CONFIDENTIALITY OF TEST RESULTS
Overall, in the areas where there were differences between teachers and parents in 1999, the
differences have been dramatically reduced and both groups have more positive perceptions.
Compared to 1998, teachers in particular, have become more accepting of WALNA. Parents
and caregivers were generally positive and had more positive views in all areas than were
reported in the 1999 survey. With familiarity over the years and the help of the Data Club,
the WALNA data is accepted as a tool which can be used diversely, for example, by individual
teachers for judging students or by the schools for assisting with curriculum planning.
Feedback from principals, teachers and parents from Remote Community Schools and
Education Support Centres indicates they view the WALNA tests are not as appropriate for
students in these schools as they are for students in mainstream schools. This is because they
see the tests as more stressful for the students and more disruptive to their learning.
Nevertheless, there is some support for the view that despite this, WALNA can still play a
positive role in measuring progress in improving student outcomes over the years.
The 2002 survey results demonstrate that the WALNA is now accepted by the three key stakeholder
groups as part of the normal cycle of reporting on student numeracy and literacy outcomes.
The School Accountability Framework
The School Accountability Framework sets out the requirements for all schools in relation to
accountability and review.
All schools are required to produce, in partnership with their school community, a school
plan setting out their objectives, priorities, major initiatives and evaluation measures. They
are required also to assess their performance in terms of standards of student achievement
and the effectiveness of the school; and make available to the public and to the District
Director a School Report that describes the school’s performance. Central to this activity is
the use of achievement data collected through the systemwide assessment programs. School
staff are accountable to the principal and school principals are accountable to the District
Director for the performance of the school.
The Framework is built on five fundamental commitments: high standards; effectiveness;

quality teaching; quality information for parents about the standards being achieved by their
schools; information for the wider community about the standards being achieved across the
government schools system. The accountability policy is based on five key concepts.
1. Accountability involves responding to information about performance.

That is, accepting responsibility for the achievement of outcomes and taking appropriate
action to improve performance. Key accountability questions for schools include:
• What are we trying to achieve?
• How well are we achieving it?
• How can we improve?

2. Accountability is linked to outcomes.

The core purpose of schools is to enable the success of each and every student. Schools
therefore must be able to demonstrate to themselves and to the communities they serve
that what they do makes a difference to student learning. Since schools do not have
control over all the variables that impact upon student achievement, they cannot be
held accountable for it in an absolute sense. However, because the school can make a
difference to the level of achievement of the students, schools are accountable for
maximising the difference they make in terms of student outcomes.
3. Demonstrating accountability is a professional responsibility of school staff.

Schools need to take responsibility for demonstrating their effectiveness to others. That
is, accountability is located within the professional responsibility of the school staff, by
creating an expectation that staff demonstrate the quality of their work as part of the
expression of their professionalism.
4. Rigorous self-assessment is at the heart of effective accountability.

While schools and systems vary in the approach they take to school accountability, the
success of any approach ultimately depends on how well the school reviews its own per-
formance. Effective internal review involves the school gathering information about the
levels of student achievement, analysing and judging the adequacy of those levels,
identifying strengths and weaknesses in its overall performance, and assessing which aspects
of the school”s operation should be changed to generate improved performance.
5. Schools are accountable to their communities and to the Director-General.

The Framework acknowledges the two-way accountability of schools. Schools listen
and respond to their local community’s needs and provide information about the school”s
performance. They also respond to systemic priorities and requirements and provide
an assurance that every government school is meeting accountability requirements.
The planning model used by a school should reflect the needs of the school. That is, so
long as the plan enables the school to respond to the community’s needs, accommodate
Departmental initiatives, and demonstrate its accountability, schools have flexibility in
terms of the planning model they adopt.
The Framework makes the role of school self-assessment clear and the changed context of
teacher assessment explicit. In the past teachers have not used a common set of outcomes on
which to base their judgements of student performance. Establishing the comparability of
teachers’ judgements against the Outcomes and Standards Framework requires a great deal of
moderation activity within and across schools. Teachers will need to have opportunities to
discuss and share examples of what constitutes a particular level of achievement by a student.
Many schools already use Monitoring Standards in Education (MSE) test materials as part of
a process of moderating teachers’ judgements about levels of student achievement. The process
of fine-tuning teachers’ judgements will also be assisted by annual literacy and numeracy
benchmark testing for year 3, year 5 and year 7 students. As well as monitoring the perfor-
mance of students across the school and at particular year levels, school staff also need to
know about the performance of particular groups of students: perhaps the girls are doing
better than the boys or the performance of the Aboriginal students is different from that of
the other students.
Analysing achievement data will assist schools to target their improvement efforts at the
points where improvement is most needed. Having gathered some information about student
achievement levels, a key question in any self-assessment is whether that level of performance
is up to expectations. Reaching an answer to this question involves comparing the school’s
performance with a standard of some kind. The standard might be a State average; it might
be based on other schools with similar student populations: it might be a national benchmark;
or it might be the schools own performance in previous years. Schools will be assisted in this
process and will be provided with information about the performance of “like” schools on
national literacy and numeracy benchmark testing.
Having reviewed their performance in terms of student achievement, school staff also need to
assess whether the school is operating as effectively as it could so that the maximum value is
added to the students” learning. They are also required to produce a School Report that gives
parents and members of local communities a clear sense of the standards being achieved and
the schools” effectiveness in reaching the objectives set out in their school plans. Parents need
to have information reported to them that shows the progress being made in the agreed
priority areas of the school plan. Any information that could identify individual students
must not be published in School Reports. In general, information about school performance
should be communicated in such a way as to ensure that the maximum amount of information
is placed in the public domain while avoiding simplistic league tables based on narrow measures
of school performance. The kinds of information that are to be included in the report are
detailed in Appendix 4.
As a final accountability measure, reviews of schools by District Directors provide assurance

to all stakeholders that schools have conducted rigorous and comprehensive self-assessments,
reported the required outcomes in their School Reports and responded with appropriate
improvement plans. This validation enables those with an interest in a school to have confidence
in the quality of the school’s review and reporting processes. It also provides schools with
valuable feedback about their performance and directions for improvement.
EDUCATIONAL IMPACT: CONCERNS12
A number of issues relating to the identification, measurement and presentation of data have
emerged in Australia. Below is a summary of the issues that are currently under consideration.
Ethical principles underpinning reporting
The performance information that is collected and published has the potential to affect greatly
the public perception of school education as well as to influence the decision-making for
government and non-government schools. It is essential, therefore, that the presentation of this
information is underpinned by a clear set of ethical principles. During 2000, this matter was
considered by one of national working groups, which proposed that the fundamental ethical
principle should be “truth in reporting”. Such a principle requires that the information reported
should be accurate and comprehensive, reporting should not be selective or partial, and all
relevant information should be reported. This principle should also be followed when data are
presented in graphical and tabular forms. Two other supporting ethical principles were proposed:
• the community’s right to information; and
• the avoidance of unwarranted harm to members of the community.
The first of these is based on the assumption that the public has the right to access information
that has been collected by government for public purposes. However, this principle has to be
considered in the light of the second, which acknowledges that, because of the potential for
harm to occur to individuals and institutions, there can be no absolute right of publication.
Ensuring privacy
Most jurisdictions have in place legislation covering privacy and/or freedom of information. Typically,
this legislation refers to matters such as the collection of personal information, solicitation of
information, its storage and security as well as access to the information. The Australian National
Report operates within these provisions and also acknowledges that some jurisdictions have agreements
with particular organisations and sections of the community that need to be respected. The over-
riding protocol being observed is that no information should be reported that allows the identification
of individual students or schools. Where information is provided about individual schools, teachers
or students (for example, in some case studies) appropriate approvals are sought prior to publication.
12 This section of the paper draws directly from

• Appendix 4 the 2000 National Report on Schooling in Australia which can be found on the Ministerial Council on Education, Employment, Training and
Youth Affairs (MCEETYA) website http://www.curriculum.edu.au
Selection and reporting of performance measures
The Ministerial Council on Education, Employment, Training and Youth Affairs has defined
key performance measures as: A set of measures, limited in number and strategic in orientation,
that provides nationally comparable data on aspects of performance critical to the monitoring
of progress against the National Goals for Schooling in the Twenty-first Century.
Initial efforts to develop such data have focused on the more measurable aspects of students’
performance. However, it is expected that performance measures will eventually be developed
to cover the broad range of educational outcomes, even though this might be conceptually and
operationally more difficult in some areas. Some general issues surrounding the reporting of
performance measures include the following:
• Time-series data are preferable to single-year data and are included where possible with
descriptive comments regarding trends.
• Caution should be taken whenever data involves comparisons between jurisdictions,

sectors or institutions, as extraneous factors can influence the data and lead to readers
drawing false conclusions.
• Comparative data of this kind should draw the attention of the reader to all relevant factors.
• Data will also be provided, where possible, for students according to gender,
socioeconomic status and geographic location.
• International studies may provide good information about the performance of Australian
students and when used, they need to be accompanied by appropriate contextual information.
Data quality
It is intended that only data that are deemed to be of high quality will be published. This
requires clear data specifications which, while not always being published, must be available
from the organisations responsible for data collection. It is important that estimates of statistical
uncertainty are published with the performance measures and not in a separate appendix.
The usefulness of data will be enhanced by the publication of contextual information that
enables the reader to take a considered and critical view of the measure. Similarly, when
information is necessary to explain the data, caveats should be included and be placed as near
as possible to the data.
Census (full cohort) versus sample data
The latest edition of the National Report on Schooling In Australia contains data derived from
whole-of-cohort census measurement as well as data obtained from samples. Both kinds of
measurement can provide useful information for nationally comparable reporting. Whole-cohort
testing is likely to be both more expensive and intrusive than sample testing and these factors may
make sample testing the preferred option. It is also recognised that sample testing may make
possible the use of richer assessment tasks than are likely to be possible in census testing. However,
sample size must be sufficient to allow reporting by jurisdictions and by sub-groups. Stratified
samples are usually constructed to ensure adequate numbers of students in some sub-groups.
A similar situation exists when using survey data. Census data can be disaggregated to report
by jurisdictions and by sub-groups but are more expensive to collect. Sample surveys must be
constructed with sufficient sample size to allow reporting by jurisdictions and by sub-groups.
School authorities are anxious to limit both the cost and the intrusiveness of data collection
exercises and will frequently seek to use data that has been collected for more than one purpose.
Where sample surveys are conducted it is often necessary for authorities to over-sample some
sub-group populations so as to be able to report on them.
Nationally consistent definitions
The use of disaggregated data in performance indicators to provide information about parti-
cular sub-groups has the potential to raise awareness of problems that these groups may be
experiencing. However, it can also generate prejudice or reinforce stereotypes and therefore
needs care in presentation. The National Report on Schooling in Australia will attempt to
provide data for the following sub-groups, preferably using consistent definitions.
• Indigenous students
• students of language backgrounds other than English
• students with disabilities.
In order to report on the outcomes of student sub-groups, including educationally

disadvantaged students, in a nationally consistent manner, significant progress in developing
definitions that can be applied consistently across the country has been made.
Sex
In March 2000, Ministers endorsed recommendations on definitions for sex, agreeing that
the Australian Bureau of Statistics (ABS) standard for identifying students’ sex be adopted for
the purposes of nationally comparable reporting of outcomes within the context of the National
Goals for Schooling in the Twenty-first Century.
Indigenous students
In March 2000, Ministers endorsed recommendations on definitions for Indigenous students,

agreeing that the ABS standard for identifying Indigenous students be adopted for the purposes
of nationally comparable reporting of outcomes within the context of the National Goals for
Schooling in the Twenty-first Century:
Is this student of Aboriginal or Torres Strait Islander origin?
No
Yes, Aboriginal
Yes, Torres Strait Islander
At the end of 2000, jurisdictions were using variants of the standard identifier in enrolment
systems, and there were often differences between how Indigenous status was collected at
enrolment and at testing time. In the light of this, the Australian Bureau of Statistics began
negotiating with jurisdictions regarding its full introduction.
Socioeconomic disadvantage
In October 1999, ACER was commissioned to develop options for a common definition and
approach to the measurement of SES. The basic characteristics of the proposed approach
recommended by ACER are that:
• Socio Economic Status (SES) of students should be based on individual measures rather
than on aggregated area measures;
• SES should be based on parental occupation and education; and
• socioeconomic disadvantage should be measured using the health-card status of the
student’s parents.
Language background other than English
In November 1999, ACER was commissioned to develop a discussion paper on the

measurement of language background, culture and ethnicity for the reporting of nationally
comparable outcomes of schooling. When the findings and recommendations of the ACER
team were received in 2000, the use of the already agreed MCEETYA definition of a student
of language background other than English, was endorsed:
• born in a non-English-speaking country,
• or born in Australia, with one or both parents born in a non-English-speaking country,
• or an Indigenous student for whom English is not the first language.
In line with the ABS standards on cultural and language diversity, the following indicators of
language background of students were also endorsed:
• country of birth of student,
• country of birth of father,
• country of birth of mother, and
• main language other than English spoken at home.
Geographic location
In November 1999, a discussion paper was commissioned to explore a common definition

and approach to the measurement of geographic location of students for the purpose of
reporting outcomes against the national goals.
Students with disabilities
Work on the definition of students with disabilities commenced towards the end of 2000,
with the establishment of a sub-group to manage a project to develop a common approach to
defining and reporting on the achievements of these students.
In addition to these national considerations, concerns have been expressed about the quality of
inferences schools draw from data. Sometimes inferences (positive or negative) are drawn from
data when observed differences in test scores are insignificant. Differences in test scores between
groups, and rises or falls in test scores over time, may be insignificant due to measurement error
and/or small sample size. This is particularly the case in schools with changing demographics
and/or small schools. Concerns of this kind are addressed by the work of the Data Club.
C ASE S TUDY 2
ENGLAND
(NATIONAL CURRICULUM
ASSESSMENT)
CONTEXTUAL FEATURES13
Geographic, demographic, economic,

political and religious context of English schooling
England is part of the United Kingdom (UK), which consists of Great Britain (England,
Wales and Scotland) and Northern Ireland. England covers an area of 50,663 square miles.
The official language in England by custom and practice is English. In 1998, English was an
additional language for around eight per cent of students in primary and secondary education
in England. The established church in England is the Church of England, which is Protestant
Episcopal. In 1998, the population in the UK was 59.2 million.
The UK economy has become one of the strongest in Europe. The International Labour
Organisation (ILO) unemployment rate for the UK (persons aged 16 and over) in 1999 was six
per cent (6.7 per cent for men; 5.1 per cent for women) and employment is generally increasing.
The United Kingdom has a constitutional monarchy and the Sovereign is Head of State and
Head of Government. The Government comprises the Legislature (Parliament), the Executive
(the Cabinet, which consists of about 20 Ministers, who are usually Heads of the Government
Departments and are chosen by the Prime Minister) and the Judiciary. Parliament consists of
the Queen, the House of Lords and the House of Commons. Most of the work of Parliament is
conducted in the House of Commons, which is composed of 650 elected Members of Parliament.
Responsibility for schooling in England
Overall responsibility for all aspects of education in England lies with the Secretary of State
for Education and Skills. Responsibility for different aspects of the service is shared between
central government (the Department for Education and Skills (DfES), local government,
churches and other voluntary bodies, the governing bodies of educational institutions and
the teaching profession. The provision of publicly-financed education in “maintained” schools
is the responsibility of local councils, (county councils, district councils, and London boroughs)
which have designated responsibility as local education authorities (LEAs).
For LEA-maintained schools, the responsibility for financial and related administration,
maintenance of schools, and the appointment, management, appraisal and dismissal of teachers
has been delegated from LEAs to the school governing bodies of the individual schools.
Central authorities have the power to intervene if individual LEAs or the governing bodies of
educational institutions do not discharge their duties satisfactorily.
13 This section of the paper draws text directly from two sources:
• the Archive of the International Review of Curriculum and Assessment Frameworks Project (INCA) http://www.inca.org.uk/
• the Qualifications and Curriculum Authority website www.qca.org.uk
The provision of schooling in England
Education is compulsory for children between the ages of 5 and 16. Children must start
school in the term following their fifth birthday (unless alternative provision is made); however,
most children start school earlier. Young people can legally leave school and take a full-time
job on the last Friday in June in the academic year (September to July) in which they reach
the age of 16. Children may be educated at home but parents have a duty to ensure that
children of compulsory school age receive an efficient full-time education suitable to their
age, ability and aptitude, and to any special educational needs they may have.
England has both public and private schools within all sectors of the education system. Primary
schools teach 5-11 year olds while the secondary sector caters for 11-18 year olds, although in
some cases there may be 16-18 year olds being taught in sixth form or tertiary colleges. Table 7
below details the relationship between phase of education, type of institution, grade level and
typical age of students.
Funding for schooling in England
Most publicly-funded primary and secondary schools are maintained by LEAs. The LEAs
meet their expenditure partly from locally raised revenue and partly from general grants
received from central government. Most LEA-maintained schools receive 100 per cent funding
for both current and capital expenditure. Voluntary-aided schools receive 100 per cent funding
for recurrent costs and are eligible for 85 per cent funding for capital expenditure; the remainder
being the responsibility of the founding body, usually a religious organisation.
In addition, the “specialist schools programme”, which forms part of the Government’s plan
to extend choice and diversity, allows schools to specialise in a particular area of the curriculum,
such as modern foreign languages, or sciences and technology, while still delivering the full
National Curriculum. Such schools receive additional funding from the Government and
sponsors in industry who are represented on the school governing body.
Private education is provided in institutions that are largely privately-funded, receiving most
of their income from tuition fees. (There is private provision at all levels of education.) Private
sector primary and secondary schools in England are generally known as independent schools14
and, in addition to fees paid by parents may, in some cases, also be funded by donations and
grants received from benefactors. A private or independent school is “any school at which
full-time education is provided for five or more children of compulsory school age (5 to 16
years) (whether or not such education is also provided for children over or under that age)”,
and which is not maintained by a LEA. Voluntary schools, which are set up and, in many
14 Although private sector schools in England are usually known as independent schools, some long-established secondary private schools are known as “public
schools”.
cases, owned by a voluntary body, usually a church body, are considered to be maintained
schools rather than independent/private schools, since they are publicly-funded and may not
charge tuition fees.)
In 2001-01 total UK public expenditure on education was £44,108 million, representing 4.6
per cent of the gross domestic product (GDP).
Shaded= compulsory esducation

* in some areas, there are separate schools for key stage 1 and key stage 2, known as infant and junior schools respectively.
** in some areas of England, there are middle schools, which normally provide a four-year course for children aged between 8 and 12
years, or 9 and 13 years. In such cases, two-tier systems of primary and secondary schools exist along three-tier systems of firts schools
(for 5- to 8- or 9- year olds), middle schools and secondary schools (for 12- or 13- to 16-/17-/18 year olds).
Agreed goals for schooling in England
Since the 1988 Education Reform Act, all state schools in the UK conform to the teaching
guidelines laid down by the National Curriculum. Before the introduction of the National
Curriculum, schools had responsibility for their own curricula, with guidance from the LEA.
(The curriculum does not apply in independent schools although those schools may choose
to follow it.15) The curriculum, which is intended to raise teaching standards and give a
consistent level of education across all schools, defines educational entitlement for pupils of
compulsory school age.
The Act requires all state schools to provide pupils with a curriculum that:
• is balanced and broadly based;
• promotes their spiritual, moral, cultural, mental and physical development;
• prepares them for the opportunities, responsibilities and experiences of adult life; and
• includes, in addition to the national curriculum, religious education, and for secondary
pupils, sex education16.
There is a strong emphasis on inclusion and providing effective opportunities for all pupils.
Schools are directed at all levels of curriculum planning to set suitable learning challenges;
respond to pupils’ diverse learning needs; overcome potential barriers to learning and assessment
for individuals and groups of pupils. Information is given in “Setting suitable learning
challenges” about the curriculum flexibility available and the action that schools should take
to ensure that all pupils are presented with learning opportunities relevant to their attainments
to enable them to progress and to achieve positive outcomes. In “responding to pupils’ diverse
learning needs”, specific reference is made to the action necessary to ensure that all pupils are
enabled to achieve, including boys and girls, pupils with special educational needs, pupils
with disabilities, pupils from all social and cultural backgrounds, pupils of different ethnic
groups including travellers, refugees and asylum seekers, and pupils from diverse linguistic
backgrounds. In “overcoming potential barriers to learning and assessment”, specific reference
is made to the provision that should be made to meet the individual requirements of pupils
with special educational needs, pupils with disabilities and pupils for whom English is an
additional language.
15 Independent schools must be registered with the Department for Education and Skills (DfES) and are subject to school inspections, in accordance with the School
Inspections Act 1996 and the curriculum is one of the major aspects considered in a school inspection. Reports of such inspections are made public. Where an
aspect of the education provision is deemed to be unsatisfactory, improvements must be made and a school that fails to meet the required standards may be deleted
from the register of independent schools. In January 2000, there were just over 560,000 students in independent schools, representing around six per cent of the
total number of school students.
16 The national curriculum does not constitute the whole curriculum for schools. Schools have discretion to develop the whole curriculum to reflect their particular
needs and circumstances.
The core subject areas of the National Curriculum are English, Mathematics, Science, Design
& Technology, Information Technology and Languages. Schools will also offer a selection of
other subjects to provide a complete curriculum17. The content of each national curriculum
subject is defined in a statutory Order that consists of:
• common requirements which relate to access to the curriculum for all pupils, pupils’ use of
language, pupils’ access to information technology and the Curriculum Cymreig (in Wales);
• the program of study which sets out the minimum knowledge, understanding and
skills for each subject at each key stage; and the
• “attainment targets” which define the expected standards of pupil performance in terms
of level descriptions or end of “key stage” descriptions.
Details of the National Curriculum can be found on the National Curriculum website
http://www.nc.uk.net or on the Qualification and Curriculum Authority (QCA)website
(QCA): National curriculum at key stage 1 (5- to 7-year-olds): Years 1 and 2

http://www.qca.org.uk/ca/5-14/revised2000/edprim2a.asp
National curriculum at key stage 2 (7- to 11-year-olds): Years 3, 4, 5 and 6

http://www.qca.org.uk/ca/5-14/revised2000/edprim2b.asp
National curriculum at key stage 3 (11- to 14-year-olds): Years 7, 8 and 9

http://www.qca.org.uk/ca/5-14/revised2000/edsec2a.asp
Changed processes for school management
Other recent significant changes in the English education system have included the
• assessment regime accompanying the National Curriculum (see the next section of this paper);
• devolution of financial management of budgets to schools;
• introduction of regular inspections of schools with failing schools subject to special measures
and close monitoring with the ultimate threat of closure (Whetton et al, 2000);
• significant changes in the way Literacy and Numeracy is taught in Primary Schools. An
hour is dedicated to prescriptive activities for Literacy and then a further hour for
Numeracy with the content and pedagogic methods prescribed. (Teachers are provided
with support materials which can be found on the national curriculum website
http://www.nc.uk.net)
17 For more details regarding the National Curriculum and what is taught within the subject areas it is you can also visit the Department of Education and skills
(DfES) website www.dfes.gov.uk.
Target setting
Of particular relevance to this paper is the introduction of target setting as a management

and motivational device. Schools are required to set targets for individual students stating
what they are to learn next. They must also set targets for the school as a whole. One of these
targets has to be in terms of the proportion of students who will achieve level 4 in the National
Curriculum tests in English. (To support the achievement of these targets, a National Literacy
Strategy was introduced in 1998 and a National Numeracy Strategy in 1999.)
To begin the process in 1997, the Government set an overall target that 80 per cent of 11-year
old children should achieve level 4 or higher in English by the year 2002. In 1997 when the
target was set, 63 per cent of students were achieving at level 4 or higher, so an increase of 17 per
cent over five years was needed. The politician responsible, the Secretary of State for Education
and Employment promised to resign if the target was not met. The test results thus became
high stakes for politicians and their opponents as well as schools, teachers and students.
Student intake
All children in England between the ages of 5 and 16 must receive full-time education, either
at school or out of school18 . Parents have the right to select a school for their child and
schools have the right to accept or reject individual students.
The local education authority (LEA) has a duty to secure a free school place for all children
who are ‘of compulsory school age’ or to provide suitable education in some other way, for
example, home tuition. This duty usually applies to pupils who are temporarily living in the
area for long enough to be practical to attend school (for example, the child of a traveller or
a child whose parent is in the armed forces) or have come from abroad, or have special
educational needs. If a pupil is permanently excluded (expelled) from any school, the local
education authority has a duty to provide other suitable education. This may be a place in
another school, or the local education authority could place the pupil in a local special
educational unit, or provide home or individual tuition. Parents have the right to appeal
against a child’s exclusion.
18 (See the Education reform act 1988, for details of education policy http://www.hmso.gov.uk/acts/acts1988/Ukpga_19880040_en_1.htm)
Parent choice
Parents have the right to express a preference for their child’s school. They do not have to
express a preference for a school in their own local education authority and maintained schools
must not give priority to children simply because they live in the local education authority
area. The local education authority has a duty to try to comply with their choice, subject to
the school’s admission policy.
School choice
Schools use different methods to decide whether to offer a child a place. Criteria include such
considerations as whether the child has a brother/sister already attending the school; how far your
child lives from the school; whether the child lives within the ‘catchment area’ of the school;
whether the school is a religious school, and whether the child follows that particular religion;
whether the school gives priority to parents who express a preference for a single sex or co-educational
school; and whether the child has been excluded from two or more schools; and whether the
school selects its pupils on the basis of academic ability. (The local education authority or governing
body can refuse to admit pupils who have been excluded from two or more schools.)
It is accepted that parent choice has had an impact on real estate in England. Three newspaper
articles are included here by way of illustration.
TAKE PRIDE IN OUR STATE SCHOOLS, BUT MORE INVESTMENT IS STILL CRUCIAL LEADER,
SUNDAY MARCH 24, 2002 THE OBSERVER
Britain’s state schools are getting better. The best are becoming very good indeed. The evidence is all
around. In A-level and GCSE performance, state schools are rapidly closing the gap with even the top
private schools, where fees are more than £10,000 per year. Many state schools routinely outstrip the
performance of many private schools.
Now the public is starting to notice. Today, we report on how much families are paying to be in the
catchment area of good primary schools; new research shows that a 25 per cent differential in school
performance leads to a 21 per cent jump in house prices. In London and the south-east, the price of
homes near desirable schools can be £45,000 higher than those that are not. Parents are recognising that
£3,000 in additional mortgage payments each year can be much better value than school fees.
Good schools have always had an impact on local house prices, but as good state schools become ever
better, the impact is becoming more marked. And as recognition becomes more widespread, there will be
a cascade effect to the next tier of improving schools. The requirement that Oxford and Cambridge lift
their intake from state schools to 67 per cent will also force many middle-class parents to reassess whether
private schools are worth the fees.
The overriding imperative now should be to maintain the framework of policy that is producing these
results - the national curriculum, testing, league tables and better inspection, along with increased resources
and smaller class sizes. What still causes concern, however, is that schools in some underprivileged
neighbourhoods with low levels of owner-occupation might never benefit from this improvement. These
now need to be singled out for additional resources. Teachers need to be paid a premium and staff-student
ratios in these schools must be significantly better than those already in the virtuous circle of improvement.
The challenge for the long term is to create neighbourhoods where social disadvantage is less concentrated,
with a greater mix of public and private housing ownership. As we address Britain’s urgent need for more
homes during the next 20 years, the opportunity should be seized to build more mixed communities
with the resulting better schools. But these are good, not bad, problems to have. The most important
conclusion is that much state education is getting better.
PARENTS PAY £50,000 MORE TO LIVE NEAR A GOOD PRIMARY SCHOOL. FAISAL ISLAM AND
MARTIN BRIGHT, SUNDAY MARCH 24, 2002 THE OBSERVER
Parents are paying nearly £50,000 extra in property prices to guarantee their children places at top
primary schools, according to a government-funded study. ‘Selection by house price’ has led to the exclusion
of those on low incomes from the benefits of good local schooling, according to economists from London
University, who carried out the first nationwide study of its kind.
The findings confirm fears that leading state primary schools have become the preserve of middle-class
families who can afford properties near top-performing schools. The problem is particularly acute in
London, the South-East and the North. In these areas a 25 per cent difference in the results of primary
schools pushes up local house prices by 21 per cent. But the phenomenon is nationwide. Even in the
South-West, where house prices were least affected by primary school results, they still rose by 10 per cent
when there was a significant difference in school performance.
‘Lower-income home-owners will be priced out of the best school catchment areas,’ said Steve Gibbons,
co-author of the research to be presented to the Royal Economics Society’s annual conference on Wednesday.
‘The sensitivity of property prices to primary-school quality implies the existence of a back-door selection
of pupils by the incomes of their families.’
The research by Steve Gibbons of University College London and Steve Machin of the London School of
Economics is the first scientific study linking house prices in the Land Registry with school league table
performance, based on the proportion of children who reach the expected level in Key Stage 2 tests for
11-year-olds. The problem is most acute in London and the South-East, where a 25 per cent difference in
school performance will cost an average £37,000. But in some parts of London the extra cost could be
£45,000. Extra mortgage payments, up to £3,000 a year, are the equivalent of the cost of an average
private prep school.
School campaigners said the research proved what many parents suspected, and that moving house was
not an option for many people. Margaret Tulloch of the Campaign for State Education said: ‘This is
what happens when the education system is driven by market forces. Parents who can afford it will
operate choice to get their children into a good school.’ But teachers’ unions said parents were mistaken
to rely on league tables to work out performance. A spokeswoman of the National Union of Teachers
said: ‘We are well aware of this phenomenon and its impact on house prices. The league tables have
always given a limited presentation of the education provided in a school. Yet parents have come to
believe that they are true measures.’
Just how much will parents pay for good schools? An extract from the ground-breaking
research which quantifies the property effect of top-performing schools which has led to a new
form of selection by house price. Steve Gibson and Stephen Machin
Sunday March 24, 2002 The report Valuing Primary Schools will be presented at the Royal
Economic Society’s 2002 Annual Conference at the University of Warwick on Wednesday. In
this extract, the authors explain their research and its social and policy consequences.
From the introduction: Severe inequalities in the measured performance of English primary
schools across geographical space have parents clamouring to get their children into the best
schools. There is a lot of anecdotal evidence to suggest that parents are prepared to move
house to try to secure admission to a good school, and that they are often prepared to pay a
high premium on property prices. Stories of soaring house prices close to good schools are
commonplace. We have heard stories from Local Education Authority staff of complaints
and appeals by families failing to gain admission to a school of their choice, despite having
moved house specifically for that purpose. One anonymous interviewee spoke of an expectant
mother calling for advice on which streets she should consider moving to in anticipation of
her unborn child’s primary education. Another family, known to one of the authors, recently
sold a three bedroom Victorian terrace in north London for a much smaller semi-detached
house just over a mile away. This move cost them around £140000. For what net gain? A
35% increase in the proportion of children at the local primary school reaching the target
level in Key Stage 2 assessment tests.
For sure, these moves may buy more than just better schools - good schools are typically in
neighbourhoods that are better in other ways: lower crime rates, quieter neighbours, cleaner
streets, better local amenities. But some component of any premium paid for a re-location
from a bad-school neighbourhood to a good-school neighbourhood may well be attributable
to the price of an improvement in school quality. This phenomenon is by now widely recognised
in the US, and several attempts have been made to quantify it. For Britain, the issue has
received much less attention in the academic arena, despite being discussed a great deal in the
media, and amongst politicians and parents.
A first aim of this paper is to start to fill this gap. We estimate the premium attracted by
improvements in primary school quality in England, using property price data from the
Land Registry and the government’s school performance tables. This sample gives us near-
universal coverage of property transactions and school performance measures in England
from 1996 to 1999. We also carefully address important empirical issues that have been
largely ignored or overlooked in the existing literature.
Teenagers are a fairly mobile group and can travel long distances to school. Mobility between
Local Education Authorities is high for children in secondary education. In contrast, primary
age children typically attend schools which are within walking distance (at least in urban areas),
and catchment areas can shrink down to just a few blocks for those in the highest demand.
Our interest in primary schools also has a sound empirical and theoretical basis. We would
expect primary school performance to be the principal object of choice by parents seeking to
improve the life chances of their offspring. For a start, there is evidence that attainments in
the early years are positively correlated with later academic and economic success. If gains
made in the primary years reap rewards in terms of achievements at secondary school, then
the payoff for the investment is higher if the investment is made early on in a child’s life.
What is more, investment in good primary education may be a pre-requisite of admission to
selective secondary schools. Given the high fixed costs of moving house, a rational parent will
make a once and for all locational choice when their first child enters the education system.
Our focus in this paper is therefore upon the associations between local house prices and
primary school performance.
From the report’s conclusion: In this paper we ask how much parents are prepared to pay to
get their children into better schools by moving house. We use postcode sector level data on
house prices and primary school performance in England to estimate the magnitude of the
association between primary school quality and local house prices. We eliminate the effects
of catchment area wealth on pupils’ achievements by concentrating on the effects within
narrow geographical areas, and by instrumenting measured pupil achievements by
characteristics of the school itself.
Our best estimates imply a premium on postcode sector house prices of between 5.2% in the
West and South West and 8.4% in the East, South East and North for each 10% improvement
in the proportion of children reaching Key Stage 2, Level 4 at age 11. This translates into
monetary valuations of the order of £4500 for the West Midlands, £5,600 for the South
West, £5,800 for the North, £6,200 for the North West, £17,600 for London and £12,600
the South East (all at 2000 property prices).
Interestingly, our estimates of the primary school effect are of the same order as those obtained
for suburbs of Boston, Massachusetts by Black (1999). She finds that a 10% increase in
primary school mean test scores attracts a 5% property price premium.
Our lowest put the figure at around 3% for a 10% school improvement. The sensitivity of
property prices to local primary school quality implies the existence of a back-door selection of
pupils by the incomes of their families. This flies in the face of notions of equality of opportunity,
is likely to restrict intergenerational mobility and generates an inequality of educational outcomes
that may be unrelated to the abilities of children. If pupil ability is related to parental incomes
then selection by income is implicitly selection by academic ability.
Indeed, this goes against the principle in the DfEE code of practice on admissions (Section
5.6) that “academic ability should not be used to decide entry into primary education”. The
equilibrium arising from local sorting by incomes on primary school quality will be inefficient
if the net marginal benefits of state school quality are greater for lower income families.
This is almost certainly true given that the alternatives - private sector schooling, private
personal tuition - are available at lower marginal cost to wealthier families with sufficient
capital or lower borrowing costs. As usual with issues of educational equity, relaxation of
borrowing constraints is a fundamental issue here. Linking of property loans to current incomes
means that the marginal costs of borrowing become infinite at lower and lower purchase
price thresholds as incomes decrease. This is sensible given the need to match lending to
borrowers’ ability to repay the debt, but leads to exclusion of those on low incomes from the
benefits of good local schooling.
The primary objective for policy seeking to remove inequities and inefficiencies arising from
income-related selection on good state schools is to eradicate differences in primary school
quality across geographical space. Current government policy is to increase competition
between schools as an incentive for good performance. However, proximity-based restrictions
on admissions, together with the house price effects shown in this paper, mean that higher
income families will inevitably benefit the most. Lower-income home-owners will be priced
out of the best school catchment areas. More public information on school performance
differences could exacerbate this problem, though there seems to be no evidence that house
prices are more sensitive to school quality over the years that the school performance league
tables have been available.
Valuing Primary Schools by Steve Gibbons and Stephen Machin will be presented at the Royal
Economic Society’s 2002 Annual Conference at the University of Warwick on Wednesday 27
March. Steve Gibbons is Lecturer in Economic Geography at the London School of Economics
(LSE) and a member of LSE’s Centre for Economic Performance (CEP). Stephen Machin is
Professor of Economics at University College London, Director of the DfES-funded Centre
for the Economics of Education and a CEP Programme Director. During the 2001/2 academic
year, he is Visiting Professor of Economics at Massachusetts Institute of Technology.
Funding
Most of the income to fund educational activity is allocated to local education authorities
(LEAs) by central government on a formula basis which includes a per student allowance.
However, there are other significant sources of additional resourcing which, at least in part,
require LEAs and/or schools to bid or otherwise negotiate with government departments,
other public agencies, the voluntary sector, or the private sector. For further detail of these
sources of funds see http://www.ofsted.gov.uk/publications/docs/3049.pdf
The government was committed to revising the calculation of the allocation formula 2003/4
to one based on:
• a national per-pupil element (varying by pupil age);
• additional funding to reflect higher client needs in some areas; and
• additional funding to reflect higher delivery costs in some areas.
An overview of the proposed funding framework can be found at

http://www.dfes.gov.uk/pns/DisplayPN.cgi?pn_id=2003_0151
The full funding statement as delivered to parliament can be found at
http://www.dfes.gov.uk/pns/pnattach/20030151/1.htm
SYSTEMWIDE ASSESSMENT19
Along with the National Curriculum, compulsory testing of all students in state schools was
introduced at the end of keys stages 1, 2 and 3; that is, at ages seven, 11 and 14. (The
relationship between age and educational level is explored on the QCA website
www.qca.org.uk/ca/edlevels.asp).
“National tests” were introduced to serve a number of different purposes:

• to give teachers information about their students’ attainment, for use in planning teaching
and learning;
• to provide information to the Government, to local education authorities (LEAs) and
to the public in general about standards in education nationally and locally; and
• to encourage school accountability.
Students working at levels 1-3 are assessed by a combination of statutory key stage 1 tests and
tasks; students working at levels 3-5 by the key stage 2 tests; and students working at levels 4-
7 in English, 3-7 in science and 3-8 in mathematics by the statutory key stage 3 tests. QCA
provides optional tasks to support the teacher assessment of students working above or below
the level of the tests20.
.
The tests were originally known as “standard assessment tasks” (SATs) but since 1991 have
been called “National Curriculum Tests”. The tests are in mathematics, in English (reading
and writing) and, except for seven-year olds, in science. The QCA overseas the development
of the tests and related supporting documents and manages the external marking of the
statutory key stage 2 and 3 tests.
Since 1997, key stage 2 test results have had an extremely high profile because they are used
to measure progress towards the Government’s targets in literacy and numeracy: that, by
2002, 80 per cent of pupils at the end of key stage 2 nationally will attain level 4 or above in
English, and 75 per cent in mathematics. The key stage 2 results of individual schools are
published as “league tables” showing the relative achievements of all schools in the area and
country. The underlying model is one of competition between schools in which pressure
from parents will raise standards. Hence the reports of the school inspections are also made
public. This is then a high stakes assessment system in which the stakes are greatest for teachers
and school managers. For 11-year olds the consequences for individual pupils are minor.
Nevertheless a great deal of parental and school pressure exists.
The assessment system is a “criterion-related” system (Sainsbury & Sizmur, 1998) with levels
of achievement broadly defined (an increase of one level represents two years of teaching and
learning). The levels describe expectations of what children should be able to do and are not
19 This section of the paper draws primarily on:
• the Qualifications and Curriculum Authority website www.qca.org.uk
• Whetton, C., Twist, E. & Sainsbury, M. (2000). National tests and target setting: maintaining consistent standards. Paper presented by the National Foundation
for Educational Research (NFER) at the American Educational Research Association (AERA), New Orleans, 2000.
20 QCA also produces support materials to provide advice and guidance for primary and secondary schools. www.qcashop.org.uk/cgi-bin/qcashop
empirically derived. For example the description of the Reading level 4 attainment target
reads: “In responding to a range of texts, pupils show understanding of significant ideas,
themes, events and characters, beginning to use inference and deduction. They refer to the
text when explaining their views. They locate and use ideas and information.” Examples of
the test materials can be found on the QCA website at
http://www.qca.org.uk/ca/tests/ks1/2003sample/2003sample_ks1_en_leaflet.asp
http://www.qca.org.uk/ca/tests/ks1/2003sample/english.asp
http://www.qca.org.uk/ca/tests/ks1/2003sample/ks1_en_long_writing.pdf
Aspects of reading, writing and spelling to be assessed, the rationale for the focus of assessment
and examples of marking schemes are all illustrated on the QCA website.
EDUCATIONAL IMPACT: POSITIVES21
System level monitoring of standards and initiatives
The Government’s overriding agenda for primary schools since its election in 1997 has been
to raise standards, especially in literacy and numeracy. The national tests play a central role in
these initiatives, by providing a measure of these standards. Teachers, headteachers and LEAs
have had responsibilities placed upon them to contribute to this improvement of standards,
and their use of test results is a direct response to this policy context.
In order to achieve results nationally, contributory targets are required from each LEA, and, in turn,
from each school. Each LEA has an annual Education Development Plan, agreed with the Department
for Education and Employment, to set out how it will act to raise standards and improve the quality
of education in its schools over the coming year. Targets for National Curriculum achievement
within the LEA form an integral part of these plans. Every school has an annual school development
plan, agreed with its LEA and governors, in which national test target levels may also figure. This is
a picture of a highly effective response to a clear policy initiative from central Government.
School level: use of results
At the school level, evidence of the impact of data provided by the National Curriculum tests
is confirmed by research undertaken in 1998 and 2000 by the National Foundation for
Educational Research.
The research project investigated the use made within individual schools of the National
Curriculum assessment information available to them. Data were collected via a questionnaire
which was sent to a representative national sample of schools in England. Responses were
requested about the practices current in school, from either the principal or a senior staff
member with responsibility for assessment.
The questionnaire was structured in three parts. The first sought an overview of the school’s
approach to the use of National Curriculum results. The second requested more detailed
responses on how the results were used for curriculum management. The third part focused
upon the use of results for curriculum planning, looking at each of the three core subjects of
English, mathematics and science separately.

• Ashby, J. & Sainsbury, M. (2001). How do schools use national curriculum test results? A survey of the use of national curriculum test results in the management
and planning of the curriculum at key stages 1 and 2. National Foundation for Educational Research E paper.
http://www.nfer.ac.uk/research/down_pub.asp and a
• detailed case study of one school by way of illustration. This text is drawn directly from Forster, M. (2003). A principal’s guide to using systemwide assessment
data. (Final draft) Melbourne: Australian Council for Educational Research.
The picture presented by the responses to the questionnaires shows the central role played by
National Curriculum results in the current educational climate. There is clear evidence that such
results are widely used, and that this use has become significantly greater over the last two years.
Test results are used overwhelmingly in schools for both curriculum management and curriculum
planning, with over 95 per cent responding affirmatively for each. Fewer respondents, only about
40 per cent, said that they had a school policy to this effect, with a further one-third in the “not
yet” category. The linking of test results to school development plans was also very widespread,
with just below 90 per cent of the sample answering “yes”. Use in Education Development Plans
was slightly less widespread, reported by about three-quarters of the sample. See Figure 4.
The most universal purposes for the use of test results in school management contexts were
checking National Curriculum attainment and monitoring performance at the end of the
key stage. Year-on-year comparisons were also very widespread. A substantial majority of
respondents also made at least some use of results for: comparisons with other schools; screening
for special educational needs; the evaluation of curriculum delivery; the evaluation of new
teaching approaches; and for planning continuity and progression.
More use was made of National Curriculum results in English and mathematics than in science.
This may be related to the high profile of the national literacy and numeracy targets, which
have no equivalent in science. The highest frequency for any purpose was the use of National
Curriculum levels for whole school planning, reported by three-quarters of the sample for English
and almost as many for mathematics. In science, the proportion was just over a half.
For all three subjects, National Curriculum levels were more widely used than either of the
other types of information, and this held true for curriculum planning from whole-school to
individual level.
By way of illustration, it is interesting to look at the work of one school to use system generated data.
Two mile ash middle school, england
Two Mile Ash (TMA) is a co-educational, non-selective school for students from Year 4
(eight-year-olds) to Year 7 (twelve-year-olds). In 2001, the school entered its fourth year of a
co-ordinated, whole-school assessment and reporting program.
The program is managed by the Deputy Head, Curriculum and Assessment. She keeps all
the assessment data in one database using Assessment Manager software. Each piece of data is
stored separately under templates organised by assessment source. To ensure security and
accuracy, teachers do not have direct access to the database–administration staff enter the
data from teacher-completed standardised grids.
The school uses achievement data drawn from three sources:

• compulsory national assessments at Key Stage 1 (Year 3) and Key Stage 2 (Year 6) in
English, mathematics and science (the Standard Assessment Tasks, SATs);
• optional tests provided by the government (Qualifications and Curriculum Authority,
optional SATs);
• and commercially available standardised tests that include normative information.
TMA maintains an assessment schedule across the four years of school with a timetable of
tests for each year group spread over three terms–see table below. The initial assessment of
Year 4 pupils’ reading, spelling and mathematics ability takes place during the first half of
their first term at the school after they have settled in. These students have been assessed
fifteen months earlier at Key Stage 1.
TMA uses the data to understand first how students are performing:
• How are students performing in relation to national targets?
• How are students performing in relation to previous school achievement?
• How are particular groups of students performing in relation to other groups of students
within the school?
then explores why students are performing at the levels indicated:
• What do these data tell us about our teaching practices?
• What do these data tell us about the success of special initiatives?
1. How are students performing in relation to national targets?
TMA examines student performance in relation to national targets by year level cohort. For
example, Year 4 intake data in 1999 showed a large tail: 66 of the 192 students arrived with
a point score of 14 or below on the national test Key Stage 1 Standard Assessment Tasks
(SAT). These students were at risk. It would be difficult for them to achieve the national
target of level 4 at the end of Key Stage 2 (Year 6).
2. How are students performing in relation to previous school achievement?
TMA monitors student performance in relation to previous school achievement cross-

sectionally by year level cohort, and longitudinally at an individual student level. Figure 2
below illustrates the percentage of Year 6 students in successive cohorts from 1996 to 1999
achieving above level 5 on the Key Stage 2 national test. Notice the increase in achievement
since a whole school approach to assessment and reporting was initiated.
TMA also monitors student performance in relation to previous school achievement

longitudinally. Figure 3 is an extract from a chart illustrating the achievements over time of
students with Special Education Needs (SEN) )–those in the lowest 2% of achievement who
are considered to be 18 months behind their peers. Achievement is summarised as a point
score. Mainstream students are expected to progress at a rate of 12 points over 4 years. Data
for English only are shown here, although achievement is monitored in mathematics and
science also.
3. How are particular groups of students performing

in relation to other groups of students within the school?
TMA disaggregates achievement data by year group, by class group, by gender, and by ability
“set” in order to examine student achievement. For example, at TMA each year group is
divided into ability sets for writing, reading, and mathematics instruction. Usually there are
7 or 8 sets for each of these three areas of learning. Students are allocated to sets, and moved
between sets, on the basis of their performance on standardised tests (see the assessment
schedule) and ongoing teacher assessments. Students will change sets if they need more
challenge or more support than others in the set: that is, if their progress is much faster or
slower than the progress made by their set peers. Movement usually occurs at the end of term
when formal and informal assessment information is available, and after a formal review has
occurred within a year group meeting.
Students may be in different sets for different learning areas. Sets are staffed by class teachers,
specialist staff, and additional teachers. TMA examines the performance of students in relation
to students from different ability sets.
4. What do these data tell us about our teaching practices?
TMA staff also look for anomalies in the results. When they disaggregated student achievement in
reading by year group, gender and ability set they found that at Year 6, girls in the higher attaining
sets were significantly outperforming boys in writing, and boys in the higher attaining sets were
significantly outperforming girls in reading. They wondered whether this might be related to the
kinds of texts they presented to boys and girls and the different texts boys and girls selected.
When TMA compared the cohort progress of students in writing at each of Years 4, 5 and 6,
they discovered different rates of progress within each year level. They wondered whether this
might be related to the way in which writing was assessed at the different year levels.
Sometimes when TMA compares the achievements of different ability sets the school finds that
a particular set is doing unexpectedly well or poorly. They then ask whether the teacher of that
set has a certain teaching style, is using or not using available resources, has engaged or not
engaged specialist support, and whether there are any further training or resource requirements.
5. What do these data tell us about the success of special initiatives?
TMA also monitors student performance over time to evaluate the success of special initiatives.
For example, the relative achievements of boys and girls in reading and writing in the Year 6
SATs were monitored from 1998 to 2000 to see whether special initiatives undertaken to
broaden the reading and writing experiences of students were having a positive effect.
The table below shows the narrowing gap between the achievement of boys and girls on the
SATs from 1998 to 2000.
In response to these examinations of data, TMA employs a number of strategies to improve learning:
• creating a school culture of accountability;
• focusing teacher professional development;
• focusing teaching resources;
• setting explicit targets for student learning; and
• establishing and monitoring special programs and initiatives.
What evidence is there that these strategies have improved student learning?
At TMA standards have risen steadily and significantly particularly in mathematics. The
graph below illustrates the rising achievements of students from 1995-1999 as indicated by
the increasing percentage of children achieving Level 4 (the national average expectation) or
above at the end of Key Stage 2 (in Year 6).
The Deputy Head argues that the strategies have improved student learning because they are
based on evidence of student achievement: the use of assessment and pupil data has led to
higher expectations, consistency of practice and a sense of collective accountability for children’s
progress.
There have also been some unexpected positives. Initial concerns that the focus on achievement
would lead to a data-oriented approach that ignored the whole student have not been realised.
Rather than marginalising other informal assessments or information such as pastoral records,
the use of achievement data has resulted in more frequent focused references to additional
sources to assist understanding of the individual student. Furthermore, teachers, parents and
students have become empowered by shared understandings about individual progress.
“Assessment of learning has become assessment for learning for all-staff, parents and pupils.”
Cutis, & Hudson, J. (2000). Using assessment and pupil data to raise standards in teaching
and learning. Unpublished Paper. Milton Keynes: Two Mile Ash Middle School.A.
EDUCATIONAL IMPACT: CONCERNS22
There has been steady criticism and much anecdotal debate about the negative impact of the
National Curriculum and its accompanying assessments. See the Assessment Reform Group
http://www.assessment-reform-group.org.uk/ and the work of Wiliam in particular.
An emphasis on summative assessment
There is a general concern about the emphasis on summative assessment at the expense of
formative assessment. This concern is based on the view that where systems embrace multiple
assessment purposes it is almost inevitable that summative purposes will displace formative
ones: National curriculum tests play a dominant role in the summative assessment of students.
That is, in providing overall scores and levels at the end of a period of instruction and learning–
“assessment of learning”. Despite the fact that teachers also use classroom assessments, the
national curriculum tests have greater status than other tests leading to a lack of emphasis on
formative assessment. That is, a lack of emphasis on the nature of students” proficiencies in
an area of learning–“assessment for learning”.
The national tests undergo a thorough development process and are good reflections of the
breadth of the National Curriculum, but they are nevertheless only a snapshot of a child’s
performance on one day. Attention to test results could go beyond an optimum level, with
the result that the curriculum becomes too focused upon test preparation, to the detriment
of pupils and staff (Ashby and Sainsbury, 2001).
Validity of the tests for summative and formative purposes
More seriously, there is concern that the National Curriculum tests, because they inevitably
sample from the domain, provide limited information for formative purposes and even areas
that are covered well will be sampled with insufficient frequency to support diagnostic inferences.
Furthermore there is concern that the tests are not robust enough to support inferences
drawn for summative and evaluative purposes.

• Newton, P. (2003). The defensibility of national curriculum assessment in England. Research Papers in Education 18 (2) June 2003, pp. 101-127.
• Newton, P. (2003). Evidence-based policy making. Research Papers in Education 18 (2) June 2003, pp. 137-140.
• Whetton, C., Twist, E. & Sainsbury, M. (2000). National tests and target setting: maintaining consistent standards. Paper presented by the
National Foundation for Educational Research (NFER) at the American Educational Research Association (AERA), New Orleans, 2000.
• Wiliam, D. (2003). National curriculum assessment: how to make it better. Research Papers in Education 18 (2) June 2003, pp. 129-136.
The end-of -key stage tests can assess only a small proportion of the national curriculum in each
tested subject. In low-stakes contexts, the limited range of achievement that is assessed in the
tests can stand as a proxy for achievement across the whole subject. However, in high-stakes
contexts, there is pressure to increase the student’s performance in those aspects of the subject
that will be tested… Standards of achievement n the tested areas will rise, but only at the
expense of untested areas. Therefore while the reported standards of achievement may rise, the
actual level of achievement across the whole subject could well be falling, and the tests are no
longer an adequate proxy for achievement across the whole domain… For all these reasons, the
whole idea of measuring standards over time in any real sense is nonsense – not just with
national curriculum tests, but with any tests of achievement. (Wilian, 2001, p.11).
As Newton points out (2003a, p. 107), the evidence for this claim is anecdotal. Research needs to be
undertaken to determine the extent to which national curriculum tests predictably fail to assess the
learning objectives of their programs of study – and the particular matters, skills and understandings
that are underrepresented; the extent to which teachers fail to foster those learning objectives in
students – and the particular matters, skills and understandings that are under-taught.
Reliability of tests for summative and evaluative purposes
In parallel with concerns over validity are concerns about the reliability of the inferences drawn
from the National Test scores, in particular the reliability of the classification of students’ test
scores to a particular level. The technical arguments put forward are discussed and rebutted in
detail by Newton (2003, pp107-109).
Target setting demands
The need to monitor at a system level against the level 4 target has raised questions about the
standard setting process. To address security needs and the changing curriculum backwash
effect the tests are changed every year. The problem of maintaining the standard of level 4 as
a constant becomes a major one for the test developers. The problem is both technical and
one of public perception.
Given the high stakes, a decision was made to use four methods to set the standard each year
and to combine the results to derive single cut scores: direct statistical equating of the new
test with the previous year’s test; equating of the new test with an anchor test which had been
used for several years; an Angoff-type standards setting meeting; and a “script scrutiny”
procedure. (That is, two methods based on empirical data and two on expert judgement.)
The procedural details and challenges of these methods are discussed in Whetton et al, 2000.
Given the political stakes it is not surprising that there were suggestions that the cut scores for
level 4 had been lowered in order to assist the Secretary of State to reach the literacy target (as
reported by a national newspaper, the Times, in 1999, Clare, 1999). To address the charge
the Secretary of State established a committee of inquiry which included representatives from
his own and other major political parties and a Times journalist. The committee examined all
the cut score setting processes and took evidence from academics, test developers, principals,
teachers and various interest groups. They concluded that the concerns about setting and
maintaining standards of the English test were without foundation. They concluded also that
there was no evidence to suggest that the Minister or any other officials had sought to influence
the tests in order to meet the national targets in English (Rose, 1999).
League tables
As noted already, the underlying model for the national assessment system is one of competition
between schools in which pressure from parents will raise standards. From the start the
unfairness of the publication of unadjusted results (“league tables”) between schools has been
trumpeted by those working in areas of social deprivation, such as inner cities and there have
been efforts to develop “value added” measures which show progress or performance adjusted
for previous attainment or social background.
Students’ experiences and perspectives of the national curriculum
QCA conducts and commissions research to inform the development of education and training
policies and the results of this research are available on the QCA website. A recent study
(Lord, 2001, 2002) explored what research had been conducted and published on pupil’s
experiences and perspectives of the National Curriculum. http://www.qca.org.uk/rs/rer/
pupils_perspectives.asp as a missing piece in reflections of the National Curriclum.
Trends in the latest body of most recently published research include:

• a notable number of additions on studies that monitor pupils’ subject choice and standards
–attractive possibly as it is now a decade since the introduction of the National Curriculum
• an increase in studies on pupils’ understanding of National Curriculum subject areas–
prominent both in the thematic approach of the studies and in the substantive findings
• an advance in the number of studies on PE – several of which emphasise gender issues,
and unusually, ethnicity as well.
Key findings in the two most popularly researched areas – relevance and enjoyment – emerge
in the recent research and seem to be corroborated by previous studies:
• pupils’ enjoyment of the curriculum seems related to perceived ability (but not necessarily
to actual ability)
• pupils seem to hold a utilitarian construction of the importance or relevance of subjects,
often based on the knowledge content of a subject, rather than a skill-base or process
• pupils rarely understand why they are learning what they are learning unless this is
made explicit (related also to pupils’ understanding of the curriculum).
The report can be found at http://www.qca.org.uk/re/rer/pupils_perspectives.asp

FINAL REFLECTIONS
There are many similarities in the Western Australian and the English assessment systems, in
particular the intention to address a number of purposes for assessment and to provide
information of different kinds to a range of stakeholders. They both also express a key intention
to impact directly on the work of the classroom teacher.
One central difference is the degree to which these intentions are driven by direct accountability
pressures. In Western Australia, although schools need to use the systemwide assessment data
in their planning, the stakes are not high and there are no public school-by-school comparisons.
Rather, the government has moved quietly forward to influence teacher practice by sending
the message that all areas of the curriculum are valued, that assessment materials will be made
available as a model for teachers, and that support will be provided to assist schools to use
their data more effectively. The use of two parallel and complementary assessment programs
has also assisted this process. And although this is partly historical (MSE was in existence
before the National agreement to undertake full cohort testing), it is still a strategy worth
contemplating.
One might ask whether the context of debate in England, in particular the emphasis on the
validity of the data for formative assessment, and the resort to value added measures, is in
direct response to the pressure of national targets, and the public nature of comparisons,
rather than in response to the systematic collection and reporting of student achievement
data. Western Australia has so far avoided a heated debate about curriculum fidelity and is
instead exploring issues of equity, and ways to better support teachers in their classroom
practice. One might also ask whether in both countries the debates are ones held by academics
and researchers, and or by teachers and the community more generally. There is some evidence
that in both countries teachers and schools have accepted the assessment programs and are
now getting on with the job of using data more effectively.
As noted by Whetton et al (2000) the experience in England illustrates that the setting and
maintenance of standards is a social and societal process. Assessment is a social activity that
we can understand only by taking account of the social, cultural, economic and political
contexts in which it operates. While the setting of standards, for example, relies on empirical
information, the information must be interpreted, and the integrity and expertness of those
making the judgements must be accepted publicly and politically (Gipps, 1999).
REFERENCES
Ashby, J. & Sainsbury, M. (2001). How do schools use national curriculum test results? A survey of the use of
national curriculum test results in the management and planning of the curriculum at key stages 1 and 2.
National Foundation for Educational Research E paper. http://www.nfer.ac.uk/research/down_pub.asp
Clare, J. (1999). Passing Off, Daily Telegraph, 29 May, 1999.
Forster, M. (2001). A policy maker’s guide to systemwide assessment programs. Melbourne: Australian Council
for Educational Research.
Forster, M. (2003). A principal’s guide to using systemwide assessment data. (Final draft) Melbourne: Australian
Council for Educational Research.
Gipps, C. (1999). Socio-cultural aspects of assessment’ in Iran-Nejad, A. and Pearson, P.P. (Eds). Review of
Research in Education, 24. Washington, DC: AERA.
Linn, R. (1995). Angoff lecture. Princeton: Education Testing Service.
Lord, P. (2001). Pupils’ Experiences and Perspectives of the National Curriculum: Updating the Research Review
(Research Report) [online]. Available: http://www.qca.org.uk/rs/rer/pupils_perspectives.asp [21 January, 2002].
Lord, P. (2002). Pupils’ Experiences and Perspectives of the National Curriculum: Updating the Research
Review (Research Report) [online]. Available:
http://www.qca.org.uk/re/rer/pupils_perspectives.asp [27 January, 2003].
Marion, SF and Sheinker, A. (1999). Issues and consequences for State-level minimum competency testing
programs. National Center on Educational Outcomes, January 1999.
Ministerial Council on Education, Employment, Training and Youth Affairs and Curriculum Corporation
(2000). National Report on Schooling in Australia. Melbourne: Curriculum Corporation.
http://www.curriculum.edu.au/mceetya
Murdoch University, Estill and Associates (2003). Evaluation of the use of the 2002 Western Australian
Literacy and Numeracy Assessment (WALNA). Paper prepared for School and System Performance Directorate,
Department of Education, Western Australia.
Newton, P. (2003a). The defensibility of national curriculum assessment in England. Research Papers in
Education 18 (2) June 2003, pp. 101-127.
Newton, P. (2003b). Evidence-based policy making. Research Papers in Education 18 (2) June 2003, pp.
137-140.
Quinlan, M. & Scharaschkin, A. (1999). National curriculum testing: problems and practicalities. Paper
presented at the British Educational Research Association Annual Conference, University of Sussex, 1999.
Rose, J. (1999). Weighing the baby: the report of the independent scrutiny panel on the 1999 key stage 2
National Curriculum Tests in English and mathematics. London: DfEE.
Sainsbury, M. & Sizmur, S. (1998). Level descriptions in the National Curriculum: what kind of criterion
referencing is this? Oxford Review of Education, 24, 2, 181-193.
Stobart, G. (1999). The validity of national curriculum assessment. Paper presented at the British Educational
Research Association Annual Conference, University of Sussex, 1999.
Wildy, H. (2003). Data Club: supporting schools to use data in Professional Educator, Vol. 2, No. 3 September
2003, pp 6-7.
Whetton, C., Twist, E. & Sainsbury, M. (2000). National tests and target setting: maintaining consistent
standards. Paper presented by the National Foundation for Educational Research (NFER) at the American
Educational Research Association (AERA), New Orleans, 2000.
Wiliam, D. (2003). National curriculum assessment: how to make it better. Research Papers in Education 18
(2) June 2003, pp. 129-136.
Wiliam, D. (2001). Level best? Levels of attainment in national curriculum assessment. London: ATL.
Wiliam, D. (2000). Integrating formative and summative functions of assessment. Paper presented to Working
Group 10 of the International Congress on Mathematics Education, Makuhari, Tokyo, August 2000.
Useful websites
General
EURYDICE, the information network on education across the Member States of the European Union, has a
website which provides current descriptions of the education systems across the Member States: http://
www.eurydice.org
The website for EURYDICE at the NFER, the National Unit for England, Wales and Northern Ireland is
accessible at: http://www.nfer.ac.uk/eurydice
INCA The Archive of the International Review of Curriculum and Assessment Frameworks
Project http://www.inca.org.uk/
The INCA website is produced in England by the Qualifications and Curriculum Authority
(QCA) and the National Foundation for Educational Research (NFER) and currently includes
national descriptions (country Archives) of the education systems in - Australia, Canada,
England, France, Germany, Hungary, Ireland, Italy, Japan, Korea, the Netherlands, New
Zealand, Singapore, Spain, Sweden, Switzerland, the USA and Wales. The site, and the faci-
lities and information it provides, are reviewed, revised and updated on an ongoing basis.
Australia
Education Department of Western Australia - http://www.eddept.wa.edu.au
The gateway to resources and services for education and training in Australia -Education Network Australia
at http://www.edna.edu.au
Commonwealth Department of Education, Science and Training (DEST) http://www.dest.gov.au/
Ministerial Council on Education, Employment, Training and Youth Affairs

http://www.curriculum.edu.au/mceetya
Curriculum Corporation http://www.curriculum.edu.au
The website of the Australian Curriculum, Assessment and Certification Authorities (ACACA) http://
www.acaca.org.au ACACA is a national forum for the consideration of curriculum, assessment and certification
issues
Curriculum Trends across Australia in 2000 http://www.boardofstudies.nsw.edu.au/acaca2
National Council of Independent Schools’ Associations http://ncisa.edu.au
National Catholic Education Commission http://www.ncec.catholic.edu.au

England
The National Curriculum website at: http://www.nc.uk.net

The National Curriculum in Action website at: http://www.ncaction.org.uk
Qualifications and Curriculum Authority (QCA) - http://www.qca.org.uk/

Office for Standards in Education (OFSTED) - http://www.ofsted.gov.uk
Department for Education and Skills (DfES) - http://www.dfes.gov.uk/
The Assessment Reform Group studies the implications of assessment policy and practice in the UK with the
aim of ensuring that public policy at all levels takes account of relevant research in assessment practice.
http://www.assessment-reform-group.org.uk/
The CERUK database of current educational research in the United Kingdom.

CERUK is a database sponsored by the National Foundation for Educational Research (NFER), the
Department for Education and Skills (DfES) and the EPPI Centre (Evidence for Policy and Practice Information
and Coordinating Centre). It is accessible at http://www.nfer.ac.uk/ceruk and aims to include all recently
completed and ongoing research in the UK from large sponsored programs to individual projects.
The Standing International Conference of Inspectorates website at http://www.sici.org.uk/
Access to the Government’s Green Paper (consultation document) on 14-19 education 14-19: extending
opportunities, raising standards is available via the DfES website at http://www.dfes.gov.uk/14-19greenpaper/
The Centre for Formative Assessment Studies (CFAS), located in the Faculty of Education of the University
of Manchester, is one of the largest research units in England in the areas of education and social science
evaluations. http://www.education.man.ac.uk/cfas

Comision Simce Anexo2 Forster

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Comision Simce Anexo2 Forster

Uploaded by

Copyright:

Available Formats

A NEXOS

Comisión para el Desarrollo y Uso del Sistema

La Comisión para el Desarrollo y Uso del Sistema de Medición de la Calidad de la Educación

LA POLÍTICA EN EVALUACIÓN Y CURRÍCULO ANTE EL DESAFÍO DE LA CALIDAD 09

T ENDENCIAS MUNDIALES EN LA OPERACIONALIZACIÓN DE CRITERIOS DE CALIDAD 13

Fijando criterios para la evaluación de la calidad 15

A SSESSMENT SYSTEMS : TWO CASE STUDIES 45

C ASE S TUDY 2: E NGLAND (N ATIONAL C URRICULUM A SSESSMENT ) 93

TWO CASE STUDIES

The MSE program is administered annually to a representative sample of students in each of

Full-cohort (census) programs also provide:

Geographic, demographic, economic, political

1 This section of the paper draws directly from two texts:

Responsibility for schooling in Australia

Provision of schooling in Australia

Some features of the structure of Australian schooling in 2001 were as follows:

• There were 9596 schools in Australia.

• There were 3.268 million full-time students in Australian schools.

• The proportion of students enrolled in non-government schools continued to rise in

Funding for schooling in Australia

Figure 1 Public funding for school education

Source: 2000 National Report on Schooling in Australia Amounts in AUD

Common and agreed national goals for schooling in Australia

The provision of schooling in Western Australia

• A child of compulsory school age is entitled to be enrolled at a local-intake school if his

Different programs different purposes

The Western Australian Department of Education conducts two systemwide assessment

The purpose of WALNA is to provide:

The purpose of the MSE program is to:

(Source: Murdoch University and Estill & Associates, 2003).

The tests are designed to:

Three types of report are provided:

• Individual student reports for parents and caregivers

• Class reports for teachers

• School reports for principals

At the classroom level, teachers use the results to:

EDUCATIONAL IMPACT: POSITIVES9

System level monitoring

The WA Department reports two major findings in relation to the standards:

2. Girls’ and boys’ numeracy results are comparable.

3. Aboriginal students’ literacy performance has shown a trend of improvement since

School level: use of results and acceptance of the program

The Data Club

Independent review of WALNA

Role of the test in assisting schools to

The quality of the information provided

Reaction of students to the test

WILL NOT BE KEPT CONFIDENTIAL CONFIDENTIALITY OF TEST RESULTS

The School Accountability Framework

The Framework is built on five fundamental commitments: high standards; effectiveness;

1. Accountability involves responding to information about performance.

• What are we trying to achieve?

• How well are we achieving it?

• How can we improve?

2. Accountability is linked to outcomes.

3. Demonstrating accountability is a professional responsibility of school staff.

4. Rigorous self-assessment is at the heart of effective accountability.

5. Schools are accountable to their communities and to the Director-General.

As a final accountability measure, reviews of schools by District Directors provide assurance

EDUCATIONAL IMPACT: CONCERNS12

Ethical principles underpinning reporting

12 This section of the paper draws directly from

Selection and reporting of performance measures

• Caution should be taken whenever data involves comparisons between jurisdictions,