You are on page 1of 13

High-stakes literacy tests and local

effects in a rural school


Phillip Cormack
University of South Australia

Barbara Comber
Queensland University of Technology

ABSTRACT

High-stakes literacy testing is now a ubiquitous educational phenomenon. However, it remains


a relatively recent phenomenon in Australia. Hence it is possible to study the ways in which
such tests are reorganising educators’ work during this period of change. This paper draws upon
Dorothy Smith’s Institutional Ethnography and critical policy analysis to consider this problem
and reports on interview data from teachers and the principal in small rural school in a poor
area of South Australia. In this context high-stakes testing and the associated diagnostic school
review unleashes a chain of actions within the school which ultimately results in educators
doubting their professional judgments, increasing the investment in testing, narrowing their
teaching of literacy and purchasing levelled reading schemes. The effects of high-stakes testing in
disadvantaged schools are identified and discussed.

Introduction educators’ work in the wake of the implementation of


I would welcome more skilling of myself to be able to
National Assessment Program: Literacy and Numeracy
unpack the mandated tests. I’m happy to administer (NAPLAN). As did other principals in our study, he
them because I think they can give some direction to grapples daily with what it means to be working in a
programming, but whether it’s good direction or token, site which has been assessed a ‘failing school’ on the
or whatever, is dependent on how well we unpack it basis of a high-stakes test.
and drill down into it. I guess I’m crying out for some
The project’s design and analysis was informed by
help in that area. (Principal, small rural school in high
poverty region) the Institutional Ethnography of Dorothy Smith (1987;
2005; 2006). The lived experiences of school-based
This statement, from an experienced primary educators are typically absent from policy discussions
school principal in a poor rural community, touches and broader political, media and educational debates
on the complexity of the work and emotions associ- concerning standardised testing. The work – emotional,
ated with mandated literacy testing. Like most princi- everyday and pragmatic – is often invisible (Black-
pals, he wants to do the right thing and willingly gets more, 2004; Nichols & Griffith, 2009; Smith, 2005).
on with ‘administering’ the tests. He hopes that the Smith (2005), however, has developed an approach
results might ‘give some direction’, however he admits to researching how the practices of everyday life are
to being unsure about how he should interpret the organised and how those practices are coordinated
results in order to decide on a ‘good direction’. There’s beyond the local level, often by texts which come to
a lot going on here and we can hear his anxiety in the rule how people go about their work. Yet it is people
request for help. The principal was one of a number of working and living in particular places who activate
school leaders from Victoria and South Australia who these texts in specific ways, eliciting chains of related
participated, along with their teachers, in our research actions with particular consequences for particular
project designed to investigate the reorganisation of people. Following Smith, we argue that it is important

78 Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013


)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPMtCORMACK & COMBER

to hear from front-line workers – educators in schools, numerical, standardised data have come to be the only
especially those who are working in challenging circum- information that counts when discussing the value of
stances – about what mandated testing actually does to schools (Comber, 2012; Comber & Cormack, 2011;
everyday practices. Hence our research endeavours to Kostogriz & Doecke, 2011). Within the proliferation
trace in the talk of educators the activation of a range of information and data about schools, cohorts of
of practices associated with high-stakes testing. In the students, national and international performance, and
case of the rural school considered here, we discuss how so on, the narrow underlying logics of this discourse
contemporary policy is enacted in situ though staff’s can be lost. These logics can be summarised as:
reported experiences of a school review, subsequent
r TUBOEBSEJTFE UFTU SFTVMUT QSPWJEF BO BDDVSBUF BOE
to the students’ poor performance on literacy tests,
reliable insight into students’ literacy and numeracy
and their efforts to improve student test performance
practice and performance
through the ways they purchase resources, group their
r MJUFSBDZ QFSGPSNBODF PS TUBOEBSET BT NFBTVSFE
students, spend their time, prioritise aspects of literacy
by these tests are the key indicator of the ‘value’
and so on.
that schools provide to students and, by proxy, to
In terms of our ongoing program of research
the students’ future performance as workers and
concerning the relationships between literacy educa-
citizens – a logic Comber and Hill (2000) identified
tion and social justice, we believe it is increasingly
as ‘literacisation’, especially for schools serving low
urgent to consider the material effects of high-stakes
SES students
tests in particular places. It is important in doing this
r JGUIFTFEBUBBSFQSPWJEFEJOXBZTUIBUBMMPXNBOBHFST
to build on the work of others who have explored the
and parents (as consumers who ‘choose’ schools) to
compounded educational disadvantage often experi-
compare the ‘value’ being added to their students by
enced by young people growing up in rural poverty
schools, underperforming schools can be identified
(Green & Letts, 2007) and the challenges faced by
and ‘incentivised’ to lift their performance, learning
their teachers (Somerville, Plunkett & Dyson, 2010).
from those schools which are performing best, thus
At the same time, we recognise that the principal’s posi-
solving the problem of unequal outcomes
tioning as a school leader has altered with devolution
to school-based management, performative cultures, For these logics to work, as Lingard points out, schools
concern with risk management, intensification of need to be recast as part of a competitive educational
work, the emergence of new accountabilities, and the ‘market’.
proliferation of educational policy and associated legal A basic assumption is that competition between
and financial reporting demands, and there has been schools and parental pressures will push up standards
useful work on these effects in policy and practice (Ball, and strengthen accountabilities. There is also at times
2001; Blackmore, 2004; Lingard, 2010; McWilliam & a parental market choice discourse underpinning the
Singh, 2004; Thomson, 2009). In what follows, we policy (Lingard, 2010, p. 132).
begin by examining the claims and assumptions upon The assumption that schools operate on a level
which high-stakes testing is built, before moving on playing field within a market can only be sustained if
to examine the impact of these policies on one rural much of the actual work of teachers and principals, and
school. of school context, is bracketed out. The emphasis in this
discourse is on the teacher as the most important factor
NAPLAN, My School and ICSEA: High in student achievement (Berliner, 2013 forthcoming),
stakes for principals and teachers eliding such issues as location, socio-economic status,
For several decades in Australia, literacy has been a race and linguistic background. What counts in deter-
focus of government policy and now with the advent mining the value of schooling becomes standardised
of the My School website’s (myschool.edu.au) annual data and the only solution for change within schools
reporting of school performance data, the standard- is teaching. Labaree (2011, p. 628) notes that statistics
ised measurement of literacy achievement has become has come to dominate research about school effective-
high stakes. The centrepiece of the My School website ness and determine what will count as trustworthy
and associated public discussions is the data generated information such that significant issues that don’t fit
through the standardised tests known as NAPLAN their frame receive virtually no attention – as he puts
conducted annually on Year 3, 5, 7 and 9 students it, ‘[w]hen you are holding a hammer, everything looks
in all Australian schools. The reforms with which like a nail’.
NAPLAN is associated operate within what can be Significantly for the work of teachers and school
labelled a ‘discourse of data’ (Poster et al., 1996) where leaders, the focus on data underpinned by competitive

Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013 79


CORMACK & COMBERt)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPM

logics is that the market will provide its own correc- the associated material realities of everyday life become
tive solutions to school underperformance. Consider statistical ‘noise’ that distracts from the (moral) imper-
the way that this logic works as it is expressed on the ative to develop better pedagogies, better programs,
My School and NAPLAN websites. In a page titled better interventions. By this account the tests identify
‘Why NAP?’ a ‘key benefit’ of NAPLAN is described the problem within the ‘student body’ and teachers
as ‘driving improvements’. are positioned as the solution, through attending to
All Australian schools benefit from the outcomes of
the insights in the data and applying the pedagogic
national testing, with aggregated results made avail- remedy. Berliner (2013 forthcoming, p. 1), in the US
able though comprehensive reports at the national context, shows these logics have failed to deliver on
and school level, accessible on-line (see Test Results). their promise and asks:
Schools can gain detailed information about how they
are performing, and they can identify strengths and What does it take to get politicians and the general
weaknesses which may warrant further attention. public to abandon misleading ideas, such as ‘Anyone
who tries can pull themselves up by their bootstraps’, or
The focus here is on the identification of ‘strengths that ‘Teachers are the most important factor in deter-
and weaknesses’ through comparison with other mining the achievement of our youth’?
schools. As explained on the My School website, key Those who do manage to pull themselves up by their
in this process of comparison is the concept of ‘like bootstraps and teachers who manage to turn around
schools’ where, through the application of the Index the learning lives of their students are the extraordinary
of Community Socio-Economic Advantage (ICSEA), cases and not the norm, hence policy should not be
schools are grouped into those serving ‘statistically based upon such exceptional instances. Bluntly Berliner
similar populations’. argues: ‘Most children born into the lower social classes
ICSEA enables student results on national tests to be
will not make it out of that class, even when exposed
understood and compared in a fair and meaningful to heroic educators’ (Berliner, 2013 forthcoming, p. 2).
way, and enables schools seeking to improve their He concludes that educational reform can only achieve
performance to learn from other schools with statisti- so much when there is great inequality of wealth
cally similar populations. in a society and that at least part of the solution to
Here we see an assumption that comparison (under- better learning outcomes involves different economic
pinned by competition) will foster a bootstrapping policies and practices – including fair wages, higher
process to occur whereby ‘underperforming’ schools taxes to better resource schools (e.g. librarians, coun-
learn from their better-performing peers. A virtuous sellors, nurses) and providing better infrastructure in
cycle of continuous improvement is thus assumed, poor areas (e.g. transportation, health care, policing).
centred on what teachers do in their classrooms and In short, there is a need in wealthy nations with high
principals in their schools, with difference in loca- inequality to provide substantive and ongoing invest-
tion, community resources, and so on, taken care of ments in education and services allied with basic human
statistically. One of our key questions in working with rights. Schools cannot solve the effects of poverty.
teachers serving students in poverty and in different This argument about the need to interrogate taken-
locations such as rural towns or poorly-resourced low for-granted assumptions about school reform and
socio-economic suburbs was how such logics worked, literacy education in low socio-economic communities
or didn’t work, for them in making a difference for underlies our discussion of the local effects of global
their students. trends toward standardisation. We wish to compli-
League tables are dangerous (Alexander, 2010) and cate the logics described above by considering the
can result in the stigmatisation of school communi- reported experiences of educators in one rural school
ties especially in the face of a dominant common-sense in a poor regional community. We begin by describing
rationality that if poor people try harder/ work harder the school’s location and population. We then turn to
they should prosper. High-stakes testing, combined the principal’s and teachers’ reported experience of
with educational policies based on meritocracy (Teese, high-stakes testing in literacy. Finally we describe how
2000; Teese & Polesel, 2003) or bootstrap mentali- students come to be understood in this context and the
ties (Berliner, 2013 forthcoming), make poor perfor- subsequent solutions designed for these ‘problems’ and
mance the problem of the poor and their teachers. If how such practices are contingent upon the resources
only the teachers taught the basics, if only the parents the school community can assemble.
read them a bed-time story, if only the children worked
harder, they would improve. These, ‘if only’ rationales School context
ignore how poverty is actually produced. Poverty and Wheatville (a pseudonym) is a small rural primary

80 Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013


)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPMtCORMACK & COMBER

Table 1

2006 2011

Median weekly … Australia-wide Wheatville Australia-wide Wheatville

individual income $466 $300 $577 $390

household income $1,027 $570 $1,234 $720

family income $1,171 $660 $1,481 $850

school with an enrolment of around 140 to 150 students, As the principal and teachers reported to us, Wheat-
with some variation due to transience associated with ville experienced a great deal of mobility of population
mobile populations. The local community suffers a high associated with casualised and seasonal labour, cheap
level of poverty and the principal and teachers see the housing and so forth. By the state education depart-
context as disadvantaged and challenging. According ment’s standards, Wheatville experienced a high level
to the My School website Wheatville’s ICSEA score was of disadvantage. Since the period in which the study
between the low 950s and high 940s from 2008 to 2010 was undertaken (2008–2010) some areas of the town
(with 1,000 being the mean for Australian schools)1. have become gentrified with the arrival of ‘sea-changers’
The My School website glossary claims that ICSEA is ‘a which would account for the slightly higher incomes
scale that enables meaningful comparisons to be made recorded in the 2011 census. However the points which
across schools’ because it identifies ‘schools serving we need to make here are concerned with the relation-
similar student populations’. It goes onto explain that: ships between the school’s literacy results on NAPLAN
The variables used in calculating a value on the ICSEA
(and other standardised measures), the actual experience
scale include student-level data on the occupation of poverty and its effects, the path to being declared a
and education level of parents/carers, and/or socio- failing school and how this impacts on educators’ work
economic characteristics of the areas where students in a small rural community. The short story here is that
live, whether a school is in a metropolitan, regional or Wheatville students and their families experienced a
remote area, proportion of students from a language
high level of relative poverty. The teachers recognised
background other than English, as well as the propor-
tion of Indigenous students enrolled at the school. this, as we will discuss below. Two other features of the
(http://www.acara.edu.au/myschool/myschool_glos- school need to be stressed at this point. First, there was
sary:.html) a very small enrolment, typically there were only about
A detailed fact sheet has been produced to elaborate 20 students (maximum) at any one level to be tested (i.e.
on how the scale has been calculated. However since its in Years, 3, 5 & 7). Second, the school population was
inception, school principals have contested the extent highly mobile, meaning it was quite likely that some
to which the ICSEA scale really does identify similar students were recently arrived at the time of testing and
populations. In the case of Wheatville the ICSEA score rare for students to remain in the school for years 3, 5,
indicates that it is below the mean but not dramatically 7. Small numbers and high mobility introduce statistical
so. However, the 2006 ABS data (http://www.census- uncertainty, so whether the ICSEA can represent the
data.abs.gov.au) indicated that for the town in which school population in terms of comparability is question-
the school was located, the median weekly income able, as is the assumption that schools will be able to
was significantly lower than national and state means analyse their results in terms of ‘value-added’. These are
(Table 1). complex questions and more than we can address in this
At the time of the study, according to the state educa- paper, however the point we wish to make is that school
tion department’s Index of Disadvantage, Wheatville context is very important in terms of how educators
Primary was a Category 2 school, where there are 7 experience their work and not sufficiently accounted for
categories with 1 being the lowest. The department’s by ICSEA. Indeed, as Smith (2005, pp. 165–182) has
Index of Disadvantage is calculated on measures of: shown, it is the work of abstract texts such as the ICSEA
to strip out the local and everyday, in order to make
r QBSFOUBMFDPOPNJDSFTPVSDFT
practices comparable and transportable. However, by
r QBSFOUBMFEVDBUJPOBOEPDDVQBUJPO
ignoring the local and the diversity of populations, they
r "CPSJHJOBMJUZ
represent schools in ways that simply don’t make sense
r TUVEFOUNPCJMJUZ
to the participants who experience the daily/nightly
1 All statistics on Wheatville have been rounded or provided in
effects of such things as student mobility, distance from
a range to strengthen anonymity. services and regional poverty.

Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013 81


CORMACK & COMBERt)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPM

The principal’s and teachers’ experience of the difficulty of securing relieving teachers in his area,
NAPLAN and being a failing school he was restricted in how much time he or his teachers
To consider the way in which the language and prac- could undertake professional learning away from the
tices of NAPLAN positioned the principal and teachers school. So his keenness for ‘outside influence’ for his
in Wheatville and the effects on their experience, we middle and upper primary teachers and for himself as
conducted an extended interview with the principal, a principal from a ‘nation-wide’ project is understand-
Barry (a pseudonym), and a focus group discussion with able. The fact that his school was judged as failing
the teachers who taught the grades where NAPLAN before he was offered this opportunity is not stressed;
was used. Significant for all the participants was that rather, he ran with the affordances it provided and posi-
the school was assessed as failing through a diagnostic tioned himself as a principal who welcomed opportu-
review process initiated by the state education depart- nities to meet educators from interstate. It is clear he
ment as a result of poor performance in the first round also welcomed the chance to learn alongside his peers.
of NAPLAN. Interestingly, the whole notion of a diag- He ‘received’ information he believed he should have
nostic review has been imported into schools from had in teachers’ college and subsequently developed
the world of business, promoting analysis by outside confidence to question his teachers. Here we see the
experts as an unproblematic process. While this review principal making the most of such a program, when
had impacts on everyone, these effects were not neces- he is repositioned as a literacy leader and given some
sarily the same for the principal and the teachers. assistance in learning practices for working with his
teachers, such as observation and questioning.
The principal However, his ‘opening doors’ and ‘opportunities’
Barry, as principal, saw that the diagnostic review and rhetoric did not transfer into how he approached
follow up ultimately provided him with the opportu- change to literacy pedagogy in the school and class-
nity to learn ‘how to lead literacy in the school’. room. Here his language altered. Keywords included:
[W]e had the opportunity to be part of the literacy
‘focusing’, ‘zeroing’, ‘digging’, and ‘drilling’. The frame
and numeracy diagnostic review process that involves of reference was narrower, deeper, and closed rather
Central Office and District Office review, and peer than wider and open. This is not surprising. As Lingard
principals and staff, going into classrooms, looking at (2010, p. 131) points out, ‘a narrowed focus on literacy
the way teachers teach, looking at the documentation and numeracy’, is what the then Minister for Educa-
we have relating to literacy and numeracy, and looking
tion, and current Prime Minister, argued was required.
at our school development plans.
The language of ‘narrowing’ and ‘focusing’ appears
He explained that, following the initial Central
ubiquitous and not contestable.
Office and District Office review, he was then invited
to be part of a national project for principals. As a school we’ve zeroed in on reading this term,
because we felt … while some of our kids are below
[O]ne of the things that process did was open a door national benchmark in reading and writing, that
for me to be part of the ‘Principals as Literacy Leaders’ reading, that’s where we’ve got most of our resources,
project (emphasis added). that’s where we’ve got some key people on staff who are
He expressed his strong enthusiasm for that project, good resource people, and to go broadly literacy, it can
sometimes water stuff down, so we just wanted to go
describing it as ‘nation-wide’ and as ‘something that
narrower and a bit deeper with that (emphasis added).
I have really valued’. In the same vein, he mentioned
the opportunities that he believes teachers need ‘to Barry explained the need to use further testing such
take them out of their comfort zone’. We were struck as the Neale Analysis for Years 4–7 students, so that
in reading the transcript by the ways in which spatial teachers ‘can dig a bit deeper and find out more’ about
metaphors featured in his talk, as is highlighted by students who may appear to read fluently but under-
terms italicised in the quotations from the transcript. stand little of what they read. He wanted to get beyond
My keenness was to try and get some outside influence,
what he described as ‘throw away lines’ from teachers.
particularly with the Year 3–7 group, so it was just This relationship with data and the need to do some-
another opportunity for them to consider how they’re thing with it was of considerable interest to all the prin-
going about things (emphasis added). cipals we have interviewed. ‘Data’ was a keyword; as it
It is important to note that Barry was working as is across the field of education as part of an emerging
principal in a small rural school. Given his institutional ‘discourse of data’. There was a sense in which princi-
locale, the spatial metaphors are telling. He did not pals needed to learn the mysteries of data interpretation
have an assistant principal with whom he could consult and the implications for practice. As Smith (2006) and
or to whom he could delegate various tasks. Due to colleagues have pointed out, data allows for experts to

82 Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013


)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPMtCORMACK & COMBER

‘know’ about people’s experiences in ways that bracket willing subject appropriating the discourses which
out their lived experience; their everyday knowing. assessed his school as failing, taking on the responsi-
There is a range of contradictions and tensions in bility that he needed to learn how to lead literacy in
Barry’s account of the impact of mandated literacy his school, assiduously meeting departmental expecta-
assessment upon his work as a school principal. In the tions, and actively researching programs to ‘close the
interview Barry described surveillance and diagnosis of gap’. Still, he portrayed himself as someone who needed
the school (from the ‘office’) and then in turn by him of ‘skilling’ in terms of interpreting what the mandated
classrooms. In Smith’s terms the key texts – namely the tests show and the implications for practice they imply.
diagnostic review and the high stakes literacy tests – He concluded with a plea for help in that regard to the
unleash a chain of action and further text-mediated interviewer. The loss of trust in the teaching profession
practices, whereby people reorganise their work around and the undermining of the reliability of their judg-
the centrality of the data. While he welcomed opportu- ments which has been noted by critical policy analysts
nities, he saw a need to narrow and focus, to drill down. were evident here (see also Ball, 2001, 2003; Codd,
He was wary of teacher knowledge and judgment and, 2005). The principal not only questioned his teachers’
indeed, doubted his own professional knowledge. The capacities to make sense of, and work productively
discourses of data and improvement are not immedi- with, data, but also his own. As Blackmore (2010,
ately meaningful yet they trigger a range of responses p. 642) observes, ‘Leadership is about fear and desire –
associated with the need to know differently. the desire to make a difference, the desire to prove and
A repeated theme of wanting not to be bound by the improve oneself.’
limits of the local and what that might entail, including This insight applies to Barry and, in terms of Smith’s
being too comfortable and having low expectations, is (2005) approach to understanding translocal discourses
reiterated. There was also a belief that solutions might and practices on people’s everyday doings, it is clear
be found elsewhere by taking opportunities and opening that from the time when Barry’s school was designated
up in-school practices to review by outsiders. The as failing in terms of the NAPLAN results and the
resultant action however, and the resources required, subsequent external review, to the time when the inter-
must be found within the school, and the responsibility view was conducted approximately a year later, he had
for change rests with the principal. significantly reorganised his approach to being a prin-
Barry: I mean I guess, just on an aside, my concern
cipal. He literally subjected himself to what he could and
for that, as principal in a busy school, is it’s should learn about how to ‘lead literacy in the school’.
all very well to identify, but if it’s then the The new imperative to lift measurable standards in his
school needing to drive some sort of program school resulted in considerable attention and resources
or project, what support is there for me as being redirected to assessment of literacy. His account
leader for our staff to do more than we’re
indicated that he allocated a significant amount of his
already doing without resources, so it’s going
to be … time and that of his teachers on activities designed to
‘bump’ up their results on NAPLAN.
Researcher: So is there a potential criticism in there that
we’re doing more and more to identify but
not necessarily more and more to respond? The teachers
There were interesting patterns of similarity and differ-
Barry: I think there is, yeah, I think so. I think there’s
ence with the principals’ account when considering
a feeling at times that the school’s asked to do
more and more of a family’s responsibility, or what the teachers had to say about their experience of
a health agency responsibility, or a counsel- NAPLAN and the broader discourse of data. For the
ling role, whatever, and we don’t have the teachers as well as the principal, the review process
resources. which followed the early NAPLAN results which repre-
Barry almost apologised for raising his concern about sented Wheatville as a failing school was a key event.
the lack of extra resources ‘to close that gap signifi- However, while the principal was able to represent this
cantly’ and emphasised that his school ‘did the best as an opportunity for opening out his own knowledge
we could with the resources we had’. In an interview and experience, for some of the teachers it was the
which is peppered with the principal’s statements about stigmatisation of failure that stood out for its implied
meeting departmental expectations for improvement, judgement of their skill as teachers.
this is one of the few criticisms he made about the Our whole school got low figures over the last few years,
impact of mandated literacy assessment on his work. and that meant that last year we had like a review. There
He reported that he redirected the resources he had were some people that came to our school, remember
to meet the new priorities as he saw them. He was a that, a team came to our school, and interviewed every

Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013 83


CORMACK & COMBERt)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPM

one of us, and it really made me feel under pressure and These frames of reference obliterate teachers’ usual
like it was questioning my performance as a teacher, interpretive resources. In contrast to the principal, the
and my abilities, and made me feel quite low and flat … teachers’ talk was less about NAPLAN as an oppor-
They sent a team out to … the aim was to review why
tunity, and more about it as a problem to be encoun-
we were getting low figures, and maybe develop some
strategies, but it didn’t come across that way at all I tered, wrestled with and overcome where possible.
didn’t feel. I felt they were very, quite intrusive and The teachers were not without resources or ideas from
really not assisting us at all. They were more critical their own experience to deal with NAPLAN, but these
and judging and not sympathetic to the scenarios that were in discourses quite different from, or even cynical
we were [working] with (Year 3 teacher).
about, the promise of standardised testing. One of the
This extract illustrates the heightened anxiety that teachers noted that NAPLAN was the latest in a long
was evident when the teachers discussed the judgements line of curriculum and assessment reforms stretching
arising from the school’s NAPLAN results. It also illus- back more than two decades and resisted seeing the
trates Smith’s (2005) insight that knowing from the latest as the last word in what to do – after all, there
standpoint of a practitioner, in this case, the teachers, would be others to come, and each had its own new
is very different from the knowing that comes from lexicon to master.
abstraction, figures, and categorisation which ignores Unlike the principal, the teachers were highly critical
people’s experience, in this case to the point where of the test, describing it as: limited to testing only a few
the reviewers were not interested at all in the actual components of literacy; emphasising simple answers;
‘scenarios’ which teachers saw as key to their work. The too much about pencil and paper; featuring content
same labels, practices and texts can come to have very irrelevant to their students’ lives; and, tellingly, not
different impacts on people’s work, depending on their providing information that was particularly useful or
location or institutional position; and an educational timely for helping their students improve. On this last
consultant can ‘read’ a failing school without qualms. point, they noted that the timing of the results left little
Another teacher emphasised that it was the potential time to act on the information, especially as it was on
for public naming and shaming that was a key concern. only some students in their multi-grade classrooms.
Are they going the English and American models of
However, they were supportive of Barry on the need
putting them in publications and then, you know, to gather more information about their students, so
people will be going, ‘Oh, look at that school compared the perceived shortcomings of NAPLAN were compen-
to that school’, and parents are going to be going, ‘Oh sated for by bringing in other standardised tests that
well, I’m going to take my child out of that school’… were administered in a more timely way. Some teachers
(Year 4/5 teacher)
noted that the information from these tests was more
The central message, according to the teachers, was user friendly than earlier standards-referenced models
that only the standardised data count, and that the of assessment being promoted by the system, as they
NAPLAN results would come to be the sole measure of provided an age or grade level and for every student
a school’s reputation and of the work of the teachers. in the class. They also pointed out how many of the
As the year 6/7 teacher said, ‘There’s so many bril- assumptions behind NAPLAN and school improve-
liant things that take place, yet they’ll just look at the ment were problematic in a small rural school. Teaching
results’. At the same time, the teachers had taken up in multi-grade classrooms, information was provided
Barry’s message that standardised data were impor- about only some students and, due to the mobility of
tant and that improving results on the tests was the their student population, even those in the grades doing
key response to be made, but there was little sense of the test might have been elsewhere when the test was
support coming from beyond the school, possibly as a given, or moved on since.
result of the negative experience of the school review. The teachers clearly recognised the importance of
Thus teachers felt thrown back onto their own resources the test data for the reputation of the school and their
and histories in deciding how to move forward. For own performance as teachers, even as they disputed its
some, too, the language and logic of the discourse of validity for those who worked with students in poverty.
data was unfamiliar. They knew they had no choice but to engage in the
discourse of data, and try to do the best they could
I tell you what, it’s very brain daunting when they do to improve results. What this meant in practice, as
all of that in facts and figures and everything. I mean
they pointed out, was a narrowing of the curriculum
you’re trying to relate that to how that works with you
and what you’re doing, and you’ve really got to have and a loss of ‘variety’. The teachers described how the
a mathematical mind to understand some of the stuff response to the tests led to a focus on literacy, then
that they’re putting out. (Year 3 teacher) within that field, to spelling and reading. The reasoning

84 Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013


)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPMtCORMACK & COMBER

was that these were the areas that were high profile, and The repeated insistence on the need for ability group-
by concentrating on these, at least improvement might ings for teaching component skills of literacy – spelling,
be possible. This seemed the only sensible response in reading, writing – was significant and reflected the
practice given that the students were already behind school’s goals to improve their measurable performance
their peers when they arrive at the school gate. As one on standardised tests of literacy.
teacher put it, ‘your kids are going to be behind the Once again it’s split into four different groups at the
8-ball from the start, you know, to keep up with the moment in the class depending on their level of spelling.
standard, so yeah, it’s a bit of a tough ask’. Led by (Year 5/6 teacher)
Barry, the school had used its limited resources to buy
I do reading groups and I do writing groups and I do
in a US-based reading scheme Lexiles (http://lexile. spelling groups … On the spelling side of things at the
com), which promised to, ‘match readers with texts lower end we’re also doing basic sight words, frequency
essential for growth and monitor their progress toward words. (Year 3 teacher)
standards’. Thus the tests, even as they failed to provide Here we can see how teachers organised their time and
teachers with useful guidance for their teaching, came their students in order to accommodate different stand-
to regulate their pedagogy to some degree and to deter- ards of performance as determined by a range of tests.
mine the (newly narrowed and globalised) curriculum The logic of NAPLAN transfers into the classroom as
they offered. In spite of their belief in the importance teachers grapple with different year level expectations –
of responding to their students in sensitive ways, the a good example of what Smith (2005) refers to as the
NAPLAN review process overwhelmed local aspi- ‘translocal’ organisation of activity through coordi-
rations through its emphasis on standard forms of nating texts. The acceptance of standards-grouping is
(evidence-based) practice (see also Nichols & Griffith, not surprising, nor it being considered part of a possible
2009 for an example from Canada). As is shown in the solution for raising test scores. However, elsewhere in
next section, the ways that students could be talked the discussion it was clear that this ethos was incon-
about was also strongly affected by the discourse of sistent with teachers’ preferred ways of operating and
data. their understandings about equity. The very pragmatics
of grouping may have unanticipated effects.
NAPLAN and the discursive construction 4/5 teacher: This year for my NAPLAN somebody else
of the poor rural student administered the test. I had only about five
A number of our research projects have investigated students that had to sit the test, so those
how normative discourses work to constitute the students worked with a …
ideal (and indeed the problem) student with respect 3 teacher: So we have multi year level classes for
to English literacy (Comber, 1997; Cormack, 2011). starters, so there’s one problem, so your class
We have always been concerned to understand the fall- had to be split.
out for students and, indeed, teachers that emanates 4/5 teacher: Yep. Those students that were going to do
from the unproblematic insistence on the norm at the that test had to work with a Student Support
expense of diversity (Luke, 2012). Given that Wheat- Officer and go out with them for a number
ville was considered a failing school with the advent of of hours over a week or two weeks I think it
high-stakes testing, we are interested in the effects on was, and go through last year’s test and look
at questions, talk about concepts, all of that
teachers’ discursive and pedagogical practices as well sort of …
as the interpretive resources they employed at this time
and place to constitute their students. 6/7 teacher: Which puts them at a disadvantage as
opposed to those that are in a classroom
In small rural primary schools, staffing formulas
where the teacher is, or has, that year level,
mean that it is inevitable that some classes are multi- and where your kids are having to go out for
age, such as 3/4 or 5/6/7 for example. However, this, and like Teacher 4 had, it’s mostly year
students are tested in year level cohorts and perfor- 5, so a bigger group of year 5s, so what she’s
mances on NAPLAN are reported in terms of year teaching she can sort of explicitly teach what
they need to be doing, whereas these other
levels for individual schools, allowing Year 3 cohorts
kids only have a school support person in
around Australia to be compared. It is not surprising order to develop these kids.
in this policy context that students came to be spoken
about in ways that differentiated them from the norm. Here the teachers explain how the logic of testing
In the teacher focus group, students were variously year-level cohorts impacts on what they do and how
described as members of a ‘group’ (year level, ability or students are prepared for the tests and by whom, with
age) and/or as ‘disadvantaged’, as we illustrate below. one teacher raising questions about equity. One group

Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013 85


CORMACK & COMBERt)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPM

of Year 5 students was actually prepared to take the 4/5 teacher: Seven might be a Suburb A or a Suburb B
test, not by their classroom teacher, but by an aide. [wealthy suburbs in the state’s capital city].
Here we can see how students’ educational disadvan- 6/7 teacher: The leafy suburbs apparently.
tage may be unwittingly escalated when the school’s
4/5 teacher: Well we’re a 2, so that’s, you know, and so …
actions, intended to give more assistance in small
groups, actually result in less teaching by a qualified 3 teacher: When you come in here you’re working hard.
educator. The Year 6/7 teacher had his students re-take 4/5 teacher: It’s those low category schools, you know,
the test so that he could use it to identify problem areas. and it’s all based on income and social disad-
vantage. I mean those low category schools
That’s the most important thing that I use it for,
are the ones that are going get most of the
because I’ve got immediate feedback as to what needs
fallout from the NAPLAN.
to happen so I can modify my program or modify what
we’re doing, just to look at what they’ve missed out on, The teachers draw on the state education depart-
where the group is lacking. (Year 6/7 teacher) ment’s Index of Disadvantage to situate their school
The mentality associated with the testing becomes on a continuum from most disadvantaged to the most
one of looking for deficits, which is also implied in the affluent, concluding that working in their school was
principal’s stated goal: ‘what I’d like us to do better is officially ‘hard’. This discussion had been initiated by
to look at individual students and identify their gaps, one of the teachers recalling the history of Wheatville in
and then try and address those gaps’. We see that if terms of class, but stressed that politicians and policy-
the student is framed within a discourse of testing and makers (‘they’) only take into account the test results
diagnosis, then the pedagogical problem is also framed and ignore ‘the brilliant things’ being achieved by the
as the need to identify and apply a cure. This is an students and teachers in the school. In the dialogue
example of the way that the discourse of data incor- above, the teachers invoke the school’s authorised
porates tiers of decisions and actions which reach into category of disadvantage to explain the test results and
daily classroom life (Griffith & Smith, 2005) and come point out that it will be schools such as theirs that ‘get
to affect how students are actually labelled and treated. most of the fall-out from NAPLAN’ (see also Comber,
As Nichols and Griffith (2009, p. 247) point out, ‘people 2012). As they continue the conversation we can see
learn to describe their work in these terms’. We do not how these students and their families become a problem
mean to suggest that there was no contestation amongst once again for their lack of literacy. The practices of
these teachers about this logic, nor that they subscribed parents are questioned in terms of their provision of
totally to deficit discourses. The situation was consid- adequate learning opportunities.
erably more complex with teachers employing a range
of (sometimes contradictory) discourses to talk about 3 teacher: When you think of that in terms of literacy
their students’ life experiences as unconnected to the I mean: What’s literate about their parents?
What’s in their home that supports literacy?
content of the tests, or to the knowledge that counts.
How are they spending their money?
Hence, while the teachers reorganised their literacy
lessons in terms of groups based on test standards, they 4/5 teacher: How is it valued?
by no means saw such tests as unproblematic. While 3 teacher: Do they get a newspaper? Can they afford
they acted pragmatically on the findings in a somewhat that? What do they do, do they tune into tele-
compliant fashion, they noted issues related to relia- vision? Do they read a book at night time?
bility and also with the fairness of the texts of the tests 6/7 teacher: How to get their attention and keep it and
themselves and reiterated the importance of considering [inaudible – overtalk].
context in terms of their work and their students. In
3 teacher: How to get them involved in what’s going
the focus group discussions, teachers grappled with the on. Do they actually understand what their
consequences of NAPLAN and the wider discursive children are doing? Are their children even
construction of the poor rural student. better off than they are?
6/7 teacher: Are they at school?
4/5 teacher: Another way of thinking about is they have,
I think it’s category 1 to category 7 levels of 4/5 teacher: So with those low categories …
disadvantage. Well we’re a category 2, so
we’re pretty … Aboriginal schools are 1s 3 teacher: Access to computers, you know, money prob-
so … lems and things.

Researcher: And so category 7 is like … 4/5 teacher: So for those low category schools, I mean like
we said before, it’s no big surprise when we
3 teacher: One is the hardest, 2 is the second hardest. get the NAPLAN stuff back.

86 Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013


)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPMtCORMACK & COMBER

These propositions are all too familiar in that here we to limit reputational damage to the school and them-
see a return to the ubiquitous circularity of educational selves. As we have seen, however, the resources for that
blame and the near impossibility of ‘getting out of response may be very limited in a rural school serving
deficit’ (Comber & Kamler, 2004) or shifting ‘default a poor community. There seemed little opportunity for
positions’ (Johnston & Hayes, 2007), whereby the the bootstrapping processes assumed in the logic of
poor are blamed for their lack of effort, inappropriate standards-reform, or for teachers to learn from those
spending, inadequate practices and so on. So, while they places where it is apparently done better. The reality
begin by recognising that Wheatville is a disadvantaged was not of a lively market for reform, but the imposition
school with high levels of transience, low income and of standards-based solutions and subsequent take-up of
social disadvantage, they then buy into to the common global, generic curriculum from a large multinational
sense argument that the parents are responsible for the company in the form of a reading scheme. Both the
problems they experience as teachers. The students may principal and the teachers were left to struggle with
not be giving their ‘attention’, or even be ‘at school’; at the implications of meaning of the data they collect
home they may not be accessing computers, books and which, ironically, only served to confirm what they
parents who should help them become literate. Lack is already knew – that their students were ‘behind the
a key theme here. The teachers then reiterate that the 8-ball’ – and simultaneously to discount what was actu-
policy-makers and politicians (‘they’) still expect the ally being accomplished.
teachers to do better. Students, in turn, are labelled, grouped and taught in
3 teacher: But they want us to be better than we are.
relation to the deficits the tests reveal. They experience a
curriculum which focuses on what they can’t do, which
4/5 teacher: Oh yeah, I know that, and we aspire to be also squeezes out other, more hopeful and energising,
better than we are, but for those low category
topics and activities. Notwithstanding the limits and
schools, when those results come back, and
if they start using it in a much wider commu- the pressures, some teachers still aim for ethical peda-
nity-based publications or whatever … gogic practices even in the face of demands to focus
on minimal standards. These teachers can call on long
3 teacher: ‘This area is a low achiever, don’t send them
there’. experience in finding ways of working with students
that don’t come from a prescribed program manual,
4/5 teacher: … [T]hose schools that are high category,
and which are tailored to their students. For example,
they’re going to go ‘Oh yeah, well we’re
doing alright, we could lift our game but one teacher reported having his students re-take the test
we’re doing OK’. It’s all the 1s, 2s, 3s, they’re in order to enable him help them with areas they did not
all the ones, and what have we learned from understand. Rather than waiting for the results to come
that, really, in the big picture things, what several months later, he took pedagogical responsibility
have we learnt? Yes, these kids are disadvan- for immediately teaching them the specific strategies
taged. Yes, their literacy and numeracy skills
and approaches to the tasks indicated as a problem by
are lower. Gee, that’s a surprise!
the tests. Despite teachers’ best intentions, however,
While the teachers accept their responsibility to aspire standardised mandated assessments do have differen-
to do better, buying into the bootstrapping rationality tial effects. Rather than ‘closing the gap’ or amelio-
and continuous improvement discourses, they fear the rating educational disadvantage, policies may, indeed,
backlash from the publicity associated with failure result in a limited and reduced education for children
and what it can do to their community. They predict growing up in rural poverty. Results from the mandated
that their enrolments may diminish as they rehearse test led to a further of investment of money and time in
what parents might say to each other, even imag- testing and the purchase of levelled reading schemes as
ining a description of Wheatville portrayed as a ‘low numerical comparative data is seen as all the matters.
achieving area’. It’s not so surprising when they end The ways in which the work of teachers is coordinated
rather cynically pointing out they learn little through beyond the district or even the state become clear along
these processes of testing and public accountability that with the significant risks of educators coming to doubt
helps them to do their work any better. Meanwhile, the their own judgments.
leafy green schools are ‘doing alright’. One key problem with the discourse of data and its
various technologies, is that the materiality of poverty
Conclusions and place is backgrounded while reconstituting
With limited resources at their disposal, teachers and teachers’ work as sets of generic techniques. In this
school leaders have little choice but to respond with regime, schools are responsible for failure and left with
respect to the logic of the discourse of data and work the problems that are identified through standardised

Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013 87


CORMACK & COMBERt)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPM

tests. Educational researchers have long stressed America’s youth’ in S.L. Nichols (Ed.) Educational Policy
the importance of contesting the deficit discourses and the Socialization of Youth for the 21st Century. New
concerning the learning capabilities of students York: Teachers College Press.
Blackmore, J. (2004). Restructuring educational leadership in
growing up in poverty. Yet we have found it difficult to
changing contexts: A local / global account of restructuring
help teachers to assemble other discursive resources for in Australia. Journal of Educational Change, 5, 267–288.
understanding the material realities of poverty without Codd, J. (2005). Teachers as ‘managed professionals’ in the
assigning blame. Not surprisingly teachers can resort to global education industry: The New Zealand experience.
deficit explanations which, in turn, pass on the blame Educational Review 57(2), 193–206.
for failure to parents and the community. This is not Comber, B. (2012). Mandated literacy assessment and the
reorganisation of teachers’ work: Federal policy and local
the only story to be told of the school, however. Both
effects. Critical Studies in Education. 53(2), 119–136.
the principal and the teachers tried to work around Comber, B. (1997). Managerial Discourses: Tracking the local
the shortcomings of the situation in which they found effects on teachers’ and students’ work in literacy lessons.
themselves and had past experience of success – limited Discourse: Studies in the Cultural Politics of Education,
and transitory as it may have been – to build on ‘the 18(3), 389–407.
brilliant things’ that have worked for students in diffi- Comber, B. & Cormack, P. (2011). Education policy media-
tion: Principals’ work with mandated literacy assessment.
cult circumstances. Unfortunately, within the discourse
English in Australia, 46(2), 77–86.
of data, such experience doesn’t ‘count’. Comber, B & Hill, S. (2000). Socio-economic Disadvan-
To return to Dorothy Smith’s (2005) desire for a tage, Literacy and Social Justice; Learning from Longitu-
‘sociology for people’, whereby people come to under- dinal Case Study Research, The Australian Educational
stand how their work is being coordinated, our hope Researcher, 27(9), 79–98.
is that research such as this which shows how policy Comber, B. & Kamler, B. (2004). Getting out of deficit:
Pedagogies of reconnection. Teaching Education, 15(3),
is enacted in particular situations will create the possi-
293–310.
bilities for change and for educators to imagine ‘richer Cormack, P. (2011). Reading pedagogy, ‘evidence’ and educa-
and more intelligent’ accountabilities (Lingard, 2010, tion policy: Learning from history? Australian Educational
p. 132). The dominance of comparative numerical Researcher, 38(2), 133–148.
data about student performance in limited aspects of Green, B. & Letts, W. (2007). Space, equity and rural educa-
literacy, as the only data that counts for student and tion: a trialectic account. In K. Gulson & C. Symes (Eds.),
Spatial Theories of Education: Policy and Geography
school success, is simply not good enough.
Matters (57–76). London: Routledge.
Griffith, A. & Smith, D. (2005). Mothering for schooling.
Acknowledgement New York: RoutledgeFalmer.
This paper is based on work from the research project, Johnston. K & Hayes, K. (2007). Supporting student success
Mandated Literacy Assessment and the Reorganisation of at school through teacher professional learning: the peda-
Teachers’ Work is an Australian Research Council (ARC) gogy of disrupting the default modes of schooling. Inter-
Discovery Project (DP0986449) between the University of national Journal of Inclusive Education, 11(3), 371–381.
South Australia, Queensland University of Technology and Kostogriz, A. & Doecke, B. (2011). Standards-based account-
Deakin University in Australia and York and Victoria Univer- ability: reification, responsibility and the ethical subject.
sities in Canada. The chief investigators are Barbara Comber, Teaching Education, 22(4), 397–412.
Phillip Cormack, Helen Nixon, Alex Kostogriz and Brenton Labaree, D.F. (2011). The lure of statistics for educational
Doecke. Partner investigators in Canada are Dorothy Smith researchers. Educational Theory, 61(6), 621–632.
and Alison Griffith. The views expressed in this paper are
Lingard, B. (2010). Policy borrowing, policy learning: testing
those of the authors only.
times in Australian schooling. Critical Studies in Educa-
tion, 51(2), 129–147.
References Luke, A. (2012). After the testing: Talking and reading
Alexander, R. (2010, 10 March). The Perils of Policy: Success, and writing the world. Journal of Adolescent and Adult
Amnesia and Collateral Damage in Systemic Educational Literacy 56(1), 8–13.
Reform. Paper presented at the Miegunyah Distinguished McWilliam, E. & Singh, P. (2004), Safety in numbers?
Visiting Fellowship Program 2010 public lecture. Teacher collegiality in the risk-conscious school. Journal
Ball, S. (2001). Performativities and fabrications in the of Educational Inquiry 5(1), 22–33.
education economy. In D. Gleeson & C. Husbands (Eds.). Nichols, N. & Griffith, A. (2009). Talk, texts, and educa-
The performing school: Managing and learning in the tional action: an institutional ethnography of policy in
performance culture (210–226). London & New York: practice. Cambridge Journal of Education, 39(2), 241–255.
RoutledgeFalmer. Poster, M., Lash, S. & Morris, P. (1996). Databases as
Ball, S.J. (2003). The teacher’s soul and the terrors of perform- discourse, or electronic interpellations. In P. Heelas (Ed.),
ativity. Journal of Education Policy, 18(2), 215–228. Detraditionalization: Critical Reflections on Authority
Berliner, David C. (2013 forthcoming) ‘Sorting out the effects and Identity (277–293). Cambridge, MA: Blackwell
of inequality and poverty, teachers and schooling, on Publishers.

88 Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013


)JHITUBLFTMJUFSBDZUFTUJOHBOEMPDBMFGGFDUTJOBSVSBMTDIPPMtCORMACK & COMBER

Smith, D. (Ed.) (2006). Institutional ethnography as practice. Teese, R. (2000). Academic Success and Social Power:
Lanham: Rowman & Littlefield Publishers. Examinations and Inequality. Carlton South: Melbourne
Smith, D. (2005). Institutional Ethnography: A sociology for University Press.
people. Lanham: AltaMira Press. Teese, R. & Polesel, J. (2003). Undemocratic Schooling:
Smith, D. (1987). The everyday world as problematic: a femi- Equity and Quality in Mass Secondary Education in
nist sociology. Boston: Northeastern University Press. Australia. Carlton, Vic.: Melbourne University Press.
Somerville, M., Plunkett, M. & Dyson, M. (2010). New Thomson, P. (2009). School leadership: Heads on the block.
teachers learning in rural and regional Australia. Asia- London & New York: Routledge.
Pacific Journal of Teacher Education, 38(1), 39 - 55.

Phillip Cormack is Adjunct Associate Research Professor of Education in the School of Education at the
University of South Australia, Australia. His research interests include the history of adolescence, and
contemporary and historical perspectives on literacy policy, curriculum and pedagogy. He currently works on
projects on the history of reading education in Australia and the impact of contemporary standardised testing
reforms on teachers’ work.

Barbara Comber is a Research Professor in the Faculty of Education at Queensland University of Technology.
She is particularly interested in literacy education and social justice. She has conducted a range of research with
teachers working in high poverty and culturally diverse communities. She has co-edited a number of books
including the forthcoming International Handbook of Research in Children’s Literacy, Learning and Culture (Hall
et al. in press), Literacies in Place: Teaching environmental communications (Comber et al., 2007) Turn-around
pedagogies: Literacy interventions for at-risk students (Comber & Kamler, 2005) Look again: Longitudinal
studies of children’s literacy learning (Comber & Barnett, 2003).

Australian Journal of Language and Literacy, Vol. 36, No. 2, 2013 89


Copyright of Australian Journal of Language & Literacy is the property of Australian Literacy
Educators' Association and its content may not be copied or emailed to multiple sites or
posted to a listserv without the copyright holder's express written permission. However, users
may print, download, or email articles for individual use.

You might also like