You are on page 1of 4

Language Assessment in Foreign Language Education: The Struggle over Constructs

Author(s): Tim McNamara


Source: The Modern Language Journal, Vol. 91, No. 2 (Summer, 2007), pp. 280-282
Published by: Wiley on behalf of the National Federation of Modern Language Teachers
Associations
Stable URL: http://www.jstor.org/stable/4626015
Accessed: 16-11-2015 00:02 UTC
REFERENCES
Linked references are available on JSTOR for this article:
http://www.jstor.org/stable/4626015?seq=1&cid=pdf-reference#references_tab_contents
You may need to log in to JSTOR to access the linked references.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://www.jstor.org/page/
info/about/policies/terms.jsp
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content
in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship.
For more information about JSTOR, please contact support@jstor.org.

National Federation of Modern Language Teachers Associations and Wiley are collaborating with JSTOR to digitize,
preserve and extend access to The Modern Language Journal.

http://www.jstor.org

This content downloaded from 161.112.232.221 on Mon, 16 Nov 2015 00:02:13 UTC
All use subject to JSTOR Terms and Conditions

TheModernLanguageJournal91 (2007)

280
Change:TheMagazineofHigherLearning,34, 5058.
Davidson,D. E., & Lehmann,S. (2005). A 25-yearlongi-

tudinalanalysis
ofthelanguagecareersofACTR

studyabroad alumni. Russian LanguageJournal,

55, 193-221.
Green,M. F., Siaya,L., & Porcelli,M. (2002). One
Attitudes
aboutinternational
education
yearlater.
sinceSeptember11. In M. Green(Ed.), Public
DC: American
opinion
pool(pp.1-8).Washington,
Councilon Education.
McPherson,P. (2005). OpenDoors 2005: Lincoln Commissionprojections.
Report on InternationalEd-

ucationalExchange2005. Retrieved
June 11,
2006, fromhttp://opendoors.iienetwork.org/?
p=69735
EdOpen Doors (2006). Reporton International
ucational Exchange Online. RetrievedJanuary2, 2007,fromhttp://opendoors.iienetwork.
org/?p=89211
theselfin thestudy
Pellegrino,V. A. (2005). Constructing

abroadcontext.
Cambridge:
Cambridge
University
Press.

Standards
the
forForeign
LanguageLearning:Preparingfor
21st Century(2006). National StandardsColabo-

rative.
KS:AllenPress.
Lawrence,

LanguageAssessmentin ForeignLanguage Education:The StruggleOver


Constructs
Australia
TIM McNAMARA,TheUniversity
ofMelbourne,
Language assessmentis likelyto playa crucial
role in policiesinvolvingforeignlanguage education,giventhatcontrolof the reportingprocess,
as governments
all overtheworldhavediscovered,
is an effective
wayof controllingcurriculum.This
phenomenon raisesthe generalissue of the relationshipbetweenlanguage policyand language
testing.How does currentassessmenttheoryunderstandthis relationship?In attemptingto answerthisquestion,we immediatelyencountera

problem.
Assessmenttheoryhas focusednot on the polof
icyoriginsof assessmentbut on the fallibility
assessment.Language testsin educationalsettings
are proceduresformakingclaimsabout learners
based on a limitedamountofevidence-often not
morethantheevidencethatitis possibleto obtain
in theshorttimeavailableforthe test.The resultingtestscoreis thebasisfora claim,nothingmore,
about thecandidate'sstandingin relationto a domainofknowledgeor skillor capacityto carryout
particularsetsof communicativetasksand hence
his or her readinessforentryintoparticularcommunicativecontexts.This pointis sometimesobscured by the factthatthe claim is expressedin
the formof a numberor a labeled categoryon
a scale, whichseems to giveit a spuriouskind of
scientificobjectivity.
Now,in orderfortheseclaimsto be defensible,
we need to have a strongrelationshipbetween
claimand supportingevidence,a relationshipthat
and challenge,a
itselfis subjectto interrogation
A
known
as
validation.
language testwithprocess
out validationresearchis like a police forcewithout a courtsystem:unfairand dangerous.But the
emphasiswithinvaliditytheoryon evidence and

itsrelationshipto claimsbegs the questionof the


source and characterof the claims themselves.
What claimsdo we wantto make about learners?
Whythose claims and not others?In whatterms
and in whatwordingwilltheclaimsbe expressed?
Whose wordingdoes thisrepresent-thatof eduTo
catorsand theirstudentsor of policymakers?
whatextentdo claimsconstitutepotentialsitesof
struggleoverwhatcountsin language education?
In thesestruggles,
whosevoice willprevail?
Such issues are addressed only in part within
validitytheory.On the one hand, major theorists
such as Messick (1989) and Kane (1992, 2001)
in generaleducational assessmentand Bachman
(1990, 2005) in language testingincluded questions of testuse withinthe scope of validation.
But the discussions,in the main, focused on investigating
potentialsourcesofunfairnessin tests,
and addressedonlyweaklythesocial and political
contextin which testsoperate. For example, in
Bachman's (1990) work,contextis understoodin
cognitiveterms,reflectingthe broader traditions
of both linguisticsand educationalmeasurement
fromwhich language testingdraws.The target
language use situationis conceptualizedin terms
of componentsof communicativelanguage ability,whichin turnis understoodas abilityto handle
thetargetlanguageuse situation.The situationor
contextis projectedonto thelearneras a demand
for a relevantset of cognitiveabilities;in turn,
thesecognitiveabilitiesare read onto thecontext.
Discussionof thewidersocial contextbeyondthe
immediatetargetuse situationis restrictedto the
intended and unintended consequences of the
testin studiesofwashback(Cheng, Watanabe,&
Curtis,2004), not to thewidersocial meaningof

This content downloaded from 161.112.232.221 on Mon, 16 Nov 2015 00:02:13 UTC
All use subject to JSTOR Terms and Conditions

281

Perspectives
the testin its context.There is a gap, in other
words,in our theorizingof the social contextin
which language testsfindtheirplace. A notable
exceptionis themovementknownas CriticalLanguage Testingrepresentedespeciallybythework
of Shohamy(1998, 2001).
In general educational assessment,Messick
(1989) stressedthesocial and politicalvaluesembodied both in assessmentconstructsand in assessmentpractice,and the need to considerthe
impact of testson classroomsand societymore
generally.But Messick'scommitmentto the need
to articulatevalueswentonlyso far,in thathe did
notseem to have envisagedthesituationin which
thevaluesimplicitin testswouldbe imposedfrom
outside,and validationof testconstructswould
be reduced to negotiatingtheiracceptabilityto
stakeholders.But this external policy intervention in assessmentisjust whatwe see happening
all overthe world,in diversesettingsof language
education.
An adequate theoryof the social contextin
which language testsoperate would bringabout
significant
changesin thewaywe thinkabout the
role of testswithinlanguage educational policy.
It would impel us to envisagethe testtakernot
as an individualwithcognitiveattributesdefined
in termsof the theorizedfeaturesof the immediate situationof use, but as a social being whose
is a functionof subjectpositionsrealsubjectivity
ized in the testitself.From thisperspective,tests
become technologiesof subjectivity.
They act as
mechanismsbothforthedefinitionofsubjectpositionsand forthe recognitionof subjects.Tests
create the identitiestheymeasure (McNamara &
Roever,2006).
Whatare theimplicationsofall thisforthecurrentdiscussionsof U.S. language policyand the
role of language testingwithinthatpolicy,given
thatwe can easilyenvisagecontextswheregovernmentpolicyand educationalvaluesmayconflict?
Answersto this question may emerge in research on the politicizationof language testing
withinother contexts.For example, in Europe,
thedictatingoftestconstructs
in foreignlanguage
educationas a functionof governmentpolicyhas
become almost routine,at everylevel of education, by the necessityof linkingall assessment
to levels on the Common European Framework
of Referencefor Languages (CEFR; Council of
Europe, 2001) in orderto secureanykindofgovernmentfunding.Analysisoftherole oftheCEFR
withineducation systemsin Europe has begun
to appear (e.g., Fulcher,2004). What is clear is
thatthe intellectualand educationalinadequacy
of the constructin the CEFR, withits originsin

1970snotional/functional
syllabusesforlanguage
teaching,is notopen to challenge,and is evenin a
sense irrelevant,
because ofitsoverridingrole as a
withineducationalsystems.
reportingframework
Also in Europe, criticalattentionis being given
to theuse oflanguage testsin contextsof debates
over immigrationand citizenship,an association
witha traditionas old as the shibbolethtest(McNamara,2005). In Japan,Akiyama(2004) has examined how deep culturalvalues in the school
contextworkagainstthe possibilityof the assessment of spoken language. In Australia,Moore
(2005) has provideda detailed analysis,drawing
on Foucault's (1975/1977) theories of governof the struggleover the constructof
mentality,
assessmentin the Englishlanguage education of
adult immigrantsin Australia.Also in Australia,
Elder (1997) has analyzedthewayin whichstruggle foraccess toscarcehighereducationresources
betweenethnicminority
communitiesand theAnglo mainstreamin Australiais covertly
playedout
in whatappear on the surfaceto be purelytechnical adjustmentsof testscoresin highstakessecond language examinations.And in the United
States,the impact of the intersectionof federal
governmentpolicy and language assessmentin
the case of the No Child LeftBehind Act is the
subjectof research,includingitsnegativeimpact
on bilingual education (Byrnes,2005; Evans &
Hornberger,2005; McNamara & Roever,2006).
There has long been a debatein theUnitedStates
over the constructimplicitwithinpolicy-related
frameworks
such as the Proficiency
Guidelinesof
the American Council on the Teaching of Foreign Languages (ACTFL), and similardebate has
begun in a smallwayon thecontentand function
of the StandardsforForeignLanguage Learning
(ACTFL, 1996; Schechtman,2005). In all thisresearch and discussion,we need a betterunderstandingof the role of scholarsin what are essentiallypoliticaldebates; the veryholding of a
conferenceon language policyand foreignlanguage educationis an interesting
example of the
issue.We should perhapsbe awareof the relative
impotenceof scholars,itselfnot a newstory.
Movingbeyondthe parameterslaid out within
currenttheoriesofvalidity,
even progressiveones
such as those of Messick (1989), Kane (1992,
theso2001), and Bachman (1990), to investigate
cial meaningoftestsand theirsocial impact,takes
us into relativelyunchartedwaterstheoretically.
The problemhere is thatwe need an adequate social theorytoframetheissuesthatwewishto investigate.The mostproductiveand relevantof these
theories,for example, Foucault's (1975/1977)
theorizationof the functionof examinationsin

This content downloaded from 161.112.232.221 on Mon, 16 Nov 2015 00:02:13 UTC
All use subject to JSTOR Terms and Conditions

TheModernLanguageJournal91 (2007)

282
are not only relativelyunfamiliarto
modernity,
language testersbutin factchallengemanyofthe
fundamentalepistemologicaland ontologicalassumptionsof the field,and thusrequireintellectual staminaand confidenceto negotiate.But as
the truesocial functionand impactof language
tests,and the involvementof language testersin
the implementationof language policy,become
clearer,this researchbecomes more significant
and more urgent.

N.H. (2005).NoChildLeftBeEvans,B.,& Hornberger,


federallanguage
hind:Repealingand unpeeling
educationpolicyin theUnitedStates.Language
Policy,4, 87-106.
Foucault,M. (1977). Disciplineandpunish:Thebirth
ofthe

(A.Sheridan,
Trans.).London:AllenLane.
prison

(Originalworkpublished1975.)
The Common
Fulcher,G. (2004). Deluded byartifices?
European Frameworkand harmonization.Lan1, 253-266.
guageAssessment
Quarterly,
Kane, M. T. (1992). An argument-basedapproach to
Bulletin,112, 527-535.
validity.Psychological

theconcernsin validity
Kane,M. T. (2001). Current

38, 319ory.JournalofEducationalMeasurement,
342.
McNamara, T. (2005). 21st centuryshibboleth:Laninto
a
T.
tests
guage tests,identityand intergroupconflict.LanAkiyama, (2004). Introducingspeaking
examination.
UnguagePolicy,4, 1-17.
Japaneseseniorhighschoolentrance
ofMel- McNamara,T., & Roever,C. (2006). Language testing:
doctoral
dissertation,
University
published
Thesocialdimension.
Malden, MA: Blackwell.
bourne,Australia.
AmericanCouncilforthe Teachingof ForeignLanMessick,S. (1989). Validity.In R. L. Linn (Ed.), Educationalmeasurement
(3rd ed., pp. 13-103). New
guages. (1996). Standards
forforeign
languagelearnYork:Macmillan.
Yonkers,NY:
ing: Preparingfor the 21st century.
"thetarget
ACTFL.
Moore, H. M. (2005) Identifying
population":
A genealogy
considerations
in lanBachman,L. F. (1990). Fundamental
forEnglishas a Second
ofpolicy-making
Press.
Oxford:
OxfordUniversity
Language(ESL) in Australianschools(194 7-1 997).
guagetesting.
andsupporting
a case
L. F. (2005).Building
Bachman,
Unpublished doctoral dissertation,Ontario Institutefor Studies in Education, Universityof
fortestuse. LanguageAssessment
2, 1-34.
Quarterly,
H. (Ed.). (2005).Perspectives.
Modern
Toronto,Canada.
Language
Byrnes,
An apSchechtman,R. (2005). Whither"communities"?
Journal,89, 248-282.
A. (2004). Washback
praisal of foreignlanguage education in lightof
Cheng,L.,Watanabe,
Y.,& Curtis,
at Berkethenationalstandards.LanguageTeaching
and methods.
in languagetesting:
Researchcontexts
Mahwah,
ley,21, 10-11.
NJ:Erlbaum.
and beE. (1998). Criticallanguagetesting
Council ofEurope. (2001). Common
EuropeanFramework Shohamy,
and asyond. Studiesin EducationalEvaluation,24, 331forlanguages:Learning,teaching
ofReference

REFERENCES

Press.
sessment.
Cambridge
University
Cambridge:

Elder, C. (1997). What does testbias have to do with


fairness?LanguageTesting,14, 261-277.

345.

perspecShohamy,E. (2001). Thepoweroftests:A critical


tiveon theusesoflanguagetests.London: Pearson.

ClosingRemarks

ROBERT BLAKE, Universityof California,Davis


CLAIRE KRAMSCH, Universityof California,Berkeley

The University
of CaliforniaConsortiumconof Californiaat Berkeferenceat the University
to
was
intended
ley
provide a forum where
and politieducators,administrators,
researchers,
cians could openly discuss the goals of foreign
language education in all its humanistic,political, economic, and professionaldimensions.We
were conscious that thiswas a tall order. Scholars in the humanitiesdo not usuallytalk to researchersin education,nor do politiciansneceswithpeople fromtheacademy.
sarilysee eye-to-eye
The tone was one
But thismeetingwas different.
of distinctconcern and urgency;participantslistened to each other,debated, argued, and did

not shy away from expressingtheir fears and


disagreements.
The issue ofhowwe understandotherpeople's
memories,aspirations,and worldviewsis inseparable fromhow we understandourselves.Any
discussionof foreignlanguage education in the
United Statesmustbe embedded in a largerdiscussion about the natureof Americanpluralistic
democracywithina multipolarglobal world.The
domesticideal e pluribusunumchallenges us to
defineour common identitybeyonda pursuitof
individualhappinessthatknowsonlythelanguage
of business.We are now confrontedwithothers
who see us as (unum) interpares,and whowonder

This content downloaded from 161.112.232.221 on Mon, 16 Nov 2015 00:02:13 UTC
All use subject to JSTOR Terms and Conditions