Professional Documents
Culture Documents
Ethics and Context in Second Language Testing: Rethinking Validity in Theory and Practice 1st Edition M. Rafael Salaberry
Ethics and Context in Second Language Testing: Rethinking Validity in Theory and Practice 1st Edition M. Rafael Salaberry
https://textbookfull.com/product/rethinking-civic-participation-
in-democratic-theory-and-practice-1st-edition-rod-dacombe-auth/
https://textbookfull.com/product/kingdom-ethics-following-jesus-
in-contemporary-context-second-edition-gushee/
https://textbookfull.com/product/practice-and-automatization-in-
second-language-research-perspectives-from-skill-acquisition-
theory-and-cognitive-psychology-1st-edition-suzuki/
https://textbookfull.com/product/taboos-and-controversial-issues-
in-foreign-language-education-critical-language-pedagogy-in-
theory-research-and-practice-1st-edition-christian-ludwig/
Electricity in fish research and management : theory
and practice Second Edition Beaumont
https://textbookfull.com/product/electricity-in-fish-research-
and-management-theory-and-practice-second-edition-beaumont/
https://textbookfull.com/product/procurement-and-transplantation-
of-abdominal-organs-in-clinical-practice-rafael-diaz-nieto/
https://textbookfull.com/product/stimulating-student-interest-in-
language-learning-theory-research-and-practice-1st-edition-tan-
bee-tin-auth/
https://textbookfull.com/product/rethinking-metonymy-literary-
theory-and-poetic-practice-from-pindar-to-jakobson-1st-edition-
matzner/
https://textbookfull.com/product/shadowing-as-a-practice-in-
second-language-acquisition-connecting-inputs-and-outputs-1st-
edition-shuhei-kadota/
ETHICS AND CONTEXT IN
SECOND LANGUAGE TESTING
This innovative, timely text introduces the theory and research of critical
approaches to language assessment, foregrounding ethical and socially
contextualized concerns in language testing and language test validation in
today’s globalized world.
The editors bring together diverse perspectives, qualitative and quantitative
methodologies, and empirical work on this subject that speak to concerns
about social justice and equity in language education, from languages and
contexts around the world – offering an overview of key concepts and
theoretical issues and field-advancing suggestions for research projects.
This book offers a fresh perspective on language testing that will be
an invaluable resource for advanced students and researchers of applied
linguistics, sociolinguistics, language policy, education, and related fields – as
well as language program administrators.
PART I
The ethical contextualization of validity 1
PART II
Agency and empowerment prompted by test adequacy 51
PART III
Sociointeractional perspectives on assessment 133
Index235
CONTRIBUTORS
Meng Yeh is Teaching Professor at the Center for Languages and Intercul-
tural Communication at Rice University (USA). Her current studies focus
on using naturally occurring conversations in general language courses and
languages for specific purposes, as well as evaluating the effectiveness of its
pedagogy.
PART I
DOI: 10.4324/9781003384922-2
4 M. Rafael Salaberry and Albert Weideman
how the various chapters give us a diversity of views on the ways that test de-
signers may meet the design challenges to assess the ability to use additional
languages in more responsible ways.
practice and is specific to that practice.” The phrase “that practice” already
indicates the contextuality of such competence, a point that we shall return
to several times. Apart from the general language resources identified in
previous models of language competence – notably grammatical structure,
lexical resources, and expressive meaning – the construct of interactional
competence highlights the role played by contextually specific identity re-
sources (i.e., participation frameworks) and interactional resources (i.e., a
variety of modes of turn-taking, sequence and preference organization, and
repair, for different types of discourse). For the purpose of assessment, there-
fore, the definition of interactional competence brings up two problems: it
is determined by the local context (thus, not necessarily generalizable), and
it is based on the co-construction of meaning (thus, language ability can-
not be attributed to one individual alone). It follows that the earlier per-
spective on language, which focuses mainly on the general structural and
semantic means of expression, is much less concerned with context than an
interactional view that sees language as communication (Weideman, 2009),
and how lingual interaction is accomplished within a specific social relation-
ship. The earlier, less contextualized one focuses on skills-based assessment,
defining skills conventionally in essentially 19th-century terms: listening,
speaking, reading, and writing. The one centered on interaction turns its at-
tention to skills-neutral, functional language use (Weideman, 2021). A pre-
dominantly structural view of language will emphasize the command of
general means of expression, whereas a socially informed perspective will
promote the mastery of communication in various kinds of typically differ-
ent discourse. The restrictive perspective will attempt to isolate a ‘skill’ to be
tested, while the open, disclosed view of language will respond to the reality
that skills cannot be separated, and that lingual interaction is characteristi-
cally multimodal.
So, the challenge, given this change in perspective, is: Can we move be-
yond a decontextualized view of language knowledge and language profi-
ciency? Can it be replaced or augmented by a functional and interactional
perspective on language competence? When one considers the critique of,
for instance, international high-stakes English language proficiency tests, in-
cluding IELTS (e.g., Hamid et al., 2019; Hamid & Hoang, 2018; Noori &
Mirhosseini, 2021; Pearson, 2019), one begins to appreciate the size of the
challenge. Moreover, have large-scale institutional tests been able to uphold
traditional skills-based divisions in assessing language ability? Most appear
to favor some accommodation of both skills-based and functional perspec-
tives in how they define what they test.
To take a further example of the issues involved, consider the ACTFL
OPI (American Council on the Teaching of Foreign Languages Oral Profi-
ciency Interview), which has become an unavoidable point of reference for
6 M. Rafael Salaberry and Albert Weideman
the US context. Since the time of its publication (1982), it has become the
prevalent testing instrument to measure speaking interactional competence in
the US. Crucially, despite early calls for modifications (e.g., Bachman, 1990;
Chalhoub-Deville, 2003; Kramsch, 1986; Raffaldini, 1988; Salaberry, 2000;
Shohamy, 1990), the ACTFL proficiency model remains mostly unchanged
since the time of its original publication. The success of the ACTFL-OPI
test in the US, however, showcases the benefit of implementing some type
of institutionalized, systematic procedure to assess language competence.
Liskin-Gasparro (2003, p. 489) concludes in this respect that “the notion of
proficiency as construed by the ACTFL Guidelines . . . seems to have found
its legitimacy in the arenas of policy, program development, and classroom
instruction.” A similar institutionally sanctioned model is to be found in the
Common European Framework of Reference (CEFR) for languages (Coun-
cil of Europe, 2011). Though it is widely used as the theoretical basis for
language assessments, also outside of Europe, and though it has been aug-
mented and modified over time, it, too, has remained relatively untouched
by substantial criticism from language testers (e.g., Alderson, 2007; Byrnes,
2007; Figueras, 2012; Fulcher, 2004; Weir, 2005).
The success of these models, despite the obvious lacunae in our under-
standing of the process of language development or the nature of language
interaction, rests primarily on their pragmatic institutional nature. For in-
stance, van Lier (1989, p. 501) argues that it is “possible to sidestep the is-
sue of construct validity altogether and be satisfied with measuring whatever
oral language use happens to be elicited by the OPI, since it is in any case
the best instrument available.” A more philosophical perspective is offered
by Spolsky (1997, p. 246), who vindicates the effort launched by such insti-
tutionalized tests to move forward with a less than ideal assessment scheme
because “(m)any if not most of the important decisions we have to make
in life are made in a state of insufficient empirical evidence.” It should be
noted, however, that Spolsky added a qualification to his position: “(o)nce
we accept the need for a gatekeeping function, we are ethically bound to
seek the most complete information available.” We return to this point later.
Arguably, there have been some modifications of the ACTFL-OPI over
the last 30-plus years. It is clear, however, that these modifications have not
introduced the type of significant changes needed to address some of the
critiques advanced in the 1980s and 1990s, especially those related to the
three ‘turns’ mentioned earlier (i.e., interactional, embodied, multilingual).
Inadequate design responses to these critiques mean that challenges remain
(see Salaberry & Burch, 2021 for an overview of those challenges faced by
the testing of speaking in context). Perhaps the construct of interactional
competence may be difficult to operationalize in conventional assessment
tasks. Indeed, the assessment of a broader definition of proficiency, as an
Context, construct, and ethics 7
achieve that goal, she analyzed the effects of two speaking test tasks on
topic initiations and their corresponding scores. One of the two tasks was an
open-topic task where the peers are supposed to chat for five to ten minutes
about several topics of their choice. The other task was a discussion task in
which participants were provided with several topics on a card, which was
drawn from a set of such cards. The two tasks were used in classroom-based
paired speaking tests designed for first-year learners of German. In the first-
year German classes selected for the study, students are explicitly taught
interactional competence (e.g., initiate topics, to expand on topics, initiate
repair in case of non-understanding, etc.). The data were evaluated follow-
ing a conversation analysis methodology. The analysis of the data revealed
that the initiation of topics between test takers who engaged in the open-
topic task was balanced. That is, both test takers initiated topics at about the
same rate. In comparison, the test taker pairs who engaged in the discussion
task, where the topics to be discussed were provided, initiated topics in a
rather unbalanced fashion, in that one of the participants took on a domi-
nant role and initiated all topics listed on the topic card. The findings sug-
gest that the discussion task may be misleading in terms of test takers’ ability
to initiate topics. Even though a test taker does not introduce any topics
for that task, it does not mean that s/he is unable to do so. The discussion
task may impact our understanding of what a student can do and test tak-
ers’ scores. In our classroom-based test setting, such a misconception would
also affect our overall classroom teaching and testing. Consequently, Kley
concludes that one option to make this type of assessment more reliable and
valid is to modify the rubric for the discussion task or the task design.
In Chapter 8, Kley, Kunitz, and Yeh specifically assess the role of the
interlocutor in two speaking tests of Chinese as a foreign language (CFL)
conducted at a US university. As is well known from previous research in
language testing (Bachman, 1990; Bachman & Palmer, 1996; McNamara,
1996), test discourse and assigned scores may be influenced by various con-
textual factors (e.g., test task, interlocutor, etc.). This study focused on the
effect of the interlocutor’s native/non-native-speakerness on the test taker’s
production of repair practices. Accordingly, 28 second semester CFL learn-
ers participated in two classroom-based speaking tests: one with a fellow
CFL student (“peer interaction”) and one with a native-speaking student at
the same university (“NS interaction”). For both tests, the test taker pairs
engaged in an open-topic task. Overall, the test takers’ lower engagement in
other-initiated repair and other-directed word searches during peer interac-
tion represented a safe strategy to avoid potentially face-threatening situa-
tions (such as displaying non-understanding or asking for help that the peer
might not be able to give). On the other hand, the increased number of
other-directed word searches in NS interaction was related to the linguistic
20 M. Rafael Salaberry and Albert Weideman
epistemic asymmetry between the two interlocutors, with the test taker ori-
enting to the NS as more knowledgeable. At the same time, the test taker
seemed to orient to the importance of establishing intersubjectivity with
the NS, which would explain the higher number of repair initiations. The
authors of the chapter conclude that the NS interaction appears to provide
more affordances for initiating repairs. In general, although not all test tak-
ers and NSs react in the same way, the findings of the study indicate that
a linguistically asymmetric test setting may be most fruitful if the testing
objective is to elicit repair practices.
In the final section, a concluding chapter by Weideman provides a reflec-
tion, from the point of view of a test designer, of what we have accomplished
in fronting juridical and ethical issues in language testing, asking whether
we have yardsticks to gauge both current trends and potential future ad-
vances. He starts by noting that we need a criterion, first, for knowing what
constitutes an advance in language test design, and what would signal re-
gression. Weideman’s argument implies that we should not succumb to the
cynical view that there can be no development, and that everything is but a
power play between less and more influential agents – the argument of shift-
ing of the deckchairs on the Titanic, only to find that nothing has changed
or will change. He argues that an evaluation of where we have progressed
or failed in the last two decades will show that, despite throwbacks and un-
met challenges, there have been substantial gains. One illustration of such
a gain is the rising level of language assessment literacy. But there are many
more. What we need in order to identify gains or losses he calls a “theory of
disclosure.” Such a theory asks: how is the meaning of language test design
opened up? With the tenets of a theory that articulates what makes designs
(more) meaningful, we are readying ourselves to generate yardsticks that we
can use to evaluate future developments. We need to articulate such princi-
ples, stipulating the requirements of how the many remaining challenges for
responsible test design can be met.
This book is an attempt to do just that.
References
Abedi, J. (2008). Classification system for English language learners: Issues and rec-
ommendations. Educational Measurement: Issues and Practice, 27(3), 17–31.
Alderson, J. C. (2007). The CEFR and the need for more research. The Modern
Language Journal, 91(4), 659–663. www.jstor.org/stable/4626093
Alderson, J. C., Haapakangas, E. L., Huhta, A., Nieminen, L., & Ullakonoja, R.
(2014). The diagnosis of reading in a second or foreign language. Routledge.
Arias, M. B., & Faltis, C. (Eds.) (2012). Implementing educational language policy
in Arizona: Legal, historical, and current practices in SEI (Vol. 86). Multilingual
Matters.
Another random document with
no related content on Scribd:
Näinä aikoina sai "Kullervo" tehdä palveluksen, jota kapteeni Malm
aina ylpeydellä muisteli. Yhtiön pääisännän tyttären häät vietettiin
eräänä kesänä. Kapteeni Malm kiirehti ulkomaamatkaltaan ja
joutuikin parhaiksi häihin. Hän tarjosi laivansa hääjoukon
käytettäväksi. Tarjous otettiin riemulla vastaan, ja päivä oli
"Kullervon" ja sen päällikön kunniapäivä.
Laivan pääisäntä, joka ensi vuosina, noina levinä aikoina, aina oli
sitä vastaan ottamassa ja piti pienet kemut jokaisen ulkomaamatkan
jälkeen, tuli tylyksi laivaa kohtaan ja ani harvoin kävi sitä
katsomassa. Ja yhtiökokouksissa osakkaat murisivat, joku kiroilikin
laivaa, kun se niin vähän tuotti. Kapteeniin se koski kipeästi. Hän oli
alati tehtävänsä tunnollisesti täyttänyt, vaan kiitoslauseita ei nyt
muistanut kukaan. Nuo osakkaat ne eivät pitäneet mistään muusta
kuin voitosta, jako-osuuksista.
Vaan sepä se oli, kun ei ollut rahoja. Hänelläkin tosin oli pääomaa,
vaan se ei ollut pankeissa eikä obligatsiooneissa, se oli pienten
kasvavien kapteenin alkujen muodossa. Se kyllä oli "liikkuvaa"
pääomaa, vaan sangen hankala sitä olisi ollut käyttää liikkeessä.
Eivät ainakaan laivayhtiöt semmoiselle suurta arvoa antaneet.
Raukka.
— Soipihan se.
— Onko se Tiina?
— On.
Minä selitin.
— Vai niin!
— Kolmannella kymmenellä.
— Eikö se koske?
Pikku Aaro.
Viimein tuli pohjalta esiin rutikuiva kanto, joka oli täynnä koloja.
Hän väänsi sen ylös ja rikkoi sen, jolloin siitä pelmahti maahan paljo
pieniä valkeita kappaleita. Aaro vähän säpsähti. Mitähän ne olivat?
Hän kumartui lähemmin katsomaan ja näki niiden olevan aivan
pieniä munan muotoisia kappaleita: Hän ei tiennyt mitä ne olivat ja
jäi uteliaana katsomaan, sydän läpättäen kiihkeästi.
Hän käsitti nyt, että nuo valkeat munat olivat kai niiden
muonavarastoa, niin kuin hänen kotonaan oli aitoissa jyviä, ja että
tässä tehtiin nyt majanmuuttoa. Hänen mieleensä muistui, kuinka,
kun naapurissa oli tulipalo, sieltä asukkaat yöllä pakenivat heille,
sylissään kantaen vaatteita, ruokatavaroita ynnä muuta, mitä olivat
ehtineet pelastaa kauheasta palosta. Se oli hirveä yö, hän muisti
vielä sen hyvästi! Tämä muurahaisten majan muutto alkoi hänestä
nyt tuntua samanlaiselta ja hän tunsi tehneensä pahan teon. Nuo
pienet raukat! Kuinka hän sääli niitä! Jos hän olisi tiennyt edeltä päin,
ei hän toki olisi tehnyt niille tätä. Mitenkä ne nyt mahtuvat asumaan
yhdessä pesässä entisten asukasten kanssa?
Aaro tuli uteliaaksi ja pani sen uudestaan veteen. Hän tahtoi antaa
sen oikein kastua. Vaan sitä lähti nyt ilman henki viemään ja se meni
avutonna, pää veden sisässä, takaruumis yläällä. Aaro haki
karahkan, jolla hän onki sen lähelle. Se tarttui erääseen karahkan
oksaan. Aaro antoi sen olla siinä ja asetti karahkan veteen, niin että
tuon oksan nenä oli vaan vähän yläällä vedestä ja karahkan tyven
hän laski kivelle. Sitte hän rupesi kalliolla mahalleen, hajasäärin ja
kädet rinnan alla, ja katseli muurahaisen hommia. Ei hän aikonut
suinkaan sitä tappaa, vaan hän tahtoi nähdä sen neuvokkaisuutta.