1

Oral Language Assessment LITeracy

Oral Language Assessment LITeracy: Choosing the Right Assessment in the Time of COVID

Rihana S. Mason1, Lisa Fitton2, Ryan L. James3 and Yaacov Petscher4

1
Urban Child Study Center, College of Education and Human Development, Georgia State
University
2
Communication Sciences and Disorders, Arnold School of Public Health, University of South
Carolina
3
Ryan L. James, Rollins Center for Language and Literacy, Atlanta Speech School
4
Florida Center for Reading Research, Florida State University

Author contact:

Rihana Mason – rihana@gsu.edu, @drrihana

Lisa Fitton – fittonl@mailbox.sc.edu, @fit_ton
Ryan L. James – leeryan3@gmail.com
Yaacov Petscher – ypetscher@fsu.edu, @yaacovp

Author contact: Funding for the authors’ work was supported, in part, by the Chan Zuckerberg
Initiative
2
Oral Language Assessment LITeracy

Abstract

The novel coronavirus (COVID-19) has altered learning environments. These changes

have shed light on several factors that make assessing oral language skill, a foundational

component of reading development, even more challenging under current conditions. Oral

language is the way that we communicate our thoughts and ideas. Three factors related to ways

in which we can raise our oral language assessment LITeracy by considering children’s: 1)

Language variation, 2) Individual differences, and 3) the Technological considerations of the

school environment during the upcoming year are discussed. Taking account of these three

factors is a first step in ensuring equitable assessment. In order to make strides in preventing

inequitable assessment practices we provide general recommendations to help overcome the

challenges faced in the current climate and future learning environments. Choosing the right oral

language assessment means: 1) being inclusive of the language, dialect, and individual variation

that is reflected in your classroom. 2) embracing all of the oral language 3) anticipating how the

data from an assessment can be linked to instructional practice and 4) making time to do your

due diligence.
3
Oral Language Assessment LITeracy

Oral Language Assessment LITeracy: Choosing the Right Assessment in the Time of COVID

Learning environments have been altered due to the novel coronavirus (COVID-19).

These changes highlight several factors that make assessing oral language skill, a foundational

component of reading development, even more challenging under current conditions. Despite the

decades of research to create appropriate assessments, we need to consider how the

characteristics of those we are designing assessments for, influence our expectations of the

results. We need to raise our oral language assessment LITeracy by considering children’s: 1)

Language variation, 2) Individual differences, and 3) the Technological considerations of the

school environment. In language assessment, we need to aim to examine the similarity between

the input a child receives and the output we observe (if the child’s output matches the signal they

receive as input, this suggests no difficulty/disorder). Taking account of these three factors is a

first step in ensuring equitable assessment.

Oral language is the way that we communicate our thoughts and ideas. The successful

use of oral language in the classroom includes the combination of several skill domains

including: the production and comprehension of words (vocabulary), knowledge of sounds

(phonological awareness/phonology), the use of words to form complete sentences (syntax), the

formation of words from smaller units of speech (morphological awareness) and the social use of

language (pragmatics). The structure of language is complex for children (LARRC, 2015), thus

measuring oral language skills is a critical component of a child’s educational development.

Against the backdrop of the pandemic and the importance of equitable assessment, we focus on

factors that influence the selection of oral language assessments from early learning (birth to

five) to early grades (K-5). We provide recommendations for choosing assessments that give

you the abiLITy to measure oral language skills centered around the backgrounds of students in
4
Oral Language Assessment LITeracy
your classroom (e.g. language variation, dialect variation, and individual variation) and your

current instructional environment (e.g. in person or virtual).

Language Variation

The landscape of U.S. classrooms is changing to reflect the expansion of language and

dialect variation of speakers. The U.S. Census Bureau reported that 67.3 million U.S. residents in

2018 spoke a language other than English at home. Multilingual students reflect similar linguistic

variation across languages. As multilingual students are educated in English-focused schools, the

influence of their home language(s) is likely to shift over time (Gottschalk, 2019). Some of these

students may test-out of the official “English language learner” classification, but variable

language experience and exposure will still limit the validity of many standardized English-based

language assessments (Bedore & Pena, 2008). Consequently, more pragmatic approaches to

distinguishing which students might be considered multilingual may be appropriate, including

any student who uses more than one language in everyday life (Uljarevic et al., 2016).

To gain an accurate picture of multilingual students’ language ability, assessment should

include all languages to which the student is regularly exposed (Duran et al., 2019). Including

multiple indicators such as parent and teacher report of child language ability coupled with direct

assessment of child skills, can yield good diagnostic classification accuracy, reliability, and

validity (Bonifacci et al., 2020). Dynamic assessment, consisting of a test-teach-retest approach,

is useful for determining students’ readiness to learn new skills or concepts (Kapantzoglou et al.,

2012). This strategy can be further adapted through graduated prompting, incorporating

increasing scaffolding support to encourage the student toward the correct response (Petersen et

al., 2020). Modeling-based assessments such as sentence repetition tasks reduce the influence of

background knowledge on performance, likely allowing for a more precise examination of

5
Oral Language Assessment LITeracy
component skills of language (Stokes et al., 2006). Any evaluation of oral language ability must

account for the linguistic variability present in a child’s language exposure and use (Duran et al.,

2019). Language assessment must similarly reflect diversity in what approaches are used to

evaluate and which skills are valued as indicators of linguistic ability.

Dialect Variation

Rule-governed variations within a language are referred to as dialects and similarly are

often not accounted for in the design of many assessments. All children and adults speak a

dialect, but in every society there is a standard dialect - often defined as the academic language

and the language used in the mainstream media, for example Mainstream American English in

the United States. Nonstandard dialects, generally spoken by racial/ethnic and cultural minority

groups, are oral dialects (i.e., do not have a written form) and vary systematically from the

standard in some or all language domains.

Dialect mismatches influence how messages are expressed and received, which has direct

implications for oral language assessment (Oetting & McDonald, 2001). This becomes a salient

factor in later development when students who speak a nonstandard dialect are tasked with

mastering the orthographic rules for reading (Terry, 2012). As an example, the distinct

morphological and syntactic characteristics of African American English, to a certain degree,

overlap with clinical indicators of language impairment. In assessment, these differences can

result in misrepresentation of the student’s language skills. When African American English-

speaking children are learning to read, these differences can influence the development of word

reading skills and reading comprehension. The issues surrounding dialect mismatch have been

documented in other languages where nonstandard, oral dialects differ systematically from the

standard, academic language (Maamouri, 2009).

6
Oral Language Assessment LITeracy
Individual Variation

More than half of the children in the US are black, indigenous, and persons of color

(BIPOC). BIPOC children are primarily enrolled in schools with lower socioeconomic indicators

(Morgan & Ameriker, 2018). Caregivers from low SES backgrounds are more likely to

communicate with their children using shorter utterances and less lexical diversity than

caregivers from higher SES backgrounds (Rowe, 2008). These differences in early language

experiences result in differences in the amounts and types of vocabulary, morphology, and

syntax children are exposed to. In oral language assessment, children’s response patterns or oral

language output may reflect, for instance, knowledge of different meanings or uses of words, less

specified or less variety in word use, and a preference for using one or more word classes

(Mason & Bass, 2020).

Other individual variation arises from differences in the communities where children live

geographically. Rural settings are associated with more extreme levels of poverty and

inequitable funding appropriation. This translates into inequitable access to resources in both

schools and homes. Despite these challenges children in rural settings are surrounded by kinship

relationships that promote learning the social expectations of language or pragmatics. Parental

language includes the predominance of -wh questions which models the expectation that children

in rural homes should both critically think and engage in a dialogue with adults (Reynolds et al.,

2019). Maternal selection of informal home literacy activities and children's enjoyment of

reading are associated with preschool children’s vocabulary in rural settings vocabulary

knowledge (Bojczyk et al., 2015). Thus, parental style of communication and the curriculum in

rural homes may not align with the style of certain tests given at school (Reynolds, et al., 2019).

The selection of assessments that closely match both the cultural and contextual features of the
7
Oral Language Assessment LITeracy
child’s home environment is encouraged particularly when examining children’s use of syntax

and vocabulary (Poolman et al 2017).

How do we get LIT?

Two promising technological solutions have emerged that support unbiased scoring -

computer adaptive assessments (CAA) and automated speech verification (ASV). CAAs are

where individuals are delivered items that are targeted to the test-taker’s ability level ensuring

that they are never administered items that are too difficult or too easy - instead they are just

right. A benefit of CAAs is that they are often completed faster than assessments where everyone

in a class is given the same items, they are shorter, and they produce more reliable scores

(Mitchell et al., 2015). Because oral language includes diverse constructs, CAAs afford an

opportunity to obtain reliable and useful information about a child’s oral language skills from

both diverse oral language constructs and across diverse languages for dual language learners.

CAAs can help to eliminate examiner implicit bias and facilitate student focus and engagement

in the task.

Automated speech verification (ASV) systems are engines that may be embedded within

assessments to automatically score students’ oral language responses. Although ASV has only

recently begun to be embedded in educational assessments, there is a promising frontier for this

technology. ASV is a form of speech recognition that evaluates what a child actually said against

what the system expected them to say. In this way, ASV may introduce greater objectivity in

oral language assessment for children because they hold the potential to challenge unintentional

biases that creep into classroom assessment practices (Petscher & Patton-Terry, 2020). Dialect

and linguistic variation can be accounted for in the models that train the ASV systems, but it is

critical that such systems are scientifically evaluated before their implementation.
8
Oral Language Assessment LITeracy
Choosing the Right Assessment

So how does this manifest in a classroom? Teachers can implement formative assessment

(FA) in their classroom with all students in a way that allows for opportunities to progress

monitor and distinguish between variations among students. FAs can be adapted to coincide with

readily available classroom materials, incorporate culturally relevant stimuli, and provide a

hierarchy of interactions that build upon children’s varying home language and language

learning environments. Examples of these assessments include asking wh-question

comprehension or story-retells after small group read alouds, language sampling analysis, and

dynamic assessment of morphological awareness.

In order to make strides in preventing inequitable assessment practices we provide

general recommendations to help overcome current challenges. Choosing the right oral language

assessment means attending to the following four factors:

1) Be inclusive of the language, dialect, and individual variation that is reflected in your

classroom. Select an assessment or battery of assessments that allows for multiple prompts

and/or response types in order to capture environmental oral language features.

2) Embrace all of the oral language domains. Carefully choose an assessment or battery

of assessments that provides a snapshot of multiple oral language domains.

3) Anticipate how the data from an assessment can be linked to instructional practice.

Review the data, identify patterns, and look for teachable moments that fit children’s home

experiences.

4) Make time to do your due diligence. Be intentional by checking the technical manual

to see if their norms reflect unbiased and reliable estimates of oral language skill for the age and

backgrounds of children in your classroom.

9
Oral Language Assessment LITeracy
References

Bedore, L. M., & Pena, E. D. (2008). Assessment of bilingual children for identification of

language impairment: Current findings and implications for practice. International

Journal of Bilingual Education and Bilingualism, 11(1), 1–29.

Bojczyk, K. E., Rogers-Haverback, H., Pae, H., Davis, A. E., & Mason, R. S. (2015). Cultural

capital theory: a study of children enrolled in rural and urban Head Start programmes.

Early Child Development and Care, 185(9), 1390–1408.

Bonifacci, P., Atti, E., Casamenti, M., Piani, B., Porrelli, M., & Mari, R. (2020). Which

measures better discriminate language minority bilingual children with and without

developmental language disorder? A study testing the combined protocol of first and

second language assessment. Journal of Speech, Language, and Hearing Research, 63,

1898–1915.

Durán, L. K., Wackerle-Hollman, A. K., Kohlmeier, T. L., Brunner, S. K., Palma, J., & Callard,

C. H. (2019). Individual Growth and Development Indicators-Español: Innovation in the

development of Spanish oral language general outcome measures. Early Childhood

Research Quarterly, 48, 155–172.

Gottschalk, B. (2019). Holding high, not hurried expectations for ELLs. Education Update,

61(12). http://www.ascd.org/publications/newsletters/education-update/dec19/vol61/

num12/Holding-High,-Not-Hurried,-Expectations-for-ELLs.aspx

Kapantzoglou, M., Adelaida Restrepo, M., & Thompson, M. S. (2012). Dynamic assessment of

word learning skills: Identifying language impairment in bilingual children. Language,

Speech, and Hearing Services in Schools, 43(1), 81–96.

10
Oral Language Assessment LITeracy
Language and Reading Research Consortium (LARRC). (2015). The dimensionality of Spanish

in young Spanish-English dual-language learners. Journal of Speech, Language, and

Hearing Research, 58, 754–766.

Maamouri, M. (2009). Language education and human development: Arabic diglossia and its

impact on the quality of education in the Arab region. Washington, DC: World Bank,

Mediterranean Development Forum.

Mason, R. S., & Bass, L. A. (2020). Just ask me again: An analysis of receptive vocabulary

performance of children from low-income environments. Early Education and

Development, 31(6), 910–926.

Mitchell, A. M., Truckenmiller, A., & Petscher, Y. (2015). Computer-adaptive assessments:

Fundamentals and considerations. Communique, 43(8), 1-22.

Morgan, I., & Amerikaner, A. (2018). Funding gaps 2018. The Education Trust.

http://repositorio.unan.edu.ni/2986/1/5624.pdf

Oetting, J. B., & McDonald, J. L. (2001). Nonmainstream dialect use and specific language

impairment. Journal of Speech, Language, and Hearing Research, 44(1), 207–223.

Petersen, D., Tonn, P., Spencer, T., & Foster, M. (2020). The classification accuracy of a

dynamic assessment of inferential word learning for bilingual English/Spanish-speaking

school-age children. Language, Speech, & Hearing Services in Schools, 51(1), 144–164.

Petscher, Y., & Patton Terry, N. (2020). Speech recognition in education: The powers and perils.

SmartBrief [Editorial]. Retrieved from

https://www.smartbrief.com/original/2020/08/speech-recognition-education-powers-and-

perils
11
Oral Language Assessment LITeracy
Poolman, B. G., Leseman, P. P. M., Doornenbal, J. M., & Minnaert, A. E. M. G. (2017).

Development of the language proficiency of five- to seven-year-olds in rural areas. Early

Child Development and Care, 187(3–4), 756–777.

Reynolds, E., Vernon-Feagans, L., Bratsch-HInes, M. & Baker, C.E. (2019). Mother’s and

father’s language input to 6 and 36 month olds in rural two parent families: Relation to

children’s kindergarten achievement. Early Childhood Research Quarterly, 47, 385-395.

Rowe, M. L. (2008). Child-directed speech: Relation to socioeconomic status, knowledge of

child development and child vocabulary skill. Journal of Child Language, 35(1), 185–205.

Stokes, S. F., Wong, A. M. Y., Fletcher, P., & Leonard, L. B. (2006). Nonword repetition and

sentence repetition as clinical markers of specific language impairment: The case of

Cantonese. Journal of Speech, Language, and Hearing Research, 49(2), 219–236.

Uljarević, M., Katsos, N., Hudry, K., & Gibson, J. L. (2016). Practitioner review:

Multilingualism and neurodevelopmental disorders – an overview of recent research and

discussion of clinical implications. Journal of Child Psychology and Psychiatry, 57(11),

1205–1217.