See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/281272769

Mathematical Communication in State Standards

Before the Common Core

Article  in  Educational Policy · July 2015

DOI: 10.1177/0895904815595723.

CITATIONS READS

3 2,580

2 authors:

Karl W. Kosko Yang Gao

Kent State University Dalian Maritime University
107 PUBLICATIONS   809 CITATIONS    42 PUBLICATIONS   86 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Validating Assessment of Multiplicative Reasoning View project

Extended Reality Initiative (XRi) @ Kent State View project

All content following this page was uploaded by Yang Gao on 15 August 2017.

The user has requested enhancement of the downloaded file.

595723
research-article2015
EPXXXX10.1177/0895904815595723Kosko and GaoEducational Policy

Article
Educational Policy
2017, Vol. 31(3) 275­–302
Mathematical © The Author(s) 2015
Reprints and permissions:
Communication in State sagepub.com/journalsPermissions.nav
DOI: 10.1177/0895904815595723
Standards Before the journals.sagepub.com/home/epx

Common Core

Karl Wesley Kosko1 and Yang Gao1

Abstract
Mathematical communication has been an important feature of standards
documents since National Council of Teachers of Mathematics’ (NCTM)
(1989) Curriculum and Evaluation Standards. Such an emphasis has influenced
content standards of states from then to present. This study examined how
effective the prevalence of various forms of mathematical communication in
2009 state standards documents was in regard to National Assessment of
Educational Progress (NAEP) 2009 achievement scores for Grade 4. Analysis
suggests mixed results with potential implications as states move toward
fully implementing the Common Core State Standards in Mathematics.
Specifically, although including language requiring mathematical descriptions
from students had a positive effect on Grade 4 NAEP 2009 achievement
scores, including language requiring rationales and justifications was not
found to have a statistically significant effect.

Keywords
mathematics education, standards, achievement, common core

1Kent State University, OH, USA

Corresponding Author:
Karl Wesley Kosko, Kent State University, 401 White Hall, P.O. Box 5190, Kent, OH 44242,
USA.
Email: kkosko1@kent.edu
276 Educational Policy 31(3)

Introduction
Many education reformers continue to advocate engaging students in math-
ematical communication through discussion and writing. First gaining trac-
tion as a reform-oriented position with the publication of Curriculum and
Evaluation Standards for School Mathematics (National Council of
Teachers of Mathematics [NCTM], 1989), mathematical communication
has been observed to engage students in developing deeper understandings
of mathematics (e.g., Hufferd-Ackles, Fuson, & Sherin, 2004; Staples,
2007; Yackel & Cobb, 1996). Furthermore, student engagement in mathe-
matical communication has been found to statistically predict students’
mathematics achievement (Hiebert & Wearne, 1993; Koichu, Berman, &
Moore, 2007; Kosko, 2012; Mercer & Sams, 2006). While evidence exists
to support teacher practices that engage students in mathematical commu-
nication, no such examination has been done from a policy perspective.
Specifically, the importance of mathematical communication was first
argued successfully from a standards document (Lampert & Cobb, 2000;
McLeod, 2003; NCTM, 1989). Therefore, it is important to evaluate the
effectiveness of various standards documents in advocating student engage-
ment in mathematical communication. As individual states transition away
from their own standards to the new Common Core State Standards in
Mathematics (CCSSM), the importance of such an evaluation is more per-
tinent than ever. It is important to evaluate the differences in the “old”
standards and the “new” CCSSM ones to determine if we are moving for-
ward effectively. Furthermore, there is a need to examine what ways differ-
ent content standards may be improved by including expectations that
students describe and justify their mathematics. The purpose of the present
study is to provide a baseline of research for future efforts in the field to
review and recommend revisions for standards-based language. Specifically,
we chose to examine the general effectiveness of including such expecta-
tions as integrated in mathematics content standards. To accomplish these
goals, we examined the effectiveness of states’ standards expectations for
mathematical communication in regard to students’ National Assessment of
Education Progress (NAEP) 2009 mathematics scores. Comparing states by
way of students’ NAEP 2009 mathematics achievement provides a common
metric between states, as well as a metric that is similar in form (i.e., stan-
dardized test) to how most states tend to assess the effectiveness of teach-
ers’ instruction in accordance with standards. In a similar manner, this study
examines the effectiveness of the various 2009 standards documents (as
opposed to teachers’ instruction) regarding conveyance of expectations for
students to communicate mathematically.
Kosko and Gao 277

A Brief History of Mathematical Communication in

the Content Standards
Although the modern standards movement can be dated as far back as the
launch of Sputnik in 1957, the first explicit national standards recommenda-
tions for mathematics came about in NCTM’s (1989) Curriculum and
Evaluation Standards for School Mathematics (Lappan & Wanko, 2003). The
new standards document was effectively the first such publication of its kind
and led to various states’ and textbook publishers’ incorporation and adapta-
tion of the standards (Lappan & Wanko, 2003; Porter, 1994). Within this ini-
tial standards document was an expectation for students to communicate
about their mathematics. As McLeod (2003) notes,

The importance of mathematics as communication was noted in early drafts,

but mostly in the context of the growing need for mathematical literacy in the
postindustrial age. At some point, communication got listed in the 5-8 group as
one of the “big ideas” that deserved attention, perhaps because the issue of
communication had been a central theme in some College Board publications
that were available. In later drafts, the Communication Standard became one of
the general standards listed for all grade levels. The emphasis on communication
was a new feature in NCTM recommendations, a new idea that later appeared
regularly in standards documents for other school subjects. (p. 777)

The communication standard was the second standard listed in each grade
band (NCTM, 1989). A useful example of expectations comes from the
Grades K-4 band in which NCTM (1989) advocated communication as a
means for children to make connections among multiple means of represent-
ing mathematical ideas (pictorial, graphical, symbolic, etc.). Discussion and
writing were encouraged as ways for students to develop better understand-
ings of mathematics. Furthermore, the Curriculum and Evaluation Standards
for School Mathematics encouraged teachers at all grade levels to engage
students in communication through the posing of questions, facilitating dis-
cussions among students, and engaging students in writing for mathematics
(NCTM, 1989).
The legacy of NCTM’s 1989 document can be seen in Principles and
Standards for School Mathematics (NCTM, 2000), a revision of their 1989
standards document. The Principles and Standards refined the ideas posed at
various grade bands in 1989 and provided a more comprehensive set of
expectations for what they referred to as the mathematical process standard
of communication. As a process standard, communication was viewed as dis-
tinct, but integrated with the mathematics content standards as a “[way] of
acquiring and using content knowledge” (NCTM, 2000, p. 29). Specifically,
278 Educational Policy 31(3)

NCTM (2000) suggested that from prekindergarten through Grade 12, stu-
dents should organize and consolidate their mathematical thinking through
communication; communicate their mathematical thinking coherently and
clearly to peers, teachers, and others; analyze and evaluate the mathematical
thinking and strategies of others; and use the language of mathematics to
express mathematical ideas precisely (NCTM, 2000). Among the specific
actions that support these usages of communication are students describing,
explaining, justifying, conjecturing, writing, questioning, arguing, listening,
and speaking about their mathematics. More recently, the Common Core State
Standards for Mathematics (CCSSM) have included expectations for various
aspects of mathematical communication within their grade-specific content
standards and their more general Standards for Mathematical Practice
(Common Core State Standards Initiative [CCSSI], 2010). These expectations
include the mathematical practice to construct viable arguments and critique
the reasoning of others. Like the NCTM communication process standard that
preceded it, this mathematical practice includes expectations for students to
provide descriptions and rationales in various formats, as well as interpreting
them via reading and listening to others’ mathematical arguments.
This brief history of mathematical communication in mathematics stan-
dards documents shows that mathematical communication has consistently
been considered an important expectation since the first serious standards
document was written. Yet, the general effect of including such expectations
in standards documents on students’ mathematics achievement has not been
examined. We argue that such an examination is necessary to determine how
effective language used to convey such expectations of mathematical
communication are in their most current form.

Interpretations of Mathematical Communication

Much of the literature suggests teachers’ self-reported familiarity and under-
standing with NCTM-aligned standards relates to what teachers do in the
classroom and to their students’ mathematics achievement (Hamilton &
Berends, 2006; Loeber, 2008; Young, 2007). However, other research reports
on discrepancies between what various state standards intend and what teach-
ers interpret they intend. H. C. Hill (2001) observed how teachers in a par-
ticular school district interpreted new mathematics standards in their states as
they aligned them to the adopted textbook series in the district (Saxon Math).
She notes that state standards were interpreted in reference to the curriculum,
rather than the curriculum being interpreted in terms of the standards.
Interpretations by the teachers were observed to represent “a wide gulf [from]
what the state wants and what teachers determined to do within this district”
Kosko and Gao 279

(H. C. Hill, 2001, p. 310). A separate study by H. C. Hill (2005) found differ-
ences between teachers’ and researchers’ assessment of teaching practices in
the classroom in regard to “mathematical communication and representa-
tion.” Examining secondary mathematics teachers interpretation of the
Common Core Standard for Mathematical Practice, construct viable argu-
ments and critique the reasoning of others, Kosko, Rougee, and Herbst
(2014) also found many teachers had interpretations of such practices that did
not match expectations of researchers.
Observations as those presented in the prior paragraph may help explain
findings from Kosko and Miyazaki (2012) who found that there was signifi-
cant variance in the effect of discussion frequency on mathematics achieve-
ment between teachers and schools. The variance was so large that in many
schools, engaging in discussion had a negative effect and in others, it was
generally positive. Yet, the cause of such variation may go beyond the teacher.
In examining various state standards for the usage of primary verbs in content
standards, Larnell and Smith (2011) noted that verbs such as “explain” and
“justify” (as well as others that would be classified here as related to com-
munication) were used relatively infrequently. The verb “explain,” which
was used more frequently than other communication-related verbs, was used
in varying ways. “Though explain is commonly considered a relatively high-
level verb in Bloom’s Taxonomy, this analysis suggests that states may be
using it to identify low-level cognitive processes” (pp. 111-112) as well as
high-level cognitive processes. Thus, Larnell and Smith’s (2011) findings
suggest that misinterpreting of standards, and mathematical communication
in standards, may generate at the state level, and not merely at the teacher
level. It also seems entirely plausible that teachers’ misinterpretations (or cor-
rect interpretations) may be influenced by the state policy documents they are
beholden.
Although it is not a purpose of this study to investigate teachers’ interpre-
tations of standards documents, they are discussed here because such factors
may influence the effectiveness of states’ standards. As useful as it would be
to incorporate teachers’ interpretations of standards in the present study, the
nature of data (as will be discussed later) does not allow for such a compari-
son at the scale of the present study.

Research Question
Research suggests that standards documents may affect how teachers incor-
porate mathematical communication in their classrooms (e.g., Loeber, 2008;
Young, 2007). In turn, students who are taught by teachers incorporating
mathematical communication appropriately tend to have higher mathematics
280 Educational Policy 31(3)

achievement (Hiebert & Wearne, 1993; Koichu et al., 2007; Kosko, 2012;
Mercer & Sams, 2006). Yet, some evidence exists that suggests teachers may
not interpret standards as intended, when it comes to mathematical commu-
nication (H. C. Hill, 2001, 2005; Kosko et al., 2014), possibly because some
states’ standards may not include consistent language in providing expecta-
tions in content standards for mathematical communication (Larnell & Smith,
2011). In essence, these various studies suggest engaging students in mathe-
matical discussions and writing in particular ways can have a statistically
significant and meaningful effect on their mathematics achievement, but it is
unclear whether or not state standards as policy documents are effective in
ensuring students engage in such actions.
The purpose of this study is to provide an initial examination of whether
and to what degree states’ incorporation of mathematical communication in
their mathematics standards documents affects students’ mathematics achieve-
ment. Our focus on mathematical communication is interpreted from NCTM’s
(2000) descriptions, in accordance with McLeod’s (2003) description of the
historical incorporation of mathematical communication in standards docu-
ments. While the term effectiveness can be interpreted in a number of ways, the
present study uses the term in regard to students’ mathematics achievement.
The reason for such an interpretation is simple: Those who drive policy (e.g.,
politicians and the public) typically interpret effectiveness of education reforms
in terms of test scores. Whether or not such usage is appropriate is a worthwhile
topic, but not one which we discuss here. Rather, we use this indicator of effec-
tiveness because of its present currency in policy decision making. Therefore,
the present study uses that metric to determine how “effective” policy docu-
ments in the form of state content standards are. To assess this study’s objec-
tive, a secondary analysis of NAEP 2009 data was conducted. NAEP 2009
provides an assessment common among all 50 states and at the last measured
point before adoption of CCSSM beginning in 2010. As will be described in the
forthcoming sections, state mathematics content standards effective in the
2008-2009 academic year were collected and examined for elements of math-
ematical communication. These data were then merged with NAEP 2009
Grades 4 restricted-use data to provide an indicator of the effectiveness of vari-
ous states’ standards emphasizing mathematical communication on students’
mathematics achievement.1 Merging of the data and the analysis that followed
allowed for the examination of the following research question:

Research Question 1: To what degree does the prevalence of states’ con-

tent standards that emphasize aspects of mathematical communication (in
terms of descriptions and explanations, as well as justification and ratio-
nale) affect students’ mathematical achievement in those states?
Kosko and Gao 281

Nature of Data
Two sources of data were used in the present study. The first were mathemat-
ics content standards documents from all 50 U.S. states. These data are pub-
licly available on request from the various governments of each of those
states. The second source of data was restricted-use data for Grade 4 mathe-
matics in the NAEP 2009 assessment (National Center for Education Statistics
[NCES], 2009b). The NAEP mathematics assessment uses a common set of
items to examine the status of mathematics learning across the United States.
Additional data related to the students, teachers, schools, and districts are
collected as well to provide for a general indication of the status of education
in the nation. The nature of NAEP’s common assessment across states, which
can have very different state-level content tests, was a necessary requirement
for the present study’s goals. Although the nature of NAEP 2009 provided for
a useful and necessary source of data for this study, use of such data came
with certain limitations. Specifically, neither teachers’ interpretations of stan-
dards nor their teaching practices were observed in NAEP 2009, and such
effects are not assessed in the present study. This limits the interpretability
and available range of focus of the present study to the effectiveness of
the standards documents alone. So, inferences regarding teaching practices
cannot be made.

Sample
The sample used for NAEP 2009 incorporated a multistage cluster sampling
design to obtain a sample where “estimates of population and subpopulation
characteristics could be obtained with reasonably high precision” (NCES,
2009a, p. 68). Therefore, certain subpopulations were oversampled (e.g.,
Native Americans) to insure that appropriate estimates for those subpopula-
tions could be obtained. To adjust for this oversampling in certain clusters,
NCES (2009a) recommends use of certain sample weights, depending on the
analysis performed. In accordance with these recommendations, the sample
weight identified in the data set with the designation ORIGWT was used in
the present study. Using this weight, the sample in this study can be inter-
preted as representative of students in those enrolled in Grade 4 in a public
school in one of the 50 U.S. states.2
While NAEP 2009 collected data from private schools, the goals of this
study were to examine how state mathematics content standards influenced
student achievement, and thus only public school data were used. In addition,
only students who resided in a U.S. state were included in the study. Other
reductions in sample were due to various independent variables included as
282 Educational Policy 31(3)

covariates in the study. The result of these reductions was an effective

sample3 of 23,340 students taught by 3,610 teachers in 50 U.S. states. Descriptive
characteristics for the sample is further provided in the measures section below,
as these characteristics were used as covariates in the present study.

Measures
Dependent variable.  Students’ mathematics achievement was used as the out-
come in this study. According to NCES (2009a), the NAEP 2009 mathemat-
ics test assessed five content strands: number properties and operations;
measurement; geometry; data analysis, statistics, and probability; and alge-
bra. The assessment used multiple-choice, short constructed-response and
extended constructed-response items in such a manner as to cover all content
strands. NAEP 2009 tests were assessed and scored using item response the-
ory (IRT). “IRT models the probability of answering a question in a certain
way as a mathematical function of proficiency or skill” (NCES, 2009a,
p. 26). NAEP created a composite score based on IRT estimations with a
range of 0 to 500. For the present analysis, the mean of variables MRPCM1
to MRPCM5, which represent composite score estimates as previously
described, were used to create the outcome measure (MathScore). MathScore
represents the mathematics composite score for students across all content
strands (M = 240.05, SD = 26.96, range = 120.60-332.77).

Independent variables.  The primary purpose of this study is to examine the

effect of mathematics content standards that require students to communi-
cate, with specific focus on how prevalence of such standards affects stu-
dents’ mathematics achievement scores. As noted in the review of literature,
the Principles and Standards for School Mathematics (NCTM, 2000)
describes a variety of ways for students to communicate mathematically.
Using NCTM’s guidelines, two codes were used to study states’ 2009 content
standards: description and rationale.
We incorporated systemic functional linguistics (SFL) in our coding pro-
cess to identify language indicative of expectations for communication
(Halliday & Matthiessen, 2004). Specifically, we examined the system of tran-
sitivity, which serves to organize the experiential function of language, to
identify standards conveying an expectation for mathematical description or
rationale. Through the system of transitivity, experience is conveyed primarily
through grammatical processes (e.g., I added 42; the answer is 20), with par-
ticipants and other grammatical elements providing appropriate context.
Although transitive processes often convey the central meaning of a gram-
matical clause, or unit of grammatical meaning, these processes can, in certain
Kosko and Gao 283

contexts, become embedded in a clause such that the meaning is not initially
at the fore. This process, referred to as nominalization, causes the transitive
process to serve as a participant in the clause. In other words, such processes
are acted on by other processes, thereby providing multiple layers of meaning
in a grammatical text. While others have looked at the difference between
nominalized and nonnominalized processes in content standards (i.e., Kosko
& Herbst, 2011), we looked only at the presence of processes eliciting expec-
tations for communication (i.e., description and rationale). This presence was
counted whether the process acted as a main transitive process in the standard
or if it was embedded in the clause as a participant. Throughout the descrip-
tions of our coding, we provide examples with italicized processes and nomi-
nalized processes to help illustrate their presence in standards language.
Description refers to standards that require students to describe mathemat-
ical procedures, strategies, steps, or concepts. According to the Principles
and Standards (NCTM, 2000), communication helps students to “gain
insights into their thinking when they present their methods for solving prob-
lems” (p. 60), and students should “learn new mathematical concepts”
through giving “verbal accounts and explanations” (p. 61). Maryland (2004)
requires students to

identify, describe, extend, analyze, and create a non-numeric growing or

repeating pattern. (Grade 4, Patterns and Functions, 4.1.2)

While this particular standard asks students to do several things regarding

patterns, asking students to describe these patterns encourages them to articu-
late procedures or concepts, which further confirms communication as “a
way of sharing ideas and clarifying understanding” (NCTM, 2000, p. 60).
Hawaii (2007) asks Grade 4 students to

explain the need to use standard units for measuring. (Grade 4, Measurement,
4.4.1)

This standard corresponds to the Principles and Standards recommenda-

tion that students should “explain their ideas in mathematic class” (p. 62).
Both example standards provided here utilize transitive process for describ-
ing some form of mathematical procedure or concept, and thus were coded as
description standards.
Rationale designated standards that require students to articulate justifi-
cations, conjecture or create hypotheses, or engage in mathematical argu-
ments. Take some Grade 4 standards as examples, Hawaii (2007) asks
students to
284 Educational Policy 31(3)

propose and justify conclusions/predictions based on data. (MA4.13.1)

Similarly, Missouri (2008) requires students to

estimate and justify products of whole numbers. (3. D)

The transitive process justify solicits the action to engage students in mathe-
matical communication and arguments, which has the potential to help them
to better understand mathematical concepts or procedures. Therefore, exam-
ples such as the ones provided in this paragraph were coded as rationale
standards.
Grade 4 mathematics content standards were collected for analysis. To
align the standards with NAEP mathematics achievement scores, standards
documents that were in effect in the academic year of 2008-2009 were col-
lected for analysis (as this coincided with the year NAEP 2009 data were
collected). These documents were collected from a variety of sources
including the websites from the Department of Education in each U.S.
state, and directly contacting each of these offices for verification we had
identified the standards document effective in the 2008-2009 academic
year. After standards documents were collected, a portion of the data
(10%) was examined by both authors, as is typical in such analyses, to
refine the coding rubric (see Hruschka et al., 2004). The remaining 90% of
data were coded by both authors individually. Afterward, both authors
compared and reconciled results. Weighted Kappa was calculated to assess
interrater reliability or prereconciled data, as it allows for ratings closer in
magnitude to be given more weight in terms of agreement than ratings
further away in magnitude (Landis & Koch, 1977). Our weighted Kappa
was .62 for description and .60 for rationale. Landis and Koch (1977) sug-
gested that kappa coefficients .41 to .60 have moderate reliability and .61
to .80 have substantial reliability. This indicates that the coding of 2009
standards was sufficiently reliable for the present study. Descriptive statis-
tics suggest that description standards (M = .16; SD = .10; range = .00-.47)
may be more prevalent than rationale standards (M = .05; SD = .05; range
= .00-.24). However, there was substantial variance in the frequency of
standards that required either form of communication. While most states
had relatively low frequencies of standards requiring descriptions or ratio-
nales from students, there was a great deal of variation in the degree to
which different states incorporated such expectations of students, with
some states having no such expectations and others requiring descriptions
for nearly half of their math content standards and rationales for nearly a
quarter.
Kosko and Gao 285

Table 1.  Descriptive Statistics for Student Covariates at Level 1.

dFemale dLunch dBlack dHispanic dAsian dNatAmer dOther

M = 0.49 M = 0.49 M = 0.16 M = 0.22 M = 0.05 M = 0.01 M = 0.02
SD = 0.50 SD = 0.50 SD = 0.37 SD = 0.41 SD = 0.22 SD = 0.11 SD = 0.12

Covariates.  While the focus of the present study is on how language in 2009
state mathematics standards affected students’ mathematics achievement in
Grade 4, other factors have historically affected mathematics achievement.
Namely, gender, race/ethnicity, and socioeconomic status (SES) have each
been consistently associated with differences in mathematics achievement.
Tate (1997) reviewed literature and found that between 1973 and 1992, the
achievement gap attributable to race closed in Grades 4, 8, and 12. This was
also found to be true of SES. However, Lubienski (2002) examined NAEP
data from 1990, 1996, and 2000 and found that the achievement gap due to
race increased between 1990 and 2000 for Grade 8 Black students, but
remained the same at Grade 4. SES did not adequately explain the achieve-
ment gaps. A later study of NAEP data found that, while small, the achieve-
ment gap due to gender was consistently present for data between 1990 and
2003 (McGraw, Lubienski, & Strutchens, 2006). This finding is consistent
with prior research (Tate, 1997). Given the consistent effects associated with
gender, race, and SES, these factors were included at the student level of
analysis in the present study.
Gender was included by taking the variable SEX from the NAEP data set
and dummy coding it (dFemale) so that the effect would associate with being
female (0 = male; 1 = female). For race/ethnicity, the variable SRACE, which
indicated several designations of race for students, was separated into 5
dummy coded variables so that the designation White/Caucasian was the ref-
erence group: dBlack (0 = not Black; 1 = Black), dHispanic (0 = not Hispanic;
1 = Hispanic), dAsian (0 = not Asian; 1 = Asian), dNatAmer (0 = not Native
American; 1 = Native American), and dOther (0 = not Other non-White
ethnicity; 1 = Other non-White ethnicity). SES was assessed using the variable
SLUNCH1, which designated a student’s eligibility for the National School
Lunch Program (i.e., free and/or reduced lunch). As the provided variable
included multiple designations, it was dummy coded into dLunch (0 = not
eligible for free/reduced lunch; 1 = eligible for free/reduced lunch). Descriptive
statistics for these student-level covariates are displayed in Table 1.
Additional covariates were included in the analysis to account for
teacher factors influential to mathematics achievement. According to find-
ings by Clotfelter, Ladd, and Vigdor (2007), the more years of experience
286 Educational Policy 31(3)

a teacher has, the higher his or her students’ mathematics achievement

scores are. This finding has been confirmed through use of experience as a
covariate in other literature (e.g., Kosko & Miyazaki, 2012). Therefore, we
included a dichotomous variable Experience (M = 0.83, SD = 0.37), which
represents whether a teacher has more than 3 years of experience4
(1 = more than 3 years; 0 = 3 years or less). The next teacher-level covari-
ate included SES_Mean, which indicates the percentage of students
enrolled in a teacher’s class who are eligible for free or reduced lunch
(M = 0.47, SD = 0.34) and has been found to influence individual student
achievement (de Fraine, van Damme, van Landeghem, Opdenakker, &
Onghena, 2003). We also included MathScore_Mean (M = 236.74, SD =
20.23), which is the average NAEP mathematics achievement score for a
teacher’s class. Mean class achievement is another teacher-level factor found
to influence mathematics achievement at the individual student level (de
Jong, Westerhof, & Kruiter, 2004; Opdenakker, van Damme, de Fraine, van
Landeghem, & Onghena, 2002) and inclusion of this variable is meant to
account for this effect.

Analysis and Results

The present study incorporated hierarchical linear modeling (HLM) to exam-
ine the effect of language used in Grade 4 mathematics content standards in
various states on students’ mathematics achievement, as assessed by the 2009
NAEP assessment. HLM accounts for the nested structure of social science
data (e.g., students are nested in teachers’ classes, teachers are nested within
the states they teach) by using multilevel regression analysis (Raudenbush &
Bryk, 2002). Specifically, the group average of a coefficient’s effects at an
individual student-level regression is treated as outcome measures for regres-
sion equations at the teacher level. Likewise, effects in teacher-level regres-
sion equations are treated as outcomes at the state level. For the analysis
presented here, maximum likelihood estimation (MLE) was used to estimate
effects at each level. The reported effects are, in actuality, averages of effects
for each grouping factor, which are in turn weighted for the specific variance
associated with each group.
While there are various versions of HLM, the present study used three-
level HLM-3 to investigate the research questions. Specifically, it is assumed
that state standards documents are read directly, and thus interpreted directly,
by classroom teachers. These teachers in turn instruct their students in accor-
dance with their interpretations of the standards documents. So, students were
assessed at Level 1 in the three-level HLM and were assumed to be nested by
teacher at Level 2, who were in turn nested within the state at Level 3.
Kosko and Gao 287

As is customary in HLM analysis, the HLM-3 models were constructed by

first analyzing the data with no predictors, and then including predictors
sequentially at each level to properly assess changes in effects and the vari-
ance due to the nested nature of data (Raudenbush & Bryk, 2002). Therefore,
in the sections that follow, the construction and modification of each model
leading up to the final model of analysis is described.

Unconditional Model
The initial model examined in all HLM analyses is the unconditional model,
or the model without conditions (i.e., variables). Analysis of the uncondi-
tional model allows for the researchers to determine the variance in the out-
come measure and how much of this variance is due to the respective levels
in the analysis. It also allows for a generalized baseline interpretation of the
outcome measure across the sample, without adjusting for other factors. In
the Equation Set 1 below, the mathematics score (MathScore) of student
i taught by teacher j in state k is represented as the outcome at Level 1. The
variable π0 jk represents the mean achievement of students taught by teacher
j in state k, and eijk represents the random variance from this mean that is
associated with the individual student i. At Level 2, the mean score per
teacher, π0 jk, is the outcome variable with β00k representing the mean
achievement for state k and r0 jk is the random variance associated with the
individual teacher j. At Level 3, β00k , or the mean score per state, is the out-
come variable with γ 000 symbolizing the grand mean, or overall average, of
math achievement scores in all 50 states. The variable u00 k represents the
deviation of state k’s mean achievement from the grand mean.
Equation Set 1.
Level 1:
( MathScore )ijk = π0 jk + eijk .
Level 2:

π0 jk = β00 k + r0 jk .
Level 3:
β00 k = γ 000 + u00 k .

Results from the unconditional model suggest the typical student in the
United States, residing in 1 of 50 states and not accounting for any other fac-
tors, has a mathematics achievement score of 239.50, which is statistically
significant from zero at the .001 level.
288 Educational Policy 31(3)

Model 1—Student-Level Covariates

Following the unconditional model, an initial conditional model was con-
structed by incorporating student factors at Level 1. These changes are repre-
sented in Equation Set 2. With all Level 1 predictors included, π0 jk now
represents the mean achievement of White male students, not eligible for free
and reduced lunch, who are taught by teacher j in state k. The remaining vari-
ables each represent the effect of belonging to a certain classification on a
student’s mathematics achievement score. Thus, π1 jk represents the effect of
being a female student, π2 jk represents the effect of being eligible for free and
reduced lunch, and the variables π3 jk through π7 jk represent belonging to an
ethnicity other than White/Caucasian (i.e., Black, Hispanic, Asian, Native
American, or Other, respectively). eijk represents the random variance asso-
ciated with student i. At Level 2, all slopes for covariates were set constant so
as to provide for a more parsimonious model. Specifically, as the focus of the
study is on the effect of certain language uses in states’ standards (at Level 3),
the variance of Level 1 and Level 2 variables between grouping factors
(teachers or states) was not addressed.
Equation Set 2:

( MathScore )ijk = π0 jk + π1 jk ⋅ ( dFemale )ijk +

π2 jk ⋅ ( dLunch )ijk + π3 jk ⋅ ( dBlack )ijk
+ π4 jk ⋅ ( dHispanic )ijk + π5 jk ⋅

( dAsian )ijk + π6 jk ⋅ ( dNatAmer )ijk

+ π7 jk ⋅ ( dOther )ijk + eijk .

Effects at Level 1 are outcome variables at Level 2. Likewise, effects in

each Level 2 equation become the outcome variables at Level 3. At Level 2
and Level 3, random effects are associated only with the intercept, meaning
that only the variation of the intercept was measured during this portion of
analysis. This was done to keep a parsimonious, or simplified, model as addi-
tional variables were included at other levels.
Results are presented in Table 2 and indicate that dFemale has a statisti-
cally significant, negative effect on students’ mathematics achievement ( γ100
= −1.12, p < .01). Other covariates found to have statistically significant
effects were dLunch ( γ 200 = −12.69, p < .001), dBlack ( γ 300 = −16.86,
p < .001), dHispanic ( γ 400 = −12.14, p < .001), dNatAmer ( γ 600 = −13.75,
p < .001), and dOther ( γ 700 = −10.34, p < .01). The results suggest academic
advantages for students who are not eligible for free and reduced lunch, who
Kosko and Gao 289

Table 2.  Level 1 Coefficient Results for Grades 4 and 8.

Coefficient SE
Intercept, γ 000 251.33*** 0.77
dFemale, γ100 −1.12** 0.17
dLunch, γ 200 −12.69*** 0.21
dBlack, γ300 −16.86*** 0.29
dHispanic, γ 400 −12.14*** 0.32
dAsian, γ 500 1.64 0.49
dNatAmer, γ 600 −13.75*** 0.77
dOther, γ 700 −10.34** 0.74

*p < .05. **p < .01. ***p < .001.

are male, and who are either White or Asian (students reported as Asian were
found not to have statistically significant math scores from White students).
These results generally follow findings from previous research and merit
their inclusion in the present analysis.

Model 2—Teacher-Level Covariates

The next step in constructing the HLM-3 models was to include covariates at
Level 2, the teacher level. Equation Set 2 represents the model at Level 1 as
no changes were made at this level from Model 1. Equation Set 3 represents
additions to the HLM-3 model, which take place at Level 2.
Equation Set 3:

π0 jk = β00 k + β01k ( Experience ) jk β02 k

( SES _ Mean ) jk β03k ( MathScore _ Mean ) jk + r0 jk .

π1 jk = β10 k .
π2 jk = β20 k .
π3 jk = β30 k .
π4 jk = β40 k .
π5 jk = β50 k .
π6 jk = β60 k .
π7 jk = β70 k .
290 Educational Policy 31(3)

Table 3.  Results for Model 2.

Coefficient SE
Intercept, γ 000 246.46*** 0.93
SES_Mean, γ 010 14.33*** 0.69
MathScore_Mean, γ 020 0.96*** 0.01
dFemale, γ100 −1.04** 0.32
dLunch, γ 200 −10.57*** 0.66
dBlack, γ300 −9.46*** 0.58
dHispanic, γ 400 −6.44*** 0.52
dAsian, γ 500 1.16 1.04
dNatAmer, γ 600 −7.69*** 0.98
dOther, γ 700 −6.90** 2.53

*p < .05. **p < .01. ***p < .001.

As can be seen in Equation Set 3, Experience, SES_Mean, and MathScore_

Mean were included at Level 2 as predictors of the intercept, π0 jk, at Level 1,
or the mean math achievement score for teacher j in state k. SES_Mean and
MathScore_Mean were group mean centered, meaning that the effects for
these particular variables are adjusted for the state mean of teacher j. So, β01k
represents the effect of a teacher having more than 3 years of experience in
the classroom on the mean math achievement score of his or her students.
Also, β02k represents the effect of having a class with larger (or smaller)
percentages of students eligible for free and reduced lunch than the average
for state k. Similarly, β03k represents this effect in regard to higher average
NAEP mathematics achievement scores relative to the average achievement
score in state k.
Initial results from Model 2 indicated that Experience was not statistically
significant. We examined whether the removal of this variable would signifi-
cantly change the model deviance, an indicator of model fit. This examina-
tion suggests that removal of the variable did not decrease model fit,
χ2(df = 1) = .13, p > .05. Therefore, the variable was removed to defer to a
more parsimonious model. Results for Model 2 are presented in Table 3 and
indicate that coefficients at Level 1 were found to be statistically significant
at levels similar to those found in Model 1. At Level 2, the effect of SES_
Mean, now designated by β01k , was found to have a positive and statistically
significant effect on math achievement scores ( γ 010 = 14.33, p < .001), indi-
cating that teachers with higher percentages of students eligible for free and
reduced lunch had higher average mathematics scores in their class. This
finding is counter to previous research examining class mean SES. Although
Kosko and Gao 291

this particular finding is outside of the scope of the present study, it is worth
noting that prior research has indicated use of eligibility for free or reduced
lunch may not be a reliable indicator of poverty, primarily because many
students are eligible for such lunch programs for 1 or 2 years and then rise out
of such indicators of poverty (M. S. Hill & Jenkins, 2001). However, a less
reliable indicator would likely result in nonstatistically significant findings,
suggesting that our small finding here needs further investigation. We
acknowledge the possible limitation of eligibility for free or reduced lunch as
a sole measure for SES, but include it due to a lack of additional information
for SES of the student population in the sample. As expected, MathScore_
Mean, β02k , was found to be a statistically significant and positive predictor
of student achievement ( γ 020 = 0.96, p < .001).

Model 3—Final Model

The final model was constructed by including the main variables of interest
at Level 3. These additions are represented by Equation Set 4.
Equation Set 4:
Level 2:
π0 jk = β00 k + β01k ⋅ (Years ) jk + β02 k ⋅ ( PD ) jk + r0 jk .

π1 jk = β10 k .
π2 jk = β20 k .
π3 jk = β30 k .
π4 jk = β40 k .
π5 jk = β50 k .
π6 jk = β60 k .
π7 jk = β70 k .

Level 3:
β00 k = γ 000 + γ 001 ⋅ ( Description )k + γ 002 ⋅ ( Rationale )k + u00 k .

β01k = γ 010 .
β02 k = γ 020 .
β10 k = γ100 .
β20 k = γ 200 .
β30 k = γ 300 .
β40 k = γ 400 .
292 Educational Policy 31(3)

β50 k = γ 500 .
β60 k = γ 600 .
β70 k = γ 700 .

Level 2 was left unchanged from Model 2. At Level 3, the independent

variables Description and rationale were added as predictors for β00k , which
represents the mean math achievement score for White male students in state
k who are taught by teachers whose classes have average achievement and
percentages of students eligible for free or reduced lunch for their respective
state. Given the nature of the independent variables, β00k also represents the
aforementioned mean score for state k, which does not have any content stan-
dards that include language encouraging mathematical communication either
in the form of description or rationale. Moreover, γ 001 represents the effect
on a state’s mean math achievement adjusted for the percentage of mathemat-
ics content standards requiring students to describe or explain their mathe-
matics. The effect of the percentage of standards requiring students to provide
conjectures, rationales, or justifications is represented by γ 002 . Finally, u00 k
represents the random variance associated with state k from the grand mean
of all states’ math achievement scores.

Final model results.  Final model results for Grade 4 are presented in Table 4.
As can be seen from Table 4, Level 1 coefficients saw little change from pre-
vious models, and Level 2 covariates were also found to be similar to find-
ings from Model 2. For Level 3, description was found to have a positive and
statistically significant effect ( γ 001 = 22.41, p < .01). However, rationale was
found not to be statistically significant ( γ 002 = −2.12, p = .90). These coef-
ficients should be interpreted in reference to the percentage of such standards
a state has included. For example, in a state with 16% of its standards including
requirements for description (the national average), the average student would
typically see an increase in his or her scores by 3.59 points. Such changes are not
meaningful in magnitude. Rather, to improve the average student’s mathematics
achievement score by just half a standard deviation, a state would need approxi-
mately 60% of its content standards to include expectations for mathematical
description. As indicated previously, no state had such high frequencies of math-
ematics standards requiring description. Therefore, these results suggest that
although the inclusion of expectations for mathematical description in content
standards does have a statistically significant effect on students’ mathematics
achievement, for most states this effect is not meaningful in size. Furthermore,
inclusion of standards requiring rationales from students was not found to have
a statistically significant effect on their mathematics achievement. Essentially,
various states’ incorporation of mathematical communication in their Grade 4
Kosko and Gao 293

Table 4.  Final Model Results.

Grade 4

  Coefficient SE
Intercept, γ 000 243.21*** 1.81
Description, γ 001 22.41*** 7.81
Rationale, γ 002 −2.12 16.94
SES_Mean, γ 010 14.32*** 0.68
MathScore_Mean, γ 020 0.96*** 0.01
dFemale, γ100 −1.04** 0.32
dLunch, γ 200 −10.56*** 0.66
dBlack, γ300 −9.47*** 0.58
dHispanic, γ 400 −6.44*** 0.51
dAsian, γ 500 1.19 1.03
dNatAmer, γ 600 −7.68*** 0.98
dOther, γ 700 −6.95** 2.53

*p < .05. **p < .01. ***p < .001.

mathematics standards documents is generally not effective in improving stu-

dents’ mathematics achievement.

Discussion
Results of the present study should be interpreted with great care. Specifically,
these findings are indicative of the quality of state standards that were in
effect in 2009 for Grade 4, but of which most are no longer in effect.
Furthermore, the results are indicative of mathematics content standards spe-
cifically and not of mathematics teaching practices generally. It is of utmost
importance that such facts be considered prior to interpreting the findings
presented and discussed here.
The present study presents a complicated view of how various states’ stan-
dards conveyance of mathematical communication affected students’ mathe-
matics achievement. Results revealed that higher frequencies of description
standards in Grade 4 documents did have a statistically significant effect on
mathematics scores. However, the size of these effects were relatively low in
magnitude. Interestingly, higher frequencies of rationale standards were found
to not have a statistically significant effect associated with mathematics achieve-
ment. On one hand, these findings could be interpreted that efforts to include
expectations for mathematical communication are not worthwhile. On the other
hand, there is a large body of research demonstrating young children’s ability to
294 Educational Policy 31(3)

describe and argue about their mathematics (e.g., Hufferd-Ackles et al., 2004;
Yackel & Cobb, 1996). Therefore, expectations for Grade 4 children to
explain and justify their mathematics are reasonable. In addition, prior
research on the effect of NCTM’s (2000) Principles and Standards document
suggest this is not the case (Hamilton & Berends, 2006; Loeber, 2008; Young,
2007). Such documents can be effective, but the majority of U.S. state’s stan-
dards documents simply were not effective, or were not effective enough.
Rather than conclude that standards do not matter, it is more likely that the
incorporation of communication expectations in content standards by various
states has not been done in an effective manner. Such a conclusion would
align with studies such as that conducted by Larnell and Smith (2011), who
found that various states’ inclusion of mathematics communication-oriented
expectations was vague, inconsistent, and not always aligning with the higher
expectations described by NCTM (2000). Although the findings of the pres-
ent study did not report on the quality of such expectations in various state
content standards, findings from this study do suggest that such expectations,
regardless of quality, were included in relatively few of the content standards
by state. Therefore, findings from the present study provide additional evi-
dence that language used in state standards is ineffective at communicated
expectations for students (particularly in regard to providing mathematical
description and rationales).
If the reason for the lackluster effects found in this study rests in the man-
ner in which such expectations were included in content standards, then it is
logical that either low frequency, the vague nature of inclusion, or a combina-
tion of both would influence conditions such that these standards would not
have a sizable effect on student achievement. Although findings from the
present study are general in nature, these findings can inform potential
improvements regarding inclusion of communication expectations in content
standards. Practical implications from this study suggest that mathematics
content standards should include expectations for mathematical communica-
tion in higher frequencies and in language more explicit and supportive of
NCTM (2000) recommendations. While approximately half of states did
include reference to communication in a separate set of expectations, there
appears to be little effect on students’ mathematics achievement whether this
separate section is included in a document or not.5 Therefore, it may be that
for expectations for mathematical communication to be conveyed effectively,
they should be embedded in content standards. However, such expectations
must be conveyed in a meaningful manner. For example, Larnell and Smith’s
(2011) analysis suggests that standards including expectations for mathemat-
ical descriptions should do so in a way that elicits higher reasoning and
engagement in the content. One solution to this mismatch is for states to
Kosko and Gao 295

provide further specification to enhance standards’ ability to convey expecta-

tions for mathematical communication. For example, Hawaii (2007) asks
Grade 4 students to “propose and justify conclusions/predictions based on
data.” The standard lacks specification on the type of data that conclusions
and/or predictions would be associated, and this relates to the format of the
justification to be solicited by students. Certain types of data elicit more or
less sophisticated justifications. For example, a pie chart illustrating the num-
ber of boys and girls in a classroom would likely elicit less interesting justi-
fications than a line plot of different heights in the classroom. In addition to
improving the wording of standards, another possibility is to provide teachers
with meaningful resources that articulate, for each standard, how students
should engage in communicating mathematically. Confrey et al. (2012) pro-
vided one example of such a resource with the website www.turnonccmath.
net. The resource provides in-depth descriptions and learning trajectories for
standards and groups of standards. However, examination of the effective-
ness of such a resource on teachers’ interpretations of standards is still needed.
Examining and improving the clarity of standards’ expectations for math-
ematical communication provides a useful starting point for improving the
quality of mathematics content standards. Yet, research findings suggest that
part of the issue may be how teachers interpret, or misinterpret, such stan-
dards (e.g., H. C. Hill, 2001, 2005). H. C. Hill’s (2005) observations of teach-
ers’ interpretations of standards documents found that teachers interpreted
the standards from the perspective of the curriculum at hand (i.e., textbook),
rather than the intent of particular standards. Newton’s (2009) observations
of Grade 6 teachers showed that teachers consistently engaged students in
relatively fewer descriptive or “how” questioning than the written curriculum
(textbook) they were assigned. If we pair the findings by H. C. Hill (2005)
and Newton (2009), an interesting problem comes to the fore. Potentially,
teachers may interpret content standards expectations for mathematical com-
munication via the expectations of their textbook, but their interpretations of
such textbooks’ expectations may also not match the intent behind the cur-
riculum. While the present study did not investigate teachers’ interpretations
of standards, such factors may have influenced the findings presented here.
Yet, it is unclear how separable the issues of unclear standards (or curriculum
in general) and teachers’ misinterpretations of such standards actually are.
The two issues are likely intertwined, and further study is needed to deter-
mine how much effort should be devoted to clarifying standards, and how
much should be devoted to professional development regarding standards.
The findings of the present study suggest that including standards that
require descriptions and rationales of mathematics does not affect students’
mathematics achievement. However, it is unclear how well NAEP measures
296 Educational Policy 31(3)

Figure 1.  NAEP extended constructed-response item that includes the questions
“why or why not.”
Source. Obtained from NAEP questions tool (NCES, 2014).
Note. NAEP = National Assessment of Education Progress; NCES = National Center for
Education Statistics.

these particular skills. Although NAEP does include some prompts that solicit
explanations, accessible items do not provide a clear assessment of rationales
on Grade 4 items. Figure 1 provides an example item from the NAEP
Question Tool that seemingly solicits a rationale. However, the scoring rubric
evaluates the correct or incorrect answer along with whether the description
of procedures is appropriate. No mention of rationale or justification is pro-
vided. A similar phenomenon was observed by Kosko and Wilkins (2011) in
which open-response items from various assessments of quantitative literacy
were examined. Specifically, while several items included requests for
descriptions and rationales, the rubrics often did not assess such data from
test takers. If a similar phenomenon is present in NAEP items, it may explain
why the effect sizes obtained in this study were not larger. However, various
studies have documented the positive association between these mathemati-
cal communication behaviors and achievement scores, where achievement
was not measured with items soliciting description or justification (e.g.,
Hiebert & Wearne, 1993; Kosko, 2012; Kosko & Miyazaki, 2012). Therefore,
it is more likely that the findings of the present study are due mainly to the
manner in which various states incorporated communication expectations in
their content standards.

The Common Core

The findings of the present study may provide some information on how the
CCSSM will impact student achievement in the coming years. Table 5 shows
the average makeup of each type of communication standard for 2009-
effective standards alongside the CCSSM.
While higher prevalence of descriptive standards was found to have a
positive effect on mathematics achievement at Grade 4, there is a much lower
Kosko and Gao 297

Table 5.  Comparison of Grade 4 Communication Standards in 2009 and

Common Core Standards.

2009 state standards Common Core State Standards

Descriptive 0.16 0.06
Rationale 0.05 0.12

prevalence of such standards in the new CCSSM. Yet, those standards

included in CCSSM soliciting descriptions appear to provide some clarity in
the kind of explanations expected of students. For example, the standard
below states that students should explain calculations of multiplying various
digit numbers with different representations. Although anecdotal, comparing
this expectation with the examples provided earlier in examples from
Maryland and Hawaii suggests a more purposeful expectation from CCSSM
than some states.

Multiply a whole number of up to four digits by a one-digit whole number, and

multiply two-digit numbers, using strategies based on place value and the
properties of operations. Illustrate and explain the calculation by using
equations, rectangular arrays, and/or area models. (CCSSI, 2010, p. 29)

While description standards are less prevalent in Grade 4 CCSSM stan-

dards, rationale standards saw subtle increases over the national mean in
2009. As with the example provided for description standards, the rationale
standards in CCSSM provide more specific expectations of what should be
included. In the example below, rationales for comparing numbers with deci-
mals is conveyed through the expectation of reasoning via comparison of the
size of numbers and justifying symbolic representations through other repre-
sentations, such as a visual model.

Compare two decimals to hundredths by reasoning about their size. Recognize

that comparisons are valid only when the two decimals refer to the same whole.
Record the results of comparisons with symbols >, =, or <, and justify the
conclusions, e.g., by using a visual model. (CCSSM, 2010, p. 30)

Although it is tempting to criticize the Common Core’s decrease in number

of content standards requiring descriptions, it seems that there is actually a shift
in focus from requiring descriptions of procedures toward providing rationales,
justifications, and conjectures. Furthermore, the expectations of what is
included with a description or justification appear, at least anecdotally, to be
more clearly communicated. Still, both percentages may be lower than needed
298 Educational Policy 31(3)

for such language to be effective in improving students’ depth of understand-

ing of mathematics. Furthermore, while the Common Core standards strongly
advocate the Standards for Mathematical Practice, there are relatively few
viable resources or direction for teachers on how students should construct
viable arguments and critique the reasoning of others for particular grade
levels or topics. Therefore, future revisions of CCSSM should consider such
choices of language thoroughly and give particular attention to the preva-
lence and use of description and rationale standards in various grade levels,
as well as integration of expectations for the mathematical practices with
specific topics at particular grades or grade bands.

Conclusion and Implications

Although mathematical communication has been considered an important
element of mathematical standards since the first of such documents (i.e.,
NCTM, 1989), the present analysis suggests relatively infrequent incorpora-
tion of description and rationale in content standards, and little meaningful
effect of those standards that have been included by various states. In other
words, although higher prevalence of content standards requiring description
shows a statistically significant, positive effect on students’ mathematics
achievement in Grade 4, such effects were not meaningfully large in size,
indicating that standards requiring students to describe their mathematics
may not be written in an effective manner. A similar argument can also be
made for the effectiveness of language used to write mathematics standards
requiring mathematical rationales from students. These findings suggest that
a primary implication from this study is that authors of policy documents
need to be mindful of how communication-endorsing language is used in
content standards.
Another implication is the need for support materials to help clarify the
intent of such standards. Anecdotal evidence from the examination of 2009
state content standards suggests relatively little supporting documents for
teachers in regard to how to interpret content standards for their mathematical
meaning. Likewise, relatively little support documentation is provided for the
new CCSSM standards by that organization. Given the potential for misinter-
pretation of standards (H. C. Hill, 2001, 2005; Kosko et al., 2014), the need
for such material is critical. Resources such as www.turnonccmath.net do
provide some assistance to teachers (Confrey et al., 2012), but it is unclear
how well known such resources are to teachers, or how well used.
The findings presented here go beyond the case of standards conveying
mathematical communication. Results suggest that although there are instruc-
tion practices that are highly beneficial to student learning (e.g., providing ratio-
nales, justifications, and conjectures in mathematics), the potential inclusion of
Kosko and Gao 299

such practices in policy documents may have little effect on mathematics

achievement. Such results beg the question of what other such language
choices in standards have little meaningful effect? With the rapid adoption
and implementation of CCSSM, such issues need further and intensive study
to ensure that steps meant to be progressive in certain aspects are not regres-
sive in others.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,
authorship, and/or publication of this article.

Funding
The author(s) received no financial support for the research, authorship, and/or publi-
cation of this article.

Notes
1. Although National Assessment of Education Progress (NAEP) 2009 also
assessed mathematics in Grades 8 and 12, we limit our focus on Grade 4 for
simplicity of discussion.
2. Although NAEP data exist for the District of Columbia and certain U.S. territo-
ries, we restricted the scope of analysis to U.S. states. States as political entities,
though different from one another, have more similarities in their organiza-
tion than various territories. Thus, a focus on states and excluding District of
Columbia and territories was meant to simplify the focus of this study.
3. Per guidelines for reporting analysis results from restricted-use NAEP 2009 data,
all sample sizes, and indicators of sample size were rounded to the nearest 10.
4. The NAEP 2009 variable used for Experience was YRSEXP.
5. We conducted a similar analysis of expectations for communication (descrip-
tion and rationale) separate from content standards (e.g., inclusion of
National Council of Teachers of Mathematics [NCTM] Process Standards and
variations of such). Results suggest no statistically significant effects of such
expectations.

References
Common Core State Standards Initiative. (2010). Common Core State Standards for
Mathematics. Retrieved from http://corestandards.org/assets/CCSSI_Math%20
Standards.pdf
Clotfelter, C. T., Ladd, H. F., & Vigdor, J. L. (2007). Teacher credentials and stu-
dent achievement: Longitudinal analysis with student fixed effects. Economics of
Education Review, 26(6), 673-682.
Confrey, J., Nguyen, K. H., Lee, K., Panorkou, N., Corley, A. K., & Maloney, A. P.
(2012). Turn-on common core math: Learning trajectories for the Common Core
State Standards for Mathematics. Available from www.turnonccmath.net
300 Educational Policy 31(3)

de Fraine, B., van Damme, J., van Landeghem, V., Opdenakker, M., & Onghena,
P. (2003). The effect of schools and classes on language achievement. British
Educational Research Journal, 29, 841-859.
de Jong, R., Westerhof, K. J., & Kruiter, J. H. (2004). Empirical evidence of a com-
prehensive model of school effectiveness: A multilevel study in mathematics in
the 1st year of junior general education in the Netherlands. School Effectiveness
and School Improvement, 15, 3-31.
Halliday, M. A. K., & Matthiessen, C. M. I. M. (2004). An introduction to functional
grammar (3rd ed.). London, England: Hodder Education.
Hamilton, L. S., & Berends, M. (2006). Instructional practices related to standards
and assessments. Pittsburgh, PA: RAND Education.
Hiebert, J., & Wearne, D. (1993). Instructional tasks, classroom discourse, and stu-
dents’ learning in second-grade arithmetic. American Educational Research
Journal, 30, 393-425.
Hill, H. C. (2001). Policy is not enough: Language and the interpretation of state stan-
dards. American Educational Research Journal, 38, 289-318.
Hill, H. C. (2005). Content across communities: Validating measures of elementary
mathematics instruction. Educational Policy, 19, 447-475.
Hill, M. S., & Jenkins, S. P. (2001). Poverty among British children: Chronic or
transitory? In B. Bradbury, S. P. Jenkins, & J. Micklewright (Eds.), The dynam-
ics of child poverty in industrialised countries (pp. 174-195). Cambridge, UK:
Cambridge University Press.
Hruschka, D. J., Schwartz, D., St. John, D. C., Picone-Decaro, E., Jenkins, R. A., &
Carey, J. W. (2004). Reliability in coding open-ended data: Lessons learned from
HIV behavioral research. Field Methods, 16, 307-331.
Hufferd-Ackles, K., Fuson, K. C., & Sherin, M. G. (2004). Describing levels and com-
ponents of a math-talk learning community. Journal for Research in Mathematics
Education, 35, 81-116.
Koichu, B., Berman, A., & Moore, M. (2007). The effect of promoting heuristic
literacy on the mathematical aptitude of middle-school students. International
Journal of Mathematical Education in Science and Technology, 38, 1-17.
Kosko, K. W. (2012). Student enrollment in classes with frequent mathematical dis-
cussion and its longitudinal effect on mathematics achievement. The Mathematics
Enthusiast, 9(1&2), 111-148.
Kosko, K. W., & Herbst, P. G. (2011). Where’s the proof? Proof in U.S. high school
geometry content standards. In L. R. Wiest & T. Lamberg (Eds.), Proceedings
of the 33rd annual meeting of the North American Chapter of the International
Group for the Psychology of Mathematics Education (pp. 329-337). Reno:
University of Nevada, Reno.
Kosko, K. W., & Miyazaki, Y. (2012). The effect of student discussion frequency on
fifth-grade students’ mathematics achievement in U.S. schools. The Journal of
Experimental Education, 80, 173-195.
Kosko, K. W., Rougee, A., & Herbst, P. (2014). What actions do teachers envi-
sion when asked to facilitate mathematical argumentation in the classroom?
Mathematics Education Research Journal, 26, 459-476.
Kosko and Gao 301

Kosko, K. W., & Wilkins, J. L. M. (2011). Communicating quantitative literacy: An

examination of open-ended assessment items in TIMSS, NALS, IALS, and PISA.
Numeracy, 4(2). Retrieved from http://services.bepress.com/numeracy/vol4/
iss2/art3
Lampert, M., & Cobb, P. (2000). Communication and language. In J. Kilpatrick,
W. G. Martin, & D. Schifter (Eds.), A research companion to Principles and
Standards for School Mathematics (pp. 237-249). Reston, VA: National Council
of Teachers of Mathematics.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for
categorical data. Biometrics, 33, 159-174.
Lappan, G., & Wanko, J. J. (2003). The changing roles and priorities of the federal
government in mathematics education in the United States. In G. M. A. Stanic
& J. Kilpatrick (Eds.), A history of school mathematics (Vol. 2, pp. 897-930).
Reston, VA: National Council of Teachers of Mathematics.
Larnell, G. V., & Smith, J. P. (2011). Verbs and cognitive demand in K-8 geom-
etry and measurement grade level expectations. In J. P. Smith (Ed.), A compan-
ion analysis of K-8 state mathematics standards (pp. 95-118). Charlotte, NC:
Information Age.
Loeber, G. H. (2008). An inquiry into teacher implementation of NCTM standards
in kindergarten through second grade instructional setting (Master’s thesis).
Walden University, Naples, FL.
Lubienski, S. T. (2002). A closer look at Black-White mathematics gaps: Intersections
of race and SES in NAEP achievement and instructional practices data. The
Journal of Negro Education, 71, 269-287.
McGraw, R., Lubienski, S. T., & Strutchens, M. E. (2006). A closer look at gender in
NAEP mathematics achievement and affect data: Intersections with achievement,
race/ethnicity, and socioeconomic status. Journal for Research in Mathematics
Education, 37, 129-150.
McLeod, D. B. (2003). From consensus to controversy: The story of the NCTM stan-
dards. In G. M. A. Stanic & J. Kilpatrick (Eds.), A history of school mathematics
(Vol. 1, pp. 753-818). Reston, VA: National Council of Teachers of Mathematics.
Mercer, N., & Sams, C. (2006). Teaching children how to use language to solve maths
problems. Language and Education, 20, 507-528.
National Center for Education Statistics. (2009a). NAEP 2009 mathematics, reading,
and science assessments restricted-use data files: Data companion. Washington,
DC: U.S. Department of Education.
National Center for Education Statistics. (2009b). NAEP 2009 national and state
mathematics assessments (grades 4 and 8) restricted-use data files. Washington,
DC: U.S. Department of Education.
National Center for Education Statistics. (2014). NAEP question tool. Retrieved from
http://nces.ed.gov/nationsreportcard/ mathematics
National Council of Teachers of Mathematics. (1989). Curriculum and evaluation
standards for school mathematics. Reston, VA: Author.
National Council of Teachers of Mathematics. (2000). Principles and standards for
school mathematics. Reston, VA: Author.
302 Educational Policy 31(3)

Newton, J. (2009). The relationship between the written and enacted curricula: The
mathematical routine of questioning. In S. L. Swars, D. W. Stinson, & S. Lemons-
Smith (Eds.), Proceedings of the 31st annual meeting of the North American
Chapter of the International Group for the Psychology of Mathematics Education
(pp. 337-345). Atlanta: Georgia State University.
Opdenakker, M., van Damme, J., de Fraine, B., van Landeghem, G., & Onghena, P.
(2002). The effect of schools and classes on mathematics achievement. School
Effectiveness and School Improvement, 13, 399-427.
Porter, A. C. (1994). National standards and school improvement in the 1990s: Issues
and promise. American Journal of Education, 102, 421-449.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications
and data analysis methods (2nd ed.). Thousand Oaks, CA: SAGE.
Staples, M. (2007). Supporting whole-class collaborative inquiry in a secondary
mathematics classroom. Cognition and Instruction, 25, 161-217.
Tate, W. F. (1997). Race-ethnicity, SES, gender, and language proficiency trends
in mathematics achievement: An update. Journal for Research in Mathematics
Education, 28, 652-679.
Yackel, E., & Cobb, P. (1996). Sociomathematical norms, argumentation, and
autonomy in mathematics. Journal for Research in Mathematics Education, 27,
458-477.
Young, L. (2007). Determining teachers’ behaviors concerning the NCTM standards
in low and high performing rural high schools in Kansas (Doctoral dissertation).
Kansas State University, Manhattan.

Author Biographies
Karl Wesley Kosko is an Assistant Professor at Kent State University in mathematics
education. His program of research centers on mathematical communication with a
focus on student engagement in and teacher facilitation of whole class discussion, and
students’ mathematical writing.
Yang Gao is currently a Doctoral Candidate and Graduate Assistant at Kent State
University. His research interests are English as a second/foreign language (ESL/
EFL) teacher beliefs and practice, and ESL/EFL reading instruction.

View publication stats