Identifying an Indicator of Growth in Early Writing Proficiency

for Elementary School Students

Erica Lembke and Stanley L. Deno
University of Minnesota
Kim Hall
Robbinsdale Area Public Schools
New Hope, MN
In this study we compared alternative measures of early writing performance to identify an indicator of
writing proficiency that teachers could use to monitor student growth and evaluate instructional effec-
tiveness. The participants were 15 second-grade students in a summer school program for students at risk.
The performance sampled was word copying, sentence copying, word dictation, and sentence dictation.
Two- to three-minute samples of performance were obtained and several different scores were examined.
The criterion validity of the early writing measures was examined using both global judgments of compe-
tence and quantitative scoring of story writing as criteria. Writing from word and sentence dictation

entered into very high correlations with both criteria, showing promise as indicators of early writing pro-
ficiency. The results are discussed in terms of desirable features of assessment procedures including tech-
nical adequacy, administrative efficiency, and consumer acceptability.

Monitoring student progress in academic reliable measures for evaluating written

areas is criticalstudent success, especially
to expression interventions include the number
for students who are at risk for academic fail- of words written, number of words spelled cor-
ure (Deno, 1992). For these students at risk, rectly, or number of correct word sequences in
progress monitoring is particularly important a 3-minute writing sample (Deno, Marston, &

during the early elementary grades (K-2), as Mirkin, 1982; Deno, Mirkin, & Marston,
this is the time when students’ progress might 1980; Videen, Deno, & Marston, 1982).
best be accelerated through improved instruc- These indicators derived from students’
are
tional interventions. To monitor student writing in response to a story prompt. The

progress in academic subjects, teachers need
to use technically adequate measures and sub-
sequently respond to student needs based on
the data from such measures ( Deno, 1985).
One technically adequate method of
progress monitoring is curriculum-based mea-
surement (CBM), which provides teachers
with reliable, time-efficient, easy-to-adminis-
ter measures that are overall indicators of aca-
demic progress (Deno, 1985). CBM indicators
have been identified in reading, math,
spelling, and written expression. Current
CBM indicators that are viewed as valid and
empirical validity of these measures as overall
indicators of written expression has been rel-
atively well established for elementary stu-
dents in grades 2-6 (Marston, 1989) and, to a
lesser extent, for secondary students (Espin,
Scierka, Skare, & Halverson, 1999; Espin,
Shin, Deno, Skare, Robinson, & Benner,
2000). However, for students in the early
stages of writing (e.g., K-2 ), or struggling
learners, the CBM measure of written expres-
sion is too difficult to capture progress in
learning to encode oral language into writing.
In grades K-2, writing is typically not empha-

23

Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015

24
sized as much as reading or math ( Bradley-
Johnson Lesiak, 1989; Parker, Tindal,
& &
Hasbrouck, 1991; Roit & McKenzie, 1985).
However, developing early writing skills is a
critical component of early literacy, and mas-
tering these skills is necessary for later success
in written expression. For students in the
early elementary grades, teachers may focus
on legible handwriting, letter and word for-
mation, and basic sentence composition
when assessing students’ writing. Criterion-
referenced assessment of specific skills can
provide information about mastery. However,
mastery of these skills does not provide infor-
mation about students’ general proficiency in
writing over time. The existing CBM measure
of written expression correlates well with
other criterion variables in writing, but young
or struggling students may have difficulty with
the advanced requirements of this task with-
out aid from the teacher. As a result, measures
of writing proficiency need to be identified
that allow teachers to assess student progress
of initial acquisition of proficiency at encod-
ing oral language into writing.
Tindal and Parker (1991) identified four
criteria that can be used to guide the search
for useful assessment procedures: (a) consis-
tent administration and reliable scoring, (b)
ability to discriminate among students at dif
ferent skill levels, (c) criterion validity (to
other accepted assessment methods), and (d)
sensitivity to growth. These criteria consti-
tute a condensed version of the criteria used
in the initial development of CBM (Deno,
1985). Another important characteristic for
the task is &dquo;face validity.&dquo; In this context, face
validity refers to the degree to which the
assessment task is perceived as valuable by the
teacher and whether the task is authentic,
indicating that a student could complete it in
the classroom.
The purpose of this study was to identify
indicators of early writing performance with
respect to the criteria identified by Tindal and
Parker: consistent administration, reliable
scoring, effective discrimination among stu-
Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015
dents different levels, criterion validity,
at
sensitivity to growth, and face validity.
Identifying these indicators is viewed as
important for teachers assessing students who
are just learning how to write. The target stu-
dents include those in grades 1 and 2 and
those who have identified leaming needs or
are at risk for academic failure. Whereas the
existing CBM writing measure is relatively
unstructured, the proposed measures for early
writing assessment ask students to encode
controlled vocabulary. If measures can be
identified for early elementary students that
are predictive of later writing, such measures
could serve as indicators, providing data to
the teacher in evaluating and planning
instruction. Ideally, an early form of ongoing
monitoring will result in implementation of
specific interventions for individual students
designed to increase writing performance and
eventually lead to better overall writing. The
alternative measures to be examined included
writing from word or sentence dictation and
word or sentence copying, each administered
for 2-3 minutes.
Method
and
Partic:ip3nts Siting
This study took place in an elementary
school in a first-ring suburb of a large, mid-
western metropolitan area. Approximately
50% of the students were from minority back-
grounds, and 52% of students received free
and reduced cost lunch. Participants were 15
students in a second-grade summer school
program for students at risk, (8 females and 7
males). General education teachers had
referred these students for the summer school
program because of their low academic
achievement. Of the 15 subjects, 6 were
receiving special education services: 1 student
with leaming disabilities, 3 with mild/moder-
ate mental impairments, and 2 with speech
and language difficulties. All students and
their parents signed letters of consent for par-
ticipation.

Pirooecbare
Students completed four writing tasks at the
beginning and the end of the summer school
session: word copying, sentence copying, word
dictation, and sentence dictation. Materials
used in these tasks included controlled vocabu-
lary, with words taken from Basic Elementary
Reading Vocabulary (Harris & Jacobson,
1972). Each task was individually administered
for 2-3 minutes by the classroom teacher using
standardized administration directions. For
each task, the student completed a sample item
before beginning. Fidelity of implementation
checks were performed by a graduate student. A
description of each task follows:
1. Word copying. Students copied printed
words onto lines below each word. The
task was composed of 30 words, and was
administered for 2 minutes.
2. Sentence copying. Students copied printed
sentences onto lines below each sentence.
The task was composed of 12 sentences,
each with five to seven words, and was
administered for 3 minutes.
3. Word dictation. Students wrote dictated
words. Each word was dictated twice by the
teacher and then written by the student.
Students were instructed to do their best
when attempting to spell words, and teach-
ers moved on to the next word after stu-
dents had finished the current word, or
when the students had paused for 5 sec-
onds. The task was composed of 30 words,
and was administered for 3 minutes.
4. Sentence dictation. Students wrote dictated
sentences. Each sentence was dictated
twice by the teacher and then written by
the student. Students were instructed to do
their best when attempting to spell words,
and teachers moved on to the next sen-
tence after students had finished the cur-
rent sentence, or when the students had
paused for 10 seconds. The task was com-
posed of 12 sentences, each five to seven
words in length, and was administered for 3
minutes.
Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015
25
Pisdktor Variables
The writing tasks were scored using stan-
dardized methods (Deno et al., 1980).
Methods used to score the writing tasks were
used as the predictor variables in the study,
with scoring method varying by task. For
example, the word copying and word dicta-
tion tasks were scored according to number of
words written, number of words spelled cor-
rectly, and correct letter sequences. Sentence
copying and sentence dictation were scored
according to number of words written, num-
ber of words spelled correctly, correct word
sequences, and correct minus incorrect word
sequences.
Words written included all words that the
student wrote, a word being defined as a series
of letters with spaces on both sides. A word
was counted even if it was spelled incorrectly.
Words spelled correctly included all correctly
spelled English words, with no regard to con-
text. Correct letter sequences were defined as
two letters adjacent to each other that are
correctly placed in the word (Deno et al.,
1980). Correct word sequences were defined as
any two adjacent correctly spelled words that
were semantically and syntactically accept-
able within the context of the sentence
according to a native speaker of the English
language (Videen et al., 1982). Word
sequences were counted at the beginning and
the end of sentences, giving students credit or
penalizing them for capitals on the first letters
of sentences, and punctuation at the end of
sentences. In all scoring methods, students
were penalized for letter reversals within
words.
The typical interrater reliabilities reported
when scoring words written, words spelled
correctly, correct letter sequences, correct
word sequences, and correct minus incorrect
word sequences range from .87 for correct
word sequences to .99 for words written
(Marston & Deno, 1981; Parker et al., 1991;
Tindal & Parker, 1989; Watkinson & Lee,
1992).
26
Ctiterim Variables
Scores obtained on each writing task using
the various scoring methods were correlated
with seven criterion variables, which includ-
ed two qualitatively different sets. Criterion
variables were grouped into atomistic and the
holistic sets (Isaacson, 1988), with the atom-
istic variables utilizing discrete, countable fea-
tures of student writing, and the holistic vari-
ables utilizing teacher ranking or writing sam-
ple rating. The four atomistic variables that
enumerated student performance included
the average number of words written, the
average number of words spelled correctly, the
average number of correct word sequences,
and the average number of correct minus
incorrect word sequences on writing samples
that the students completed at the end of
summer school. The three holistic variables,
which provided a more global judgment of
writing performance, included the classroom
teacher’s ranking of students’ general writing
ability, the average of two holistic ratings of
students’ initial writing samples, and the aver-
age of two holistic ratings of students’ final
writing samples.
The atomistic variables included scoring
methods that were the same as those for the
writing tasks, with scoring completed on stu-
dents’ writing samples taken at the end of the
summer program. The holistic variables
included rankings or ratings of students’ writ-
ing. Writing samples completed at the end of the
summer program involved students writing for
5 minutes on a picture story starter where key
words were listed. To obtain scores on this
task, students completed three writing sam-
ples, which were scored according to number
of words written, number of words spelled cor-
rectly, correct word sequences, and correct
minus incorrect word sequences. This criteri-
on variable is similar to the current written
expression measure in CBM. To obtain the
teacher’s ranking, the summer school teacher
was asked to rank her students from most
skilled to least skilled on overall writing abil-
Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015
ity, based on all of her experience with the
students’ writing. Independent ratings from ini-
tial and final writing samples consisted of two
independent raters ranking three of the stu-
dents’ journal writings from the beginning of
the summer and three from the end of the
summer on a bipolar rating scale from 1 to 7,
with 1 being the worst and 7 being the best.
This scale did not include rubrics. Raters were
instructed to examine all sets of initial and
final journal writings, and identify samples
that were the best out of all that they had
seen, samples that were average, and samples
that were poorest. They were instructed to
look at punctuation, grammar, spelling, hand-
writing, spacing, and indentation of the first
sentence and to try to rate them holistically,
that is, they were not to pick out all the punc-
tuation and grammar mistakes in each writing
sample, but to look at each child’s overall
writing ability based on these samples. After
examining the students’ samples, they rated
each sample on a scale of one to seven. The
interscorer agreement between the two inde-
pendent raters was 84%.
Results
To identify writing measures that would be
the best indicators of early writing proficien-
cy, data analyses focused on the relation
between students’ performance on the poten-
tial writing measures and performance on the
criterion scores representing general writing
proficiency. In conducting our analyses, we
were interested in measures that embodied
several different characteristics. Ability to
discriminate among students at different skill
levels and sensitivity to growth (Tindal &
Parker, 1991) were two of the criteria that we
hoped our writing measures would possess.
The potential of a measure to discriminate
among students at different skill levels
increases if it produces a wide dispersion of
student scores (a large standard deviation).
Sensitivity to growth would be established if
there was not a floor or ceiling effect for the

measure and if students had room to grow on
the measure, without reaching the maximum
score.
Performance en Vkitin _g 7&#a
In our first analysis we examined the
descriptive data for each writing measure to
determine whether sensitivity to growth and
discrimination criteria were met. The means
and standard deviations for each of the writ-
ing measures are reported in Table 1.
Measures given at the beginning of summer
school are referred to as Initial, whereas mea-
sures given at the end of summer school are
referred to as Final. As seen in Table 1, none
of the mean scores for the measures approach-
es the total possible, so students had room to
improve on all measures. In other words, they
were not approaching a ceiling. In addition,
variability among participants on each mea-
sure was considerable, with standard devia-
tions ranging from 6.92 to 23.27. Thus, there
was a large dispersion of student scores for
each measure. Measures with the largest stan-
dard deviations included Word Copying-
Correct Letter Sequences, Word Dictation-
Correct Letter Sequences, Sentence
Dictation-Correct minus Incorrect Word
Sequences, and Sentence Copying-Correct
Word Sequences. Thus, all measures were
able to discriminate among students at differ-
ent skill levels and were sensitive to growth,
with several measures showing particular
strength on these criteria. It should be noted
that one student was able to complete the
word copying task at both initial and final test
sessions, although the total possible for this
task was still not met.
Cbrrelaticns Between the Criterion
variables
The next step in our analyses was to deter-
mine whether the criterion variables were surf
ficiently related to be appropriate for use in
subsequent analyses. For that purpose, the cri-
terion variables were grouped into sets of
Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015
27
atomistic and holistic measures. Atomistic
criterion variables included the average num-
ber of words written, words spelled correctly,
correct word sequences, and correct minus
incorrect word sequences. These scores were
taken from students’ writing samples given at
the end of the summer program. Holistic cri-
terion variables included the classroom
teacher’s holistic ranking of students’ writing
ability, the average of two holistic ratings
from students’ initial writing samples, and the
average of two holistic ratings from students’
final writing samples. The correlations
between the atomistic and holistic criterion
scores are presented in Table 2. Inspection of
this table reveals that the coefficients ranged
from .53 to .98, indicating that the two sets of
criterion measures thought to be broadly asso-
ciated with overall writing performance were,
in fact, related.
Qimfm Validity
The key characteristic to be demonstrated
by the early writing indicators is criterion
validity with other accepted assessment meth-
ods (Tindal & Parker, 1991). The criterion
validity of our measures was evaluated
through a series of correlational analyses.
Results of the analysis between predictor vari-
ables and the atomistic criterion variables for
the initial and final data collection are pre-
sented in Table 3. Predictor variables are list-
ed horizontally, whereas atomistic criterion
variables are listed vertically. Due to the large
number of correlation coefficients, we set a
stringent alpha level of p < .001. When exam-
ining the columns for each measure, it is evi-
dent that most of the strong, statistically sig-
nificant correlations with the atomistic crite-
ria are obtained for the dictation measures.
Dictation measures included both word dicta-
tion (14 out of 24 correlations were signifi-
cant, p < .001, column two) and sentence dic-
tation (17 out of 32 correlations were signifi-
cant, p < .001, column four). Sentence
Copying, in the third column, also showed
28

- ç: C’CI o0 z

<c s

o 6ê s
N
~
&Scaron; x
G
-£ &copy; .- <
i !
_#f 0
.y. 2 x~ s
J
°~ 3
i
?
C’CI _£
.:;::

.~M
>1
M
~ .s

3
S %m
o
Uh
--
S? ~0
w
rA
-’sf - fl
i
n. .~
E
ed
-
en
.~CU
_£
ç:
w
u
c
m N
c~4 I g’§fi< ’&dquo;

E O U
L ~~]0
a
n
.~ ~
.9 ~ C’CI
P

a~ S ~
’O t0 G
z

fi
E t ~ fi
0 ~=! ~ ’~ E
L
(A
~-
±
&dquo; __-~ oj3
p S* :E
0
u .~ ,_ £
u .,¡:::
L

10- o .-

H
e .a 0v. ~ ~^v. c:
0
~5
>< ~ . S ~ : I
~#~ ä08~
Gw.
°§
a .. S
I’f&dquo;B
&6scaron; ~~~ E
16.
? ..9~~
% b
-T3
c
]..gä3
m ~ o
(on -S~E
s 8 ~ ¡;
§§ °~%f,&cop£y;n~8(+
TS
m
(A
c
m
_~~) K m # #~~ &copy;~< o < .5- o £ õ
0)
~ ~ ~ g ~
!-~!!
1 0P3
j
’p < _ Kti
-
VJ
.il -o
m
9 C Z &scaron;~..g

Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015

 29

(A
QJ
:3
e_~0
.¡:

C
o
L.
QJ
°11
U
U
°$g
.4 1
E
0
Q
&dquo;&dquo;C
C
~
. -L,
’$g
.~
O
:::J:
c
cu
QJ
A
QJ
m
CII
C
o
°fl
. tv
GJ
-

0
U 0
9
N vi
QJ 1-1.
» *

Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015

30
significant correlations with some of the
atomistic criterion variables for the initial and
final data collections (7 out of 32 correlations
were significant, p < .001 ).
Examining the columns again, but now
noting differences in individual scoring meth-
ods (i.e., WW, WSC, CLS, CWS, C-IVVS),
when the word dictation, sentence copying,
and sentence dictation tasks were scored
using words written, words spelled correctly,
correct letter sequences or correct word
sequences, correlations were higher than
when the writing measures were scored using
correct minus incorrect word sequences, with
many correlations ranging from .80 to .92. In
the case of the word copying task, however,
correlations were in the low to low-moderate
range for all scoring methods. Additionally,
the majority of correlations increased from
the initial to the final data collection (as can
be seen when examining the horizontal rows
in Table 3). Thus, it appears that word dicta-
tion, sentence dictation, and sentence copy-
ing were the most promising measures com-
pared to the atomistic criteria, particularly
when scored using words written, words
spelled correctly, correct letter sequences, or
correct word sequences.
Correlations between predictors and holis-
tic criterion variables are presented in Table
4. Few significant correlations were obtained
between the writing measures and the holistic
criteria. However, as can be seen in columns
two and four, correlations between the writ-
ing measures and the holistic criteria were
strongest for word dictation (8 out of 18 cor-
relations ranging from .62 to .83, with one sig-
nificant correlation, p < .001 ) and sentence
dictation ( 17 out of 24 correlations ranging
from .61 to .84, with three significant correla-
tions, p < .001 ). No particular scoring method
appeared to produce higher correlations for
the holistic variable set. The rows in Table 4
also show correlations increasing from the ini-
tial to final data collection for most measures,
with the exception of sentence copying, for
which many correlations decreased.
Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015
Compared holistic criteria, once again,
to
word and sentence dictation had the strongest
criterion validity.
Discussion
The purpose of this study was to examine
alternative approaches to sampling early writ-
ing performance to identify measures that
might be used to monitor individual student
growth and to formatively evaluate the effec-
tiveness of writing instruction. Scores from
four writing tasks were examined to deter-
mine their criterion validity and evaluated
with respect to criteria that are important
when selecting assessment procedures: consis-
tent administration, reliable scoring, effec-
tively discriminating among students at differ-
ent levels, sensitivity to growth, and face
validity (Tindal & Parker, 1991 ). The alter-
native performance scores examined were
taken from writing from word or sentence dic,
tation and word or sentence copying.
An examination of the descriptive data for
all of the performance scores reveals that all
tasks result in a wide dispersion of student
scores. The largest standard deviations occur
for sentence copying. The large standard
deviations were apparent when the sentence
copying task was scored using any of the scor-
ing methods (i.e., words written or words
spelled correctly). In addition, standard devi-
ations for all measures apparently are larger
when the measures are scored using correct
letter sequences (for word dictation and word
copying), correct word sequences (for sen-
tence copying), or correct minus incorrect
word sequences (for sentence dictation).
When considering the range of possible
scores, it appears that students do not score
zero or reach a ceiling on any of the tasks. In
particular, student performance for sentence
copying and sentence dictation measures is
low compared to the total possible.
Consideration of both the dispersion and
range of scores is important because these
technical characteristics limit the ability of a
 31

1% £ enen u
&dquo; g
’ C63
~
N

! s
en ;:I ~ V en
VY
£ O 3 &copy;
O
UD %
e <u
’E 6
.8~OJ) £
v
v
...c:: v
9
» &dquo;t7

N
9 .9
’

1
c
G
~
~ V s
::. v ~ .- >- gf 0- 5 ;:I
E
tv
<u~ 0 0. ~ UJ 7
E
cu
fl n n~ ’:&dquo;;0>~.
v
’&dquo;0

w
. L-1
-_J~i ~
<S
-~
S’-’ ~
ot
~5
M)
~£ Y ~

Q
0 iN S~ ~Y ~.w-6 S!
.- ¡::
S
v ¡:
fl

T3
c4 fl X S
c
m
V)
JJ
’&dquo;0 ~ ~ O c1 U
S
Ot
~
v~
F
cu t
.9 ~
m

Q- ~ ~

’&scaron; c: s rfJ
’&dquo;0 ~ ¡:: OJ)
U s
</) tv w ’§~
w
u
c
s
0 s :
3 ’&dquo;0 t) (f)~ ~vs
jQ
E
E ~ i ~
~) o
~
M)
_

C;;
- y’i
~.0- G
°>. °2 %
v
i Gw
s ~.~
~
E ’&dqOuo;0 . . . <.) . .c: ’-’.£¡: u N
ll
0 3 i M
i
i G h
M) ~-! U

p
0
£ £ / ’ q~
y OJ) 1- x
i*
i8~
~
ul ê [0
c ’&scaron; t ll~
n
8
%
°
w
40
3 ?
’n ~ ~
fl j 0-
UJ
8 /J fl , %
’S q~ §£ £ 9 r11
en
m
c
O
~~~ ~
~~ .

c
0 ~~vll,..~n
~ ~~ ~ &5
m a~ Y
~
i
0
3e~~&dquo;
Y~ ~G5 ’~~~ §
n.au .~2 U’EvI 5

<J ~ ~ .s ~ ~
M i ri, 3
K < # ) , fl §£ w ~ £ m G * I ~ m # I l 8 £ Vl
.R
.

ja <ü $: vi
¡¡¡ &dquo;

9 o~.’ain,
z s..s.&scaron;~:*

Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015

32

*g 55o
%
g
~- s s 6
~ N
S
~
4.
~ °i
¡::

3S j~ ?
.G
~
N
O
# j
-5 6
M) 6 U
...c;¡ ’&dquo;0
GJ
9
:&scaron;
.-

i
~ w

G ~ 3 &c U

_~#sjfl n f.§oOy£i~~w ~c55fgl§8.

SS
.f
~

’&dquo;0 c; U ’S

% £ &copy; 3 ) _ W
fi = $ S
tv
S! .5i~ ~ -

Q,/
h..
1 ~j .S ~
~! 1S 6g
u
u
M)

0
°4 g
% <g ~
I ~a~r -~s c1) l
c
(V i & u

~
ri)
M > e
]/.-P@fW#_
Q.
E ~ # c
m
CA
£
.~ ’b cJ
n
Q,/
w
c
i ~ &dquo;&dquo;8
M _J )
tv
E ’I # Dc b£ st
0~
’&dquo;0 . &dquo; 8(f) I
I’ll o3 s
E , Z£ Qu)
9~e- ~£~ fi
0
It: s! ~0 30 u% W
s ê ~~ ~
0 ’&scaron; ~.r; 8
u
U) ~!&dquo;
s§~ K
~
c
Q,/
&dquo;’&dquo; rJ). &
’b£ w°i~ y ~11I I ·t (Cn
i .~ ’õ ~ c;
’s
OQ ’s’s’<~ ~
’6 5 &dquo;ti’ ~ ~
c ~~~~&S
0
~0
4m, £ _ , ,
-tv
S
3 ‘~ GII .S ~
0
~~¡õê
n ,h , <
a
’r

U
.4
3 N-ow 3 -oo ’~,~g
a
J3 :<~~~~8
~o ~~.~~’Qj ~~ II.a’Vlvi
,

9 2 ~t

Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015


measure to discriminate change in individual
student scores across time. It appears then,
that the measures of sentence copying and
sentence dictation showed the greatest
promise as indicators of early writing profi-
ciency when considering their potential for
discriminating performance within and
between individuals.
The criterion validity of a measure is one of
its most important characteristics. To forma-
tively evaluate instruction and to make pre-
dictions about the acceptability of student
growth, it is essential to establish an empirical
basis for using data to inform those decisions.
In the present study we examined both atom-
istic and holistic criteria in an effort to broad-
en the base for identifying the measures that
might be most informative. The results of the
analyses related to criterion validity provide a
basis for concluding that word and sentence
dictation can function as measures of early
writing proficiency. That is, scores from both
approaches to sampling performance enter
into high correlations with the criterion vari-
ables. Performance on the word and sentence
dictation tasks shows particularly close associ-
ation with the number of words spelled cor-
rectly, and the number of correct letter or
word sequences. Results indicate that sen-
tence copying might also be used to measure
writing performance, since it entered into
many significant relationships with atomistic
criteria; however, the strength of the sentence
copying relationship is not as strong as those
for the dictation measures. Additionally, the
range of reliable relationships between sen-
tence copying and the holistic criteria seems
narrower, which makes it less certain as an
indicator of broad writing proficiency.
Generally, the relationship between the
potential indicators of writing proficiency and
atomistic criteria was higher than that
between the writing indicators and the holis-
tic criteria. One notable exception is that the
word dictation scores seem to be strongly
related to independent ratings, where moder-
ate to strong correlations were found at the
Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015
33
beginning of the summer program. The
stronger relationships between the scores
obtained from the performance samples and
atomistic criteria are not surprising, and are
most likely associated with the technical
characteristics of the criterion scores. The
CBM procedures used to score student’s writ-
ten expression produce a wide dispersion of
scores. The holistic ratings, in contrast, have
a restricted range. Differences in the distribu-
tion of scores necessarily result in differences
in the strength of correlations based on those
distributions. A related point is that the valid-
ity of teacher judgment of student writing
proficiency, while widely accepted, is not
always certain.
With few exceptions the correlations
increased from the initial to the final data col-
lection. From this we conclude that the writ-
ing measures provide a stable measure of per-
formance and that the criterion validity is
likely to be maintained in future explorations.
One concern is that student scores on the per-
formance samples did not increase from the
beginning to the end of the summer program.
We are not certain whether this means that
the students did not improve in their writing
skills during that relatively short period of
instruction, or whether the prospective indi-
cators are not sufficiently sensitive to growth
in the short term. Since the summer program
in which the students were enrolled was only
5 weeks believe that there was not
long, we
enough time for these students to significant-
ly grow. Nevertheless, future studies are nec-
essary to examine whether student growth on
the early writing measures will be revealed
when measures are utilized on a frequent basis
(e.g., once per week).
Face validity is often a concern when rec-
ommending procedures for teachers to use in
formatively evaluating their instruction. In
selecting measures we were mindful of this
issue and tried to examine measures that
involve straightforward scoring of actual sam-
ples of student writing. We thought that the
teachers would view the early writing tasks as
34
appropriate to use with their students and
that the tasks might provide clear data on
student performance in writing. The measures
are easy to create, as teachers can construct
probes with words and sentences taken from
high-frequency word lists. In addition, the
graphed results of these short measures of stu-
dent performance in writing can be valuable
to teachers as they examine instructional
effectiveness. Another positive feature of the
measures is that they can be completed in 3
minutes, thus taking up very little instruc-
tional time. Further research must be con-
ducted to determine whether differences in
sample duration affect the ability of the mea-
sures to predict writing proficiency. Perhaps
even shorter samples (1 or 2 minutes) of per-
formance will demonstrate potential as indi-
cators of writing proficiency.
Based on the results of this study, it appears
that indicators of early writing proficiency
can be identified that will allow us to monitor
growth in early writing proficiency and evalu-
ate instruction. This is important because the
current CBMs of written expression function
well after students become fluent in encoding
oral language into written, but they are less
serviceable for beginning writing instruction.
The results provide a basis for optimism that
these early writing measures will serve teach-
ers and researchers as well as CBM has done
in early reading instruction. We hope that the
findings presented here will encourage others
to conduct research examining the effective-
ness of these early writing measures with larg-
er groups of students in grades 1 and 2, and
over longer periods of time. Eventually, per-
haps, normative development and benchmark
standards will be established that enable edu-
cators to not only describe, but also improve
the writing instruction of beginning writers.
Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015
References
Bradley-Johnson, S., & Lesiak, J. L. ( 1989). Problems in
written expression: Assessment and remediation. New
York: Guilford.
Deno, S. L. (1985). Curriculum-based measurement:
The emerging alternative. Exceptional Children,
52(3), 219-232.
Deno, S. L. (1992). The nature and development of cur-
riculum-based measurement. Preventing School
Failure, 36(2), 5-10.
Deno, S. L., Marston, D., & Mirkin, P L. (1982). Valid
measurement procedures for continuous evaluation
of written expression. Exceptional Children, 48, 368-
371.
Deno, S. L., Mirkin, P. K., & Marston, D. (1980).
Relationships among simple measures of written expres-
sion and performance on standardized achievement tests
(Research Report No. 22). Minneapolis: University
of Minnesota Institute for Research on Learning
Disabilities.
Espin, C. A., Scierka, B. J., Skare, S., & Halverson, N.
(1999). Curriculum-based measures in writing for
secondary students. Reading and Writing Quarterly,
15, 5-27.
Espin, C. A., Shin, J., Deno,S. L., Skare, S., Robinson,
S., & Benner, B. (2000). Identifying indicators of
written expression proficiency for middle school stu-
dents. The Journal of Special Education, 34, 140-153.
Harris, A. P., & Jacobson, M.D. (1972). Basic elementary
reading vocabularies. New York: Macmillan.
Isaacson, S. (1988). Assessing the writing product:
Qualitative and quantitative measures. Exceptional
Children, 54, 528-534.
Marston, D. ( 1989). A curriculum-based measurement
approach to assessing academic performance: What
it is and why to do it. In M. Shinn (Ed.), Curriculum-
based measurement: Assessing special children (pp. 18-
78). New York: Guilford.
Marston, D., & Deno, S. L. (1981). The reliability of sim-
ple, direct measures of written expression (Research
Report No. 50). Minneapolis: University of
Minnesota, Institute for Research on Learning
Disabilities.
Parker, R., Tindal, G., & Hasbrouck, J. (1991).
Countable indices of writing quality: Their suitabili-
ty for screening-eligibility decisions. Exceptionality,
2, 1-17.
Roit, M., & McKenzie, R. ( 1985 ). Disorders of written
communication: An instructional priority for LD
 35

students. Journal of Learning Disabilities, 19, 258-260. Watkinson, J. T, & Lee, S. W. (1992). Curriculum-
Tindal, G., & Parker, R. (1989). Assessment of written based measures of written expression for learning-
expression for students in compensatory and special
disabled and nondisabled students. Psychology in the
education programs. The Journal of Special Education,
23 , 169-183. Schools, 29, 184-191.
Tindal, G., & Parker, R. (1991). Identifying measures
for evaluating written expression. Learning Disabilities
Research and Practice, 6, 211-218. Author Note
Videen, J., Deno, S. L., & Marston, D. (1982). Correct
word sequences: A valid indicator of proficiency in writ- We thank Andrea Thompson for her assis-
ten expression (Research Report No. 84). tance with rating the students’ writing sam-
Minneapolis: University of Minnesota, Institute for
Research on Learning Disabilities. ples.

Downloaded from aei.sagepub.com at DALHOUSIE UNIV on June 21, 2015