You are on page 1of 17

Journal of

Language and Translation


Volume 8, Number 2, June 2018, (pp. 39 - 55)

Impact of Dynamic Assessment on the Writing Performance of English as


Foreign Language Learners in Asynchronous Web 2.0 and Face-to-face
Environments

Zohreh Zafarani1, Parviz Maftoon2*


1
Department of Foreign Languages, Science and Research Branch, Islamic Azad University, Tehran, Iran
2
Department of Foreign Languages, Science and Research Branch, Islamic Azad University, Tehran, Iran

Received: 05 August, 2017 Accepted: 4 November, 2017

Abstract
This study sought to investigate dynamic assessment (DA) - an assessment approach that embeds inter-
vention within the assessment process and that yields information about the learner’s responsiveness to
this intervention - and the writing performance of the second language (L2) learners in Web 2.0 contexts.
To this end, pre and post-treatment writings of 45 participants were analyzed to examine the impact of
asynchronous collaborative computer mediation and face-to-face collaborative mediation on L2 learners’
writing performance. Three textual features of syntactic complexity, vocabulary complexity, and quantity
of the overall information conveyed in the learners’ pre and post-tests served as the basic units of analy-
sis. The findings of the present study indicated that using blogging as a Web 2.0 tool to provide mediation
contributed more to the enhancement of the overall writing performance. Moreover, the asynchronous
collaborative computer mediated group, as compared to the face to face mediated group, showed signifi-
cant improvement in the vocabulary complexity, syntactic complexity, and quantity of overall infor-
mation presented in a single paragraph. The findings of the present study also revealed that DA proce-
dures were applicable via Web 2.0 tools and were advantageous to L2 learners’ writing performance sug-
gesting that L2 practitioners and instructors should dynamically consider the integration of Web 2.0 tech-
nology into L2 writing courses.

Keywords: Asynchronous collaborative computer mediation, Dynamic assessment, Process writing, Syn-
tactic complexity, Vocabulary complexity

INTRODUCTION empirical research studies about the assessment


During the past several years, there has been a procedure of writing as a process is limited
change in English as Foreign Language (EFL) (Xiaoxia, & Yan, 2010). The empirical studies of
writing instruction i.e. the product and process writing assessment are restrained to one assess-
approach to writing. Despite the recent wide- ment method across either all stages of writing
spread use of EFL process writing, the number of process or several assessment methods in one
*Corresponding Author’s Email: stage of writing process. Some other studies (e.g.
pmaftoon@srbiau.ac.ir Ableeva, 2010) examined the modified assess-
40 Zafarani, Maftoon. Impact of Dynamic Assessment on the Writing Performance of English …
ment methods of EFL writing assessment, e.g., assessment and intervention through focusing on
peer assessment and self-assessment. the ability of the learner to respond to interven-
While traditional assessment aims to summa- tion (Heywood & Lidz, 2007).
rize students’ learning at some point (e.g., at the Dynamic assessment (DA) has several ad-
end of the course), DA looks for the immediate vantages over non-dynamic types of assessment.
and contextualized feedback to assist the instruc- One of the DA’s advantages is that it provides a
tor and the learners during the learning process plenty of information about learners’ abili-
(Lantolf, & Poehner, 2013). DA stems from the ties. DA reveals the degrees of knowledge that
reciprocally constitutive relationships between learners gained at certain points of instruc-
methodology and epistemology. It is deeply root- tion. As such, it provides instructors with means
ed in the theory of Vygotsky’s sociocultural theo- of identifying those learners who have trouble
ry (SCT) of mind (as cited in Poehner, 2008). with learning. In addition, DA enables instructors
Vygotsky’s SCT advocates the primacy of social to provide detailed descriptions of the abilities of
constructivist theory in which social interaction is these learners so as remedial programs can be
the main thrust in language development. Social planned (Lantolff & Poehner, 2011).
constructivist theory is chiefly applied to address Another advantage of DA is that it increases
the learning through social interaction as repre- the number of opportunities for learners to inter-
sented by the much-heralded concept of zone of act with the instructor via both mediation and
proximal development (ZPD), which is the dis- feedback (Lantolf & Poehner, 2013; Shrestha &
tance between one’s actual cognitive capacity Coffin, 2012). Unlike the traditional summative
and the level of potential development through assessment, DA is more suitable for process writ-
mediation or scaffolding (Poehner, 2008). Subse- ing because in DA the instructor’s major role is
quently, under collaborative conditions learners to promote and to provide immediate and situated
reveal certain emergent functions, which have not mediation during the whole procedure. Both ver-
been yet fully internalized or have not been part bal and written feedback are effective tools if
of ZPD. provided while learners are in the process of writ-
As such, DA is a way of assessing the true po- ing drafts and revising them. Comments made by
tential of learners, which takes the interactive the instructor and the peers on learners’ writing
nature of leaning into the process of assessment. drafts, provide learners with information about
The instructor and the learners enter into dia- the clarity and impact of their writing. When
logues to catch on the learners’ existing level of learners receive feedback while they are drafting,
performance on any task and share with each they are more apt to use that feedback to revise
other the possible ways in which that perfor- their drafts. In addition, they have an immediate
mance might be enhanced in the future. This pre- chance to examine the comments in their writing,
arranged mediational instruction and the assess- which may lead to a meaningful application of
ment become a continuous process (Lantolf & what they have learned from the feedback. In
Poehner, 2004, 2011; Poehner, 2008; Poehner & fact, this learning phase is one of the concepts
Lantolf, 2013). that distinguishes DA from other forms of as-
DA is neither an assessment tool nor a method sessment.
of teaching but a framework for conceptualizing This study investigated the applicability of
teaching and assessment as an integrated activity DA procedures to the assessment of writing per-
of traditional assessment. It is an interactive ap- formance in Iranian EFL settings via Web 2.0
proach to undertake the assessment that follows a asynchronous collaborative computer mediation
test-intervene-retest format. This approach focus- (ACCM) using blogging. If one endorses the
es on the learning processes and modifiability. It Vygotskian view of cognition, and if the aim of
provides the possibility of direct linkage between education is to improve the learners’ cognitive
Journal of language and translation, Volume 8, Number 2, June 2018 41
development, the logical conclusion is that as- from one another is how they conceptualize me-
sessment and instruction cannot be separable. diation. While some of DA approaches employ a
Instructors need to interact with individual learn- flexible approach to examiner-examinee interac-
ers in the process of performing a task to enable tions, some others provide mediation.
themselves to manage the task when they cannot Lidz and Gindis (2003) argue that two funda-
do so on their own and discover those hidden mental points of difference exist between DA and
abilities that learners may use in the future. non-DA approaches. The first point of departure
DA takes on significant meaning when con- is the purpose for which the assessment is con-
sidered in Iranian context, where tests are still ducted. In DA, instruction, assessment, and me-
used as a single information-gathering tool to diation are inseparable activities. In essence, the
make judgments about individual learners. In a main goal of assessment is to promote develop-
country like Iran, where in-depth empirical re- ment.
search on DA still appears to be limited, doing In the same vein, Poehner, Zhang, and Lu
further research zeroing in on DA appears to be (2015) maintain that the integration of assess-
necessary. ment and instruction leads to test-takers’ learning
By integration of assessment with EFL writ- during assessment. This is recognized as instru-
ing instruction based on blogging, the research- ment decay in the assessment literature, which
ers’ aim was to determine the usefulness of using poses a problem for test reliability. However, the
weblogs to enhance L2 writing skill by providing counter-argument is that DA and non-DA ap-
learners with a space to reflect on what has al- proaches do not fall on a continuum, as they are
ready been covered in class, mediate one another, epistemologically different. DA approaches aim
engage in writing, and to express themselves. The at understanding examinees’ skills, promoting
researchers also suggested some implications to development of their skills, and predicting exam-
the EFL practitioners about using weblogs in the inees’ future course of development, whereas the
L2 classroom within DA framework in educa- goal of non-DA approaches is to determine and
tional contexts. measure the current abilities of examinees. In
other words, the difference between DA and non-
An Overview of Dynamic Assessment DA approaches is not a matter of instrument but a
Dynamic assessment (DA) resides in Vygotsky’s matter of purpose and procedure.
notion of zone of proximal development (ZPD) The second point of difference between DA
and emphasizes the unity of assessment and in- and non-DA approaches is how the abilities un-
struction with the goal of learners’ development. derlying the procedures are viewed. From the DA
DA proposes that mediation of the examinee’s perspective, individuals’ abilities can be com-
performance is an integral part of the assessment pletely assessed on condition that the examiner
process (Lantolf & Poehner, 2004). In general, collaborates with examinees during the comple-
DA follows a test-teach-retest format over a peri- tion of assessment tasks. In fact, DA transforms
od of several weeks. Although models of DA the examiner-examinee relationship to that of the
vary, most share the incorporation of a test- mediator-learner’s with the mediator intervening
intervene-retest format, as well as a focus on during the assessment through providing learners
learner modifiability and on underlying meta- with help ranging from implicit (e.g. prompts) to
cognitive processes that facilitate learning. The explicit (e.g. explanation). Deeply rooted in so-
role of the assessor as an active interventionist, cio-cultural theory, which advances that the de-
rather than a passive recorder, is also a common velopment of higher psychological functions
ingredient (Poehner, Zhang, & Lu, 2015). takes place via social interaction, DA holds that
The core to any DA approaches is mediation. collaboration with the examinee is necessary for
In effect, what makes DA approaches different prompting and assessing development.
42 Zafarani, Maftoon. Impact of Dynamic Assessment on the Writing Performance of English …
An intensive review of the related literature munications systems, has technologically revolu-
reveals that the majority of studies investigating tionized the means of knowledge production and
the effects of DA in ESL focus on listening and delivery (Warschauer, 1996). From a socio-
speaking skills. Shrestha and Coffin (2012), cultural perspective, CMC, with the high interac-
however, studied the effectiveness of instructor tivity of its communication capacity, is not only a
text-based mediation in academic writing con- tool but also a medium of social interactions.
texts. The outcome of their study showed that DA Thorne (2008) holds that CMC shapes social
helps instructors to find areas that learners need communities, within which diverse interactions
the most support and that DA contributes to take place and thus provide ample collaborative
learners' writing enhancement. learning opportunities for L2 learners.
Birjandi and Ebadi (2012) studied the scaf- One of the emerging technologies that allow
folding aspect of DA during the instruction of the L2 CMC is Web 2.0. As a paradigm shift in the
writing process. The outcomes of their study manner by which the long-established World
specified that teaching in a dialogic manner was Wide Web (retroactively referred to as Web 1.0)
advantageous to the learners’ learning interest is used, Web 2.0 refers to the Read-Write Web
and enhancement of the writing competence. The which empowers Internet users either synchro-
framework they used was developed with the nously or asynchronously to generate their own
goal of integrating the assessing and assisting ideas rather than merely reading someone else’s
aspects of DA. and thus fostering greater collaboration among
Alavi and Taghizadeh’s (2014) study showed them (Thomas, 2009). Warschauer (2004) main-
that DA positively contributed to the L2 learners’ tains that learning English through computer-
development of writing skill. Their study showed assisted instruction and becoming computer lit-
that the expert/novice interaction led to the sub- erate through learning English is a trend in many
stantial improvement of their participants’ writ- ESL/EFL learning and teaching programs in
ing performance as demonstrated in their third many countries across the world.
and fourth essays. Alavi and Taghizadeh con-
cluded that the mediation provided by the instruc- Asynchronous Model of Teaching L2 Collabo-
tor paved the way for the learners to move from rative Writing
other-regulation to self-regulation. Asynchronous model of collaborative writing
Lantolf and Poehner (2011) investigated the allows for delayed interactions between the in-
effects of implicit and explicit feed- structor and learners and among learners. In this
back/mediations in forms of verification and way, learners have enough time to uncover what
elaboration on ESL writing. Their findings indi- is learned. As such, it provides learners with
cated that both types of feedback have positive more time to brainstorm ideas, read, understand,
impact on the learners’ writing performance. In reflect, edit, and respond to the written texts of
another study, Lantolf and Poehner (2013) devel- the instructor or peers.
oped a DA computerized framework for English Warschauer (1996) found that asynchronous
writing instruction. Their outcome proved that interactions led to more syntactically complex
mediation in form of interactions was beneficial texts as demonstrated by the use of subordinate
in enhancing learners’ writing performance. clauses and longer sentences that are indicative of
active cognitive processes involved in text con-
Computer-Mediated Communication in EFL/ struction. Related literature indicates that asyn-
ESL chronous interactions provide learners with more
Computer-mediated communication (CMC), as a time to construct text, thus enabling learner writ-
process by which people create, exchange, and ers to prepare responses. This, in turn, encour-
perceive information using networked telecom- ages learners to think more critically and focus
Journal of language and translation, Volume 8, Number 2, June 2018 43
on both meaning and form than during the syn- Vocabulary is one of the main features of L2
chronous computer-mediated communications writing and that L2 writing studies need to use
(Beauvois, 1997; Warschauer, 1999; Warschauer, scoring methods that rank vocabulary (Santos,
2004). 1988; Sarieva, 2007). A number of different
Moreover, asynchronous writing instruction measures have been proposed for examining the
benefit the L2 learners (Ableeva, 2010; Montero- vocabulary development of language learners.
Fleta, & Pérez-Sabater, 2010). First, they can Nation (2001) proposes the Lexical Frequency
receive instruction when and where it is most Profile as a vocabulary scoring method and de-
appropriate for them. This puts less pressure on scribed it as “the most complete profile that
learners to instantly respond and/or reflect on the avoids the drawbacks of measures such as lexical
information and provides them with more time to originality, lexical sophistication, and lexical
process information. Still, another advantage of quality” (p. 46).
asynchronous writing instruction is that it is pos- In compliance with what have been discussed
sible to collaborate through a variety of tools in- above, the researchers aimed to investigated writ-
cluding e-mail and blogs, asynchronously. ing assessment in relation to the writing instruc-
tion via blogging as a Web 2.0. Consequently, the
Text Construction following null hypotheses were formulated.
T-units. The concept of T-unit has long been ex- 1- There is no difference between the
amined as one of the primary constructs in the impact of DA procedures if conducted
learners’ writing (Grabe & Kaplan, 1996; Hunt, by means of ACCM and OCM on L2
1965) and has been used by the first researcher of learners’ writing performance.
the present study, the instructor, for the analysis 1.1 ACCM does not affect Iranian EFL
of the text surface level. Hunt (1965) defines a T- learners’ writing performance.
unit theoretically as “the main clause of a sen- 2.1 OCM does not affect Iranian EFL
tence including the subordinate clauses” (p. 46). learners’ writing performance.
Sarieva (2007) defines a T-unit operationally as 2- There is no deferential impact be-
the average number of words per error-free main tween ACCM and OCM on the quali-
clause of a sentence. ty of L2 learners’ writing.
Idea Units. An idea unit allows measuring the 1.1 The syntactic complexity of the pre-
information that is jammed in a single focus. In and post-test paragraphs of the partic-
other words, an idea unit is linguistic expressions ipants does not differ within and be-
of focuses of consciousness (Chafe, 1980). tween ACCM and OCM.
Chafe suggests that most readable writing shows 2.1 The vocabulary complexity of the pre-
idea units quite noticeably. Comprehensible writ- and post-test paragraphs of the partic-
ers use punctuation marks to show idea unit. ipants does not differ within and be-
Moreover, Chafe assert that written idea units, tween ACCM and OCM.
compared with spoken idea, tend to be longer; the 3.1 The quantity of the overall infor-
mean length of idea units produced in spoken mation in the pre- and post-test para-
discourse is 6-7 words, whereas the mean number graphs of the participants does not dif-
of words per written idea unit is approximately 11. fer within and between ACCM and
Hildyard and Hidi (1985) provide a measurable OCM.
definition of the idea unit as a clause containing a
main verb, subject, and objects plus modifiers. METHODS
Vocabulary Complexity. Research on L2 writ- Participants
ing confirms that richness of L2 learners’ vo- The present study included one instructor (the
cabulary immensely affects L2 quality of writing. first researcher of the present study), two raters,
44 Zafarani, Maftoon. Impact of Dynamic Assessment on the Writing Performance of English …
and 45 undergraduate students in the field of tests from among which the participants were to
English studies from the Islamic Azad Universi- choose one that interested them the most. The
ty, North Tehran Branch. The participants had a pre-test topics included their favorite national
105-minute English writing class each week and holiday, their favorite course, and their favorite
overall they attended the academic paragraph city they like to travel. The post-test topics in-
writing class for 16 weeks. The participants in- cluded the uniqueness of their hometown, de-
cluded male and female Persian native speakers scription of a famous person or statue in their
(their age range was 19 to 31 years). country, and the benefits of learning a foreign
language. The pre- and post-test topics new to the
Instruments participants i.e. the topics were not discussed dur-
To ensure the homogeneity of the participants’ ing the semester. The participants were instructed
level of language proficiency, a sample of Oxford to write a paragraph with a clear topic sentence,
Placement Test (OPT) was administered. The minimum two major and two minor supporting
OPT, developed by Oxford University in collabo- sentences, and a clear concluding sentence.
ration with the University of Cambridge ESOL Furthermore, the ACCM group was given
Examinations, has 60 multiple-choice questions written instructions for blog assignments. To
with each question carrying one mark, giving a smooth the process of collaborative work among
total of 60. The time allowed to complete the test both ACCM and OCM groups and to assign them
is 30 minutes. In the current study, the reliability to work collaboratively, the instructor introduced
of the test was 0.80 as estimated by Cronbach’s what they were required to do and involved both
Alpha. groups in discussions about the advantages of
Before the inception of the instruction, both collaboration. Prior to the first online session,
OCM and ACCM group members took a para- three different types of weblog, namely, tutor
graph-writing pre-test to measure their baseline weblog (TB), class weblog (CB), and learner
writing performance. In addition, a paragraph- weblog (LB) were created. Table 1 below sum-
writing post-test was administered in the 16th marizes the types of weblog used in the present
session to verify the effect of the treatment. Three study and presents their functions.
topics were chosen for each of the pre- and post-

Table 1.
Types of Weblog and their Functions Used in the Present Study
Type of Weblog Run by Functions/Usage
Provided:
1- Instructions
2- Follow-ups on difficult areas of work covered in
TB The instructor
online sessions
3- Guidelines to assignments
4- Upcoming topics of instruction
The participants and the Used each week by four participants to post their assign-
CB
instructor ments
Individually used for practicing writing and posting assign-
LB Each participant
ments

Procedure teaching sessions in which online instruction was


The OCM group attended the class, but the given.
ACCM group met online once a week. The As for the mediation, the revised version of
online sessions served as instructor-controlled the regulatory scale designed by Aljaafreh and
Journal of language and translation, Volume 8, Number 2, June 2018 45
Lantolf (1994) was used in this study. Based on and peer-initiated prompts. Table 2 below com-
this scale, mediation was ranged from implicit to pares medium of delivery of instruction and me-
explicit and took form of both instructor-initiated diation for both OCM and ACCM groups.

Table 2 .
Differences in Medium of Delivery of Instruction and Mediation for ACCM & OCM Groups
Procedure ACCM OCM
administered in the first class session paper-
Pre-test administered in the first online session on CB
based
Instructions on TB and in an online session in a conventional classroom

pre-writing tasks on CB and LBs pre-writing tasks in a conventional classroom


Assignments sentence/paragraph writing tasks on CB and sentence/ paragraph writing tasks on paper
LBs post-writing tasks on CB and LBs post-writing tasks on paper

face-to-face instructor and peer mediation in a


instructor and peer mediation via CB and LBs
Mediations conventional classroom
written corrective feedback on CB and LBs
written corrective feedback on papers
Post-test administered in the 16th session on CB dministered in the 16th session paper-based

To operationalize DA procedures, the instruc- Table 3 below summarizes the DA framework


tor adopted Xiaoxia and Yan (2010) framework used in the present study.
of DA with features of EFL process writing.

Table 3
DA Framework of EFL Process Writing (adopted from Xiaoxia and Yan, 2010, p. 29)
Assessment Steps
Pre-task Mediation Post-task

Assigning task
Dialogues & media- Independently modifying
Choosing topics & independently choosing
tional moves selves’ topics again
topics
Writing Stages

Generating ideas,
Independently generating ideas Dialogues & media- Independently modifying
outlining, & structur-
& structuring tional moves selves’ structures again
ing

Macro- & Dialogues, mediation- Independently modifying


Independently writing drafts
micro- revising al tools moves selves’ drafts again

Additionally, the instructor adopted Poehner developed for face-to-face interactions and thus
(2008) typology for tutor and learner mediation was different to the present study, this typology
moves of advanced French learners’ speaking had to be modified in order to reflect the different
skills. However, since Poehner’s study was modes of communication (as in Table 4 below).
46 Zafarani, Maftoon. Impact of Dynamic Assessment on the Writing Performance of English …
Table 4.
Poehner’s Mediational Moves
Mediation Moves
Providing example Request for re-narration Asking for explanation
Accepting response Offering a choice Providing explanation
Request for repetition Specifying error Providing correct response
Request for verification Metalinguistic clues Identifying specific site of error
Reminder of directions Translation Helping move narration along

A chain of pre-task, mediation, post-task was topic if necessary. Next, the instructor assigned
used for each of the stages of process writing in for the participants to make outlines for their par-
the present study as discussed below. agraphs according to their own topic. Then, the
Stage I. Choosing topic. This stage intended instructor discussed participants outlines with
to assist the participants in generating writing them and encouraged peer negotiation on other
topics through the instructor’s mediation. For participants’ outlines. Next, the participants were
generating preliminary ideas, the instructor em- instructed to modify their outlines based on the
ployed brainstorming technique. This stage start- mediations they had received.
ed by the instructor’s giving the participants a Finally, the elements of a pedagogic para-
broad topic and instructing them to narrow it graph, namely, the topic sentence, major support-
down to a more specific one. Then, for the ing sentence, minor supporting sentence, and
pre-task ring, the participants worked on their closing sentence was introduced. As the final step
own to find a unique topic worth writing. To do in this stage, the participants drafted their para-
so, mind mapping was used as a visual way of graph according to their mind map and outline.
brainstorming. The participants literally drew a Stage III: Macro- and micro-revising. Once
tree with the stem, branches, and twigs and put the participants finished drafting, there came the
the topic in the stem, then put the ideas to the stage of revising with macro revising at the outset
branches or twigs consistent with their relations. and micro revising soon after. In the pre-task
For the mediation ring, the instructor started ring, the instructor’s concern was to identify ma-
dialogues with the participants to negotiate the jor problems in the participants’ first draft with
topic with them and encouraged peer negotiation regard to subject, audience, and purpose to gen-
on the topic. The instructor also asked the partic- erate logical coherence. To this end, the instruc-
ipants to go over their branching in pairs in the tor shared a sample paragraph with both groups
classroom for the OCM group and in the LB for and explained why it was considered an accepta-
the ACCM group. Next, both groups were to ble paragraph in terms of content and organiza-
generate ideas for their own topics by drawing a tion.
mind map, which was reviewed and commented The instructor incorporated Feuerstein’s three
on, by the instructor and the other participants. mediated learning experience attributes (Poehner,
For the post-task ring, the participants were 2008), namely, intentionality, reciprocity, and
encouraged to modify their topics using the expe- transcendence in the analysis of the sample para-
rience they gained through mediation provided graph.
by both the instructor and their peers. Intentionality. In order for the participants to
Stage II: Generating ideas, outlining, and understand the criteria of a good paragraph in
structuring. In this stage, DA was operationalized terms of content and organization and also to be
by the instructor’s providing instruction on how able to improve their drafts in content and in or-
to generate ideas and structure them into an out- ganization, the instructor took several activities
line. She provided the participants with some from Writing to Communicate 2; Paragraph and
tasks to generate ideas and modify the writing Essays, 3rd ed. (Boardman & Frydenberg, 2008).
Journal of language and translation, Volume 8, Number 2, June 2018 47
Moreover, the difficulty level of the sample para- reconstructive recaps (indicating the transcend-
graphs was moderated by focusing on the two ence of the concrete learning). To wrap up the
aspects of difficulty level and content in order to macro-revision phase, the participants were asked
adjust the activity level to the participant’s ZPD. first to start their individual and then their peer
The sample paragraphs were selected from the revision.
written productions of learners from the partici- In the micro-revision phase, the instructor
pants’ upper grade. As for the content, it was re- employed an expert- and peer-response technique
lated to the topic already discussed among the via which language choice, syntax, and grammar
group in stage II. were reviewed. The objective of micro-revision
Reciprocity. In this study, the interaction be- for this study was clarity; that is, to write para-
tween the participants and the mediator (the in- graphs that could be read and understood in one
structor) as well as peers was emphasized. To reading. The micro-revision phase focused on
maximize interactions, situated dialogue was proofreading for grammatical, typographical er-
used in the process of sample paragraph analysis. rors, and checking for redundancy and consisten-
Transcendence. In order to enable learners to cy of tone or level of formality. This phase also
write a well-organized and rich-content para- focused on replacing some passive verbs with
graph on their own in the future, the instructor more descriptive action verbs and improving the
recapped what was learnt in various ways includ- syntax of sentences and phrasing.
ing restating the whole activity in form of sum- Finally, the participants were assigned to rewrite
marizing its main points. The summary consisted their original draft by responding to revisions
‘we’ statements (linking the past experience to through proofreading and editing their first draft.
the present), literal recap (including a summary The Table 5 below shows the design employed in
of the immediate objective of the activity), and the current study (i.e. experimental design).

Table 5.
Design of the Study
pre-test post-test
complexity

complexity

complexity

complexity

quantity of
quantity of

presented
info. pre-

Syntactic
syntactic
Overall

Overall
writing

writing
overall

overall
perfor-

perfor-
mance

mance
sented

group

info.
Voc
voc

ACCM
OCM

DATA ANALYSIS two experimental groups of OCM and ACCM.


To ascertain that the sample included only partic- However, due to data attrition, only the data
ipants who had the same language proficiency, gathered from 45 participants were analyzed in
only those who scored one Standard Deviation the present study.
(SD) above and one SD below the mean on the The participants’ pre and post-treatment para-
OPT were included in the study. The mean graphs were normalized and later scored by two
obtained was 40.79, and the SD was 11. Those independent raters: the instructor and a Canadian
who scored below 30 and above 52 were ex- ESL writing instructor who was trained by the
cluded from the study. Out of 97 English first writer of this paper. Spelling and ambiguity
sophomores who had enrolled in three Ad- normalization were the two procedures used in
vanced Writing classes, 58 participants were the present study. For spelling normalization, the
homogenous in terms of their language profi- texts were normalized at word level. The inter-
ciency and were randomly categorized into
48 Zafarani, Maftoon. Impact of Dynamic Assessment on the Writing Performance of English …
rater reliability for the raters was found to be 0.98 came to an agreement. To obtain comparable
using Miles and Huberman’s 1 formula. For am- scores for the statistical analysis, all scores were
biguity, the texts were normalized at both word converted to z-scores and were compared through
and sentence levels. High inter-rater reliability of the parametric tests of independent paired-sample
94% was achieved. t-tests. Miles and Huberman’s (1994) formula
To answer the first overarching research ques- was used to calculate the inter-rater reliability for
tion, the two raters using the revised version of the scores. The inter-rater reliability calculated
the rubric of ESL Composition Profile (Jacobs et was 93%.
al., 1981) marked the pre- and post-tests of the To answer the second overarching research
OCM and ACCM groups. question, the researchers measured syntactic
As for the criterion-referenced validity of the complexity, vocabulary complexity, and quantity
writing tests, the results of the Pearson correla- of the overall information of the participants’
tions (Table 6 below) ran between OPT and pre- paragraph writings. Syntactic complexity was
and post-tests of writing indicated that OPT had measured through T-unit analysis. In this study,
significant correlations with both pre-test of writ- both syntactically correct T-units and incorrect
ing (r (19) = .631, p = .000, representing a large but unambiguous T-units (i.e., T-units that had
effect size), and the post-test of Writing (r (19) = minor grammatical errors but still conveyed the
.943, p = .000, representing a large effect size). author’s thoughts) were analyzed. Vocabulary
complexity score was calculated using Automatic
Table 6. Analysis of Lexical Sophistication (TAALES)
Pearson Correlations; Criterion Referenced (Kyle & Crossley, 2014). Quantity of the overall
Validity information that the participants presented in their
OPT paragraph was measured through the number of
Pearson Correlation .631** idea units. The instructor adopted Hildyard and
Pre-WR Sig. (2-tailed) .002 Hidi (1985) definition of idea units which includes
N 21
a clause containing a main verb, subject, and ob-
Pearson Correlation .943**
jects plus modifiers and measured quantity of the
Post-WR Sig. (2-tailed) .000
overall information in the participants’ writing
N 21
through measuring mean length of idea units. The
**. Correlation is significant at the 0.01 level
(2-tailed). measures of syntactic complexity, vocabulary com-
plexity, and quantity of overall information were
Concerning the reliability of the writing tests, presented with continuous scores; thus, they were
the pre- and post-test instruments had been exam- analyzed using three independent t-tests.
ined based on the results of a pilot test ran in a
semester prior to the commencement of the pre- RESULTS
sent study. The pilot study aimed at validating the A paired-samples t-test was run to compare the
scores gained by the multiple-trait rubric used in ACCM group’s mean scores on the pre- and
the present study. For the pilot pre-and post-test post-test of writing in order to probe the minor
paragraphs, the inter-rater reliability for the ru- null-hypothesis 1.1. The results of the paired-
brics scores had been calculated to be 91%. samples t-test (t (20) = 19.81, p = .000, r = .97
Following the procedures employed in the pi- representing a large effect size) (Table 7) indi-
lot test, the pre and post-test paragraphs were cate that the ACCM had a significantly higher
marked by the two raters. To secure high reliabil- mean on the post-test of writing. Thus, the mi-
ity of scores, if a score differed by more than nor null-hypothesis 1.1 was rejected. This is
10%, the raters discussed that very paragraph and supportive of DA procedures conducted by
1. Number of agreements/Number of possible agreements c= Inter-rater Reliability
Journal of language and translation, Volume 8, Number 2, June 2018 49
means of ACCM and proves that it positively affects EFL adult learners’ writing performance.

Table 7.
Paired-Samples Test; Pre- and Post-test of Writing (ACCM)
Paired Differences
95% Confidence Interval of the Sig.
Std. Std. Error T df
Mean Difference (2-tailed)
Deviation Mean
Lower Upper
6.167 1.426 .311 5.518 6.816 119.818 20 .000

A paired-samples t-test was run to compare OCM had a significantly higher mean on the
the OCM group’s mean scores on the pre- and post-test of writing. Thus the minor null-
post-test of writing in order to test the minor null- hypothesis 1.2 was rejected. In other words, DA
hypothesis 1.2. The results of the paired-sample procedures if conducted by means of face-to-face
t-test (t (23) = 12.23, p = .000, r = .93 represent- collaborative mediation improves Iranian EFL
ing a large effect size) (Table 8) indicate that the adult learners’ writing performance.

Table 8.
Paired-Samples Test; Pre- and Post-test of Writing (OCM)
Paired Differences
95% Confidence Interval of Sig.
Std. Std. Error T f
Mean the Difference (2-tailed)
Deviation Mean
Lower Upper
4.771 1.911 .390 3.964 5.578 12.231 3 .000

Furthermore, an independent sample t-test ACCM and OCM on L2 learners’ writing per-
was run to compare the ACCM and OCM formance. Based on the results displayed in Table
groups’ mean scores on the gain score of writing 9, it can be claimed that the ACCM had a higher
(post-test minus pre-test) in order to examine gained mean score (M = 6.17, SD = 1.42) than
whether there is any difference between the im- the OCM (M = 4.77, SD = 1.91).
pact of DA procedures if conducted by means of

Table 9.
Descriptive Statistics; Gain Score of Writing by Groups
Group N Mean Std. Deviation Std. Error Mean
ACCM 1 6.17 1.426 .311
Gain-WR
OCM 4 4.77 1.911 .390

The results of the independent sample t-test (t concluded that ACCM enhances EFL learners’
(43) = 2.74, p = .009, r = .38 representing a mod- writing performance when compared to face-to-
erate effect size) (Table 10) indicate that the face collaborative mediation. Nevertheless, this
ACCM was more effective for enhancing EFL should be interpreted cautiously due to the mod-
adult learners’ writing performance. Thus, it was erate effect size value of .38.
50 Zafarani, Maftoon. Impact of Dynamic Assessment on the Writing Performance of English …
Table 10.
Independent Samples Test, Gain Score of Writing by Groups
Levene's Test
for Equality of t-test for Equality of Means
Variances
95% Confidence
Sig. Mean Std. Error Interval of the
F Sig. T df
(2-tailed) Difference Difference Difference
Lower Upper
Equal
variances 2.035 .161 2.74 3 .009 1.396 .509 .370 2.422
assumed
Equal
variances not 2.79 42.0 .008 1.396 .499 .389 2.403
assumed

Three criteria of syntactic complexity, vocab- length of the T-units for the ACCM was higher
ulary complexity, and quantity of the overall in- (M = 4.87) than that of the OCM (M = -.06). The
formation presented were used to measure partic- standard deviation of the two groups was also
ipants’ language competencies as demonstrated different; for the ACCM it was 3.12, whereas for
in their pre- and post-treatment paragraphs. the OCM it was 4.74.
An independent samples t-test was run to The results of the independent samples t-test
compare the ACCM and OCM groups’ mean (t (43) = 4.05, p = .000, r = .52 representing a
scores on the gain score of syntactic complexity large effect size) (Table 11) indicate that the
(post-test minus pre-test scores) in order to exam- ACCM was more effective for enhancing EFL
ine null-hypothesis 2.1. The analysis revealed adult learners’ syntactic complexity. Thus, the
that in the post-treatment paragraphs, the mean null hypothesis was rejected.

Table 11.
Independent Samples Test, Gain Score of Syntactic Complexity by Groups
Levene's
Test for
t-test for Equality of Means
Equality of
Variances
95% Confidence
Sig. Mean Std. Error Interval of the
F Sig. T Df
(2-tailed) Difference Difference Difference
Lower Upper
Equal
variances 3.949 .053 4.05 43 .000 4.928 1.217 2.474 7.381
assumed
Equal
variances 4.16 40.1 .000 4.928 1.184 2.534 7.321
not assumed

In addition, an independent samples t-test was probe null-hypothesis 2.2. The results of the in-
run to compare the ACCM and OCM groups’ dependent samples t-test (t (43) = 5.67, p = .000,
mean scores on the gain score of vocabulary r = .65 representing a large effect size) (Table 12)
complexity (post-test minus pre-test) in order to indicate that the ACCM was more effective for
Journal of language and translation, Volume 8, Number 2, June 2018 51
enhancing EFL adult learners’ vocabulary com- Thus, it was concluded that ACCM contributes to
plexity. Thus, the null-hypothesis was rejected. vocabulary complexity more than what OCM does.

Table 12.
Independent Samples Test, Gain Score of Vocabulary Complexity by Groups
Levene's
Test for
t-test for Equality of Means
Equality of
Variances
95% Confidence In-
Sig. Mean Std. Error terval of the
F Sig. T Df
(2-tailed) Difference Difference Difference
Lower Upper
Equal
variances .016 .899 5.67 43 .000 3.515 .620 2.265 4.766
assumed
Equal
variances
5.71 42.9 .000 3.515 .615 2.274 4.756
not
assumed

To test the null hypothesis 2.3, the instructor An independent samples t-test was run to com-
used morphosyntactically correct idea units, as pare the ACCM and OCM groups’ mean scores on
well as idea units that had certain errors but were the gain score of overall writing quantity (post-test
unambiguous in the context of participants’ writ- score minus pre-test score) in order to probe
ings. In the context of the present study, dependent null-hypothesis 2.3. The results of the independent
clauses conjoined by a different conjunctions, such samples t-test (t (43) = 6.45, p = .000, r = .70 repre-
as after, although, as, as if, if, in order to, and so senting a large effect size) (Table 13) indicate that
forth; appositives; and participial clauses were the ACCM was more effective for enhancing EFL
considered as separate idea units. Conversely, re- adult learners’ overall writing quality. Thus, the
strictive relative clauses, indirect questions, and null-hypothesis was rejected and it was concluded
indirect quotations were counted as part of the that overall writing quality is much enhanced by
main clause, thus forming no separate idea unit. ACCM as compared to OCM.

Table 13.
Independent Samples Test, Gain Score of Quantity of Overall Info. by Groups
Levene's Test
for Equality t-test for Equality of Means
of Variances
95% Confidence
Sig. Mean Std. Error Interval of the
F Sig. T Df
(2-tailed) Difference Difference Difference
Lower Upper
Equal
6.4
variances .006 .941 43 .000 6.247 .969 4.294 8.201
5
assumed
Equal
6.4
variances not 42.6 .000 6.247 .965 4.301 8.194
7
assumed
52 Zafarani, Maftoon. Impact of Dynamic Assessment on the Writing Performance of English …
DISCUSSION AND CONCLUSION which have emphasized the effectiveness of Web
The results of this study showed that both ACCM 2.0 tools in L2 teaching and testing (Akcay, &
and OCM environments used to undertake DA Arslan, 2010; Montero-Fleta, & Pérez-Sabater,
significantly enhanced the participants’ writing 2010; Mueller, 2009; Noytima, 2010; Radia, &
performance across both ACCM and OCM Stapleton, 2009; Richardson, 2006; Zaini,
groups. This outcome appears reasonably rational Kemboja, & Supyan, 2011).
in that DA compares current performance of each Based on the instructor’s observation of the
individual with his/her earlier performance, participants’ activities in each task during pre-
makes inferences about each individual’s im- writing, drafting, and post-writing, OCM might
provement on such grounds, and mediates each be more appropriate for generating ideas as
individual moving a stage above his/her present learners engage in oral conversation, providing
level of ability. If on-the-spot mediation is pro- immediate feedback on one another’s brain-
vided, no one remains unaffected and thus his/her stormed ideas. This is while ACCM might be
ability would advance. These findings commen- more appropriate for organizing these ideas, as
surate with a number of other studies, including well as multi-drafting and revising, since this
Ableeva (2010); Lantolf and Poehner (2011, mode of collaboration provides round the clock
2013), Poehner and Lantolf (2010, 2013), access to the outcome of the previous stages in a
Poehner, Zhang and Lu (2015), Shrestha and written form. One might conclude that different
Coffin (2012), and Xiaoxia and Yan (2010). modes of collaboration and mediation might
DA procedures employed in the present study mingle to provide a window of opportunities for
placed high emphasis on all of the stages of the practicing different stages of process writing.
process writing, particularly the first two stages Moreover, the findings of the current study
(i.e. choosing topic - generating ideas, and outlin- reveal that participants’ knowledge of syntax and
ing, and structuring), which are usually over- vocabulary and the amount of information they
looked by most L2 instructors. In the process- presented in their writing is much enhanced by
oriented instruction, the endeavor made for en- ACCM as compared to OCM. It is abstracted
couraging writing is a dynamic, continuous, and from the data analysis that the ACCM group pro-
mutual effort of both the instructor and the learn- duced a significantly greater number of error-free
er (Xiaoxia & Yan, 2010). In the present study, T-units than the OCM group. One of the reasons
the dynamic and mutual effort was made possible is that the ACCM made it possible for the partic-
via interpreting the potential of the participants ipants to exercise a higher degree of control over
(pre-task), the interaction between the mediator writing mechanisms. It offered the ACCM group
(instructor and peers) and the participants, use of more time to prepare posts in form of topic initia-
language (dialogue), and use of mediational tion moves or responses to the instructor or to
moves. Since the mediation was made in the ZPD other learners. Asynchrony of collaboration and
of learners, remarkable progress occurred in both mediation made it possible for the ACCM group
groups of ACCM and OCM. to focus on both form and meaning to a greater
The results of the present study indicated that extent than the OCM group when generating ide-
weblogs provide a favorable environment for de- as, planning writing, editing spelling, grammar,
livering focused and tailored-to-the-learners’ and punctuation, and also writing longer sentenc-
ZPD mediations. Weblogs have the capacity of es and clauses.. The OCM participants’ immedi-
providing learners with an environment in which ate attention was directed toward expressing
they can interact with one another, share their meaning rather than form thus produced signifi-
ideas, build up their knowledge, and enhance cantly shorter and simple sentences.
their writing performance collaboratively. This The present study possesses limitations asso-
complies with an increasing number of studies ciated with time and sample-size. This study was
Journal of language and translation, Volume 8, Number 2, June 2018 53
run over the course of 16 weeks with the last ses- Social and Behavioral Sciences, 2(2),
sion devoted to the post-test. Bearing in mind that 1195-1199.
progress in writing skills demonstrable both on Alavi, M.S., & Taghizadeh, M. (2014). Dynamic
the surface--vocabulary and syntactical com- assessment of writing: the impact of
plexity-- and on deep levels--presentation and implicit/explicit mediations on L2
development, overall language use, and rhetorical learners' internalization of writing skills
soundness--takes place over long time. As the and starategies. Educational Assessment,
instructor did not have access to participants once 19 (16), 335-378.
the semester was over, delayed effect of DA on Aljaafreh, A., & Lantolf, J. P. (1994). Negative
improving the writing performance of the L2 feedback as regulation and second
learners was not tested. language learning in the zone of
Another main limitation of the present study proximal development. The Modern
was the sample-size. The difference between the Language Journal, 78(4), 465-483.
groups (effect size =.38) could be more noticea- Beauvois, M. H. (1997). High-tech, high-touch:
ble with a relatively larger sample size. Conceiv- From discussion to composition in the
ably, with a larger group of participants, the networked classroom. Computer Assisted
study would have had more power to detect a Language Learning, 10(1), 57-69.
more significant difference. Birjandi, P. & Ebadi, S. (2012). Microgenesis in
The present study offers some suggestions dynamic assessment of L2 learners’
for further research. First, the findings of the pre- sociocognitive development via web 2.0.
sent study provide some awareness for L2 in- Procedia Social and Behavioral
structors on the use of Web 2.0 to offer collabo- Sciences, 32, 34-39.
rative mediation during planning, idea genera- Chafe, W. L. (1980). The deployment of
tion, drafting, and revising stages of process writ- consciousness in the production of
ing. Weblogging is text-based in nature, which narrative. In W. L. Chafe (Ed.), The pear
enables learners to individually practice and re- stories: Cognitive, cultural and linguistic
flect on what has been done previously. Never- aspects of narrative production (pp. 9-
theless, the mode of mediation and collaboration 50). NJ: Norwood.
(asynchronous, synchronous, and offline) should Grabe, W., & Kaplan, R. B. (1996). Theory and
be considered in the design of the tasks for each Practice of Writing. United States of
stage. This is because learners participate in the America: Addison Wesley Longman
same task completion differently contingent on Limited.
the mode of collaboration. It would also be worth Hildyard, A., & Hidi, S. (1985). The production
conducting further research that studies the type and recall of narratives. In D. R. Olson,
of tasks and activities suitable for each mode of N. Torrance, & A. Hildyard (Eds.),
communication. Literacy, language, and learning: The
nature and consequences of reading and
References writing (pp. 285-306). Cambridge:
Ableeva, R. (2010). Dynamic assessment of Cambridge University Press.
listening comprehension in second Hunt, K. W. (1965). Grammatical structures
language learning. Pennsylvania State written at three grade levels. IL:
University, Pennsylvania Retrieved from Champaign.
https://etda.libraries.psu.edu/files/final_s Kyle, K., & Crossley, S. A. (2014). Automatically
ubmissions/5374 assessing lexical sophistication: Indices,
Akcay, A., & Arslan, A. (2010). The using of tools, findings, and application. TESOL
blogs in Turkish education. Procedia Quarterly, 49(4), 757-786.
54 Zafarani, Maftoon. Impact of Dynamic Assessment on the Writing Performance of English …
Lan, Y., & Cheng, L. (2005). A research on and L2 education: The case for dynamic
domestic research on EFL writing. assessment. Mind, Culture, and Activity,
Retrieved from 17(4), 312-330.
http://en.cnki.com.cn/Article_en/CJFDT Poehner, M. E., & Lantolf, J. P. (2013). Bringing
OTAL-WYJY200505000.htm the ZPD into the equation: Capturing L2
Lantolf, J. P., & Poehner, M. E. (2004). Dynamic development during Computerized
assessment: Bringing the past into the Dynamic Assessment (C-DA). Language
future. Journal of Applied Linguistics, 1, Teaching Research, 17(3), 1-21.
49-74. Poehner, M. E., Zhang, J., & Lu, X. (2015).
Lantolf, J. P., & Poehner, M. E. (2011). Dynamic Computerized dynamic assessment (C-
assessment in the classroom: Vygotskian DA): Diagnosing L2 development
praxis for second language development. according to learner responsiveness to
Language Teaching Research, 15(1), 11- mediation. Language Testing, 32(3), 337-
33. 357.
Lantolf, J. P., & Poehner, M. E. (2013). The Radia, P., & Stapleton, P. (2009).
unfairness of equal treatment: Objectivity Unconventional sources as a new
in L2 testing and dynamic assessment. convention: The shifting paradigm of
Educational Research and Evaluation, undergraduate writing. The Internet and
19(2), 141-157. Higher Education, 12, 156-164.
Lidz, C. S., & Gindis, B. (2003). Dynamic Richardson, W. (2006). Blogs, wikis, podcasts,
assessment of the evolving cognitive and other powerful web tools for
functions in children. In A. Kozulin, B. classrooms. Thousand Oaks, CA: Corwin
Gindis, V. S. Ageyev, & S. M. Miller Press.
(Eds.), Vygotsky’s educational theory in Santos, T. (1988). Professors’ reactions to the
cultural context (pp. 99-118). academic writing of nonnative speaking
Cambridge: Cambridge University Press. students. TESOL Quarterly, 22, 69-90.
Montero-Fleta, B., & Pérez-Sabater, C. (2010). A Sarieva, L. (2007). The communicative two-way
research on blogging as a platform to pre-writing task performed via
enhance language skills. Procedia Social asynchronous and synchronous
and Behavioral Sciences, 2(2), 773-777. computer-mediated communication and
Mueller, D. N. (2009). Digital underlife in the its influence on the writing expertise
networked writing classroom. Computers development of adult English language
and Composition, 26, 240-250. learners: A mixed design study Doctoral
Nation, I. S. P. (2001). Learning vocabulary in Dissertations. University of South
another language. Cambridge: Florida. Retrieved from
Cambridge University Press. http://scholarcommons.usf.edu/etd/2352
Noytima, U. (2010). Weblogs enhancing EFL Shrestha, P., & Coffin, C. (2012). Dynamic
students’ English language learning. assessment, tutor mediation and
Procedia Social and Behavioral academic writing development.
Sciences, 2, 1127-1132. Assessing Writing, 17, 55-70.
Poehner, M. E. (2008). Dynamic assessment: A Thomas, M. (2009). Handbook of research on
Vygotskian approach to understanding web 2.0 and second language learning.
and promoting L2 development. New New York: Information science
York: Springer. reference.
Poehner, M. E., & Lantolf, J. P. (2010). Thorne, S. L. (2008). Mediating technologies and
Vygotsky's teaching-assessment dialectic second language learning. In D. Leu, J.
Journal of language and translation, Volume 8, Number 2, June 2018 55
Coiro, C. Lankshear, & M. Knobel Biodata
(Eds.), Handbook of Research on New Zohreh Zafarani is a PhD candidate in the field
Literacies (pp. 417-449). NJ: Lawrence of Teaching English as a Foreign Language
Erlbaum. (TEFL) at the Islamic Azad University, Science
Warschauer, M. (1996). Computer-mediated and Research Branch, Tehran, Iran. She holds a
collaborative learning: Theory and master’s degree in TEFL from Tehran University.
practice. Honolulu: University of She has taught undergraduate courses at the Is-
Hawaii. lamic Azad University, Tehran North Branch for
Warschauer, M. (1999). Electronic literacies: the past 15 years. Her areas of interest include
Language, culture, and power in online Web 2.0 technology and distant virtual teaching.
education. NJ: Lawrence Erlbaum Email: zohrehzafarani@gmail.com
Associates.
Warschauer, M. (2004). Technological change Dr Parviz Maftoon is associate professor of
and the future of CALL. In S. Fotos, & teaching English at the Islamic Azad University,
C. Brown (Eds.), The use of weblogs in Science and Research Branch, Tehran, Iran. He
language education: New perspectives received his PhD from New York University in
on CALL for second and foreign Teaching English to Speakers of Other Lan-
language classrooms (pp. 15-25). NJ: guages (TESOL). His primary research interests
Lawrence Erlbaum Associates. concern second language acquisition, sec-
Xiaoxia, L., & Yan, L. (2010). A case study of ond/foreign (SL/FL) language teaching method-
dynamic assessment EFL process ology, and language curriculum development. He
writing. Chinese Journal of Applied has published nationally and internationally, writ-
Linguistics, 33(1), 24-40. ten, and edited a number of English books. He is
Zaini, A., Kemboja, I., & Supyan, H. (2011). currently on the editorial board of several lan-
Blogs in language learning: Maximizing guage journals in Iran.
students’ collaborative writing. Procedia Email: pmaftoon@srbiau.ac.ir
SBS(18), 537-543.

You might also like