System 105 (2022) 102712

Contents lists available at ScienceDirect

System
journal homepage: www.elsevier.com/locate/system

The effects of using an online collaboration tool on college

students’ learning of academic writing skills
Jia Li a, *, Lillian Mak b
a
Ontario Tech University, Canada
b
Centennial College, Canada

A R T I C L E I N F O A B S T R A C T

Keywords: Expository writing skills, integrating perspectives and evidential supporting sources, are critical
Intervention study for students to succeed in college. However, many students at the onset of postsecondary edu­
Expository writing cation lack adequate reading and writing skills to complete integrated language tasks, including
Reading to writing skills
paraphrasing and synthesizing information critically and coherently from multiple sources to
Online collaboration tool
College students
elicit deep thinking and make sophisticated arguments. This article reports on a 10-week inter­
vention using a cloud-based technology application to support students’ collaboration in reading-
to-write learning tasks. The intervention focused on systematically helping students develop
distinct expository writing skills progressively with embedded reading strategies. Sixty-nine first-
year students enrolled in an English course at a large Canadian community college participated in
the study. The scores of pre- and post-writing tests and standardized reading tests measuring three
reading attributes were analyzed to examine the control and treatment groups’ performance. A
statistically significant result was found in the writing scores of the treatment subgroup with
lower language proficiency levels, indicating that the instructional framework with the
technology-supported collaborative setting had a positive effect. This study provides evidence
that college students with underdeveloped language skills can benefit from a well-structured,
online intervention supporting a collaborative learning environment for academic writing.

1. Introduction

Post-secondary students need effective skills in academic writing, especially source-based expository writing for their academic
success. Brockman et al.’s (2011) study found that critical analysis and research-based writing predominates the types of assignments
given to university students across disciplines. These expository writing assignments often require students to accurately read and
interpret an author’s main idea, and objectively convey information in writing. Moreover, writing assignments in college and uni­
versity often expect students to summarize, synthesize, and critique multiple readings, in turn demonstrating their comprehension and
integration of learning of a subject matter (Bartolomeo-Maida, 2016; Maaka & Ward, 2000). Students are required to read supple­
mentary materials beyond course textbooks that deepen, expand, and contrast perspectives (Taraban et al., 2000; Tomasek, 2009).
Then, based on their understanding of the sources, students can develop their arguments of a topic while reflecting on and connecting
to their experiences and observations. Grabe and Zhang (2013) describe source-based writing as follows.

* Corresponding author. Faculty of Education, Ontario Tech University, EDU, 513, 11 Simcoe Street North, P.O. Box 385, Oshawa, Ontario,
L1H7R7, Canada.
E-mail addresses: jia.li@ontariotechu.ca (J. Li), lmak13@my.centennialcollege.ca (L. Mak).

https://doi.org/10.1016/j.system.2021.102712
Received 23 March 2021; Received in revised form 5 December 2021; Accepted 30 December 2021
Available online 3 January 2022
0346-251X/© 2022 Elsevier Ltd. All rights reserved.
J. Li and L. Mak System 105 (2022) 102712

Learning to write from textual sources (e.g., integrating complementary sources of information, interpreting conceptually
difficult information) is a challenging skill that even native speaking students have to work hard to master … Tasks that require
reading/writing integration, such as summarizing, synthesizing information, critically responding to text input, or writing a
research paper, require a great deal of practice. (p. 9)
Expository writing demands higher-level cognitive skills to submit a position clearly with well-developed supporting evidence
(Beck et al., 2013; Perin et al., 2017) by evaluating the alignment between the source content and thesis statements, using proficient
reading and writing skills to synthesize, paraphrase, and quote relevant information across texts. However, research has shown that
some students are inadequately prepared for the complexities of reading and writing tasks at post-secondary levels (Gruenbaum, 2012;
MacArthur & Philippakos, 2013). Particularly, some first- and second-year post-secondary students not only lack skills in organizing
writing to convey major and supporting ideas, but also have difficulties in reading with basic comprehension, main idea recognition,
and critical perspectives. These students find generating a thesis or developing supporting evidence from source material complicated
(Brockman et al., 2011). Cumming et al. (2005) suggest that some of the difficulty may stem from weaker reading comprehension
skills; this was evidenced by less proficient writers who tended to use verbatim phrases with unevenly developed ideas, whereas
proficient writers were more likely to summarize central ideas from source material and include adequate and coherent supporting
ideas. Wang’s (2009) study showed that only 55% of first-year university participants correctly identified explicit main ideas of in­
dividual paragraphs, and 11% correctly stated main ideas of a multiple-paragraph expository text. Apart from the content decisions
that critically depend on their reading comprehension, students must also contend with selecting the vocabulary and linguistic
structures appropriate for a specific writing task. Students need skills to evaluate, organize, and present ideas to establish their
knowledge and objectivity (Schleppegrell, 2001). Thus, it is critical to develop effective instructional strategies to support these
students who often struggle with a combination of academic writing and reading skills at the onset of post-secondary studies.
Much of the research has addressed general reading and writing challenges, with fewer studies on developing integrated skills in
expository reading and writing. In addition, research has often examined reading and writing skills separately, while acknowledging a
close connection between these two language skills. Research has identified that cloud-based technology tools, such as Google Docs,
enable students to access learning materials, practice collaborative writing activities synchronously and provide real time feedback.
Limited research, however, is available addressing both reading and writing skills with an intervention design using the technology
that is becoming increasingly popular in schools and among students. Therefore, to bridge these gaps, this paper reports on a study
examining the impact of an instructional intervention using OneNote to support first-year college students’ collaborative writing along
with group reading activities.

2. Literature review

2.1. Correlations between writing performance and reading skills

To understand the writing and reading relationship, and their shared knowledge and skills, research has examined the correlations
between writing and a subset of reading skills, including reading comprehension, spelling and vocabulary knowledge. A moderate to
strong positive range of correlations has been reported. Parodi (2007) analyzed test scores of reading comprehension and writing in
Spanish with eighth graders in Chile and discovered a strong positive correlation (r = 0.72). This indicated 51.8% commonality of
written argumentative text, explaining “a quite extensive intersecting area between comprehension and production from the cogni­
tive/textual perspective” (p. 236). The high correlation may be partly attributed to the same genre—argumentative text—used in
reading and writing tests. Allen, Snow, Crossley, Jackson, and McNamara’s (2014) study has shown a strong positive correlation
between American university students’ reading comprehension and writing performance (r = 0.57), indicating their shared common
knowledge sources and higher-level cognitive skills. Students’ performance was assessed using argumentative essays responding to a
prompt, and Gates-MacGinitie (4th ed.) tests for reading comprehension and vocabulary knowledge. Vocabulary was strongly posi­
tively correlated with both reading comprehension (r = 0.79) and writing performance (r = 0.55). The reading process appeared more
associated with vocabulary knowledge than writing.
A recent study by Schoonen (2019) examined the correlations between reading and writing skills in Dutch as a First Language (L1)
and English as a Foreign Language (EFL) with secondary school students in the Netherlands, as well as the extent that linguistic
knowledge and fluency subskills might explain the reading and writing connection. Results covering grades 8-10 revealed a strong but
declining correlation trend between students’ English reading and writing while their grades progressed (r = 0.81, r = 0.73, r = 0.63),
with eighth graders having the strongest correlation in English. However, the correlations between their reading and writing in Dutch
remained relatively lower than those of English and stable across grades (r = 0.67, r = 0.65, r = 0.68). By grade 10, the correlations in
L1 and EFL became comparable (r = 0.68 versus r = 0.63). Schoonen suggests that early linguistic knowledge shares more common
individual differences in reading and writing skills in EFL before other factors intervene, compared with later stages. The results also
showed that vocabulary and spelling (orthographic) knowledge had positively moderate to strong correlations with reading ability and
writing performance in both L1 (r = 0.45-0.77) and EFL (r = 0.36-0.81) across grades. Several interesting correlational patterns of
students’ vocabulary and spelling knowledge were observed by grade and language. Their vocabulary knowledge had a stronger
correlation with reading than writing in Dutch for grades 8-9, but the differences diminished as students progressed to grade 10,
whereas this is the opposite for their linguistic knowledge in EFL—vocabulary had a stronger correlation with writing than reading for
grades 8-10, and the differences increased as students progressed. The L1 results of Schoonen’s (2019) study for grades 8-9 are
consistent with Allen et al.’s (2014) study with college students whose vocabulary knowledge correlated more with reading than
writing performance. However, both EFL vocabulary and spelling knowledge of Schoonen’s (2019) participants were more strongly

2
J. Li and L. Mak System 105 (2022) 102712

correlated to writing than reading. Their spelling knowledge demonstrated a very strong positive correlational trend with writing
across grades (r = 0.74-0.81).
Perin et al. (2017) conducted a study in two American community colleges with low-skilled students who attended developmental
education courses where integrated reading and writing instruction was provided. Results showed that general reading and writing
skills, measured by comprehension and writing fluency subtests of the Nelson-Denny Reading Test and the Woodcock–Johnson III, had
different relationships with summary and persuasive writing, though within an overall small-to-low moderate correlation range.
General reading skills were more correlated with summary writing (r = 0.31) compared to persuasive writing quality (r = 0.16),
whereas general writing skills were more correlated with persuasive writing (r = 0.28) compared to summary writing quality (r =
0.14). These studies have made important contributions to our understanding of the academic reading-writing connection; the
research, however, remains limited regarding instructional strategies to address new college students’ challenges while concurrently
developing their expository reading and writing skills.

2.2. Effect of technology-supported collaborative writing

Writing instruction has evolved from a product-oriented to a process-oriented approach, encouraging specific writing skills to be
taught in stages with more learning opportunities through peer interaction which enhance students’ metacognitive skills during
writing, such as planning, monitoring, and evaluation activities (Storch, 2005; Teng, 2021). Collaboration in writing includes small
group discussions to brainstorm ideas that enable a dual effect of enhancing reading comprehension as well as writing quality (Shen,
2013; Yeh, 2014). This is also achieved through collectively giving and receiving feedback with more comfort, which can consolidate
knowledge to resolve errors that students are otherwise unable to do individually (Loretto et al., 2016; Wang, 2014; Wigglesworth &
Storch, 2012). Additionally, making revisions focusing on higher-level edits to writers’ ideas and organization are critical to devel­
oping expository writing skills (Kathpalia & See, 2016; Zheng et al., 2015). Technology use in writing instruction has been explored
extensively, including as a tool to facilitate collaboration for writing instruction (Ducate et al., 2011; Miyazoe & Anderson, 2010), to
assist students with academic writing (Kessler et al., 2012; Yanguas, 2020), as well as to enable younger students’ peer collaborative
learning of reading and writing, e.g., summarizing and paraphrasing of digital text in a multimodal environment (Fitzgerald & Pal­
incsar, 2017).
Research on the effect of technology-supported collaboration on writing tasks varies widely in research design and assessment, yet
the question remains whether the benefits render a measurable effect on students’ performance. Google Docs, a cloud-based platform, is
one of the most used tools to facilitate collaborative writing practices, and research has reported mixed results. Two studies examined
academic writing performance of EFL university students using Google Docs in several conditions. Suwantarathip and Wichadee (2014)
compared Thai university English learners’ collaborative writing performance using Google Docs and face-to-face interaction, and they
found that students using Google Docs had significantly higher post-test scores than their counterparts (p = 0.03). Alsubaie and
Ashuraidah (2017) studied the effect of individual and pair writing with Saudi college EFL students with and without the use of Google
Docs. Results showed that in both individual and paired writing scenarios, students’ writing scores were statistically significantly
higher when using Google Docs than not. However, Google Docs seemed to have more influence on their writing individually than
collaboratively; scores of individual writing were higher than pairs’ writing (M = 11.18 versus M = 6.00). Yanguas (2020) examined
the writing quality of English-speaking college students’ online collaborative writing using Google Docs with L1 text-based chat, L2
text-based chat, no chat, or individual writing. Significant positive results were reported in the accuracy of the writing for groups
collaborating and chatting in L1 although no significant differences were observed between L2 collaboration and chat, collaboration
with no chat, or individual writing.
With a very different study design, Liu et al. (2018) examined the impact of the use of a group awareness feature in an online
collaborative writing tool, Cooperpad, on Chinese undergraduate EFL learners’ writing performance. This feature, enabling students to
monitor and compare their behavior and contributions to the collaboration with that of other group members, aimed at improving
their engagement in a group-writing task. The results showed that the experimental group significantly outperformed the control group
in writing quality (p < 0.05) when performing difficult tasks. The researchers found that the experimental group tended to stay on task
and collaborate more on challenging components, although they pointed out that the small groups of three participants may have been
a contributing factor to engagement.
Most of the research that has evaluated the effects of technology-supported collaboration on writing performance has focused on
specific indicators of writing quality, such as content, organization, and grammar, and found a positive trend of results. Passig and
Maidel-Kravetsky (2016) compared summary writing scores of graduate students in an Information and Communication Technology
and Education program. One group of students read a chapter and digitally wrote its summary collaboratively while the other group
read and hand-wrote face-to-face collaboratively. The findings indicated that the former group of students had higher quality sum­
maries than the latter. The summaries written collaboratively online were significantly better in six out of eight performance indicators
than those from face-to-face collaboration, that is, main ideas, integration of ideas, organization, style, objectivity and holistic writing
(style, use of language, objectivity, and succinctness) (p < 0.05 - p < 0.001). Similarly, Tai (2016) studied collaborative writing in an
asynchronous online forum where nursing students in Taiwan, EFL learners, wrote individual drafts and received feedback from their
group for revisions. The results showed that the students’ writing performance improved significantly from pre-to post-test scores (p <
0.001), and post-hoc comparisons further identified a significant increase in scores for two of six indicators: content (p < 0.001) and
holistic writing (p < 0.05), in addition to the improvement in another three indicators: organization, grammar, and mechanics. For
overall performance in individual writing, Bikowski and Vithanage’s (2016) study found that L2 students in a U.S. university had
statistically significant gains after in-class web-based collaborative writing activities than their peers who wrote on their own in the

3
J. Li and L. Mak System 105 (2022) 102712

same condition.
A few studies that explored the effect of another form of online collaboration—peer review—on writing performance using cloud-
based online platforms, Google Docs, wiki, or weblog also reported mixed results. Zheng et al. (2015) investigated the effect of using
Google Docs for writing, revising, peer editing, and reviewing on reading and writing achievement of middle school students in the U.S.
Most student collaboration on Google Docs entailed peer feedback for the work of single authors, while nearly 30% of writing was a
collaborative effort between two or more students. Their results from analyzing a large number of student writing samples and
comparing students’ pre- and post-test writing and reading scores from a statewide student assessment program indicated that stu­
dents’ use of Google Docs had no significant effect on either language skill measures, though students had a positive attitude towards
using Google Docs for editing and exchanging feedback. Interestingly, the authors mentioned that students’ pre- and post-test scores in
reading and writing were found to be positively associated. Higher levels of collaboration with more technology support (interactive
whiteboard), enabling students to write and receive feedback synchronously, were found to lead to statistically better writing per­
formance than students using less (traditional whiteboard) or no technology (blackboard) (Teng, 2021).
To conclude, writing collaboratively online has been found to effectively engage students and encourage them to become more
committed to the task (Chao & Lo, 2011; Matthew et al., 2009). Research indicates that given the convenient access to online
collaboration platforms, students tend to be more motivated to participate in editing and peer review, investing more time in revising
their writing and deepening their understanding of information (Reynolds & Anderson, 2015; Yeh, 2014). Students tend to pay more
attention on how a task is completed, gain more awareness of the different stages in the writing process, and gradually become more
self-reliant in their learning (Elola & Oskoz, 2010; Sotillo, 2002).

2.3. Research questions

The use of collaborative technology tools, such as OneNote, is beneficial for sharing and constructing knowledge and enhancing the
peer review process. It has great potential to facilitate community college students’ learning of expository writing through collabo­
ratively undertaking a series of online reading and writing activities. Therefore, to bridge the research gap and meet student needs, we
developed a writing intervention using OneNote to answer three research questions.

1. What, if any, is the effect of an integrated academic language intervention using an online collaboration tool on first-year com­
munity college students’ expository writing performance?
2. What, if any, is the effect of this intervention on their learning of reading skills, specifically in spelling, vocabulary, and reading
comprehension?
3. Are there any correlations between their reading skills, including reading comprehension, vocabulary knowledge and spelling
skills, and expository writing performance in source-based writing tasks?

3. Methods

3.1. Context and participants

This study took place at a large urban community college in Canada that comprises about 45,000 students with a culturally diverse
student population. The project received approvals from the Research Ethics Boards of researchers’ affiliated university, and the
college where participants were recruited. Participants in the study were enrolled in a first-year English course entitled, “College
Communication I” based on their program requirement and their post-admission English placement test. Ninety-four students regis­
tered in four sessions of the course were invited, and 69 volunteered to participate in the study. Participants’ ages ranged 18-24 years.
Males (n = 33) and females (n = 36) were evenly represented. Students self-identified as native English speakers, but it is likely that
some students spoke another language besides English at home.
The first-year English course aims to develop students’ academic writing skills, and they learn to write structured paragraphs and
essays containing a clear thesis and supporting arguments in response to readings. This lecture-based course meets once per week for 3
h. The instructor, also one of the researchers, has three years of experience teaching the course and had used OneNote in limited
classroom activities prior to the intervention. Students had some computer skills; however, many were unfamiliar with cloud-based
collaborative tools.

3.2. Research procedure

The research, with a pre- and post-factorial design, was conducted over one 14-week semester. The intervention with 10 lesson
units were taught during weeks 2-12. The writing pre-and post-tests were administered during weeks 1 and 14, respectively. The time
allocated for the test was 1.5 h. The standardized pre-and post-tests of reading comprehension, vocabulary, and spelling were
administered during week 2 and week 13, respectively. Following the guidelines of standardized tests, the spelling test took about 20
min. Students were given 15 min for the vocabulary test and 20 min for the reading comprehension test. The instructor administered all
tests during regular class time in the classroom.
For the intervention, the four classes were randomly assigned as the control (n = 34) or treatment (n = 35) groups, with two classes
in each group. The group members were blended with higher language proficiency (HP) and lower language proficiency (LP) students
(see details about HP and LP subgroups in the data analysis section). In each class, students were taught an element essential to

4
J. Li and L. Mak System 105 (2022) 102712

developing complex expository writing skills, including paraphrasing, summarizing, integrating sources, crafting topic sentences, and
idea organization. These skills were modelled by the instructor with the entire class participating before they were divided into groups
of 2-3 students to collaboratively practice the target skills.
The intervention took place over 10 weeks (weeks 2-12). Both control and treatment classes received the same instruction, cur­
riculum materials and writing assignments, except that the treatment classes used OneNote and could sit wherever they wanted in class,
and the control classes were required to sit in groups to collaboratively complete a series of reading-to-write tasks face-to-face.
Although students in treatment classes were not prevented from communicating face-to-face, we observed that they communicated
mainly via OneNote, which is consistent with their feedback through interviews (see Li, Mak, Hunter & Cunningham, submitted, for the
full report of students’ perception of the intervention). Each weekly task required students to write about 150 words within 30 min.
The instructor provided both the treatment and control groups guided instruction and formative feedback as needed in class. The
treatment groups received primarily written feedback on their collaborative writing in OneNote; the control groups received primarily
oral feedback. Both groups received written feedback after class.
Prior to the intervention in week 2, the instructor provided students in the treatment classes a tutorial, including an overview and
short tasks to help them learn to navigate OneNote to write and edit jointly using different features, such as online resources, adding
pages, spellchecker, and dictionaries (also see Figs. 1–8 in Appendix B). In addition, for each weekly class, a specific collaborative
reading-to-write task was uploaded in OneNote, and the instructor provided instruction and demos on how to use its relevant features to
effectively learn required skills to complete collaborative tasks.

3.3. Intervention

The intervention aimed to use OneNote to close the skill gap in source-based expository writing and associated reading skills for
students transitioning from high school to college. OneNote is part of the college’s network services to which all registered students
automatically have access. Students in the treatment group were asked to bring their own computers and mobile devices, and those
without a device borrowed a laptop from the college library. The design principle of the intervention was to facilitate student
collaboration throughout the reading-to-write process. Using OneNote, students can read the text, compose a response, exchange
feedback, and edit their group members’ writing instantly. OneNote pages enable group members to synchronously write on the same
page and combine pieces of writing into one composition. Additionally, they can read other groups’ work, expanding their support
network beyond their immediate group. Moreover, OneNote may enable reticent students to participate in activities; often in group
work, some students may be more dominant, while more reserved students may find it easier to contribute by writing their ideas. Since
OneNote pages permit several students to work separately on the same page, it may create an encouraging work environment where all
students can actively participate and minimize potential conflict.
Each unit of the intervention lasted 45-60 min during the regular 3-h weekly class. The 10 lesson units were developed by two
researchers responding to the course curriculum (see Appendix A for the lesson unit topics). To ensure coherent exposure to the content
between control and treatment groups, we adopted readings, skill development exercises, and tasks from the course textbook, The
Canadian Writer’s Workplace 8ed. (Lipschutz et al., 2017). Lesson content and sequencing with instructional scaffolding were organized
to support all students to develop their academic language skills in manageable segments. Instruction focused on the elements of
expository writing and associated skills, gradually progressing to more complex skills to culminate in an essay with references. Specific
strategies were provided to aid students in self-monitoring their strengths and weaknesses in their progressive reading and writing
skills, such as skills in rearranging the order of ideas and using synonyms when paraphrasing (see screen captures in Appendix B for
sample activities using OneNote).

3.4. Measures

Eight tests (four pre-tests and four post-tests) were administered, including a writing test developed by the researchers, adhering to
the course curriculum. According to the literature reviewed above, reading skills consist of major subskills: reading comprehension,
spelling, and vocabulary knowledge. To ensure that student participants in control and treatment groups had comparable reading and
writing skills prior to the intervention, we administered three pre-tests of Nelson-Denny Reading Test, Wechsler Individual
Achievement Test (WIAT-III) and self-developed reading-to-write test. To thoroughly examine the direct and indirect effects of the
intervention on these skills, as well as the correlations between students’ reading and writing skills, we also issued the same tests after
the intervention as post-tests.

1. The Nelson-Denny Reading Test. This multiple-choice test was used to measure reading comprehension and vocabulary knowledge.
The reading comprehension section includes 38 questions covering seven reading passages; the vocabulary section consists of 80
words, and each comprehension and vocabulary question has five choices. Form G of the test was used for the pre-test, and Form H
for the post-test (Brown, Fishco, & Hanna, 1993).
2. Wechsler Individual Achievement Test (WIAT-III). Taken from the WIAT-III, the spelling test includes 44 words. The same test was
used for the pre- and post-test (Wechsler, 2009).
3. Self-developed reading-to-write test. Participants completed a pre- and post-test to assess the impact on writing performance before
and after the intervention. The writing tests we developed contained a reading and an essay writing component; we consulted with
the course coordinator and made three rounds of revisions of the tests. Students were required to read a short article and write an
essay in response to a choice between two writing prompts. The same writing test was used for both the pre- and post-test.

5
J. Li and L. Mak System 105 (2022) 102712

A course-approved writing rubric was used to assess students’ essays, considering four major aspects. These include purpose—the
degree that the response clearly addresses the topic; organization and development—the degree that ideas are logically developed and
supported; vocabulary—the range and use of vocabulary; and grammar—the range and accuracy of writing conventions. One additional
criterion, citation was included to assess the degree that source material is integrated and cited relevantly and accurately. The rubric is
aligned with Task 2 writing rubric by the International English Language Testing System (IELTS); both include four criteria under
different labels (see Appendix C and Appendix D).

4. Accuplacer—a post-admission English placement test. To understand participants’ English language skills at the onset of the inter­
vention, the college where this study was conducted provided us administrative data, including results of the post-admission
English placement test of first-year students measured by Accuplacer. The Accuplacer tests, developed by the College Board
(2020), assess listening, reading, and essay writing competence. It is used by colleges and universities globally to determine a
suitable level English course for students transitioning to post-secondary studies. The listening test includes a mix of conversations
in academic and day-to-day settings (e.g., grocery store); thus the Accuplacer tests assess students’ cognitive academic language
proficiency (CALP) as well as basic interpersonal communicative skills (BICS) (Cummins, 2008).

3.5. Data coding

The pre- and post-test scores from the writing tests, Nelson-Denny Reading Test and WIAT-III spelling test were coded, following
the test manuals. Each pre- and post-writing test was rated by one researcher and an independent rater using the rubric. Prior to scoring
the tests, they discussed the rubric to mutually agree on the grading treatment for common student writing errors within the pa­
rameters of the rubric categories and ranges. Test scores were assigned in multiples of 5 (e.g., 45, 50, 55) to a maximum score of 100.
Where the two raters’ scores differed by 10 points or more, the test results were discussed until agreement was reached. Inter-rater
reliability for the writing pre-tests was r = 0.95 and for the writing post-tests r = 0.96, indicating high rater agreement. The inde­
pendent rater also reviewed 15% randomly selected pre- and post-tests for reading, vocabulary, and spelling to verify the accuracy of
the scores.

3.6. Data analysis

To answer Research Question 1, descriptive statistics, independent t-tests, and one-way ANOVA were used to analyze students’ pre-
and post-test writing scores before and after the intervention and compare the writing scores between the control and treatment
groups. To answer Research Question 2, descriptive statistics, independent t-tests, and one-way ANOVA were used to analyze students’
pre- and post-test scores from the Nelson-Denny Reading Test and WIAT-III and compare these test scores between the control and
treatment groups. To answer Research Question 3, Pearson correlation analyses were conducted using pre- and post-test writing,
reading, vocabulary, and spelling scores of the control and treatment groups. To explore students’ performances more fully in
responding to these research questions, participants in control and treatment groups were divided into higher language proficiency
(HP) and lower language proficiency (LP) subgroups based on Accuplacer criteria (listening L85 and reading R51 scores), creating four
subgroups. Descriptive statistics and one-way ANOVA were used to compare the results between the control and treatment subgroups.
The details of the threshold used to subdivide, and its rationale are based on Cummins’ (2008) distinction in language proficiency
between BICS and CALP; proficiency in BICS can develop quickly while proficiency in CALP is more gradual. Based on their Accuplacer
scores, participants in this study exhibited proficiency in BICS and a lag in CALP. The threshold used to subdivide the control and
treatment group participants for more in-depth examination was a combination of listening and reading scores. Participants with a
listening score (L) equal to or greater than 85 (advanced low proficiency) and a reading score (R) equal to or greater than 51
(low-intermediate proficiency) were categorized in the higher proficiency subgroup; participants whose scores were below these
thresholds were categorized in the low proficiency subgroup. The listening threshold was determined based on two considerations.
First, listening skills are fundamental for the learning environment. Participants were domestic students presumably with high pro­
ficiency in BICS; thus, a weakness in listening skills could foreshadow that a student may be underprepared and face potential aca­
demic difficulties. The listening score cut-off of L85 was also determined by the college as the minimum score for placement in the next
higher-level English course. The reading score cut-off of R51 was the minimum level for this first-year English course; a score lower
than R51 denotes low reading proficiency and indicates remedial support may be needed. While students enrolled in this course were
identified to have some weakness in academic language skills, students with a combined listening and reading score below L85 and
R51 faced greater challenges and needed additional support.

4. Results

Data distributions. The Shapiro-Wilk test results indicate normal distributions for the control group’s pre-test scores in writing (p =
0.19), reading (p = 0.30), vocabulary (p = 0.77), and spelling (p = 0.34), as well as the treatment group’s pre-test scores in writing (p =
0.06), reading (p = 0.10), vocabulary (p = 0.35), and spelling (p = 0.30). Shapiro-Wilk tests indicate normal distributions in writing (p
= 0.33), reading (p = 0.16), vocabulary (p = 0.16), and spelling (p = 0.18) post-tests for the control group, and in vocabulary (p = 0.60)
and spelling (p = 0.47) post-tests for the treatment group. Non-normal distributions were detected for the treatment group’s post-test
scores in writing (p = 0.03) and reading (p = 0.01). However, skewness and kurtosis show that the data is not far from being normally
distributed. Skew for the treatment group’s post-writing scores was 0.16 (-1 to 1 is acceptable for normal distribution) and kurtosis was

6
J. Li and L. Mak System 105 (2022) 102712

-1.33 (-3 to 3 is acceptable for normal distribution). The treatment group’s post-reading scores had a skew of 0.92 and kurtosis of -0.44.
Therefore, both t-tests and ANOVA are considered adequate for analyzing the present data (Schmider et al., 2010).
Comparability tests between control and treatment groups and subgroups. To establish comparability before treatment, pre-test scores
between control and treatment groups were analyzed using independent t-tests and descriptive analyses for writing, reading, vo­
cabulary, and spelling. The results deem that the two groups in all four language skills were comparable before the treatment (see
Table 1). The minor variations in sample size in discrete skill tests were due to absent participants when the test was administered. An
outlier was removed from the control group’s writing pre-test data for submitting a blank writing test.
Comparability was also tested for subgroups at the same proficiency levels, that is, between the higher proficiency control and
treatment subgroups, and between the lower proficiency of the control and treatment subgroups using one-way ANOVA and post-hoc
tests (see Table 2). None of the differences were significant between the control and treatment subgroups at each proficiency level in
any of the language skills.
To answer Research Questions 1 and 2, the results from independent t-tests showed no significant differences in the post-test scores
between the control and treatment groups in any of the four language skills (see Table 3). The results of one-way ANOVA of post-test
scores between the control and treatment higher and lower proficiency subgroups also found no significant differences (see Table 4).
To further explore the effect of the intervention, independent t-tests and one-way ANOVA were applied to compare mean differ­
ences of pre- and post-test scores of four language skills within control and treatment groups and subgroups. First, independent t-tests
were conducted on the pre- and post-test scores for the control and treatment groups. Significant differences were found in both
treatment group’s writing performances between pre-test (M = 51.9, SD = 11.0) and post-test [(M = 67.8, SD = 8.4), t = -6.28, p <
0.001] and control group’s writing performances between pre-test (M = 54.5, SD = 8.2) and post-test [(M = 66.7, SD = 11.3), t = -5.03,
p < 0.001]. Results for reading, vocabulary, and spelling were nonsignificant for both control and treatment groups (see Table 5).
One-way ANOVA and post-hoc results revealed a significant difference between writing pre- (M = 55.5, SD = 8.80) and post-test
[(M = 69.7, SD = 10.90), p < 0.001] in the control HP subgroup. A significant difference was also found for the treatment HP subgroup
between writing pre- (M = 56, SD = 11.4) and post-test [(M = 71.3, SD = 7.8), p < 0.001]. There were no significant results in reading,
vocabulary, or spelling scores in the HP sample of the control or treatment groups.
When comparing pre- and post-test writing scores of the control and treatment LP subgroups, the former showed no significant
difference between pre- (M = 51.3, SD = 5.2) and post-test scores [(M = 58.6, SD = 8.1), p = 0.70] whereas there was a significant
difference in the treatment LP subgroup between pre- (M = 46.1, SD = 7.3) and post-test scores [(M = 60.3, SD = 3.4), p = 0.01]. No
significant differences were found in either the LP control or treatment subgroups in reading, vocabulary, or spelling test scores (see
Table 6).
To answer Research Question 3 regarding correlations between these first-year college students’ reading skills and performance of
their expository writing, Pearson correlation tests were applied to all participants’ scores of the writing tests, the Nelson-Denny
Reading Test and WIAT-III spelling test, combining pre- and post-tests. The results indicated that the correlations between their
four skills were all statistically significant with effect sizes ranging from small to large: reading-writing (r = 0.41, p < 0.001),
vocabulary-writing (r = 0.45, p < 0.001), spelling-writing (r = 0.25, p = 0.01), vocabulary-reading (r = 0.58, p < 0.001), spelling-
reading (r = 0.31, p < 0.001), and spelling-vocabulary (r = 0.53, p < 0.001) (see Table 7).

5. Discussions

5.1. Role of reading-to-write collaboration and technology scaffolding and their effect

First, although the results showed no statistical differences in performance between students who wrote collaboratively with and
without the technology scaffolding, the treatment subgroups outperformed the control subgroups in writing, evidenced by the mean
differences (MD) between pre- and post-test writing scores [HP MDs = 15.3 (treatment) versus 14.2 (control); LP MDs = 14.2
(treatment) versus 7.3 (control)]. Second, the present results indicate that except for students in the control subgroup with lower
proficiency level, students in both control and treatment subgroups with higher proficiency level and students in the treatment
subgroup with lower proficiency level had significant improvement in their writing performance compared to their own pre-test scores
in writing. These results mostly agree with previous research findings that indicated collaborative writing with additional opportu­
nities to interact with peers led to better quality writing (Yanguas, 2020), as well as demonstrated the advantages of collaborative
writing over individual writing using the same technology tools (Bikowski & Vithanage, 2016), and collaborative writing with

Table 1
Comparison of pre-test results for all participants in control and treatment groups.
Skill Population Group N M SD t p

Writing Control 33 54.5 8.2 1.08 0.28

Treatment 34 51.9 11.0
Reading Control 33 15.0 6.6 -0.75 0.46
Treatment 35 16.1 6.0
Vocabulary Control 34 36.2 13.3 0.87 0.39
Treatment 35 33.6 11.4
Spelling Control 34 26.1 8.5 1.91 0.06
Treatment 35 22.1 8.7

7
J. Li and L. Mak System 105 (2022) 102712

Table 2
Comparison of pre-test scores for control and treatment subgroups.
Skills Groups N M SD p

Writing Control HP 25 55.5 8.8 1.00

Treatment HP 20 56.0 11.4
Control LP 8 51.3 5.2 0.90
Treatment LP 14 46.1 7.3
Reading Control HP 25 15.6 6.8 0.91
Treatment HP 21 18.0 5.1
Control LP 8 13.3 5.8 1.00
Treatment LP 14 13.4 6.4
Vocabulary Control HP 25 40.0 11.6 1.00
Treatment HP 21 39.0 9.4
Control LP 9 25.8 12.8 1.00
Treatment LP 14 25.5 9.3
Spelling Control HP 25 27.4 8.1 1.00
Treatment HP 21 26.0 7.5
Control LP 9 22.4 8.9 0.66
Treatment LP 14 16.4 7.4

Table 3
Comparison of post-test scores between control and treatment groups.
Skills Groups N M SD t p

Writing Control 33 66.7 11.3 -0.44 0.66

Treatment 25 67.8 8.4
Reading Control 26 18.5 6.9 0.57 0.57
Treatment 22 17.2 8.0
Vocabulary Control 28 38.7 13.0 0.36 0.72
Treatment 23 37.4 14.0
Spelling Control 25 27.1 8.3 0.98 0.33
Treatment 22 24.4 10.4

Table 4
Comparison of post-test scores between control and treatment subgroups.
Skills Groups N M SD p

Writing Control HP 24 69.7 10.9 1.00

Treatment HP 17 71.3 7.8
Control LP 9 58.6 8.1 1.00
Treatment LP 8 60.3 3.4
Reading Control HP 17 20.7 7.0 1.00
Treatment HP 14 20.3 8.5
Control LP 9 14.3 4.5 0.99
Treatment LP 8 11.9 2.7
Vocabulary Control HP 19 42.7 12.8 1.00
Treatment HP 15 43.3 11.1
Control LP 9 30.3 9.6 0.99
Treatment LP 8 26.1 12.3
Spelling Control HP 17 28.8 8.6 1.00
Treatment HP 14 28.7 8.0
Control LP 8 23.4 6.8 0.73
Treatment LP 8 16.8 10.1

technology support over those with less and no technology support (Liu et al., 2018; Passig & Maidel-Kravetsky, 2016; Teng, 2021).
The nonsignificant differences between treatment and control groups of this study may be attributed to the fact that participants in
both conditions used the same curriculum materials and engaged in consistent, collaborative writing practice either using OneNote, or
face-to-face interaction in the classroom that has been found in other studies to improve writing quality (Storch, 2005; Storch &
Wigglesworth, 2007). In addition, all students benefited from structured instruction with other learning activities scaffolded by the
instructor. These results are in line with previous observations and results indicating that students who wrote collaboratively
significantly improved in writing performance with and without technology scaffolding (MacArthur et al., 2015; Shehadeh, 2011; Yeh,
2014).
Third, the results are nonsignificant in reading, vocabulary, and spelling scores between treatment and control groups and from
pre- to post-test scores after students took the course using or not using the OneNote-supported intervention. A few reasons might be
plausible to explain these results. First, although reading tasks were integrated with writing practice in the present intervention, to

8
J. Li and L. Mak System 105 (2022) 102712

Table 5
Comparison of pre- and post-test scores for control and treatment groups.
Skills Groups Time N M SD t p

Writing Control pre 33 54.5 8.2 -5.03 0.00**

post 33 66.7 11.3
Treatment pre 34 51.9 11.0 -6.28 0.00**
post 25 67.8 8.4
Reading Control pre 33 15.0 6.6 -1.95 0.06
post 26 18.5 6.9
Treatment pre 35 16.1 6.0 -0.55 0.59
post 22 17.2 8.0
Vocabulary Control pre 34 36.2 13.3 -0.75 0.46
post 28 38.7 13.0
Treatment pre 35 33.6 11.4 -1.07 0.29
post 23 37.4 14.0
Spelling Control pre 34 26.1 8.5 -0.45 0.66
post 25 27.1 8.3
Treatment pre 35 22.1 8.7 -0.84 0.41
post 22 24.4 10.4

Note: **p < 0.001.

Table 6
Comparison of pre- and post-test results between control and treatment subgroups.
Skills Groups Time N M SD p

Writing Control HP pre 25 55.5 8.8 0.00**

post 24 69.7 10.9
Treatment HP pre 20 56.0 11.4 0.00**
post 17 71.3 7.8
Control LP pre 8 51.3 5.2 0.70
post 9 58.6 8.1
Treatment LP pre 14 46.1 7.3 0.01*
post 8 60.3 3.4
Reading Control HP pre 25 15.6 6.8 0.19
post 17 20.7 7.0
Treatment HP pre 21 18.0 5.1 0.96
post 14 20.3 8.5
Control LP pre 8 13.3 5.8 1.00
post 9 14.3 4.5
Treatment LP pre 14 13.4 6.4 1.00
post 8 11.9 2.7
Vocabulary Control HP pre 25 40.0 11.6 0.99
post 19 42.7 12.8
Treatment HP pre 21 39.0 9.4 0.94
post 15 43.3 11.1
Control LP pre 9 25.8 12.8 0.99
post 9 30.3 9.6
Treatment LP pre 14 25.5 9.3 1.00
post 8 26.1 12.3
Spelling Control HP pre 25 27.4 8.1 1.00
post 17 28.8 8.6
Treatment HP pre 21 26.0 7.5 0.98
post 14 28.7 8.0
Control LP pre 9 22.4 8.9 1.00
post 8 23.4 6.8
Treatment LP pre 14 16.4 7.4 1.00
post 8 16.8 10.1

Notes: *p < 0.05; **p < 0.001.

students this was a course for writing, so they tended to focus on writing, not other aspects of language skills. Additionally, we did not
explicitly teach vocabulary and spelling, although students in the treatment group had access to the online look-up feature with which
they could check word spelling, pronunciation, definitions, sample phrases, and synonyms from online dictionaries and thesaurus. The
instructor also emphasized the importance of vocabulary and spelling in class, and all students were prompted to check grammar and
proofread during the collaborative activities. Secondly, teamwork may not lead to productive outcomes for learning some language
skills. Shehadeh (2011) suggested that collaboration may be less beneficial to students acquiring more straightforward skills that could
be learned independently, such as spelling. According to our observation, this also applies to learning basic and discrete language
knowledge, including vocabulary to a certain degree. However, we also believe group practice can effectively enhance students’

9
J. Li and L. Mak System 105 (2022) 102712

Table 7
Pearson correlations among language skills based on combined pre- and post-test scores.
Skills Writing Reading Vocabulary Spelling

Writing 1.00
Reading 0.41** 1.00
Vocabulary 0.45** 0.58** 1.00
Spelling 0.25* 0.31** 0.53** 1.00

Notes: Writing/reading correlation (n = 115), writing/vocabulary correlation (n = 119), writing/spelling correlation (n = 115), reading/vocabulary
correlation (n = 116), reading/spelling correlation (n = 112), vocabulary/spelling correlation (n = 116). *p < 0.05; **p < 0.001.

understanding of pragmatics, such as appropriate use of words in writing. The other factor concerns students’ priority on certain
language aspects during collaborative activities. Kessler et al. (2012) noted that students seemingly overlooked errors they considered
less important during collaborative writing although they could accurately make the corrections, such as vocabulary and spelling.

5.2. Variations in intervention effect by language proficiency levels

It is interesting to discuss the present results in relation to the students’ proficiency level from several aspects. First, an encouraging
finding is that students with lower listening and reading skills had statistically significant improvement in writing after the inter­
vention, while their counterparts in the control group did not. This result is in line with Bikowski and Vithanage’s (2016) study
showing that technology-supported collaborative writing may be “particularly beneficial” for lower proficiency students. Specifically,
they found that the collaborative writing group who started with “considerably lower” writing scores gained scores comparable to
those of the individual writing group by the end of the semester. Lower proficiency subgroup students in the present study, who had a
listening score less than 85 (advanced low proficiency) and a reading score less than 51 (low-intermediate proficiency), faced greater
challenges and needed additional support. They not only lacked cognitive academic language proficiency such as reading compre­
hension skills, but also had limited basic communication skills. Thus, it is highly likely that these students encountered difficulty in
understanding the readings assigned for writing tasks, as well as the lectures and assignment instruction in the classroom. The
completion of source-based writing tasks online collaboratively might have provided these students more opportunities for instant,
explicit clarification for assignment requirements and reading comprehension support. More importantly, as per their feedback
through interviews, these students particularly appreciated a few OneNote features, including easy access to online learning resources,
e.g., readings, assignments and peer feedback. In particular they could view other members’ writing within their group and across
groups in OneNote, thus, refer to several samples of the same writing activity (see Li et al., submitted, for the full report of students’
feedback on the intervention). It is plausible the visibility of multiple solutions along with other features offered the LP students with
much needed support, leading to significantly better quality of writing. To conclude, the technology platform for this study can better
support students with lower proficiency levels in collaborative writing than face-to-face interaction.
Secondly, there are further interpretations of the subgroups’ results by proficiency levels. On one hand, the results support the
findings from studies conducted in traditional classroom settings, suggesting that compared to their higher proficiency counterparts,
lower-proficiency students do not benefit fully from collaboration (Shehadeh, 2011; Storch, 1998), likely leading to less learning gains
in some aspects of language knowledge and skills. Since knowledge-sharing is selective during collaborative writing practice, team
members with weaker language skills may be unable to share or construct knowledge effectively in traditional collaborative writing
contexts (Vorobel & Kim, 2017). In the present study, students with higher proficiency had a larger gain than their peers with lower
proficiency in the control group [MDs = 14.2 (HP control) versus 7.3 (LP control)] as well as in the treatment group [MDs = 15.3 (HP
treatment) versus 14.2 (LP treatment) when writing collaboratively. On the other hand, compared to the control subgroups, the gap of
the gains from pre- to post-test writing scores between the treatment lower and higher proficiency subgroups was much smaller (1.1
versus 6.9). Given that students with limited language skills in listening and reading had an average writing score gain similar to that of
their peers with higher proficiency, again it is possible that the intervention may have created a more encouraging online collaboration
environment for these students, adjusting their learning pace and help-seeking that they may have otherwise hesitated to do in the
face-to-face settings.
Furthermore, another explanation for the significant gains of the LP treatment subgroup is that online collaboration of students
with mixed language proficiency levels may have facilitated a greater number of higher-quality interactions within groups. In this
study, small groups comprised higher and lower proficiency students in both treatment and control settings. Often in group work, HP
students tend to be more dominant, while LP students may be more reserved. It is feasible that the use of the online tool supported
lower proficiency students’ interactions better with higher proficiency students than their counterparts in control group who
collaborated face-to-face. Since cloud-based features of OneNote permitted several students to work on the same page synchronously,
they read each other’s contributions during collaborative writing activities and asked for clarification when required. Students could
even listen to recorded messages to communicate. These may have made it easier for lower proficiency students, often also less vocal
members, to contribute in a work environment where potential face-to-face interaction conflict can be minimized. In the treatment
condition, we noticed that LP students who often were reluctant to speak out in class actively participated in group activities online and
interacted with HP students through written communication. LP students in control group communicated mainly face-to-face, where
they might not be able to comprehend and react to their HP team members’ writing immediately. In turn, they may have difficulty
remembering the writing process as well as their peers in treatment group. Therefore, the online intervention may have benefited

10
J. Li and L. Mak System 105 (2022) 102712

lower proficiency students, leading to increased time-on-task, and higher quality revisions integrating the feedback. Research points
out that groups who shared ideas and participated equally demonstrated better writing than groups with uneven contributions or
dominant behaviours. Li and Zhu (2017) studied interactions of four groups using a wiki outside of class to collaboratively write a
research proposal, finding the “collective” group wrote a higher quality proposal, especially in structure and coherence.
Lastly, different instructor feedback delivery—written versus oral—might have played in a role in students’ learning experiences
and outcomes. LP students in treatment group who received primarily instructor’s written feedback on their collaborative writing in
OneNote may have better understood and recalled the feedback to improve their work than LP students in control group who received
primarily the oral feedback.
A couple of reasons may explain the insignificant gain of higher proficiency students in the treatment group. One limitation of the
intervention which used an earlier version of OneNote was that revision history tracking writing changes was not available. Color
coding to differentiate authors was not automatic with the last author’s editing superseding previously typed text. Otherwise, it would
have allowed students, in particular higher proficiency students to analyze and discuss different versions of writing and decision-
making on their best version. Discussions on these aspects are critical for developing writing content and skills. As a group,
everyone could readily see how their writing was modified by peers, thus enhancing their learning. Lastly, lower proficiency students,
having underdeveloped language learning strategies, availed themselves fully of the online collaborative resources, while higher
proficiency students continued with their already established strategies and needed more advanced scaffolds that were not provided in
the present intervention. We believe limitations as such disadvantaged higher proficiency students and led to insignificant differences
in their learning gains between (control versus treatment) and within (pre- and post-tests) the HP subgroups.

5.3. Writing-reading correlations by language proficiency levels and context

The present results support previous studies that found correlations between college students’ reading and writing skills, and their
spelling and vocabulary knowledge, but with smaller effect sizes, ranging from 0.25 to 0.58 rather than 0.57 to 0.79 in Parodi’s (2007)
and 0.55 to 0.79 in Allen et al.’s (2014) study. This may be related to the overall lower language skills of our participants than those of
participants in other studies, as exemplified by Perin et al. (2017) who found an overall small-to-low medium correlation between
reading and several academic writing genres (0.14-0.31) among college students with low proficiency levels. The results, however, are
somewhat different from Schoonen’s (2019) results with high school Dutch students that found that reading-writing correlations were
0.45-0.77 for L1 and 0.36 to 0.81 for EFL. The author suggests that there are stronger correlations between reading ability and writing
performance in early years of language development, as linguistic knowledge plays a more important role in reading and writing skills
than later. We believe that participant age and language context may be accountable for the differences. College participants in the
present study, who self-identify as English speakers but many of whom with limited listening skills, may be able to draw more
non-linguistic and metacognitive knowledge than EFL high school students, such as those in Schoonen’s study. These findings will help
decision-making for intervention timing, strategies, and student populations with specific levels of language skills that can optimally
benefit innovative reading to writing instruction.
Furthermore, we found that the strongest correlation was between students’ reading comprehension and vocabulary knowledge (r
= 0.58), and the latter was more correlated with their reading comprehension than writing performance. These results are in
agreement with Allen et al.’s (2014) finding with the strongest correlation between vocabulary and reading (r = 0.79) and Schoonen’s
(2019) results indicating that for both L1 and EFL, vocabulary was more correlated with reading ability than writing performance,
except for L1 tenth graders. Therefore, it is important for integrated literacy tasks that aim at enhancing students’ academic writing to
provide vocabulary and reading comprehension instruction at the same time, particularly for students with limited proficiency skills.

6. Conclusions

The interviews with students have shown that the intervention was well-received with enhanced motivation observed by the
instructor; most students described the experience as “fun.” The students preferred learning activities using OneNote over the tradi­
tional face-to-face setting, and particularly enjoyed the social interaction and instantaneous feedback from peers and the instructor
(see Li et al., submitted). OneNote’s features in making multiple collaborative writing samples visible to all students in and out of class
provided students different perspectives to support their comprehension and writing development, particularly for lower proficiency
students needing additional support. To our knowledge, this intervention study may be the first that developed a structured curriculum
in an attempt to improve college students’ academic writing skills by focusing on teaching and practice of key aspects of source-based
expository writing skills in an online collaborative environment. The significant improvement in writing performance of students with
lower proficiency skills resulting from the intervention demonstrates great potential for methodologically bridging the gaps of teaching
college students academic writing skills by developing technology-supported instructional designs.
Several limitations of the present study warrant future research. First, there were limited and uneven number of participants in
lower proficiency control and treatment subgroups [8-9 (8 in reading and writing measures, 9 in vocabulary and spelling measures)
versus 14, respectively]. Over the course of the study, nine students from the treatment group and one student from the control group
dropped the course, therefore attrition occurred that may have caused slight deviations from a normal data distribution observed.
Second, the division of the course sections into control and treatment groups resulted in an unequal gender distribution; the control
group had 12 males and 22 females, and the treatment group had 21 males and 14 females. As the instructor observed that female
students seemed to be more organized with their studies in the course, this may also have an impact on the results. The behaviours and
attitudes of group members may have also influenced the quality of collaboration; group members who are more collaborative tend to

11
J. Li and L. Mak System 105 (2022) 102712

have better quality writing. For example, the teams that encouraged constructive feedback and had a receptive attitude to instructor’s
feedback improved their work through more frequent revisions. Also as noted above, with the intervention students with lower
proficiency levels were observed to be more engaging and having more interactions on tasks with their peers with higher proficiency
levels. Thus, it would be valuable to develop measures to capture students’ online collaborative behavior patterns, their perception on
the quantity and quality of each member’s contribution and analyze the correlations between these factors and their learning out­
comes. Third, since the intervention took place only for one semester, this may have led to nonsignificant results for most measures
between the control and treatment groups. Future research will be needed with a larger sample size and even gender distribution to
document student progress through longer intervention durations, such as writing programs for one to two years. Fourth, as the present
intervention effect varied greatly between different proficiency levels, a “customizable” approach to developing future writing in­
terventions would be of great value (Centre for Excellence in Universal Design-CEUD, 2021). This includes using artificial intelligence
technology to develop dynamic, adaptive instructional strategies that can be effective for diverse students in intact classes.
Fifth, vocabulary and spelling were not explicitly taught as part of the intervention since our aim was to focus on improving the
complex skills used in expository writing e.g. paraphrasing, summarizing and organizing that many students find more challenging.
Research on the types of peer feedback (e.g., spelling, vocabulary versus content and structure) during technology-assisted collabo­
rative writing, as well as the impact of explicit instruction of spelling and vocabulary on the expository writing of students with
different proficiency levels will be required. Moreover, it will be worthwhile to examine the impact of the group formation/compo­
sition on students’ writing performance and experience, for example, group cohesion and interaction patterns by strategically orga­
nizing students with different strengths to collaborate on reading-to-write tasks. Another limitation that may have impacted the
current intervention outcomes was the unstable Internet connection at the college. This was one of the major complaints reported by
the students during the interviews, indicating the poor Internet connection interrupted their group work using OneNote. This is in line
with previous studies that concluded that participants’ familiarity with technology, connectivity, and the way that technology fa­
cilitates the completion of a task, play a critical role in their experiences (Li & Zhu, 2013; Liu & Lan, 2016; Matthew et al., 2009). While
we provided students tutorials with tutorials on OneNote features for collaborative reading-to-write tasks, and weekly training in using
specific features for specific collaborative tasks, some did not always use all of them. Most students in the present study used basic
features of OneNote to complete their tasks. Had we had more time for the project, its other features, e.g., the Immersive Reader that
highlights parts of speech and the Smart Lookup that enables an easily accessible dictionary, could have contributed to students’
writing performance. Therefore, future intervention studies may benefit from more explicit instruction, and student training and
monitoring on their use of a broader spectrum of online collaborative tools.
Nonetheless, the present results indicated that college students with underdeveloped language skills can benefit from a well-
structured, online intervention that supports their collaborative learning of academic writing in a step-by-step manner. This allows
instructors to adapt their teaching methods, effectively use their limited class time, and also leverage peer support and technology as
scaffolds in the learning process to improve these students’ writing skills to meet the academic demands of post-secondary education.

CRediT author statement

Jia Li: Conceptualization, Methodology (Research Procedure, Instruments Development, Intervention Design), Data Collection and
Coding (Coaching and Guidance), Validation, Formal Analysis, Visualization, (Most) Writing - Original Draft, (Most) Writing - Review
& Editing, Supervision, Project administration. Lillian Mak: Methodology (Instruments Development, Intervention Design), Data
Collection and Coding, Formal analysis, Visualization, (Section) Writing - Original Draft, (Assisting) Writing - Review & Editing.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.system.2021.102712.

References

Allen, L. K., Snow, E. L., Crossley, S. A., Jackson, G. T., & McNamara, D. (2014). Reading comprehension components and their relation to writing. Annee
Psychologique, 114(4), 663–691.
Alsubaie, J., & Ashuraidah, A. (2017). Exploring writing individually and collaboratively using Google docs in EFL contexts. English Language Teaching, 10(10), 10–30.
https://doi.org/10.5539/elt.v10n10p10
Bartolomeo-Maida, M. (2016). The use of learning journals to foster textbook reading in the community college psychology class. College Student Journal, 50(3),
440–453.
Beck, S. W., Llosa, L., & Fredrick, T. (2013). The challenges of writing exposition: Lessons from a study of ELL and non-ELL high school students. Reading & Writing
Quarterly, 29, 358–380. https://doi.org/10.1080/10573569.2013.758938
Bikowski, D., & Vithanage, R. (2016). Effects of web-based Collaborative writing on individual L2 writing development. Language, Learning and Technology, 20(1),
79–99. Retrieved from http://llt.msu.edu/issues/february2016/bikowskivithanage.pdf.
Brockman, E., Taylor, M., Kreth, M., & Crawford, M. K. (2011). What do professors really say about college writing? English Journal, 100(3), 75–81.
Brown, J. I., Fishco, V. V., & Hanna, G. (1993). Nelson-Denny reading test: Manual for scoring and interpretation, forms G & H. The Riverside Publishing Company.
Centre for Excellence in Universal Design. (2021). The 7 principles of universal design. Retrieved from https://universaldesign.ie/what-is-universal-design/the-7-
principles/.
Chao, Y. J., & Lo, H. (2011). Students’ perceptions of wiki-based collaborative writing for learners of English as a foreign language. Interactive Learning Environments,
19(4), 395–411. https://doi.org/10.1080/10494820903298662

12
J. Li and L. Mak System 105 (2022) 102712

College Board. (2020). Accuplacer. https://accuplacer.collegeboard.org/.

Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for
next generation TOEFL. Assessing Writing, 10, 5–43. https://doi.org/10.1016/j.asw.2005.02.001
Cummins, J. (2008). BICS and CALP: Empirical and theoretical status of the distinction. In B. Street, & N. H. Hornberger (Eds.), Encyclopedia of language and education
(2nd ed., vol. 2, pp. 71–83). New York, NY: Springer. Literacy.
Ducate, L. C., Anderson, L. L., & Moreno, N. (2011). Wading through the world of wikis: An analysis of three wiki projects. Foreign Language Annals, 44(3), 495–524.
https://doi.org/10.1111/j.1944-9720.2011.01144.x
Elola, I., & Oskoz, A. (2010). Collaborative writing: Fostering foreign language and writing conventions development. Language, Learning and Technology, 14(3),
51–71.
Fitzgerald, M. S., & Palincsar, A. (2017). Peer-mediated reading and writing in a digital, multimodal environment. Reading & Writing Quarterly, 33(4), 309–326.
https://doi.org/10.1080/10573569.2017.1294514
Grabe, W., & Zhang, C. (2013). Reading and writing together: A critical component of English for academic purposes teaching and learning. TESOL Journal, 4(1), 9–24.
https://doi.org/10.1002/tesj.65
Gruenbaum, E. A. (2012). Common literacy struggles with college students: Using the reciprocal teaching technique. Journal of College Reading and Learning, 42(2),
110–116. https://doi.org/10.1080/10790195.2012.10850357
Kathpalia, S., & See, E. (2016). Improving argumentation through student blogs. System, 58, 25–36. https://doi.org/10.1016/j.system.2016.03.002
Kessler, G., Bikowski, D., & Boggs, J. (2012). Collaborative writing among second language learners in academic web-based projects. Language, Learning and
Technology, 16(1), 91–109.
Li, J., Mak, L., Hunter, W., & Cunningham, T. (submitted). Structured instructional design for integrated language skill development: College students’ perspectives on
collaborative reading-to-write activities on a cloud-based platform.
Lipschutz, G., Scarry, S., & Scarry, J. (2017). The Canadian Writer’s Workplace 8ed. Toronto, ON: Nelson.
Liu, S. H., & Lan, Y. J. (2016). Social constructivist approach to web-based EFL learning: Collaboration, motivation, and perception on the use of Google docs.
Educational Technology & Society, 19(1), 171–186.
Liu, M., Liu, L., & Liu, L. (2018). Group awareness increases student engagement in online collaborative writing. The Internet and Higher Education, 38, 1–8. https://doi.
org/10.1016/j.iheduc.2018.04.001
Li, M., & Zhu, W. (2013). Patterns of computer-mediated interaction in small writing groups using wikis. Computer Assisted Language Learning, 26(1), 61–82. https://
doi.org/10.1080/09588221.2011.631142
Li, M., & Zhu, W. (2017). Good or bad collaborative wiki writing: Exploring links between group interactions and writing products. Journal of Second Language Writing,
35(2017), 38–53.
Loretto, A., DeMartino, S., & Godley, A. (2016). Secondary students’ perceptions of peer review of writing. Research in the Teaching of English, 51(2), 134–161.
Maaka, M. J., & Ward, S. M. (2000). Content area reading in community college classrooms. Community College Journal of Research and Practice, 24(2), 107–125.
https://doi.org/10.1080/106689200264240
MacArthur, C. A., & Philippakos, Z. A. (2013). Self-regulated strategy instruction in developmental writing: A design research project. Community College Review, 41
(2), 176–195. https://doi.org/10.1177/0091552113484580
MacArthur, C. A., Philippakos, Z. A., & Ianetta, M. (2015). Self-regulated strategy instruction in college developmental writing. Journal of Educational Psychology, 107
(3), 855–867. https://doi.org/10.1037/edu0000011
Matthew, K. I., Felvegi, E., & Callaway, R. A. (2009). Wiki as a collaborative learning tool in a language arts methods class. Journal of Research on Technology in
Education, 42(1), 51–72. https://doi.org/10.1080/15391523.2009.10782541
Miyazoe, T., & Anderson, T. (2010). Learning outcomes and students’ perceptions of online writing: Simultaneous implementation of a forum, blog, and wiki in an EFL
blended learning setting. System, 38(2), 185–199. https://doi.org/10.1016/j.system.2010.03.006
Parodi, G. (2007). Reading–writing connections: Discourse-oriented research. Reading and Writing, 20, 225–250. https://doi.org/10.1007/s11145-006-9029-7
Passig, D., & Maidel-Kravetsky, J. (2016). The impact of collaborative online reading on summarizing skills. Education and Information Technologies, 21(3), 531–543.
https://doi.org/10.1007/s10639-014-9337-5
Perin, D., Lauterbach, M., Raufman, J., & Kalamkarian, H. S. (2017). Text-based writing of low-skilled postsecondary students: Relation to comprehension, self-
efficacy and teacher judgments. Reading and Writing, 30(4), 887–915. https://doi.org/10.1007/s11145-016-9706-0
Reynolds, B. L., & Anderson, T. A. (2015). Extra-dimensional in-class communications: Action research exploring text chat support of face-to-face writing. Computers
and Composition, 35, 52–64. https://doi.org/10.1016/j.compcom.2014.12.002
Schleppegrell, M. J. (2001). Linguistic features of the language of schooling. Linguistics and Education, 12(4), 431–459. https://doi.org/10.1016/S0898-5898(01)
00073-0
Schmider, E., Ziegler, M., Danay, E., Beyer, L., & Bühner, M. (2010). Is it really robust? Reinvestigating the robustness of ANOVA against violations of the normal
distribution assumption. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 6(4), 147–151.
Schoonen, R. (2019). Are reading and writing building on the same skills? The relationship between reading and writing in L1 and EFL. Reading and Writing, 32(3),
511–535. https://doi.org/10.1007/s11145-018-9874-1
Shehadeh, A. (2011). Effects and student perceptions of collaborative writing in L2. Journal of Second Language Writing, 20(4), 286–305. https://doi.org/10.1016/j.
jslw.2011.05.010
Shen, F. (2013). Using group discussion with Taiwan’s EFL college students: A comparison of comprehension instruction for book club, literature circles, and
instructional conversations. English Language Teaching, 6(12), 58–78. https://doi.org/10.5539/elt.v6n12p58
Sotillo, S. M. (2002). Constructivist and collaborative learning in a wireless environment. TESOL Journal, 11(3), 16–20. https://doi.org/10.1002/j.1949-3533.2002.
tb00091.x
Storch, N. (1998). A classroom-based study: Insights from a collaborative text reconstruction task. ELT Journal, 52(4), 291–300. https://doi.org/10.1093/elt/52.4.291
Storch, N. (2005). Collaborative writing: Product, process, and students’ reflections. Journal of Second Language Writing, 14(3), 153–173. https://doi.org/10.1016/j.
jslw.2005.05.002
Storch, N., & Wigglesworth, G. (2007). Writing Tasks: The effect of collaboration. In M. Mayo (Ed.), Investigating tasks in formal language learning (pp. 157–177).
Multilingual Matters.
Suwantarathip, O., & Wichadee, S. (2014). The effects of collaborative writing activity using Google docs on students’ writing abilities. The Turkish Online Journal of
Educational Technology, 13(2), 148–156.
Tai, H. (2016). Effects of collaborative online learning on EFL learners’ writing performance and self-efficacy. English Language Teaching, 9(5), 119–133. https://doi.
org/10.5539/elt.v9n5p119
Taraban, R., Rynearson, K., & Kerr, M. (2000). College students’ academic performance and self-reports of comprehension strategy use. Reading Psychology, 21(4),
283–308. https://doi.org/10.1080/027027100750061930
Teng, M. (2021). Interactive-whiteboard-technology-supported collaborative writing: Writing achievement, metacognitive activities, and co-regulation patterns.
System, 97. https://doi.org/10.1016/j.system.2020.102426
Tomasek, T. (2009). Critical reading: Using reading prompts to promote active engagement with text. International Journal of Teaching and Learning in Higher Education,
21(1), 127–132.
Vorobel, O., & Kim, D. (2017). Adolescent ELLs’ collaborative writing practices in face-to-face and online contexts: From perceptions to action. System, 65, 78–89.
https://doi.org/10.1016/j.system.2017.01.008
Wang, D. (2009). Factors affecting the comprehension of global and local main idea. Journal of College Reading and Learning, 39(2), 34–52. https://doi.org/10.1080/
10790195.2009.10850317

13
J. Li and L. Mak System 105 (2022) 102712

Wang, Y. (2014). Using wikis to facilitate interaction and collaboration among EFL learners: A social constructivist approach to language teaching. System, 42,
383–390. https://doi.org/10.1016/j.system.2014.01.007
Wechsler, D. (2009). Wechsler Individual Achievement Test (3rd ed.). San Antonio, TX: Psychological Corporation.
Wigglesworth, G., & Storch, N. (2012). What role for collaboration in writing and writing feedback. Journal of Second Language Writing, 21(4), 364–374. https://doi.
org/10.1016/j.jslw.2012.09.005
Yanguas, I. (2020). L1 vs L2 synchronous text-based interaction in computer-mediated L2 writing. System, 88, 102169. https://doi.org/10.1016/j.
system.2019.102169
Yeh, H.-C. (2014). Exploring how collaborative dialogues facilitate synchronous collaborative writing. Language, Learning and Technology, 18(1), 23–37.
Zheng, B., Lawrence, J., Warschauer, M., & Lin, C.-H. (2015). Middle school students’ writing and feedback in a cloud-based classroom environment. Technology,
Knowledge and Learning, 20(2), 201–229. https://doi.org/10.1007/s10758-014-9239

Jia Li is an Associate Professor at the Faculty of Education at Ontario Tech University, and a Canada-U.S. Fulbright Scholar at the Harvard Graduate School of Education
(2011–2012) and John A. Sproul Research Fellow at the University of California, Berkeley (2018–2019). Her research focuses on data-driven innovative language and
literacy instruction using emerging technologies for linguistically diverse students. Li has published with journals including Computers & Education, Computers in Human
Behavior, Language Learning and Technology, Computer Assisted Language Learning, Canadian Modern Language Review.

Lillian Mak is a writing instructor at Centennial College, Toronto. Her research interests focus on academic reading and writing skills for college students, specifically
the reasoning skills for expository essays to bridge students’ writing skills from high school to post-secondary studies.

14