Electronic Outline

Computers & Education 78 (2014) 352e366
Contents lists available at ScienceDirect
Computers & Education

journal homepage: www.elsevier.com/locate/compedu
Electronic outlining as a writing strategy: Effects on students' writing

products, mental effort and writing process
Milou J.R. de Smet a, b, *, Saskia Brand-Gruwel a, Marie
€lle Leijten b, c, Paul A. Kirschner a
a
Open University in the Netherlands, Welten Institute, Research Centre for Learning, Teaching and Technology, P.O. Box 2960, 6401, DL Heerlen,
The Netherlands
b
University of Antwerp, Faculty of Applied Economics, Department of Management, Prinsstraat 13, 2000, Antwerp, Belgium
c
Research Foundation e Flanders (FWO), Belgium
a r t i c l e i n f o a b s t r a c t
Article history: This study addresses to what extent and how electronic outlining enhances students' writing perfor-
Received 28 September 2013 mance. To this end, the focus of this study is not only on students' final writing products but also on the
Received in revised form organisation of the writing process (i.e., planning, translating, and reviewing) and perceived mental
16 June 2014
effort during writing. In addition, effects of repeated electronic outlining were examined. A combined
Accepted 20 June 2014
Available online 5 July 2014
within and between subjects design was implemented in which 93 10th-grade students wrote two
argumentative texts with or without using electronic outlining. Analyses showed that using electronic
outlining for planning and writing significantly improved the presentation of the argumentative struc-
Keywords:
Secondary education ture. However, effects were less clear for correctly and completely establishing a text structure and no
Writing strategies effects were found on the elaboration of students' argumentation. Process data showed that electronic
Writing process outlining increased total process time, but no effect was found on students' overall planning and revision
Electronic outlining activities. Finally, self-reports showed no effect of electronic outlining on students' perceived mental
Mental effort effort. Nevertheless, repeated use of the same writing strategy enhanced writing fluency.
© 2014 Elsevier Ltd. All rights reserved.
1. Introduction
In the current knowledge-driven society, writing elaborated and coherent texts is an important skill for both one's educational and
professional career (U.S. Department of Education, 2003). The development of students' writing skills is therefore an explicitly formulated
educational goal. At the same time, writing is a complex and demanding skill to master (Flower & Hayes, 1980; Hayes, 1996; Kellogg, 1994).
The complexity of writing can be explained by the fact that writers must simultaneously perform a set of distinctive cognitive activities.
During composition, writers must simultaneously plan, translate, and review their text; they should consider a content problem of what to
write, and a rhetorical problem of how to express their ideas in a way that suits both the topic and the audience (Flower & Hayes, 1980;
Lindgren & Sullivan, 2005; Torrance, Thomas, & Robinson, 2000).
The alternation of these cognitive processes puts a heavy load on a writer's working memory. Empirical studies by Kellogg (1996) and
Baddeley (2010), have shown that this working memory has only limited capacity to process and store information. Writers, therefore, often
experience cognitive overload while composing a text and may be unable to adequately attend to any of these processes at all (Flower & Hayes,
1981; Kellogg, 1988). Such writing may be inefficient and might lead to poorly structured and/or incoherent texts. Kellogg (2008) showed that
the efficiency of writing is affected by expertise as certain processes become automated with expertise (i.e., they no longer require cognitive
processing). Moreover, expert writers make use of well-developed writing strategies to enhance their writing. Different studies have shown
that these writing strategies, in particular outlining, may improve students' writing products and decrease mental effort during writing (e.g.,
Erkens, Kanselaar, Prangsma, & Jaspers, 2002; Kellogg, 1988, 1990; Kozma, 1991). However, until now, not much is known about how these
effects are achieved. To understand how outlining exerts influence on writing, the three main components of the writing process e planning,
translating, and reviewing e serve as a starting point in this study. The purpose of this study is to examine the effects of electronic outlining on
students' organisation of the writing process and its influence on students' writing products and perceived mental effort.
* Corresponding author. University of Antwerp, Faculty of Applied Economics, Department of Management, Prinsstraat 13 (C.459), B2000, Antwerp, Belgium. Tel.: þ32 3
265 4116.
E-mail addresses: milou.desmet@uantwerpen.be, milou.desmet@ou.nl (M.J.R. de Smet), Saskia.Brand-Gruwel@ou.nl (S. Brand-Gruwel), Marielle.Leijten@uantwerpen.be
(M. Leijten), Paul.Kirschner@ou.nl (P.A. Kirschner).
http://dx.doi.org/10.1016/j.compedu.2014.06.010
0360-1315/© 2014 Elsevier Ltd. All rights reserved.
M.J.R. de Smet et al. / Computers & Education 78 (2014) 352e366 353
Traditionally, many studies have focussed on whether writing strategies work, but not on how they are used and how they work. Un-
derstanding how electronic outlining is used and how it affects the organisation of the writing process is an important first step which may
help to fill this gap and begin to provide a deeper understanding of how and why electronic outlining works. Understanding what works in
electronic outlining may subsequently help indicate where teachers can and should provide extra support in writing education. This is an
important step towards developing effective pedagogical instruction for using electronic outlining in education.
1.1. The writing process
Since the early 1980's, emphasis in writing research shifted from the writing product to the writing process, focussing mainly on the
cognitive processes involved in writing so as to unravel what happens in the writer's mind during composition (Flower & Hayes, 1981;
Hayes & Flower, 1980). From then on, several researchers attempted to grasp the complexity of the writing process in a model.
The first and most prominent model is Hayes and Flower's (1980) cognitive model of the writing process which contains three main
components: the task environment, the writer's long term memory and the writing process. Through the years, various revisions have been
made of the initial Hayes and Flower model (Chenoweth & Hayes, 2001; Flower & Hayes, 1981; Hayes, 1996, 2012; Leijten, Van Waes,
Schriver, & Hayes, 2014). However, as Hayes (2012) states, “despite its age, the [initial] model contains features that are still current in
modern representation of writing” (p.370). For the purpose of this study, the writing process component from the original model is used as a
theoretical base because it clearly distinguishes three subcomponent processes, namely planning, translating, and reviewing. This study
focuses on the interaction between, and the management of these three subprocesses in the writing process.
Hayes and Flower's model makes clear that the writing process is not so much a linear process but much more a recursive cyclical one in
which planning, translating, and reviewing may occur at any time. They constantly alternate and interact with one another throughout
composition (Flower & Hayes, 1981). Regarding the alternation of the subprocesses, several researchers (Braaksma, 2002; Graham & Harris,
2000; McCutchen, 2000; McCutchen, Covill, Hoyne, & Mildes, 1994) used the term ‘orchestration’ to emphasise the temporal management
of the writing processes and the fact that the subprocesses in writing can be activated and coordinated by the monitor.
Several studies (Beauvais, Olive, & Passerault, 2011; Berninger, Fuller, & Whitaker, 1996; Braaksma, Rijlaarsdam, Van den Bergh, & Van
Hout-Wolters, 2004; Breetvelt, Van den Bergh, & Rijlaarsdam, 1994; Levy & Ransdell, 1995; McCutchen, 1988) have shown that the ability to
manage and distribute the interacting subprocesses of writing is a decisive factor for both text quality and cognitive load. It might therefore
be expected that writing performance can be enhanced by using writing strategies that help writers efficiently manage the different
subprocesses during writing.
1.2. Writing strategies
Student writers, who are considered to be novice writers, may especially benefit from using writing strategies that help manage
orchestrating the writing process (Kozma, 1991; Torrance, Thomas, & Robinson, 1994). The premise underlying this is that novices profit
most from strategies that divide the writing process into separate stages, allowing them to focus effort on one single subtask a time,
reducing the number of simultaneous constraints (Kellogg, 2008). In this study, a novice is defined as a writer who may have sufficient
domain or genre knowledge but lacks process and procedural skills to effectively and efficiently write a coherent text. For these novices,
little of the writing process is automated and therefore they must devote close attention to a variety of tasks and processes simultaneously
(Flower & Hayes, 1981; Kellogg, 2008).
This study focuses on the effects of using a planning strategy which was expected to positively influence writing performance through not
only organising but also generating content to set up an elaborated and structured text. In line with this, Olive and Passerault (2012) suggested
that structuring ideas was important for generating new ideas. Also, Pouit and Golder (2002) found that in argumentative texts, students
included more properties for the defended position when they drew up a list of ideas in advance. Finally, Walvoord et al. (1995) found that
outlining was helpful for generating text.
1.3. Outlining as a writing strategy
Outlining is possibly the most recommended planning strategy for novices to enhance writing performance (Galbraith, Ford, Walker, &
Ford, 2005; Hayes, 2006; Murray, 2011). An outline is a specific type of text plan drawn up by the writer before fully elaborating a text. It is a
vertical list of items that is organised in the sequence which the writer intends to use for the final text, using one or more levels of hierarchy
(University of Chicago, 2003; Walvoord et al., 1995). Outlining allows generating, clustering, and ordering ideas at an early stage in the
writing process, and forces writers to consider both hierarchical and structural relations. Piolat and Roussey (1996) found that when stu-
dents set up an organised list of ideas, the chronology helped them to linearise their ideas during the translation phase leading to essays
with significant higher grades. Outlining can improve text quality because it shifts the writer's focus from a lower level of text-bound
considerations to a higher, more structural level, leading to better structured texts (Kozma, 1991).
Outlining not only influences text quality, but according to Favart and Coirier (2006) the cognitive load induced during the writing
process might be reduced when a text structure has been established in a prewriting phase. A written outline serves as an external rep-
resentation of the plan and allows the writer to focus on non-planning activities during later phases of the writing process (Kellogg, 1988).
Making a written outline prior to composing a full text could thus reduce cognitive load during writing (Collins & Gentner, 1980; Galbraith &
Rijlaarsdam, 1999; Glynn, Britton, Muth, & Dogan, 1982).
1.4. Electronic outlining
Although outlines are traditionally set up with pen and paper, writers can nowadays profit from using electronic outline tools embedded
in standard word-processing programs, such as MS® Word (Deacon, Jaftha, & Horwitz, 2004; Kozma, 1991; Price, 1997). These electronic
tools enable writers to easily create outlines in which they arrange the sequence and subordination of their ideas. In this study, the use of
354 M.J.R. de Smet et al. / Computers & Education 78 (2014) 352e366
such an electronic outline tool and the corresponding outline strategy is referred to as ‘electronic outlining’. The advantage of electronic
outlining above using pen and paper lies in its increased flexibility in that it enables writers to (1) create better visible hierarchical structures
through automatic formatting, (2) directly attach elaborated text to the different original headings from the outline, which automatically
become part of the text, (3) present the outline and the full text simultaneously on the computer screen (cf. Erkens, Jaspers, Prangsma, &
Kanselaar, 2005; Price, 1997), (4) fold and unfold parts of the text to selectively display what they are working on and hide intervening text,
(5) navigate easily through the text on the basis of the outline, and (6) easily and continuously revise their outlines (i.e., insert, delete,
promote, demote and move topics and subtopics). Price suggests that a traditional outline is a fixed document “that acts as a rigid blueprint
the student must follow when drafting” (p. 410) while an electronic outline can be part of the ongoing writing- and thinking process. He
states that the purpose of outlining (i.e., developing a meaningful structure for a text) becomes lost amid the use of pen and paper which
made changing difficult.
This study focuses on the effects of electronic outlining on students' argumentative writing. Results from previous studies (De Smet,
Brand-Gruwel, Broekkamp, & Kirschner, 2012; De Smet, Broekkamp, Brand-Gruwel, & Kirschner, 2011) suggest positive effects of elec-
tronic outlining on the quality of students' argumentative texts. The tool helped students better present their texts' structure and
decreased perceived mental effort. These studies showed that it was important to practise using the tool in order to benefit from its effects.
However, these studies only focused on the effects on students' final writing products and not on how these texts were produced or how
the outlines affected the organisation of the writing process. Moreover, in these studies, a control group was missing which hindered a
clear differentiation between the effects of electronic outlining and a common learning effect of repeatedly performing a similar writing
task.
To understand the functioning of electronic outlining, this study examines the extent to which the effects of electronic outlining on text
quality may be caused by differences in the organisation of the writing process and alleviation of attentional (over)load.
Pre-task planning such as electronic outlining may result in greater fluency (cf. Ellis & Yuan, 2004) and reduce students' online planning
and reviewing behaviour. Students could use their initial outlines as a guide while elaborating their text, which may influence the course of
the writing process. Students who do not use electronic outlining in advance may lack such an explicit text structure, which might cause the
writing process to be more fragmented and leading to a higher degree of recursion. That is, these students may need to consider content,
structure and formulations in a more extensive way while writing. Less initial planning might affect pausing and revision behaviour in the
further writing process.
1.5. Analysing the writing process
The recursive nature of the writing process often leaves traces as pauses and revisions in the process output. Differences in pausing and
revision behaviour have been found to indicate differences in the underlying cognitive processes in sentence production (Quinlan, Loncke,
Leijten, & Van Waes, 2012; Van Waes, Leijten, & Quinlan, 2010) and full text production (Butterworth, 1980; Leijten, Janssen, & Van Waes,
2010; Schilperoord, 1996; Wengelin, 2005). Many researchers analysed pause patterns to study the underlying cognitive processes in
writing and used different definitions of a pause. Spelman Miller (2005) defines a pause as a “visible trace of nonwriting activity” (p. 24)
which, as such, offers measureable clues to cognitive processes such as planning activities. Wengelin (2005) takes the practical interference
of the computer into account when defining a pause. According to her, a pause is a transition time between two keystrokes that is longer
than the time that may be expected for a writer to find the next key. In her view, a pause threshold of 2000 ms is twice as long as the normal
transition time between two keystrokes, even for the slowest typist and it is, therefore, sufficient to eliminate the practical problem of
pauses between keystrokes.
In this study, a pause is defined as a period of 2000 ms (2 s) or more (Wengelin, 2005) between two consecutive keystrokes. Although
recent research has shown that there may be important cognitive activities between 200 ms en 2000 ms (Van Waes & Leijten, 2013) for this
study, we were more interested in longer pauses as they are considered indicators of higher-level global planning processes (Schilperoord,
1996). Using a pause threshold of 2000 ms allows the elimination of shorter pauses that are caused by activities associated with lexical
retrieval and other lower-level local planning. As discussed, electronic outlining may shift a writer's focus to a higher more structural level of
planning (Kozma, 1991) which would suggest that electronic outlining leads to longer, globally-oriented pauses and reduces lower level
planning and shorter pauses.
Since the 1980's, researchers also focused on revision processes in writing research (Lindgren & Sullivan, 2006; Lindgren, Sullivan, &
Spelman Miller, 2008). Matsuhashi (1982) defines a revision as an episode in which the writer stops writing and makes a change in the
previously written text. The purpose of a revision is emphasised in the definition of Leijten and Van Waes (2006) who define revisions as
changes in the text aiming at the text's content, wording and lay-out. The analysis and description of pausing and revision behaviour may
be a first step towards understanding the effects of electronic outlining on students' cognitive strategies that subtend writing
achievement.
1.6. Research questions and hypotheses
The present study investigates the effect of electronic outlining on students' (1) writing products, (2) perceived mental effort, and (3) the
organisation of the writing process. In addition, it examined the effect of repeated use of electronic outlining as a writing strategy.
Based on the theoretical notions and prior studies on (electronic) outlining described above (De Smet et al., 2011; 2012; Kozma, 1991;
Walvoord et al., 1995) it is expected (H1) that electronic outlining leads to better structured texts. Second, it is hypothesised (H2) that
electronic outlining decreases perceived mental effort while writing (cf. Favart & Coirier, 2006; Kellogg, 2008). Furthermore, it is
hypothesised (H3) that electronic outlining causes a less fragmented organisation of the writing process in which initial planning through
electronic outlining reduces the need to plan or revise later in the writing process. Finally, based on previous studies (De Smet et al., 2012) it
is expected (H4) that repeated electronic outlining enhances effective use of this strategy and thus leads to further improvement of text
quality. Tool-practice might reduce the demands of tool appropriation to free up working memory space for the writing process.
2. Method
2.1. Participants
Participants were 93 Dutch 10th-grade students from Higher General Secondary Education (HGSE). The group consisted of 38 male and
55 female participants between the age of 14 and 17 (M ¼ 15.50; SD ¼ 0.69). Four intact classes were randomly assigned to one of the three
conditions in this experiment. In order to equalise the number of participants in each condition, the two largest classes were each assigned
to one condition and the two smallest classes were both assigned to one and the same condition. Participation in the experiment was
compulsory and part of students' regular curriculum.
2.2. Design
The central research question was examined by means of a combined within-subjects and between-subjects design in which three
conditions were compared. In each condition, students carried out two argumentative writing tasks with an interval of one week. According
to the design as shown in Table 1, students were or were not required to use the electronic outline tool to make an outline before fully
elaborating their text.
This combined design allowed determination of the effects of electronic outlining while controlling for learning effects and possible
pretest differences between conditions. The control condition (in which students used no electronic outlining for both tasks) helped to
distinguish between the effects of electronic outlining and a common learning effect of repeatedly performing a similar writing task. Finally,
the design allowed determination of the effects of repeatedly using electronic outlining (repeated outline condition). Tool- and strategy
practice might enhance effective use of electronic outlining and enhance students' writing performance.
2.3. Materials
2.3.1. Questionnaire concerning control variables

Before carrying out the writing tasks, all participants filled out a general self-report questionnaire which focused on students' (1)
knowledge of available writing tools in their word processor (2) self-efficacy for argumentative writing (3) report grade in the Dutch
language, and (4) their writing style.
Questions on knowledge of available writing tools were partly based on the questionnaire used by Leijten (2007). Students indicated
whether they had previously used any of 12 writing functions incorporated in their word processor (e.g., generating a table of contents,
using headings, using the outline tool, etcetera).
Questions on self-efficacy for argumentative writing were based on an existing self-efficacy test for argumentative writing (Braaksma,
Rijlaarsdam, & Van den Bergh, 2009). Self-efficacy beliefs may predict actual writing performance (cf. Duijnhouwer, 2010) and are there-
fore important as a control variable. Cronbach's alpha for the self-efficacy questionnaire (19 items) in this study was a ¼ .92. A 10-point scale
was used instead of the original 100-point scale used by Braaksma et al. because the Dutch educational rating system runs from 1 to 10.
Students were therefore expected to be more familiar with this scale.
Questions on students' writing style were based on a writing process questionnaire developed and tested by Kieft, Rijlaarsdam, and Van
den Bergh (2006). Using a 5-point Likert scale, students indicated the degree to which they usually engage in planning (12 items, a ¼ .71)
and revising activities (10 items, a ¼ .63) when composing an argumentative text.
2.3.2. Writing tasks

Students carried out two argumentative writing tasks about two timely topics (i.e., surveillance cameras in inner city areas and mobile
phone use (cf. Van Weijen, 2009)). Texts were all written in Dutch, which was students' L1. Apart from the specific topics and the explicit
instruction to use the outline tool or not, all other task features (e.g., genre, goal, target audience, instructions, available time and infor-
mation, topic difficulty, and students' involvement in the topic) were kept constant.
To control for possible topic-effects, the topics of the writing tasks were counterbalanced. Students were randomly assigned to a specific
order of topics. For each task, 75 min were given to write a text of 600e800 words; students were allowed to stop earlier when they
completed the task before the end of the lesson. They all knew in essence how to write an argumentative task as this was part of their regular
curriculum.
The genre of argumentative writing is particularly interesting in this context, due to its complexity and the natural requirement to
hierarchically connect and order ideas. In an argumentative text, writers should support their standpoint with arguments which in turn
should be supported with subordinated arguments. However, writers often have problems establishing a well ordered coherent argu-
mentative structure (Favart & Coirier, 2006). In argumentative writing, writers cannot structure their text through simple approaches such
as causal, chronological, or spatial organisation. Instead, to satisfy complex argumentative constraints, they must transpose their often
multidimensional mental structures into a linguistic sequence of sentences and an organised text (Coirier, Andriessen, & Chanquoy, 2000;
Coirier, Favart, & Chanquoy, 2002; Erkens et al., 2005; McCutchen, 1987). Setting up an outline may help writers to organise their argu-
mentative structure.
Table 1
Design of the study.
Task 1 Task 2
Control condition (N ¼ 28) Without outline tool Without outline tool

Single outline condition (N ¼ 29) Without outline tool With outline tool
Repeated outline condition (N ¼ 36) With outline tool With outline tool
Students who were not required to use electronic outlining started writing their text immediately after receiving the instructions, while
students using the tool spent the first part of the writing task on their outlines. No specific time instruction was given for setting up the
outline. Students could divide their own time between electronic outlining and writing the full text. To guarantee high ecological validity,
students performed the writing tasks in their own school's computer room in the presence of their own teachers.
2.3.3. Tool instruction

Prior to the first writing task that required students to use the outline tool, students received a 10-min training in which the tool and its
working were explained. For the repeated outline condition, the training was given prior to the first writing task, while for the single outline
condition training was only given prior to the second task. The training was restricted to a technical instruction showing the available
buttons and their functions, and not on how to use the outline tool to write an argumentative text so as to prevent differences between
conditions caused by instruction instead of the effects of electronic outlining. The instruction was followed by a 5-min practice session
where students individually practised using the outline tool. The tool used in this study is the outline function incorporated in the ‘view’
menu in MS® Word 2007 (see Fig. 1) which was available in the school's computer room and with which the students were familiar.
The advantage of this specific outline tool is that it is directly available in the standard settings of a widely used word processor.
Moreover, students were familiar with composing writing tasks in MS® Word, so they only needed specific instruction on the outline tool
and not on the writing environment in general.
2.3.4. Keystroke logging via Inputlog

Detailed data on the writing process was collected via the keystroke logging program Inputlog (Leijten & Van Waes, 2013). Inputlog is a
research tool for logging and analysing writing process data in Windows™ environments. It gives researchers the opportunity to capture
detailed information concerning the organisation of the writing process.
Inputlog records students' writing activity on the computer and registers and stores, from beginning to end, all keystrokes and mouse
movements and their distribution across time. The writing process can, thus, be studied in real time as it unfolds, enhancing the under-
standing of how a text develops. Using keystroke logging as an observational tool has several advantages above, for example, think aloud
procedures or retrospective interviews in that (1) it yields a detailed picture of the writing process, (2) results from different respondents are
quantitatively comparable, and (3) it is an ecological, nonintrusive method that does not interfere with the writing process or influence
students' course of writing.
2.3.5. Retrospective self-reports on perceived mental effort

To measure perceived mental effort, students indicated after each writing task how much mental effort they perceived during the
composition of the task on a 5-point Likert scale (1 ¼ very little effort, 5 ¼ very much effort). The scale is based on the 9-point cognitive load
scale developed and validated by Paas (1992).
2.4. Data analysis
2.4.1. Writing products

The quality of the writing products was evaluated with an analytic assessment protocol for argumentative text quality. This protocol was
adapted from Erkens et al. (2002) and distinguishes the degree to which students (1) correctly and completely establish text structure, (2)
clearly present this structure, and (3) produce complex and elaborated argumentation. A high score on these three characteristics would
indicate a well-structured and elaborated argumentative text.
Credit for Total Text Structure was given when students differentiated between title, introduction, text body, and conclusion. Moreover,
they needed to elaborate the different characteristics of each text part. A scoring rubric was used in which the presence (1) or absence (0) of
these different characteristics were scored. The maximum total score for Total Text Structure was 16 points (see Appendix A).
Credit for Structure Presentation was given when students established an explicit presentation of the argumentative structure by correctly
and sufficiently distinguishing paragraphs, using headings and subheadings, by correctly and sufficiently using linguistic organisers such as
connectives and anaphora (cf. Chanquoy, 1996) and by presenting the text towards the reader. The maximum score for Structure Presen-
tation was 14 points (see Appendix A).
Fig. 1. Screen dump of outline function in MS® Word 2007.

The validity and reliability of this assessment instrument has been established by De Smet et al. (2011) who used the protocol for the
same population and the same text genre. Moreover, to calculate interrater reliability for the present study, a random sample of 20 texts (i.e.
10.8% of the total sample) was independently scored by the first author and a research assistant who had a Master degree in modern Dutch
language and literature and had experience with teaching Dutch language in secondary education. A significant correlation at the .01 level
was found for both Total Text Structure (r ¼ .84) and for Structure Presentation (r ¼ .78) between both raters.
The third and last variable for the writing products was Hierarchical Elaboration of Arguments which focused on students' ability to
produce complex and elaborated argumentation. The procedure for scoring this variable started with segmentating all writing products.
Each text was manually divided into segments with regard to distinct idea units based on their argumentative function. In some cases this
would mean that sentences were split, based on argumentative and organisational markers, so that the different constituents of a sentence
could be properly coded.
Subsequently, all individual segments were evaluated in terms of their relevance to the main issue and were coded according to an
extensive coding system (see Appendix B) which incorporated 13 different labels focussing on the segments' argumentative function. A
unique code was assigned to each segment (cf. Reznitskaya, Kuo, Glina, & Anderson, 2009). In total 10% of all texts was independently coded
by the first and second author. The number of segments per text ranged from 25 to 46, with an average of 35 segments per text, thus giving a
large amount of data that were used to measure interrater reliability. Overall, interrater agreement was 78%.
Eventually, analyses concentrated on the segments focussing on the arguments and their elaboration. Based on coded segments, stu-
dents' hierarchical elaboration of arguments was measured. All arguments were counted and a mean score was calculated as follows. For
each (elaborated) main argument, students could obtain 0 or 1 point (0 ¼ a main argument is presented without any elaboration, 1 ¼ a main
argument is elaborated with at least one subargument). In other words, students could obtain a score for their argument only when an
argument was hierarchically elaborated and underpinned. Finally, an average hierarchical elaboration score (between 0 and 1) was
calculated for each student.
2.4.2. Writing process

To analyse the effects of electronic outlining on students' organisation of the writing process, the focus is mainly on the pausing and
revision behaviour during writing. Data collection via Inputlog allows observing, detecting, and analysing students' pausing-, and revision
behaviour in extensive ways. To analyse students' pausing behaviour, a threshold of 2000 ms (2 s) was used to define a pause.
First, to describe the writing process data, general process information (i.e., Total Process Time and the number of Words Produced per
Minute (WPM)) was analysed and compared between conditions. The average number of WPM is a standard measure used to analyse verbal
fluency (Chenoweth & Hayes, 2001; Gould, 1980; Kellogg, 1996; Le Bigot, Passerault, & Olive, 2012; Van Waes & Leijten, 2013). The analyses
of students' pausing behaviour are based on Mean Pause Time and the Pause Ratio. Regarding students' revision activities, a global measure
of revision (i.e., the Revision Ratio) was calculated. Students generally produce more words during the process of writing than what appears
in the final text. The Revision Ratio takes this difference into account and consists of the total number of words in the final text divided by the
total number of words produced during the writing process, subtracted from 1. Higher scores on the revision ratio indicate greater amounts
of revision (cf. Baaijen, Galbraith, & De Glopper, 2010).
In addition to these global process measures, it is important to study students' pausing behaviour per interval. Several researchers (e.g.,
Beauvais et al., 2011; Braaksma, 2002; Braaksma et al., 2004; Breetvelt et al., 1994) have shown that intervention effects on the writing
process are best visible in the temporal organisation e the orchestration e of the writing process. Global analyses may not show these
differences in the writing process. Electronic outlining as a preplanning strategy might above all influence the first part of the writing
process. Therefore, the temporal organisation of writing activities was taken into account to analyse the writing process in a more sensitive
way (Breetvelt et al., 1994; Olive, Kellogg, & Piolat, 2008).
Because of technical issues, process information of two students (both on the second writing task) was excluded from analyses. However,
for these two students, the product and self-report data were retained. We carefully controlled the writing process data by means of the
replay function in Inputlog to insure that students actually made an outline before elaborating their text. These process data showed that in
four cases students who were required to use electronic outlining did not start making an outline on their computer. Though explicitly
instructed to do so, and in spite of supervision in classrooms, they started writing their texts immediately. For these students, both product
and process information were excluded from analyses since these data do not reflect the effects of electronic outlining. For the first writing
task, this was the case for one student and for the second, data from three students were excluded from analyses for this reason. The final
data set used for analyses thus only includes texts that were performed as required by the design.
2.5. Statistics
To study the effect of electronic outlining, an ANOVA was performed on the first writing task to compare the repeated outline condition
with the control and single outline condition. On the second writing task, an ANOVA was performed to compare the effect of non-
outlining, single electronic outlining and repeated electronic outlining. Furthermore, ANOVAs were used to analyse the difference
scores for the control and the single outline condition to compare the effects of electronic outlining with a mere learning effect of
repeatedly performing a similar task. Repeated measures could not be used in this study, because there was no equal starting point for the
three conditions.
3. Results
3.1. Control variables
Before presenting the results related to the main research question, the general findings from the control variables are reported. Results
of the ANOVA comparing the three conditions (see Table 2) revealed that there were no significant a priori differences between conditions.
Moreover, no difference was observed in gender distribution between conditions (c2(2) ¼ 3.171; p ¼ .205).
Table 2
Means, Standard Deviations and ANOVA results of the Control Variables.
Control Single outline Repeated outline E^2 F p

M (SD) M (SD) M (SD)
Knowledge of writing tools (max ¼ 12) 7.23 (2.01) 8.11 (2.05) 7.52 (1.73) .034 1.452 .240
Self-efficacy for argumentative writing (max ¼ 190) 147.15 (11.61) 135.54 (25.44) 138.08 (15.53) .067 2.960 .057
Planning strategy (max ¼ 60) 33.72 (6.21) 35.15 (6.64) 35.71 (4.31) .021 .823 .443
Revision strategy (max ¼ 50) 31.04 (4.68) 32.26 (4.45) 32.28 (3.99) .017 .703 .498
Report grade Dutch language (max ¼ 10) 7.00 (.61) 6.73 (.85) 7.03 (.74) .034 1.398 .253
Student age (years) 15.65 (.85) 15.63 (.63) 15.52 (.62) .008 .341 .712
3.2. Effects of electronic outlining on students' writing products
Students' performance on the first writing task was analysed and compared to answer the first part of the research question concerning
the effect of electronic outlining on students' writing products (see Table 3).
3.2.1. First writing task

Students in the control condition and in the single outline condition carried out the first task without electronic outlining while students
in the repeated outline condition used the tool during the first task to organise and write their text. Results of the ANOVA showed a sig-
nificant effect of condition on Total Text Structure, F(2, 90) ¼ 11.312, p < .001, s2 ¼ .201. Post hoc analyses (Bonferroni) indicated that students
in the repeated outline condition elaborated the different characteristics of an argumentative text better than students in both the control
condition (p < .001) and the single outline condition (p < .001). No significant differences were found between both nonoutline conditions
regarding Total Text Structure (p ¼ .989).
Second, a significant effect was found for electronic outlining on Structure Presentation, F(2, 90) ¼ 8.972, p < .001, s2 ¼ .166. Post hoc
analyses revealed that students in the repeated outline condition scored significantly higher on the variable Structure Presentation than
students in both the control condition (p ¼ .001) and the single outline condition (p ¼ .003). This suggests that electronic outlining enhances
the presentation of the argumentative structure. There were no significant differences between the two non-outline conditions regarding
Structure Presentation (p ¼ .926).
Finally, a marginal effect was found for Elaboration of Argumentation F(2, 90) ¼ 2.855, p ¼ .063, s2 ¼ .060. A post hoc analysis was
conducted, but showed no significant differences between conditions.
3.2.2. Second writing task

Students' second writing tasks were analysed using a between-subject analysis (ANOVA) comparing all three conditions. Focus is on the
difference between the control condition and the single outline condition to determine the effect of electronic outlining as compared to a
learning effect. Furthermore, the difference between the single outline condition and the repeated outline condition is examined to compare
the effects of first time electronic outlining to the effects of repeatedly using this strategy for writing.
Analyses of the second writing task showed a significant effect on Total Text Structure, (F(2, 88) ¼ 10.77, p < .001, s2 ¼ .197). Post hoc
analyses revealed that students in the repeated outline condition scored significantly higher on Total Text Structure than both other con-
ditions (p < .001 for the control condition; p ¼ .013 for the single outline condition). However, no difference was found between the control
and the single outline condition (p ¼ .278).
Regarding Structure Presentation, a significant effect of condition (F(2, 88) ¼ 28.39, p < .001, s2 ¼ .392) was found. Post hoc analyses
revealed that the control condition scored significantly lower than both conditions using electronic outlining (p < .001 for both outline
conditions). No significant differences were found between both outline conditions (p ¼ .388). This suggests that electronic outlining
improves the presentation of the argumentative structure, but no increased beneficial effect was found for repeated use.
Finally, analyses on the second writing task revealed no effects of electronic outlining on students' Elaboration of Argumentation
(p ¼ .817).
3.2.3. Comparing difference scores: electronic outlining vs. learning effects

Difference scores in this study indicate the amount of change between the first and the second writing task. Comparing difference scores
between the control condition and the single outline condition helps distinguishing between common learning effects of repeatedly per-
forming a similar writing task, and effects of electronic outlining.
Table 3
Mean scores and standard deviations of text quality on the first and second writing task (T1 ¼ Task 1; T2 ¼ Task 2).
Control Single outline Repeated outline

(N ¼ 28) (N ¼ 29) (N ¼ 36)
M SD M SD M SD
T1: Total Text Structure 8.46 2.69 8.55 2.35 10.89 2.05
T2: Total Text Structure 8.04 2.67 8.96 1.94 10.63 2.13
T1: Structure Presentation 7.86 1.88 8.07 2.40 9.89 2.11
T2: Structure Presentation 7.68 1.96 10.46 1.72 11.09 1.85
T1: Elaboration Arguments 0.68 0.15 0.71 0.21 0.79 0.20
T2: Elaboration Arguments 0.73 0.19 0.73 0.21 0.76 0.17
Table 4
Means and standard deviations of perceived mental effort.
M SD M SD M SD
Task 1 (max ¼ 5) 2.93 0.78 3.00 1.07 2.86 0.83

Task 2 (max ¼ 5) 2.37 0.84 2.67 1.06 2.57 0.96
Regarding Total Text Structure, results from the ANOVA showed a marginally significant difference between the control and the single
outline condition (F(2, 54) ¼ 1.51, p ¼ .058, s2 ¼ .045) suggesting a marginal beneficial effect for electronic outlining as compared to a simple
learning effect.
For Structure Presentation, results from the ANOVA showed a significant difference between the control condition and the single outline
condition (F(1, 54) ¼ 22.64, p < .001, s2 ¼ .295) suggesting positive effects of electronic outlining for Structure Presentation over a simple
learning effect.
Finally, analyses revealed no significant differences for the variable Elaboration of Argumentation between the control and the single
outline condition (p ¼ .320).
3.3. Effects of electronic outlining on students' perceived mental effort
The second part of the research question focused on the effect of electronic outlining on students' perceived mental effort while carrying
out the writing tasks. Results of students' self-reports on mental effort are presented in Table 4. Results from the ANOVA showed no sig-
nificant differences between the three conditions both on the first (p ¼ .888) and the second writing task (p ¼ .531). Electronic outlining did
not significantly affect perceived mental effort during writing.
Similarly, comparing the difference scores between the control and the single outline condition, showed no significant effect of electronic
outlining (p ¼ .555).
However, within-subject analyses, comparing students' perceived mental effort over both writing tasks, pointed out that students in the
control condition (t(25) ¼ 2.487, p ¼ .01) and students in the repeated outline condition (t(36) ¼ 2.157, p ¼ .019) reported a significant
decrease in perceived mental effort over time. Students in the single outline condition reported only a marginal decrease in perceived
mental effort over both writing tasks (t(28) ¼ 1.440, p ¼ .085). This suggests beneficial effects on perceived mental effort as a result of
repeatedly using the same writing strategy (i.e., repeated electronic outlining or repeated non-outlining).
3.4. Effects of electronic outlining on students' writing process
3.4.1. General process information

The final part of the research question focused on the effect of electronic outlining on students' organisation of the writing process. First,
the Total Process Time and the number of Words Produced per Minute (WPM) were analysed and compared between conditions and over
tasks (see Table 5). Participants were given a manimum of 75 min to write their texts, but they were allowed to stop earlier when they
finished their text.
3.4.1.1. First writing task. Analyses between the three conditions on the first writing task revealed a significant main effect of condition on
total process time (F(2, 90) ¼ 12.97, p < .001, s2 ¼ .224). Post hoc analyses showed that electronic outlining significantly increased the Total
Process Time. Students in the repeated outline condition showed a significantly longer process time than students in both the control
(p < .001) and the single outline condition (p ¼ .004). No differences were found between both non-outline conditions (p ¼ .270).
It is, however, also relevant to relate the Total Process Time to the number of words in the final product to examine overall writing
fluency. A marginal effect on WPM was found (F(2, 90) ¼ 2.53, p ¼ .086, s2 ¼ .054). A post hoc analysis, however, revealed no significant
differences between conditions. Although students in the outline condition showed a significant longer process time, electronic outlining
did not affected overall writing fluency.
3.4.1.2. Second writing task. Total Process Time on the second writing task was analysed and compared between conditions. Analyses on the
second writing task revealed a significant effect for condition (F(2, 87) ¼ 11.76, p < .001, s2 ¼ .217). Post hoc analyses confirm the finding on
the first writing task that electronic outlining significantly increases Total Process Time. Here, both conditions in which students used
electronic outlining, showed a longer total process time as compared to the non-outline condition (single outline condition, p < .001;
repeated outline condition, p ¼ .001). No significant difference was found between both outline conditions (p ¼ .527).
Table 5
Means and standard deviations of total process time and words produced per minute (WPM).
M SD M SD M SD
T1: Total process time 420 3300 90 0900 470 0900 100 1700 560 2400 130 0200
T2: Total process time 320 0900 110 1100 460 4600 120 4500 430 2600 110 3800
T1: WPM 14.86 4.86 13.93 4.61 12.31 4.31
T2: WPM 20.72 8.36 14.19 5.44 16.26 5.94
Table 6
Means and standard deviations of pausing- and revision behaviour.
M SD M SD M SD
T1: Pause ratio .44 .11 .44 .10 .46 .10

T2: Pause ratio .37 .11 .36 .11 .36 .10
T1: Mean pause time (s) 10.37 3.74 11.47 4.07 10.71 3.69
T2: Mean pause time (s) 9.37 3.30 8.96 3.77 8.77 2.77
T1: Revision ratio .24 .12 .24 .10 .22 .09
T2: Revision ratio .16 .06 .22 .10 .21 .08
Analyses on the WPM showed a significant difference between conditions (F(2, 88) ¼ 7.10, p ¼ .001, s2 ¼ .140). Post hoc analyses revealed
a significant effect of electronic outlining on the second writing task. Students in the control condition produced significantly more words
per minute compared to the single outline condition (p ¼ .001) and the repeated outline condition (p ¼ .027). No significant difference was
found between both outline conditions (p ¼ .444).
3.4.1.3. Comparing difference scores: electronic outlining vs. learning effects. Difference scores from students in the control and single outline
condition were analysed and compared to examine the difference between electronic outlining and learning effects.
Regarding Total Process Time, analyses showed a significant main effect for condition (F(2, 54) ¼ 16.17, p < .001, s2 ¼ .230) suggesting that
the decrease in total process time was significantly larger for students in the control condition. Regarding difference scores on WPM, no
significant differences were found between conditions (p ¼ .950). Although students in the control condition showed a larger decrease in
total process time, their improvement in fluency did not differ from the single outline condition.
3.4.2. Pausing- and revision behaviour

Data collection via Inputlog allowed analyses of students' pausing and revision behaviour (see Table 6). Because there was a significant
difference between conditions on Total Process Time, it was important to use relative measures (e.g., Pause Ratio) instead of absolute
measures (e.g., Total Pause Time, Number of Pauses) of the writing process. A longer process time may cause more pauses during the writing
process. Using a relative measure, initial differences between conditions are controlled and the data become more comparable (cf. Leijten &
Van Waes, 2005). The Pause Ratio indicates the relative time students spent pausing as opposed to active writing time. The higher the Pause
Ratio, the more time students spent relatively on pausing and the less they spent on active writing.
3.4.2.1. First writing task. Analyses of the Pause Ratio revealed that there were no significant between-subject differences (p ¼ .588). Also,
analyses regarding the Mean Pause Time in seconds, showed no difference between conditions (p ¼ .539). Finally, students' Revision Ratio
was analysed and compared between conditions. Analyses showed no significant difference regarding students' revision ratio (p ¼ .768).
Electronic outlining did not influence students' overall revision activities on this first writing task.
3.4.2.2. Second writing task. With respect to the Pause Ratio, analyses on the second task showed no significant differences between
conditions (p ¼ .808). Moreover, no effect was found on the Mean Pause Time (p ¼ .773).
However, analyses on the Revision Ratio showed a significant difference between conditions (F(2, 85) ¼ 5.03, p ¼ .009, s2 ¼ .106). Post hoc
analyses revealed that students in the control condition had a significant lower Revision Ratio as compared to both the single outline
(p ¼ .012) and repeated outline condition (p ¼ .035). Students who used electronic outlining as a writing strategy made here more revisions
during writing, compared to students who did not use electronic outlining.
3.4.2.3. Comparing difference scores: electronic outlining vs. learning effects. No difference was found on the difference scores for Pausing
Ratio (p ¼ .621) and Mean Pause Time (p ¼ .215). However, analyses on the difference scores for Revision Ratio showed a significant difference
between the control and the single outline condition (F(1, 54) ¼ 4.19, p ¼ .046, s2 ¼ .073). Students in the control condition demonstrated a
significantly larger decrease of revisions during writing as compared to the single outline condition.
3.4.3. Temporal organization of pausing activities

In addition to global process measures, students' pausing behaviour (Number of Pauses and Pause Time) across intervals was studied.
Electronic outlining as a preplanning strategy might above all influence the first part of the writing process. Therefore, the temporal
Table 7
Means and standard deviation of pausing behaviour per interval.
M (SD) M (SD) M (SD)
Interval 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
T1: Nr of 24.54 22.77 22.00 23.19 17.85 26.41 24.90 23.62 22.66 13.97 34.44 27.73 26.07 28.27 16.70
Pauses (7.12) (8.53) (23.19) (5.51) (8.18) (8.40) (8.22) (10.35 (8.03) (6.38) (10.99) (8.12) (9.37) (10.06) (9.08)
T2: Nr of 18.93 15.57 14.71 15.57 12.43 27.66 24.21 21.83 22.10 17.10 24.03 22.97 18.43 19.37 14.13
Pauses (8.89) (8.34) (8.54) (7.30) (7.25) (8.78) (7.99) (10.10) (9.50) (9.51) (9.62) (10.86) (8.56) (8.05) (7.69)
T1: Pause 4.22 4.36 4.80 4.73 4.50 4.39 4.16 4.18 4.53 4.42 4.35 4.07 3.94 3.76 3.95
Time (s) (0.87) (1.44) (1.66) (1.51) (1.24) (0.88) (1.38) (1.00) (1.38) (1.18) (0.91) (1.03) (0.84) (0.80) (1.02)
T2: Pause 4.58 3.83 4.13 4.28 4.18 5.35 3.76 3.45 4.05 4.23 4.84 3.56 3.76 3.62 3.95
time (s) (2.16) (1.02) (1.53) (1.25) (1.53) (2.17) (1.07) (0.62) (1.01) (1.16) (1.76) (0.69) (1.03) (0.95) (1.13)
organisation of writing activities was taken into account to analyse the writing process in a more valid and sensitive way (Breetvelt et al. 1994;
Leijten & Van Waes, 2006; Olive et al., 2008). In this study, each writing process was split into five equally long time intervals. In this study, five
intervals were used because a detailed study of all the general analyses files in Inputlog showed that students spent on average 10 min outlining.
Dividing a writing process of approximately 50 min (i.e., the average total process time in this sample of students using the outline tool) in five
intervals results in five episodes of 10 min each, meaning, thus, that the first interval could roughly be considered as the outline interval.
Regarding students' Pause Time, median pause times were used instead of mean pause times. Because all types of pausing times (e.g.,
within words, between sentences) were combined, the duration of the pauses varied extremely because times considerably differ between
higher and lower level locations (Schilperoord, 1996; Spelman Miller, 2005). Due to this high deviation on pause times, mean pause times
were less suitable for analyses and therefore a more stable measure, the median, was used.
A mixed design ANOVA with repeated measures for the five intervals across time (factor interval) and with experimental condition as
between subjects factor (factor condition) was conducted for the first and second writing task. Results are shown in Table 7.
Analyses of the Number of Pauses per interval on the pretest (as shown in Fig. 2) showed a significant main effect of interval (Wilks
Lambda ¼ .274, F(4, 79) ¼ 52,255, p < .001, s2 ¼ .726) indicating that the number of pauses significantly differed across intervals.
Furthermore, a significant main effect of condition was found here (F(2, 82) ¼ 3.84, p ¼ .025, s2 ¼ .086). Post hoc analyses comparing the
three conditions showed that the repeated outline significantly differed from the control condition (p ¼ .018) and the single outline con-
dition (p ¼ .021). Electronic outlining increased the number of pauses overall. Finally, a significant interaction effect between interval and
condition was found (Wilks Lambda ¼ .712 F(8, 158) ¼ 3,651, p ¼ .001, s2 ¼ .156). To understand where in the writing process conditions
differed, difference scores between intervals (within conditions) were calculated and computed. Post hoc analyses of these difference scores
revealed a significant difference between conditions from the first to the second interval (p ¼ .013) in which the repeated outline condition
significantly differed from the control (p ¼ .003) and the single outline condition (p ¼ .012). Students using electronic outlining showed a
significant decrease in the number of pauses between the first and the second interval while the number of pauses did not decrease for
students who did not use electronic outlining. This suggests that electronic outlining as a preplanning strategy decreased online planning.
On the second writing task, there is again a significant main effect of interval (Wilks Lambda ¼ .424, F(4, 81) ¼ 27,479, p < .001, s2 ¼ .726).
Also, a significant main effect of condition was found (F(2, 84) ¼ 6.48, p ¼ .002, s2 ¼ .134). Post hoc analyses indicated that the control
condition significantly differed from the single outline condition (p ¼ .001) and the repeated outline condition (p ¼ .031). Students in both
outline conditions showed more pauses overall. However, on this second writing task, no significant interaction effect was found between
condition and interval. The finding on the first task in which preplanning in the form of electronic outlining reduced online planning was not
confirmed in this second task.
Regarding the Median Pause Time per interval (see Fig. 3), results on the first writing task showed that there was no significant main effect
of interval. A significant main effect of condition was found (F(2, 67) ¼ 3.70, p ¼ .030, s2 ¼ .100). Post hoc analyses revealed that the repeated
outline condition significantly differs from the control condition (p ¼ .010) and marginally differs from the single outline condition
(p ¼ .086). Electronic outlining decreased the Median Pause Time across overall. No significant interaction effect between interval and
condition was found.
On the second writing task, a significant main effect of interval was found (Wilks Lambda ¼ .715, F(4, 64) ¼ 6,39, p < .001, s2 ¼ .285). No
significant main effect was found, however, of condition. On the second writing task, there was no longer a difference between students who
did or did not use electronic outlining. Again, no significant interaction effect was found.
3.4.4. Repeatedly using the same writing strategy

The results from the writing process analyses, as described above, suggest an appropriation effect for both the control and the repeated
outline condition. Students in these two conditions seem to become more fluent across tasks. However, this does not seem to be the case for
the single outline condition. Judging from this first impression, it would be interesting to examine the effect of repeatedly using the same
writing strategy. Therefore, additional paired-samples t-tests for all three conditions were performed to examine the change within subjects
across both writing tasks.
Analyses on students' Total Process Time showed a significant decrease over both writing tasks for the control (t(27) ¼ 5.956, p < .001) and
the repeated outline condition (t(30) ¼ 6.119, p < .001). No significant decrease in time was found for the single outline condition (p ¼ .333).
A similar result was found for WPM. A significant increase in WPM over both writing tasks was found for students in the control
(t(27) ¼ 4.743, p < .001) and the repeated outline condition (t(31) ¼ 4.161, p < .001). No significant difference was found for the single
outline condition across tasks (p ¼ .314).
Fig. 2. Number of pauses per interval for the first and second writing task.
Fig. 3. Median pause time per interval for the first and second writing task.
Regarding Total Number of Pauses, analyses demonstrated a similar finding. Students in the control condition showed a decrease in total
number of pauses (t(27) ¼ 5.549, p < .001) as did the repeated outline condition (t(30) ¼ 6.640, p < .001). Again, this decrease was not found
for the single outline condition (p ¼ .186).
With respect to Total Pause Time, a significant decrease was found for the control (t(27) ¼ 3.524, p ¼ .001), the single outline
(t(27) ¼ 3.775, p < .001) and the repeated outline condition (t(30) ¼ 7.321, p < .001).
Analyses of the Mean Pause Time, showed no significant decrease over writing tasks for the control condition (p ¼ .156), but a decrease
was found for both the single outline (t(27) ¼ 2.950, p ¼ .003) and repeated outline condition (t(30) ¼ 3.689, p < .001).
Students' Pause Ratio decreased for the control (t(27) ¼ 2.461, p ¼ .011), the single outline (t(30) ¼ 4.652, p < .001) and the repeated
outline condition (t(30) ¼ 6.162, p < .001). During the second writing task, all conditions spent relatively more time actively writing
compared to the first writing task.
Finally, analyses from the paired-samples t-test revealed a significantly decreased Revision Ratio for the control condition
(t(26) ¼ 4.040, p < .001). However, no significant difference was found across tasks for either the single outline (p ¼ .621) or the repeated
outline condition (p ¼ .678). Both conditions using electronic outlining did not change across tasks regarding the revision ratio, though
students in the control condition showed a decreased Revision Ratio indicating less revisions during the second writing task.
4. Discussion
The goal of this study was to determine how electronic outlining affects students' writing. In order to do so, it examined the effects of
repeated and single use of electronic outlining on the quality of students' argumentative texts, the organisation of the writing process and
perceived mental effort.
Consistent with the hypothesis (H1), beneficial effects were found for electronic outlining on the variable Structure Presentation which
focused on reader orientation and correct and sufficient use of headings, paragraphs and connectives. On the first writing task, the repeated
outline condition significantly outperformed both conditions in which students did not use electronic outlining, and on the second task both
conditions in which students used electronic outlining scored significantly higher than students in the control condition. Furthermore,
difference scores regarding structure presentation showed that across both tasks, there were positive effects of electronic outlining
compared to mere learning effects. Electronic outlining helped to improve Structure Presentation, and although this technique explicitly
encourages students to use headings in their text as the original headings from the outline become part of the elaborated text, additional
analyses showed that the effects of electronic outlining on Structure Presentation remain intact when headings are discounted. This allows
making more general conclusions about the role of electronic outlining on linguistic features beyond those relating to headings. Results
showed, thus, that electronic outlining enhances the presentation of the argumentative structure.
Second, on the first writing task, students in the repeated outline condition scored significantly higher than both non-outline con-
ditions on the variable Total Text Structure, which focused on the presence of all argumentative characteristics in the text. Electronic
outlining improved students' elaboration of the different characteristics of an argumentative text. On the second writing task, students in
the repeated outline condition again scored significantly higher than the other two conditions. Regarding difference scores on Total Text
Structure, students in the single outline condition showed a marginally greater improvement over time compared to the control condition
suggesting a marginal positive effect of electronic outlining over a learning effect. However, students in the single outline condition who
used the tool for the first time during the second task did not score significantly higher than the control condition. This is somewhat at
odds with the hypothesis since one would expect this condition also to perform better when using electronic outlining, as was the case for
the repeated outline condition. This may partly be accounted for by a lack of practice in electronic outlining. Previous research (De Smet
et al., 2012) showed that beneficial effects of electronic outlining on text quality were most prominent for repeated tool-use. During the
second writing task, students in the single outline condition used the tool for the first time which might explain why electronic outlining
did here not improve their scores on Total Text Structure. However, it is not clear why electronic outlining did enhance Total Text Structure
for students in the repeated outline condition when using the tool for the first time but not for students in the single outline condition. A
priori differences between conditions should not be the cause of this result, since control variables for relevant pretest differences were
analysed and no a-priori differences were found between conditions. An alternative explanation is that changing strategies over time
might here have interfered with the learning process. Dembo and Praks Seli (2004) indicated that changing learning strategies requires
considerable commitment, effort and time. Further research must then be done to investigate the effects of changing writing strategies
over time.
Regarding students' Elaboration of Arguments, no effects of electronic outlining were found. This can probably be accounted for by the
minimal instruction students received in this experiment. Instruction focused only on the technical aspects of electronic outlining and not
on how to produce elaborated argumentation with an outline tool. Although argumentative writing and elaborated argumentation was part
of students' regular curriculum, it was not explicitly repeated here. As a consequence, students may not have used the outline tool effectively
for setting up an elaborated argumentative structure. For educational purposes, it would therefore be important to focus instruction on
hierarchical and structural relations and integrate this instruction on elaborated argumentation with the technical outline instruction.
Research has shown that for higher-order skills such as general problem solving, metacognitive processing and writing, integrated in-
struction is much more effective than separate instruction (Brown, 1997). Future studies should therefore offer students a more integrated
instruction focussing not only on the technical aspects of tool use, but also on the theoretical principles of argumentative writing, setting up
an argumentative structure and how an electronic outline tool can help planning and structuring a complex and hierarchical argumentation.
In conclusion, the first hypothesis regarding the quality of students' writing products was partially confirmed. Although there are clear
positive effects of electronic outlining on Structure Presentation, the effects are still indistinct regarding Total Text Structure and absent for
Elaboration of Argumentation.
Analyses of students' Perceived Mental Effort (H2) showed no significant differences between conditions on either the first or the second
writing task. However, a significant beneficial effect was found for repeatedly using the same writing strategy. Practice in applying a specific
writing strategy (electronic outlining, or not) decreased perceived mental effort over time. This is in line with other studies (Shaffer, Doube,
& Tuovinen, 2003; Sweller, Van Merrie €nboer, & Paas, 1998) which found that once a particular skill is acquired, automatic processing can
circumvent the limits of working memory. It is therefore important to practise using a specific strategy to decrease perceived mental effort
while writing an argumentative text.
Previous research (De Smet et al., 2012) concluded that beneficial effects on perceived mental effort resulted from repeated use of
electronic outlining. However, the present study suggests that it is not only the effect of repeated electronic outlining, but the effect of
repeatedly using the same writing strategy in general which reduces mental effort across writing tasks. Using electronic outlining for the
first time, or changing strategies, did not reduce perceived mental effort. The process of strategy exploration and appropriation might here
have interfered with the writing process. Since working memory is limited, when a learner is confronted with a complex task and a new tool
designed to aid in carrying out the task, the learner may at first experience extraneous load caused by the tool. Van Bruggen, Kirschner, and
Jochems (2002) found this, for example, when students were required to produce external visual representations that were actually
intended to reduce load.
Analyses regarding the hypothesis on students' writing process (H3) showed that electronic outlining significantly increased Total Process
Time. On the first writing task, the repeated outline condition showed a significantly longer writing process than the control and the single
outline condition. Similarly, on the second writing task, both outline conditions showed a significantly longer process time than the control
condition. Students using electronic outlining spent on average more than 10 min longer on their writing task than students who did not use
electronic outlining. This difference in Total Process Time might be the result of the time students need to plan and set up their outlines. It
might, however, also be the case that students using electronic outlining think more deeply about what they are doing due to the
requirement to plan and that they therefore showed a longer writing process, even when excluding the outlining phase. Regarding the latter,
Kellogg (1988) indeed found that students' mean time spent writing was about seven minutes greater for students who outlined (excluding
their prewriting time). Regarding the number of Words Produced per Minute, no significant effects were found on the first writing task;
though students using electronic outlining showed significantly longer process times, their writing fluency did not differ from students in
both non-outline conditions. Nevertheless, on the second writing task, students in the control condition produced significantly more words
per minute compared to both outline conditions. Contrary to our hypothesis, students performing both tasks without electronic outlining
became more fluent on the second writing task as compared to both outline conditions. Similar to findings in Kellogg's (1988) study,
electronic outlining improved the quality of the writing product, but there was no evidence that electronic outlining improved the efficiency
of the writing process. It is, however, important to note that the production of the outline is included in these overall analyses. Setting up an
outline, which requires the determination of discrete elements, may initially involve high cognitive effort and may thus be a less fluent
activity than directly starting the translation of a text; a strategy which may have become automated across time. This could consequently
decrease students' average fluency on the whole writing task. Overall analyses do not take into account these possible differences in fluency
across ‘sub phases’ of outlining and text elaboration (cf. Van Waes & Leijten, 2012). Future research may focus on these different phases of
writing in order to confirm this finding.
Regarding students' pausing behaviour, results on the Pause Ratio showed that there are no significant differences on both the first and
the second writing task regarding students' overall pausing ratio. Also, regarding the Mean Pause Time, no effects of electronic outlining were
found on the first and second writing task. Although it was hypothesised (H3) that electronic outlining might lead to less but longer pauses,
instead of many short pauses, this was not confirmed by means of students' overall pausing behaviour. This is not surprising because
intervention effects on the writing process are mostly not found on global measures but much more in the temporal organisation. This study,
therefore, also analysed students' pausing behaviour on interval level. However, results from this temporal analysis did not provide clear
answers to the question how electronic outlining influenced students' pausing behaviour. Results on the first writing task suggested that
electronic outlining in advance decreased the number of pauses between the first and second interval once the outline was finished. In other
words, preplanning through electronic outlining reduced online planning, but only immediately after the outlining phase. This was only a
short effect. Moreover, this effect was not affirmed by the results on the second writing task. These analyses therefore not clearly provide an
answer to the effects of electronic outlining on students' pausing behaviour.
Analyses on students' revision behaviour showed no effects of electronic outlining on the first writing task, however, on the second
writing task, both outline conditions showed a significantly higher Revision Ratio indicating more revisions during writing. Contrary to the
hypothesis, students using electronic outlining revised more during writing than students who did not use this strategy. Although more
revision is often seen as an indication for better writing, it was expected in this study that students using electronic outlining would need
less revision since they planned their text more extensively in advance. Apparently, this was not the case. Students in the control condition,
who wrote both texts without electronic outlining, may have focused less on revision during the second writing task because their writing
became more automated over time without focussing extensively on planning or revising activities.
Analyses on the writing process revealed that practice in using a specific writing strategy (i.e., repeated electronic outlining, repeated
direct writing) enhanced writing fluency in terms of total process time, words produced per minute, total number of pauses, total pause time
and perceived mental effort. Results thus underline the importance of strategy practice.
With respect to the last hypothesis on the effects of repeated electronic outlining (H4), it was found that repeated use showed beneficial
effects on the writing product regarding Total Text Structure. However, as mentioned above, it may not be repeated electronic outlining that
caused this, but rather repeated use of a writing strategy that enhanced writing fluency.
In conclusion, results from this study suggest beneficial effects of electronic outlining on students' writing products. However, no clear
answer was found on the question how these effects are achieved. To this end, the following adaptations could be made concerning the
methodology to insure univocality of results found. First, future studies should elaborate instruction and not only focus on the technical
aspects of electronic outlining but incorporate instruction on setting up a complex and elaborated argumentation. Furthermore in addition
to the questionnaire concerning control variables, future studies should include a pretest writing task to insure that the performance profile
of all three conditions is equal at the beginning. This may gain better insight in current results on Total Text Structure.
Results from this study help understand how electronic outlining affects argumentative writing. However, not much is known about the
quality and characteristics of these outlines. Future research should also analyse the outlines apart from the final text. Students' outlines may
naturally differ in terms of completeness, elaborateness, and hierarchical organisation. A study of the General Analyses file from Inputlog
revealed that students' outlines range from very generic to content specific (cf. Walvoord et al., 1995). Similar to Piolat and Roussey (1996),
who distinguish different types of drafts (i.e., note drafts, organised drafts, and composed drafts), a distinction can be made for different
types of outlines that may evoke different effects for the final text. Future studies should consider these different types of outlines, analysing
them in terms of content and structure and relate this to the quality of the final texts.
Finally, although keystoke logging has advantages for research purposes, it also causes some challenges. The data can measure the
occurrence of pauses and revisions and show exactly when and how often they occur, but it remains difficult to relate these events to the
underlying cognitive processes and the reasons for pausing and revision activities (Baaijen et al., 2012; Lindgren & Sullivan, 2005; Wengelin,
2005). The issue with cognitive processes is that they can never be directly observed. Future studies may therefore complement keystroke
logging data with think aloud protocols (i.e., concurrent and retrospective) or with eye tracking data. This may allow researchers to examine
why students perform a writing task in a certain way and where students focus on during pauses and revisions.
This study suggests that electronic outlining has the potential to improve students' text quality and, thus, creates perspectives for further
research on optimising the effects of electronic outlining and setting up an effective instruction for electronic outlining and argumentative
writing in the classroom. Based on the findings in this study, it can be recommended for educational practice that future instruction
regarding electronic outlining should provide sufficient practice and explicit strategy instruction. Although results were not univocal, it
seemed that more pausing at the beginning of the writing process, reduced online planning which may result in better quality texts. In-
struction should thus focus on the importance of initial planning and the support of electronic outlining in this preplanning process.
Role of the funding source
The funding source had no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in
the decision to submit the article for publication.
Acknowledgement
This research project was generously funded by the Kennisnet Foundation, the knowledge centre for information and communication
technologies and education in the Netherlands.
Appendix A. Supplementary data
Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.compedu.2014.06.010.
References
Baaijen, V. M., Galbraith, D., & De Glopper, K. (2010). Writing: the process of discovery. In S. Ohlsson, & R. Catrambone (Eds.), Proceedings of the 32nd Annual Conference of the
Cognitive Science Society (pp. 1774e1779). Austin, TX: Cognitive Science Society.
Baaijen, V. M., Galbraith, D., & De Glopper, K. (2012). Keystroke analysis: reflections on procedures and measures. Written Communication, 29, 246e277. http://dx.doi.org/
10.1177/0741088312451108.
Baddeley, A. (2010). Working memory. Current Biology, 20(4), 136e140.
Beauvais, C., Olive, T., & Passerault, J.-M. (2011). Why are some texts good and others not? Relationship between text quality and management of the writing process. Journal
of Educational Psychology, 103, 415e428.
Berninger, V. W., Fuller, F., & Whitaker, D. (1996). A process model of writing development across life span. Educational Psychology Review, 8, 193e218. http://dx.doi.org/
10.1007/BF01464073.
Braaksma, M. A. H. (2002). Observational learning in argumentative writing. Unpublished doctoral dissertation. The Netherlands: University of Amsterdam.
Braaksma, M. A. H., Rijlaarsdam, G., & Van den Bergh, H. (2009, August). Hypertext writing: Effects on writing processes and writing products. Paper presented at the EARLI
Conference, Budapest, Hungary.
Braaksma, M. A. H., Rijlaarsdam, G., Van den Bergh, H., & Van Hout-Wolters, B. H. A. M. (2004). Observational learning and its effects on the orchestration of writing processes.
Cognition and Instruction, 22(1), 1e36.
Breetvelt, I., Van den Bergh, H., & Rijlaarsdam, G. (1994). Relations between writing processes and text quality: when and how? Cognition and Instruction, 12(2), 103e123.
Brown, A. (1997). Transforming schools into communities of thinking and learning about serious matters. American Psychologist, 52, 399e413.
Butterworth, B. L. (1980). Evidence from pauses in speech. In B. L. Butterworth (Ed.), Language production 1: Speech and talk (pp. 155e176). Londen, England: Academic Press.
Chanquoy, L. (1996, October). Connectives and argumentative text: A developmental study. Paper presented at the first international workshop on argumentative text pro-
cessing, Barcelona, Spain.
Chenoweth, N. A., & Hayes, J. R. (2001). Fluency in writing: generating text in L1 and L2. Written Communication, 18(1), 80e98.
Coirier, P., Andriessen, J. E. B., & Chanquoy, L. (2000). From planning to translating: the specificity of argumentative writing. In P. Coirier, & J. E. B. Andriessen (Eds.),
Foundations of argumentative text processing (pp. 1e29). Amsterdam, the Netherlands: Amsterdam University Press.
Coirier, P., Favart, M., & Chanquoy, L. (2002). Ordering and structuring ideas in text: from conceptual organization to linguistic formulation. European Journal of Psychology of
Education, 17(2), 157e175.
Collins, A., & Gentner, D. (1980). A framework for a cognitive model of writing. In L. W. Gregg, & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 51e72). Hillsdale, NJ:
Lawrence Erlbaum Associates.
Deacon, A., Jaftha, J., & Horwitz, D. (2004). Customising Microsoft Office to develop a tutorial learning environment. British Journal of Educational Technology, 35, 223e234.
Dembo, M. H., & Praks Seli, H. (2004). Students' resistance to change in learning strategies courses. Journal of Developmental Education, 27(3), 2e11.
De Smet, M. J. R., Brand-Gruwel, S., Broekkamp, H., & Kirschner, P. A. (2012). Write between the lines: electronic outlining and the organization of text ideas. Computers in
Human Behavior, 28(6), 2107e2116.
De Smet, M. J. R., Broekkamp, H., Brand-Gruwel, S., & Kirschner, P. A. (2011). Effects of electronic outlining on students' argumentative writing performance. Journal of
Computer Assisted Learning, 27(6), 557e574.
Duijnhouwer, H. (2010). Feedback effects on students' writing motivation, process, and performance. Unpublished doctoral dissertation. the Netherlands: University of Utrecht.
Ellis, R., & Yuan, F. (2004). The effects of planning on fluency, complexity and accuracy in second language narrative writing. Studies in Second Language Acquisition, 26, 59e84.
http://dx.doi.org/10.10170/S0272263104261034.
Erkens, G., Jaspers, J. G. M., Prangsma, M. E., & Kanselaar, G. (2005). Coordination processes in computer supported collaborative writing. Computers in Human Behavior, 21,
463e486.
Erkens, G., Kanselaar, G., Prangsma, M. E., & Jaspers, J. (2002). Using tools and resources in computer supported collaborative writing. In G. Dr. Stahl (Ed.), Computer support for
collaborative learning: Foundations for a CSCL community (pp. 389e399). Hillsdale, NJ: Lawrence Erlbaum Associates.
Favart, M., & Coirier, P. (2006). Acquisition of the linearization process in text composition in third to ninth graders: effects of textual superstructure and macrostructural
organization. Springer, 35, 305e328.
Flower, L. S., & Hayes, J. R. (1980). The dynamics of composing: making plans and juggling constraints. In L. W. Gregg, & E. R. Steinberg (Eds.), Cognitive processes in writing: An
interdisciplinary approach. Hillsdale, NJ: Lawrence Erlbaum Associates.
Flower, L. S., & Hayes, J. R. (1981). A cognitive process: theory of writing. College Composition and Communication, 32(4), 365e387.
Galbraith, D., Ford, S., Walker, G., & Ford, J. (2005). The contribution of different components of working memory to knowledge transformation during writing. Educational
Studies in Language and Literature, 5, 113e145.
Galbraith, D., & Rijlaarsdam, G. (1999). Effective strategies for the teaching and learning of writing. Learning and Instruction, 9, 93e108.
Glynn, S. M., Brittton, B. K., Muth, K. D., & Dogan, N. (1982). Writing and revising persuasive documents: cognitive demands. Journal of Educational Psychology, 74(4), 557e567.
Graham, S., & Harris, K. R. (2000). The role of self-regulation and transcription skills in writing and writing development. Educational Psychologist, 35(1), 3e12.
Gould, J. D. (1980). Experiments on composing letters: some facts, some myths, and some observations. In L. W. Gregg, & E. R. Steinberg (Eds.), Cognitive processes in writing
(pp. 97e127). Hillsdale, NJ: Lawrende Erlbaum Associated.
Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing. In C. M. Levy, & S. Ransdell (Eds.), The science of writing: Theories, methods, individual
differences, and applications (pp. 1e28). Mahwah, NJ: Erlbaum.
Hayes, J. R. (2006). New directions in writing theory. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 28e40). New York, NY: The Guilford
Press.
Hayes, J. R. (2012). Modeling and remodeling writing. Written Communication, 29, 369e388. http://dx.doi.org/10.1177/0741088312451260.
Hayes, J. R., & Flower, L. S. (1980). Identifying the organization of writing processes. In L. W. Gregg, & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 3e30).
Hillsdale, NJ: Erlbaum.
Kellogg, R. T. (1988). Attentional overload: effects of rough draft and outline strategies. Journal of Experimental Psychology: Learning, Memory and Cognition, 14, 355e365.
Kellogg, R. T. (1990). Effectiveness of prewriting strategies as a function of task demands. American Journal of Psychology, 103, 327e342.
Kellogg, R. T. (1994). The psychology of writing. New York, NY: University Press.
Kellogg, R. T. (1996). A model of working memory in writing. In C. M. Levy, & S. Ransdell (Eds.), The science of writing: Theories, methods, individual differences, and applications
(pp. 57e71). Mahwah, NJ: Erlbaum.
Kellogg, R. T. (2008). Training writing skills: a cognitive developmental perspective. Journal of Writing Research, 1(1), 1e26.
Kieft, M., Rijlaarsdam, G., & Van den Bergh, H. (2006). Writing as a learning tool: testing the role of students' writing strategies. European Journal of Psychology of Education,
12(1), 17e34.
Kozma, R. (1991). The impact of computer-based tools and embedded prompts on writing processes and products of novice and advanced college writers. Cognition and
Instruction, 8(1), 1e27.
Le Bigot, N., Passerault, J.-M., & Olive, T. (2012). Visuospatial processing in memory for word location in writing. Experimental Psychology, 59(3), 138e146. http://dx.doi.org/
10.1027/1618-3169/a000136.
Leijten, M. (2007). Writing and speech recognition. Observing error correction strategies of professional writers. Unpublished doctoral dissertation. Utrecht, the Netherlands: LOT.
Leijten, M., Janssen, D., & Van Waes, L. (2010). Error correction strategies of professional speech recognition users: three profiles. Computers in Human Behavior, 26, 964e975.
Leijten, M., & Van Waes, L. (2005). Writing with speech recognition: the adaptation process of professional writers with and without dictating experience. Interacting with
Computers, 17, 736e772.
Leijten, M., & Van Waes, L. (2006). Inputlog: new perspectives on the logging of on-line writing. In K. P. H. Sullivan, & E. Lindgren (Eds.), Studies in writing: Vol. 18. Computer
keystroke logging and writing: Methods and applications (pp. 73e94). Oxford, England: Elsevier.
Leijten, M., & Van Waes, L. (2013). Keystroke logging in writing research: using Inputlog to analyze and visualize writing processes. Written Communication, 30(3), 358e392.
http://dx.doi.org/10.1177/0741088313491692.
Leijten, M., Van Waes, L., Schriver, K., & Hayes, J. R. (2014). Writing in the workplace: constructing documents using multiple digital sources. Journal of Writing Research, 5(3),
285e337.
Levy, C. M., & Ransdell, S. E. (1995). Is writing as difficult as it seems? Memory & Cognition, 23, 767e779. http://dx.doi.org/10.3758/BF03200928.
Lindgren, E., & Sullivan, K. P. H. (2005). Analysing online revision. In G. Rijlaarsdam (Ed.), Computer keystroke logging and writing: Methods and applications (pp. 157e188). UK/
Nederland: Elsevier.
Lindgren, E., & Sullivan, K. P. H. (2006). Writing and the analysis of revision: an overview. In K. P. H. Sullivan, & E. Lindgren (Eds.), Studies in writing: Vol. 18. Computer keystroke
logging and writing: Methods and applications (pp. 31e44). Oxford: Elsevier.
Lindgren, E., Sullivan, K. P. H., & Spelman Miller, K. (2008). Development of fluency and revision in L1 and L2 writing in Swedish high school years 8 and 9. International Journal
of Applied Linguistics, 156, 133e151.
Matsuhashi, A. (1982). Explorations in real-time production of written discourse. In M. Nystrand (Ed.), What writers know. The language, process, and structure of written
discourse (pp. 269e290). New York, NY: Academic Press.
McCutchen, D. (1987). Children's discourse skills: form and modality requirements of schooled writing. Discourse Processes, 10(3), 267e286.
McCutchen, D. (1988). Functional automaticity in children's writing: a problem of metacognitive control. Written Communication, 5, 306e324. http://dx.doi.org/10.1177/
0741088388005003003.
McCutchen, D. (2000). Knowledge, processing, and working memory: Implications for a theory of writing. Educational Psychologist, 35(1), 13e23.
McCutchen, D., Covill, A., Hoyne, S. H., & Mildes, K. (1994). Individual differences in writing: implications of translating fluency. Journal of Educational Psychology, 86, 256e266.
Murray, R. (2011). How to write a thesis. Maidenhead, UK: Open University Press.
Olive, T., Kellogg, R. T., & Piolat, A. (2008). Verbal, visual and spatial working memory demands during text composition. Applied Psycholinguistics, 19, 669e687.
Olive, T., & Passerault, J.-M. (2012). The visuospatial dimension of writing. Written Communication, 29, 326e344. http://dx.doi.org/10.1177/0741088312451111.
Paas, F. G. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: a cognitive-load approach. Journal of Educational Psychology, 84, 429e434.
Piolat, A., & Roussey, J.-Y. (1996). Students' drafting strategies and text quality. Learning and Instruction, 6(2), 111e129.
Pouit, D., & Golder, C. (2002). Idea retrieval in argumentative text writing by 11e18 year old students. European Journal of Psychology of Education, 17, 309e320.
Price, J. (1997). Electronic outlining as a tool for making writing visible. Computers and Composition, 14, 409e427.
Quinlan, T., Loncke, M., Leijten, M., & Van Waes, L. (2012). Coordinating the cognitive processes of writing: the role of the monitor. Written Communication, 29(3),
345e368.
Reznitskaya, A., Kuo, L., Glina, M., & Anderson, R. C. (2009). Measuring argumentative reasoning: what's behind the number? Learning and Individual Differences, 19, 219e224.
Schilperoord, J. (1996). It's about time. Temporal aspects of cognitive processes in text production. Amsterdam/Atlanta: Rodopi.
Shaffer, D., Doube, W., & Tuovinen, J. (2003). Applying cognitive load theory to computer science education. In M. Petre, & D. Budgen (Eds.), Proceedings of the Joint Conference
for Evaluation and Assessment in Software Engineering and the Psychology of Programming Interest Group 2003 (pp. 333e346).
Spelman Miller, K. S. (2005). Pausing, productivity and the processing of topic in online writing. In G. Rijlaarsdam (Ed.), Computer keystroke logging and writing: Methods and
applications (pp. 131e155). UK/Nederland: Elsevier.
Sweller, J., Van Merrie €nboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251e296.
Torrance, M., Thomas, G. V., & Robinson, E. J. (1994). The effect of outlining and rough drafting strategies on the quality of short essays. Paper presented at EARLI SIG Writing
Conference, Utrecht, the Netherlands.
Torrance, M., Thomas, G. V., & Robinson, E. J. (2000). Individual differences in undergraduate essay-writing strategies: a longitudinal study. Higher Education, 39, 181e200.
University of Chicago. (2003). The Chicago manual of style (15th ed.). Chicago, IL: University of Chicago Press.
U.S. Department of Education. Institute of Education Sciences. National Center for Education Statistics. (2003). The Nation's report card: Writing 2002, NCES 2003e529, by H. R.
Persky, M. C. Daane, and Y. Jin. Washington, DC.
Van Bruggen, J. M., Kirschner, P. A., & Jochems, W. (2002). External representation of argumentation in CSCLS and the managment of cognitive load. Learning and Instruction,
12(1), 121e138.
Van Waes, L., & Leijten, M. (2012, July). Writing fluency revisited. Paper presented at EARLI SIG Writing Conference, Porto, Portugal.
Van Waes, L., & Leijten, M. (2013). Vlot schrijven: Een multidimensioneel perspectief op ‘writing fluency’ [Fluent writing: A multidimensional perspective on writing fluency].
Tijdschrift voor Taalbeheersing, 35(2), 160e182.
Van Waes, L., Leijten, M., & Quinlan, T. (2010). Reading during sentence composing and error correction: a multilevel analysis of the influences of task complexity. Reading and
Writing: An Interdisciplinary Journal, 23(7), 803e834.
Van Weijen, D. (2009). Writing processes, text quality, and task effects. Empirical studies in first and second language writing. Unpublished doctoral dissertation. Utrecht, the
Netherlands: LOT.
Walvoord, B. E., Anderson, V. J., Breihan, J. R., McCarthy, L. P., Robison, S. M., & Sherman, A. K. (1995). Functions of outlining among college students in four disciplines. Research
in the Teaching of English, 29, 390e421.
Wengelin, A. (2005). Examining pauses in writing: theory, methods and empirical data. In G. Rijlaarsdam (Ed.), Computer keystroke logging and writing: Methods and Ap-
plications (pp. 107e130). UK, Nederland: Elsevier.

Electronic Outline

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Electronic Outline

Uploaded by

Copyright:

Available Formats

Computers & Education 78 (2014) 352e366

Contents lists available at ScienceDirect

Computers & Education

Electronic outlining as a writing strategy: Effects on students' writing

1.1. The writing process

1.2. Writing strategies

1.3. Outlining as a writing strategy

1.4. Electronic outlining

1.5. Analysing the writing process

1.6. Research questions and hypotheses

2.3.1. Questionnaire concerning control variables

2.3.2. Writing tasks

Control condition (N ¼ 28) Without outline tool Without outline tool

2.3.3. Tool instruction

2.3.4. Keystroke logging via Inputlog

2.3.5. Retrospective self-reports on perceived mental effort

2.4. Data analysis

2.4.1. Writing products

Fig. 1. Screen dump of outline function in MS® Word 2007.

2.4.2. Writing process

3.1. Control variables

Control Single outline Repeated outline E^2 F p

3.2. Effects of electronic outlining on students' writing products

3.2.1. First writing task

3.2.2. Second writing task

3.2.3. Comparing difference scores: electronic outlining vs. learning effects

Control Single outline Repeated outline

Control Single outline Repeated outline

Task 1 (max ¼ 5) 2.93 0.78 3.00 1.07 2.86 0.83

3.3. Effects of electronic outlining on students' perceived mental effort

3.4. Effects of electronic outlining on students' writing process

3.4.1. General process information

Control Single outline Repeated outline

Control Single outline Repeated outline

T1: Pause ratio .44 .11 .44 .10 .46 .10

3.4.2. Pausing- and revision behaviour

3.4.3. Temporal organization of pausing activities

Control Single outline Repeated outline

M (SD) M (SD) M (SD)

3.4.4. Repeatedly using the same writing strategy

Role of the funding source

Appendix A. Supplementary data

Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.compedu.2014.06.010.

You might also like