Journal of English for Academic Purposes 37 (2019) 22e33

Contents lists available at ScienceDirect

Journal of English for Academic Purposes

journal homepage: www.elsevier.com/locate/jeap

Learning to evaluate through that-clauses: Evidence from a

longitudinal learner corpus
Deliang Man, Meng Huat Chau*
Faculty of Languages and Linguistics, University of Malaya, Malaysia

a r t i c l e i n f o a b s t r a c t

Article history: The language of evaluation in academic writing has been increasingly studied. However, little
Received 17 August 2018 is known about how students learn to make evaluative meanings. This article reports on a
Received in revised form 7 October 2018 study of evaluative that-clauses based on a longitudinal corpus of 632 argumentative essays
Accepted 9 November 2018
by 158 Chinese undergraduate students. The findings of the study show a non-linear tra-
Available online 10 November 2018
jectory of change in the use of evaluative that-clauses over time. It was found that the
proportion of verb that-clauses decreased whereas the proportions of noun that-clauses and
Keywords:
adjective that-clauses increased. Further, while the range of the lexical words co-occurring
Academic writing development
Evaluation and language learning
with evaluative that-clauses expanded, the authorial visibility in stance expression
Empowering ideology decreased. These changes indicate signs of agency and suggest that the development of
Longitudinal learner corpus evaluative language resources is an organic, dynamic process. Overall the study points to the
Evaluative meaning making value of developing and exploring longitudinal learner corpora for studying language
learning in general and development in evaluative language in particular. Taking an approach
to treating learner language in its own right, the study further contributes to a small but
growing body of research based on an empowering ideology and to the empirical literature
on evaluation and language learning. Implications for writing instruction are considered.
© 2018 Elsevier Ltd. All rights reserved.

1. Introduction

The language of evaluation has been studied in academic discourse under various labels such as evaluation (Hunston, 1989,
2011; Hunston & Su, 2017; Hunston & Thompson, 2000), stance (Biber & Finegan, 1989; Conrad & Biber, 2000), metadiscourse
(Hyland, 2005) and appraisal (Martin & White, 2005). The importance of evaluative language has aroused research interest in
its use in student writing (e.g., Hong & Cao, 2014; Jiang, 2015; P.; Thompson, 2012) and motivated scholarly attempts to
investigate differences between second language (L2) and first language (L1) learner writing. There are, however, few studies
which examine how students of English develop their linguistic resources over time to express evaluation.
This study, based on a longitudinal corpus, seeks to uncover how university students go about developing language re-
sources based on that-clauses to convey evaluative meaning. The use of that-clauses has been found to be prevalent in ac-
ademic writing: this linguistic feature serves an important means for expressing stance, defined here as the personal feelings,
attitudes and value judgments conveyed by the speaker or writer (Biber, 2006; Biber, Johansson, Leech, Conrad, & Finegan,
1999; Groom, 2005; Hyland & Tse, 2005a, 2005b; Jiang & Hyland, 2015; Parkinson, 2013). Empirically, this study shows
that the developmental trajectories for evaluative that-clauses are not straightforward and that development in the use of
evaluative language is more than just an increase in the employment of linguistic features over time. Learners also constantly

* Corresponding author.
E-mail address: chaumenghuat@yahoo.co.uk (M.H. Chau).

https://doi.org/10.1016/j.jeap.2018.11.007
1475-1585/© 2018 Elsevier Ltd. All rights reserved.
 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33 23

engage in choice making in the selection of linguistic resources for communication as they develop more experience with
language over the course of their studies. Methodologically, the study demonstrates how learner language can be studied in
its own right without involving any comparison with native speaker data or an external point of reference.

2. Literature review

In English for Academic Purposes (EAP) studies, more and more scholars emphasize the importance of raising learners'
awareness of linguistic markers of stance (Charles, 2006; Hong & Cao, 2014; Jiang & Hyland, 2016; Thomas, Thomas, &
Moltow, 2015; Zhang, 2015). For explicit instruction, for example, EAP teachers would find it helpful to be aware of the
use of linguistic markers of stance in learner writing and how the use of these linguistic markers changes over time. Such
awareness can help EAP teachers to conceptualize academic writing development, which is necessary for selecting the focus
of explicit instruction. As will be seen in this review, while past studies have offered insights into the use of stance markers in
learner writing, development in the use of evaluative language remains to be an area largely overlooked in the literature.

2.1. Evaluative language in L2 writing

In corpus-based studies of linguistic features in L2 academic writing, there is considerable interest in comparing L2 learner
texts against the texts written by expert academic writers. Hewings and Hewings (2002), for example, examined a corpus of
published journal articles from the field of Business Studies and a corpus of MBA dissertations written by non-native speakers
of English in their study of the use of clauses with an anticipatory it and extraposed subject (as in ‘It is interesting to note that
no solution is offered’). Parkinson (2013), on the other hand, compared the use of that-complement clauses in L1 Zulu students'
reports and in journal articles for the expression of academic values. Studies of this kind usefully highlight the differences
between native and non-native academic writing. The observed differences, however, may be understood in the light of the
context in which academic texts are produced. Arguments may arise as to how unrealistic it is to compare L2 learners' writing
against published research articles. Hyland and Milton (1997), for example, pointed out that journal articles have gone
through rigorous reviews and revisions prior to publication. This is certainly not the case with many L2 texts produced as
reports or dissertations at university level.
Another strand of research is to compare L2 learners with L1 learners, with learner factors (e.g., age and education level)
and text characteristics (e.g., mode, genre and production condition) controlled. Following this approach, Adel € (2006), for
example, compared the use of metadiscourse in L1 Swedish university students' essays with those written by American and
British native speaker (NS) university students. Jiang (2015), on the other hand, compared the use of noun that-complement
clauses in argumentative essays written by L1 Chinese learners and in those produced by American NSs. In another study,
Staples and Reppen (2016) compared the use of eight lexico-grammatical items in English for stance expression in the writing
of two groups of L2 learners (Chinese and Arabic) and in the writing of NSs at a US university. The findings from these studies
are generally consistent. First, L2 learners' writings exhibit some linguistic features characteristic of conversation (Gilquin &
Paquot, 2008; Parkinson, 2013). Second, compared with L1 learners, L2 learners tend to use a more restricted range of lin-
guistic items (Hyland & Milton, 1997; Jiang, 2015; Parkinson, 2013; Staples & Reppen, 2016) and overuse high-frequency
linguistic items (Biber & Reppen, 1998; Larsson, 2016; see also; Granger, 1998).
The rationale underlying the two approaches featured in previous studies is to uncover the extent to which L2 writing
approximates expert-like or native-like usage. However, as Werner (2017) observes, the extent of approximation largely
depends on the norm, or choice of norm, against which learner language is compared. If the reference point is different, the
result of comparison would be different.
Further, from the perspective of multilingualism, it can be expected that many L2 learners simply do not use evaluative
language the same ways as their native-speaker monolingual counterparts. For L2 learners that have different exposures to
English, “the next language learned is not a totally new entity but an additional component in a multilingual system in which the
new system interacts with the components acquired earlier” (de Bot, Lowie, Thorne, & Verspoor, 2013, p. 201). It seems safe to
assume that their meaning-making resources would differ from those of their monolingual counterparts (see, e.g., Cook, 2007).
One important way to address concerns of the kind noted above, we argue, is to conduct longitudinal research on the same
learners and observe changes in language use based on longitudinal evidence. All too often, the frequency of linguistic
features observed in a corpus based on cross-sectional data, as opposed to longitudinal data, only shows learner performance
at a particular point in time. If a major goal of writing instruction and research is to understand students' developmental
writing practices, including their choices in language use, a longitudinal design would be essential.

2.2. Language development and L2 writing

Many studies on language development focused on complexity, accuracy and fluency (e.g., Biber, Gray, & Staples, 2016;
Knoch, Rouhshad, Oon, & Storch, 2015; Larsen-Freeman, 2006; Mazgutova & Kormos, 2015; Parkinson & Musgrave, 2014;
Yoon, 2017). These studies have offered valuable insights. The development of complexity, accuracy and fluency has been
suggested to be a dynamic process that has a non-linear trajectory (Larsen-Freeman, 2006; Verspoor, Schmid, & Xu, 2012).
During the developmental process, learner performance may progress or regress and sub-systems of language interact with
each other (de Bot et al., 2013; Larsen-Freeman, 2006). Additionally, some linguistic features are shown to be more reliable
24 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33

measures of language development than others. For instance, studies show that phrasal complexity features are better
predictors of academic writing development than clausal complexity features (Biber, Gray, & Poonpon, 2011; Parkinson &
Musgrave, 2014; Staples, Egbert, Biber, & Gray, 2016). Whereas the body of longitudinal research on complexity, accuracy
and fluency continues to grow, few studies in this line of inquiry directly focus on how learners go about developing linguistic
resources for meaning making.
In fact, a large body of SLA and learner corpus research is cross-sectional in nature (see Chau, 2012; Hasko, 2013; Ortega &
Iberri-Shea, 2005). In SLA, for example, when longitudinal studies are conducted, the sample of participants tends to be small.
Also, there is no strict control over the writing task (e.g., Morton & Llinares, 2018). As topics and individual differences can
have a significant effect on writing performance (Lowie, van Dijk, Chan, & Verspoor, 2017; Yang, Lu, & Weigle, 2015), it is
particularly important to track changes in language use among the same group of students with a controlled writing task.
This article reports on a longitudinal corpus study of evaluative that-clauses in Chinese EFL argumentative writing based
on a common task. The questions to be addressed in the article are as follows:

1) How does the frequency of evaluative that-clauses in learner writing change over time?
2) How does the degree of variability within evaluative that-clauses change over time?
3) Does the change in the use of language resources based on that-clauses impact the way evaluative meanings are expressed
in the text?

3. Method

3.1. Learner profile

A total of 158 Chinese EFL learners in their first semester studying at a Mainland China university participated in this study.
Half of them were female and the other half male. They were all first-year students enrolled in a university course known as
College Comprehensive English. The participants were classified as low-intermediate learners of English based on their
performance in a standardized test as a sub-component of their college entrance examination. All the participants reported
speaking Chinese as their first language and having learnt English as a compulsory subject in their secondary school for at
least six years. Only one participant had experience studying as a visiting student in Australia for two months. During the
study, all the participants enrolled in the English course taught by the same instructor. The course focused on listening,
speaking, reading and writing skills for general purposes. During the course, the participants completed a variety of learning
tasks, such as extensive reading, gap-filling exercises and making sentences with the words and phrases that had appeared in
the teaching materials. No feedback or instruction was given to the students on the use of that-clauses in the course, which
was conducted by the first author of the article. None of the participants majored in linguistics or reported enrolling in other
English courses. Courses other than English were taught in Mandarin.

3.2. Corpus

The longitudinal corpus in the present study consists of timed argumentative essays written by the participants as
described above. These essays were completed within sixty minutes under examination conditions. The participants per-
formed the same writing task at four points in time, with a three-week interval in between. The topic of the controlled writing
task is:
The Internet brings more harm than good to students. To what extent do you agree or disagree with this statement?
The corpus consists of 632 texts, with a total of 138,942 words. The details are described in Table 1. Each text is assigned a
number and a letter for reference. The number range is 001e158, reflecting the number of students (158) who participated in
the research. Four letters (i.e., a, b, c and d) were used to indicate the four different points in time (i.e., Time 1, Time 2, Time 3
and Time 4) when the text was produced. Thus, 001a and 001b, for example, refer to the texts written by the same student
(001) at Time 1 and Time 2, and 023c and 095d refer to the texts written by different students (023 and 095, respectively) at
Time 3 and Time 4.

3.3. Analytical framework

Among the most structurally complex grammatical devices used to express stance in English is the that-clause, a gram-
matical structure consisting of two components: one presenting a proposition and the other presenting a stance towards that

Table 1
Composition of longitudinal corpus.

Time 1 Time 2 Time 3 Time 4

Total words 33, 386 33, 717 34, 533 37, 306
Average text length 211 213 219 236
 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33 25

proposition (Biber et al., 1999, p. 969). This grammatical structure has great evaluative potential in that the evaluated entity,
evaluative stance and evaluative source embedded can be controlled to construct evaluative meanings (Charles, 2006; Hyland
& Tse, 2005a, 2005b; Parkinson, 2013), as illustrated in the following sentences taken from the longitudinal learner corpus:

(1) And we [Evaluative source] can't ignore the fact [Evaluative stance] that internet has been brought our a lot of
convenient [Evaluated entity]. (033d)
It [Evaluative source] is very important [Evaluative stance] that students can study in the Internet [Evaluated entity].
(006d)
I [Evaluative source] believe [Evaluative stance] that the Internet can help us finish the work quickly [Evaluated entity]. (051d)

This structure creates space for writers to overtly express their evaluative stance by offering them a range of predicates,
including verbs (2), nouns (3) and adjectives (4) (Biber et al., 1999; Hyland & Tse, 2005b):

(2) So some people think that the Internet brings more harm than good to students. (139d)
So, I insist that the development of the internet get more benefits than harm to students. (071d)
A growing number of people realize that the Internet is indispenable to a life. (067d)
(3) It is no doubt that the Internet has changed our life and made our life more and more convenient. (063c)
And we can't ignore the fact that internet has been brought our a lot of convenient, everyone is closely related to internet.
(033d)
I can't hold the same opinion that the Internet brings more harm than good to students. (148d)
(4) It is clear that the Internet can help us solve this problem. (120c)
Last, it is most important that education can reap the benefits of the Internet. (153d)
It is undeniable that Internet is essential our life. (013d)

The choice of predicate forms in evaluative that-clauses varies across registers. While verb predicates are extremely
common in conversation and relatively rare in academic prose, noun predicates are extremely rare in conversation and
moderately common in academic prose (Biber et al., 1999, p. 984).
The grammatical subject of the predicates indicates the source of evaluation. The subject can be moved to the position
following the original predicate or be replaced by it as subject (Hyland & Tse, 2005b). The possibility of extraposition allows
writers to choose the attribution of stance, which can be presented in an explicit, implicit or ambiguous manner. Writers can
also choose to attribute the stance to themselves to show high level of authorial visibility or conceal the source of stance to
create a sense of objectivity.

3.4. Analytical procedures

Instances of evaluative that-clauses were all retrieved from the corpus with the assistance of AntConc 3.4.4 (Anthony,
2016), including cases of zero that:

(5) I think internet is one of the most important things. (001a)

Experts said internet have all kinds of informations. (052b)
I feel the Internet library is very good. (113d)

Zero that-clauses are considered for two reasons. First, L2 learner writing is shown to possess characteristics of con-
versations (Gilquin & Paquot, 2008), in which there is a heavy reliance on that-complement clauses to mark stance,
typically with that omitted (Biber et al., 1999, p. 984). Second, 46% of verb that-clauses in the study omitted the comple-
mentizer that. To retrieve all the zero-that instances, we searched in the corpus for the most commonly used verbs in zero-
that structures.
After that-clauses were all extracted from the corpus, a manual analysis was conducted to exclude instances that are not
used as stance markers. It should be emphasized that only those evaluative that-clauses that occur at least in two texts in the
respective sub-corpus were included for analysis. This practice is to eliminate potential idiosyncrasy. The remaining clauses
were further categorized and coded according to the degree of explicitness of attribution (i.e., explicit, implicit or ambiguous)
with MAXQDA Pro 12. The resulting frequency of that-clauses at Time 1 and Time 4 was then compared with the log-
likelihood and effect size calculator (accessible at http://ucrel.lancs.ac.uk/llwizard.html).

4. Results

The results show a significant increase in the total number of evaluative that-clauses (p < .05) (see Table 2) from Time 1 to
Time 4. While there is a statistically significant increase in the number of noun that-clauses (p < .01) and adjective that-
clauses (p < .01), the increase in the number of verb that-clauses is not statistically significant (p > .05).
26 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33

Table 2
The number of evaluative that-clauses.

Time 1 Time 4 Significance test (p value)

Verb-that 332 375 0.886
Noun-that 28 68 0.000**
Adjective-that 6 27 0.000**
Total number 366 470 0.045*

Note: In the present study, we calculated log-likelihood value with assistance of a spreadsheet developed by Paul Rayson, available at http://
ucrel.lancs.ac.uk/llwizard.html.
*p < 0.05.
**p < 0.01.

4.1. Predicate forms of evaluative that-clauses

The normalized frequency of evaluative that-clauses increased from 109.6 to 126.0 per 10,000 words (see Fig. 1). The
normalized frequency of noun that-clauses increased consistently whereas the trajectory for verb that-clauses and adjective
that-clauses went through some fluctuations. The fluctuation is most obvious in the use of verb that-clauses. The frequency of
verb that-clauses first decreased at Time 2, then increased at Time 3 and decreased again at Time 4. The overall frequency of
all evaluative that-clauses has a similar trajectory with that of verb that-clauses.
As can be seen from Fig. 2, the proportion of different types of that-clauses underwent changes over time. On the whole,
the proportion of verb that-clauses, for example, decreased (from 90.7% to 79.8%) whereas the proportions of noun that-
clauses and adjective that-clauses increased (from 7.7% to 14.5% and from 1.6% to 5.7%, respectively). The trajectories for verb
that-clauses and noun that-clauses appear to be straightforward. The trajectory for adjective that-clauses, however, slightly
decreased after it reached its peak at 6.0%. The findings based on Figs. 1 and 2 indicate that the development of linguistic
resources associated with the use of that-clauses in academic writing is dynamic and complex (cf., Larsen-Freeman, 2006). It
is dynamic because the overall trajectory is not straightforward and complex because the interaction between and among
types of that-clauses changes across different points in time.
A closer look at the data reveals more interesting findings about students' development in academic writing. As can be
seen from Table 3 which presents findings concerning lexis co-occurring with that-clauses at Time 1 and Time 4, the learners
first relied heavily on a small set of verbs (e.g., think, believe, say) and nouns (e.g., opinion, no doubt) as the controlling words to
express stance at Time 1. Over time, their use of controlling verbs as part of their linguistic resources in academic writing has
expanded. This is evidenced in the longer list of both verbs and nouns used at Time 4, where a greater number of different
types of verbs and nouns were observed. At this later point in time, too, adjectives were seen to form part of the learners'
linguistic repertoires to express stance with that-clauses:

(6) I believe it is important that we tell him how to do. (155d)

It is well-known that the delveopment of the Internet is a smoybol of progress. (126d)
Although the Internet brings more harm to students, it is undeniable that the Internet can also brings some good for
students. (023d)

In addition, the choice of lexis was becoming more academic-like. Words like tell and remember, for example, were
observed in the writing at Time 1, but they were no longer used with that-clauses at Time 4 (7); At Time 4, new words such as
realize and suggest became the preferred controlling words for that-clauses (8):

Fig. 1. Normalized frequency of grammatical stance markers (per 10, 000 words).
 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33 27

Fig. 2. Percentage of different that-clauses.

(7) She tell me that Internet let she know much knowledge about world that is the book can't give she. (040a)
So in order to change his mind, we always tell him that he shoud stop to doing this, don't playing the computer game so
crazy. (065a)
I still remember that I had been lost in many mate problems. (051a)
(8) Today, a growing number of people realize that internet is indispensable to their life. (046d)
But people are beginning to realize that the Internet has exerted a negative impact on our life. (048d)
Some people suggest that the Internet brings us a lot of benefits. (009d)

4.2. Source of stance in evaluative that-clauses

In argumentative writing, the source of stance plays a crucial role in persuading readers. The source of stance can be
concealed to create a sense of objectivity or be highlighted to add credit. Usually, the grammatical subject of the main clause
holds responsibility for the stance towards the proposition made in a that-complement clause. The subject can also be
extraposed to the position following the original predicate or be removed.
In the study, the proportion of that-clauses with explicit stance sources decreased over time, from 95.4% at Time 1e79.1%
at Time 4 (see Fig. 3). In contrast, the proportion of that-clauses with implicit and ambiguous stance sources increased. The
curves for implicit and ambiguous that-clauses are not straightforward.

Table 3
Lexis controlling that-clauses (Freq.
2).a

Rank Time 1 Time 4

Verb Freq. Noun Freq. Verb Freq. Noun Freq. Adjective Freq.
1 think 238 opinion 12 think 205 no doubt 16 important 6
2 believe 18 no doubt 8 believe 24 opinion 14 well known 5
3 say 18 say 18 fact 12 /well-known
4 agree 13 said 12 conclusion 6 undeniable 4
5 hope 11 know 11 reason 3 obvious 2
6 said 9 agree 7 view 3 harmful 2
7 see 9 consider 7 idea 2
8 know 7 found 6
9 disagree 6 saying 6
10 find 5 known 5
11 feel 4 feel 5
12 remember 3 insist 5
13 says 3 realize 4
14 tell 3 see 4
15 admit 2 thinks 3
16 pledge 2 deny 3
17 thinks 2 denying 3
18 estimated 2
19 means 2
20 reported 2
21 says 2
22 suggest 2
a
Note: Only controlling lexis that occurs more than once and across two different texts was considered.
28 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33

Fig. 3. Explicitness of stance attribution (in percentage).

One surprising observation is that while the frequency of noun that-clauses and adjective that-clauses increased, some of
these syntactic structures attribute stance explicitly by using the personal pronoun as the subject of the superordinate clause:

(9) I hold the view that the Internet brings more benefits to us. (102b)
But I hold the belief that the Internet brings more harm than good to students. (027c)
I draw a conclusion that the Internet brings more good than harm to students. (097d)

This is a feature that emerged at a later point of development (i.e., Time 2, Time 3 and Time 4). At Time 1, all the twenty
noun that-clauses conceal the source of stance. But at Time 2, four noun that-clauses attribute stance to personal pronouns,
accounting for 19.2% of all the twenty-six noun that-clauses. The percentage dropped to 8.2% at Time 3, where four out of
forty-nine noun that-clauses made use of the personal pronoun as the subject of the superordinate clause. This percentage,
however, increased at Time 4, where nine out of fifty-six noun that-clauses attribute stance to personal pronouns, amounting
to 16.1% of the total number of noun that-clauses. This suggests how the learners are diversifying their linguistic means to
make similar meanings.

4.3. Evaluative meaning making with that-clauses

In written communication, writers acknowledge alternative views so that they can create space for negotiation of their
stance with the reader (G. Thompson, 2001). In the present study, we find that learners presented conflicting views by
manipulating the grammatical subject of the superordinate clauses. At Time 1, for instance, students used the third person
pronoun to bring in alternative voices:

(10) People also worry the compute is bad for their eyes and body and so on. So more and more people think that the Internet
brings more harm than good to student. But I still disagree with this opinion …. (007a)
Someone think the Internet brings more harm than good to students, but I don't think so. (018a)
Some people say that Internet brings harm to teenagers. (152a)

In these examples, included in the text is a voice that could be attributed to the reader. What the writers are doing is to
bring in alternative views and then contradict them. There are also examples that writers used a third-party source to support
their own stance:

(11) I remember Hawkim says that human depend on the AI. I agree him opinion, because human difficult to give up
Internet. (003a)
 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33 29

In my opinion, the internet brings more good than harm to students. For example, University students do homework use
internet more than writing without anything. Experts said internet have all kinds of informations. (052b)
A study found that some students play computer games day and night, as a result, they fail to their exam. So we must use
Internet in a true way. (079c)

From these examples, it can be seen that the students select different grammatical subjects to express their evaluative
meanings. To acknowledge alternative views, they use subjects such as someone and some people, whereas to support their
own stance, they use grammatical subjects such as Hawkim, expert and a study that could give credit to the source of in-
formation. This use of evaluative language also appears at Time 4:

(12) Someone think Internet makes us corrupt. But I think Internet is a bridge. (001d)
A great number of people think that the Internet will harm their children, but the Internet must be more and more
important that no one can leave it. (016d)
But other people think that the Internet brings more harm than good to students. A growing number of student feel that the
computer online games indispensable to them. It is estimated that more and more students are addicted to online games.
(126d)

At Time 1 and Time 2, the nouns controlling that-clauses were mostly those that represent ideas and beliefs such as
opinion, belief, point and thinking:

(13) I disagree with this opinion that the Internet brings more harm than good to students. (096a)
It is my thinking that Internet give great help for my study and my job in the future. (153a)
A group of people who have the same interests that have the stronger belief that you are able to do things well. (087b)

In contrast, at Time 4, learners used nouns that represent objectivity and evidence to control that-clauses:

(14) Despite the fact that the internet has some negative aspects, but we have to recognize it let our life and study better.
(008d)
Now, it's a truth that the human can't live in a world without the Internet. (088d)
It is hard to deny the fact that Internet becomes more and more important in our life. (156d)

Overall, the argumentative writing produced by the participating students is found to be increasingly conforming to
conventions of academic writing over time. The way that stance is expressed has become less explicit, though the process of
change is not straightforward. While the percentage of explicit stance expression decreased from Time 1 to Time 4, the
development went through some fluctuation in between. More interestingly, the use of noun that-clauses has increased over
time, suggesting that students now place greater emphasis on constructing objectivity and evidence in persuading their
readers.

5. Discussion

The results suggest that the changes in both the frequency and proportion of evaluative that-clause and its structural
variants are complex and dynamic. The fluctuations observed in the trajectories indicate that development in the expression
of evaluation is not only about an increase in the use of specific linguistic features over time. As learners develop more
experience with language over the course of their studies, the development is also about making choices in the selection of
linguistic resources for communication. Similarly, the degree of explicitness to which evaluative meanings are made has been
observed to change across different points in time in a rather non-linear manner, suggesting learners' fluid use of their
developing linguistic resources that are constantly adjusted and reorganized to meet the demands of written communication.
As far as co-occurring lexis is concerned, we found a “lexical teddy bear” tendency in our study (Hasselgren, 1994, p. 237):
The learners initially depended on a small set of verbs and nouns as the controlling words to express stance. This finding
seems to be in line with previous research which suggests that learners tend to rely heavily on a small set of linguistic items
(Biber & Reppen, 1998; Hyland & Milton, 1997; Larsson, 2016; Parkinson, 2013; Staples & Reppen, 2016). Over time, however,
not only was a wider range of verbs and nouns observed, adjectives as a new category of linguistic resources that co-occur
with that-clauses were also seen to occur in their writing. Also, while Parkinson (2013) finds that controlling verbs for
evaluative that-clauses in learner writing are more reflective of vocabulary use in conversation, this study observed an in-
crease in the use of controlling verbs over time that are more characteristic of academic writing. The choice of lexis among the
learners in this study, as noted above, was becoming more academic-like. These findings further confirm the value of lon-
gitudinal research: while previous cross-sectional research such as Hyland and Tse (2005b) and Parkinson (2013) is helpful in
identifying ‘problematic’ use of evaluative that-clauses in L2 learner writing, the present study sheds light on how learners'
use of this stance marker changes over time and reveals signs of agency in academic writing development.
30 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33

It must be pointed out that although the overall frequency of evaluative that-clauses increased over time, the frequency of
some structural variants fluctuated across different points in time. This is particularly noticeable in the case of verb þ that-
clauses. The frequency of noun and adjective þ that-clauses also went through some fluctuation, but to a lesser extent. These
findings provide evidence for a dynamic perspective on language development (de Bot et al., 2013; Larsen-Freeman, 2006).
Further, verb that-clauses are arguably acquired in earlier points of development as a linguistic feature of conversation (see
also Biber & Gray, 2010). Over time, learners develop alternative language resources (other than verb that-clause) for stance
expression.
Meanwhile, in noun that-clauses, placing personal pronouns as the grammatical subject of the superordinate clause seems
to be an emerging feature of Chinese student writing in this study. This is, to some extent, consistent with Jiang’s (2015)
finding that Chinese EFL learners tend to be personally involved in expressing their stance:

(15) I have an idea that the Internet brings more good than harm to student. (104d)
I have an opinion that we not only make the most of the Internet resours but also reduse students are influented the
harmful resours. (126d)
So, I insist on my view that the Internet bring more good than harm to student. (145d)

The learners were also observed at Time 1 to use the grammatical subject of the superordinate to develop their argument.
They acknowledged possible alternative views and contradicted them or supported their own stance by quoting from third-
party sources (see examples in Section 4.3). It is possible that learners at undergraduate level already have these meaning-
making resources in their L1. As Bardovi-Harlig (2014, p.128) points out, adult ESL/EFL learners have access to “the full range of
semantic concepts from their previous linguistic and cognitive experience”. Further research on how undergraduate students
make use of their previous experience in stance expression is warranted, however.
The explicitness of meaning expressed in the student writing is found to have decreased over time as the range of linguistic
resources expanded. At Time 1, for example, all stances were explicitly or implicitly attributed. At Time 4, however, the
learners used grammatical devices that conceal the source of stance. The instances of implicit attribution also increased in
terms of normalized frequency. Such changes indicate that learners began to refrain from making subjective judgment and
started to construct language that reflects greater objectivity. As academic writing has been argued to be less explicit than
conversation (Biber & Gray, 2010), the reduced explicitness indicates that the students were over time producing texts that
are more characteristic of formal writing. This finding complements the research by Hyland and Tse (2005b) on the use of
evaluative that-clauses in abstracts from postgraduate dissertations and journal articles. They found that L2 postgraduate
students were more reluctant to use personal voice than scholarly writers. Taken together, all this suggests that in learning
how to evaluate, student writers appear to progress from being relatively explicit in stance expression to becoming less
explicit as they go through their undergraduate years and into postgraduate-level study. For some of them who go on pur-
suing an academic career in their later lives, they may start developing or showing a more authorial voice or authorial
presence in their writing. This hypothesized developmental progression of course remains to be confirmed in future research.
A number of limitations of this study may be noted. First and most obvious of all, the period of the study only spanned 11
weeks of one semester of study. Ideally, a longitudinal study of language development would span over a year or two for the
researcher to observe substantial changes (see, e.g., Ortega, 2003). For practical reasons, however, this was not possible in this
study. Nevertheless, in the light of the findings of our study, it is important to note that even a study with as short a duration
as 11 weeks like the present one surprisingly yielded results that reveal noticeable changes in the use of linguistic resources
over time. This points to the possibility then that the duration or time that is required to observe linguistic changes may vary,
depending on the linguistic features under examination. As linguistic items grow organically, as evidenced in this study, it can
be argued that there is no ‘standard’ timeframe that applies to all linguistic investigations.
Second, the present study is essentially a corpus study investigating patterns of linguistic resources used over time to
express evaluative meaning. It is beyond the scope of this study to consider such issues as the role of input or feedback in
changing the ways language is used, important though such factors are. Neither has the concern been addressed of individual
€
student development and variability in individual performance (see, e.g., Adel, 2015). Future research might consider this and
other issues such as whether and to what extent there would be differences in results between using the same task and
different tasks over time in studies that investigate language development.
It should be pointed out that the repeated-task design used in this study reflects the need for a controlled writing task for a
common group of students, who were also interested in knowing their changing language use over time. As noted before, the
use of different topics can have a significant effect on writing performance and may affect the extent of comparability of texts
written by the same students over time. Future research could potentially build upon the findings from this study by using a
similarly controlled method to track learners' written (or spoken) output over longer periods of time and by observing
whether learners' use of other features of academic discourse in their language output over time follows a similarly non-
linear trajectory.
A final important point to make is a direct response to what a reviewer has commented:
… as an EAP writing instructor, I find it somewhat surprising that you didn't observe or comment on changes in ac-
curacy over time (i.e. where these structures were being formed more or less in a grammatically correct way over time).
Perhaps this is beyond the scope of the study, but I think it is still worth mentioning.
 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33 31

We must first make it clear that not only do we welcome this comment; we appreciate having the opportunity to engage in
this dialogue. The present study focuses on students' development in evaluative meaning making: it is indeed not concerned
with changes in accuracy in the student writing (e.g., does the writing become more grammatically correct over time?). This is
not because it is beyond the scope of our study; it has to do with the position we adopt. Our position is that learner language is
a variety of human language that is to be respected in its own right and that language learning is an organic process that
requires no comparison with an external point of reference, idealized or otherwise (Chau, 2015). Space concerns do not allow
for a good discussion of this issue; for now, we would just note that learner language is not an imperfect version of native-
speaker competence or performance. It is not even a ‘thing’: It is a process (see also García & Wei, 2014; Larsen-Freeman,
2006).
Further, like World Englishes or English as a (multi)lingua franca (see Jenkins, 2018; Mauranen, 2018; Seidlhofer, 2012),
how learner language is to be studied or perceived, we argue, should go beyond superficial concerns for mere linguistic
features: issues of ideology and identity need to be considered, so are learners' intentions and capabilities. A study based on
accuracy would inevitably involve a specific external norm, which is against the whole idea of treating learner language in its
own right (Selinker, 1972). We hope that the position we hold, together with the methodology we use (studying longitudinal
learner data without recourse to native speaker data, involving no comparison with an idealized competence and embracing
interpretations of findings based on what is present in the data, not based on a monolingual ideology), represents a conscious
choice: a choice based on an empowering ideology that shows our commitment to what Ortega (2018, p. 75) has called a
“project of reconceptualizing linguistic development under non-essentialist lenses”. We exemplify both the position and
approach in this study, and we hope that more studies in EAP, learner corpus research, SLA, second language writing and ELT/
TESOL will be conducted with a similar respect for learners and learner language (i.e., involving no comparison with an
external point of reference and reflecting no introduction of monolingual bias). EAP instructors who are interested in knowing
and developing alternative ways of teaching language without accuracy in mind may refer to Pallotti (2017) for useful insights
and examples.

6. Conclusion

This article has reported on a longitudinal study of undergraduate students' argumentative writing, highlighting the value
of developing and exploring longitudinal learner corpora for studying development in evaluative language. A number of
observations have been made. First, the results show that the frequency of evaluative that-clauses increased significantly and
that the range of the lexical words co-occurring with that-clauses expanded. Second, there is considerable variability in the
use of evaluative that-clauses. Specifically, the proportion of verb that-clauses decreased whereas the proportion of adjective
and noun that-clauses increased. Further, the changing use of that-clauses has an impact on the evaluative meanings
conveyed by the learners. As discussed above, the explicitness of stance marking decreased as the learners used more ad-
jective that-clauses and noun that-clauses. The manner in which learners expressed their stance became more characteristic
of academic writing. Overall then, it seems clear that the developmental trajectories for evaluative that-clause and its
structural variants were not straightforward, suggesting that the development of evaluative language is an organic, dynamic
process, with lexis and grammar interacting with each other to co-construct meanings.
What does all this suggest to the writing teacher? First, an appreciation for viewing language development as a
dynamic process is important. Both the increase and the decrease in the use of specific linguistic items might not be
simply taken to suggest one's language ability. In assessing students' academic writing, for example, the complex
interplay between and amongst different language resources might require closer attention. As shown in this study, the
decreased use of a specific linguistic item can be a matter of choice rather than a matter of regression. In such a case,
simply opting for more conventional forms of corrective feedback might not always be the best option. In fact, this would
enact and reinforce a monolingual worldview, as briefly noted above. Second, the changing use of linguistic items as the
controlling lexis for that-clauses suggests that L2 learners are expanding their evaluative meaning-making resources. It
would be helpful for instructors to raise their students' awareness about what makes appropriate choices in different
contexts of language use and expose students to linguistic choices that are characteristic of academic writing in specific
contexts.
The discussion in this article raises a central issue for EAP instructors: What are the expected linguistic outcomes of EAP
teaching and how to assess such outcomes? In this study, for example, when changes in language use are considered based on
averaged group performance, a complex picture of language development emerges. Taking into account individual differ-
ences among students in the classroom would further complicate the situation. On the one hand, imposing a unified, fixed
standard of assessment would show little respect for learners; on the other hand, assessing the learning outcomes on an
individual basis would almost be impractical. There is a further concern about meeting institutional demands for a target-
oriented approach to language assessment if we are truly committed to treating learner language in its own right. There
are no easy answers to these concerns, but we hope that the discussion here invites a critical reflection upon the nature and
goals of language learning, and encourages a reassessment of our practices in EAP instruction.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.jeap.2018.11.007.

32 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33

References
€
Adel, A. (2006). Metadiscourse in L1 and L2 English. Amsterdam/Philadelphia: John Benjamins Publishing Company.
€
Adel, A. (2015). Variability in learner corpora. In S. Granger, G. Gilquin, & F. Meunier (Eds.), The Cambridge handbook of learner corpus research (pp. 401e422).
Cambridge: Cambridge University Press.
Anthony, L. (2016). AntConc (Version 3.4.4) [Computer Software]. Tokyo, Japan: Waseda University. Available from: http://www.laurenceanthony.net/
software.
Bardovi-Harlig, K. (2014). Documenting interlanguage development. In Z.-H. Han, & E. Tarone (Eds.), Interlanguage 40 years later (pp. 127e146). Amsterdam:
John Benjamins.
Biber, D. (2006). Stance in spoken and written university registers. Journal of English for Academic Purposes, 5(2), 97e116.
Biber, D., & Finegan, E. (1989). Styles of stance in English: Lexical and grammatical marking of evidentiality and affect. Text, 9(1), 93e124.
Biber, D., & Gray, B. (2010). Challenging stereotypes about academic writing: Complexity, elaboration, explicitness. Journal of English for Academic Purposes,
9(1), 2e20.
Biber, D., Gray, B., & Poonpon, K. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? Tesol
Quarterly, 45(1), 5e35.
Biber, D., Gray, B., & Staples, S. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied
Linguistics, 37(5), 639e668.
Biber, D., Johansson, S., Leech, G., Conrad, S., & Finegan, E. (1999). Longman grammar of spoken and written English. Harlow: Pearson Education.
Biber, D., & Reppen, R. (1998). Comparing native and learner perspectives on English grammar: A study of complement clauses. In S. Granger (Ed.), Learner
English on computer (pp. 145e158). London: Longman.
de Bot, K., Lowie, W., Thorne, S. L., & Verspoor, M. (2013). Dynamic systems theory as a comprehensive theory of second language development. In M. P. G.
Mayo, M. J. G. Mangado, & M. M. Adri an (Eds.), Contemporary approaches to second language acquisition (pp. 199e220). Amsterdam/Philadelphia: John
Benjamins Publishing Company.
Charles, M. (2006). The construction of stance in reporting clauses: A cross-disciplinary study of theses. Applied Linguistics, 27(3), 492e518.
Chau, M. H. (2012). Learner corpora and second language acquisition. In K. Hyland, M. H. Chau, & M. Handford (Eds.), Corpus applications in applied linguistics
(pp. 191e207). London: Continuum/Bloomsbury.
Chau, M. H. (2015). From language learners to dynamic meaning makers: A longitudinal investigation of Malaysian secondary school students’ development of
English from text and corpus perspectives. Unpublished PhD thesis. University of Birmingham.
Conrad, S., & Biber, D. (2000). Adverbial marking of stance in speech and writing. In S. Hunston, & G. Thompson (Eds.), Evaluation in text: Authorial stance and
the construction of discourse (pp. 56e73). Oxford: Oxford University Press.
Cook, V. J. (2007). Multi-competence: Black-hole or worm-hole for second language acquisition research. In Z. Han (Ed.), Understanding second language
process (pp. 16e26). Clevedon: Multilingual Matters.
García, O., & Wei, L. (2014). Translanguaging: Language, bilingualism and education. New York: Palgrave Macmillan.
Gilquin, G., & Paquot, M. (2008). Too chatty: Learner academic writing and register variation. English Text Construction, 1(1), 41e61.
Granger, S. (Ed.). (1998). Learner English on computer. London: Longman.
Groom, N. (2005). Pattern and meaning across genres and disciplines: An exploratory study. Journal of English for Academic Purposes, 4(3), 257e277.
Hasko, V. (2013). Capturing the dynamics of second language development via learner corpus research: A very long engagement. The Modern Language
Journal, 97(S1), 1e10.
Hasselgren, A. (1994). Lexical teddy bears and advanced learners: A study into the ways Norwegian students cope with English vocabulary. International
Journal of Applied Linguistics, 4(2), 237e260.
Hewings, M., & Hewings, A. (2002). “It is interesting to note that.”: A comparative study of anticipatory 'it' in student and published writing. English for
Specific Purposes, 21(4), 367e383.
Hong, H. Q., & Cao, F. (2014). Interactional metadiscourse in young EFL learner writing: A corpus-based study. International Journal of Corpus Linguistics,
19(2), 201e224.
Hunston, S. (1989). Evaluation in experimental research articles. Unpublished PhD thesis. University of Birmingham.
Hunston, S. (2011). Corpus approaches to evaluation: Phraseology and evaluative language. London: Routledge.
Hunston, S., & Su, H. (2017). Patterns, constructions, and local grammar: A case study of ‘evaluation’. Applied Linguistics. https://doi.org/10.1093/applin/
amx046.
Hunston, S., & Thompson, G. (2000). Evaluation in text: Authorial stance and the construction of discourse. Oxford: Oxford University Press.
Hyland, K. (2005). Metadiscourse: Exploring interaction in writing. New York: Continuum.
Hyland, K., & Milton, J. (1997). Qualification and certainty in L1 and L2 student's writing. Journal of Second Language Writing, 6(2), 183e206.
Hyland, K., & Tse, P. (2005a). Evaluative that constructions: Signaling stance in research abstracts. Functions of Language, 12(1), 39e63.
Hyland, K., & Tse, P. (2005b). Hooking the reader: A corpus study of evaluative that in abstracts. English for Specific Purposes, 24(2), 123e139.
Jenkins, J. (2018). The future of English as a lingua franca? In J. Jenkins, W. Baker, & M. Dewey (Eds.), The Routledge handbook of English as a lingua franca (pp.
593e605). Abingdon: Routledge.
Jiang, F. (2015). Nominal stance construction in L1 and L2 students' writing. Journal of English for Academic Purposes, 20, 90e102.
Jiang, F., & Hyland, K. (2015). 'The fact that': Stance nouns in disciplinary writing. Discourse Studies, 17(5), 529e550.
Jiang, F., & Hyland, K. (2016). Nouns and academic interactions: A neglected feature of metadiscourse. Applied Linguistics. https://doi.org/10.1093/applin/
amw023.
Knoch, U., Rouhshad, A., Oon, S. P., & Storch, N. (2015). What happens to ESL students' writing after three years of study at an English medium university?
Journal of Second Language Writing, 28, 39e52.
Larsen-Freeman, D. (2006). The emergence of complexity, fluency, and accuracy in the oral and written production of five Chinese learners of English.
Applied Linguistics, 27(4), 590e619.
Larsson, T. (2016). The introductory it pattern: Variability explored in learner and expert writing. Journal of English for Academic Purposes, 22, 64e79.
Lowie, W., van Dijk, M., Chan, H. P., & Verspoor, M. (2017). Finding the key to successful L2 learning in groups and individuals. Studies in Second Language
Learning and Teaching, 7(1), 127e148.
Martin, J. R., & White, P. R. R. (2005). The language of evaluation: Appraisal in English. London: Palgrave.
Mauranen, A. (2018). Conceptualising ELF. In J. Jenkins, W. Baker, & M. Dewey (Eds.), The Routledge handbook of English as a lingua franca (pp. 7e24).
Abingdon: Routledge.
Mazgutova, D., & Kormos, J. (2015). Syntactic and lexical development in an intensive English for Academic Purposes programme. Journal of Second Language
Writing, 29, 3e15.
Morton, T., & Llinares, A. (2018). Students' use of evaluative language in L2 English to talk and write about history in a bilingual education programme.
International Journal of Bilingual Education and Bilingualism, 21(4), 496e508.
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics,
24(4), 492e518.
Ortega, L. (2018). Ontologies of language, second language acquisition, and world Englishes. World Englishes, 37, 64e79. https://doi.org/10.1111/weng.12303.
Ortega, L., & Iberri-Shea, G. (2005). Longitudinal research in second language acquisition: Recent trends and future directions. Annual Review of Applied
Linguistics, 25, 26e45.
 D. Man, M.H. Chau / Journal of English for Academic Purposes 37 (2019) 22e33 33

Pallotti, G. (2017). Applying the interlanguage approach to language teaching. International Review of Applied Linguistics in Language Teaching, 55(4),
393e412.
Parkinson, J. (2013). Adopting academic values: Student use of that-complement clauses in academic writing. System, 41(2), 428e442.
Parkinson, J., & Musgrave, J. (2014). Development of noun phrase complexity in the writing of English for Academic Purposes students. Journal of English for
Academic Purposes, 14, 48e59.
Seidlhofer, B. (2012). Corpora and English as a lingua franca. In K. Hyland, M. H. Chau, & M. Handford (Eds.), Corpus applications in applied linguistics (pp.
135e149). London: Continuum/Bloomsbury.
Selinker, L. (1972). Interlanguage. International Review of Applied Linguistics, 10, 209e231.
Staples, S., Egbert, J., Biber, D., & Gray, B. (2016). Academic writing development at the university level. Written Communication, 33(2), 149e183.
Staples, S., & Reppen, R. (2016). Understanding first-year L2 writing: A lexico-grammatical analysis across L1s, genres, and language ratings. Journal of
Second Language Writing, 32, 17e35.
Thomas, D. P., Thomas, A. A., & Moltow, D. T. (2015). Evaluative stance in high achieving Year 3 persuasive texts. Linguistics and Education, 30, 26e41.
Thompson, G. (2001). Interaction in academic writing: Learning to argue with the reader. Applied Linguistics, 22(1), 58e78.
Thompson, P. (2012). Achieving a voice of authority in PhD theses. In K. Hyland, & C. Sancho-Guinda (Eds.), Stance and voice in academic writing (pp.
119e133). Basingstoke: Palgrave MacMillan.
Verspoor, M., Schmid, M. S., & Xu, X. (2012). A dynamic usage based perspective on L2 writing. Journal of Second Language Writing, 21(3), 239e263.
Werner, V. (2017). Adversative pragmatic markers in learner language: A cross-sectional perspective. Corpus Pragmatics, 1(2), 135e158.
Yang, W., Lu, X., & Weigle, S. C. (2015). Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and
judgments of writing quality. Journal of Second Language Writing, 28, 53e67.
Yoon, H.-J. (2017). Linguistic complexity in L2 writing revisited: Issues of topic, proficiency, and construct multidimensionality. System, 66, 130e141.
Zhang, G. (2015). It is suggested tha or it is better to? Forms and meanings of subject it-extraposition in academic and popular writing. Journal of English for
Academic Purposes, 20, 1e13.

MAN Deliang is a PhD student in the Faculty of Languages and Linguistics at the University of Malaya. His research interests include corpus linguistics,
academic writing and language development.

CHAU Meng Huat (PhD, Birmingham) is Senior Lecturer at the University of Malaya, where he coordinates the Master of Linguistics/M.A. (Linguistics), and
teaches and supervises research in Applied Linguistics. His research interests include corpus linguistics and educational linguistics.