Journal of English for Academic Purposes 7 (2008) 27e37

www.elsevier.com/locate/jeap

Evaluative language in peer review referee reports

Inmaculada Fortanet*
Dep. d’Estudis Anglesos, Facultat de Ciències Humanes i Socials, Universitat Jaume I, Avgda. Sos Baynat s/n, 12071 Castelló, Spain

Abstract

Most international journals and conferences currently use the peer review system to ensure the quality of their contributions.
Among the various types of peer review, the ‘‘blind’’ and the ‘‘anonymous’’ review seem to be the most common. Reviewers,
or referees, usually write reports anonymously to indicate to authors what they should change in their papers in order to meet
the requirements of journals or conferences, and to help editors to decide whether to publish the article or not. Despite their rele-
vance in the review process and their importance for researchers all over the world, as an academic genre referee reports have not
received the attention that might be expected, maybe due to their ‘‘occluded’’ genre status. This study presents an analysis of 50
referee reports from the fields of Applied Linguistics and Business Organisation, based on a taxonomy which combines formal and
functional features. It provides an overview of some of the most significant evaluative features of this genre, and their pragmatic
value as direct or indirect requests. It is hoped that this will facilitate the task of researchers who need to understand and interpret
these reports, as well as those who need to write them.
Ó 2008 Elsevier Ltd. All rights reserved.

Keywords: Academic writing; English for Academic Purposes; Evaluative language; Genre analysis; Peer review; Referee reports

1. Introduction

Peer review is a common practice in today’s research publication process. Papers submitted to academic journals
and conference proposals both undergo this selection procedure. The editors of journals send the contributions they
receive to a number of experts in the field, i.e., referees, who review the papers and send back their evaluative reports
to the editors. This process can be ‘‘open’’ (both parties know the other party’s identity), ‘‘anonymous’’ (the referees
know the author’s names, but the authors do not know the referee’s identity) or ‘‘blind’’ (neither party knows the iden-
tity of the other) (Gosden, 2001, p. 7). ‘‘Blind’’ and ‘‘anonymous’’ reviews are the types considered by many editors,
readers and authors to provide the best guarantee of quality, since referees may feel freer to criticise the articles. Ref-
eree reports bring together three actors in the review process: the reviewer, the submitting author, and the editor, who
acts as a mediator. The report has two functions depending on the reader: to help the editor determine whether to pub-
lish the paper or not by means of an evaluation that is also transmitted to the author, and to persuade the author to make
changes to render the paper suitable for publication in the journal. Thus, the reviewer becomes an advisor to the editor
and an evaluator/assessor to the author. From the author’s point of view, the referee report can clearly cause a certain

* Tel.: þ34 964729592; fax: þ34 964729261.

E-mail address: fortanet@ang.uji.es

1475-1585/$ - see front matter Ó 2008 Elsevier Ltd. All rights reserved.
doi:10.1016/j.jeap.2008.02.004
28 I. Fortanet / Journal of English for Academic Purposes 7 (2008) 27e37

amount of anxiety, as it may be seen as a face-threatening act (Brown & Levinson, 1987) which comments on the value
of the research. However, the document is also a request to improve the article in a way that meets the journal’s stan-
dards for publication, to which the author will have to respond satisfactorily.
As a teacher of English for research purposes to advanced non-native students, I have often been faced with the
difficulties that these newcomers to research encounter when dealing with the whole chain of review genres (guide-
lines for contributors, editors’ letters, answers to these letters, reports, etc.). Among these, referee reports are espe-
cially problematic, not only because they represent a highly conventionalised genre, but also an ‘occluded’ one
(Swales, 1996). In other words, as private and sometimes sensitive documents, access to them is typically restricted
to close friends and colleagues. In addition, it is not only novice authors who have difficulties with these genres, but
also experienced researchers who may need guidelines in order to write appropriate reports. The latter problem has
been somewhat overcome by manuals of writing for certain disciplines (Zobel, 1997), or by journals that provide
Guidelines for Reviewers (e.g., English for Specific Purposes, New Journal of Physics, or Marketing Science). This
issue has led some researchers to write ‘how-to-review’ articles which have been published in specialised discipline
journals or bulletins (Peters & Ceci, 1982; Radford, Smillie, Wilson, & Grace, 1999; Shebilske, 1997; Wright & Orbe,
2003).

1.1. Research on peer review and referee reports

Räisänen (1999, pp. 121e126) traces the origin of the peer review system back to the first editor of the Philosoph-
ical Transaction of the Royal Society, and describes the review process as ‘‘a negotiation process between reviewers
and author(s) with the editor of the journal, as mediator and arbitrator’’ (p. 124). Other researchers have also dealt with
the genres of the peer review process (Hamp-Lyons, 1997; Okamura & Shaw, 2000; Swales, 2004). However, studies
dedicated exclusively to referee reports, referee comments, or simply to peer reviews of research articles (RA) as they
have also been called, are rather scarce. One of the first researchers to deal with this topic was Kourilova (1996), who
analysed the interactive functions of language in a corpus of peer reviews of medical papers by non-native users of
English. She focused on the rhetoric of evaluative features and objectivity, and conducted a quantitative study of com-
mands, doubt, suspicion and criticism, though only a few examples are provided. Also interesting for the purpose of
my research is her organisation of peer reviews into purpose-based proto-sections. She distinguished these as criti-
cism, compliments and polite suggestions and further described their patterning in the texts (p. 8). Even though it
is a very valuable contribution, this study considers language only from a functional perspective. Therefore, it seemed
that a more in-depth study focusing on the role of evaluative language within this functional perspective would offer
further important insights.
Another researcher who has looked at referee reports is Gosden (2001, 2002, 2003). Gosden (2001) analysed part of
the genre chain of the research article and focuses his studies on the contents of the reports and how authors respond to
the criticism they receive. He also proposed a structure for referee reports which is pertinent to the present research:
[T]hey often consist of a summarizing judgement regarding suitability for publication, followed by comments,
sometimes numbered by point or page, which may track criticisms in a sequential manner from beginning to
end of the manuscript. (2003, p. 91)
His main conclusions are related to the nature of the comments by referees. He also pointed out the difficulties
novice writers may encounter when faced with the ‘‘conflicting signals’’ in referee reports, which are very often
formed by evaluative expressions, though he does not further investigate this type of language.

1.2. Evaluative language

Evaluative discourse has been analysed by a number of researchers over the years (Biber & Conrad, 1999; Biber &
Finegan, 1989; Halliday, 1985; Hunston & Thompson, 1999). Although it has been given several names and defini-
tions, I will consider evaluation as ‘‘the broad cover term for the expression of the speaker or writer’s attitude or stance
towards, viewpoint on, or feelings about the entities or propositions that he or she is talking about’’ (Hunston &
Thompson, 1999, p. 5).
The perspective from which the referee report is analysed in this article takes as a general premise that, once the
author has read the referee report, it is no longer a document meant for criticism, but a request for improvement. It is
 I. Fortanet / Journal of English for Academic Purposes 7 (2008) 27e37 29

not only a text to be read, but must also be answered (as a letter), and an action is required - to modify the manuscript.
However, a request to modify a manuscript is too general and the referee usually focuses on some specific actions
which are expressed at several levels of explicitness.
From the perspective of speech act theory, according to Blum-Kulka (1989), there are three main strategies to for-
mulate requests in English: direct requests, conventional indirect requests and non-conventional indirect requests. Di-
rect requests are easily identifiable by the reader since they are expressed by means of specific grammatical, lexical and
semantic features. For the purpose of this research, direct requests in referee reports are considered to be those expressed
by means of recommendations, which include imperatives, performatives, and the use of some modal auxiliaries.
Conventional indirect requests use ‘‘fixed linguistic conventions established in the speech community, but the il-
locutionary act has two or more functions’’ (Cook & Liddicoat, 2002, p. 21). Among some common conventional in-
direct requests in English are questions starting with ‘‘could/can you’’ (Blum-Kulka, 1989). These requests are not so
evident for somebody who has learned the language formally, but who may not be familiar with its pragmatic con-
ventions. The intention of the writer when choosing this type of question is not to ask for a reply, which would be
the result of interpreting the literal meaning of the evaluative act, but to politely request that the reader do something.
The third type is the non-conventional indirect request, which needs not only knowledge of the language, but also
of the genre and its context for an accurate interpretation. There can be different levels of difficulty in the interpre-
tation of these requests. In fact, requests of this are more difficult for novice researchers to interpret because the reader
has to infer pragmatically (Grice, 1975) what is meant by the writer. In referee reports, this group includes questions,
criticism patterns, and even sometimes plain statements.
Taking instead a systemic functional perspective, language in evaluative genres can be related to Halliday’s (1985)
metafunctions: textual, ideational and interpersonal. In this paper the three metafunctions will be considered through
an analysis of the linguistic resources offered by the English language that are selected by the writer (textual meta-
function), the evaluative language used to convey propositional content (ideational metafunction), and the relationship
between reviewer and reviewed (interpersonal metafunction). Even though descriptive language may sometimes have
implied evaluation, in this study the parts of the text which do not encode positive or negative criticism, or recommen-
dation have not been considered. However, the importance of the description of the text or the part of it that the referee
wants to focus on should not be ignored since it prevents misunderstandings and also mitigates the impact of criticism
and recommendations (Moreno & Suarez, 2007).
There are two important aspects to communicate in a review. First, what has to be improved in some way and, sec-
ondly, the urgency or importance the revision of these points has for the reviewer. The resources used to highlight these
two aspects can be syntactic (order of the elements in a sentence), lexical (choice of certain verbs, adjectives, or
nouns), and pragmatic (especially the use of implicature). For example, in ‘‘The references should be carefully re-
vised’’ (ORG-25) the reviewer has chosen the passive voice as a syntactic device. Additionally, s/he has chosen the
verb ‘‘revise’’ instead of others, such as ‘‘check,’’ and selected the adverb ‘‘carefully’’ instead of expressions like
‘‘in detail,’’ as lexical devices. Pragmatically, this is a polite request easily understandable by the reader. In contrast,
the choice of an imperative form would have seemed inappropriate and constituted a more face-threatening pattern of
criticism. The present research will only focus on the syntactic and lexical devices of the evaluative language of ref-
eree reports and their pragmatic value as requests for modification of the article.
Before describing the research, it seems useful to define the unit of analysis. I have selected the definition proposed
by Suarez (2006) for evaluative acts in book reviews, adapting it to the analysis of referee reports by adding a second
part dealing with recommendations:
We have counted as an evaluative act any structural unit, irrespective of its lexico-grammatical configuration,
that contains both the (sub)aspect commented upon and what is said about it, or what action is recommended.
(p. 153)
For example ‘‘The problematics of metatext is well presented and ample room is given to the categories of the
model’’ (LIN-10) includes two units of evaluative language in the same sentence and linked by the conjunction
‘‘and.’’ In the first unit, the aspect commented upon is ‘‘the problematics of metatext’’ and what is said about it is
‘‘is well presented.’’
The objective of this research is to provide a description of the language of referee reports that helps both referees
and authors to improve the coding and decoding of referee reports, and consequently the relationship established be-
tween these two actors of the review process.
30 I. Fortanet / Journal of English for Academic Purposes 7 (2008) 27e37

2. Method

A corpus of referee reports was compiled specifically for this study. Experienced researchers from Universitat
Jaume I were asked to send blind or anonymous reviews of their papers submitted to international journals, as well
as some reviews that they had written at the request of journal editors. As these are private documents, I had to guar-
antee confidentiality. A total of 50 reviews related to two broad disciplines were collected: Business Organisation
(ORG) and Applied Linguistics (LIN). All of them were first reviews; for each disciplinary group I received 20 re-
views and 5 that were written by the participants who were all Spanish researchers and non-native users of English.
The total number of words in the Linguistics sub-corpus was 31,000, with an average of 1,240 words per report and
the length of each ranged from 180 to 3,214 words. The articles had been submitted to 10 different journals. There
were 14,933 words in the Business Organisation sub-corpus, with an average of 597 words per report and the length
of each ranged from 201 to 1,413. The articles had been submitted to 18 different journals.
Firstly, I analysed the characteristics of the evaluative language in referee reports based on Suarez’s (2006) defi-
nition of an evaluative act. Kourilova’s (1996) proto-sections were also taken into account. However, after a prelim-
inary search of the corpus, I decided to focus on three main groups of patterns to be analysed: criticism (including
Kourilova’s compliments and criticism); recommendations; and requests through questions (which take the form
of interrogative clauses). The last group (requests through questions) was included by Kourilova as part of the criti-
cism proto-section. However, the questions in this group are different from criticism in several ways. They do not use
attributes or identifiers to evaluate some aspect, and they convey an invitation to reflection or a requirement for ad-
ditional information or explanation. In a quantitative analysis, the frequency of evaluative patterns was measured
as units per 1,000 words of text. A qualitative analysis of the patterns was carried out on their syntactic and lexical
features in order to identify patterns in relation to particular types of requests.
Secondly, as seen in the Introduction, Gosden (2003) proposed a structure for referee reports in two parts: summa-
rizing judgement regarding suitability for publication, and comments with criticism. The presence of this structure
was tested in the corpus in order to check its validity and application for the teaching of the genre, as well as to reveal
evaluative patterns that may be used in each of the parts.
Using this methodology, the research questions to be answered were:

(a) Can the evaluative language of referee reports be described according to the criteria adopted?
(b) When compared according to these criteria, are there differences between the two sub-corpora that may affect
the reader’s understanding and interpretation of the text?
(c) Is there a relationship between the use of certain patterns and the structure of the referee reports?

3. Results and discussion

One of the most significant differences between the two sub-corpora under analysis is the total number of words
(31,000 in LIN, and 14,933 in ORG), and consequently, the average number of words per referee report (1,240 vs.
597). The Business Organisation sub-corpus is half the size of the Linguistics sub-corpus. However, the frequency
rate of clauses with evaluative language is much higher in the latter (35.5 evaluative patterns per thousand words)
than in the former field (25.6 evaluative patterns per thousand words). Though there are some exceptions, most referee
reports in the corpus conformed to this trend. Therefore, the initial finding of this research indicates that referee reports
in Business Organisation are shorter and are more evaluative in nature than their counterparts in Linguistics.

3.1. Linguistic patterns

The total number of evaluative patterns found in the whole corpus is 1,353, of which 793 (58%) belong to the Lin-
guistics sub-corpus, and 560 (42%) to the Business Organisation sub-corpus. As shown in Table 1, criticism patterns
are the most common in the whole corpus (47%), as well as in each sub-corpus (45% and 50%, respectively), whereas
recommendations and requests through questions rank second and third.
In the following sections, I will describe the patterns found in the three categories under investigation and provide
examples of them.
 I. Fortanet / Journal of English for Academic Purposes 7 (2008) 27e37 31

Table 1
Distribution of evaluative patterns in the two sub-corpora
LIN ORG Total
Criticism 356 (45%) 279 (50%) 635 (47%)
Recommendation 347 (44%) 186 (33%) 533 (39%)
Questions 90 (11%) 95 (17%) 185 (14%)
Total 793 (100%) 560 (100%) 1,353 (100%)

3.1.1. Criticism patterns

Most of the criticism patterns in the referee reports could be considered either attributive or identifying clauses
(Halliday, 1985). In attributive clauses, the (sub)aspect commented upon, or evaluated aspect, is followed by an at-
tribute which denotes that the evaluated aspect is ascribed or given a value as a member of a class (what is said about
the (sub)aspect, or criticism).

(1) The analysis is very interesting (LIN-23)

EVALUATED ASPECT: The analysis
CRITICISM: is very interesting

In this example, the structure of the clause is Subject þ Copular verb þ Attribute. Another structure is also possi-
ble, in which the criticism is expressed by means of an attributive adjective of a noun representing the evaluated as-
pect, as for example:

(2) This paper makes an interesting, quantitative contribution to both the competence-based perspective of the firm
and to the field of quality management (ORG-19)
EVALUATED ASPECT: the quantitative contribution of the paper
CRITICISM: (is) interesting

This structure will be named as Pattern CA1 for the purpose of this research. This is the most common pattern found
in the referee reports. It is used at the beginning of the document to give the general evaluation, and is usually
addressed to the editor in an implicit manner (the editor is very rarely addressed directly in a referee report), using
paper or article as a subject. This pattern can be used for positive or negative criticism, though it should be noted
that positive criticism only accounts for 11% of the patterns in both sub-corpora. Negative evaluation is conveyed
either by means of the negation of the verb, followed by an adjective with a positive meaning or by means of a negative
adjective (e.g., difficult, poor). It is an example of ‘blunt’ criticism (Kourilova, 1996), which can be mitigated by pre-
ceding or following clauses of positive criticism or suggestions for improvement, or by an explanation of the criticism.
There is another type of attributive clause (Pattern CA2), in which the attribute denotes a quality equivalent to
a mental process (Halliday, 1985, p. 121), the ‘carrier’ of that quality is usually the writer of the text (i.e., the referee),
though it can be expressed in a personal or impersonal way, as in the following examples:

(3) I am still not certain why Lecture 5 was included in the analysis. (LIN-2)
EVALUATED ASPECT: Lecture 5 was included in the analysis
CRITICISM: I am still not certain why
(4) it is difficult to develop enthusiasm for another article on industrial districts (ORG-7)
EVALUATED ASPECT: another article on industrial districts
CRITICISM: it is difficult to develop enthusiasm

In example (3) the referee seems to show doubts about her or his own criticism. This attitude may serve to soften the
negative evaluation, or it could just reflect a certain irony on the part of the reviewer. It is not always easy for the reader
to discern one from the other, even less for non-native users of the language. The negative effect is produced by the
adjective, which can either have a negative meaning (difficult) or a negative prefix (unclear). Again this pattern is used
mainly for negative evaluation and its function is to emphasize the aspects of the article that should be modified in
order to meet the quality standards of the journal.
32 I. Fortanet / Journal of English for Academic Purposes 7 (2008) 27e37

Table 2
Occurrences of criticism patterns in the corpus
Patterns LIN ORG Total
CA1 204 (57.3%) 176 (63.1%) 380 (59.8%)
CA2 59 (16.6%) 33 (11.9%) 92 (14.5%)
CB 62 (17.4%) 47 (16.8%) 109 (17.1%)
CC 31 (8.7%) 23 (8.2%) 54 (8.5%)
Total 356 (100%) 279 (100%) 635 (100%)

There is still another variety of pattern CA2 in which the writer expresses a mental process to introduce the negative
or positive value s/he attributes to the evaluated aspect, as shown in example (5):

(5) I find difficult to position the contribution of the article within the literature on the topic (ORG-3)1
EVALUATED ASPECT: to position the contribution of the article within the literature on the topic
CRITICISM: I find (it) difficult

As in examples (4) and (5), the evaluated aspect is postponed to the end of the clause; this gives the writer the op-
portunity to give it more emphasis as end focus, and also to explain it as much as necessary.
As mentioned above, identifying clauses are also common as criticism patterns. In these clauses, the evaluated
aspect is identified by means of certain characteristics. This type of pattern can be observed in examples (6) and (7):

(6) the background does not specify or discuss entrepreneurial orientation to a sufficient degree (ORG-13)
EVALUATED ASPECT: the background
CRITICISM: does not specify or discuss entrepreneurial orientation to a sufficient degree
(7) none of the studies (.) consider historical changes in academic style in actual use (LIN-2)
EVALUATED ASPECT: studies
CRITICISM: none of [them] consider historical changes in academic style in actual use

In this pattern (Pattern CB) the evaluative category is the verb, which is usually a negative verb or a verb in its
negative form as in example (6) above, but can move to the subject when it is this part of the sentence that conveys
the negation, such as in example (7).
There are also other patterns that were present in the referee reports, though not so common. One of these makes
use of existential clauses (Pattern CC). The verb there is/are is followed by a noun group, which includes either a neg-
ative adjective, or a noun with a negative meaning and is used for negative evaluation.

(8) there also is no discussion of the history of the region or the historical development of the area (ORG-7)
EVALUATED ASPECT: discussion of the history of the region or the historical development of the area
CRITICISM: there is no

Table 2 shows the distribution of criticism patterns for the corpus as a whole and for each sub-corpus.
As can be seen from the table, the most common pattern in both sub-corpora is CA1, which accounts for about 57%
of the occurrences of criticism patterns in LIN, and 63% in ORG, which is a similar percentage. If we consider all
attribution patterns together (CA1 and CA2), the percentage amounts to about 74% in both Linguistics and Business
Organisation referee reports. Attribution patterns are followed at a distance by identifying patterns (CB), which show
a frequency of about 17% in both disciplines, and existential clauses (CC) (about 8%).

3.1.2. Recommendation patterns

The second group of patterns identified in the referee reports is directly related to the interpersonal characteristics
of these documents. The patterns in this group could be considered directives following Yates’s (2004, p. 72) defini-
tion, ‘‘attempts by the teacher (reviewer, in this case) to get a student (reviewee, in this case) to do something concrete

1
Some of the referee reports presented grammar or spelling mistakes. As may have been noticed, this example is grammatically incorrect.
 I. Fortanet / Journal of English for Academic Purposes 7 (2008) 27e37 33

in the future,’’ and they belong to the group of direct requests, since the reader should only respond to the literal mean-
ing of the utterance.
Among the evaluative acts in the documents in the corpus, we have found direct requests in the form of imperatives,
as in example (9), though they are not the most common:

(9) Close bracket on definition here. (ORG-11)

RECOMMENDED ACTION: close bracket

This pattern (RA) represents a command or an order, using the imperative form or sometimes passive infinitives.
These could seem too direct for a formal document such as the referee report. However, imperatives are used only to
recommend very specific modifications, such as changes in linguistic form or expression, or the inclusion of a refer-
ence that is missing. Typically, what has to be added or replaced is provided by the referee. In this way, the imperative
does not seem to be so imposing and other longer and more polite expressions such as ‘‘Could you remove line 3?’’
would seem pragmatically inappropriate.
Also as direct requests, there are some examples of performatives (RB) with verbs such as ‘‘recommend’’ and
‘‘suggest,’’ as in (10):

(10) I would recommend the author to focus more in depth on rhetorical devices. (LIN-21)
RECOMMENDED ACTION: to focus more in depth on rhetorical devices

This pattern expresses the direct recommendation of the referee to the author. The referee uses the first person sin-
gular for himself and either addresses the author in the third person in the recommendation, or avoids it by choosing
a passive form or an eing clause. Sometimes, but very rarely, the second person is used to address the author.
The use of ‘‘would’’ adds politeness or indirectness to the sense of the verb. This pattern is generally used to sum-
marize the indications previously given in the report, and is often found either at the end of the document, or in the first
paragraph.
Another pattern of direct requests includes modals and semi-modals like ‘‘need to’’ (RC). According to Smith
(2003), ‘‘‘need’ has the potential to be used as an indirect means of laying down obligations’’ (p. 263). First person
plural and passivised third person are common as subjects of ‘‘need to,’’ as well as a reference to the reader as a third
person through ‘‘the authors’’; it is also common to find an impersonal use of ‘‘need to’’ using ‘‘there needs to be.’’
Though this semi-modal usually conveys a need for action in a vague way, ‘‘as if the source of requirement is the sit-
uation itself’’ (Smith, 2003, p. 263), the interpretation of obligation imposed can be pragmatically inferred. Some-
times, other verbs in the passive voice are used with the same illocutionary force of ‘‘need to’’ as in ‘‘deserves to
be considered’’ (¼needs to be considered) or ‘‘are required’’ (¼are needed).
Other modal verbs are also used, such as ‘‘should,’’ which conveys a stronger illocutionary force to the directive, or
‘‘might’’ which is softer, and suggests rather than obliges. The structure of these evaluative patterns can be Passivised
subject þ Modal þ Passive verb as in example (11), and Subject þ Modal þ Verb, as in example (12):

(11) Your use of the term innovation needs to be discussed in the light of the various types of innovation found in the
management literature (ORG-10)
RECOMMENDED ACTION: to discuss the term innovation

(12) I think the authors’ argument might be that entrepreneurs with a well-balanced, highly useful network with
whom they have frequent contact, will also exhibit higher EO. (ORG-5)
RECOMMENDED ACTION: to introduce the argument that entrepreneurs with a well-balanced, highly useful
network with whom they have frequent contact, will also exhibit higher EO

An alternative structure found in this group of patterns of recommendation is: it þ would þ be þ Adjective þ to/
wh- clause, as in example (13):

(13) it would be helpful to explain who the relevant ESP learners are. (LIN-5)
RECOMMENDED ACTION: to explain who the relevant ESP learners are
34 I. Fortanet / Journal of English for Academic Purposes 7 (2008) 27e37

Though the grammatical structure of this pattern resembles that of criticism patterns with attributive clauses as in
example (4), there is a difference in meaning and intention of the reviewer. Instead of pointing out the problem of the
evaluated aspect through criticism (mainly by means of a negative adjective), the reviewer emphasizes the recommen-
ded action by creating a hypothetical situation in the future with ‘‘would’’ and positive adjectives such as ‘‘helpful,’’
‘‘useful,’’ ‘‘desirable,’’ or ‘‘clearer.’’
Another type of recommendation pattern consists of conventional indirect requests. Because these are structured as
yes/no questions, their literal meaning could induce somebody knowing little about the pragmatics of the English
language to believe that questions with this pattern asks for a positive or a negative answer. However, in English, ques-
tions starting with ‘‘can/could you’’ are very often used as polite requests (RD). There are some examples in the
corpus, e.g., (14):

(14) Could the author move up the explanation of quantitative analysis to p.4? (LIN-3)
RECOMMENDED ACTION: to move up the explanation of quantitative analysis to p. 4

Table 3 summarises the frequencies of the various patterns for recommendations. It shows that the most frequent
pattern in both Linguistics and Business Organisation is RC, i.e., modal verbs followed by an infinitive verb, though
the percentage is much higher in Business Organisation than in Linguistics. As has been observed before, referee
reports in Linguistics are much longer than in Business Administration. Both types of documents deal with general
aspects often related to originality, significance of the contribution, or methodology, but only Linguistics referee
reports deal with more specific aspects regarding presentation of data, linguistic expressions, or references. Referee
reports often use imperatives to point out the problems with these more specific aspects, which makes the percentage
of these patterns more than double in Linguistics than in Business Organisation. Patterns with performative verbs (RB)
are also much more common in Linguistics, perhaps due to the presence of general recommendations at the beginning
and final part of the referee report which, as will be seen in 3.2, are not usual in the Business Organisation documents.
Polite requests starting ‘‘can/could you’’ (RD), on the contrary, seem to be preferred by referees in Business Organ-
isation, with a higher percentage in this field than RA and RB patterns.

3.1.3. Requests through questions

There was a great variety in the interrogative clauses used by the referees to request clarification or more informa-
tion. The aims of these questions were varied and they do not seem to fit either into criticism or recommendation pat-
terns. In general, and based exclusively on syntax, two types can be distinguished: yes/no questions and wh- questions.
All the questions in this group share an added difficulty for the reader since they do not give clear clues about what has
to be done, and therefore should be classified as non-conventional indirect requests whose interpretation will very
much depend on the context. Generally speaking, it can be said that both types of questions may mean lack of com-
prehension or disagreement, as in (15):

(15) why ‘‘some types’’ of metonymy? (LIN-4)

RECOMMENDED ACTION: provide an explanation for introducing only some undetermined types of
metonymy

The reviewer may also not agree with some part of the document and by using such questions, they can indicate
a need for clarification or further information, as in (16):

Table 3
Occurrences of recommendation patterns in the corpus
PATTERNS LIN ORG Total
RA 68 (19.6%) 14 (7.5%) 82 (15.4%)
RB 50 (14.4%) 15 (8.1%) 65 (12.2%)
RC 214 (61.7%) 135 (72.6%) 349(65.5%)
RD 15 (4.3%) 22 (11.8%) 37 (6.9%)
Total 347 (100%) 186 (100%) 533 (100%)
 I. Fortanet / Journal of English for Academic Purposes 7 (2008) 27e37 35

Table 4
Occurrences of request through questions patterns in the corpus
Patterns LIN ORG Total
Yes/no question 56 (62.2%) 45 (47.4%) 101 (54.6%)
Wh-question 34 (37.8%) 50 (52.6%) 84 (45.4%)
Total 90 (100%) 95 (100%) 185 (100%)

(16) Is there support for using this approach? (ORG-10)

RECOMMENDED ACTION: either to add theoretical support or change the approach

These types of questions can be quite face-threatening for the authors since they are addressed directly to them.
Moreover, the meaning that is conveyed is not literal but has to be inferred, and very often several questions come
together, sometimes referring to the same aspect and other times to different aspects. Table 4 presents the quantitative
results on the request through questions patterns.
Though the number of questions found in both corpora is very similar, there seems to be a preference in Linguistics
for yes/no questions, contrary to what happens in Business Organisation. However, as indicated above, some notice-
able differences have been observed in both sub-corpora at the individual level.

3.2. Structure

Regarding structure, when Gosden’s (2003) model was applied to the corpus, it was found that two additional parts
were often present. As a result, a four-part structure has been proposed, using the term Moves, following Swales
(1990):

Move 1 e Summarizing judgement regarding suitability for publication.

Move 2 e Outlining the article.
Move 3 e Points of criticism.
Move 4 e Conclusion and recommendation.

The findings indicate that in all 50 reports in the corpus of this research Move 3 is present, and is even the only
move present in three referee reports in Business Organisation (ORG-7/-12/-17). It is in this move where the assess-
ment of the reviewed article is made and where the specific suggestions and recommendations for improvement are
clearly explained. The points assessed are introduced either following the order of the manuscript or from more gen-
eral to more specific aspects of the text. As for the rest of the moves, there seem to be some differences. While Move 1,
in which the referees summarize their opinion about the suitability of the article for publication, is present in 23 of the
Linguistics reports, it only appears in 14 Business Organisation reports. Move 2, the outline of the article, is less fre-
quent, though more common in Business Organisation reports (12 vs 8 in Linguistics). Regarding Move 4, there is
a similar frequency in both Linguistics and Business Organisation with less than half of the reports including it
(see Table 5).
From these results it could be concluded that a four-move structure is more appropriate for this genre, though there
may be some disciplinary differences, especially for Move 1. A study using a larger corpus would be needed to cor-
roborate these results.
Finally, Table 6 shows the distribution of the three types of evaluative patterns in the moves established for the
structure of referee reports.

Table 5
Distribution of Moves in the two sub-corpora
Move 1 Move 2 Move 3 Move 4
LIN 23 8 25 10
ORG 14 12 25 9
36 I. Fortanet / Journal of English for Academic Purposes 7 (2008) 27e37

Table 6
Distribution of evaluative patterns in the Moves of the referee reports
Move 1 Move 2 Move 3 Move 4 Total
LIN ORG LIN ORG LIN ORG LIN ORG LIN ORG
Criticism 42 (11.8) 34 (12.2) 8 (2.2) 3 (1.1) 294 (82.6) 233 (83.5) 12 (3.4) 9 (3.2) 356 (44.9) 279 (49.8)
Recommendation 25 (7.2) 4 (2.1) 1 (0.3) 0 (0) 308 (88.8) 176 (95) 13 (3.7) 6 (3.2) 347 (43.8) 186 (33.2)
Questions 0 (0) 0 (0) 0 (0) 0 (0) 90 (100) 94 (98.9) 0 (0) 1 (1.1) 90 (11.3) 95 (17)
Total 67 (8.4) 38 (6.8) 9 (1.2) 3 (0.5) 691 (87.2) 503 (89.8) 25 (3.2) 16 (2.9) 793 (100) 560 (100)
Note: The number in brackets corresponds to the percentage.

As can be seen from the table, the distribution of the evaluative patterns in the moves of the referee report does not
show many disciplinary differences. All types of patterns concentrate in Move 3, Points of criticism. The only relevant
finding can be the location of some recommendation patterns in Move 1 in Linguistics, which implies a lower percent-
age of this type of evaluative patterns in Move 3 in comparison to Business Organisation. Criticism patterns are the
only ones with a notable presence in Move 1, and a small percentage both of Criticism and Recommendation patterns
can be found in Move 4 in both sub-corpora.
The presence or absence of certain moves in a number of referee reports in a discipline does not seem to have an
important repercussion on the amount of evaluative acts in those moves probably because the language found in them
is descriptive rather than evaluative.

4. Conclusion

To answer the first of the three research questions outlined in the Method section, the proposed taxonomy has
proven to be very useful for the analysis of referee reports in these two fields of research. Referees seem to prefer
Criticism, and especially attribution patterns. Recommendation patterns are placed second, with a tendency to use
modal verbs. Requests through questions rank third with an unequal distribution. However, it is important to point
out that most referees use a great variety of these resources. This description of the evaluative language of referee
reports can be of help to teachers and learners of EAP who wish to learn more about the language of referee reports.
With reference to the second research question, throughout the study we have seen that there are some marked dif-
ferences between the referee reports in Business Organisation and Linguistics. The first clue was given by the number
of words in the two corpora. Even though the same number of texts was taken into consideration, the length of the
Linguistics sub-corpus was double the size of the Business Organisation sub-corpus. Secondly, the frequency of eval-
uative acts in Business Organisation reports is much higher (35.5 per 1,000) than in Linguistics (25.6). Another
difference concerns requests. As discussed in the Introduction, requests can be direct, conventionally indirect or
non-conventionally indirect. While recommendation patterns only require a good command of the language, conven-
tional indirect requests and especially non-conventional direct requests require a much greater knowledge of the genre
and comprehension of the context to be interpreted. In Business Organisation referee reports criticism patterns and
requests through questions, classified as non-conventional indirect requests, amount to almost 70% of the evaluative
patterns compared with only 56% in Linguistics. According to Cook and Liddicoat’s (2002) study, decoding non-
conventional indirect requests takes L2 learners much time and effort, and they need to be exposed to contextual in-
formation, as well as to the genre for a certain period of time before being able to interpret the appropriate function of
an evaluative act. Referees and editors, especially those in the field of Business Organisation, should be aware of this
difficulty and consider the target reader of their reports. In this way, it will be easier for authors who are not native
speakers of English to understand the reports and comply with the referee’s requests.
In response to the third question, though most patterns were found in Move 3, Points of criticism, a few were found
with some frequency in other Moves, such as Move 1 for positive and negative criticism, or Move 1 and Move 4 for
recommendation for publication (implicitly addressed to the editor). However, these disciplinary differences would
need further research to be corroborated.
Further research would also be needed on other aspects. As Kourilova (1996) noticed, most evaluative acts do not
appear in isolation. It would be interesting to see how the patterns described are interrelated in referee reports of this
study. This could be complemented with an analysis of the combinations between evaluative acts and claims or jus-
tifications, which have been found to be very frequent in other evaluative genres such as book reviews.
 I. Fortanet / Journal of English for Academic Purposes 7 (2008) 27e37 37

4.1. Implications for researchers

The final aim of my research was to foster better comprehension of referee reports. The results of this study can be
useful for novice researchers who are not yet familiar with the characteristics of the genre. The description of the lan-
guage used can help them to understand the demands of the reviewers. Additionally, these findings may be of help to
referees who need to write reports in English, which include the necessary criticism and still are ‘‘supportive, con-
structive, thoughtful, and fair.’’2

References

Biber, D., & Conrad, S. (1999). Lexical bundles in conversation and academic prose. In H. Hasselgard & S. Oksefjell (Eds.), Out of corpora:
Studies in honour of Stig Johansson (pp. 181e190). Amsterdam: Rodopi.
Biber, D., & Finegan, E. (1989). Styles of stance in English: Lexical and grammatical marking of evidentiality and affect. Text, 9(1), 93e124.
Blum-Kulka, S. (1989). ). Playing it safe: the role of conventionality in indirectness. In S. Blum-Kulka, J. House, & G. Kasper (Eds.), Cross-
cultural pragmatics: Requests and apologies (pp. 37e70). Norwood, NJ: Ablex.
Brown, P., & Levinson, S. (1987). Politeness. Some universals in language use. Cambridge: Cambridge University Press.
Cook, F. H., & Liddicoat, A. J. (2002). The development of comprehension in interlanguage pragmatics: The case of request strategies in English.
Australian Review of Applied Linguistics, 25(1), 19e39.
Gosden, H. (2001). ‘‘Thank you for your critical comments and helpful suggestions’’: Compliance and conflict in authors’ replies to referees’
comments in peer reviews of scientific research papers. Ibérica, 3, 3e17.
Gosden, H. (2002). Thematic content in peer reviews of scientific papers. Seminarios de Linguı́stica, 5.
Gosden, H. (2003). ‘Why not give the full story?’: Functions of referees’ comments in peer reviews of scientific research papers. Journal of En-
glish for Academic Purposes, 2(2), 87e101.
Grice, H. P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.), Syntax and semantics (pp. 41e58). New York: Academic Press.
Halliday, M. A. K. (1985). An introduction to functional grammar. London: Arnold.
Hamp-Lyons, L. (1997, March). Editor’s interpretations of submission reviews: how occluded is the genre? Paper presented at the TESOL
Convention, Orlando, FL.
Hunston, S., & Thompson, G. (Eds.). (1999). Evaluation in text. Oxford: Oxford University Press.
Kourilova, M. (1996). Interactive functions of language in peer reviews of medical papers written by non-native users of English. UNESCO-
ALSED LSP Newsletter, 19(41), 4e21.
Moreno, A., & Suarez, L. (2007). A framework for comparing evaluation resources across academic texts. Text & Talk.
Okamura, A., & Shaw, P. (2000). Lexical phrases, culture, and subculture in transactional letter writing. English for Specific Purposes, 19, 1e15.
Peters, D., & Ceci, S. (1982). Peer-review practices of psychological journals: the fate of submitted articles, submitted again. Behavioral and
Brain Sciences, 5, 187e255.
Radford, D. R., Smillie, L., Wilson, R. F., & Grace, A. M. (1999). The criteria used by editors of scientific dental journals in the assessment of
manuscripts submitted for publication. Research, 187(7), 376e379.
Räisänen, C. (1999). The conference forum as a system of genres. Göteborg: Acta Universitatis Gothoburgensis.
Shebilske, L. (1997). How to review a journal article. ISSPR Bulletin, 13(2), 19e20.
Smith, N. (2003). Changes in the modals and semi-modals of strong obligation and epistemic necessity in recent British English. In R. Facchinetti,
M. Krug, & F. Palmer (Eds.), Modality in contemporary English (pp. 241e266). Berlin: Mouton de Gruyter.
Suarez, L. (2006). Modes of evaluation and rhetorical patterns: a contrastive study of English and Spanish book reviews. Unpublished doctoral
dissertation, Universidad de León, Spain.
Swales, J. (1990). Genre analysis: English in academic and research settings. Cambridge: Cambridge University Press.
Swales, J. (1996). Occluded genres in the academy. In E. Ventola & A. Mauranen (Eds.), Academic writing (pp. 45e58). Amsterdam: John
Benjamins.
Swales, J. (2004). Research genres. Cambridge: Cambridge University Press.
Wright, T. J., & Orbe, M. (2003). Turning the tables of analysis in intercultural communication research: studying the facework strategies used by
‘‘anonymous’’ European American reviewers. The Howard Journal of Communications, 14, 1e14.
Yates, L. (2004). Negotiating an institutional identity: individual differences in NS and NNS teacher directives. In K. Bardovi-Harlig &
B. S. Hartford (Eds.), Interlanguage pragmatics. Exploring institutional talk (pp. 67e97). London: Lawrence Erlbaum Associates.
Zobel, J. (1997). Writing for computer science. Singapore: Springer.

Inmaculada Fortanet is a lecturer and researcher at Universitat Jaume I (Castellón, Spain), where she coordinates the Group of Research on
Academic and Professional English (GRAPE). Her current research interests are related to the analysis of spoken and written academic discourse.
Some of her publications can be found in journals such as English for Specific Purposes and Discourse Studies.

2
Journal of Marriage and Family. How to Review a Journal Article: Suggestions for First-Time Reviewers and Reminders for Seasoned Experts
[http://www.ncfr.com/jmf/review_journal_howto.htm].