Copyright Waming Notice ‘cae ny bet oy Ui een nets, Nan eas of yi tcp scp ss, Yon tcp bd a sat oe coeepant 2 oe porn Whee couse onde bya exson acy ay pron Nayou om we, You yh ae te cy oy Sher pase. aloe waa whet wanna oyu seep ogoes a Scpny yh e. * Sengupta, S. (1998). Peer evaluation: ‘1 am not the teacher’. ELT Journal, 52/1, 19-28. Peer evaluation: ‘I am not the teacher’ Sima Sengupta ‘This article is based on an exploratory investigation of a secondary school writing cless in Hong Kong. Through examination of the way learners in this study viewed the roles of the teacher and learner as ‘readers’ of the ‘compositions they had written, it explores the extent to which the broader educational context and its belief system shaped six ESL students’ per- ception of peer evaluation. Finally, the article questions whether notions of collaborative construction of knowledge in the classroom are viable options within an examination-driven, accuracy-oriented L2 curriculum which may preclude learners (and teachers) from re-conceptualizing their traditional roles. Introduction Many researchers have studied peer and self-evaluation in Li and L2 writing (for example Mangelsdorf 1992, Mendonga and Johnson 1994). Findings suggest that student writers take selective account of peer comments when they revise, preferring to depend more on their own knowledge. Mendonca and Johnson (1994: 762) also discovered that student writers may not always trust their peers, but the same comment ‘from a teacher will be taken into account when they revise. Mangelsdort (1992: 280) reports that peer reviews were always rated negatively by Asian students, and raises the question of the effect of teacher-centred cultures on the way students regard peer comments. Carson and Nelson (1996: 18) believe that cultural factors, such as harmony-maintenance strategies, guide Chinese learners when they participate in peer evaluation. Even in Western cultures, Freedman et af. (1986), for example, found that even when peer evaluation is planned and controlled by the teacher, there may be social implications behind the responses which are determined by the way students maintain social relations (ibid.: 15). They point out that peer evaluation often turns out to be an exercise in futility because students are more busy figuring out easy ways to complete the evaluation sheets than evaluating the text (ibid.:21), So, learners may pay only lip-service to the task (Mangelsdorf 1992: 280). ‘However, it must be stressed that the peer evaluation described in this study was not implemented in the same way as the peer conferencing/ review procedures in the studies mentioned above. In the context of this study in Hong Kong, the demands of the English language syllabus allowed only limited time and resources to be spent on peer and self- evaluation, ELT Joural Volume 52/1 January 1998 © Oxford University Press 1998 19Background In the first five years of secondary school in Hong Kong, reading, writing, speaking, and listening skills are taught separately. There are regular examinations at the end of each term, and at the end of the fifth year, English language skills are assessed by means of @ public examination (the Hong Kong Certificate of Education Examination). ‘The examination includes a composition paper that accounts for 25 per cent of the total grade. The examination is marked using double impression holistic grading. Tse (1993) points out that in Hong Kong schools studying to pass ‘examinations is common, and the teaching of writing is very examina- tion-oriented. Almost all compositions are done in the classroom and, in many respects, are considered tests. Language teachers give a topic, set a time limit, and specify the number of words to be written. The usual practice is to correct the grammatical errors in each composition and then move on to a new topic (Sengupta 1996). The product-centred culture of Hong Kong emphasizes accuracy at the expense of self- ‘expression (Harris 1993: 516). At the outset, the learners with whom this study was conducted matched this description. For this study, however, they received additional instructional input aimed at developing reader awareness both before and after writing the first draft. At the post first draft stage, input consisted of audience analysis with questions such as, ‘What does the reader need to know in order to understand the writer”. The learners were first taught revision strategies with guided tasks, such as analysing a text by posing questions on the content. Then two copies of the evaluation sheet for the task (see Appendix) were handed out, and the teacher elicited what the questions required, sometimes changing, correcting, or simplifying them, where necessary, Students were first asked to read their own compositions and complete the evaluation sheets. Then they exchanged their first draft with the person sitting mext to them, read each other's first drafts, completed the evaluations sheet, and then returned the evaluation sheet and the ‘composition to the writer. Each student read the evaluation sheet completed by the other student and revised the first draft. The teacher's role was that of a facilitator, explaining difficult words or demonstrating how to answer the questions in the evaluation sheets. In this way peer evaluation sheets were used as a support for revising first drafts. The evaluation students were expected to do was not in itself extensive but it was hoped that by giving them an opportunity to evaluate their peers’ texts, a number of pedagogic and practical goals would be accomplished. Firstly, it would help students understand a reader's perspective because someone other than the writer would read the text. Secondly, it would enable students to become independent writers invoking the authority of personal experience as ‘potential readers’ (Reid 1993: 46). Thirdly, it would enable the teachers and the researcher to examine whether the strategies taught in order to develop writing and revising skills had actually been learnt. Sima SenguptaThe research Sample Data collection and analysis Findings Changes made on the basis of peer evaluation Interviews The research project was designed to answer the following questions: 1. Did the evaluation sheets and the resulting revisions indicate that there were textual changes arising from peer evaluation? 2. Did the students believe that peer evaluation led to awareness of themselves as real readers? ‘The sample consisted of a class of fifteen to sixteen year-old girls, whose native language is Cantonese. The majority of the girls came from middle or lower-middle class backgrounds. The medium of instruction in the school was English. The teacher was a native English speaker. Firstly, the self and peer-evaluation sheets were compared in order to identify peer suggestions that were distinct from those made by the ‘writers themselves. Then the revisions were examined to see whether there was any evidence that the peer suggestions had been incorpo- rated.! The aim of the analysis was to separate the changes that reflected peer suggestions from those that did not. Twelve students’ compositions pairs) were chosen for this analysis, and six students agreed to be interviewed. Arising out of the analysis of the evaluation sheets, semi- structured interview questions were framed and analysed within the following inter-dependent constructs? 1. perceived usefulness 2 peresption of ability to respond and way of responding (Le. how the writer evaluated the text of a peer) 3 perception of real reader 4 concerns (e.g. the public examination) 5. beliefs about knowledge and perception of teacher's role. Self and peer-evaluation of the same composition did not differ substantially in the compositions selected. Although there were differ- ences in the way these students executed peer evaluation, there were notable similarities in their revision behaviour. Not one of the twelve seemed to perform any revisions arising out of their peers’ suggestions unless they were problems the learners had already detected themselves. For the interview the researcher started with a general question about what the interviewee thought of peer evaluation. To this question all six, with varying duration of pauses, said that the experience was okay or useful. Only on further probing did the ideas that are reported emerge. None of the six interviewees believed that peer evaluation led to awareness of themselves as real readers. Moreover, the interviews revealed a number of consistent concerns. Table 1 shows that four categories of comments were made by all six subjects. The examination is mentioned most often, followed by beliefs about the role of the teacher as an evaluator, and a related perception of their own inability to correct grammatical errors. These findings indicate how student perceptions of the value of peer evaluation are shaped. Peer evaluation aTable 1: Summary of the main concerns expressed by ‘the students Perceived usefulness Construct Comments, Number of Number of {800 interview Interviewees times data analysis) who mentioned ‘mentioned it « ‘Concerns voiced inthe context 6 2 ‘of the examination 5 Evaluation i the teacher'sjob 6 Fy 2 Cannot eveluste without 6 ” knowledge of grammar Sends theteacheror 6 “ peer 2 Do not know how to 5 " ‘order to evaluate other ‘composition: can only evaluate ‘own composition 2 Embarrassed to have peers read 4 8 the composition 2 Evalustion depended on aspects 4 3 that could be dealt with in the revision 1 Not taken seriously by peers and 4 10 50 did not doit seriously 2 Do not have schemata for text 4 a ‘ype a Parceiving sof loss able 3 19 ‘compared with peers 4 [No similarity with real reading in 3 7 school The interviewees seemed to imply that the instructional input had not prepared them for peer evaluation because they could not be the real reader. In these students’ world, the real reader is the teacher or the examiner ‘who knows correct English’, not a peer with a questionable command of English. Although the students were being taught how to look at discourse-related aspects of writing, they seemed to be 50 conditioned by their experience of error correction as a follow-up to the first draft that they had come to believe that accuracy could solve most problems in writing, and that accuracy is the province of the teacher. The interviews seemed to indicate that these writers did not perceive any value in peer evaluation, and two learners called the whole exercise a ‘waste of time’. This typical comment makes it clear that these students ‘were completing the evaluation sheets to please the teacher: Extract 1 Student: I think organization is better if teacher tells me what to do. I think I do not like my neighbour to read my composition. I have many mistakes. I am not . .. 1 do not like . . . my class friend will laugh. So we read quickly because . .. it is because [the teacher] tells me. But I do not want to read many times and think many Sima Senguptatimes .. am not the real reader. So I read quickly and. ‘write something on—in—the sheet. Interviewer: What do you mean by ‘something"? Student: Something, I mean, a comment or something. 1 fill it—ah—just flit, Intrinsically merged with the perception of usefulness was the context of the approaching public examination and the role of the teacher in preparing these students for it. Role of the teacher Every interviewee felt that by completing these evaluation sheets they were being asked to take on the role of the teacher. It seemed that the students viewed the evaluation as something to be carried out by an expert, and the teacher, as the one who ‘knows English’, was therefore better able to evaluate than the students, whose English was seen as ‘weak’ Extract 2 Student 1: No not at all useful for me. I think it was useless. It is because I want to know from the teacher how to make this composition better . . . to get more . .. to get good. HIKCE pass. I cannot tell Sarah and Rachel how to get. a better result... she is already better than me. Interviewer: But you can read and see if there are questions you ask as reader, just like the exercises in the notes—the way you were taught, Did these exercises help you? Student 2: But they helped me to read my own composition but only {the teacher] can tell me how to make my composition better. It is her... it is her work . .. job. Since linguistic feedback from the teacher was readily available before the study began, the strategies for evaluation were seen as unnecessary.? ‘The interviewer's suggestions that even grammatical errors could be corrected by discussing them with peers or referring to a grammar book ‘were met with comments such as, ‘But the teacher must tell me’, The reader and There was constant reference to the readers’ English proficiency, as if to writing purpose indicate that there was an idealized target reader in the writer's mind ‘who did not in any way match the peer reader. This idealized reader derived his or her status not only from knowing English, but also because he or she gave the grade. The students’ perception of themselves as real readers was closely connected with this conceptualization of the idealized reader, as seen in Extract 3: Extract 3 Interviewer: So you think the real reader can only be someone ‘whose English is excellent? Student: Of course. Interviewer: Why? Student: It is because the reader is giving me a F. (Laughs) Yes because this reader is not reading like reading the Peer evaluation 2)Classroom reading Summary ‘newspaper but reading to pass or fail. So she must know English. Providing them with a real reader and a real writing purpose was seen as peripheral by these students. It seemed that they were so conditioned by the examination and their expectation that the teacher would prepare them for it that they did not give the imaginary reader or the purpose a second thought. Extract 4 is an example of whet one student thought of the reader and the purpose of Task 2 (see Appendix). Extract 4 Interviewer: But if the police received such a letter what will they expect? Did you read thinking of yourself as a policeman? Student: We know that the police are not going to read this, only the teacher will, so we have to try to read like the teacher .. . but my English is not so good, s0 how can I read like the teacher? Why will my classmate learn from me? She knows I do not know how to write... not good in this subject. I cannot help her get a good result. The idea that learning is only possible from an authority figure who knows ‘correct’ English is not surprising in 2 context where the focus on accuracy naturally precludes risk taking, Even in L1 writing, Tse (1993: 389) observes that students seemed not to have been encouraged in school to express themselves freely and to attempt uncertain forms of expression. ‘The mental configuration of the idealized real reader's perspective was not only influenced by the students’ perception of the teacher's role in “helping them get a good result’; they also felt that neither reading in an English lesson in school nor reading in real life was performed in this evaluation mode. As pointed out by one student: Extract 5 Interviewer: So you think that your neighbour was not a real reader? Student: Of course no. How can she be a real reader? We don't read like this. We read a passage and answer questions. The passage is in correct English and good writing—it is because it is in a book. When we read the passage we do not give suggestions or see which paragraph is not so good. This is not reading. I do not read anything and think if it is helping me as a reader. I read a passage to answer questions. Reading in school is done to retrieve information and thus, although they were learning to evaluate texts in their writing lesson, these students failed to perceive the connection between the task of reading and evaluating and reading done in the L2 classroom: ‘The demands of the educational context made these students unable to value learning to write beyond the instrumental need for a good grade. Sima SenguptaConclusion The whole exercise of writing, evaluating, and revising was reduced to one main concern: the grade. The belief that there was only one real reader— the person who gave the grade—was voiced repeatedly by all the interviewees, indicating that the source of their perception of the teacher's role in the L2 curriculum is intrinsically linked with the importance attached not only to the examination but also to accuracy. In the classroom the teacher, with her knowledge of ‘correct English’, gives the grades, is thus the only ‘real’ reader, and is responsible for teaching accurate writing. Tt seemed clear from the interviews that the leamers see their responsibility as to do what the teacher asks. The possibility of learning from their peers by looking at a potential reader's suggestions, or asking for genvine reader impressions, did not occur to these learners as their Jack of knowledge’ prevented such a possibility within the accuracy- oriented context. Clearly peer evaluation was not able to bring a real reader's perspective. ‘A number of reasons for this may be considered. It is likely that the way instruction in revision was designed and executed had failed to help students to become the real reader, and that the input may not have Prepared students with adequate linguistic and cognitive maturity to evaluate and act upon the evaluation. In spite of the negative feedback from students, the evaluation sheets demonstrated that students were, in fact, reading for evaluation, and were able to do this adequately in many cases. However, providing students with an evaluation sheet may have encouraged a prescriptive stance (see Mangelsdorf and Schlumberger 1992: 248) rather than a collaborative one. The question is that if peer evaluation were introduced as a collaboration between writer and peer, would students value the experience more positively? Although much more focused research is essential to address this question in any depth, the data regarding students’ perceptions seem to indicate that these students were not able to benefit from this exercise because, to them, the teacher was the only reader. The traditional roles of the teacher and learner in the school curriculum seem so deep-rooted that the only possible interpretation of knowledge appears to be that it is ‘transmitted from the teacher to the student, and not constructed by the classroom community. Unless these perceptions regarding teachers’ roles are addressed, it is probable that little value will be attached to peer evaluation, and collaborative and autonomous learning by secondary school ESL students may not become a reality. As Onore (1989; 231-2) points out ‘As long as judgements of what may be ‘better’ or ‘worse’—that is, of ‘what constitutes improvement in writing—remain the province of the teacher alone, then the writer cannot fully and authentically engage in choice making and problem solving, Received May 1996 Peer evaluation 25Notes 1 However, through this analysis itis claimed that the revision changes performed on the texts were solely due to peer evaluation 2 Segments of transcripts were given to a col- league to categorize in order to establish reliability. For eight out of the ten segments there was agreement, 13 The fact that the feedback received prior to the study was predominantly linguistic did not seem to matter to these learners, to whom ‘accuracy’ ‘was synonymous with a ‘good’ composition, in 4 Bi of the dacourge elated inp. is may be partly due to the current division of the English curriculum into the four kil areas Kong schools. 5 Beash {1959878 points out that i is proble matic to demonstrate to large groups how evaluation should be done because. students hhave disparate problems and abilities. However, considering the realities of the large classrooms ‘of Hong Kong, itis essential to get teaching toa mixed ability class. How this can be done effectively needs further investigation. References Ansoa, CM. (ed). 1989. Writing and Response, ease Noam ol Tesora Beach, R. 1989. ‘Showing students how to assess: demonstr students’ perception of ESL peer response group interaction. Journal of Second Language Writing. SM: 1-19. Freedman, S.C. Greenleaf, and M. ee Report No. 23). Us of Feachers of Bags IL: National Council 6 Sima Sengupta Haris, J. 1993. ‘Isee what I mean! Exploring the relationships between writing and learning and learning to write’ in N. Bird, J. Harris, and M Ingham (eds): Language and Content, Hong Kong Josute of Languages in Education Mangledort, 1992 Peet reviews in the ESL. ition classrooms: what do the students .”” ELT Journal 46/3: 274-84, Mangelsdort, K. and A. Schlumberger. 1992. “ESL student response stances in a peer-review task’. Journal of Second Language Writing. V3: 235-4. Mendonca, C.0. and KE. Johnson, 1994, “Peer review negotiations: revision activities in ESL ring instruction’. TESOL Quarery 284 148 Onore, C. 1989, “The student, the teacher and the text: negotiating meaning through response and revision’ in CM. Anson (ed.). Reid, J. 1993, “Historical perspectives on writing and ‘reading in the ESL classroom’ in 4.G. Carson and I. Leki (eds). Reading in the Composition Classroom. Boston: Heinle and einle, Sengupta, S. 1996. ‘A Study of the Effects of the ‘Systematic Teaching of Revision on the 12 Learners in a Secondary School in Hong Kong’. PhD thesis, Department of Curriculum Studies, University of Hong Tse, SK, 1993, ‘The ing Process of Hong ‘Kong Children in Primary Schools." PRD thesis, University of Nottingham. The author is a lecturer in the Department of English, “Hong Polytechnic University, where she teaches to undergraduate students. She has just completed her PhD in ‘writing and revision. Her research interests include second language writing, reading, and teacher education.Appendix: Evaluation sheets Task 1 ‘Your views are sought. Please respond: ‘To keep the Hong Kong beaches clean it has been suggested that people should have to buy a ticket worth HIKS 50 to enter the beach. They will not be allowed to carry food, drink, or plastic bags. Do you think that this is a good idea? ‘Why/Why not? Read the composition and fill inthe evaluation sheet. Name of evaluator. provided for longer answors the intention? Agrees/Disagrees withthe policy statement '3, How soon does the reader know about your intention? in the firsvsecond’ ‘thirdfast paragraph. Meccowtructure 4 Does the composition have a dafinite macrostructure? Yes/No 55 Has the macrostructure been signalled with specific words e.g. problem, solution? Yaeto {6 Do the problems follow 2 logical sequence? 1. Yas, the main probloms are mentioned first. B. Yes, the problem of the HKS 50 charge has been dealt with frst, and then ‘the question of plastic begs has been discussed. ‘& No, no sequence. 4. Other sequence, 6. 17 Do you have ny suggestions about how to improve the mscrostructure? Arguments '8 What are the main arguments? (please write in note form) o rGuments have the three elements: claim, data, and warrant? ‘Argument () ‘Argument (i) ‘Argument ii ‘10 Do you have any suggestion sbout revising the arguments? 11 Does each paragraph have a main point? What ie i? Paragraph 1 Paragraph 2 Paragraph 3 Paragraph 4 Peer evaluation a112 Do you have any suggestions for changes in the paragraphs? 118 What is the best feature of this composition? (Circle one only) @. doas 1. Paragraphs are well developed B, Mactostructure {Purpose is clear ‘Arguments Readers tnd teary to reed ‘Clerty stated main points . Other {4 What s the woakest feature? Choose one from the options for question 13. Task 2 Context: A series of bullying incidents which led to the hospitalization of a schoolboy. The police are seeking information, (Three picture prompts are siven, with details such as clothes, appearance, time, street name, etc.) ‘Write a report tothe police describing what you saw on three separate occasions, Read the composition and fill in the evaluation she. 11 Have Uimy classmate given the police enough information? ‘Whole essay: Yes/No (please circle) Paragraph 1 Yeso Paragraph 2 Yee Paragraph 3 Yeamoe. Paragraph 4 YesNoo.. any detail can be added put e plus sign (+) you feel any importantinformation can be added, puta cross (x) next tothe paragraph number and give examples. Ifyou feel some information is irrelevant put 8 question mark (2). 2 Have Uy clasemste made it easy for the police to read and picture the events as they happened, or will hey have to read tho report many times to know exactly what happened? Whole essay: Yes/No Can the organization be made more reader-tendly by reorgenizing aragraphs? Which peragrapt Why? ‘2. Confusing for the reader. Too long. ‘The police must be told the most important details fret. dd. The order or sequence of events is wrong. This isa report and thus must have facts not personel views. '3 Does the language help the police understand whet is meant? Sima Sengupta