RATERS’ BACKGROUND KNOWLEGDE AND EVALUATION OF

L2 LEARNERS’ WRITING COMPOSITION

STUDENT NAME: SYAHIRAH BT MOHD SOFIAN / SHARVNI ARUMUGAM

STUDENT NUMBER: PQE180019 / PQE180021
COURSE NAME: Masters in ESL
DEPARTMENT: Department of Language & Literacy
COURSE CODE: PQE 7011

LECTURER: DR ZUWATI HASIM

DATE OF SUBMISSION: 14 / 12 / 2018

 CONTENTS
ABSTRACT.........................................................................................................................................................................4

INTRODUCTION..............................................................................................................................................................5

USE HEADING 1 FROM THE SELECTION ABOVE FOR YOUR MAIN HEADING. USE ALL CAPS, DO
NOT USE ANYTHING ELSE AS THE TABLE OF CONTENTS HAS BEEN AUTOMATED TO USE THIS
SETTING............................................................................................................................................................................5

SUB HEADING...................................................................................................................................................................5
Sub-Sub-Heading........................................................................................................................................................5
Referencing..................................................................................................................................................................5

PROBLEM STATEMENT................................................................................................................................................6

OVERVIEW........................................................................................................................................................................6
PRESENTING ISSUES AND GAPS IN RESEARCH, CONCLUDE WITH YOUR RESEARCH AIMS.................................................6

RESEARCH OBJECTIVES..............................................................................................................................................7

OVERALL OBJECTIVE........................................................................................................................................................7

BACKGROUND AND SIGNIFICANCE OF THE STUDY..........................................................................................8

RESEARCH DESIGN AND METHODS........................................................................................................................9

OVERVIEW........................................................................................................................................................................9
POPULATION AND STUDY SAMPLE...................................................................................................................................9
SAMPLE SIZE AND SELECTION OF SAMPLE......................................................................................................................9
SOURCES OF DATA...........................................................................................................................................................9
COLLECTION OF DATA......................................................................................................................................................9
DATA MANAGEMENT.......................................................................................................................................................9
DATA ANALYSIS PROCEDURES.........................................................................................................................................9
ETHICS CONSIDERATIONS.................................................................................................................................................9
GANTT CHART..................................................................................................................................................................9

REFERENCES.................................................................................................................................................................10

APPENDICES...................................................................................................................................................................11

APPENDIX 1: QUESTIONNAIRE........................................................................................................................................11
Appendix 2: Interview Question....................................................................................................................................12

2
 ABSTRACT

This research was carried out to see how the number of teaching experience influences the
evaluation of the essay compositions and what aspect do the raters evaluate in the writing
composition without scheme of rubric provided. Using rater interviews and L2 learners’
writing composition as both quantitative and qualitative approaches, this study examines
the raters’ background knowledge and evaluation of L2 leaners’ in writing composition. 20
secondary teachers from various part of states in Malaysia with different experience years
of teaching were selected to rate Form 5 students argumentative writing compositions. Out
of 80 essay compositions, 30 were selected. Each raters were given the same set of 30
essays to evaluate and rate. Raters were given a month time interval to evaluate holistically
and analytically. Rubrics were not given to the raters as it will influence the findings. A wide
berth in the marking of essay compositions was seen when compared to experience raters
and without experience raters. Similarly, qualitative results showed that all raters tended to
use holistic assessment method because without specific scheme, it helped them not only
assign fair and objective scores for the essays but also facilitate the evaluation process. It
can be concluded that content of an essay was the most important factor that most affected
their holistic scoring decision marking of an essay was their experience in teaching and
how it shaped their thinking to evaluate which aspect to mark in the writing composition.

3
 PROBLEM STATEMENT

Overview

In recent years, issues of poor English writing skill have become a

major issue even writing is the most demanding skill among the four. In
general, English practitioners have a belief that their experience or
years of teaching, own experience in writing and the types of students
they had taught can affect the way they evaluate their students' writing.
However, a small empirical study was done to evaluate this to find out
the actual truth about the belief. As far as we are concerned, no
concrete research has been found that explores the number of
experiences of raters in rating English Language papers or the
differences in evaluation identified. However, we had managed to find
certain researches that explored the effect of rater’s general English
teaching experience. We noted that several researchers had observed
how influential rater’s experience especially when it involved with the
culture and language of ESL writers on L2 writing evaluation. Hamp-
Lyons (1989) described that native English speakers have the tendency
to be favourable or biased either in a positive or negative way towards
English writing compositions based on their experience with the culture
and language of the learners. In an effort to explore how rater’s
background knowledge can shape one’s ability to rate essay
compositions, we want to conduct the present study to examine how
the number of years of teaching experience will influence the evaluation
of L2 learner’s writing composition.

4
 RESEARCH OBJECTIVES

Overall Objective

THE OBJECTIVES OF THIS RESEARCH ARE:

TO DETERMINE WHETHER THE NUMBER OF YEARS OF A

RATER’S TEACHING EXPERIENCE INFLUENCES MARKING OF
L2 LEARNERS’S ESSAY COMPOSITION

TO IDENTIFY WHAT CRITERIA OR ASPECTS CONSIDERED

WHEN EXPERIENCE RATERS AND NON-EXPERIENCE RATERS
MARK ESSAY COMPOSITION

TO IDENTIFY RATER’S TEACHING EXPERIENCE DETERMINES

THE RELIABILITY OF EVALUATION OF L2 LEARNERS’
WRITING COMPOSITION

5
 BACKGROUND AND SIGNIFICANCE OF THE STUDY

2.1 ESL Academic Writing

Writing is an important prerequisite for active participation in modern democracy,

but it is also important source of modern inequality in education. Writing can be
difficult skill to learn and teach as it is not an easy cognitive activity; more it is
believed that this is a complex mental process that requires ‘careful reflection’,
discipline and concentration (Grami, 2010, p.9).

2.2 Writing Assessment Challenge

Problems and issues in writing has changed and become more difficult. The key
issue in the written assessment certainly goes beyond the specific issues, problems
or technical problems. Different interpretations of evaluative headings used by
different teachers lead to unreliable evaluations which should be addressed by
developing appropriate headings at the local level for specific groups of language
learners (Rezaei and Lovorn, 2010).

2.3 Scoring Perception of Types of Tasks

The concern of teachers about the evaluation of the essay is understandable and
not just aimed at measuring the effectiveness of their work. In fact, this largely
determines the content of their work. Teachers even feel more powerful that they
can evaluate and often have little impact on the evaluation criteria. Douglas (2000)
points out that the above comments light up the determination of the objective, the
procedures to be followed, the format of the task and finally the evaluation criteria
as they illustrate the relative weight associated with several features of writing. This
acts as a guiding language which gives real account of the context.

6
Research Design and Methods

Overview

Population and Study Sample

The population of this study is 80 students and 20 raters.

Sample Size and Selection of Sample

Sample size of this study is 50 participants (30 students and 20 raters).

Sources of Data

The sources of the data were taken from students’ writing composition and audio
recorder and rubric scheme prepared just for researcher’s guidance.

Collection of Data

The data collection method used semi-structured interview and document analysis.

Data Management

Data stored in Google drive and the control bias is the random selection of raters.

Data Analysis Procedures

Pie chart and data analysis were used to show the frequency of raters’ marking the
essay composition and how the teaching experience influences the reliability of
their marking based on rubric scheme.

Ethics Considerations

The data and name of the participants were not going to be revealed as it was
confidential.

Gantt Chart

7
 REFERENCES

Douglas, D. (2000). Assessing languages for specific purposes. Ernst Klett Sprachen.

Grami, G. M. A. (2010). The Effects of Integrating Peer Feedback into University-Level ESL
Writing Curriculum: A.

Rezaei, A. R., & Lovorn, M. (2010). Reliability and validity of rubrics for assessment through
writing. Assessing writing, 15(1), 18-39.

8
 APPENDICES

Appendix 1: Argumentative Essay

Social networking has caused a lot of problems. How far do you agree? Your composition
should be not more than 500 words.

______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________

9
Appendix 2: Semi-structured Interview Questions
Answer from each respondent will be videotaped and consent forms are handed out before interview takes
place.

Name of interviewer :
Name of respondent :
Place of interview :
Date of interview :

1) What is your qualification to be a teacher?

2) When was your first posting and where are you working at present?

3) How many years of teaching experience?

4) Have you been a SPM marker in the past, present or going to be one in future?

5) Can you recall any bitter or sweet moments in marking?

6) When you start to evaluate an essay, what is the first thing that you do as a rater?

7) What aspects do you take into account when you mark a student’s writing compositions?

8) Do you give pity marks when marking student’s compositions even when their writing does not
reflect the mark that you gave?

9) What is the hardest decision as a rater you have to take when evaluating student’s essay
compositions?

10) As a teacher, have you ever felt that you had been bias towards students when marking their
compositions? Can you honestly tell that you never been bias in marking? Why? How do you justify
it?

10