You are on page 1of 5

PAPER

A SURVEY OF EFL COLLEGE LEARNERS PERCEPTIONS OF AN ON-LINE WRITING PROGRAM

A Survey of EFL College Learners Perceptions


of an On-Line Writing Program
http://dx.doi.org/10.3991/ijet.v11i04.5459

Huaqing He
China West Normal University, Nanchong, China

AbstractThis survey adopted quantitative and qualitative were high. However, these studies also pointed out Pigai-
methods to investigate 209 Chinese college learners percep- wangs drawbacks.
tions of an on-line writing program, Pigaiwang, in a normal Above all, because of the various pitfalls in the use of
university. The survey questionnaire and interview results writing programs with EFL learners, writing teachers need
indicated that EFL college learners attach great importance to observe the possible benefits and drawbacks for their
to English writing, but they find English writing difficult, students. Thus, the present study aims to gain a deeper
particularly in terms of vocabulary and grammar; the ma- understanding of a specific on-line writing program, Pi-
jority of college learners showed slightly more positive atti- gaiwang, from the learners viewpoints, in an EFL con-
tudes toward using Pigaiwang as a writing tool than as an
text.
essay grader; computermediated feedback could enlarge
learners vocabulary size, and reduce the grammar and A. Brief Overview of the Writing Software, Pigaiwang,
spelling linguistic errors in their writing compositions; the
Used in this Study
majority of college learners thought writing with Pigaiwang
could improve learners English writing learning autonomy Currently, more and more universities in China are us-
and they hoped to use on-line writing programs in future ing computerassisted writing programs in their teaching
English writing. of writing. One of the most commonly used on-line writ-
ing programs is Pigaiwang (http:// www.pigai.org), based
Index Termson-line writing program, EFL, computer- on corpuses and cloud computing, which claims to have
mediated feedback, college learner attracted over 100,000 customers from more than 1000
universities. Pigaiwang was offered to college students in
China West Normal University in the spring semester of
I. INTRODUCTION the 2014 academic year to supplement classroom instruc-
With the rapid development of computer technology tion due to the insufficient number of English teachers in
and the introduction of the internet, the increasing im- the university. In fact, this was the first time computer-
portance of computer-assisted language learning and based writing software had been applied to any English
computer-assisted language instruction has greatly influ- composition class in the university.
enced both writing instruction and writing research in Pigaiwang was used both as an essay grader and as a
recent years. writing tool in the writing course. As an automated essay
The history of Automated Essay Scoring (AES) can be evaluation tool, teachers assign writing tasks online and
traced back to the 1960s in America with the development give students the composition number. Students then write
of Page Essay Grade (PEG), a program that used multiple multiple drafts before the deadline and receive immediate
regression analysis of measurable text features to build a feedback in the form of both holistic scores and diagnostic
scoring model based on a corpus of essays previously comments on grammar, organization, lexical usage, and
graded by hand [1]. Recently, more and more AES pro- content. The electronic grader in the writing program was
grams have appeared and can be used both as assessment designed to check for lexical complexity, syntactic varie-
and learning tools. These AES programs, such as Criteri- ty, topic content, and grammatical errors, and students
on, My Access, Writing Roadmap, and Pigaiwang, com- could then revise and submit their compositions repeated-
bine a scoring engine, an editing tool (offering grammar ly, according to each simultaneous feedback.
and spelling feedback), and some support resources (such The on-line writing program also has other important
as dictionaries). additional functions, such as a plagiarism checker, dead-
Research on AES systems is classified into three types. line setting, peer evaluation, sample composition reading,
The first type of research is on the validity of the software teacher grading, and similarity comparison, which writing
[2], while the second focuses on the learning outcome and teachers can choose to use when assigning composition
explores whether the writing software improves learners tasks.
writing skills [3], [4]. The third type of research has con-
centrated on the process of using the writing software in II. RESEARCH METHODOLOGY
the classroom [5], [6].
Previous studies conducted in China have discussed the A. Research Question
use of Pigaiwang in the writing classroom [7], [8], [9] and This study examined quantitative and qualitative as-
their results have indicated that its use could enhance pects of using the on-line writing program Pigaiwang in
learners English writing proficiency to some degree. Ref. an EFL writing course based on learners perceptions. The
[10] found that the reliability and validity of Pigaiwang

iJET Volume 11, Issue 4, 2016 11


PAPER
A SURVEY OF EFL COLLEGE LEARNERS PERCEPTIONS OF AN ON-LINE WRITING PROGRAM

following questions were specifically discussed in the Items 1 to 7 relate to the importance and degree of dif-
study: ficulty of college English writing (see Tables I, II, and
What are the learners perceptions of college English III). The results of the data analysis in Tables I and II
writing? demonstrate that the mean scores were above 3.0 for all
seven items, and above 3.50 for the majority of items.
What are the learners attitudes toward the use of Pi- Items 1 to 7 in Table I indicate the degree of students
gaiwang in college English writing? agreement about the importance and degree of difficulty
What are the learners perceptions of using Pigai- of college English writing.
wang in the future? As shown in Table I, the mean score of Item 1, I find
B. Participants college English writing important, was 4.25 and suggest-
ed that the majority of learners have high positive attitudes
The participants in this study comprised 209 sopho- toward college English writing. To be specific, 93.8% of
mores, including 64 Chinese education majors, 51 chemis- learners find college English writing important (see Table
try education majors, and 94 geography education majors III).
in China West Normal University, who volunteered to fill
out a questionnaire. The questionnaire was given to these Table I also shows that the mean score of Item 2, I find
college English writing difficult, was 3.94, and Table III
sophomores who were divided into four classes on June 8
shows that 85.0% of students find college English writing
and 9, 2015 during the English class at the end of the
difficult.
spring semester. All of these Chinese students had regis-
tered on the website www.pigai.org at the beginning of the Items 3 to 7 investigate what aspects cause difficulty in
spring semester and had completed five compositions on- writing. These aspects include English vocabulary, gram-
line. On average, they revised each composition six times. mar, spelling, compositions content, and organization.
Among the 209 students who participated in the study,
28.9% were male and 71.1% were female. TABLE I.
DISTRIBUTION OF MEAN SCORES OF ITEMS 12 (N=209)
C. Research Instruments
Item Description mean S.D.
The data used in the study came from a questionnaire 1. I find college English writing important. 4.25 .75
completed by 209 sophomores and semi-structured fol-
2. I find college English writing difficult. 3.94 .49
low-up interviews conducted with eight students.
Questionnaire: The questionnaire consisted of three
TABLE II.
parts containing 25 questions in total. The survey was DISTRIBUTION OF MEAN SCORES OF ITEMS 37 (N=209)
carried out anonymously to reduce the potential for un-
comfortable feelings among the participants. Part 1 con- Item Description mean S.D. Rank
tained 20 items that were designed around a five-point 3. During English writing, I find vocabulary
3.87 .84 1
Likert-type scale (1 for strongly disagree, 2 for disagree, 3 difficult.
for not sure, 4 for agree, and 5 for strongly agree) to iden- 4. During English writing, I find grammar
3.67 .99 2
tify the learners attitudes toward college English writing difficult.
and their perceptions of using Pigaiwang in college Eng- 5. During English writing, I find content
3.02 .99 5
lish writing. After collection!data were coded using difficult.
SPSS21. For the purposes of analysis, part 2 included two 6. During English writing, I find organization
3.14 1.02 4
open-ended questions: Item 21, designed to elicit students difficult.
preference for computer feedback or teacher feedback, 7. During English writing, I find spelling
3.50 .90 3
and Item 22, designed to elicit students judgments on difficult.
Pigaiwang. Part 3 contained three items concerning partic-
ipants background information, such as gender, major, TABLE III.
and their perceptions of using Pigaiwang in the future. PERCENTAGE OF ITEMS 17 (N=209)
Interview: Interviews with the eight student participants Items Strongly Disagree Not Agree (%) Strongly
took place individually on June 11. The semi-structured disagree (%) sure agree (%)
interviews involved open-ended questions concerning (%) (%)
perceptions of using Pigaiwang, which may not have been 1 1.9 1.4 2.9 57.7 36.1
captured by the Likert-type questionnaire items.
2 0 0 15.0 75.8 9.2
3 0.5 10.1 9.6 62.0 17.8
III. RESULTS
4 1.4 15.9 14.4 51.0 17.3
A. Research Question 1: What Are the Learners 5 1.9 38.3 19.9 35.4 4.4
Perceptions of College English Writing? 6 1.4 34.8 19.3 37.7 6.8
The learners perceptions of college English writing 7 0.5 17.0 24.3 48.5 9.7
were discussed from two aspects, i.e., its importance and
As shown in Table II, Item 3, the mean score of Item 3
degree of difficulty. From the questionnaire results, the
is the highest, which suggests that students find English
importance and degree of difficulty of college English
vocabulary the most difficult aspect of college English
writing were analyzed in terms of the mean scores on a
writing, followed by English grammar, spelling, organiza-
five-point Likert-type scale and percentages of their an-
tion, and content. To be specific, Table III vividly reflect-
swers. Standard deviations, means, and percentages of
ed the degree of difficulty. In total, 79.8 % of students find
participants responses are listed and reported in Tables I,
English vocabulary difficult during college English writ-
II, and III.

12 http://www.i-jet.org
PAPER
A SURVEY OF EFL COLLEGE LEARNERS PERCEPTIONS OF AN ON-LINE WRITING PROGRAM

ing (Item 3), 68.3% find English grammar difficult (Item mean scores of the other six items were between 3.0 and
4), 58.2% find English spelling difficult (Item 7), 44.5% 4.0. Items 15 to 20 relate to the learners perceptions of
find writing organization difficult (Item 6), and only using Pigaiwang, particularly as a writing tool.
39.8% find writing content difficult (Item 5). As Table VI illustrates, apart from the mean scores of
only two items (17 and 18) that were below 3.50, the
B. Research Question 2: What Are the Learners mean scores of the other four items were above 3.50. This
Attitudes Toward the Use of Pigaiwang in College suggests that the majority of students seemed to have
English Writing? more positive attitudes toward the use of Pigaiwang as a
From the questionnaire results, the student participants writing tool.
attitudes toward the use of Pigaiwang were analyzed in In addition, the results presented in Table VII show that
terms of the mean scores of their answers on a five-point 70.3% of students agreed that they would revise their
Likert-type scale. Standard deviations and means of par- compositions according to the computer feedbacks, while
ticipants responses are listed and reported in Tables IV, only 11.0% of them disagreed (Item 15). A total of 75.4%
V, VI, and VII. of students agreed that writing compositions with Pigai-
Learners attitudes toward using Pigaiwang as an es- wang helps them reduce their compositions mistakes in
say grader: Items 814 relate to the use of Pigaiwang as grammar and spelling (Item 16). A total of 71.7% of
an essay grader (see Table IV). The results of the data learners agreed that writing compositions with Pigaiwang
analysis showed that the mean scores were above 3.0, but helps them to enrich their compositions vocabulary (Item
below 4.0 for all seven items. 19). However, less than half of students agreed that writ-
As shown in Table IV, the mean score of Item 8, I feel ing compositions with Pigaiwang helps them to improve
satisfied with the automated grading system of Pigai-
wang, was 3.45 and the mean scores of items 9 and 10, TABLE V.
PERCENTAGE OF ITEMS 814 (N=209)
which were related to the use of Pigaiwang as an essay
grader, were both above 3.0, but below 3.50. The mean Strongly Disagree Not sure Agree Strongly
scores of the other items, which related to the use of Pi- Items
disagree (%) (%) (%) (%) agree (%)
gaiwang as an essay grader, were all above 3.50. 8 1.4 18.2 27.3 45.5 7.6
Items 8 to 14 in Table V also indicated the degree of 9 2.9 19.6 22.5 50.7 4.3
students agreement on the use of Pigaiwang as an essay 10 1.5 18.9 30.1 46.6 2.9
grader. In fact, the result of Item 8 showed that 53.1% of
11 1.0 10.6 26.1 59.9 2.4
students felt satisfied with the automated grading system
of Pigaiwang; 19.6% of learners were dissatisfied with it 12 1.0 8.6 20.7 67.8 1.9
while 27.3% said they were not sure. In particular, over 13 1.4 13.9 19.1 63.2 2.4
50% of learners agreed with the other items (Items 9, 11, 14 1.9 6.9 15.5 69.9 5.8
12, 13, and 14), except for Item 10, which was related to
the use of Pigaiwang as an essay grader. Therefore, more TABLE VI.
than half the learners were satisfied with Pigaiwang as an DISTRIBUTION OF MEAN SCORES OF STUDENTS ATTITUDES TOWARD
THE USE OF PIGAIWANG AS A WRITING TOOL (N=209)
essay grader.
Learners response to the use of Pigaiwang as a writing Item Description mean S.D. Rank
tool: The results of the data analysis revealed that the 15. I will read the computer feedback and revise
3.69 .85 4
my compositions after using Pigaiwang.
TABLE IV. 16. Writing compositions with Pigaiwang helps
DISTRIBUTION OF MEAN SCORES OF STUDENTS ATTITUDES TOWARD me reduce my compositions mistakes in gram- 3.76 .72 1
THE USE OF PIGAIWANG AS AN ESSAY GRADER (N=209) mar and spelling.
Item Description mean S.D. Rank 17. Writing compositions with Pigaiwang helps
3.27 .80 6
me improve my compositions organization.
8. I feel satisfied with the automated grading
3.45 1.22 5 18. Writing compositions with Pigaiwang helps
system of Pigaiwang. 3.28 .84 5
me improve my compositions content.
9. I feel satisfied with the comments on my
compositions provided by the automated grading 3.34 .94 6 19. Writing compositions with Pigaiwang helps
3.71 .72 2
system of Pigaiwang. me enrich my compositions vocabulary.
10. I feel satisfied with the computer feedback to 20. Writing compositions with Pigaiwang helps
3.70 .80 3
English grammar provided by the automated 3.31 .86 7 me improve my English writing.
grading system of Pigaiwang.
11. I feel satisfied with the computer feedback to TABLE VII.
the organization of my composition provided by 3.52 .76 3 PERCENTAGE OF ITEMS 17 (N=209)
the automated grading system of Pigaiwang.
Not
12. I feel satisfied with the computer feedback to Strongly Disagree Agree Strongly
Items sure
the content of my composition provided by the 3.61 .71 2 disagree (%) (%) (%) agree (%)
(%)
automated grading system of Pigaiwang.
15 1.4 9.6 18.7 59.3 11.0
13. I feel satisfied with the computer feedback to
the sentences of my composition provided by the 3.51 .82 4 16 0 8.2 16.4 66.7 8.7
automated grading system of Pigaiwang. 17 0.6 16.4 42.5 36.2 4.3
14. I feel satisfied with the computer feedback to 18 1.5 17.2 36.4 41.6 3.3
English vocabulary provided by the automated 3.71 .76 1 19 1.5 5.3 21.5 66.0 5.7
grading system of Pigaiwang.
20 1.4 8.7 16.7 65.1 8.1

iJET Volume 11, Issue 4, 2016 13


PAPER
A SURVEY OF EFL COLLEGE LEARNERS PERCEPTIONS OF AN ON-LINE WRITING PROGRAM

their compositions organization and content (Items 17 that 70.57% of the students find college English writing
and 18). A total of 73.2% of learners agreed that writing difficult. In particular, the majority of students find Eng-
compositions with Pigaiwang helps them improve their lish vocabulary most difficult, followed by grammar and
English writing, while only 10.1% of them disagreed spelling, during the English writing process.
(Item 20). In general, the students responses to the use of A number of studies have discussed the negative and
Pigaiwang as a writing tool were positive. positive effects of EFL learners attitudes toward comput-
er-assisted writing programs in English writing instruc-
C. Research Question 3: What Are the Learners tion. Ref [15]s survey in Hong Kong suggested that EFL
Perceptions of Using Pigaiwang in the Future? writers drafts in traditional classes improved more than
Learners reactions to the suggestion of using Pigai- those in classes using LAN software programs. However,
wang in the future came from both the questionnaire data [5]s study (2010) and the present study found that the
and the interview data. A total of 88.9% of learners sug- majority of learners had favorable attitudes toward the use
gested that Pigaiwang be used in the future English writ- of computer-assisted writing programs as a writing tool.
ing class, while only 11.1% did not want to use it any- Furthermore, the results of the survey questionnaire and
more. The eight interviewees all felt happy that they could interview indicated that the participants gave positive
obtain immediate computer feedback; they then revised responses to the timely computer-mediated feedbacks for
their compositions according to that feedback and felt a revising their compositions, when using on-line writing
sense of achievement if they achieved high scores in their programs. In particular, they benefited greatly from com-
composition. Seven of the interviewees mentioned that puter-mediated feedbacks in English vocabulary, gram-
Pigaiwang offered them good opportunities to practice mar, and spelling, which they found very difficult in Eng-
English writing in and out of class and revise their compo- lish writing. In addition, the majority of interviewees
sitions without time limitations; six of them indicated that mentioned that writing essays with the on-line writing
writing compositions with Pigaiwang would reduce the program could promote their confidence and learning
teachers workload; five interviewees thought Pigaiwang autonomy. Moreover, both the survey questionnaire and
provided them with a large amount of vocabulary interview results showed that essays written with the on-
knowledge, could enlarge their vocabulary size, and could line writing program could improve learners English
improve their learning autonomy. Four of the eight inter- writing and learners suggested that Pigaiwang be used in
viewees said they often read peers compositions and future English writing class. Thus, the adoption of on-line
sample compositions recommended by teachers after they writing programs is worth taking into consideration in
submitted their compositions. Only one interviewee did EFL writing classes.
not suggest using Pigaiwang in the future class, because On the other hand, the results of the survey question-
he did not like to do homework online. Six interviewees naire revealed that slightly more than half of students were
thought the computer feedbacks were inaccurate and un- satisfied with the use of the on-line writing program as an
specific and suggested that teacher feedback and computer essay grader. It suggested that students held less positive
feedback be combined. attitudes toward the use of the on-line writing program as
an essay grader than as a writing tool and thus further
IV. DISCUSSION corroborated [5]s study. Additionally, the findings from
From the results of the survey questionnaire, college the open-ended Item 21 showed that the majority of par-
students attached great importance to English writing, but ticipants (80%) preferred teachers written feedback to the
found English writing very difficult. More importantly, computer feedback given by Pigaiwang. Moreover, the
the students attitudes toward writing compositions with findings from the open-ended Item 22, designed to elicit
the on-line writing program Pigaiwang have been investi- students judgments on Pigaiwangs merits and pitfalls,
gated. The results have revealed the students favorable found that computer-mediated feedback is too general and
attitudes toward the use of Pigaiwang as a writing tool, but unspecific despite being timely, so students sometimes did
a slightly less positive attitude toward its use as an essay not know exactly how to revise their compositions. Fur-
grader. thermore, the results of interview showed that the auto-
mated grading system was sometimes inaccurate, because
Some studies have discussed the importance of English
the compositions scores decreased after being revised
writing. For example, a survey by [11] showed that among
based on computer-mediated feedbacks. Therefore, a large
the four English language skills, writing would be badly
portion of learners suggested that computer feedback and
needed in the future in China and approximately 86% of
teacher feedback be combined and that computers cannot
the interviewees hold the opinion that the ability to write
replace the teachers role.
scientific reports or essays is the most important skill. The
present survey corroborated these previous studies, with
93.7% of learners finding college English writing im- V. CONCLUSION
portant. This study has a number of limitations. The first and
At the same time, a few studies investigated the degree biggest limitation of this study may stem from the limited
of difficulty in writing for EFL learners. In comparison sample size. Only 209 students were investigated as sur-
with the other three language skills, writing is the most vey subjects and 8 students were selected as interviewees,
difficult [12]. Ref. [13] also confirmed this belief, pointing all of whom came from the same normal university. No
out that writing is a complicated process and the ability to doubt, a larger sample size may lead to a more objective
write well is usually considered to be the final skill in analysis. The second limitation is that the study did not
language learning. According to [14]s survey, 65% of compare learners perceptions of on-line writing programs
subjects agree that writing is the most difficult skill to with regard to different English proficiency levels.
acquire. This result proved the above researchers results,

14 http://www.i-jet.org
PAPER
A SURVEY OF EFL COLLEGE LEARNERS PERCEPTIONS OF AN ON-LINE WRITING PROGRAM

While the results of the study did shed light on some [7] C. Gu and L. Wang, Empirical study of college English writing
aspects related to the use of the on-line writing program, teaching based on Juku correcting network, Journal of Yangzhou
University, vol. 16(4), pp. 9296, 2012.
most findings were that the majority of learners attach
[8] X. Shi, A tentative study on the validity of online automated
great importance to English writing, which they find diffi- essay scoring used in the teaching of EFL writing exemplified
cult, and preferred the use of Pigaiwang as a writing tool by http//www.pigai.org, Modern Educational Technology, vol.
to its use as an essay grader. In addition, the majority of 22(10), pp. 6771, 2012.
learners hope to use Pigaiwang in future English writing [9] F. Chen, A study on the application of pigai.org software in
classes. Moreover, most learners benefited from comput- English writing, Journal of East China Institute of Technology,
er-mediated feedbacks in their English linguistic Vol. 33(2), pp. 173177, 2014.
knowledge, particularly in English vocabulary, grammar, [10] X. He, Reliability and validity of the assessment by the Pigai-
and spelling, which they found very difficult during Eng- wang on college students writings, Modern Educational Tech-
nology, vol. 23(5), pp. 6467, 2013.
lish writing. Additionally, writing English essays with
Pigaiwang could enhance learners learning autonomy and [11] M. Li and L. Gong A survey on English need in the science and
technology world, Foreign Language World, vol. 4, pp. 2024,
writing level. Therefore, it is crucial that the on-line writ- 1994.
ing program be a useful supporting tool and is worth [12] L. Sun, English Writing , Shanghai: Shanghai Foreign Language
adopting in EFL writing classes. Education Press,1995.
[13] D. Nunan, Language Teaching Methodology . New York: Prentice
REFERENCES Hall, 1991.
[1] E. Page, Project essay grade: PEG, in Automated Essay Scoring: [14] Y. Gu, An analysis on errors made by senior high school students
A Cross-disciplinary Perspective, M. D. Shermis and J. Burstein, in English writing, unpublished masters thesis, Shanghai Inter-
Eds. Mahwah, NJ: Lawrence Erlbaum Associates, 2003, pp. 43 national Studies University, 2006.
54. [15] G. Braine, A study of English as a foreign language (EFL) writ-
[2] J. Wang and M. S. Brown, Automated essay scoring versus ers on a local-area network (LAN) and in traditional classes,
human scoring: A comparative study, Journal of Technology, Computers and Composition, vol. 18, pp. 275292, 2001.
Learning, and Assessment, vol. 6(2), pp. 428, 2007. http://dx.doi.org/10.1016/S8755-4615(01)00056-1
[3] M. D. Shermis, J. Burstein, and L. Bliss, The impact of automat-
ed essay scoring on high stakes writing assessments, Paper pre- AUTHOR
sented at the Annual Meeting of the National Council on Meas-
urement in Education. San Diego, CA, April 2004. Huaqing He is with School of Foreign Languages,
[4] J. Tang and Y. Wu, Using automated writing evaluation in the China West Normal University, 637009, Sichuan, China.
classroom: A critical review, Journal of Foreign Language
This work was supported in part by the the Department of Education of
Teaching and Research, vol. 43(2), pp. 115124, 2011.
Sichuan Province, China under Grant No. 13SA0015 (sponsor and finan-
[5] Y. Fang, Perceptions of the computer-assisted writing program cial support acknowledgment goes here).
among EFL college learners, Educational Technology & Society, Submitted 31 July 2015. Published as resubmitted by the authors 13
vol. 13(3), pp. 246256, 2010. September 2015.
[6] J. Tang, Technology-enhanced English language writing assess-
ment in the classroom, Chinese Journal of Applied Linguistics,
vol. 35(4), pp. 385399, 2012.

iJET Volume 11, Issue 4, 2016 15