Professional Documents
Culture Documents
CALL
CALL
net/publication/232868537
CITATIONS READS
52 762
1 author:
SEE PROFILE
All content following this page was uploaded by Shao-Ting Alan Hung on 18 July 2015.
To cite this article: Shao-Ting Alan Hung (2012) A washback study on e-portfolio assessment in an
English as a Foreign Language teacher preparation program, Computer Assisted Language Learning,
25:1, 21-36, DOI: 10.1080/09588221.2010.551756
Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
Computer Assisted Language Learning
Vol. 25, No. 1, February 2012, 21–36
Downloaded by [NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY] at 21:55 06 November 2013
Background
Washback refers to ‘‘the effect of testing on teaching and learning’’ (Hughes,
2003, p.1). More specifically, it is generally known as the positive or negative
influences tests have on teachers’ instruction and students’ learning. A number of
studies have explored washback effects that standardized exams have brought to
language learning and teaching. For instance, Watanabe (1996) examined the
effect of the university entrance examination on the use of the grammar-
translation method in Japan and found that the entrance examination failed to
play any significant role in the choice of teaching methodology. Rather, it was
teacher factors that dictated how the course would be taught. Cheng, Klinger &
Zheng (2007) also conducted a washback study to investigate the impact of a
*Email: alanhung123@gmail.com
large-scale literacy test on second language (L2) students in Canada. The results
indicated that the reading test formats, text types, skills, strategies, and writing
tasks impacted L2 and L1 learners differently and significantly. Finally, Alderson
and Hamp-Lyons (1996) discovered that the Test of English as a Foreign
Downloaded by [NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY] at 21:55 06 November 2013
Several efforts have been made to research how e-portfolios work as a learning tool
in teacher training programs. MacDonald et al. (2004) carried out a case study,
investigating graduate-level preservice teachers’ perspectives on the development of
electronic portfolios. The findings suggest that the most commonly reported discovery
Downloaded by [NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY] at 21:55 06 November 2013
was that the sharing and peer review portion of the class led to the most learning. In
other words, seeing other people’s work was noted as a continuous process for revision,
reselection of artifacts, and interface design. Briefly put, participants went through a
great deal of reflection and spent much time in redesign and reselection of portfolio
entries. Yang (2009) investigated the use of blogs as a reflective platform in the English
as a Foreign Language (EFL) student teachers training program. The results indicated
that student teachers were active in discussing language teaching theories and how these
theories can be used in real classrooms. Student teachers also critically reflected on their
learning and made valuable comments. Next, Sung et al. (2009) uncovered the
effectiveness of digital teaching portfolios in an in-service teacher training program. The
results showed that most teachers demonstrated moderate levels of reflection, but only
one-third of them showed the highest level of reflection. They also found that e-
portfolios with self-assessment, peer assessment, discussion, and journal writing might
enhance the professional growth of teachers. Finally, van Olphen’s (2007) study
focused on language teacher candidates’ views on developing digital portfolios and
concluded that digital portfolios could document evidence of students’ progress and
offer an opportunity for reflective thinking. In summary, these studies used e-portfolios
as learning tools to enhance teachers’ professional development. However, using
e-portfolios as an assessment tool still remained under-explored.
Blogs as e-portfolios
Because of the increasing interest in Web 2.0 technologies, blogs have experienced
phenomenal growth in recent years (Churchill, 2009; Godwin-Jones, 2006, 2008).
Downloaded by [NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY] at 21:55 06 November 2013
Serving as one type of e-portfolios, blogs enable users to interact with and have their
work viewed by others inside and outside the classroom (Richardson, 2006) and to
move from being observers to participants. In other words, participatory practice is
greatly emphasized in public blog domains, also called the blogosphere (Bloch,
2007). Moreover, Godwin-Jones (2003) asserts that language learners could use a
personal blog as an electronic portfolio to show development over time. Learners are
given a chance to write for audiences beyond classmates, which encourages them to
be more thoughtful and responsible for what they write. Finally, the archiving of
blog entries enhances learners’ reflection and cultivates metacognitive strategies for
monitoring the learning process (Richardson, 2006).
Research questions
Method
The setting and participants
The e-portfolio assessment project was implemented in a graduate-level content course
in a national university located in the southern part of Taiwan. The content course,
Language Assessment, aimed to provide prospective EFL teachers with an overview,
theories, and practices of language assessment and to enhance their skills of developing
effective language tests and assessment instruments. The course met for one three-hour
session each week, for 18 weeks. The participants were 18 full-time first-year graduate
students in a Master’s program with a concentration on Teaching English to Speakers
of Other Languages (TESOL). Among them, five were males and 13 were females. The
ages ranged from 24 to 35 years. All of the participants had formal or informal
experiences of teaching EFL to children, teenagers, or adults for 1 to 10 years.
Computer Assisted Language Learning 25
ready-made site that requires no high-tech skills and allows users to upload
artifacts in multiple formats, such as texts, graphics, sound, and video files. It is
also equipped with functions not merely for users to organize and categorize their
entries based on various themes and topics but also for visitors to post
individualized feedback as a medium for asynchronous communication with users.
Second, Wretch is a well-known blog site among Taiwanese students. According
to statistics by Alexa Internet, Inc., a subsidiary company of Amazon.com known
for its browsing behavior analyses and web traffic reporting, Wretch is ranked the
second most visited website among the top 100 sites in Taiwan. Lastly, most of
participants in the current study had created their own e-portfolios on Wretch in
other classes or in their leisure time. Hence, these three reasons supported
the selection of Wretch as a platform for participants to maintain their
e-portfolios.
Implementation
Table 1 describes the detailed procedures of the e-portfolio project.
Stage 2: implementation
At the second stage, spanning from the third week to the end of the semester,
participants worked on their required assignments. The participants were required to
complete six types of assignments, which are explained as follows:
Stage Task
I. Project orientation and preparation Setting up e-portfolios at www.wretch.cc
(1st to 2nd week) Learning concepts of e-portfolio assessment
Forming portfolio groups of 2–3 persons for
peer commenting
II. Implementation (3rd to 18th week) Working on the assignments
Giving peer feedback for each assignment
Teacher commenting on the participants’
entries
Revising the assignments according to the
comments
26 S.-T.A. Hung
(2) Critical responses: The instructor occasionally posed some pedagogical scenarios
for discussion. For instance, one scenario was ‘‘The teacher feels that his students
revise what they write only minimally, and he is eager that they learn the value of
revision. How can be use portfolios to achieve this?’’ The participants responded
to the scenarios and commented on each other’s responses.
(3) Group assessment project: Students worked with their group members and
constructed an assessment project on specified skills, such as listening,
speaking, reading, and writing. They applied theories they learned from class
into the projects. The final project consisted of a rationale, target test-takers,
test specifications, test items, and grading criteria.
(4) Conference notes: Students attended two assessment-related paper presenta-
tions at a conference and wrote one-page conference notes on arguments and
findings of presentations.
(5) Self- and peer-assessment questionnaire: After each annotated journal article
was completed, participants filled out an open-ended self- and peer-
assessment questionnaire, analyzing their own and peers’ strengths, weak-
nesses, and areas for future improvement. (See Appendix 1 for self- and peer-
assessment questionnaire.)
(6) Reflective journal: At the end of the semester, each student wrote a reflective
journal, discussing what they learned in this content course. (See Appendix 2
for a description of the reflective journal assignment.)
Instruments
The current study collected data via interviews, observations, and document
analysis. First, interviewing is an effective means of eliciting information when the
researcher cannot observe participants’ behavior and other visible cues as to their
feelings or how they interpret the world around them. It is also important to collect
data through interviews when the researchers are interested in past events and
experiences that are impossible to replicate (Merriam, 1998). Therefore, aiming to
investigate washback effects of e-portfolio assessment on EFL student teachers’
learning of content knowledge, the researcher and research assistants conducted
three 30-minute semi-structured interviews with each of the students (see Appendix 3
Computer Assisted Language Learning 27
for detailed interview questions). Second, participant observations also allowed the
researcher to learn firsthand how the actions of the participants corresponded to
their words and to observe patterns of behavior. Third, document analysis was
performed because documents ‘‘corroborate the interviews and thus make the
Downloaded by [NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY] at 21:55 06 November 2013
findings more trustworthy. Beyond corroboration, they may raise questions about
the researchers’ hunches and thereby shape new directions for interviews’’ (Glesne,
1999, p. 58). In the current study, the documents in students’ e-portfolios included
required assignments (i.e., annotated journal articles, critical responses, self- and
peer-assessment questionnaire, conference notes, assessment project, and reflective
journal), self-chosen entries, peers’ responses, and teacher’s feedback.
Data analysis
The data analysis procedure followed a series of steps in an effort to generate themes
capable of providing triangulation information. First, all the qualitative data were
reviewed by the researcher and another qualified researcher who holds a doctorate in
TESOL. This peer debriefing process served to establish the credibility of the
interpretation of the data. While reviewing the data, a number of codes were
generated. Second, the researchers re-read the entire data set closely and labelled all
the data with the generated codes. Next, they carefully reviewed the codes and as
such combined the relevant ones into seven themes that represented positive and
negative washback effects. The detailed coding scheme that includes themes, codes
and examples is presented in Table 2.
Positive washback
According to Pearson (1988), assessments’ washback effects will be positive if they
are beneficial and encourage desired changes on teaching, learning, and curriculum.
Similarly, Cheng and Curtis (2004) argue that positive washback effects will be
generated when teachers and learners have a positive attitude toward the
assessment and work willingly and collaboratively toward assessment and learning
objectives. In the present study, a number of positive washback effects from e-portfolio
assessment were discovered, such as building a community of practice, facilitating peer
learning, enhancing learning of content knowledge, promoting professional develop-
ment, and cultivating critical thinking. Corresponding to what Cheng and Curtis
(2004) posited, these effects increased learners’ willingness to collaborate with peers
and provoked desired changes in learning content knowledge.
E-portfolios created a learning space for the class members. In addition to classroom
learning, we were given one more opportunity to exchange information and discuss
pedagogical issues.
Computer Assisted Language Learning 29
Downloaded by [NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY] at 21:55 06 November 2013
E-portfolios were like a place where we could view others’ entries to get some different
viewpoints on the same topics and learn from different perspectives. Sometimes when I
view other peers’ entries, and then I tell myself ‘wow, why didn’t I think of this idea?’
It is clear that this community serves as an open space for language teachers to
explore and describe their ideas. Similarly, it allows language teachers to
experiment with content knowledge and express themselves in a relaxed
environment. Therefore, as indicated by Wenger (1998), the community
encourages these language teachers to take responsibility for information sharing
and problem solving, to develop their personal identities in the community, and
to foster unification of the community.
I think e-portfolio is one of the useful tools for me to learn from others. I could read
other students’ assignments. I could learn from their ideas and their writing styles so
that I can improve my assignments.
In this portfolio approach, I benefited a lot from others’ feedback. For instance, I found
several constructive suggestions on my annotated journal articles from classmates. So,
when I give feedback, I also need to be more responsible and serious, not just
commenting on others’ assignments carelessly.
30 S.-T.A. Hung
I know more about the content knowledge after a semester of constructing e-portfolio.
When I write entries in my portfolio, I have to review the principles of language
assessment mentioned in class and in the textbook. I think my e-portfolio documents my
learning progress in this content course.
Because e-portfolio documents all my entries, it helps me review what I have learned in
this class.
I think my e-portfolio helped me organize my learning in this content course. For
example, in my e-portfolio, I included the summaries and reflections of some journal
articles under the topic of self-assessment. Every time when I write my assignment, I can
easily review my previous work and the feedback from peers and instructor.
During the learning, I can see my improvement by looking at those documents that I
collected. When I am teaching, I can always connect theories with teaching practices and
try to find some ideas from those documents.
With all the entries in my portfolio, it became easier to review principles and theories of
language assessment, like reliability and validity. When I design tests for my English
classes, I can go back to my portfolio and improve the tests.
Computer Assisted Language Learning 31
teachers observe, review, and discuss the content of portfolios, they learn to clarify
their previous conceptions about their own practices and think deeper on the merits
and drawbacks of their own work.
Moreover, some of these teacher trainees projected their future use of e-portfolios in
their EFL classrooms. For instance, one noted that she would use e-portfolios in her
children’s English classes to help pupils collect their learning products such as English
journal entries and self-created glossaries. Another described his future plan of using e-
portfolios as an assessment tool in a college writing class, saying that he would rate
each of his students’ writing samples as a formative assessment and grade the entire e-
portfolio at the end of semester as a summative assessment. Hence, it became clear that
some teacher trainees were already planning and evaluating how e-portfolios could be
integrated in their future classes.
Using e-portfolios to evaluate our learning is very different from using tests. E-portfolio
assessment encouraged me to think deeper on some learning issues and challenge both
my own and peer’s perspectives.
In this type of assessment, I usually re-evaluate my assignments in other people’s eyes. I
ask myself some questions and try to answer them from different angles.
Negative washback
In the traditional testing context, negative washback effects refer to the undesirable
effects on teaching and learning, such as overemphasis on memorization, practicing
exam techniques rather than language learning tasks, unnecessary test anxiety, and
failure to promote general understanding (Alderson & Wall, 1993; Cheng & Curtis,
2004). On the other hand, in alternative assessments context, particularly e-portfolio
assessment in the current study, negative washback was explored and described as
follows.
Anxiety
While e-portfolio assessment produced positive washback, it also generated some
negative impacts on learning. The first negative washback was anxiety derived from
the presence of the audience. Since e-portfolios created on Wretch are public domain,
32 S.-T.A. Hung
they were visited by not only class participants but also unknown users beyond the
classroom. These language teachers were somewhat concerned about their written
performance being observed by people other than the class instructor. Hence,
although giving and receiving feedback could enhance interaction and promote
Downloaded by [NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY] at 21:55 06 November 2013
I know my classmates will always read my postings. I am a little bit worried about the
quality of my assignments and the feedback I give to others. I sometimes keep quiet not
because I don’t have any ideas but because I don’t want all of the members to read my
postings.
I became anxious every time when I post my entries because I don’t know how my
classmates will look at them and how they will evaluate them.
In the traditional tests, learners’ performance is judged only by the teachers; however,
in e-portfolio assessment, their performance is observed and evaluated by larger
populations. When the audience increases, anxiety also increases. In Ward’s (2004)
blog-writing study, students reported that they felt that they had to write really good
topics because everyone would read their works. Worried about a similar issue, the pre-
service teachers in the present study experienced a certain level of apprehension.
Sometimes I can’t post my feedback on others’ entries. I tried several times but still
failed. I had to wait until the next day to dispatch my messages. It’s inconvenient.
The e-portfolio system sometimes can’t display the full text of my assignments properly.
It cut out words to the right of every line. I had to do the formatting again and again or
spend a lot of time adjusting the formats.
Compared with paper-based assignments, online assignments created more troubles. If
the system is down, my assignments can’t be posted on time.
Corresponding to van Olphen’s (2007) argument that technology was not infallible,
the current study found that although technology played an important role in the
portfolio development process, it could create some frustration for some student
teachers. Hence, solving technical difficulties would need to be addressed before e-
portfolio assessment could become effective in a language teacher education program.
Pedagogical implications
Based on the findings, two pedagogical implications are proposed to maximize the
efficacy of e-portfolio assessment. First, with all the positive washback effects,
Computer Assisted Language Learning 33
teachers’ growth may lead to a more balanced assessment approach that places equal
weight on the process and product of professional development.
Second, the negative washback on anxiety from larger audiences may derive from
participants’ discomfort of revealing personal competence on the Internet. In other
words, these EFL student teachers were worried that their entries might fail to meet
peers’ expectations or their postings may jeopardize their friendship. Hence, to tackle
their concern about overt performance, guidance should be designed to sustain
collaboration and socialization throughout the course. Alternatively, helping these
student teachers realize peers as learning resources rather than judges becomes
crucial.
Acknowledgement
My appreciation goes to anonymous reviewers for their comments and suggestions. The project
was sponsored by National Science Council, Taiwan (Project no. NSC 97-2410-H-327-029).
34 S.-T.A. Hung
Notes on contributor
Dr. Shao-Ting Alan Hung is an assistant professor in the Department of English and co-
ordinator of the Foreign Language Program at National Kaohsiung First University of
Downloaded by [NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY] at 21:55 06 November 2013
Science and Technology, Taiwan. His research interests include CALL, L2 writing pedagogy
and language assessment.
References
Alexa Internet, & Inc. (2010). Top sites in Taiwan. Retrieved February 2, 2010, from http://
www.alexa.com/topsites/countries/TW/
Alderson, J.C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: A case study.
Language Testing, 13, 280–297.
Alderson, J., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115–129.
Barrett, H. (2000). Create your own electronic portfolio. Learning and Leading with
Technology, 27(7), 14–21.
Bloch, J. (2007). Abdullah’s blogging: A generation 1.5 student enters the blogosphere.
Language Learning & Technology, 11(2), 128–141.
Camp, R., & Levine, D.S. (1991). Portfolios evolving. In P. Belanoff & M. Dickson (Eds.),
Portfolios: Process and product (pp. 194–205). Portsmouth, NH: Boynton/Cook.
Cheng, L. (2005). Changing language teaching through language testing: A washback study.
Studies in Language Testing 21. Cambridge: Cambridge University Press.
Cheng, L., & Curtis, A. (2004). Washback or backwash: A review of the impact of testing on
teaching and learning. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language
testing: Research contexts and methods (pp. 3–18). Mahwah, NJ: Lawrence Erlbaum
Associates.
Cheng, L., Klinger, D., & Zheng, Y. (2007). The challenges of the Ontario Secondary School
Literacy Test for second language students. Language Testing, 24, 185–208.
Churchill, D. (2009). Educational applications of Web 2.0: Using blogs to support teaching
and learning. British Journal of Educational Technology, 40(1), 179–183.
Darling-Hammond, L., & Snyder, J. (2000). Authentic assessment of teaching in context.
Teaching and Teacher Education, 16, 523–545.
Glesne, C. (1999). Becoming qualitative researchers. New York: Longman.
Godwin-Jones, R. (2003). Emerging technologies. Blogs and wikis: Environments for on-line
collaboration. Language Learning & Technology, 17(2), 12–16.
Godwin-Jones, R. (2006). Emerging technologies. Tag clouds in the blogosphere: Electronic
literacy and social networking. Language Learning & Technology, 10(2), 8–15.
Godwin-Jones, R. (2008). Emerging technologies. Web-writing 2.0: Enabling, documenting,
and assessing writing online. Language Learning & Technology, 12(2), 7–13.
Hancock, C.R. (1994). Alternative assessment and second language study: What and why?
Eric Digest, ED376695.
Hawisher, G.E., & Selfe, C.L. (1997). Wedding the technologies of writing portfolios and
computers: The challenges of the electronic classrooms. In K.B. Yancey & I. Weiser
(Eds.), Situating portfolios: For perspectives (p. 305). Utah State: UP.
Hirvela, A., & Pierson, H. (2000). Portfolios: Vehicles for authentic self-assessment. In G.
Ekbatani & H. Pierson (Eds.), Learner-directed assessment in ESL. Mahwah, NJ:
Lawrence Erlbaum Associates.
Hughes, A. (2003). Testing for language teachers. New York: Cambridge University Press.
MacDonald, L., Liu, P., Lowell, K., Tsai, H., & Lohr, L. (2004). Graduate student
perspectives on the development of electronic portfolios. TechTrends, 48(3), 52–55.
Murphy, A. (1994). Writing portfolios in K-12 schools: Implications for linguistically diverse
students. In L. Black, D.A. Daiker, J. Sommers, & G. Stygall (Eds.), New directions in
portfolio assessment (pp. 140–156). Portsmouth, NH: Boynton/Cook.
O’Malley, M.J., & Valdes Pierce, L. (1996). Authentic assessment for English language learners:
Practical approaches for teachers. White Plains, NY: Addison-Wesley.
Ok, A., & Erdogan, M. (2010). Prospective teachers’ perceptions on different aspects of
portfolio. Asia Pacific Education Review, 11, 301–310.
Pearson, I. (1988). Tests as levers for change. In D. Chamberlain & R.J. Baumgardner (Eds.),
ESP in the classroom: Practice and evaluation (pp. 98–107). London: Modern English.
Computer Assisted Language Learning 35
Pullman, G. (2002). Electronic portfolios revisited: The efolios project. Computers and
Composition, 19, 151–169.
Qi, L. (2005). Stakeholders’ conflicting aims undermine the washback function of a high-
stakes test. Language Testing, 22, 142–173.
Downloaded by [NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY] at 21:55 06 November 2013
Read, J., & Hayes, B. (2003). The impact of IELTS on preparation for academic study in New
Zealand. In R. Tulloh (Ed.), International English language testing system research reports,
4. Canberra, Australia: ITLTS.
Richardson, W. (2006). Blogs, wikis, podcasts, and other powerful Web tools for classrooms.
Thousand Oaks, CA: Corwin Press.
Sung, Y., Chang, K., Yu, W., & Chang, T. (2009). Supporting teachers’ reflection and learning
through structured digital teaching portfolios. Journal of Computer Assisted Learning, 25, 375–
385.
van Olphen, M. (2007). Digital portfolios: Balancing the academic and professional needs of
world language teacher candidates. In M. Kassen, R. Lavine, K. Murphy-Judy, & M.
Peter (Eds.), Preparing and developing technolgy-proficient L2 teachers (pp. 265–294). San
Marcos, TX: CALICO.
Ward, J.M. (2004). Blogging assisted language learning (BALL): Push button publishing for
the pupils. TEFL Web Journal, 3(1), 1–16.
Watanabe, Y. (1996). Does grammar translation come from the entrance examination?
Preliminary findings from classroom-based research. Language Testing, 13, 318–333.
Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New York, NY:
Cambridge University Press.
Yang, S.H. (2009). Using blogs to enhance critical reflection and community of practice.
Educational Technology & Society, 12(2), 11–21.
Name:
Peer:
Self assessment
1. What are your strengths in this assignment? Please critically analyze your strengths by
presenting specific examples.
My strengths:
2. What are your weaknesses in this assignment? Please critically analyze your weaknesses by
presenting specific examples.
My weaknesses:
3. How would you make the assignment better? Discuss some suggestions.
5. Other comments?
36 S.-T.A. Hung
doing the most? (3) Which parts didn’t you like to do? (4) Do you think constructing the
portfolio helped you? If it did, in what ways? (5) What were the disadvantages? (6) What
difficulties did you encounter?, and (7) Do you like this kind of assessment as compared to
paper-and-pencil tests?