Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
The Need for a Taxonomy to Facilitate E-Assessment

The Need for a Taxonomy to Facilitate E-Assessment

Ratings: (0)|Views: 98|Likes:
Published by Bobby Elliott
Paper presented at Computer Assisted Assessment 2012 Conference in July 2012.
Paper presented at Computer Assisted Assessment 2012 Conference in July 2012.

More info:

Published by: Bobby Elliott on Jul 06, 2012
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





 Need for a Taxonomy toFacilitate E-Assessment
Bobby ElliottScottish Qualifications Authoritybobby.elliott@sqa.org.ukAbstract
Online writing, in various forms, is common in education but there has been little research into ways of analysing and evaluating its quality. This paper reports on original research into ways of assessing online writing. It addresses the following research questions: (1) Can a traditional taxonomy be used to analyse the academic quality of online writing?; and (2) Can a taxonomic approach be used to aid the assessment of online writing? It uses content analysis techniques to study two forums, used within a post-graduate online learning programme, and reports on the nature of learning in this environment. The techniques used are well known methods of analysing academic discourse; one is commonly used in the online environment (the practical enquiry model) but the other is not (a 
derivation of Bloom’s Taxonomy). This paper seeks to assess their 
suitability to the online domain.The paper concludes that traditional taxonomies can be used to analyse online writing (specifically online forums but potentially generalisable to other forms of online writing), and that the use of a taxonomy would improve assessment practice. It recommends the creation of an 
e- taxonomy 
for this purpose, based on traditional taxonomies but modified to better reflect the new affordances of the online environment.
Need for an Taxonomy to Facilitate E-Assessment
Page 1
The use of computers in education has a long tradition. Higher Education (HE) has usedasynchronous communication tools (such as e-mail and discussion boards) since the late20th Century. More recently, a range of Web 2.0 tools, such as blogs and wikis, have beenadded to this toolset. Students enjoy the use of technology in their classes (Clarke, Flaherty& Mottner, 2001), and its use is supported by a wealth of long-established pedagogicalresearch, such as
theories on language and construction of knowledge (1987),
and Kolb’s emphasis on the importance of learning through participation
(1984). Subsequentresearch has highlighted the importance of the social construction of knowledge (see, forexample, Pea, 1993). A large body of empirical evidence supports the use of computers in the learning process. meta-analysis carried out by the Department of Education in the United States (USDepartment of Education, 2009)
concluded that: “Students who took all or part of the class
online performed better, on average, than those taking the same course through traditionalface-to-
face instruction”. The study reported that instruction which combined online learning
with face-to-face learning was most effective. These findings were true irrespective of themeans of implementation [of online learning] and were found to be applicable irrespective of content or learners.
 This research paper looks at one particular form of online learning, that which involves theuse of online discussion boards. Online discussion boards are one instance of what has been
variously described as “online writing”, “digital writing”, “digital learning narratives” and “Web 2.0 writing” 
, which can take various forms including online forums, blogs, wikis, socialnetworks, instant message logs, and virtual worlds. Many of the issues raised in this paperare relevant to all forms of online writing.This new medium provides new affordances
new ways of utilising the medium tocommunicate and collaborate. For example, online writing may make visible such things asco-operation, collaboration, and self-
reflection; the learner’s thought processes are also
more apparent (Knox, 2007). The inclusion of multimedia (such as audio and video) isstraight-forward. Referencing (hyperlinking) to related resources or information is simple.The asynchronous nature of many online communications makes the time and place of 
contributions more flexible, and provides more “wait time” to improve opportunities for
reflective writing (Berliner, 1987). The dialogue may have an audience beyond the walls of the university (perhaps at campus, national or global level). The scope of the tasks that maybe set can be greater, with faculty setting large-scale projects such as collaborative book writing or large-scale software construction (Gray
et al 
, 2009). Authenticity can be improvedby tackling real-world issues, and seeking feedback from peers and experts across the world(Gray
et al 
, 2009). The potential for producing authentic, large-scale, co-constructed,continuously improved, media-rich works of local, national or international interest is unique.
These “new affordances” have implications for assessment. They provide an opportunity to
assess skills that were previously considered difficult or impossible to assess. For example,collaboration has long been recognised as an important skill but is rarely assessed due toinherent difficulties in measuring this competence through traditional methods. The newaffordances offer opportunities to set new kinds of assessment activities
real activities with
Over 1000 research studies were included in this meta-analysis. Most related to the post-schoolsector; few studies have been carried out for younger learners.
The report noted that the benefits of online learning may not necessarily relate to the medium butcould be the effects of other factors such as time spent learning, the curriculum, or pedagogy.
Need for an Taxonomy to Facilitate E-Assessment
Page 2
real value rather than the contrived tasks often used to assess learners’ knowledge and skills
et al 
, 2009). Traditional assessment has been criticised as being artificial or trivial orirrelevant (see, for example, Brown
et al 
, 1997). The online environment offers the prospectof addressing these criticisms, and rethinking current practice.But there are currently no standards for describing online writing. The lack of an online
pedagogy, an “e
pedagogy”, has been noted (Elliott, 2009) but
the absence of a taxonomyfor describing online activities is, arguably, even more fundamental. Online writing is arelatively recent form of communication, and its educational assessment is evolving. Currentpractice in the creation of rubrics to assess students
writing has been criticised with manymarking schemes found to have low validity and fidelity, and confused about what it is theyare assessing:
There appears to be confusion about what faculty is seeking to assess. Many of theassessment criteria relate to knowledge, skills and attitudes that have little, or nothing, to dowith learning objectives. Many rubrics focus on participation rather than achievement, andinputs (such as effort) rather than outputs (such as understanding).
(Elliott, 2010)
Research Methods
The research questions were:1.
Can a traditional taxonomy be used to analyse the learning that takes place in anonline environment?2.
Can a taxonomic approach to the analysis of online writing be used to aidassessment?Two forums were selected to address these questions. Both forums were part of theUniver
sity of Edinburgh’s
Master’s Degree in E 
, which has been offered by theuniversity since 2006.
The forums were part of the following courses within thatprogramme:
Introduction to Digital Environment for Learning (IDEL)
Understanding Learning in an Online Environment (ULOE).These courses were selected because they both required students to use an associatedonline discussion forum, and one was assessed (ULOE) and one was not (IDEL), permittingthem to be compared with respect to the impact of assessment.In total, 33 students undertook these courses between September and December 2009; 21participated in the IDEL course and 12 in the ULOE course. Each course lasted 12 weeks.Some time, in each course, was dedicated to assessment towards the end of the course;this varied from one non-teaching week in IDEL (week 12) to two non-teaching weeks inULOE (weeks 11 and 12).The IDEL course is an introductory course, normally, but not exclusively, undertaken by newstudents to the programme. The ULOE course is generally undertaken by more experiencedstudents.
ULOE was assessed using four elements, one of which related to learners’ participation in
the associated course forum. This contributed 10% to the overall course grade. A rubric wasused to assess each student, based on that described by Rovai (2000). The other assessed
The programme was piloted in 2004.

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->