Blended Learning For Programming Courses A Case ST

Joseph Fong Fu Lee Wang (Eds.
Blended Learning
Workshop on Blended Learning 2007
Edinburgh, United Kingdom
15-17 August, 2007
Proceedings
Organized by: The Hong Kong Web Society
Sponsored by: City University of Hong Kong

Preface
The Workshop on Blended Learning 2007 provides an open forum for two kinds of
professionals: Computer Scientists and Educators, to share their ideas and experiences
on supplementing e-Learning with classroom learning. In the reality of today’s
education, students can learn anywhere and at any time through Internet. Very often,
students can learn new advanced computer technologies through self learning on the
Internet, and universities can only provide basic knowledge in the classroom. As a
result, there is a need to combine both learning methods to speed up the students'
learning abilities.
This workshop focuses on e-Learning as a supplementary learning to classroom

learning. The era of e-Learning enables students learning any time and anywhere
through Internet. It can reduce teachers’ administration work to a minimum by
computerizing clerical work. It can also personalize each student’s need by adaptive
learning. However, traditional classroom learning is needed for face-to-face lecturing
and tutoring. Above all, the experimental work must be supervised and monitored for
its progress. As a result, it is important to combine the benefits of both e-Learning and
classroom learning into an enhanced teaching method, namely blended learning.
Lecturers can then motivate students to learn both in breadth by getting important
referenced course materials and practice online, and also in depth by detailed
discussion with the lecturers in the face-to-face classroom. The issue is how to create
a curriculum which can fit students’ personal needs. Education theory, peer-to-peer
learning, and web-based technologies should be applied together in defining
techniques and guidelines for the creation of the e-Learning contents and blended
curricula.
The Workshop on Blended Learning 2007 has record a total of 139 papers. The
Workshop had the collaboration of an 18-member program committee composed of
well-known blended learning experts and researchers from America, Asia, Australia
and Europe. The wide coverage of program committee meant that the Workshop
reach out to more participants and thus ensured good-quality paper submissions. The
reviewing of papers was carried out by reviewers chosen among the Program
Committee members and invited reviewers. The reviewing process was managed
using the OpenConf Conference Management System developed by Zakon Group.
Twenty-eight papers were finally selected.
On behalf of the Program Committee of WBL 2007, we thank all authors for their
submissions and camera-ready copies of papers, and all participants for their active
participation in the workshop. We also acknowledge the sponsor, program committee
members and individuals who gave their continuous help and support in making the
workshop a success. We hope that the workshop will continue to grow.
Joseph Fong (City University of Hong Kong)

Fu Lee Wang (City University of Hong Kong)
August 2007
Editors
Joseph Fong
Department of Computer Science
City University of Hong Kong
Kowloon Tong
Hong Kong
e-mail: csjfong@cityu.edu.hk
Fu Lee Wang
Department of Computer Science
Kowloon Tong
Hong Kong
e-mail: flwang@cityu.edu.hk
Conference Organization
The Workshop on Blended Learning 2007 (WBL 2007) was organized by the Hong
Kong Web Society and sponsored by City University of Hong Kong.
Workshop Co-chairs
Joseph Fong, City University of Hong Kong
Fu Lee Wang, City University of Hong Kong
Program Committee
Bressan, Stéphan (National University of Singapore, Singapore)
Cheng, Yin Cheong (Hong Kong Institute of Education, Hong Kong)
Corkindale, David (University of South Australia, Australia)
Fong, Joseph (Hong Kong Web Society, Hong Kong)
Hughes, John (National University of Ireland, Ireland)
Karlapalem, Kamal (International Institute of Information Technology, India)
Kwan, Reggie (Caritas Francis Hsu College, Hong Kong)
Lau, Rynson (University of Durham, U.K.)
Le, Gruenwald (University of Oklahoma, U.S.A.)
Li, Qing (Hong Kong Web Society, Hong Kong)
Lim, Cher Ping (Edith Cowan University, Australia)
McLeod, Dennis (University of Southern California, U.S.A.)
Meersman, Robert (Vrije Universiteit Brussel, Belgium)
Orlowska, Maria (The University of Queensland, Australia)
Sharman, Rex (English Unlimited, Hong Kong)
Smith, Peter (Sunderland University, U.K.)
Zhao, Wei (Texas A & M University, U.S.A.)
Zhu, Zhiting (East China Normal University, China)
Reviewers
Alberts, Philip (Brunel University, U.K.)
Barker, Philip (University of Teesside, U.K.)
Bressan, Stéphan (National University of Singapore, Singapore)
Chan, Fun Ting (The University of Hong Kong, Hong Kong)
Fong, Joseph (Hong Kong Web Society, Hong Kong)
Huang, Shi-Ming (National Chung Cheng University, Taiwan)
Jones, Norah (University of Glamorgan, U.K.)
Lau, Rynson (University of Durham, U.K.)
Lim, Cher Ping (Edith Cowan University, Australia)
McLeod, Dennis (University of Southern California, U.S.A.)
Smith, Peter (Sunderland University, U.K.)
Siu, Cheung Kong (The Hong Kong Institute of Education, Hong Kong)
Wang, Fu Lee (City University of Hong Kong, Hong Kong)
Zhu, Zhiting (East China Normal University, China)
Table of Contents
Keynote Paper
Towards a Definition and Methodology for Blended Learning 1
Won Kim
Papers
Towards Blended Learning of Computer Programming Supported by an
Automated System 9
M. Choy, S. Lam, C.K. Poon, F.L. Wang, Y.T. Yu, L. Yuen
Web-based Logging of Classroom Teaching Activities for Blended Learning 19

Joseph Fong
Blended Learning for Programming Courses:

A Case Study of Outcome Based Teaching & Learning 30
Fu Lee Wang, Joseph Fong, Marian Choy
Blended Learning with Webs, Wikis and Weblogs 42

Philip Barker
Blended Learning: Beyond Web Page Design for the Delivery of Content 53
Philip P Alberts, Linda A Murray, Darren K Griffin, Julia E Stephenson
Discovery of Educational Objective on e-Learning Resource:

A Competency Approach 66
Shi-Ming Huang, Hsiang-Yuan Hsueh, Jing-Shiuan Hua
Blended Learning: Experiences of Adult Learners in Hong Kong 79

P. W. R. Lee, F. T. Chan
An Observational Study on Blended Learning for Japanese Language

Studies in a Local University in Hong Kong 88
Kenneth K.C. Lee, Melody P.M. Chong
Structured Blended Learning Implementation for an Open

Learning Environment 101
Jason K. Y. Chan, Ken C. K. Law
The Disruptive Effect of Technology: a University Case Study 114

Norah Jones
The Marriage of Rousseau and Blended Learning: An Investigation of

3 Higher Educational Institutions’ Praxis 123
Esyin Chew, Norah Jones, David Turner
Developing an On-line Medical Curriculum Management Platform 136
Jenny Fang, Francis Wong, Raymond Chu
Concepts of Blended Learning for Different Content Types 150

Andreas Henrich, Stefanie Sieber
Blended Learning: Towards a Mix for SMEs –

Stakeholders and their Priorities 162
Sabine Moebs, Stephan Weibelzahl
Supporting Self-Regulated Learning in a Blended Course 174

Giuliana Dettori, Donatella Persico
A Study on the Impact of the Use of an Automatic and Adaptive

Free-text Assessment System during a University Course 186
Diana Perez-Marin, Ismael Pascual-Nieto, Enrique Alfonseca,
Eloy Anguiano, Pilar Rodriguez
A Wiki-Based Collaborative Learning Design and Its Effects in

Secondary Math Studies 196
Liming Zhang, Chan Lam Wong
Implementation of an E-Learning System – Learning Design and Learning

Objects Management and Evaluation through Stardardization of Resources 206
Hugo Rego, Tiago Moreira, Francisco José Garcia
Visualisation of Learners’ Contributions in Chat Conversations 217

Stefan Trausan-Matu, Traian Rebedea,
Alexandru Dragan, Catalin Alexandru
HECACEJ: B-Learning Tool for Static Content Creation in Joomla! 227

Angel Mora-Bonilla, Domingo López-Rodríguez,
Enrique Mérida-Casermeiro, Salvador Merino-Córdoba
MAAS – Mobile Administrative and Assessment System 237

Apple W P Fok, Horace H S Ip, Chilli C K Chan
Large-scale Computer-Assisted Assessment in Computer Science

Education: New Possibilities, New Questions 257
Mario Amelung, Michael Piotrowski, Dietmar Rösner
Exploratory Research on an Affective e-Learning Model 267

Liping Shen, Enrique Leon, Victor Callaghan, Ruimin Shen
Design and Development of Blended Learning through LMS 279

K. P. Hewagamage, S.C. Premaratne, K.H.R.A Peiris
An Access Monitoring Tool Based on Cookies for Course
Management Systems 292
Raquel Hijón-Neira, Ignacio López-López, Ángel Velázquez-Iturbide,
Francisco Domínguez-Mateos
Learning News Writing Using Emergent Collaborative

Writing Technology Wiki 303
Will Wai-kit Ma, Allan Hoi-kau Yuen
Description Patterns in Learning Design Authoring Systems 315

Lei Xu, Li Zheng, Jing Liu, Yintao Liu, Fang Yang
M-Learning: A Pedagogical and Technological Model for Language

Learning on Mobile Phones 327
Yannick Jolliet
Author Index 340

Towards a Definition and Methodology for Blended
Learning
Won Kim
School of Information and Communication Engineering

Sungkyunkwan University, Suwon, S. Korea
wonkim@skku.edu
Abstract. Blended learning is known roughly as combining the

traditional instructor-led classroom learning and technology-based e-
learning. Although there have been attempts to define the term blended
learning, and to understand how to make blended learning work best,
considerable further research and experimentation are needed. This
paper offers a precise definition of blended learning and blended
learning program, and proposes a methodology for making blended
learning work best.
Keywords: blended learning, blended learning methodology, e-

learning.
1. Introduction
The recent rapid advances in information technology, including the Internet

and the Web, have had significant impact on numerous aspects of the daily
living of the mankind and the society of the industrialized parts of the world.
One of the aspects is education. Technological advances and wide availability
of personal computers, CDs, the Internet, the Web, broadband access to the
Internet, etc. have resulted in e-learning (also known as distance learning or
Web-based learning). Numerous corporations (and government branches)
have adopted e-learning to train their employees and inform their business
partners and customers (citizens). Universities (and schools of other types,
including vocational schools, informal municipal classes for their citizens)
have adopted e-learning to some extent to supplement or augment the
traditional instructor-led classroom courses. People on their own have adopted
e-learning as a vehicle for self-study on numerous subjects of interest for
various purposes.
The traditional instructor-led classroom learning is a proven and effective
means of learning, with full opportunities for interaction between the
instructor and students, the learning-inducing stress of exams and homework,
Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 1-8, Pearson, 2007.
Workshop on Blended Learning 2007, Edinburgh, United Kingdom.
2 Won Kim
and relationship forming among students, etc. However, the requirement for
the students to be in the classroom on designated days and times makes it
difficult for certain students. Further, the lack of equipment in the classroom
may make it difficult for the instructor to teach certain topics effectively. The
technology-enabled e-learning can help address such difficulties posed by the
limitations of the traditional classroom learning. There is a variety of types of
e-learning, ranging from degree-conferring cyber universities and vocational
schools, certificate-conferring programs, corporate training programs using
videos of instructors delivering instructions – overlaid with PowerPoint pages,
to people watching CDs on their computers or browsing the Web for materials
on specific topics of interest. However, pure e-learning has its own
limitations. Depending on the types of e-learning and how the e-learning is
designed, these may include the absence of an instructor, poorly created or
inconsistent materials (course contents) – compounded by the absence of an
instructor or tutor, the absence of a deadline pressure – which tends to lead to
lower learning efficiency, etc.
As both the traditional classroom learning and e-learning simultaneously
offer strengths and suffer from limitations, it is only natural to combine the
strengths of the two into blended learning. In this sense, it may be said that
today the most natural form of learning is blended learning. The traditional
classroom learning has required research, or at least thoughtful considerations
on the part of the instructors for generations, to make it effective for learning.
Similarly, research is still needed to realize the full potential of e-learning.
Blended learning requires yet additional research to make the combination of
the traditional classroom learning and e-learning as effective as possible.
There have been several earlier attempts to define blended learning
[1][2][3][4][5][6][7][8][9]910][11][12]. There have been efforts to define a
methodology for designing blended learning programs [6][8][9][10]. In this
paper, I will first give a precise definition of blended learning and blended
learning program, and then, based on the definitions, propose a methodology
for making a blended learning program work best.
2. A Precise Definition of Blended Learning
As a first approximation, blended learning may be defined as a combination

of classroom learning and e-learning. This definition can, and needs to, be
made more precise, although I will use the term e-learning loosely to mean
learning outside the traditional classroom using information technology for
the delivery of the learning materials (i.e., course contents). The approach I
will take is to first consider the different types of learning that are possible.
There are at least three key dimensions to consider in classifying learning:

physical class-based or virtual, formal or informal, and scheduled or self-
paced. Learning may be classified as physical class-based or virtual class-
based. Physical class-based learning is the traditional classroom learning
where the instructor and the students must physically be present in the
classroom at the same time. Virtual class-based learning does not require the
instructor and students to be present in the classroom. E-learning is
necessarily virtual class-based.
Learning may be classified as formal or informal. Formal learning is tied to
earning a degree, certificate, or official credit. Informal learning is not.
Learning may also be classified as scheduled or unscheduled. Scheduled
learning proceeds on a fixed schedule, while there is no fixed schedule for
informal self-paced learning. To be precise, there are two types of schedule:
one is the schedule for the class (i.e., lectures), and another is the schedule for
a course (i.e., start and end of a semester, and schedules for homework and
exams). The traditional course in a university has both the class schedule and
the course schedule. Self-paced learning does not come with a class schedule.
However, one type of self-learning comes with a course schedule, and another
type comes with no course schedule.
If we are to go with the three dimensions of learning outlined above, there
are eight possible combinations. However, upon a closer examination, the
number of possible combinations is much larger than eight. First, it makes
sense to refine the notion of schedule, from scheduled and self-paced, into
class schedule, course schedule, and no schedule. Further, it is possible to
introduce one additional dimension of learning, namely, availability of face-
to-face interactions outside the class between the students and the
instructor/tutor. For example, an independent study or an industry co-op
course does not involve a lecture class; however, it requires face-to-face
interactions outside the class between the students and the
instructor/supervisor/tutor. These additional considerations will potentially
result in 24 learning types. I will not list them here, but the following are
some of the possible learning types.
1. Formal, class-scheduled and course-scheduled, physical class learning. This
is the traditional learning.
2. Formal, course-scheduled, no physical class, but face-to-face interaction-
based learning. Examples of this type of learning include dissertation
research, independent studies and industry co-op courses offered by
universities.
3. Formal, class-scheduled and course-scheduled, e-learning. This is distance
learning, and is the e-learning equivalent of the traditional learning.
Students do not need to be in the classroom, and can participate in the class
from anywhere. However, the class is offered on a fixed schedule, just like
the traditional class.
4 Won Kim
4. Formal, course-scheduled e-learning. This type of e-learning has been

adopted by universities and corporations and government branches.
5. Informal, class-scheduled and course-scheduled, physical class learning:
This is the traditional learning corresponding to auditing where students do
not receive a formal credit.
6. Informal, unscheduled e-learning. This type of e-learning is adopted by
people who would like to simply learn something about a topic on their
own from anywhere at any time.
I can now define blended learning precisely as “a combination of two or more
of all possible learning types” discussed above. There is one important
qualifier to this definition. At least one of the learning types must be a
physical class-based type (regardless of whether there is a course-schedule; or
whether it is formal; or whether there is face-to-face interaction outside the
physical classroom), and at least one other learning type must be e-learning
type (regardless of whether there is a class schedule or a course-schedule; or
whether it is formal; or whether there is face-to-face interaction outside the
physical classroom).This is to make sure blended learning remains a
combination of some form of traditional learning and some form of e-
learning.
Now that I have defined blended learning precisely, I will define blended
learning program. A blended learning program is a coherently designed
learning program that is applied to a range of learning activities, ranging from
a topic or a course fragment, to a course, and even a curriculum. A blended
course is a course that consists of some course fragments of traditional
learning types and other course fragments of e-learning types. A blended
curriculum is a set of courses, where some of the courses are blended, some
are purely e-learning courses, and others are purely traditional courses. A
blended learning program would generally refer to a number of courses, or an
entire curriculum, although it is possible to talk about it in the context of a
single course, a single topic, or even a course fragment (a part of a course,
involving multiple topics), or even an entire institution (i.e., across multiple
curricula).
I am not sure what percentage of a course must be e-learning types before the
course can reasonably be called a blended course. Similarly, I am not sure
how many of the courses must be blended courses or purely e-learning
courses, before a curriculum can reasonably be called a blended curriculum. It
appears that for both the course and curriculum the percentage may be at least
33 in order to prevent unreasonable claims from some schools or corporations
who only seek propaganda value while essentially offering only the traditional
learning or only formal e-learning.
3. Methodology for Creating a Blended Learning Program
Blended learning, and e-learning for that matter, does not just happen by
simply injecting some e-learning elements into a traditional course, or some e-
learning courses into a traditional curriculum. There are several important
issues in designing an effective blended learning program, both at the level of
a course and a curriculum. Any methodology for creating a blended learning
program must include three elements, as follows.
1. Setting the scope and objectives of the blended learning program.
2. An analysis of the relevant issues under the overall objectives set.
3. Addressing any problems identified in the analysis.
4. Measuring the results of the blended learning program.
The scope of a blended learning program, as already discussed, would

normally be an entire curriculum or at least a number of courses. Appropriate
objectives must be selected and used to guide the design and implementation
of a blended learning program. There are various potential objectives for
implementing a blended learning program, as follows.
1. Increased learning effectiveness (for the students or employees), over either
pure traditional classroom learning or pure e-learning.
2. Increased convenience (for the students or employees). In the case of
corporations, blended the e-learning component of a blended learning
program can make it easier for the employees when business trips or high-
priority business meetings come up to prevent them from attending
scheduled in-class training.
3. Enhanced image (for the school or the corporation). The progressive image
may be projected both internally (to own students or employees) and
externally (the general public, customers, the government, news media, the
financial analysts, etc.).
4. Cost savings (for the school or the corporation). The cost savings may
result from possibly reducing the number of instructors, and, in the case of
corporations, having employees not use critical work hours for scheduled
in-class training.
5. Classroom space savings (for the school or the corporation). The e-learning
component of a blended learning program can help ease the classroom
space needs by having students and employees learn more from outside the
classrooms. The freed-up classroom space can potentially be used for other
purposes.
6. Reduced traffic and parking congestion on the campus or the corporation.
Key issues to be carefully analyzed follow.

6 Won Kim
• Cost issue: To add the e-learning element into a traditional course or

curriculum, equipment, hardware, networking and software are needed in
general. There is the associated cost of installing, maintaining and
upgrading them; possibly hiring and paying technical support staff;
possibly adding administrative staff; training the staff and instructors;
content development and upgrade, etc.
• Management issue: A decision must be made as to how much of the work
should be done with in-house staff or outsourced to service organizations.
This issue is closely related to the cost issue, and includes the following
considerations.
1. Purchase, installation, and management of the equipment, hardware,
networking and software.
2. Selecting technologies to use. This has to do with setting and
observing school-wide or corporation-wide standards on the
equipment, hardware, networking, and software, so that there is overall
consistency, relative ease of receiving services and volume discounts
from the vendors, etc.
3. Hiring and training technical and administrative staff.
• Selection of vendors: Once it is decided to outsource some of the work and
people needs, vendors must be selected. There needs to be a set of criteria
for selecting the vendors, including cost estimations provided, strategic
importance they attach to the work being outsourced, their capabilities and
limitations, their financial stability, reputations from their existing
customers, etc.
• Considerations of the students: In designing the e-learning component of
a blended learning program, the students’ and employees’ ability to access
the e-learning contents must be taken into account. The access ability
includes the computing power of the PCs they will use, the types of
peripherals that come with the PCs, operating systems and their versions
that run on the PCs, web browser versions they will use, Internet access
bandwidth, availability or accessibility to video conferencing, etc. Course
contents should not include parts that a lot of students and employees
(accessing from their homes) cannot view or listen with the computers and
Internet bandwidth available to them. Further, the students’ and employees’
special needs and situations that may affect the scheduling of face-to-face
meetings or exams, the reaction to the contents of the course materials, etc.
should be taken into account. These include religion, culture, national
origin, physical handicap, etc. In particular, course contents should not
include statements, illustrations, examples, etc. that may offend the
students and employees.
• Considerations of the classroom space: Since the e-learning part of a
blended learning program does not in general require the use of a
classroom, the classroom can be put to good use. Good scheduling of a

blended program that consists of a number of courses in the same course
period can yield a fair number of free classrooms on a regular basis during
the course period. If the classroom space is at a premium, the freed-up
classrooms can be quite helpful.
• Determination of the composition of learning types: To make a blended
learning program work best, both at the curriculum and course level, the
best composition of the learning types must be determined. The
determination of the best composition must be done on the basis of the
precise definition of blended learning discussed earlier. Further, it must be
done by considering all of the issues discussed above.
It is important that there be some pre-determined set of measures and a

methodology for measuring the degree of success of the blended learning
program. The measures would include actual costs incurred and actual
benefits achieved. The benefits in turn would include all the objectives chosen
for the blended learning program.
4. Concluding Remarks
The emergence of e-learning, and the realization of the strengths and

limitations of the traditional classroom learning and e-learning, have naturally
led to the notion of blended learning. In this paper, I first provided a much
more precise definition of blended learning and blended learning program
than available in the literature. Then, based on the definition, I proposed a
practical approach for developing a methodology for designing a blended
learning program.
Looking forward, in my view, just as large corporations have been the first
to adopt e-learning to train their employees on a wide variety of subjects, such
as company orientation, leadership, project management, six sigma,
interpersonal skills, new products, etc., it will be the corporations that will
combine e-learning with the traditional learning into blended learning.
Further, municipal governments are likely to add e-learning components to
the continuing education programs they run for their citizens on a wide variety
of subjects, such as first-aid, the Web basics, etc. As such, I would like to urge
academic researchers interested in advancing the methodology for designing
blended learning programs to set their bearings to corporate training programs
and municipal government’s continuing education programs, and the very
wide and general nature of the subjects on which they conduct training and
education.
8 Won Kim
Acknowledgments. This research was supported by the Korean Ministry of

Information and Communication under the ITRC IITA-2006-(C1090-0603-
0046) grant.
References
1. Bersin & Associates, “Blended Learning: What Works?”, May 2003,

http://www.e-learningguru.com/wpapers/blended_bersin.doc
2. Josh Bersin, “Blended Learning: Finding What Works”,
http://www.clomedia.com/content/templates/clo_feature.asp?articleid=357&zon
eid=30
3. Charles Dziuban, Joel Hartman, Patsy Moskal, “Blended Learning”, University
of Central Florida, Educause Center for Applied Research, Research Bulletin,
vol. 2004, number 7, March 30, 2004.
http://ecampus.nl.edu/resources/webct/BlendedLearningUCFStudy.pdf
4. Epic Group plc., “Blended Learning” Epic white paper,
http://www.epic.co.uk/content/resources/white_papers/Epic_Whtp_blended.pdf
5. Charles Graham, “Blended Learning Systems: Definition, Current Trends, and
Future Directions”, in Handbook of Blended Learning: Global Perspectives,
Local Designs, Pfeiffer Publishing, San Francisco, USA, 2006.
http://www.uab.edu/it/instructional/technology/docs/blended_learning_systems.
pdf
6. Michael McGinnis, “Building a Successful Blended Learning Strategy”,
http://www.ltimagazine.com/ltimagazine/article/articleDetail.jsp?id=167425
7. Martin Oliver and Keith Trigwell, “Can ‘Blended Learning’ Be Redeemed?”,
E-Learning, vol. 2, no. 1, 2005.
http://www.wwwords.co.uk/pdf/viewpdf.asp?j=elea&vol=2&issue=1&year=20
05&article=3_Oliver_ELEA_2_1_web&id=59.13.102.130
8. Allison Rossett, Felicia Douglis, and Rebecca Frazee, “Strategies for Building
Blended Learning”, Learning Circuits, 2003,
http://www.learningcircuits.org/2003/jul2003/rossett.htm
9. Harvi Singh and Chris Reed, “A White Paper: Achieving Success with Blended
Learning”, Centra Software, 2001.
http://www.centra.com/download/whitepapers/blendedlearning.pdf
10. Harvi Singh, “Building Effective Blended Learning Programs”, Educational
Technology, Nov./Dec. 2003, pp. 51-54.
http://www.bookstoread.com/framework/blended-learning.pdf
11. Richard Voos, “Blended Learning – What Is It and Where Might It Take Us?”,
http://sloan-c.org/publications/view/v2n1/blended1.htm
12. Wikipedia, “Blended Learning”, http://en.wikipedia.org/wiki/Blended_learning
Towards Blended Learning of Computer Programming
Supported by an Automated System1
M. Choy, S. Lam, C.K. Poon2, F.L. Wang, Y.T. Yu and L. Yuen
Department of Computer Science, City University of Hong Kong
Abstract. Learning computer programming is known to be difficult for many

beginners. With the primary aim to improving the practice of teaching and
learning of computer programming, we have developed a web-based automated
system, known as PASS, for use in our courses. PASS has provided
capabilities with opportunities for new pedagogy and innovative strategies for
both teaching and learning. This paper describes the design and functionalities
of PASS, how it can be used to enhance students’ interest of learning and
monitor their progress, and illustrates several approaches with which PASS can
facilitate blended learning of good practices in computer programming.
Keywords: Computer programming, automated system, learning.
1 Introduction
Thanks to the booming development of the Internet, there is currently a trend

towards the convergence between learning via on-campus classes and remote learning
through the network. Blended learning, combining online and face-to-face instruction,
has become one of the hot topics in both the academic and corporate circles [7]. It is
believed that, in the long run, the word “blended” will eventually be dropped as it
becomes the norm in pedagogy.
Learning computer programming has been known to be difficult for many
beginners [3]. Programming skill has to be acquired through lots of practice [4].
When teaching programming, the instructor typically gives many programming
exercises and assignments to students, hoping that they will gain enough hands-on
practice. However, assessment of programming exercises and assignments by manual
inspection of code on paper is notoriously inefficient and error-prone [9].
Alternatively, the instructor may require students to perform demonstrations of the
execution of their code, or to submit their program source code so that the instructor
may execute the programs manually one by one. In this way, the correctness of the
students’ programs can be more reliably evaluated, but still this method is labour-
intensive and time-consuming. More importantly, such a practice is educationally
undesirable because students generally have to wait for a long time before they can
receive feedback from the instructors to help them improve their work.
1 This work is supported in part by a Teaching Development Grant (project no. 6000118) from City University of Hong Kong.
2 Corresponding author. Phone: +852-27887157. Fax: +852-27888614. Email: ckpoon@cs.cityu.edu.hk .
10 M. Choy et al.
With the primary aim to improving the practice of teaching and learning of
computer programming, we have developed a web-based automated Programming
Assignment aSsessment System, better known as PASS, for use in our courses [5]. The
basic function of the system is to allow the student to submit their program code,
automatically execute the program with predefined test cases (determined by the
instructor beforehand), and return the execution results to the student. In this way, the
student gets instant feedback from PASS as to whether his/her program is correct with
respect to the selected test cases. When the submitted program is incorrect, PASS will
also indicate to the student exactly at which position the actual output differs from the
correct (expected) output. For assignment submissions, the instructor can execute
students’ programs with a much larger set of hidden test cases in a single batch so that
a more comprehensive assessment of the submissions can be performed efficiently.
Afterwards, all the execution results can be released to the students by the instructor
so that they know exactly which test cases their programs fail, if any. This kind of
prompt feedback to students in the whole class was rarely possible before PASS was
developed [6].
Since its first introduction a few years ago, PASS has proved to be a valuable
and indispensable tool for both students and instructors [6, 11]. PASS has now
undergone significant revisions, with enhanced capabilities that have created
opportunities for new pedagogy and innovative strategies for both teaching and
learning. This paper describes the design and functionalities of PASS, how it can be
used to enhance students’ interest of learning and monitor their progress, and
illustrates several approaches with which PASS can facilitate blended learning of
good practices in computer programming.
In the next section, the system functions and design of the current version of
PASS will be described. Section 3 describes, with the aid of a running example,
several approaches to blended learning using PASS, followed by concluding remarks
in Section 4.
2 The Programming Assignment aSsessment System (PASS)
PASS was first developed in our university in 2004 as an executable prototype

for demonstration [5], but it was soon re-engineered to a working system for
production use [11]. Although the first version of PASS contained the core
functionalities which were already very helpful to students, it nevertheless carried its
built-in design limitations that significantly hindered its enhancement. To facilitate its
sustained use, we have recently substantially re-designed it with major revisions. In
this section, we shall first present the core entities and their relations, and then some
newly enhanced functionalities in the current version (version 3) of PASS. These
functions have provided capabilities that enable new approaches to the teaching and
learning of computer programming that are later described and illustrated in the next
section.
Towards Blended Learning of Computer Programming 11
2.1 System functions and design
Core entities.
Central to PASS are several core entities: problem, submission, test result,
course and various roles of user. A problem refers to a programming exercise or
assignment issued to students for practice or assessment. It includes a description
(specification) of the program to be written, supporting files (such as additional
documentation, library code or data files) and a set of test cases for assessing the
correctness of program submissions. Each problem is associated with a release date
(when the problem is publicised), a submission period (during which students may
submit their programs), and a close date (after which submission is not allowed). A
student may view a given problem, work on it to produce a program to solve the
problem, and then submit the program to PASS for testing. A program plus any
auxiliary files submitted to PASS is a submission. The result of executing a submitted
program with preset test cases in PASS is called a test result, which includes the
information of test runs, memory and time spent, and other related information.
The creator of the problem, usually the instructor, may specify the compiler with
which the submitted programs are compiled and executed. For practice problems,
students can view all the test cases (including both test inputs and expected outputs)
when their programs are tested in PASS. These test cases are thus said to be open.
For assessment problems, the instructor may specify some test cases to be open, and
leave the remaining (usually a larger number of) test cases to be hidden, which are not
revealed to students before the submission period ends. The hidden test cases are
only executed for the purpose of grading students’ submissions. They are designed to
prevent students from tailoring their programs to test cases and to encourage students
to ensure that their programs are generally correct.
PASS can handle many courses at the same time. Apart from the system
administrator who looks after the technical administration and support, there are four
other roles of user in PASS, namely course leader, course developer, tutor and
student. Course leader is overall in charge of a course, normally the course lecturer.
A course leader has all rights to access and manage the records of all students and
problems in his/her course. Students can view the problems in their course(s), submit
programs and check the results and statistics of their program executions. Tutor and
course developer are the roles of people who assist the course leader to manage the
course with limited access rights. A tutor can access the details and performance of
the students of his/her group(s). A course developer is responsible for designing the
problems and test cases, but is not allowed to access the private records of students.
Such a design enables the course leader to designate part of the work load of problem
administration to course developers without worrying about the disclosure of privacy
of students. The relationships among the core entities in PASS are summarised in
Figure 1.
12 M. Choy et al.
Fig. 1. Relationships among core entities in PASS.
Submissions monitoring and analysis.

PASS preserves all the submissions made by students and analyzes such
information to facilitate the monitoring of the overall progress and performance of the
course, selected groups of students, or each individual student. The computed
statistics can be grouped by different attributes. For example, when grouped by
students, the instructor can easily identify the students with outstanding performance,
as well as those who consistently lag behind. On the other hand, when grouped by
exercises or assignments, common difficulties or problems encountered by students
can be revealed. The instructor may further inspect finer details of the problems by
checking the effectiveness of each test case in detecting program faults. Temporal
statistics are also available to view the usage and activity profiles of the students. All
statistics are displayed in easy-to-read charts, and the instructor may focus attention to
selected groups or problems. Selected statistics can be disclosed to students so that
they can evaluate their own performance in relation to that of their peers.
Problem sharing in problem repository.

In general, the problems created for a course are only accessible by the
participants in the course. To facilitate collaboration among instructors, a mechanism
is designed in PASS for sharing and reuse of problems. An instructor may choose to
release his/her problems to a repository in PASS so that instructors in other courses
can access and reuse them as appropriate. The instructor can retrieve a problem from
the repository, amend it to suit his/her own course, and optionally release this new
version to the repository. The original version of the problem will be retained intact
to make sure that any amendment will not affect other PASS users who are making
use of the original problem.
3 Approaches to Blended Learning using PASS
The teaching and learning of computer programming is a challenging job. A well-

designed pedagogy can make the instruction most effective. However, the traditional
mode of education imposes a lot of constraints to the design of pedagogy. Certain
types of teaching and learning activities are extremely difficult to conduct within a
standard classroom setting. Students often find it difficult to master those advanced
programming paradigms without adequate practice. Blended learning combines
different media and methods to maximise students’ learning. PASS enables us to
blend our courses to render the teaching and learning of computer programming more
effective.
It is believed that effective instruction involves working the content to provide
stepwise learning experiences with checks along the way to assure that each step is
learned [10]. It is important to ensure that students are well-trained in the
fundamentals to the extent that they can eventually consider some problems with
high-level complexity. We pay extra care to design the teaching and learning
activities to incorporate stepwise learning. Students taking computer programming
course very often come with various backgrounds and ability levels. PASS allows us
to design exercises with different levels of difficulty to fulfil the need of students with
various backgrounds and ability levels. Also, we can create a series of test cases with
different levels of difficulty for the same programming problem (see Figures 2 and 3).
To illustrate the idea, we take the programming exercise of solving a quadratic
equation as a running example (Figure 2). We may group the test cases into three
levels of difficulty, namely, the beginner level, intermediate level and advanced level
(Figure 3a). The equations which have two distinct real roots are considered
relatively easier; and therefore we classify the corresponding test cases as at the
beginner level (Figure 3b). The test cases which correspond to quadratic equations
with one repeated root or two complex roots are classified as at the intermediate level
(Figure 3c). The exceptional cases (such as those corresponding to the cases when the
equations become linear or identities) are classified as at the advanced level (Figure
3d). PASS allows us to tell the students the level of difficulty of each exercise (Figure
3a). For the same problem, students can attempt the exercises based on their
capability.
14 M. Choy et al.
For example, the less talented students may design a simple program to solve
the problem at the beginner level of difficulty (Figure 3b). If they submit their
programs to attempt exercises at other levels of difficulty, they will fail in those test
cases (Figure 2c-d). Instead, they must enhance their programs in order to solve the
problem at the intermediate level of difficulty. However, the talented students may
work directly to solve the problem at the intermediate level of difficulty. They can
even try to challenge the exceptional test cases by submitting their programs to solve
the problem at the advanced level of difficulty. Eventually, they should come up with
a single program which can solve the problem up to a certain level of difficulty. This
approach allows the students to regulate their own learning pace. Additionally, we
may require students to study the given test cases for each exercise to figure out how
we select the test cases so as to learn how to test their programs on their own.
A Programming Exercise of Solving a Quadratic Equation
Write a program to solve a quadratic equation. The general form of a quadratic

equation is ax2 + bx + c = 0, where a, b, c are real numbers. When a ≠ 0, the solution of
the equation is given by the quadratic formula x = − b ± b − 4ac .
2
2a
In this exercise, we assume that all the inputs are integers, and that the outputs are to
be displayed in descending order.
Test Cases at the Beginner Level of Difficulty
Input Expected Output

a = 1, b = –5, c = 6 x = 3, 2
a = 2, b = –7, c = –15 x = 5, –1.5
a = 1, b = 6, c = 8 x = –2, –4
Test Cases at the Intermediate Level of Difficulty

a = 1, b = –2, c = 1 x=1
a = 1, b = 2, c = 5 x = –1+2i, –1–2i
a = 2, b = 12, c = 18 x = –3
Test Cases at the Advanced Level of Difficulty

a = 0, b = 2, c = 4 x = –2
a = 0, b = 0, c = 0 x = any real numbers
a = 0, b = 0, c = –4 No solution
Fig. 2. Description of a programming exercise at different levels of difficulty.

(a) Programming Exercises at Different Levels of Difficulty
(b) A Sample Run of Submission to an Exercise at the Beginner Level of Difficulty
(c) A Sample Run of Submission to an Exercise at the Intermediate Level of Difficulty
(d) A Sample Run of Submission to an Exercise at the Advanced Level of Difficulty
Fig. 3. Exercises at different levels of difficulty and sample runs of submissions in PASS.
16 M. Choy et al.
Furthermore, a programming course typically has a large class size. It is

difficult to closely monitor individual student’s learning progress in the traditional
teaching mode. The instructor does not have enough time to interact with all students
in a class of hundreds of students within a few hours of lectures and tutorials each
week. PASS has recorded all the program submission activities of students. These
data provide very useful information of students’ performance. For example, we can
easily know a student’s progress in the course based on the number of attempts to
each exercise and the number of exercises completed. The summary of student
performance supports fast decision making as well. We can identify the slow learner
quickly and offer corresponding help. At the same time, we encourage the talented
students to offer peer assistance to other students. Moreover, we find that the talented
students put in additional efforts to study all related materials before they offer help.
Besides, they can further enhance their understanding by explaining the programs to
others. This approach improves the learning of both groups of students.
As instructors of computer programming courses, we not only have to teach
students how to write programs, but also need to develop our students with good
programming practices, such as modularity, reusability and information hiding. With
the support of PASS, we have incorporated and promoted these good programming
practices in designing our teaching and learning activities.
When we design a programming activity, we break the program into smaller
modules in the form of functions and classes. The students are required to implement
their individual modules in separate files. The files are then uploaded to PASS for
testing by using the test driver which is pre-loaded to PASS by the instructor. This
approach highlights the modularity of computer programs. The students are exposed
to programs that are built from modules so that they learn the concept of modularity
of program by implicit learning [2]. When developing an application, students will
have to divide their solutions into modules as functions and classes.
Moreover, we require the students to archive all the files developed in their
activities. When designing a programming activity, we intentionally require the
students to make use of some modules developed in previous activities by importing
the corresponding files. For example, we may require students to develop a program
to solve a quadratic inequality (Figure 4) based on the module developed earlier in the
programming exercise of solving a quadratic equation (Figure 2). Similarly, as before,
we create exercises at different levels of difficulty (Figure 3a). The least talented
students can solve the inequality by using their simple programs that solve a quadratic
equation, while the talented students can try some challenging test cases such as when
the quadratic inequality has one solution or no solution. In this way, students will
naturally acquire the concept of code reuse through their own experience of reusing
the previously developed code, as concrete experience is important in the learning
cycle [8]. On the other hand, we sometimes ask students to exchange files and
develop their applications based on modules written by other students. For different
exercises of the same programming problem, we sometimes provide different
implementations of the same module as separate exercises in PASS. Students are
required to submit their solutions to all exercises of the problem. In order to pass all
the exercises, the students need to write the code purely based on the module interface.
By doing these exercises in PASS, students will gradually recognise the importance of
information hiding.
A Programming Exercise of Solving a Quadratic Inequality
Write a program to solve a quadratic inequality based on the module you developed
earlier in the programming exercise of solving a quadratic equation. In general, a quadratic
inequality can be written in one of the following standard forms, where a, b, c are real
numbers:
ax2 + bx + c ≥ 0
ax2 + bx + c > 0
ax2 + bx + c ≤ 0
ax2 + bx + c < 0
Suppose that the equation ax2 + bx + c = 0 has two real roots x1 and x2 , where x1 < x2 .
If a > 0 , the solution sets of the inequalities are, respectively, as follows.
Inequality: ax2 + bx + c ≥ 0 . Solution: ( –∞, x1 ] ∪ [ x2 , +∞ )
Inequality: ax2 + bx + c > 0 . Solution: ( –∞, x1 ) ∪ ( x2 , +∞ )
Inequality: ax2 + bx + c ≤ 0 . Solution: [ x1 , x2 ]
Inequality: ax2 + bx + c < 0 . Solution: ( x1 , x2 )
Fig. 4. A programming exercise based on a previously completed module.
Finally, traditional pedagogy focuses on the development of small applications.

Without the support of related technology, students usually develop small applications
by writing the code solely on their individual effort. The student may become an
analyst programmer in the future and may be involved in some large scale projects.
Students often find it difficult to manage large software development jobs when they
work in the industry. It is very important to provide students with experiences of
software development in large scale applications while they are studying. However,
there are practical difficulties to require students to develop a large application. First
of all, students’ learning motivation drops very fast as the time they have to spend on
study increases. If we require the student to code a large application, they are usually
unable to see their results before the completion of the whole application. They will
lose their interests in programming soon after they started. Lack of motivation is one
of the major resistances to learning [1]. As we foresee the need, we consider large
application development as an essential part of an advanced programming course.
Some special arrangements have to be made to keep the students’ learning motivation.
When we design a large application, we may divide the application into several
modules. After the student has completed one module of the application, he/she can
submit the modules to PASS. Some stubs or test drivers can be provided for testing
their individual modules. It is important to reinforce the student’s success upon
his/her completion of one module. This approach also increases the student’s
confidence in learning. The intermediate results can keep students’ learning
motivation constantly high. The students will develop the application in a progressive
manner. After the students have completed the entire application, they can submit it
to PASS, which will test all the modules together as a single integrated application.
18 M. Choy et al.
4 Conclusion
This paper has described the design and functionalities of the latest version of PASS.
PASS now allows the instructor to effectively locate the slow learners and identify the
high achievers by the statistical data it collects while students are using it to test their
own programs. We have also explained, with concrete illustrative examples, how
classroom teaching and learning activities may be blended with practice exercises that
are supported and monitored by PASS. By designing exercises at different levels of
difficulty, PASS can provide stepwise learning experiences to students, such that they
can solve problems pertaining to their corresponding ability levels. Instructors can
also define problems in various ways in PASS so as to make students familiar with
modules programming and be prepared for large projects.
We have demonstrated just a few ideas on how PASS can be used to support
blended learning. With the flexibility and extensibility of PASS, we anticipate that
PASS will also facilitate other new and innovative strategies for blended learning.
References
1. J.S. Atherton (1999), “Resistance to learning: A discussion based on participants in in-

service professional training programmes”. Journal of Vocational Education and Training
51(1):77-90.
2. D.C. Berry (ed.) (1997). How Implicit is Implicit Learning. Oxford University Press.
3. B. du Boulay (1989), “Some difficulties of learning to program”. In E. Soloway and J.C.
Spohrer (eds.), Studying the novice programmer. Hillsdale, N.J.: L. Erlbaum Associates.
4. B. Cheang, A. Kurnia, A. Lim, A., and W.-C. Oon (2003), “On automated grading of
programming assignments in an academic institution”. Computers & Education 41(2):121-
131.
5. S.L. Chong and M. Choy (2004), “Towards a progressive learning environment for
programming courses”. In R. Cheung, R. Lau and Q. Li (eds.), New Horizons in Web-
based Learning: Proceedings of the 3rd International Conference on Web-based Learning
(ICWL 2004), World Scientific Publishing Co. Ltd., pp. 200-205.
6. M. Choy, U. Nazir, C.K. Poon and Y.T. Yu (2005), “Experiences in using an automated
system for improving students’ learning of computer programming”. In Proceedings of the
4th International Conference on Web-based Learning (ICWL 2005). Lecture Notes in
Computer Science (LNCS no. 3583), Springer, pp. 267-272.
7. C.R. Graham (2006), “Blended learning systems: definition, current trends, and future
directions”. In C.J. Bonk and C.R. Graham (eds.), Handbook of Blended Learning: Global
Perspectives, Local Designs. San Franciso, CA: Pfeiffer Publishing.
8. D.A. Kolb (1984). Experiential Learning: Experience as the Source of Learning and
Development. New Jersey: Prentice-Hall.
9. U. Nazir, C.K. Poon, Y.T. Yu and M. Choy (2005), “Automated assessment for improving
the learning of computer programming: Potentials and challeges”. In Proceedings of the
9th Global Chinese Conference on Computers in Education (GCCCE 2005), pp. 634-639.
10. M. Schulman (2001), “Basic understandings for developing learning media for the
classroom and beyond”. Learning Technology Newsletter 3(1).
11. Y.T. Yu, C.K. Poon and M. Choy (2006), “Experiences with PASS: Developing and using
a programming assignment assessment system”. In Proceedings of the 6th International
Conference on Quality Software (QSIC 2006), IEEE Computer Society Press, pp. 360-365.
Web-based Logging of Classroom Teaching Activities
for Blended Learning
Joseph Fong
Department of Computer Science, City University of Hong Kong, Hong Kong

csjfong@cityu.edu.hk
Abstract. Nowadays, eLearning has been adapted in all educational institutes,

starting from kindergartens, primary schools, high schools, to the universities.
Teachers are encouraged to use computer and Internet as a teaching medium in
addition to the classroom teaching. The result is a blended learning which
combines eLearning as a supplementary learning means to classroom learning.
The issue becomes what is the best approach for blended learning for the
effective and productive teaching methods. This paper suggests to record (log)
all teaching activities into a web site for students to self study after classroom
learning. The objective is to provide students more flexibility in learning with
more facilities of eLearning, classroom learning and/or blended learning. A
case study is illustrated in the paper for discussion.
Keywords: blended learning, teaching activities, classroom learning, eLearning,

Education
1 Introduction
Education in general are undergoing rapid transition from the traditional learning and
teaching to the more self-motivated mode constructivism through the application of
IT. Many education institutes have made a tremendous step forward in the setting up
of EDO (education office) in the promotion of eLearning. Efforts have been made on
the implementation of a university wide educational Extra/Internet with the aim of
facilitating a better learning and teaching environment as well as improving the
overall student administration. Products such as WebCT [1] and Blackboard [2] have
been in use for the past few years with a degree of success. As a rule, these products
all suffer on two counts:
1. They focus more on course management rather than learning process.

2. Little consideration is given to areas such as workshops and projects. There is a
complete absence of semi-automatic assessment facilities with performance
tracking for facilitating an understanding of the strength and weakness of
students.
Basically, eLearning can be categorized into different areas as follows:
20 Joseph Fong
eLearning for adaptive learning –In order to speed up the learning curve of
learners, we aim to supplement classroom learning with eLearning. The students can
do self learning from web-based learning exercises on the Internet, which will assess
the students＇ academic level and provide them with suitable online exercises to
work on. As a result, the students can learn everywhere, any time through Internet.
eLearning for teacher helpers – In order to reduce teachers ＇ workloads in
authoring exercises for their students, in getting student feedback and in
communicating with students more effectively, many eLearning systems facilitate
these functionalities for teachers as their helpers. For example, eLearning authoring
tools can help teachers to prepare exercises, and BLOG journal files can help teachers
get student feedback on a particular learning problem or subject.
eLearning for distance learning –Besides traditional classroom learning, students
can also access learning facilities remotely. For example, a virtual cyber laboratory
can help students to perform their laboratory exercises at home through the Internet.
An online tutorial session can help students access their tutors through their notebook
computers. Consequently, students can learn in a very comfortable environment
without incurring traveling time, cost and hassle.
eLearning technology infrastructure – In order to make eLearning a success,
technology infrastructure is a must. In fact, eLearning is more cost effective whenever
it can be accessed by mass learners, because individual costs will be less with
increased numbers of learners for eLearning while the eLearning development time
and effort is fixed. In other words, once an eLearning package has been produced, it
can be reused many times with minimal operating cost. However, such mass learner
operations need the support of technology. Thus, research on eLearning technology
infrastructure is another important area for researchers to explore.
On the educational side, Blended Learning provides educators with the opportunity
to showcase best practice by reporting the match between learning or outcome
objectives and the development and implementation of this hybrid (blended) learning
strategies and material to engage students in achieving those objectives. Ideally
underpinned by learning theory, we should also attempt to demonstrate the value
added to learning by using the blended approach. Both components of hybrid
learning — the classroom and the eLearning contributions — should be presented to
the students..
2 Related work
A definition of Blended Learning [3] is the combination of multiple approaches to

learning, such as self-paced, collaborative or inquiry-based study. Blended learning
can be accomplished through the use of 'blended' virtual and physical resources.
Examples include combinations of technology-based materials, face-to-face sessions
and print materials.
Another definition of Blended learning [4] is a powerful training solution that combines e-
learning with a variety of other delivery methods for a superior learning experience.
A third definition of Blended learning [5] combines face-to-face and computer mediated
instruction.
Some educators [6] give their experience of implemented of Outcome-based

learning as blended learning in University of Sydney. As standards for the use of
Information and Communication Technology in teaching and learning are a relatively
new aspect of the blended learning, their adoptions by the academics are still in its
early stage.
Other educators [7] valuated blended learning as a mixture of online and fact-to-
face learning using a variety of learning resources and communications options
available to students and lecturers. In so doing a learning environment is created that
is richer than either a traditional face-to-face environment or a fully online
environment.
ICWL2006 keynote [8] states that Learning theory, collaborative learning, and
deep consideration of the technologies that may be leveraged should all be pulled
together in defining techniques and guidelines for the creation of great Web-based
contents and blended curricula.
The objective of this paper is to implement the idea in the keynote.
3 Classroom teaching activities logging
In general, classroom activities include lectures, tutorial session, laboratory, project,

review, questions and answers, open forum and presentation etc. Each course focuses
in certain aspect of activities due to its syllabus requirements. To the students, they
are mainly interested in two issues: how much do they learn and what is their grade
for the course. If the lecturer can log these activities into a web site for student’s
reference, it will be easier for students to know the priority importance of each
teaching activity as follows:
z Lectures: - We can video record lectures for students to review them at home.
However, this involves facilities management and operations. Many long
distance education institutes provide such services for remote learning. Also,
sometimes students complain about spending too much time copying the written
notes from the blackboard. In this case, lecturer may consider storing the
images of the written notes into the web site for students to download after the
lecture. Furthermore, references can also be put into the web site for students to
enhance their knowledge in the subject.
z Tutorial sessions – Students come to tutorial sessions for small group learning.
They can ask more questions in the sessions for peer-to-peer learning, with
more in-depth discussion of the subject. An effective approach is for teachers
assess the students’ knowledge by giving tutorial exercises in the sessions. After
marking the answers from students, the lecturers can discuss the answers with
the students. Thus, the questions and answers are logged into the web site for
students to review them. In this way, the students can learn even if the students
are absent in the sessions.
z Laboratory exercises – Students use laboratory for their hands-on exercises.

22 Joseph Fong
However, they may not have enough time to finish them in the laboratory. An
alternative is to do the exercises on the Internet at home. Sometimes, for long
distance education institutes, they allow students access laboratory facilities
through remote logging. Both techniques can serve the students well.
z Open forum – It is important for students discuss questions about the lecture
among themselves. An open forum in the classroom and in a web site can be
very helpful. A BLOG system is good to log these open forum conversation.
Students can refer them even though they were not involved at the beginning of
the discussion. This open forum is managed by the lecturer just in case incorrect
message or information is passed among students.
z Mind set diagram – A course in general covers many subjects. It is important

for students to know the learning sequence of these subjects. A mind set
diagram of the course can help students realize the position of each subject in
the course. In other words, how the subjects are related to each other. Students
should also be allowed to put their comments in the subjects which are the
syllabus of the course. A picture is worth thousands words. The diagram is very
helpful for the students’ understanding of the course.
z Grading system – A computerized grading report system can assess students

knowledge effectively and make them realize their understanding of the course
materials immediately. The lecturers post the students’ grade of each
assignment online along with the model answers so that the students can correct
their mistakes at once. This will trigger students ask more questions in the class.
z Project management – Lecturers act as project supervisors to student’s projects.

They define the project requirements and monitor student’s progress through
face-to-face interview during the duration of the project. Furthermore, an
eLearning system allows students to ftp their work to a departmental server on a
regular basis. A test log of submitted work and simple version control tracks
students’ performance. A particular server allows students to install software in
pre-specified folders. This reduces the chance of hard coding, which is a bad
practice normally committed by most students. Video conferencing facilities
can be in place to allow formal communication between students and supervisor
at pre-defined interval, such as once a week, and a log on students’ progress and
supervisors can be maintained.
.
z Assessment – The assessment system usually composes of two elements: quiz
and test. Quiz provides immediate feedback to enhance the learning process
whereas tests are for evaluation of students progress with the following
functions:
9 Calculate marks automatically
9 Generate reports on the performance for quiz as well as test for the
individual, the class and for different cohorts.
9 Students can learn at their own pace and are free to choose the time and
level for their quiz with summary feedback on the performance indicating
weakness in certain areas.

9 Students’ learning pattern rules can be derived through statistical analysis
of their learning results be the course web site using data mining approach.
z Course Work Management – Students can submit coursework with specific

instructions and assigned readings. The collection of assignments is based on a
predetermined schedule and at the pace of students’ progress.
9 Create lockers with unlimited number and level
9 Edit the lockers
9 Remove the lockers
9 Submit homework to the locker through the web browser
9 Acknowledgement on receipts to avoid disputes
z Scheduling – This acts as a communication channel between lectures and

students. Course assignments, submission deadlines and requirements can be
broadcasted to students by lecturers. The lecturers can also mark down project
meetings and agreed work schedule with students. Shareware can be modified
for this purpose.
4 A Case Study
To illustrate the teaching techniques of blended learning, we choose teaching a course

on “Data warehousing and Data mining” as a case study. The subject is rich in content,
and each syllabus can be taught in depth for several lectures. The course aims to focus
on the practical application of the data warehousing and data mining.
The whole course web site hierarchy is designed as simple as possible. It is done
according to Hick’s law [9]. It has predicting for a web site which consumes the user
lesser time to browser if the hierarchy is small. In turns, the user will get the
appropriate data easily with less short-term memory load.
The home page covers the course teaching plan, scheduling, and outcome based
teaching and learning objectives. The coursework consists of the project assignment,
review and tutorial questions, hands-on and eLearning exercises, and references etc.
The Grade web page is to list out students’ grades for each submitted assignment. The
Open forum is for students communicating with each other and the lecturers for
general discussion on the lecture. The Mind Mapping web page describe the learning
sequence of syllabuses in the course.
In the whole web sites, every web page contains a header and a navigation bar, the
consistency layout let the user easily browse the web site without getting lost.
Furthermore, there is no table in the layout. It is good for both search engine
optimization as well as the accessibility of the users who use the screen readers as
shown in Figure 1.
24 Joseph Fong
Figure 1 System Architecture of a course web site
In each lecture, lecture notes, written notes, references, review questions, tutorial
questions, hands-on and eLearning exercises are all put into the course web for
students to review and comprehend.
For lecturers who will be the web masters, instead of uploading and downloading
the files from the web site and local drive for each update, the system provides an
easy way for the web masters to change the content of the website so as to make the
website more user-friendly.
For students who will be the end users, they can download lecture notes, written
notes, references, review questions and tutorial questions from the course web site as
shown in Figure 2 below:
Figure 2 Course material web pages for the case study
In order to help students learn effectively, a mind set diagram is set up on the
course web site for the students to know the learning sequence of the course. They are
the syllabuses of data cleaning, data integration, star schema design, data cube loading,
followed by different data mining techniques of Association Rules, Clustering,
Decision Tree, Neural Network, Web Mining and Genetic Algorithm. XML
technology is introduced in the course for report generation and transmission on the
Internet. The arrows signs show the sequence of learning syllabus in the course.
For students, there is a mind-map learning page provided for them as end users.
They can give out their opinion about the structure of the course with fully
understanding the basic structure of the web site as shown in Figure 3.
26 Joseph Fong
Figure 3 Mind Set diagram for course on Data Warehousing and Data Mining
Instead of uploading and downloading of excel file to update the grades of students,
the system can provide dynamically function for the web master to record the grade
with the address [10].
After marking the review and tutorial questions, the grades are put on the course
web site. The score ranges from-0 (no score) to 1 (full score). The review questions
are on theories and the tutorial questions are on the application of the theories taught
in the class. Students are encouraged for peer-to-peer learning through discussion to
each other before submitting their answers as shown in Figure 4.
CW = 25% |PT = 25% |Exam= 50% |
Student ID Rev 1 Tut 1 Rev 2 Tut 2 Rev 3 Tut 3 Rev 4 Tut 4 Rev 5 Tut 5 25% CW
50184799 1 0.3 0.3 0.3 0.3 1 1 1 1 1 0.72
50453076 1 0.3 0.3 0.3 0.3 1 1 0.3 0.3 1 0.58
50478728 1 0.3 0.3 1 0.3 1 1 1 1 1 0.79
50797788 1 1 0.3 0.3 0.3 1 1 0.3 0.3 0.3 0.58
50832039 1 1 0.3 1 1 1 1 1 1 1 0.93
50854068 1 1 0.3 1 0.3 1 1 1 1 1 0.86
50860456 1 0.3 1 1 1 1 1 1 1 1 0.93
Average 1 0.6 0.4 0.7 0.5 1 1 0.8 0.8 0.9
Figure 4 Course grading report of the case study
In order to encourage peer-to-peer learning, an open forum is put on the course

web site for students discuss problems and solution among themselves. They can also
ask lecturers on the lectures. The open forum applies blogging system such that
students are free to append their comments to each particular subject that they are
interested in. Security need to be implemented to block non-students intruders erase
the blog messages as shown in Figure 5.
28 Joseph Fong
Figure 5 Open forum for the course web site
5 Conclusion
Ever since IT has been widely used in teaching, blended learning is the trend of
teaching method for all levels of educational institutes. Combined with eLearning as
supplementary learning avenue to class room learning, this paper introduces many
ways of logging teaching activities into the web site which can help students learn
more effectively inside and outside of the classroom. The result will be a very close
partnership between teachers and students in teaching and learning.
Reference
1. URL http://www.WebCT.com
2. URL http://www.Blackboard.com
3. URL http://en.wikipedia.org/wiki/Blended_learning.
4. URL: http://www.e-learningcentre.co.uk/eclipse/Resources/blended.htm
5. Graham. Charles (2003), "Blended Learning Systems" in Bonk, C. J. & Graham, C. R.
(Eds.). Handbook of blended learning: Global Perspectives, local designs. San Francisco,
CA: Pfeiffer Publishing. (An Imprint of Wiley)
6. Applebee, A., Elis, R., and Sheely, S.,(2004) "Developing a blended learning community
at the U of Sydney", in R. Atkinson, C. McBeath, D. Joans-Dwyer and R. Philips (Eds),
“Beyond the Comfort Zone” Proceedings of the 21st ASCILITE Conference, pp.58-66.
7. Harding, a., Kaczynski, D., and Wood, L.,(2005) “Evaluation of blended learning
analysis of qualitative data”, Proceedings of UniServe Science Blended Learning
Symposium, pp.56-61.
8. Won Kim (2006), “Directions for Web-Based Learning”, LNCS 4181, Proceedings of
The 5th International Conference on Web-Based Learning, pp.1-7.
9. URL http://www.hockscqc.com/articles/hickslaw.htm
10. URL http://www.cs.cityu.edu.hk/~jfong/grade/CS5483
Blended Learning for Programming Courses:
A Case Study of Outcome Based Teaching & Learning
Fu Lee Wang, Joseph Fong, Marian Choy
Department of Computer Science, City University of Hong Kong,

Kowloon Tong, Hong Kong
{flwang, csjfong, csmchoy}@cityu.edu.hk
Abstract. Teaching of programming has created significant difficulties.

Blended learning is a mixture of face-to-face instruction and computer-assisted
instruction. This paper will share our experiences in City University of Hong
Kong (CityU) as we teach programming courses by blended learning.
Evaluation has showed that blended learning provide great flexibilities to both
teaching and learning of computer programming. The students’ academic
results have been greatly improved in programming course.
Keywords: Blended Learning, Computer Programming, Outcome Based

Teaching & Learning.
1 Introduction
Computer programming is an essential fundamental skill required in many

curriculums for higher education nowadays. It is commonly believed that the students
would develop their general problem-solving skills through learning programming.
However, teaching of computer programming has created significant difficulties for
high-school and university students, and has failed to catalyze the development of
higher order thinking skills [13]. A number of challenges have been identified for
both teaching and learning programming [13]. Related research has showed that
intelligent computer-assisted instruction (ICAI) technology can be a more effective
way of teaching introductory programming courses - for certain populations [3].
Blended learning is to combine face-to-face instruction with computer-assisted
instruction [7, 8]. Blended learning is the convergence of two representative learning
environments. The traditional face-to-face learning environment has been used for
centuries. On the other hand, the rapid development of technologies provides
distributed learning environment as an alternative. In the past, these two
environments are separated because they use different media/method combinations.
Therefore, they are used to address the needs of different audiences [7].
Traditionally, distributed learning is used as an expansion to the face-to-face learning.
Taking the university education as an example, the face-to-face learning
environment is used in a teach-directed synchronous environment where the
interpersonal interaction is a key component. On the other hand, the distributed
Workshop on Blended Learing 2007, Edinburgh, United Kingdom.
Blended Learning for Programming Courses 31
learning environment is usually used in distanced learning, which focus on

asynchronous self-paced learning and learning-material is the key component [8].
The rapid development of technology has a significant impact on the learning
environment. In fact, there is an increasing trend to integrate the two learning
environments as a single system. Nowadays, more and more universities conduct the
learning activities under both environments.
As there is an increasing need for blended learning systems, efforts have been
continuously devoted into the research of blended learning [7]. A number of blended
learning platforms have been developed in real world. Blackboard Academic Suite
is a well-developed learning platform [17] which has been widely used in educational
institutes. WebCT is used to be a competitor of Blackboard [18]. At this moment,
two leading companies co-operate to provide services to the education industry.
The City University of Hong Kong (CityU) has put a lot of effort in development
e-learning. The university has deployed the Blackboard Academic Suite (Bb) [17] as
its unified campus-wide e-learning platform. Many e-learning activities are currently
conducted in this platform. The Department of Computer Science is one of the
departments who are devoted in e-learning development.
Last year, the department underwent a major restructure in the course design in
order to incorporate the implementation of Outcome Based Teaching & Learning
(OBTL). We have successfully implemented blended learning to teach computer
programming courses. This paper is going to share our experiences of blended
learning. We have combined the advantages of both learning environments to deliver
computer programming courses. The statistics has shown that students are greatly
benefited with this mode of study.
2 Teaching & Learning Programming Course at CityU
City University of Hong Kong (CityU) is one of the eight institutions of higher
education which are financially supported by the government of Hong Kong. The
University has achieved phenomenal growth since its establishment in 1984. The
University ranks the top 154 world-wide [15]. The University is strong in technical
science. It ranks the world’s top 51 in the field of engineering, technology and
computer science [1]. We will share our experiences in teaching computer
programming courses at CityU.
The Department of Computer Science is one of the funding departments under the
Faculty of Science and Engineering. In addition to the courses offered to the students
who are majored in Computer Science, the department also offers a lot of service
courses to the students from other departments. Computer programming is one of the
common courses offered to students from various departments. We have
implemented a blended mode of teaching and learning programming course at CityU.
32 Fu Lee Wang, Joseph Fong, Marian Choy
2.1 Outcome Based Teaching & Learning
As a strategic plan, the City University of Hong Kong focuses on enhancement of

teaching and learning qualities. The University has implemented Outcomes Based
Teaching and Learning (OBTL), which is a student-centered approach for education.
The curriculum topics in a program and the courses contained in it are expressed as
the intended outcomes for the students to learn. Teaching is designed to directly
encourage the students to learn those outcomes and assessments will then be done to
confirm that. It is an approach in which the students themselves are actively engaged
in their learning while the teachers are trying to facilitate them to do so.
Derek Bok, President Emeritus and Research Professor at Harvard University,
believes that we need a reformulation in the Undergraduate Education. He identified
some core purposes of undergraduate education [5]. On the other hand, Harvard
University has conducted a review of their entire undergraduate educational programs
[10]. They are going to implement a new curriculum for undergraduates. The OBTL
is developed on similar concept. The university identified some core outcomes (or
objectives). Those outcomes state what the students are expected to be able to do at
the end of a program. Based on the program outcomes, the teachers decide the course
outcomes, which state what the students are expected to be able to do at the end of a
course. The OBTL has been implemented in a number of institutes over many
countries, for example, USA [16], United Kingdom [12], Australia [2], South Africa
[11], etc.
The OBTL is developed based on the concept of constructive alignment [4]. The
key elements of a course, such as learning activities and assessment tasks, must be
aligned to each other so that the intended outcomes may best be achieved. Teaching
and learning activities are designed such that the students are required to enact the
learning activities and therefore they will most likely to achieve the intended
outcomes. The activities can be teacher, peer, or self-initiated. The students actively
gain knowledge through engaging in appropriate learning activities.
Traditional teaching starts from the perspective of the teachers. OBTL works from
the perspective of the learners. Under a constructively aligned curriculum, students
are actively engaged in their learning processes where the teaching are focused on
what the students, rather than on what the teachers, are doing. The teacher needs to
consider what outcomes have the students achieved, how to demonstrate those
outcomes have indeed been achieved, and to what standards. It encourages all
teachers to ask these questions by providing points for reflection on teaching. Unlike
traditional teachings, the teachers have to pay a lot of efforts to design the course
outcomes and related activities, instead of simply deciding on what topics to be
covered.
2.2 OBTL with Blended Learning
One of the major components in OBTL is the design of teaching and learning
activities. These activities must be coherently aligned with the intended learning
outcomes. However, the traditional classroom education does not meet the
requirements of OBTL. Traditionally, the learning activities are limited inside the
classrooms and assessment tasks are usually in the form of examinations. In order to
enhance the qualities of teaching and learning, blended learning is introduced to the
programming courses in CityU to implement OBTL.
Blended learning combines classroom education with e-learning technology. It
provides a large degree of flexibilities to the teachers for course design. With the
support of technologies, the teachers at CityU are able to deliver the course materials
in multiple channels. In the following sub-sections, we will explain more details how
program courses are delivered in mixed channels. Activities in different formats can
be provided to the students with time and geographical constraints. This approach not
only promotes active learning, it also challenges the students to take control of their
own learning.
On the other hand, assessments play an important part in OBTL. Assessments are
designed to align with the course outcomes to provide evidences on how well each
student has achieved the outcomes. Moreover, assessments must be able to measure
students in multiple dimensions. The students are evaluated by their performances in
each outcome Such evidence could be provided by project work, case studies,
assignments, examinations, laboratory work and reports, practicum, etc.
2.3 Computer Programming Course at CityU
In the past, computer programming courses at CityU are taught in a traditional mode.
The course was delivered in a mixture of large-sized face-to-face lectures and small-
sized face-to-face tutorials (which might be in the form of laboratory sections). The
students were evaluated by coursework and final examinations. The coursework was
usually in the format of programming assignment or written quiz, and the final
examination was in the format of paper-based written examinations.
After implementation of Blended Learning, the courses are delivered in multiple
channels:
• The teachers present the primary course materials in the large-sized face-to-face
lectures.
• Small-sized face-to-face tutorials are conducted by the tutors to allow students to
do some programming practices.
• Supplementary course materials are delivered in Internet through the university e-
learning platform. For example, extensive examples are provided to help students
to appreciate the good programming skills.
• Computer programming clinic scheme (subsection 2.3) is setup to provide
consultations to students.
• Online intelligent computer-assisted instruction system (subsection 2.4) is
developed to provide a programming practice platform to the students.
There is one major difference between the OBTL and traditional teaching. The
learning outcomes are clearly stated at the beginning of the courses. Each learning
activity is aligned with the learning outcome. It provides a high level of transparency
to the students. The students have a clear picture about the course structure. They
know what learning activities they must enact in order to achieve the intended
outcomes. On the other hand, the students are assessed by how well they have
achieved the outcomes. The assessments are usually measured in multiple
dimensions. In CityU, the assessments of a programming course include the

followings
• Both online and offline short quizzes will be conducted to evaluate the student’s
performances during the semester.
• Students are required to do some programming assignments to demonstrate their
programming capabilities. The e-learning platform allows a great flexibility in
assignment design. For example, we provide some testing modules in the e-
learning platform. This allows the student to complete the assignments stepwisely.
• The data collected in the intelligent computer-assisted instruction system may also
be used to evaluate the students.
• The e-learning environment in blended learning makes it feasible to conduct online
programming quizzes.
2.4 Computer Programming Clinic Scheme at CityU
For effective learning, it is important to provide the students with a good learning
environment of computer programming. A pilot scheme of Computer Programming
Clinic has been setup in CityU with the support of the Department of Computer
Science.
The clinic recruits students of senior years who are good at programming as
programming consultants. The consultants will share their programming experiences
with students of junior years. The senior students have similar backgrounds as the
junior students. They understand clearly what problems the junior students are
currently facing, and therefore be the most suitable persons to offer helps. This clinic
is developed based on the idea of “help desk” system. We have setup a face-to-face
clinic in the Computer Laboratory of Department of Computer Science. The junior
students can visit the clinic for consultations during school hours. The consultants
will perform the following tasks:
• answer students’ questions related to general programming,
• help students to identify the bugs in their program at high levels,
• help students to formulate high-level pesudocode before programming,
• suggest some appropriate readings for the students if necessary,
• demonstrate a small segment of programming code to the students to help them to
understand their program if necessary,
• assign some simple tutorial problems to the students to help them to understand the
programming concepts.
Another obstacle for the students to learn programming is that they do not know
where to seek help when they have encountered problems. Usually, the students
encounter problems when they are doing some programming work after they returned
home. Most students will put the problems aside and forget to solve their problems
when they return to school. It greatly reduces their enthusiasm to study programming,
if they lack of instant supports.
In order to provide instant supports to the students beyond normal school hours, we
have setup a virtual extension of the Computer Programming Clinic in Internet.
CityU has deployed the Blackboard Academic Suite (Bb) as its unified e-learning
platform [17]. To align with the e-learning strategic development of the University
and to eliminate the development cost, we have implemented online clinic based on
the Blackboard (Figure 1). During school hours and after school hours, consultants
will be on duty to offer help to the junior students.
We devote actively in monitoring the students’ learning. In addition to face-to-
face and on-line consultations, the consultants will analyze the coursework submitted
in the programming courses and data in the electronic systems to identify the slow
learners. Moreover, the instructors of the programming courses will also refer the
students who have difficulties in programming to our clinic. Corresponding
personalized learning program will be provided to the students. This project will
greatly enhance the learning environment of computer programming.
Fig. 1. Computer Programming Clinic – Online Clinic
2.5 Online Intelligent Computer-Assisted Instruction System for Programming
Instant support to the student is a critical factor to the success of teaching and learning
of programming. However, it introduces a huge pressure in the resources, and it may
not be affordable by some universities. It has been showed that intelligent computer-
assisted instruction technology can be a more effective way of teaching introductory
programming courses [3]. We have implemented a computer-assisted instruction
system to supplement our supports.
Figure 2 shows the Programming Assignment aSsessment System (PASS). The
PASS system is a web-based computer-assisted instruction system for computer
programming developed at CityU [6]. The PASS system is a fully automated system
to help students to study programming.
Fig. 2. Programming Assignment aSsessment System (PASS)
The PASS system allows the instructors to setup some tutorial problems. The
instructors provide the input and the corresponding output to each test case. The
students then submit their program for testing. The system automatically complies
and executes the program submitted. By comparing the output generated by the
students’ program and the expected output provided by the instructor, the system will
then provide feedbacks to the students. For example, if the student gets wrong in
certain type of inputs, the system will show the attached annotation provided by the
instructor to give some hints of possible mistakes to the student. The instant feedback
provided by the system provides concrete assistances to students to revise their

programs, and debugging will become more interesting.
On the other hand, some teaching strategies are incorporated with the intelligent
computer-assisted instruction system:
• In OBTL, a student need to demonstrate their skills in each intended learning
outcome in order to pass the programming course. When the teachers design the
programming activities in the PASS, they will associate each activity with
specified outcome. This approach helps the teachers to review the effectiveness of
their programming activates to each intended outcome.
• The PASS system has a powerful statistical function of students’ performance in
each programming problems. It can tell individual student’s performance in each
outcome. Therefore, the system allows the teacher to measure students in multiple
dimensions. Moreover, teachers have a close monitoring of the students’ learning
progress.
• The system allows the teachers to give test cases in different difficulty levels. For
example, some normal cases will be tested as warm-up exercises to the students.
Later on, some boundary cases will be used to test the robustness of the students’
programs. The students may even try some exceptional cases. This mechanism
allows the students to control their paces of learning. On the other hand, we can
cater the needs of students with various learning capabilities.
• Traditionally, the students are required to complete a program before they can do
some testing. Our system allows the teachers to provide the main body of the
program, and students to submit their implementation of functions to the system, or
vice versa. Then, the system will integrate the source codes together as a single
program. This type of exercise is welcomed by the students, as they can speedily
see the outcomes of their program without writing many lines of code. It
significantly improves student’s incentive of learning. Moreover, it makes the
students understand more about the concept of modular programming and also
implementation hiding.
• Finally, the system allows the students to submit components of a program as
separate files. The components files may be developed by individual student. The
system will integrate the source files together as a single project. This team-work
style exercise makes the students understand the paradigm of software
development.
The PASS system allows a tailor-made learning pace and style for individual student.
It has provided a quick and convenient channel for students to test their work without
manual involvement. Instant feedback to students encourages them to enhance their
programming skills. The introduction of PASS has made the learning of computer
programming more rewarding than before.
3 Evaluation and Discussion
A number of extensive evaluations have been conducted to measure the effectiveness

of our teaching model. Evaluation results have shown that the blended learning is a
promising approach in teaching and learning programming.
3.1 Evaluation of Course Structure
It is suggested that the blended learning can be measured by interviews and

questionnaires [9]. We have conducted both evaluations to measure the effectiveness
of the blended learning.
A focus group session has been held with students who enrolled for any blended
courses of computer programming in the Department. A set of interview questions
are designed by professionals in education development. The students are
interviewed by an independent interviewer and none of the course lecturers were
presented.
All the students in the focus group believe that the blended learning model can help
them to learn the programming course more effectively. Most of the students
appreciate the flexibilities provided by the blended learning. The students can self-
control their learning paces. The anytime/anywhere studying mode allows them to
work at the time when they have the highest productivities. Moreover, the students
become more independent and self-disciplined in their learning. Their time
management skills are also enhanced. The preliminary results of interview suggest
that blended learning is a good teaching and learning model.
On the other hand, all the students appreciate interactivity of the online assessment
system. However, some of the students are less satisfied. They hope that the online
intelligent computer-assisted instruction system can provide more feedbacks to help
them to debug their programs. This provides some directions for future enhancement
of the system.
Effectiveness of self-paced learning
Effectiveness to identify student's weakness
Effectiveness to learning programming
Effectiveness to encourage continuous learning
Effectiveness to achieve learning outcomes
Effectiveness of tutorials
Effectiveness of programming assignment
Effectiveness of supplementary web
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.0
0
Fig. 3. Result of Questionnaires about the Course Structure
In order to get a more quantitative measurement for the course structure of

programming courses, we have conducted a survey by questionnaires. The
questionnaires are designed by professionals in education development in the similar
way as [9]. The students are asked to score each dimension of the course structure on
the scale from 0 to 10, where a score of 10 represents the highest satisfaction, while 0
represents the least satisfaction. 250 students have participated in the survey. The
results are summarized as Figure 4.
In the Figure, we can clearly see that the students are highly satisfied with the
course structure. The students are happy with the flexibilities provided by the
blended learning. They most believe that the mixture of teaching channels, such as
mixture of tutorial, assignment, supplementary web and online assessment system,
can effectively teach them about computer programming. They help the students to
identify their weakness and control their own learning paces. Therefore, the students
can achieve the intended learning outcomes effectively (Figure 3).
An ideal blended learning is a mixture of classroom learning and electronic
learning. Self-paced learning is one of the major advantages of electronic learning [7,
8]. However, the students are generally less satisfied with the effectiveness of self-
paced learning (Figure 3). As a result, there is potential to further blend our courses.
In the future, we will investigate on how to improve the self-pace learning in
programming courses.
3.2 Evaluation of Students’ Performances
In the past, the students taking programming courses in CityU are assessed by
coursework and final examinations. The coursework was usually in the format of
programming assignments, and the final examination was in the format of written
examinations. After implementation of OBTL, the students are assessed in multiple
dimensions. We have compared the results of the students before and after the
implementation of OBTL.
Table 1. Statistics of Course CS2362 Computer Programming for Engineers & Scientists
Year 2004 Year 2005 Year 2006

Total no. of students 277 253 251
Grade Score Boundary % of Students % of Students % of Students
A 69.5 7.94% 7.11% 26.00%
B 54.5 16.25% 17.79% 22.40%
C 39.5 35.38% 23.72% 23.60%
D 34.5 9.75% 12.65% 4.80%
F below 34.5 30.69% 38.74% 23.20%
We have selected the course CS2362 Computer Programming for Engineers &
Scientists as an example. This is a typical programming course at the introductory
level. Because the class size of this course is very large, the statistical information of
this course is worthy trusted. On the other hand, the materials of assessment are
moderated by peer review to ensure the standard of assessment. No scaling of score
has been conducted in this course. The score boundary for each grade has been fixed
by the department. As a result, this graded distribution of students is a very important

indicator to show the performances of teaching and learning.
In years 2004 and 2005, the programming courses were taught in traditional mode.
Only a small percentage of students got grade “A”, while a large percentage of
students failed the course in these two years (Table 1). These students who failed the
course may retake the course in the next year. After the blended learning was
implemented in year 2006, the percentage of grade “A” students increased
dramatically from 7~8% to 26% (Table 1). At the same time, the percentage of
failure decreased significantly. As shown in the table, the students’ performance in
the programming course increase significantly after the implementation of the
blended learning. This is a strong evidence to show the success of the blended
learning.
4 Conclusion
This paper has shared our experiences in implementing blended learning in teaching
computer Programming in City University of Hong Kong. Outcome Based Teaching
and Learning is a new pedagogy. However, traditional teaching model impose a lot
of constraints in implementing OBTL. In contrary, blended learning provides great
flexibilities to both the teachers and the students. At the same time, it can cater
students with different learning paces. The interviews and questionnaires have shown
that blended learning is very effective in teaching and learning computer
programming. The students’ performances in the assessments have further confirmed
our findings. We think that blended learning can be applied to other courses in the
future as well.
5 Acknowledgement
The PASS System is a System developed by C.K. Poon, Y.T. Yu, Marian Choy, Fu
Lee Wang, Celine Chong, Usman Nazir, P. F. Tam, Isaac Yeung, Sam Lam and Leo
Yuen at Department of Computer Science, City University of Hong Kong. The
projected was substantially supported by a grant from City University of Hong Kong
(Project No. 6000118). The Computer Programming Clinic is a project lead by Fu Lee
Wang, Kenneth Lee and Jiying Wang at Department of Computer Science, City
University of Hong Kong. We would like to thank individuals for their significant
contributions to the paper.
References
1. Academic Ranking of World Universities by Broad Subject Fields – 2007, Top

100 world universities in Engineering/Technology and Computer Sciences,
available at http://ed.sjtu.edu.cn/ARWU-FIELD2007/ENG.htm
2. Applebee, A.C., Ellis, R.A., & Sheely, S.D., (2004), Developing a blended
learning community at the University of Sydney: Broadening the comfort zone,
Proceedings of the 21st ASCILITE Conference, pp. 58-66, Perth, Western
Australia, 5-8 December 2004.
3. Anderson, J. R., & Skwarecki, E. (1986). The automated tutoring of introductory
computer programming. Communications of the ACM, 29(9):842-849.
4. Biggs, J. (2003) Teaching for Quality Learning at University, 2nd Edition,
McGraw-Hill Education, Open University Press.
5. Bok, D. (2005), Our Underachieving Colleges: A Candid Look at How Much
Students Learn and Why They Should Be Learning More, Princeton University
Press.
6. M. Choy, U. Nazir, C.K. Poon and Y.T. Yu, Experiences in using an automated
system for improving students learning of computer programming. In
Proceedings of the 4th International Conference on Web-based Learning (ICWL
2005), Lecture Notes in Computer Science (LNCS), volume 3583, Springer,
Hong Kong, 31 July 2005 - 3 August 2005, pages 267-272.
7. Graham, C.R., (2005) Blended learning system: definition, current trends, and
future directions. In Bonk, C. J. & Graham, C. R. (Eds.). Handbook of blended
learning: Global Perspectives, local designs (pp 3-21), San Francisco, CA:
Pfeiffer Publishing.
8. Graham, C.R., & Allen, S. (2005). Blended learning: An emerging trend in
education. In C. Howard, J.V. Boettecher, L. Justice & K.D. Schenk (Eds.),
Encyclopedia of Distance Learning (99. 172-179). Hershey, PA: Idea Group Inc.
9. Harding, A., Kaczynski, D. & Wood, L.N., (2005), Evaluation of blended
learning: analysis of qualitative data, Proceedings of the Symposium of Blended
Learning in Science Teaching & Learning, pp. 56-62, The University of Sydney,
Australia, 28th-30th September 2005.
10. Harvard University, Faculty of Arts and Sciences, (2004), A Report on the
Harvard College Curricular Review, April 2004.
11. Mason, M.B. Outcomes-based Education in South African Curricular Reform.
Cambridge Journal of Education, Cambridge, UK, Carfax, 1999, 29(1): 137-143.
12. Ross N & Davies D (1999). Outcome-based learning and the electronic
curriculum at Birmingham Medical School, Medical Teacher 21(1): 26-31
13. Sleeman, D., (1986), The Challenges of teaching computer programming.
Communication of the ACM, 29(9):840-841.
14. Soloway, E., (1986), Learning to program = learning to construct mechanisms
and explanations, Communications of the ACM, 29(9):850-858.
15. Times Higher Education 2006 World University Rankings, available at:
http://www.thes.co.uk/worldrankings
16. Zahorchak, G.L. and Boyd, W.L., “The politics of outcome-based education in
Pennsylvania”, Annual Conference of the University Council for Educational
Administration, Philadelphia, PA, October 1994.
17. Blackboard Academic Suite, http://www.blackboard.com/us/index.Bb
18. WebCT, http://www.blackboard.com/webct
Blended Learning with Webs, Wikis and Weblogs
Philip Barker
School of Computing, University of Teesside,

Middlesbrough, TS1 3BA, UK.
P.G.Barker@tees.ac.uk
http://www.philip-barker.net
Abstract. This paper discusses blended learning in terms of the optimal use of
multiple learning channels and the educational messages that they can each
embed. Blending algorithms are introduced as a mechanism for specifying the
composition of blends and a learning spirals model is introduced as a vehicle to
relate stimuli (embedded within educational messages) to the cognitive
processes that these are intended to stimulate. Webs, wikis and weblogs are
then discussed in terms of their importance as components within a blended e-
learning environment. Their significance both as a learning resource and as a
knowledge management tool is discussed.
Keywords: blended learning, blending algorithms, learning spirals, webs,

wikis, weblogs
1. Introduction
The fundamental axiom underlying this paper is that there is a need to pass skills
and knowledge from one generation of people to another using whatever technologies
that we have at our disposal. This could involve the transmission of material during
a face-to-face encounter between a teacher and one or more students. It could
involve the sending of an educational document from an author who is an expert in a
particular area to someone else who is less well-versed in that subject. Such a
transmission could involve a postal service or a computer network system. Of
course, an educational transmission could also involve listening to a radio programme
or a podcast or it might require watching a film or television programme. There are
also many books, journals and other communication artefacts (both electronic and
paper-based) that can be used to support the fundamental process of educational
delivery.
In this current age of ‘digital computing systems’ much learning now takes place
through the medium of computers. These may be ‘stand-alone’ devices that exist
within a learning organisation (such as a school or university) or in someone’s home
or workplace. Increasingly, however, most computers are now networked together
using either a private network system or a public facility such as the Internet. With
the advent of cellular wireless networks, we are moving towards an era in which we
have ubiquitous electronic inter-connectivity.
Bearing in mind what has been said above, it is important to realise that we now
have a tremendous range of educational resources available. This raises the rather
simple, but very important question of which ones should be used (in any given
situation) in order to achieve a particular educational objective. Unfortunately,
although the question is a simple one, finding an answer is much more complex.
This complexity arises because of the many factors that need to be taken into account
within any given teaching and learning situation. The primary concern of educators
(that is, those involved in the educational delivery process) will be the effectiveness
with which ‘educational messages’ can be created, delivered to and assimilated by a
particular target audience. Of course, other stakeholders within an educational
system (such as politicians, bureaucrats, lawyers and managers) will be more
concerned with other issues such as the legality, validity, relevance and cost of the
messages that are being broadcast to consumers.
Being an academic, my primary concern is therefore with a consideration of how
we generate educational messages, transmit them to students and then measure their
effectiveness in terms of the behaviour changes that they induce. So, what is the best
way of doing this? My honest answer would be: “I don’t really know”. I say this
because the question must be made much more specific before an attempt can be
made to answer it. Thus, what would work well in one situation might fail
completely in another. In some situations, a lecture course could be used to encode
relevant educational messages - and may work very well. In other situations, a text
book may be a better alternative while in others, practical involvement may be a
necessity for successful delivery. In my own discipline of computing, for example,
it is inconceivable that I could teach computer programming without letting students
actually become involved in programming activities. Effective teaching and
learning in this particular context involves (minimally) two basic types of learning
activity. First, attending lectures (or a suitable substitute) in order to learn about the
theory and principles of computer programming and how to apply these using a
particular programming language. Second, undertaking practical exercises that are
designed to develop the skills needed to produce efficient and effective computer
programs. Effective learning in this situation thus involves an appropriate ‘blend’ of
two different types of learning activity. In this example, two ‘learning channels’ are
being used in order to achieve the best possible learning experience for the people
involved. More channels could be used (at extra cost) or either one could be used
alone (but with reduced efficiency of learning).
If we try to generalise what has been said above, we can define blended learning as
an approach to pedagogy that mixes various types of educational resource - or
learning channel - (that may originate from many different sources) in order to create
an optimal learning experience for a consumer. The consumer could be an
individual person or a group of people - such as a university class or an online
community of learners. Of course, the larger the group of people involved, the more
difficult it becomes to tune an educational message to the needs of particular
individuals within that group. This is an age-old problem that has always plagued
conventional face-to-face teaching (involving group sizes of more than one) but, for
44 Philip Barker
which, solutions can be found within the domain of electronic teaching and learning.
The way in which blended learning might address this issue is briefly discussed later.
In this paper I would like to try to explain my model of blended learning and its
relationship to the cognitive processes that this approach to learning is intended to
stimulate and support. Naturally, an important aspect of learning is the acquisition
of knowledge and skills. I am particularly interested in the ways in which people
acquire knowledge, manage it and share it with others. The latter part of the paper
will therefore deal with how electronic artefacts such as webs, wikis and weblogs can
help to support blended learning and the acquisition, sharing and management of both
private and public knowledge.
2. Underlying Theory
According to Carman (2002) blended learning theory arises from the merging of
three basic philosophies of knowledge and skill acquisition: cognitivism,
constructivism and performance support. Each of these areas is concerned with a
slightly different aspect of the ways in which individuals develop their knowledge and
skills. Cognitivism is concerned with understanding the way in which the human
mind works (cognitive behaviour) in relation to the various mental states in which it
can exist. It can be used to explain thinking processes and creativity in terms of
various ‘information processing’ models - see, for example, the ‘HIP’ model proposed
by Card, Moran and Newell (1983). Constructivism deals with how individuals
develop knowledge as a result of the various experiences to which they are exposed
(Barker, 2004). The performance-support dimension of blended learning is more
concerned with the development of skills (both physical and cognitive) and how
different types of performance aid can be developed in order to overcome
fundamental human limitations with respect to the different types of task that they
undertake (Barker, 1995; Barker, van Schaik and Famakinwa, 2007).
From a purely pragmatic perspective, the roots of blended learning lie within the
multi-channel approach that was discussed in the previous section. This model of
blended learning is depicted schematically in Figure 1. As can be seen from this
diagram, each channel within a given learning environment can host a number of
different ‘modes’ of educational communication. Each mode, in turn, will require an
appropriate linguistic framework within which to encode the educational messages
that were previously referred to in the Introduction to this paper. Of course, the
messages that are embedded in a channel can also employ different styles or genres of
communication.
Naturally, it is hoped that the educational messages carried by the channels
depicted in Figure 1 will embed appropriate stimuli that will activate and motivate a
learner to assimilate the message content in an appropriate way, build new and/or
augment existing cognitive structures and modify his/her behaviour accordingly.
Naturally, an understanding of the principles and techniques of semiotics is an
important asset in relation to designing educational messages and predicting their
affect on human behaviour (Chandler, 2006; de Souza, 2005). The role of semiotics
in relation to message transmission (in general) and educational messages (in

particular) is discussed elsewhere (Barker, 2006a).
Fig. 1. Blended Learning in terms of multiple learning channels.
My own definition of blended learning and its realisation in terms of the activation
of learning processes within the cognitive space of a learner is described in detail
elsewhere (Barker, 2006a; 2006b). Within that work, an attempt was made to define
blending algorithms in terms of the contributions made by the various learning
channels that contribute to a particular learning scenario. It is my contention that the
learning components of a given educational environment can be represented by the
vertices of a polygon - the area contained within the polygon then defines the
allowable blend space for the components concerned. Any given blend can then be
represented by the geometric distances of the blend point from the vertices of the
polygonal structure. These concepts are illustrated schematically in Figure 2 for a
situation involving the use of three learning channels (components).
Fig. 1. A blend space produced by three types of learning resource.
By combining the geometrical distances depicted in diagrams like that shown in

Figure 2 it is possible to derive blending algorithms similar to that shown below:
Blend(X) = F({A}/a, {B}/b, {C}/c)
46 Philip Barker
In the above equation [{A}, {B}, {C}] represents the detailed specification of the
resource mix used to create blend X. Each item in this list (such as {A}) denotes
the pragmatic resource set that makes up the conceptual learning mode denoted by A.
Having identified a mechanism for specifying the composition of a blend, it
remains now to explore how different points within the blend space influence the
effectiveness of learning. It is therefore proposed that the blend (denoted by X in
Figure 2) along with the cost of producing a course (having that blend composition)
influences the course outcomes in the following way:
Success(X, Attrition, Achievement) = F(Blend, Cost)
An obvious extension of the model presented in Figure 2 would be to consider the
implications of accommodating different learning styles and preferred mechanisms of
learning (Honey and Mumford. 2000; Felder and Silverman, 1998). In this situation,
the point X in Figure 2 would now need to represent a ‘generic blend’ for a particular
course and a given cohort of students. This is referred to as X{generic} for the
course/cohort combination. It assumes that exactly the same blend is used for every
student in the cohort. An attractive advantage of using blended learning is its ability
to use different combinations of resources for each individual student. A number of
different variants of X{generic} therefore need to be made available in which the
actual blend composition can be influenced by the learning styles and preferences of
each individual student in a cohort. This is referred to as X{specific(i)}. This latter
expression describes the specific composition of the learning blend administered to a
particular learner (within a given cohort) that belongs to ‘student type i’. Using this
approach it is now possible to consider the total set of blends for the whole cohort of
students as forming an area (that defines the ‘mix variability’) within the underlying
blending polygon.
Naturally, there is likely to be some level of mis-match between any given
student’s ‘ideal’ learning requirements and those suggested by the blending strategy
for that student. It is therefore important to reflect the effect of this factor within the
blending relationships that were presented earlier. This can be done by introducing an
‘index of fit’ (iFit) into the equations in order to represent how well a chosen
X(specific) maps on to the ideal mix for a particular student. The new relationship
between learning outcomes and blending variables therefore now becomes:
Success(Xi, Attrition, Achievement) = F(iFit, Blend, Cost)
It is important to emphasise that the detailed specification of the functionality
referred to in the equations introduced above has yet to be identified through further
research. We are in the process of exploring the implications of the above
relationships within different types of learning environment. Initially, we are
investigating the situations that prevail in small-scale studies that we are more easily
able to monitor and control (Barker, 2006c; Webb et al, 2004).
Of course, as has been suggested earlier in this paper, it is important to consider the
effect that a blended learning mix has on the cognitive behaviour of those who are
exposed to it. In our research we have been using the term learning spiral as a
metaphor for a cognitive process related to a particular dimension of knowledge that
an individual has. Our use of learning spirals is illustrated schematically in the
model presented in Figure 3. This figure shows six cognitive processes (labelled P1
through P6) each one of which is represented by a corresponding learning spiral. Of

course, the spirals shown in this diagram are not static (as they are depicted in the
figure) - unless for some reason they have become dormant.
Fig. 3. The learning spirals model for cognitive processes.
Spirals are characterised by a number of important properties such as rotational

speed, breadth of coverage, skewness, trajectory and depth of reach. Spirals can
interact with each other and are strongly influenced by the underlying knowledge
structures that reside in a person’s long-term memory. As was mentioned above, the
state of a learning spiral can be perturbed by the stimuli embedded in the learning
resources that a person uses. An individual’s preferred ways of learning - that is,
his/her individual learning styles - can also have an effect on the properties of the
learning spirals. When considering the design of blended learning resources, it is
therefore imperative that we consider how the embedded stimuli (and their containing
environments) can be targeted at specific learning spirals/processes. A simple Web-
based model depicting our vision of the co-ordinated activity of learning spirals can
be found at the URL specified in (Barker, 2006b). A detailed description and
discussion of this model is presented elsewhere (Barker, 2006c).
3. Knowledge Sharing
As was mentioned at the beginning of this paper, the sharing of knowledge and
skills is a fundamental requirement of all teaching and learning systems. Naturally,
within these systems, people (in general) and academics (in particular) have important
roles to play in relation to being originators of educational messages. They must
therefore be considered as an essential component of a blended learning environment.
Depending upon the nature of the learning scenarios and the blending mix in which
they are employed, people may play an active (primary or ‘front-line’) role or they
may take on a more passive (secondary or indirectly supportive) function. For
48 Philip Barker
example, a lecturer or tutor involved in delivering a face-to-face course plays an

active role. However, the author of a course text book might only play a very
passive role in the delivery of that course.
It is important to remember that many academics spend a ‘life-time’ studying a
particular discipline and, as a result of this they acquire vast amounts of knowledge,
skills and experience which they make available to their colleagues, their students and
the world at large. Bearing in mind the importance of this process of knowledge
sharing, it is imperative to consider ways in which it might be optimised.
The underlying concept of a shareable medium is central to a knowledge sharing
processes. This is needed for two purposes: personal survival and community
sharing. Because of the limitations of the human mind, we all need to use ‘external
media’ in order both to remember things and to communicate our ideas (and
knowledge) to others. This process is referred to as ‘exteriorisation’. Commonly
used media include paper, audio tape and film (both celluloid and digital).
Naturally, the ‘power’ and usefulness of a medium depends critically upon the range
of material that it can ‘hold’ and the knowledge representation (and manipulation)
facilities that it provides - these are denoted by the ‘linguistic frameworks’ that were
referred to earlier in Figure 1.
The problem with ‘real spoken words’ (as a shareable medium) is their transience
or short livedness. Unless a listener can rapidly decode the educational message that
is encoded in a spoken utterance, it becomes lost forever. Recorded words can be
‘rewound and replayed’ in order to get a second (or third or, even more) opportunity
to capture the content of the utterance. Because of its more stable properties, paper
has (for centuries) been the medium of choice for knowledge sharing. However, it
lacks many of the attractive features that more modern electronic media possess -
such as ease of transmission from one location to another, inherent interactivity, ease
of updating and the ability to easily represent change and movement (Barker, 2007a).
Naturally, the transition from paper to electronic modes of writing will require the
development of new types of writing skill (Barker, 2005a)
Until the advent of computers and the ubiquitous availability of global
communication networks, there has been no significantly dynamic, interactive and
spontaneous way of sharing skills and knowledge - other than through face-to-face
exchanges. Of course, nowadays this has all changed. The advent of online
electronic artefacts such as those described in the following section of this paper,
means that people can share knowledge and ideas literally ‘as they are being created’.
Furthermore, through the ‘new’ electronic media, the participative (or communal)
creation of shared knowledge has at last become a reality. However, having said
this, it is very important that we consider the distinction between public knowledge
(that we wish to share) and private knowledge (that we do not wish to expose to
others).
Naturally, each and every one of us has a ‘preferred set’ of knowledge-storage
media and techniques that we use to support our everyday existence. For example,
these may be conventional devices (such as notepads, the books we write, voice
memo-pads, and so on) or they may be electronic tools such as a laptop computer,
tablet device or a PDA. Unfortunately, however, when we want to share our
material with others, we often come up against three basic types of problem - all of
which are related to compatibility issues. First, one person’s media may not be
compatible with those of another person. Second, even though a given medium may
be commonly accepted and standardised across a given population of users, the ways
in which each individual represents his/her knowledge (using that medium) may be
different - see the range of knowledge representation frameworks depicted in Figure
1. Finally, even though the members of a sharing group may agree on a common
knowledge representation framework, they may not necessarily agree upon how
collections of knowledge items should be classified and aggregated into larger units.
Again, like the ‘individualisation of learning’ problem that was described in the
previous section, this is an age-old problem that has existed ever since people began
to make knowledge collections public and shareable. However, the problem is
magnified in globally available digital collections of public knowledge - primarily
because of the tremendous number of people involved and the different views that
they hold about how knowledge archives should be structured, classified and
accessed. Of course, just as the problem is more difficult in the digital era, it is also
easier to find workable solutions - such as the use of personal and shared ontologies.
But these are not the only problems that we face with shared distributed knowledge
systems. As I have discussed elsewhere (in the context of ‘distributed blogs’) a
more difficult problem to solve is the emergence of the ‘knowledge holes’ that arise if
knowledge is lost or retracted.
Because of their importance within contemporary and future blended learning
environments, it is important to consider how the new electronic artefacts (such as
webs and wikis) can best be used to support learning processes. This issue is
discussed in the following section of the paper.
4. Using Webs, Wikis and Weblogs
As has been suggested earlier in this paper, when designing a course that depends
upon the use of blended learning, it is important to consider the nature of the
components that will be used, the educational functions that they will fulfil and the
extent to which they will be deployed. A course designer therefore has to balance a
complex inter-play between the different learning channels that are available for the
implementation of a given learning scenario and the ways in which these are used to
achieve given educational objectives.
Undoubtedly, over the last decade or so, computer-based web structures have
become an important mechanism for providing access to course resources that are
available in electronic form. Because of their importance, the remainder of this
section briefly outlines some of the major roles that webs, weblogs and wikis might
play as resources within a blended environment.
In general, there are two broad approaches that are commonly used for harnessing
an electronic web facility; these are referred to as the disseminative and participative
modes of use. Because of the advantages that are offered by the electronic mode of
delivery, there is an increasing tendency to use web-based approaches for the
dissemination of learning materials. This is an attractive approach since a major
imperative in education has to be a reduction in the volume of paper that is currently
50 Philip Barker
consumed. This can be easily achieved using web-based approaches - provided they
are appropriately designed.
As well as the disseminative role of webs, an equally important function that web-
based systems can provide is mechanisms for the support of participation. This can
take a variety of different forms depending upon the nature of the involvement that a
course designer wants to develop for his/her course members. One of the simplest
types of involvement involves participation in online testing and assessment
procedures. Much more participative roles can be developed through the use of
online conferencing using synchronous and/or asynchronous techniques. These
approaches usually involve groups of students and staff forming an online community
and using the conferencing system to communicate with each other. More advanced
approaches to this type of participation can involve the use of powerful online
computer-supported collaborative learning environments. These enable groups of
people to work together in real-time in order to solve particular problems in a
collaborative and participative way.
Two other very important web-based tools that facilitate participation and
collaboration are weblogs and wikis. Each of these provides a different type of
participation. They can each be used in a variety of different ways to facilitate the
collaborative development and assessment of electronic knowledge. The potential
roles that wikis and weblogs can play within a blended learning environment have
been discussed in detail elsewhere (Barker, 2005b; 2005c; 2007a; 2007b). This
section therefore only summarises the essential features of each medium.
A weblog (or ‘blog’ as it is often called) is a web-based structure that enables an
individual (the blogger) to make postings to it. A posting is similar to an electronic
mail message. These are then ‘visible’ to all those people who have read access to
the blog. This visibility will depend upon whether the blog lies within the public
domain (anyone can read it) or whether it resides within a closed user-group (where
only members of the group can read it). Those people who can read the weblog will
usually also be able to post comments relating to the particular entries that it contains
- of course, these may be scrutinised and moderated before they are actually
published. A typical example of such a blog (created using the Moveable Type
system) can be found at the Web address given in (Barker, 2007c). The attractive
feature of a weblog lies in the fact that its owner can get feedback from other people
about the ideas and/or opinions that he/she posts to the blog. Weblogs therefore have
obvious potential as an educational tool for providing feedback to learners - both
tutor-based and peer-feedback.
Within a given blog, the locus of control usually resides with the individual person
who has created that artefact. In contrast, a wiki is a structure that enables a group
of people to post material to a shared web-space. The locus of control now lies with
a group of people rather than an individual. It is therefore a facility that supports
truly collaborative knowledge development.
As was mentioned earlier in this section, webs, wikis and weblogs are now starting
to become a standard part of blended learning courses - see, for example, Gallop
(2006) and Doolan (2006). It is therefore important to consider how these resources
fit in alongside the other types of resource that are used for course development. This
issue is discussed in more detail elsewhere (Barker, 2007b).
5. Concluding Remarks
There is a growing amount of empirical and pragmatic evidence to support the
positive utility of the use of blended learning as an affective way of passing across
knowledge and skills from one generation to another (Carman, 2002; Bielawski and
Metcalf, 2005; Bonk and Graham, 2005). However, as important as these
contributions are, there is no substitute for an adequate ‘theory of blended learning’
that is rooted firmly within its two primary parent domains: instructional design and
cognitive science. The first of these should advise us about how we should prepare
blended environments to meet particular types of educational need while the second
should guide us with respect to being able to predict the effects that these
environments will have on the states of mind of those who are exposed to them.
References
Barker, P.G., (ed) (1995). Electronic Performance Support Systems, Special Issue of
Innovations in Education and Training International, 32(1), 1-73.
Barker, P.G., (2004). Implementing Constructivism Using an e-Science Paradigm, 3803-3810
in EDMEDIA 2004: Proceedings of the World Conference on Educational Multimedia,
Hypermedia and Telecommunications, Lugano, Switzerland, Edited by L. Cantoni and C.
McLoughlin, Association for the Advancement of Computing in Education, Norfolk,
Virginia, USA.
Barker,P.G., (2005a). Teaching and Assessing Electronic Writing Skills, 167-177 in
Proceedings of the CBLIS 2005 International Conference on Computer-Based Learning in
Science - Integrating New Technologies into Science, 2-6 July, University of Zilina,
Slovakia, Edited by Z.C. Zacharia and C.P. Constantinou, University of Zilina Press,
Slovakia.
Barker, P.G., ( 2005b). A Role for Weblogs in Electronic Course Delivery; 677-682 in
Proceedings of the EDMEDIA 2005 World Conference on Multimedia, Hypermedia and
Telecommunications, Montreal, Canada, Edited by P. Kommers and G. Richards, AACE,
Norfolk, Virginia, USA.
Barker, P.G., ( 2005c). Potential Uses of Weblogs in Electronic Course Delivery, 515-522 in
Proceedings of the E-LEARN 2005 World Conference on E-Learning in Corporate,
Government Healthcare & Higher Education, Vancouver, Canada Edited by G. Richards,
AACE, Norfolk, Virginia, USA.
Barker, P.G., (2006a). Motivation, Learning Spirals and Blended Learning, 91-97 in
Proceedings of the First Annual Blended Learning Conference: ‘Blended Learning -
Promoting Dialogue in Innovation and Practice’, 15th June 2006, University of
Hertfordshire Press, Hatfield, Hertfordshire, UK.
Barker, P.G., (2006b). Learning Spirals Model, Available online at: http://www.philip-
barker.demon.co.uk/LearningSpirals/
Barker, P.G., (2006c). Motivating Students - Can Blended Learning Help? 1849-1854 in
Proceedings of the E-Learn 2006 World Conference on E-Learning in Corporate,
Government, Healthcare and Higher Education, Honolulu, Hawaii, 13-17 October, Edited
by T. Reeves and S. Yamashita, AACE, Norfolk VA, USA.
Barker, P.G., (2007a). Using Wikis as a Teaching and Learning Resource, Paper to appear in
the Proceedings of the CBLIS 2007 International Conference on Computer-Based Learning
in Science, Crete, Greece.
52 Philip Barker
Barker, P.G., (2007b). Blended Learning the Wiki Way, Paper to appear in the Proceedings of
the Second Annual Blended Learning Conference: ‘Supporting the Net Generation Learner’,
University of Hertfordshire, Hatfield, Hertfordshire, UK.
Barker, P.G., (2007c). The Stockton Rambling Club TS20 Weblog, available online at:
http://ts20.gazettelive.co.uk/clubssocieties/stockton_rambling/
Barker, P.G., van Schaik, P. and Famakinwa, O., (2007). Building Electronic Performance
Support Environments for First-Year University Students, Innovations in Teaching and
International, 44(3), in press.
Bielawski, L. and Metcalf, D., (2005). Blended e-Learning - Integrating Knowledge,
Performance Support and Online Learning, HRD Press Inc, Amherst, MA, USA.
Bonk, C.J. and Graham, C.R., (2005). The Handbook of Blended Learning: Global
Perspectives, Local Designs, Pfeiffer, San Francisco, CA, USA.
Card, S.K., Moran, T.P. and Newell, A.P., (1983). The Psychology of Human-Computer
Interaction, Lawrence Erlbaum Associates, Hillsdale, NJ, USA.
Carman, J.M., (2002). Blended Learning Design: Five Key Ingredients, KnowledgeNet Inc.,
this document is available online at:
http://www.knowledgenet.com/pdf/Blended%20Learning%20Design_1028.PDF
(accessed 22nd February, 2007).
Chandler, D., (2006). Semiotics: The Basics, 2nd Edition, Routledge, London, UK.
de Souza, C.S., (2005). The Semiotic Engineering of Human-Computer Interaction, The MIT
Press, Cambridge, MA, USA.
Doolan, M., (2006), Effective Strategies for Building a Learning Community Online Using a
Wiki, 51-63 in Proceedings of the First Annual Blended Learning Conference: ‘Blended
Learning - Promoting Dialogue in Innovation and Practice’, 15th June 2006, University of
Hertfordshire Press, Hatfield, Hertfordshire, UK.
Felder, R.M. and Silverman, L.K., (1998). Learning and Teaching Styles in Engineering
Education, Engineering Education, 78(7), 674-681.
Gallop, R., (2006). Blending by Blogging: how “Wikied” can it get?, 25-32 in Proceedings of
the First Annual Blended Learning Conference: ‘Blended Learning - Promoting Dialogue in
Innovation and Practice’, 15th June 2006, University of Hertfordshire Press, Hatfield,
Hertfordshire, UK.
Honey, P. and Mumford, A., (2000). The Learning Styles Questionnaire (80-item version),
Peter Honey Publications, Maidenhead, UK.
Webb, E.R., Jones, A.D., Barker, P.G. and van Schaik, P., (2004). Using E-Learning
Dialogues Within a Blended Learning Environment, 2790-2796 in EDMEDIA 2004:
Proceedings of the World Conference on Educational Multimedia, Hypermedia and
Telecommunications, Lugano, Switzerland, Edited by L. Cantoni and C. McLoughlin,
Association for the Advancement of Computing in Education, Norfolk, Virginia, USA.
Blended Learning: Beyond Web Page Design for the
Delivery of Content
Philip P Alberts1, Linda A Murray, Darren K Griffin & Julia E Stephenson
1: Learning & Teaching Development Unit, Brunel University, United Kingdom

Phil.Alberts@brunel.ac.uk
Abstract. The growing use of computers in all walks of life has inevitably
raised the question of the efficiency of e-learning in higher education.
Technology has entered university teaching in a variety of ways including
content delivery, resources for additional reading, student interaction and
learning assessment. It is evident that e-learning offers significantly increased
flexibility to the student in terms of time, place, pace and process; however, the
provision of too much flexibility may disadvantage the learning process and
encourage inappropriate learning strategies. In this presentation, we consider
that the uptake of e-learning is inevitable. Moreover, it can offer significant
benefits to both lecturer and learner provided inappropriate practices are
avoided. We propose that the design of the e-learning environment is all-
important, as is the imposition of structures and boundaries for the students. We
conclude that e-learning strategies are likely to find widespread use if take-up
offers significant advantages to lecturers as compared to their current practices.
Introduction
A lecture is the process by which the notes of the lecturer are transferred to the
notes of the student without passing through the mind of either
Attributed to R K Rathbun (Gilstrap and Martin, 1975)
When the wit R K Rathbun first uttered these words, he could scarcely have imagined
that this would become the mantra of many proponents of e-learning in higher
education. The traditional lecture has been the mainstay of teaching in universities
since their inception, but since at least 1931 (Holt, 1931) their efficacy has been
questioned. Thus, before considering the benefits and prospects of e-learning as an
alternative to lectures, it seems appropriate to consider why the traditional lecture has
remained so popular and ask the question whether it is appropriate to consider
replacing it, even partially.
The main benefit of the traditional lecture is that it is efficient; one person can
deliver a pre-determined body of knowledge to a large number of students and at the
end of an hour each of them should at least have a cognitive reference structure for
revision. Second, lectures are flexible in that they are only minimally constrained by
time and location. Of course all the students have to be in one place at a given time,
but this is only limited by timetabling and the availability of a large enough space for
54 Philip P. Alberts et al.
all. Traditional lectures are by their very nature lecturer-centred; the notion of the
lecturer as the “expert” delivering wisdom to the student body is central to this.
Therefore such a system is attractive to the lecturer - he or she maintains significant
control over the student learning experience and can implement learning strategies
accordingly.
In higher education, we therefore have a system for the delivery of a course that is
efficient, flexible, tested and popular amongst lecturers. There should be no reason to
consider a change in delivery, unless change is inevitable or desirable (to both lecturer
and student). What follows is a view supported by literature that change is inevitable.
We argue that there are three good reasons why this change will occur. We then go
on to propose guidelines that lecturers should consider when adapting their pre-
existing course material. In particular, we will consider the increased use of
information technology as a means of delivering lectures i.e. a workable alternative to
traditional lectures. For the sake of clarity, the following definitions will be used:
• Traditional lecturing: A lecturer makes a presentation to a group of students in a
lecture room using visual aids such as the writing board and overhead projector, or
even a digital visual aid such as PowerPoint projection.
• E-learning: Information technology is used as the primary means of delivering
courses and lecturer-student interaction takes place online. Distance learning
courses are often delivered in this way.
• Blended learning: Both traditional lecturing and information technology are used
as the primary means of delivering courses. The lecturer decides on a ‘mix’ of
components from each.
Finally, we indicate what can be done to encourage lecturers to adopt e-learning.
Three reasons why the adoption of e-learning is inevitable
Reason 1: Use of computers and the Internet is widespread and growing in all walks
of life
The introduction of the Apple Macintosh computer in 1984 revolutionised the
computing world. The integration of features such as the Graphical User Interface and
a mouse for navigation to replace the cumbersome commands of MS-DOS was one of
the first stepping stones to enable members of society to interact with computers in
their own home and office environments. Its impact on education, however, was not
immediate due to factors such as high costs (Alessi & Trollip, 2001). Over subsequent
years, information technology diversified rapidly and the presence of computers in
higher education institutions is now commonplace, as are personal computers in many
societies. E-learning has experienced exponential growth in education over the last
few years. The sophistication of software packages has increased and the Internet has
expanded dramatically. All of these factors have provided and will continue to
provide the opportunity for greater involvement of e-learning in higher education;
they have already led to growth in online delivery of learning materials in many fields
(Inglis et al., 2002).
Reason 2: There is an increased demand for higher education world-wide and

increased pressure on lecturers to use technology to cope with it
Many parts of the world are experiencing an increase in the demand for education
and training; this is manifested in universities by larger intakes of students and a
significantly increased proportion of members of the public benefiting from higher
education (Dearing Report 1997; http://www.leeds.ac.uk/educol/ncihe/). In turn, this
leads to increased demands on the lecturer, who on top on his or her dual teaching and
research roles, may have been given insufficient extra time or resources to cope with
the increase in demand. Daniel (1996) refers to this as an ‘educational crisis’ that can
be resolved by the use of information technology. The use of information technology
is already evident in Universities, as visual aids in lectures e.g. PowerPoint (Bartsch,
2003), video-streaming lectures (Shephard, 2003) computer-based learning
programmes (Dewhurst et al., 2000), ‘data mining’ facilities, computer-based
assessment (Thelwall, 2000), realistic simulation-based learning environments (Imber
et al, 2003) and interactive presentation systems such as SMART boards
(www.smarttech.com).
Many lecturers are turning to the Internet as a means of aiding their teaching
practices. The introduction of e-learning environments such as Blackboard
(www.blackboard.com), FirstClass (www.firstclass.com), and Moodle
(http://moodle.org/) has made technical issues involved in electronic delivery less
onerous. There is also evidence that the tools provided by such systems are beneficial
to learning. Romanov & Nevgi (2006) provide evidence that students using
Blackboard (WebCT) with its accompanying tools such as ‘Discussion’ generated
significantly higher assessment results compared to students using a conventional web
site displaying the same course material but without the tools.
A problem for university lecturers is that while many skills gained through
experience of traditional lecturing can translate directly into effective skills for the
design and delivery of online learning, others do not. Although many students are
now “information technology natives” i.e. they have used computers from a very
early age, most of their lecturers are “information technology immigrants” i.e. they
have had to learn the use of computers in adulthood (Prensky, 2001). Thus lecturers
may tend to use lecture notes, overhead transparencies and handouts as a starting
point for the design of e-learning and often find it problematic to convert these into e-
learning resources from which students can learn effectively.
Reason 3. Learning by means of information technology can now provide significant

learning advantages for the student provided appropriate design is employed (four Ps)
The proponents of e-learning as an alternative to the traditional lecture argue that
such an approach provides significant advantages for student learning in terms of
flexibility. For the purposes of this paper, we have dubbed these advantages the ‘four
Ps’ as follows:
• Place: Students have the flexibility to learn in a location of their own choosing
(provided there is an available computer connected to the Internet).
• Pace: Students can learn at the speed suited to them.
• Peace: Students can determine the time in which they learn, choosing moments of
peace and quiet most appropriate for learning.
• Process: Students can choose the means by which they learn, selecting the
learning process most suitable for themselves. For instance, the non-sequential
nature of most e-learning allows students to skip topics they are confident with and
concentrate on unfamiliar topics.
If we take an utopian ideal that “the widest possible choice is best”, it is easy to see
how the above advantages might appear attractive to students and foster learner
independence. Left to their own devices, however, and not constrained by the
structure of a regular routine it may be argued that there is a risk for students that the
advantages afforded by information technology may be misused:
• Students may choose a place of learning that is not conducive to learning
concentration.
• In the absence of face-to-face encouragement and peer pressure and where time
restrictions need to be self-imposed, the pace of learning may slow considerably.
• The freedom to choose the time in which to learn may result in procrastination.
• Given the freedom to choose their own learning process, students may choose one
that is inappropriate for the learning task at hand.
Information technology has now advanced to the point where the potential advantages
of e-learning can be realised. In the following section we put forward
recommendations for ‘good practice’ to facilitate the change from traditional lecturing
to blended learning which maximises the benefits for both student and lecturer. It is
our opinion that a reasonable first step would be the adaptation of existing course
material to a useable electronic format. We therefore focus on realistic “good
practice” that lecturers may follow in order to achieve meaningful blended learning.
Ten good practices in the design of online course material
The challenge for the designer of blended learning is to make the e-learning
environment sufficiently engaging in order to promote the advantages of the ‘four Ps’,
thereby promoting independent learning amongst students.
Clearly, if students are learning via information technology rather than a traditional
lecture, they are excluded from face-to-face human interaction including verbal and
non-verbal communication. The design of e-learning should therefore seek to replace
this by maximising the opportunities for interaction via information technology. Thus
lecturer-centred traditional lectures will at least in part be replaced or supplemented
by student-centred e-learning in a blended learning situation.
In a previous study, Evans et al. (2004) examined the pitfalls of “simply putting
notes on the web” as a means of delivering lecture material. (In fairness, such
educational materials placed online are usually intended to be additional learning
resources.) One possible solution is to use more sophisticated tools (Benyon et al.,
1997). However this may not be practicable for financial reasons. The ten good
practices provided below have been compiled through collating information from
empirical studies. Some are based on Jakob Nielsen’s “Top Ten Mistakes in Web
Design” (http://www.useit.com/alertbox/9605a.html). However, it is recognised here
that studying for a degree in higher education is different from gathering information
by surfing the Internet, which was the focus of Nielsen’s work. Additional
considerations for the use of the web technology in formal learning contexts are
included.
Good practice 1: Specify your learning outcomes and design your assessment
strategies with them in mind
Learning outcomes should be considered as a feature of any learning environment.
However, we suggest that they are particularly relevant in the e-learning environment.
Learning outcomes make electronic content more “three-dimensional” by allowing
students to develop a “mind map” of the material and understand exactly what they
should attain in their learning. Students subsequently do not have to guess the
intentions of the materials and the depth to which they must be mastered. Race (1994)
outlines reasons for specifying learning outcomes for open learners and we propose
that all of these apply to e-learning. Learning outcomes are effective as they inter alia:
• Build self-confidence in students
• Alert the students to the standards that should be attained and the reasons why
learning needs to occur in the chosen subject area
• Alert the students to the challenges that are ahead of them.
McMahon (2006) proposes seven maxims to promote deeper learning as opposed to

surface learning, one of which is the incorporation of suitable learning outcomes.
Empirical evidence suggests that learning outcomes need to be well-designed and that
assessment strategies should be designed with them in mind.
Good practice 2: Bear in mind that the lecturer is not physically available within the
online environment to explain the environment to students; the environment itself
should compensate for this.
Text-based descriptions should be “stand-alone” and self-explanatory. Good
practice here is to undertake ‘user-testing’ with selected students to determine
understanding during the development of online material.
According to Giardiello (2006) it is important to keep PowerPoint presentations
simple and remember “less is more”; furthermore, when converting PowerPoint
presentations to an online format, it is important to bear in mind that the bullet points
were designed to be aides memoire to the verbal presentation. It is possible to add
audio or video recordings / clips to a PowerPoint presentation. Unlike a traditional
lecture, students can view the PowerPoint file as many times as they need to.
Evidence has shown that providing students the opportunity to explore the material in
their own time and pace reinforces the material and promotes active learning (Henly
& Reid, 2001).
Good practice 3: Make use of available multimedia facilities

It is particularly important, especially in a highly visual subject, to exploit the
potential of multimedia in the e-learning environment. The meaning and impact of
the course content can be strengthened by the use of multimedia elements.
Animations, graphics, digital photographs (Swartz, 2004), video clips and sound
recordings can engage the learner and can often be utilised better in an electronic
rather than a paper-based format. Whitson et al., (2006) created online multimedia-
based lecture packets for surgical education using a combination of Macromedia
Breeze (subsequently named Adobe Acrobat Connect Professional) and WebCT Vista.
This study illustrates the powerful software programmes available which allow the
production of high quality multimedia rich environments; PowerPoint can be
synchronised with audio / video clips and also real-time video and audio conferences
can be created.
Presenting information in varied ways provides the learner with an enriched
learning environment and an increased likelihood of maintaining interest. One such
example is the “Intelligent Classroom” project (Winer & Cooperstock, 2002) that
integrated various technologies such as audio, video, slides and digital notes on an
electronic whiteboard / digital tablet during a live lecture which was recorded and
made available for the students to access. The impact was measured qualitatively,
and demonstrated overwhelming support amongst students for the use of this means
of teaching. Another suggestion for using multimedia to maintain interest is reported
in a study by Peachey, Jones and Jones (2006); results showed that the
implementation of online quizzes and games applicable to the course material had the
potential to increase student participation and also improve the learning process.
Good practice 4: Structure material to more than one level of navigation and present
information in “bite-size chunks”.
Chapters in traditional textbooks were written with little structuring over and above
the use of paragraphs. In recent years textbooks have become more visually
interesting and have provided additional structure through the use of more
subheadings, images, textboxes and so on.
Similarly in providing content online, it is important that lecturers avoid lengthy
scrolling web-pages in their e-learning environment. Rather, the learner’s visual
memory should be engaged by introducing a hierarchy of topics and sub-topics. We
suggest that material should have at least two levels of structuring in the hierarchy.
This also assists learners to assess the relative importance of any given part of the
material and enables the material to be placed in context (Evans & Edwards, 1999).
Moreover, constructivist teaching theories suggest that students make mental
connections and construct mind maps more readily when material is presented in this
way (Novak, 1998). Without this, content is presented in a linear, one-dimensional
fashion akin to a traditional textbook.
Moreover, empirical cognitive psychology studies have demonstrated that learners
benefit from material being sub-divided (a practice known as “chunking”: Lynch &
Horton, 1999). The idea of chunking was first suggested by George Miller (1956) in a
study on short-term human memory. Miller concluded that humans are capable of
retaining between five and nine chunks of information. In incorporating the concept
of chunking, scrolling should definitely be avoided (Nielsen, 1995 & 1998).
Furthermore, a study by Lee (2005) showed that students least preferred the method
of scrolling when compared with other computer text formats.
Good practice 5: Provide sufficient navigational information to the student,

particularly about where they are in the structure and from whence they came
Disorientation or being “lost in hyperspace” is a frequent problem associated with

navigation in online teaching and learning materials (Conklin, 1987; Dix et al., 1998).
Links leading to other pages may cause the user to get lost within an intricate
structure and it may be impossible to retrace the learning path. In order to circumvent
this, the e-learning environment should help the learner answer the questions “Where
am I” and “From where did I come?”
A frequently used practice on the Internet is to include “breadcrumbs”, which
indicate the hierarchical steps that have been taken to reach the current page. This
addresses the second question (“From where did I come?”) but not the first (“Where
am I?”) because no indication is given as to where the user is in the hierarchy.
Clear information about position within the context of the e-learning environment
is all-important. A study by Gullikson et al., (1999) considered the effect of
information architecture on an academic web site. The authors recommended
including sufficient navigational aids at all times. Such navigational aids should be
represented as a standard menu on each page and include the ability to:
1. get to the top-level menu from anywhere on the site (including sub-sites)
2. use any of the access tools from anywhere on the website
3. determine easily one’s location within the hierarchy.
Ways to overcome problems of disorientation would be to incorporate navigation
elements such as “next page”, “previous page” and links to the “main page” or “table
of contents” as well as implementing a simple hierarchical structure with navigational
tools such as maps, menus or toolbars. Fortunately, many e-learning environments
provided by commercial vendors (or available as open source software) have these
features built-in.
This good practice is akin to the advice given to lecturers in relation to the
traditional lecture. As well as providing an overview of the structure of the lecture at
the outset, lecturers are encouraged to provide ongoing signposting throughout.
When this is not done, the chance of a learner declaring “I am lost” arises – and in a
traditional lecture students are rarely in a position to retrace their steps.
Online, those features that make the learner safe in the knowledge of where they
are and from whence they came, also provide confidence in addressing the question
“Where can I go next?”
Good practice 6: Provide an e-learning environment that strikes an appropriate

balance between freedom of navigational choice and appropriate constraints
An e-learning environment with too few navigational choices can be likened to the
turning of pages in a book (“electronic page-turning” - Nielsen, 1998). This limits the
choices open to the learner and is regarded as lecturer-centred.
A student visiting online content for the first time may wish to read all of it; they
may however be confident in their knowledge of certain topics and thereby decline
the opportunity to visit the topics. A student revising content will most likely wish to
visit only certain topics.
In either event, the student should easily be able to go from topic to topic. In order
to cater for individual learning requirements, it is appropriate to furnish the learner
with choice in the selection of topics or sub-topics. This can inter alia be achieved by
means of in-built menu options, easily recognisable buttons labelled with words or
pictures, hot spots and hot text (Harding et al., 1995).
However, too much of a good thing brings its own disadvantage. One of the early
celebrated advantages of hypermedia was that it provided complete control to the user
(Nielsen, 1995; 1998). Imagine a scenario where the user is given a page with fifty
links to other topics. It would take some time to determine which link to choose and
reading the descriptions of all the links would be time-consuming. In the process of
learning, this freedom can lead to the learner becoming overwhelmed or even straying
from the e-learning environment. Benyon et al., (1997) describes this in terms of
students becoming ‘bewildered’ and losing the coherence of the material. In subjects
that involve a cumulative build-up of knowledge, it is advisable that a student does
not encounter new material without first having mastered the prior material that is
essential for its understanding.
The assessment tools available in most e-learning environments provide the
opportunity for learners to self-assess their progress and thus make sensible decisions
in relation to the choices available to them. Use of these and the functions for
conditional release of material allows the lecturer to ensure an appropriately
structured learning experience.
Good practice 7: Use hyperlinks only for additional information

One of the disadvantages of hyperlinks is the possibility of disorientation (as
mentioned above). Hyperlinks are unidirectional; this means that the e-learning
environment can show the user the links that have the current node as their departure
point, but not the ones that have it as their arrival point. In other words, the system
will tell you where you can go next, but not in what alternative ways by which you
might have arrived there. In this way, the structure is hidden and is in the mind of the
designer. A partial solution is to open the page in a new window that can be closed
easily, or consider the use of expanding hypertext (a newer hybrid hypertext format).
Expanding hypertext is demonstrated in an empirical study by Lee (2005), placing the
content of the hyperlink in the same page as the hyperlink, thereby decreasing
disorientation as it retains a linear arrangement.
The ability to add external hyperlinks with relative ease is considered to be a
highly attractive feature, thereby taking the student to sites outside the e-learning
environment. However, this can lead to problems because the quality of the external
content may change and the links can become “dead” (Alessi & Trollip, 2001). A
study by Cook & Dupras (2004) identified practical guidelines to develop successful
web-based learning, one of which being “hyperlinks should be tested periodically”.
Good Practice 8: Provide adequate feedback to the learner

Feedback is generally defined as information about the quality of the student’s
response. Interaction between the learner and the lecturer can be instant or delayed
and feedback can be tailored to the individual learner. Feedback should be
constructive and supportive and can be delivered through numerous media such as
text, graphics, audio and video (Alessi & Trollip, 2001). Advice and guidelines on
feedback have been suggested; for example a study on effective feedback by the UK
Higher Education Academy proposes seven principles for good feedback
(http://www.heacademy.ac.uk/806.htm).
Feedback can bring a beneficial influence to the learning environment. For
example, in a study by Thelwall (2000), a statistics test was delivered to students via a
computer-based assessment and it was concluded that the two factors that were
thought to make the test a success was its ease of use and the fact that the 91% of the
students found the feedback helpful. Feedback is a crucial aspect of interactivity in
that it engages learners with the material. It should also be immediate in order to be
effective. The more interactive a system, the more actively the students will learn
(Schwier & Misanchuck, 1993).
Different types of feedback should be implemented depending on the course
content and the time available for the lecturer to formulate the feedback. Some
methods are more time-consuming than others. Examples include personal feedback
by the instructor, model-answers, peer-evaluation provided by the student(s) and
automatic feedback provided by the computer system (Collis et al., 2001).
Good practice 9: Provide some interactions in the e-learning environment and vary
them
E-learning environments provide courseware designers with a great deal of scope
for interactivity. The level of interactivity is a factor that will determine to what
degree the student engages with and remains interested in the environment.
Interactivity can be very simple (Clarke, 2004). The lowest level may be considered
as page turning or scrolling (Oliver, 1996). The highest level however may be
considered as receiving input from the student who is active in the processing of
information, for example interpreting graphs (Dewhurst et al., 2000).
Research has revealed that interactivity can have advantageous effects for the
learner causing their memory skills and understanding to increase to approximately
70% - in contrast to 20% when they only ‘see’, and 40% when they ‘see and hear’
(Lindstrom, 1994).
Other levels of interaction have been identified. Moore (1989) identified three
types of interactions within learning environments: a) interaction between learner and
content, b) interaction between learner and other learners, and c) interaction between
learner and teacher. The ultimate aim is to incorporate all three types of interaction in
a blended learning approach. However, as a first step it is of primary importance to
ensure interaction with the material by requiring some learner responses to questions
and tasks.
It is interesting to note that in the case of the traditional lecture, lecturers are
increasingly challenged to incorporate these types of interaction. With larger class
sizes and more heterogeneous groups, the management of lectures to use other than a
‘transmit-receive’ mode requires the development of a high level of skill.
Good practice 10: Encourage students to put their learning into practice
Interactive multimedia allows the learner to “try something out”, for i.e. by
manipulating a simulation and allowing the possibility to “fail” in a safe environment.
An example of this notion is simulating resuscitation training (Perkins, 2007) as the
lives of patients are not put at risk. In addition, a randomised controlled study by
Tanoue et al. (2005) demonstrated that the implementation of a simulation for the
training of endoscopic surgery was considerably effective to improve skills and
furthermore, such equipment will become increasingly essential because it seems the
use of animals for surgical training will lessen worldwide in the future. A powerful
learning situation is to provide the opportunity for the learner to interact extensively
with the content; a scenario where a response is invited from the user and feedback is
provided. Paper-based teaching materials of course cannot provide such a degree of
interaction.
Perhaps the greatest need of students working independently, is knowledge of how
they are progressing. In a traditional lecture it may be difficult for a student to know
whether they have understood something because they cannot (or they feel they
cannot) interrupt the lecturer. No such barriers exist within an e-learning environment.
Interaction with the lecturer can be achieved using an online communication tool such
as ‘Discussion’ and can also be done effectively by responding to self-assessment
questions (SAQs) - such as multiple-choice, drag and drop, or text-entry (inviting a
response from the learner and providing automated feedback). In this way common
misapprehensions may be addressed.
Race (1994) gives a total of eleven reasons for including SAQs and they are
paraphrased below: They
• give students the opportunity of experiential learning
• confirm to students that which they have already grasped
• help students to discover what they have yet to grasp
• help “nip problems in the bud”
• help students prioritise the importance of aspects of their course or module
• help in the interpretation of the learning outcomes
• provide a means of confidence-building
• give students essential practice in responding to questions
• help alleviate boredom
• help in student orientation in terms of how much they have learned
• help students regulate the pace of their learning
Evans & Fan (2002) provide evidence that learners attach substantial value to the use
of SAQs in multimedia courseware, suggesting that students should benefit from at
least one SAQ for every sub-topic in the material.
The “take-home message” therefore is that the e-learning environment need not be
a “poor relation” of a traditional lecture; it could indeed be superior.
What can be done to encourage lecturers to adopt blended learning?
Although adoption of all or many of a the above ‘good practices’ can provide
significant benefits for student learning, take-up by lecturers can be slow, unless a
clear route to adoption can be found (Lieblein, 2000). We propose that the primary
advantage to lecturers in adoption of blended learning is the reduction in time on their
feet in the lecture venue. University class sizes are increasing, and universities
themselves are expanding. As a result, lecturers can find themselves having to give
their lectures to very large audiences with widely differing learning needs, or having
to give the same lecture more than once because of timetabling, room size or other
campus issues.
We contend that adoption of blended learning should not reduce student-lecturer
interaction; rather it should increase it - in spite of the lecturer and student spending
less time in the lecture venue. E-learning still requires the online presence of the
lecturer. Free from the need to deliver every last part of content before the end of the
lecture, the lecturer has the opportunity to talk with students (not at them) in the form
of a seminar or tutorial – a process that can be far more rewarding.
The switch from traditional lecture to blended strategies requires a significant time
investment on the part of the lecturer, before he or she can experience the benefits.
Lecturers need help in converting what they have already. We contend that existing
teaching resources can be easily converted. Many lecturers already have PowerPoint
presentations or have notes on the Internet.
A second consideration is that there are increasing numbers of well-designed e-
learning resources available from the Internet. “Open source” repositories of learning
objects are being developed very rapidly, but the take-up is slow. Further
encouragement of their use from those charged with supporting e-learning seems to
be needed.
The following are practical suggestions for the lecturer in designing an e-learning
environment:
• Get some help; lobby your computer officer (if you have one) to provide e-learning
support. Many institutions also have central support units devoted to the promotion
of e-learning e.g. by providing infrastructure, mentoring development projects and
providing introductions to colleagues who have successfully made the transition.
• Take a training course in a web-authoring programme (e.g. Dreamweaver ). Such
software is relatively simple to use.
• Don’t re-invent the wheel! Make use of what’s out there already. Many e-learning
resources are available on the Internet or through organisations such as the UK
Higher Education Academy.
• Adopt a procedure for the design of the e-learning environment. The minimum
requirements can be regarded as providing learning outcomes, structure, chunking,
navigational information and some SAQs. Be inventive if you feel like it, but
establish the basics first.
• Don’t abandon the lecture venue or your students - support your e-learning
environment with seminars that might include more in-depth explanation and
discussion of concepts.
• Incorporate some online automated assessment; this can reduce your marking time.
The pace of disciplinary research (measured through the increase in the number of
journal articles and books) increases all the time. The present day lecturer needs to be
a facilitator of learning, not an oracle of all wisdom in the discipline. Adoption of
blended learning is by far the most promising means through which this can be
achieved. The only remaining question is: despite the advantages it offers, how
popular will blended learning strategies be given that so few of us appreciate change?
References
Alessi, S.M. & Trollip, S.R. (2001) Multimedia for learning: methods and development. Boston:
Allyn and Bacon.
Bartsch, R.A. & Cobern, K.M. (2003) Effectiveness of PowerPoint in lectures. Computers &
Education, 41(1), 77-86.
Benyon, D., Stone, D., & Woodroffe, M. (1997) Experience with developing multimedia
courseware for the World Wide Web: the need for better tools and clear pedagogy.
International Journal of Human-Computer Studies, 47, pp.197-218.
Clarke, A. (2004) e-Learning Skills. Hampshire: Palgrave Macmillian.
Conklin, J. (1987) Hypertext: An introduction and survey. IEEE Computer, 20, 17–41.
Collis, B., De Boer, W. & Slotman, K. (2001) Feedback for web-based assignments. Journal of
Computer assisted Learning, 17(3), 306-313.
Cook D.A. & Dupras D.M. (2004) A practical guide to developing effective web-based learning.
Journal of General Internal Medicine, 19(6), 698-707.
Daniel, J. (1996) Mega-Universities and Knowledge Media: Technology Strategies for Higher
Education. London: Kogan Page.
Dearing, R. (1997) The Dearing Report - The National Committee of Inquiry into Higher
Education. London. Retrieved May 3, 2006, from the University of Leeds Web site:
http://www.leeds.ac.uk/educol/ncihe/
Dewhurst, D.G., Macleod, H.A. & Norris, T.A.M. (2000) Independent student learning aided
by computers: an alternative acceptable to lectures? Computers & Education, 35, 223-241.
Dix, A., Finlay, J., Abowd, G. & Beale, R. (1998) Human-Computer Interaction (2nd edition)
Prentice Hall Europe.
Evans, C. & Edwards, M. (1999). Navigational Interface Design for Multimedia Courseware.
Journal of Educational Multimedia and Hypermedia, 8 (2), 151-174.
Evans, C. & Fan, J. (2002) Lifelong learning through the Virtual University. Journal of Campus
Wide Information Systems, Multimedia Systems for Lifelong learning: Success and
Challenges, 19(4), 127-134.
Evans, C., Gibbons, N., Shah, K., Griffin, D.K. (2004) Virtual learning in the biological
sciences – pitfalls of simply “putting notes on the web”. Computers & Education, 43(1-2),
49-61.
Gilstrap, R. & Martin, W. (1975) Current Strategies for Teachers. Pacific Palisades, California,
Goodyear Publishing Company.
Giardiello, F.M. (2006) Powerful PowerPoint presentations. Gastrointestinal Endoscopy, 64(3)
393-394.
Gullikson S., Blades R., Bragdon M., McKibbon S., Sparling M. & Toms E.G. (1999) The
impact of information architecture on academic web site usability. The Electronic Library,
17(5), 293-304.
Harding, R.D., Lay, S.W., Moule, H. & Quinney, D.A. (1995) Multimedia interactive
mathematics courseware: The mathematics experience within the Renaissance Project.
Computers & Education, 24(1), 1-23.
Henly, D.C. & Reid, A.E. (2001) Use of the web to provide learning support for a large
metabolism and nutrition class. Biochemistry and Molecular Biology Education, 29, 229-
233.
Holt. H., 1931, cited in “The College Lecture, Long Derided, May Be Fading” by William
Honan, New York Times, August 14, 2002.
Imber, S., Shapira, G., Gordon, M., Judes, H. & Metzger, Z. (2003) A virtual reality dental
simulator predicts performance in an operative dentistry manikin course. European Journal
of Dental Education, 7(4), 160-163.
Inglis, A., Ling, P., & Joosten, V. (2002) Delivering Digitally: Managing the Transition to the
Knowledge Media (2nd edition) London: Kogan Page.
Lee, M.J. (2005) Expanding hypertext: Does it address disorientation? Depends on
individuals’ adventurousness. Journal of Computer-Mediated Communication, 10(3),
article 6. Retrieved online from http://jcmc.indiana.edu/vol10/issue3/lee.html
Lindstrom, R. (1994) The Business Week Guide to Multimedia Presentations: Create Dynamic
Presentations That Inspire, McGraw-Hill, New York, NY, Ch 2, pp. 33-46.
Lieblein, E. (2000) Critical factors for successful delivery of online programs. The Internet and
Higher Education, 3, 164-174.
Lynch, P.J. & Horton, S. (1999) Web Style Guide: Basic Design Principles for Creating Web
Sites. Yale University Press.
Miller, G. A. (1956) The magical number seven plus or minus two: Some limitations on our
capacity for processing information. Psychological Review, 63, 81-97.
McMahon, T. (2006) Teaching for more effective learning: Seven maxims for practice.
Radiography, 12(1), 34-44.
Moore, M.G. (1989) Editorial: Three types of interaction. American Journal of Distance
Education, 3(2), 1-16.
Nielsen, J. (1995) Multimedia and Hypertext: the Internet and beyond. Boston, London: AP
Professional.
Nielsen, J. (1998) Electronic Books – A Bad Idea http://www.useit.com/alertbox/980726.html
Novak, J. (1998) Learning, Creating, and Using Knowledge: Concept Maps as Facilitative
Tools in Schools and Corporations. Lawrence Erlbaum Associates Publishers, Mahwah, NJ.
Oliver, R. (1996) In C. McBeath and R. Atkinson (Eds), The Learning Superhighway: New
world? New worries? Proceedings of the Third International Interactive Multimedia
Symposium, 303-308. Perth, Western Australia, 21-25 January. Promaco Conventions.
http://www.aset.org.au/confs/iims/1996/lp/oliver.html
Peachey, P., Jones. P., & Jones, A. (2006) Encouraging student participation in an online
course using “pull” initiatives. Electronic Journal of e-Learning, 4(1). Retrieved online
from http://www.ejel.org/volume-4/v4-i1/v4-i1-art-9.htm
Perkins, G.D. (2007) Simulation in resuscitation training. Resuscitation, 73, 202-211.
Prensky, M. (2001) Digital Natives, Digital Immigrants
http://www.marcprensky.com/writing/Prensky%20-
%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf
Race, P. (1994) The Open Learning Handbook. 2nd Ed., London: Kogan Page.
Romanov, K. & Nevgi, A. (2006) Learning outcomes in medical informatics: Comparison of a
WebCT course with ordinary web site material. International Journal of Medical
Informatics, 75(2), 156-62.
Swartz, M.L. (2004) Digital photographs and PowerPoint. American Journal of Orthodontics
and Dentofacial Orthopedics,126(5) 639.
Schwier, R.A. & Misanchuk, E.R. (1993) Interactive Multimedia Instruction. Englewood
Cliffs, New Jersey, Educational Technology Publications.
Shephard, K. (2003) Questioning, promoting and evaluating the use of streaming video to
support student learning. British Journal of Educational Technology, 34(3), 295-308.
Tanoue, K., Yasunaga, T., Konishi, K., Okazaki, K., Ieiri, S., Kawabe, Y., Matsumoto, K.,
Kakeji, Y., & Hashizume, M. (2005) Effectiveness of training for endoscopic surgery using
a simulator with virtual reality: Randomised study. International Congress Series, 1281,
515-520.
Thelwall, M. (2000) Computer-based assessment: a versatile educational tool. Computers &
Education, 34(1), 37-49.
Whitson, B.A., Hoang, C.D., Jie, T., & Maddaus, M.A. (2006) Association for Academic
Surgery, 2006: Technology-Enhanced Interactive Surgical Education. Journal of Surgical
Research, 136, 13-18.
Winer, L.R. & Cooperstock, J. (2002) The “Intelligent Classroom”: Changing teaching and
learning with an evolving technological environment. Computers & Education, 38, 253-266.
Discovery of Educational Objective on
e-Learning Resource: A Competency Approach
Shi-Ming Huang, Hsiang-Yuan Hsueh, and Jing-Shiuan Hua
National Chung Cheng University, ChiaYi, Taiwan

{smhuang, hyhsueh, jshua}@mis.ccu.edu.tw
Abstract. It is indeed important to implement e-Learning platforms for

education and training purposes in a variety of domains. However, facing the
enormous amount of learning resources, guidance on e-Learning platform for
meeting the educational objectives is still a neglected issue. The learning
experiences cannot be optimized and personalized. In this study, a semantic and
systematic mechanism is proposed to discover the educational objectives of
learning resources with competency information on e-Learning platform. To
verify the feasibility of mechanism proposed in this study, the design and
implementation of prototype system for the assessment of e-Learning
effectiveness using the proposed mechanism are also discussed and
demonstrated. By feasibility verification with system implementation for
practical requirement, the effectiveness of the proposed mechanism can be
basically certified.
Keywords: e-Learning; Bloom taxonomy; Competency; O*Net competence

database
1 Introduction
It is indeed for importance of e-Learning implementations and platforms for education

and training purposes in a variety of domains, including educational purposes in
organizations or individuals and human resource development facilities in enterprises.
It is indicated that the notion of e-Learning has widely been adopted as a proper
solution for human resource development in enterprises [14]. The related investment
of IT for e-Learning and web-based training programs is therefore expected to
increase human resources and talents with better quality [6,8].
With the adoption of e-Learning platform, e-Learning mechanisms and strategies
are expected to be dramatically switched from traditional advisor-lead instructions to
learner-oriented paradigms [10]. It emphasizes the interaction among actors on e-
Learning platforms including learners, instructors, developers, and managers. Users'
requirement and behavior played dominant roles during the development and
refinement phases of e-Learning experiences [4,10,16]. However, facing enormous
amount of learning resources on e-Learning platform, guidance on e-Learning
platform for educational objectives is still a neglected issue. The learning experiences
cannot be optimized and personalized, since learners cannot be guided to use learning
Discovery of Educational Objective on e-Learning Resource 67
resources to satisfy their personal learning objectives, particularly the overflowing

learning resources on e-Learning platform. It is easy to retrieve any categories of
learning contents from a computer-based e-Learning platform, but it is still necessary
to look for additional solutions to determine the educational objectives for
optimization. In such assumption, the importance rank of learning resource is needed
to be established in a personalized manner in order to fit learners' requirement.
In this study, the authors argued that additional information for educational
objectives of learning resources should be systematically generated and appended on
e-Learning platform. As for the standard of such additional information, one potential
solution is the competency information specifications proposed as the O*Net
competence database [13], which is the common standard for taxonomy, collection,
description, evaluation, and publication of competency information with the form of
dimensional factors. Several factors including a set of characteristics of employees,
occupational opportunities, skills, and abilities are included in the standard. With an
open descriptive meta-data and language, it is easy to express, identify, and clarify the
competency of job searchers and competency requirements for one job opportunity in
an enterprise with quantitative approaches. Information system solutions for
competency management are also possible to implement with the aim of such
underlying notion and open description language.
In this article, a semantic and systematic mechanism is proposed to discover
educational objectives, which is formulated by well-known Bloom taxonomy for
educational objectives in this study, of learning resources on e-Learning platform.
Strategically, the required educational objectives for competency information
specified in O*Net metadata should first be determined. Educational objectives of
learning resources in e-Learning platform, which reflect a set of competency
information, can then be determined in a systematic and semantic manner. In order to
verify the feasibility of mechanism proposed in this study, the design and
implementation of a prototype system for the assessment of e-Learning effectiveness
using the proposed mechanism are also discussed and demonstrated. By feasibility
verification with system implementation for practical requirement, the effectiveness
of the proposed mechanism can therefore be basically certified.
2 Theoretical Foundations
2.1 Computer-Based Learning Platforms
Information Technology has played an important role in modern learning program,

which is one of key enablers for individually tailored learning experience. As for the
technical e-Learning platform, it is widely accepted that the advent of World Wide Web
dramatically changed traditional computer-based technology. It provided a widely
accessible communication topology based on open and common standards to access
information and knowledge with unified manners [1]. The specification of Shareable
Content Object Reference Model (SCORM) therefore provided a set of referential
guideline to support the development of web-based e-Learning platform and
collaboration of learning resources and contents. With the view of Learning Management
68 Shi-Ming Huang, Hsiang-Yuan Hsueh, and Jing-Shiuan Hua
Systems (LMS), SCORM specification covers technical details in each layer of such
information system. By definitions of technical foundations for web-based learning
environment, it is possible to construct and deploy e-Learning platform that assumed the
presence of strong, server-side, LMS-based learning content distribution [1].
2.2 Competency Information for Characterizing Occupational Profiles
The notion of competency management is important for human resource management

and optimization in enterprises. The term competency can be defined as the
measurable corresponding knowledge, skill, ability, and other behavioral
characteristics (KSAOs), which always come out in reference to a given context, for
the determination of efficiency and effectiveness of certain missions or tasks [7]. It is
therefore applicable to determine distinguished talents in a systematical manner.
Competency management therefore concerns the way in which competencies are
properly organized and controlled [11]. It is responsible for the management of
intangible assets of human resources, such as knowledge, know-how, and behavior,
for competency either required by system or acquired by individuals.
A variety of studies and standards are presented currently. One main category is
the modeling and assessment of competency from the available competency
information with the view of human resource management. For example, Linder
investigated the status of competency assessment and human resource management
performance as an empirical case study for the maturity measurement of competency
information utilization [12]. An important standard of competency information is the
O*Net, the Occupational Information Network, which is a comprehensive database
for the occupational information of employee properties and job characteristics. It
also provides a unified language for defining and describing the corresponding
occupations. Its flexible design also captures various job requirements with the aim of
information technologies [13]. In this article, the authors attempted to apply such
complete competency information schema for expression and further analysis on
competency management processes.
2.3 Bloom Taxonomy for Educational Objectives
The term “educational objective” of learning resources or activities can be referred to

as the educational goal which should be identified during the design phases of
learning or tutoring activities. Educational objectives should also be reviewed after
learning activities in order to determine the effectiveness of learning activities.
The most well-known taxonomy of educational objectives is originated by Bloom
[5]. In the taxonomy, educational objectives can be categorized in six levels including
knowledge, comprehension, application, analysis, synthesis, and evaluation phases
[5,15]. In this study, the authors recommended to formulate educational objectives by
extended Bloom taxonomy of educational objectives, which is the refinement to the
original taxonomy, proposed by Anderson [2]. Rather than one-dimension approach,
the extended taxonomy involves two dimensions, including knowledge dimension and
cognitive dimensions. The cognitive process dimensions can be categorized as:
Remember: Exhibit the memory of previous-learned materials by recognizing

or recalling facts, terms, basic concepts and answers.
Understand: Understanding of facts and ideas by interpreting, exemplifying,
classifying, summarizing, inferring, comparing, and explaining main ideas.
Apply: Using the available knowledge to execute and implement solutions in
different ways.
Analyze: Differentiating, organizing, and attributing knowledge by
manipulating information using certain criteria.
Evaluate: Checking and Judgments about information, validity of proposed
ideas, or quality of work by certain criteria.
Create: Generating, planning, and producing information or knowledge
together and proposing new solutions.
On the other hand, the knowledge dimension can be categorized as:
Factual Knowledge: Knowledge about terminology and specification details.
Conceptual Knowledge: Knowledge about generation, classification, and
structural modeling of certain concept.
Procedural Knowledge: Knowledge about workflows, algorithms, methods,
procedures, and events.
Meta-Cognitive Knowledge: Knowledge about strategies and decisional
conditions.
Strategically, the authors attempted to apply taxonomy as a tool to discover the
characteristic of competency information and learning resources with the form of
location in the taxonomy matrix. For example, the learning resources about
“Normalization forms in relational data models” can be inferred as (Apply,
Procedural). That is, its educational objective is to provide procedural knowledge to
learners so that they can apply learned algorithms to improve the quality of database
design.
3 Discovery of Educational Objectives using Bloom Taxonomy
In this study, feature mapping involves the discovery of potential keyword from
competency information specified in O*Net metadata and identification of
educational objective for learning resources inferring from competency information.
Figure 1 illustrates the mechanism.
3.1 Specification of Competency Information in Competency Reservoir
In this study, the competency reservoir is required to store and retrieve competency
information. According to competency definition from O*Net, training program or
occupation requirement can be measured by 17 dimensions. In the simulation
experience, with the view of learning facilities, 4 dimensions (tools, technologies,
knowledge, and skills required by training programs or occupation requirement) are
selected, as shown in Figure 2. In specifying O*Net metadata, the competency
information can be categorized by occupational requirement. For example, the
competency information of Computer Programmer (No. 15-1021.00) defined in

O*Net specification can be expressed by the following metadata.
Table 1. Partial competency information for sample occupational requirement.

Tools required Computer Servers, Desktop Computers, and Mainframe Computers,
etc.
Technologies required Analytical or Scientific Software, Application Server Software, and
Compiler and de-compiler software, etc.
Knowledge required Computers and Electronics, English Language, and Mathematics, etc.
Skills required Programming, Critical Thinking, and Complex Problem Solving, etc.
Fig. 1. Feature discovery and mapping on competency information.
Fig. 2. Taxonomy of competency information.
3.2 Specification of Educational Objectives with Bloom Taxonomy
On the other hand, the educational objectives of learning resources, which should be
discovered by extended Bloom taxonomy, can be systematically identified by
synonym thesaurus is provided in this study. Table 2 shows partial contents of
synonym thesaurus for cognitive process dimensions [2].
Table 2. Synonym thesaurus to discover educational objectives.

Cognitive Process Dimension Partial Potential Synonyms (3 synonyms are selected)
Remember Recognizing, Recalling, Retrieving
Understand Interpreting, Clarifying, Representing
Apply Executing, Implementing, Carrying out
Analyze Distinguishing, Discriminating, Organizing
Evaluate Critiquing, Testing, Monitoring
Create Planning, Designing, Constructing
On the other hand, synonym of knowledge dimensions should be manually defined,
since knowledge is domain specific. Sample contents of synonym thesaurus used for
Computer Programmers are shown in Table 3.
Table 3. Synonym thesaurus to discover educational objectives.

Knowledge Dimension Partial Potential Synonyms (3 synonyms are selected)
Factual Computers, Servers, Mail
Conceptual Markup Languages, Data Models, Programming Languages
Procedural Problem-Solving, Decision-Making, Troubleshooting
Meta-Cognitive Deductive Reasoning, Active Learning, Active Listening
3.3 Semantic Identification
The first step of educational objectives discovery is the potential semantic

identification from competence information, which is responsible to extract potential
keywords which have more likelihood to reflect the educational objective defined in
Bloom taxonomy. For example, considering the skill competency information defined
in O*Net metadata for Computer Programmers: {Active Learning — Understanding
the implications of new information for both current and future problem-solving and
decision-making.}, potential keywords reflecting cognitive processes include
Potential_ Cognitive_Keywords = {Learning, Understanding, Problem-Solving,
Decision-Making}. On the other hand, in the sentence, a set of potential knowledge
keyword Potential_Knowledge_Keyowrds = {Learning, Understanding, Problem-
Solving, Decision-Making, Implementation, Information} can also be extracted.
3.4 Feature Discovery
The second step of educational objectives discovery is the feature discovery by

filtering the potential keyword lists using predefined synonym thesaurus. For example,
the keyword lists generated in the last section can be pruned as Cognitive_Keywords
= {Understanding} and Knowledge_Keyowrds = {Problem-Solving, Decision-
Making}.
3.5 Semantic Mapping
The last step of educational objectives discovery involves the mapping of competency
information into educational objectives with Bloom taxonomy. In the example
discussed in previous sections, it can be interpreted that the skill competency
information “Active Learning” involves the “understanding” of “procedural
knowledge” including “Problem-Solving and Decision-Making”. It should be noticed
that for any competence information, it is possible to reflect a set of educational
objectives. For example, the competency information “Active Learning” also involves
the interpretation for “understanding” of “meta-cognitive knowledge” about
“implication”. The following Figure 3 shows the corresponding matrix of the sample
skill competency information “Active Learning” and “Complex Problem Solving”.
Fig. 3. Interpretation of educational objective for competence information.
3.6 Discovery of Educational Objective on e-Learning Resource
As the educational objectives of competency information are discovered, the

educational objective of learning resources reflected a set of competencies can
therefore be identified. For example, learning resource {R: Understanding Case
Based Reasoning} reflects a set of competency information defined in O*Net
metadata including {“Complex Problem Solving”, “Active Learning”}, the
educational objectives can be inferred as shown in the following Figure 4.
Fig. 4. Interpretation of educational objective for learning resource.
4 Application: Assessment of e-Learning Effectiveness
In this section, the authors attempted to discuss the practical application applying the
proposed mechanism to evaluate the effectiveness of e-Learning on users using
competency information and Bloom taxonomy in order to demonstrate the feasibility

and practicality of proposed mechanism. The application of the proposed mechanism
for the assessment of e-Learning effectiveness is based on the belief that:
Effectiveness of users’ e-Learning experience can be measured by the status of
educational objectives owned by learners.
Users attempted to access learning resources for certain learning purposes
which can be reflected by occupational requirements.
Assessment of Learning experiences involves quizzing with questions reflected
a set of competency information.
Quiz, which reflects a set of competency requirements can also measure the
status of educational objectives owned by learners.
Figure 5 illustrates such belief.
Fig. 5. Belief of proposed mechanism applying on assessment of e-Learning experience.
4.1 Assessment of e-Learning Effectiveness with Bloom Taxonomy
In this study, the assessment of e-Learning effectiveness can be performed as in the

following. A quiz is given with a set of questions randomly generated by Sequential
Probability Ratio Test (SPRT) model [9]. Each question can be reflected by a set of
competency information, and each competency can be reflected by a set of cognitive processes.
That is, each question ( qi ) can be properly reflected by a set of cognitive processes as:
(1)
q i = C C iT , ∀ i ∈ N
T
, where C iT denotes the presence of cognitive process T on qi . For example, a

question of a quiz may be reflected by a set of cognitive processes including “Apply,
Analyze” of “Conceptual knowledge” for the competency “Complex Problem Solving”.
The mastery of a quiz reflected user-specified occupational requirement can be

evaluated by the union of score of each cognitive process dimension that user
obtained in the quiz:
S T = C G ( qi ) × P ( qi ) (2)
qi ∈T
, where the function G ( qi ) means the score of question i gained by a testee and the
function P ( qi ) denotes the importance of question i . The effectiveness of e-Learning
can be measured by the mastery matrix with cognitive process dimensions, as shown in
Table 4.
Table 4. Measurement of e-Learning effectiveness by Bloom cognitive taxonomy.

Remember Understand Apply Analyze Evaluate Create
S t = ' REMEMBER ' S t = 'UNDERSTAND ' S t = ' APPLY ' S t = ' ANALYZE ' S t = ' EVALUATE ' S t = 'CREATE '
For instance, the mastery of a quiz reflected a testee’ occupational requirement,
Computer Programmer, is measured. Given a quiz with three questions randomly
generated by SPRT models for testing the mastery of the testee. The gained score,
allotted score, importance, and reflected cognitive processes of each question are
displayed in Table 5. The gained score which presented mastery for cognitive
processes are shown in Table 6. It can be inferred from Table 6 that the effectiveness
of learning on the testee is poor in the case illustration.
Table 5. The related information of each question of the quiz in the case illustration.
Question Gained Allotted
Importance Reflected cognitive processes
number score score
“Apply, Analyze” of “Conceptual knowledge” for
1 7 10 5
the competency “Complex Problem Solving”
“Apply” of “Factual knowledge” for the
2 0 15 10
competency “Analytic and scientific software”
“Analyze, Create” of “Procedural knowledge” for
3 0 10 5
the competency “Programming”
Table 6. Summarized the estimated mastery of cognitive processes in the case illustration.
Cognitive processes dimension
Gained score / Full marks
Remember Understand Apply Analyze Evaluate Create
Question 1 35/50 35/50
Question 2 0/150
Question 3 0/50 0/50
Total N/A N/A 35/200 35/100 N/A 0/50
4.2 Prototype System Architecture
In this study, the authors have developed a web-based prototype system for
assessment of e-Learning effectiveness using the proposed mechanism. Figure 6
shows the architecture.
Fig. 6. Architecture of the prototype system.
The prototype, which was developed for “Project of Personnel Training of Electronic
Business” executed by the Ministry of Economic Affairs in Taiwan, has been deployed
on Microsoft Windows server platform with Microsoft SQL Server and Internet
Information Service (IIS). Some important features of the implementation include:
Feature Mapping Module: This is the core module that implements the
proposed mechanism. It is responsible for the systematic and semi-automatic
discovery educational objectives of quizzes and competency information.
User Interface: The interface is used for retrieving users’ learning requirements
with the form of occupational requirements defined by O*Net metadata.
Quizzing Module and Evaluation Module: These two modules are responsible
for assessing the e-Learning effectiveness of users by applying SPRT based on
user-specified occupational requirement.
Figure 7 shows the screenshot of sample questions in the user interface generated
by the quizzing module. Figure 8 shows the screenshot of mastery matrix with
cognitive process dimensions after quizzing. Practically in the prototype system, the
matrix can be visualized by the radar diagram for better interpretation by users. It
should be noted that it is not necessary for all cognitive process dimensions to appear
in the radar diagram. Dimensions involved are highly depended on the nature of
questions in the quiz.
To test the preliminary feasibility of the proposed mechanism embedded in the

prototype system, an internal test is performed by surveying twenty-two internal users
and adopting T-test for statistical inference. The calculated t value is 8.57 >
t 0.95 (21) = 1.721 under significance level α = 0.05, and therefore information
satisfaction of users is significant. It means that the proposed mechanism in this paper
is potential.
Fig. 7. Screenshot of sample questions in user interface.
Fig. 8. Visualized result of e-Learning effectiveness by Bloom cognitive taxonomy.
5 Conclusion Remarks
In this article, a semantic and systematic mechanism is proposed to discover the

educational objectives, which is formulated by well-known extended Bloom
taxonomy, of learning resources on e-Learning platform. In the first phase, the

required educational objectives for competency information specified in O*Net
metadata should be determined. The educational objectives of competency
information can be visualized with a two-dimensional matrix. Educational objectives
of learning resources in e-Learning platform, which reflect a set of competency
information, can therefore be inferred in a systematic and semantic manner. In order
to verify the feasibility of mechanism proposed in this study, the authors selected the
assessment of e-Learning effectiveness as potential practical application using the
proposed mechanism. A prototype system is designed and implemented for
assessment of e-Learning effectiveness using the proposed mechanism in order to
demonstrate the feasibility of the application using the proposed mechanism. By
feasibility verification with system implementation for practical requirement, the
effectiveness of the proposed mechanism can therefore be basically certified.
As for future research with respect to the mechanisms of educational objective
discovery and management, the authors recommended that the proposed mechanism
could be widely adopted in individual or distributed learning facilities on e-Learning
platform. With automatic discovery or retrieval of learning resources with user-
specified educational objectives or competency requirements, it is possible for
educational objectives as a behavioral guidance of learning resources to satisfy users
learning requirement, since the characteristics and profiles of learning resource can be
predicted by information provided by proposed mechanism.
Acknowledgments. The work presented in this paper has been supported by National
Science Council, Taiwan, R.O.C, under Grant No. 95-2524-S-194-004-. The authors
deeply appreciate their financial support and encouragement.
References
1. ADL: SCORM 2004 2nd Edition Overview [On-line]. Available: http://www.adlnet.org/

downloads/70.cfm (2004)
2. Anderson, W., Krathwohl, D.: A taxonomy for learning, teaching, and assessing: A revision
of Bloom’s Educational Objectives. Longman, New York (2001)
3. Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley Press.
(1999)
4. Baltes, C.: The e-Learning Balancing Act: Training and Education with Multimedia. IEEE
Multimedia, Vol.8, Issue 4 (2001) 16-19
5. Bloom, B.: Taxonomy of Educational Objectives: The Classification of Educational Goals.
Susan Fauer Company, Inc. (1956)
6. Brennan, M.: U.S. Corporate and Government e-Learning Forecast 2002-2007. IDC. (2003)
7. Catano, V.: Appendix 1: Competencies: A Review of the Literature and Bibliography [On-
line]. Available: http://www.cchra-ccarh.ca/en/phaseIreport/appendix.asp (1998)
8. Chang, K., Sung, Y., Lee, C.: Web-based collaborative inquiry learning. Journal of
Computer Assisted Learning, Vol. 19, Issue 1 (2003) 56-69
9. Ferguson, R.: Computer-Assisted Criterion-Referenced Measurement (Report WP-41).
Pittsburgh University, Learning Research and Development Center, Pittsburgh (1969)
10. Friedman, R., Deek, F.: Innovation and Education in the Digital Age: Reconciling the Roles
of Prdagogy, Technology, and the Business of Learning. IEEE Transactions on Engineering
Management, Vol. 50, Issue 4 (2003) 403-412
11. Harzallah, M., Berio, G., Vernadat, F.: Analysis and Modeling of Individual Competencies:
Toward Better Management of Human Resources. IEEE Transactions on Systems, Man,
and Cybernetics, Part A, Vol. 6, Issue 1 (2006) 187-207
12. Linder, J.: Competency Assessment and Human Resource Management Performance of
County Extension Chairs in Ohio. Journal of Agricultural Education, Vol.42, Issue 4 (2001)
21-31
Blended Learning: Experiences of Adult Learners in
Hong Kong
P. W. R. Lee and F. T. Chan
HKU SPACE, The University of Hong Kong

Pokfulam Road, Hong Kong, China
{ruby.lee, ft.chan}@hkuspace.hku.hk
Abstract. The development in information and communication technology

brings along many impacts to education and training. Though delivery of
courses solely based on e-learning on its own is not totally satisfactory to
students and course providers, distance learning and face-to-face courses
benefit from the use e-learning as a support in the teaching and learning
process. Surveys of Hong Kong part-time students clearly indicated that they
preferred a blended learning mode that retains some form of face-to-face
teaching and utilizes e-learning at the same time. However, effective blended
learning is not simply using the technology as an additional communication
means or to organize extra learning activities. The pedagogy has to be adjusted
to incorporate e-learning as part of the teaching and learning process.
Key words: blended learning, adult learners, distance learning
1. Introduction
The advancement in information and communication technology (ICT) brings a lot

of hope to education institutes, especially those providing distance education. A wide
range of e-learning tools, such as computer-mediated education software, online
course materials, and online forum have been developed. Some institutes make use of
the technologies to enhance their delivery of distance education courses. Some go for
new ventures in offering courses solely using the electronic platform. However, these
purely e-learning courses are largely not successful in delivering learning experience
to the satisfaction of the students and in achieving cost effectiveness to the
satisfaction of the institutes.
The potential benefits of e-learning nevertheless flourish when used together with
existing models of course delivery. This article reports the experiences and research
findings of the authors in the last 12 years. A comprehensive discussion on the
development of blended learning and utilizing e-learning to enhance teaching and
learning effectiveness can be found in Macdonald (2006) and Naidu (2003).
80 P. W. R. Lee and F. T. Chan
In the following sections, the development of e-learning activities introduced into

distance education courses experienced by the authors is reviewed; the trend of
students’ preferences on delivery modes is analyzed based on surveys conducted in
various years; and a case study of comparing two blended learning approaches is
discussed. Lastly, the Conclusion section summarizes the major findings about
blended learning based on the surveys results and the case study.
2. Stages of Blended Learning
Since 1990, the School of Professional and Continuing Education, The University
of Hong Kong (HKU SPACE) collaborated with Charles Sturt University (CSU) in
Australia their first joint course using a distance education delivery mode. Students
were largely working adults and studied part-time. For each subject, the adult learners
were given a set of distance learning package that consisted of a Subject Outline, a
Study Guide, and a collection of required Readings. The Subject Outline stated the
objectives and overview of the subject, information of the Subject Coordinator, the
proposed self-learning schedule, the assessment items, the marking criteria, and the
list of reference materials. About 8 sessions of face-to-face teaching/tutorial sessions
of 2 to 3 hours each, depending on the subject, were organized by HKU SPACE using
a local teacher to support the students. These sessions were scheduled after office
hours or during the weekends to match the availability of the students.
Students could communicate with CSU Subject Coordinator through email. At that
time, email was far less popular than today. Most students have not used email before
they joined the course. They needed to use a modem to dial in to a computer server
operated by the University Computer Centre. Many did not have computers at home
and few had modems. Before 2000, less than 50% of the households in Hong Kong
had one or more PCs at home. Out of these households with PC, about 73% had
access to the Internet (Census and Statistical Department website). The local Course
Coordinator acted as the communication bridge between students in Hong Kong and
the CSU Subject Coordinator in Australia.
Since 1994, CSU started to supplement the printed learning package with a
computer disk. Hyperlinks and computer animations of flowcharts and computer
programs were added to increase the interactions between students and the learning
materials. Students generally felt that printer materials had the advantage of physical
portability. They could bring along the printed materials and read them during
traveling or waiting at some places. Electronic materials simply mirroring the printed
version was not welcomed. Additional features such as hyperlinks to relevant
resources, computer animations, and multimedia presentations were needed to justify
the development and production of electronic materials (Messing and Chan, 1999).
Starting from 1999, a more comprehensive online support services were offered in
the form of a Subject Web Page. In addition to CSU, Monash University (Monash) in
Australia also jointly offered degree programs with HKU SPACE since 2002 using a
web-enhanced distance education delivery mode, which supplemented by face-to-face

teaching by local teachers in Hong Kong. For each subject, the set of distance
learning materials was put online. In addition, a Forum was set up to enhance
communication between the overseas University subject teacher and the students from
different places. The Forum also facilitates discussions among the students
themselves. Students from different places used the Forum as a platform to share their
learning experiences. However, connection to the Internet in Hong Kong was largely
through modems. Students needed to occupy a phone line at home and suffered from
annoying unexpected disconnections.
Recently more Hong Kong people had access to Internet at home. In 2003 the
percentage of households that had PCs at home increased to 67.5% (more than 30%
increase in three years) and out of these households 88.8% had access to the Internet.
Effectively, the percentage of households that had access to Internet at home
increased from 36.5% to almost 60%. An increase of 64% of households in Hong
Kong had Internet access at home in the three years from 2000 to 2003. The latest
survey in 2006 revealed that 71.7% of households had PCs at home and 93.6% of
them had access to Internet. The percentage of households that had both PCs and
Internet access rose to 67% (Census and Statistical Department website). The
advancement in information and communication technologies supported the
development of blended learning. Yet, to the students, there were learning elements of
face-to-face sessions that could not be totally replaced by online learning.
3. Student Surveys on Preferred Learning Modes
In order to understand learners’ preferences of various learning modes, surveys of

students studied in the blended learning programs described above were conducted
from 2000 to 2005. The first questionnaire survey was carried out in 2000 (Chan and
Messing, Chan and Mills, 2000) with two groups of students. One group of 24
students studied a graduate diploma in library and the other group of 21 students
studies a Master of Information Technology. Both courses utilized a blended learning
mode that some subjects were taught face-to-face and some subjects were based on
distance learning but with some supporting face-to-face tutorials. These students
studied part-time while working full-time. They were asked to select the teaching
mode they preferred most from the following options, assuming they were given a
choice:
- face-to-face teaching for all the subjects (FtF)
- some face-to-face taught subjects and some distance learning subjects with
supporting tutorials (BL1)
- distance learning with face-to-face tutorials (BL2)
- pure distance learning mode (DL)
The survey findings revealed that almost all students rejected the pure distance
learning mode. Yet, not too many of them preferred a purely face-to-face teaching
mode, probably due to the fact that they were working full-time. The majority wanted
to have the benefits of both world, having face-to-face sessions and distance learning
at the same time. A summary of the findings is given in Table 1. IT students did not
necessarily favored distance learning supported by technology when compared with
the library students.
Table 1. The Most Preferred Teaching Mode – Students’ Perspective in 2000
Learning Mode IT Students Library Students Combined

n=21 n=24 n=45
FtF 9% 21% 15.6%
BL1 67% 50% 57.8%
BL2 24% 25% 24.4%
DL 0% 4% 2.2%
Similar surveys on the learning experiences of part-time postgraduate students

taking distance learning courses with blended learning were conducted in 2002 and
2003. In February 2002, a survey on the distance learning experience of the two
Master’s degree programs jointly offered by HKU SPACE and Australian
Universities was conducted. 58 successfully completed questionnaires were received
for the February 2002 survey (Lee and Chan, 2002). During December 2002 and
January 2003, students from a diploma course, an undergraduate program and five
postgraduate programs that were jointly operated by HKU SPACE and universities
from Australia and United Kingdom were invited to participate in the 2002-2003
survey. In the 2002-2003 survey, the full-time face-to-face study was refined to
include both part-time and full-time study (Lee, Dooley and Chan, 2003). In early
2003, 130 returns were received. The results of all the surveys on thee most preferred
learning mode are summarized in Table 2.
Table 2. The Most Preferred Teaching Mode – Students’ Perspective in 2000 - 2003
Learning Mode 2002-03 Feb 2002 2000

n=130 n=58 n=45
FtF (part-time) 15% 11% 15.6%
FtF (full-time) 16% N/A N/A
BL1 52% 54% 57.8%
BL2 15% 35% 24.4%
DL 2% 0% 2.2%
The latest survey was conducted in late 2005 with an extended scale (Lee, Dooley
and Chan, 2006). More programs and recent graduates as well as active students from
four undergraduate program and four postgraduate programs were included. 274
successfully completed questionnaires were received out of 1,357 distributions. The
response rate was about 20%. As e-learning was much more developed and became
so pervasive, the preferred teaching modes were redefined and the survey results on
most preferred learning modes are shown in Table 3.
Table 3. The Most Preferred Teaching Mode – Students’ Perspective in 2005
Learning Mode 2005

n=274
FtF supplemented with e-Learning 42%
BL1 22%
BL2 27%
Total online learning with some FtF support 6%
Total online learning with no FtF 3%
The findings of the 2005 survey about the preferred learning mode are similar to
the previous survey results of 2000, 2002, and 2003. Blended learning modes are
highly preferred by Hong Kong’s adult learners. The results of the 2000-2005 surveys
indicated that blended learning modes with more face-to-face elements were more
welcomed by the respondents. Web-enhanced distance learning with no face-to-face
elements were not preferred by students respondents from 2000 to 2005 as only 2% to
3% of them preferred such learning mode. E-learning was simply viewed as a
supplement in the blended mode. The results of these surveys confirm the
predominant acceptance of the mixed delivery modes over pure distance learning and
online learning. This further indicated that Hong Kong adult learners perceived face-
to-face sessions as highly valuable.
The findings of these surveys lead the authors to think what blended learning
strategies can better integrate e-learning tools with face-to-face sessions to achieve
higher teaching and learning effectiveness, especially for the part-time adult learners.
The main features of the blended learning strategy that we have been adopting so far
are as follows:
- Distance learning package available online and in printed form
- Face-to-face sessions at regular internals throughout the semester
- Deployed asynchronous online environment (email, forum, subject webpage) for
communication, discussion and access to resources
The number of face-to-face sessions ranges from 6 to 10 for a semester-long

subject. Each session lasts for 2 to 3 hours. The actual arrangement will depend on the
nature of the subject and the level of the program. For example, more sessions will be
organized for a bachelor degree program than a postgraduate program. Longer hours
per session will be organized for a subject involving practical components. During the
face-to-face sessions, the local teacher can choose to do one or more of the following
activities:
- teach the more challenging topics in the subject
- discuss questions raised by students,
- discuss the assessment items
These face-to-face sessions also serve as checkpoints to keep students’ pace of

progress and allow for peer-sharing and support.
The key questions are how to integrate face-to-face sessions and e-learning to achieve
the greatest synergy. For the good practice in blended learning, Macdonald (2006)
stated that “if there is currently a recipe for a blended strategy, it is a broth of
pedagogy, heavily peppered with pragmatism”. In the next section, we describe a case
study, comparing two blended learning approaches in an education program.
4. Case Study of Two Blended Learning Approaches
In 2005-2006, the delivery approaches of two subjects in an Education Diploma

program were examined. The two subjects were taken by two groups of 20 students.
Both subjects were supported by the same e-learning platform that supported the
following functions online:
Announcement – releasing of announcements related to the subject or
program
Resources – presentation slides, handouts, and other reference materials
Forum – discussion forum for students to post views and questions as well
as responses to other students’ submitted views and questions.
For Subject A, students were encouraged to put up interesting teaching incidents

on the Forum for sharing and discussions. Some in-class teaching activities had to put
up to the Forum for the teacher and fellow students to read before the class. Fellow
students could also post responses to the items posted. These postings, however,
would not count as formal assessment items that contributed to scores affecting the
final grades of the students. The posting was entirely voluntary. The number of
postings of each lesson ranges from 6 to 37. The average number of postings per
lesson was 14.7. The average number of postings per lesson per student was 0.74.
For Subject B, students were divided into three groups. Students received a set of
pre-class reading materials for the next weekly class meeting. Each group was
assigned to work on an activity as stated in the materials. Each student was required
to post his/her views or proposed solutions in the Forum. Students were encouraged to
post responses or follow up questions to items posted by fellow classmates. Each
activity provided a scenario for the students to analyze, they were then asked to
prepare their responses to some questions. Typical questions, for example, read
something like:
What do you think about these views? Which one do you like more?
How do these teachers see motivation differently? Or can you
integrate these different approaches to motivate the class to learn?
During the class meetings, members of each group would discuss their postings
and any follow up postings. They formulated some concluding findings and presented
these findings in the class. A summary report of the findings was then posted back to
the Forum. As part of the assessment, the timely submission of the postings in the
Forum and the presentations in class meetings contributed up to 20% of the final
score. In addition, each student was required to compile the set of their own postings
in the Forum together with their responses to other students’ postings as part of the
final assignment for submission to the teacher at the end of the teaching term.
The number of postings of each lesson for Subject B ranges from 30 to 50. The
average number of postings per lesson was 36.8. The average number of postings per
lesson per student was 1.84. This is 1.5 times more than Subject A.
It is not only the quantity that implies Subject B has achieved a more effective use
of the online Forum than Subject A. Students learnt more effectively in Subject B in
several ways due to the design of the blended learning. In each class meeting, students
were required to follow up on what they have prepared by reading the reading
materials and posting on the Forum. Without such pre-class learning, they would not
be able to work effectively the in-class activity. It is by design the integration of the
face-to-face session and the use of the online Forum outside class that student
learning was built upon. The grouping of students helps not only achieving
collaborative learning but also exerting positive group pressure for each group
member to submit his/her own posting on time. Otherwise, other members cannot
read and prepare for the in-class discussion.
It is also by design the weekly postings not only contribute to part of the final score
but also form an essential part of the final assignment submission. The importance of
using the online Forum is appropriately reflected in its weighting among assessment
items.
5. Conclusion
Learning at a distance from the campus was not a new thing. With the development
of ICT, e-learning has firmly established its importance in education and training
courses, no matter these courses are conducted in conventional face-to-face or
through distance learning. In our studies, we find that throughout the last decade most
students in Hong Kong studying part-time postgraduate and undergraduate programs
indicated their preferences in retaining some form of face-to-face teaching while at
the same time utilizing the advantages of e-learning. A pure form of delivery mode,
whether it is face-to-face teaching or distance learning, is not appealing to the adult
learners.
Cheng (1998) commented that regarding students in Asian culture, it was uncertain
whether they prefer to study at home and communicate electronically with their
teachers.
“Students in Asian culture are also not used to expressing themselves
and exchanging views. They are more used to listening, keeping
analyses in their minds, and express themselves only when it is very
necessary. As such it remains to be seen whether the extension of the
physical classroom to the cyberspace would further discourage or

encourage interaction among students.” (Cheng 1998 p204)
Through the 2000- 2005 surveys and the case study, it is concluded that Hong
Kong students take a pragmatic approach towards e-learning. They can be active
learners in the cyberspace if a proper pedagogy is adopted. To make blended learning
more effective, it is more than introducing the technology component in the teaching
and learning process. The right teaching approach and assessment strategies have to
be employed. For example, it is evidenced that participation in online forum
discussion becomes more active and fruitful when such activity is designed as an
integral part of class teaching and contributing to the assessment.
As mentioned by Macdonald (2006 p.54), different parts of a blended strategy were

inter-related and there was much to learn about the ways to integrate e-learning with
face-to-face support. With the high access rate to Internet at home nowadays, it is
now more practical for Hong Kong teachers to explore how they can deploy online
tools with their class teaching.
References
Census and Statistical Department, HKSAR Government website (March 2007):

http://www.censtatd.gov.hk/hong_kong_statistics/statistics_by_subject/
Chan, F. T. and Messing, J. (2000) "A Joint Venture in Distance Education Program between
Hong Kong and Australia", IVETA Conference 2000
Chan, F. T. and Mills, J. (2000) "Collaboration for Success in Open and Distance Education: A
Case Study of Australia and Hong Kong", ‘Distance Education, an open question?’
Conference 2000
Cheng, Kai-Ming (1998) "Institutional collaboration in higher education: challenges of the

information era", in Ronnie Carr (edn 1999), The Asian Distance Learner, p196-206, Open
University of Hong Kong Press: Hong Kong, 1999.
Lee, P. W. R. and Chan, F. T. (2002) "Mixed Mode of Delivery - An Effective Collaboration

Model", ASAIHL 2002 Lifelong Learning Conference, Proceedings of ASAIHL 2002:
Lifelong Learning, 17-19 June 2002, Nanyang Technological University, Singapore, 155-
163.
Lee, P. W. R., Dooley, L. and Chan, F. T. (2003) "Enhancing Adult Learning via E-Learning:
the Perspectives of Students and Teachers in Hong Kong", VIEWDET 2003 Vienna
International Conference on eLearning, eMedicine, eSupport, 26 – 28 November, 2003,
Vienna University of Technology, Austria.
Lee P.W. R., Dooley L.S and Chan, F.T., (2006) "Effective E-Learning: Perspective of Adult
Learners in Hong Kong” , HKITEC 2006: Hong Kong International IT in Education
Conference – “Capacity Building for Learning through IT", Proceedings of HKITEC 2006,
6 – 8 February 2006, EMB, Hong Kong, pp. 66-75.
Macdonald, J. (2006) Blended Learning and Online Tutoring: A Good Practice Guide, Gower,
Hampshire
Messing, J. and Chan, F.T. (1999) "Hands Across the Ocean: Using Information Technology in
a Distance Education Course", Proceedings of the Fifth Hong Kong Web, F. Castro, R. Lai,
and Sr. M. Wong (ed.), pp. 227-241.
Naidu, S. (ed.) (2003) Learning and Teaching with Technology: Principles and Practices,
Kogan Page, London.
An Observational Study on Blended Learning for
Japanese Language Studies in a Local University in
Hong Kong
Kenneth K.C. Lee1, Melody P.M. Chong 2

1
Department of Computer Science, City University of Hong Kong
2
Department of Chinese, Translation and Linguistics, City University of Hong Kong
{kenkclee, ctchong}@cityu.edu.hk
Abstract. This paper discusses how educators and learners can be benefited
from blended learning. The research employed a qualitative research method.
An unstructured observational approach was used to assess how the current
teaching activities can be improved efficiently and effectively with blended
learning. Four dimensions of language learning in relation to vocabulary,
listening, grammar and oral skills are addressed. The results reveal that
vocabulary and listening skills of students can be improved by offering more
input with web-based learning whereas grammar and oral skills are suggested to
maintain more face-to-face interactions in the classroom between instructor and
students. Implications for effective learning and recommendations for future
research are addressed.
Keywords: blended learning, language studies, Japanese language, e-learning
1 Introduction
Thanks to the current advancement in the Internet technology, there have been
revolutionary changes in our way of life. We can enjoy a bunch of advantages with
reduced costs and improved services. The Internet shows its influences in the areas
from business, mass media, entertainment and short messaging to e-learning. In Hong
Kong like most developed countries, web-based learning or the so-called e-learning
has been a success that the paradigm was widely adopted from university colleges to
primary schools. Education is one of the major elements in governmental expenditure
in Hong Kong. According to the governmental budget in 2006–07 [1], $56.5 billion
was spent on education. For every university graduate who has passed through the
education system, the government's total investment exceeds $1 million. A further
$1.1 billion in total was injected into the language fund with a view to raising
students' linguistic proficiency. Enhancing students’ learning performance with the e-
technology is a vital mission that educational practitioners face today.
An Observational Study on Blended Learning for Japanese Language Studies 89
Yet a recent study conducted by Thomson and NETg [2] indicates that pure e-
learning approach does have its deficiencies. The study investigated the business
skills of participants who received a sequence of e-training courses. The performance
of participants was compared with another group who received “blended” training
courses which consisted of a combination of online media and instructor-led supports.
The study showed that the speed and accuracy performed by the “blended” group was
considerably superior to that of the pure e-learning group by 30-40%. With the
uprising urge of “enhanced productivity” and “cost effectiveness”, blended learning
has become an ideal and only solution to educational practitioners and business
training experts. What remains unsolved, however, is the question of “What is the
correct blend?”. Similar to what the bartenders used to do, we are now doing a
delicate blending with multiple media and thoughtless ah-Hoc blend giving us nothing
but failure, as what Roger Schank commented [3]:
“It is in vogue for a simple reason. No one wants to spend that much on e-
learning and people in general want to preserve what they have, so they have made
up this nice name for not changing much and called in blended learning.”
1.1 Blended Learning in Language Studies
Among all those academic and vocational subjects, language studies are probably the
most challenging areas. The reason lies in the fact that language is not a single skill
but a collection of literary and communicational skills which requires high degree of
proficiency. It is a long and painstaking process that it takes years to train up the
language ability of a student to a level which enables direct communication with
native speakers. Language study is also distinctive in a way that the learning of high
level concepts or syntaxes heavily depends on how well a student can grasp the
fundamental ones. For example, to learn the passive voice in English, students have to
master the past participles. This implies that foundation building is particularly
important during the course of study. In this paper, we demonstrate that blended
learning, which is characterized by a combination of free-form e-media and instructor
lead teaching (ILT), is the natural and ideal solution to quality language learning.
1.2 Pros and Cons of Pure e-learning Approach
In order to acquire “the best blend” for a course, one should begin with “knowing
your subject, knowing your delivery media”. The e-learning paradigm provides us
with a variety of instruction media and formats including websites, online discussion
groups, knowledge base, video conferencing and mobile systems, to name a few.
There are also well established Virtual Learning Environments (VLE) including the
open-source ATutor [4] system and the proprietary WebCT/Blackboard [5] system
which provide full solution to the learning and knowledge management processes.
Considering the old days in which an instructor would like to release handouts to
students, the only way to do so was to photocopy the handouts in mass volume and
distribute it to students during scheduled tutorial sessions. Meanwhile, the use of
hyperlinks and e-documents nowadays not only reduces the time and cost, but also
90 Kenneth K.C. Lee and Melody P.M. Chong
allows rapid information dispersion at any time and anywhere. Apart from cost
effectiveness and accessibility, e-learning can also solve two major problems in
education in the long run in terms of “variance in progress” and “short memory
duration”.
As an educational practitioner, the greatest challenge comes from the control of

class progress. Since students in class are not created equal, some students may need
more time on certain topics. At times, the progress of class is "dictated by the slowest
learners in the group" [6]. With e-learning, students can make use of non-contact
hours to revise and refine their skills in particular areas they consider necessary, and
hence progress among students could be synchronized. In addition, e-learning also
excels in a way that it facilitates knowledge acquirement and makes students active
learners. Traditional classrooms focus on the transfer of new knowledge from
instructor to students without highlighting the revision process. (Revision is
considered as something that students should handle themselves). According to a
study on knowledge retention [7], the lack of revision and memory reinforcement may
cause memory lost of 85% within a week. With online assessment and knowledge
management systems, students can now take a more active role in the purchase of
knowledge and self assessment. These can improve the quality of learning.
However, e-learning is no silver bullet. Online resources and VLE do show their
short sides from time to time. One major problem is that students are generally
inexperienced and hence personal guidelines and study models are needed in order to
keep them from wandering around the sea of online resources without target. Another
issue is that e-solution generally lacks personal interaction which is important in
disciplines such as second language acquisition [8]. As mentioned earlier, language
involves a collection of skills. In this research, we attempted to investigate different
learning dimensions to figure out a combination of pedagogic approach and teaching
media for effective language learning. We started by knowing the subject. The next
sections describe the methodology of the research and highlight the attributes of
Japanese language.
1.3 Objectives of the Study
This paper contributes by sharing our views on the application of blended learning in
language studies. In particular, Japanese language study is discussed. The objectives
of the study are to justify blended learning by analyzing the respective merits of
traditional classroom instruction and e-learning. In addition, the paper discusses what
it takes to help students master the Japanese language with reference to the existing
online and e-resources adopted in a local university in Hong Kong. Finally, the paper
also suggests possible development and directions for blended learning. It is hoped
that our propositions of our recipe of blended learning could bring incites and insights
to educational practitioners in similar academic disciplines.
2 Methodology
The present research employed a qualitative research approach known as

observational study. Observational studies can be of either the nonparticipant-
observer or the participant-observer type. Both of these can be either structured or
unstructured. Observational study is that an observer predetermines a set of categories
of activities or phenomena to be studied whereas an observer of unstructured
observational study has no definite ideas of the particular aspects that need focus. The
study employed the participant-observer type and unstructured study method. With
this research method, the investigators entertain a set of tentative research objectives
that serve as a guide as to who, when, where and how the individual will observe [9].
One of our investigators has joined a local university since 2000. The researcher is a
member of a work team in the Japanese language section taking a role of coordinator
and instructor, and observed the dynamics in classrooms for this study. This research
method allows the researchers to study the central phenomenon in depth. The
observation period was between 2004 and 2006. The courses observed included
elementary to advanced Japanese language courses. All participants observed were
university students across different disciplines and aged between 19 and 22.
3 An Observational Study for Japanese Language Study
Contents
Level Criteria
Sections Hours Points
The examinee has mastered grammar to a

Writing-vocabulary 45min. 100points high level, knows around 2,000 kanji and
Listening 45min. 100points 10,000 words, and has an integrated command
1
Reading-grammar 90min. 200points of the language sufficient for life in Japanese
Total 180min. 400points society. This level is normally reached after
studying Japanese for around 900 hours.
Fig. 1. Japanese Language Proficiency Test (Level 1)

Source: Society of Japanese Language Education, Hong Kong
In this study, we use the Japanese Language Proficiency Test (JLPT) [10] as a
benchmark to generate the ideas and examine how the current teaching activities can
be benefited from blended learning. The JLPT is a well-known international language
test consisting of four levels. Each test is made up of three sections including writing-
vocabulary, listening and reading-grammar. The contents and criteria of the test are
presented in Fig. 1. To highlight the importance of blended learning, we should first
understand the nature of Japanese language. Like other languages, the study of
Japanese requires intensive training in different aspects such as grammar and oral
skills. However, the picture is further complicated by distinctive linguistic features
which exist in the Japanese language. For example, Japanese words can be written as
kanji (Chinese character) or kana (Japanese character) and contain a huge amount of
homophones. This section describes the problems faced in Japanese language study
with respect to vocabulary, listening, grammar and oral skills.
3.1 Study of Japanese Vocabulary
In most instructor-led classes, instructors teach students about 180 new vocabularies
in one semester. Whereas those advanced learners have to remember over 6,000 to
10,000 words when they intend to attend the level 1 or 2 of the JLPT. For decades,
compilation of personal wordlist is the only means for students to master a huge
amount of vocabularies. But yet, compilation of word list is not a no-brainer and
considerable efforts are required.
As mentioned in the “Body & Brain” magazine [11], it would be beneficial for
recollection if information is “organized into a smaller set of subcategories”. This
implies that a practical wordlist must be well categorized. For instance, it is more
efficient to study the words with similar (synonym) or opposite (antonym) meanings.
As the mental database of vocabularies grows, memory tends to “interfere with each
other”. Our observations reveal that many Japanese learners find the vocabularies
difficult to memorize because some Japanese words may contain two or more
different pronunciations. For example, the word ‘human-being or person’ has at least
three different pronunciations. It includes hito, jin and nin. The situation is
complicated due to the existence of homophones. Some examples are given in Fig. 2.
Pronounciation Possible meanings

koto ‘capital’, ‘affair’ or ‘piano’
kōtō ‘oral’, ‘nice pitching (baseball)’, or ‘high level’
kōdo ‘height’, ‘brightness’ or ‘hardness’
kōdō ‘action’, ‘highway’, or lecture hall’
Fig. 2. Homophones in Japanese
3.2 Study of Japanese Listening Skill
In traditional in-class training, most instructors find difficulties in improving the

listening skills of their students. Given that the contact-hours are limited, most of the
times have been spent on vocabulary and grammatical explanations. Nowadays,
standard teaching aids used are those companion tapes or CDs included in textbook.
Each chapter accounts for about 5-7 minutes of recording, including the recitation of
main text/article, sound-only roleplay and listening questions. Considering that time
needed for students to write down the answers and for instructors to reveal the correct
answer together with vocabulary and grammatical explanations, one could easily draw
a conclusion that instructors are simply not able to afford playing the tape many times
within the standard 50-minute listening lab session in the classroom. Needless to say,
playing real-life recordings such as news broadcast or radio program in class is simply
out of the question. The outcome of all those is that the listening abililty among
students in class could be highly asynchronus, which in turn, affects the progress of
class. It is especially problematic that a considerable number of Japanese courses are
taught in Japanese itself by native teachers.
3.3 Study of Japanese Grammar
Most Japanese words are derived from Chinese. The situation is similar to English in
which some sixty percent of English words are derived from Latin languages.
However, Japanese grammar did not significantly change [12]. When teaching a
foreign language like Japanese, we found that most of our instructors spend most of
the times to teach the grammatical concepts to students. Some scholars [13] also urge
that, in particular, the Japanese particle system is a major hurdle for second-language
learners in college-level Japanese courses, due to its complexity and its absence from
the learners’ first language. In Japanese, each noun phrase is followed by a particle
that indicates its grammatical and semantic role. It is essential for most students to
understand the rule of the particles especially for those who intend to attend the JLPT.
Examinees are always asked to fill in the blanks by selecting a correct particle from a
list of multiple choice questions. Most elementary students find the rule of particles
difficult to understand.
3.4 Study of Japanese Oral Skill
Like most languages, the accent and intonation of spoken Japanese may be the most
difficult parts to learn. In Japanese language, there are many words with same
pronunciations. However, a change in the accent would change the meaning of the
____ __
word. For instance, ame can be ‘candy’ (ame) or ‘rain’ (ame). Unlike English which
has stress accent, Japanese has pitch accent which means that after an accented
syllable, the pitch falls. Another distinct feature of Japanese language is the honorific
expressions. Japan is well-known as a hierarchical society. Japanese has an extensive
system to express politeness and formality. The position of a person is determined by
a variety of factors including job, age, experience, or even psychological state [14].
Japanese polite expressions are divided into two types that are honorific words and
modest words. The former is used to express a speaker's respect while the latter is
used to express a humble attitude of a speaker. This makes Japanese expressions
difficult for the overseas learners [15].
4 Discussion
So far, we have discussed the four dimensions of Japanese language learning. In this
section, we go further to discuss how educators and learners can be benefited from
blended learning with proposed activities in relation to vocabulary, listening,
grammar and oral skills. Suggested activities are highlighted in Fig. 3 whereas each
dimension is discussed sufficiently in the followings.
Ability Distinctive attributes Blending of media Suggested activities

z Numerous! z Share and exchange
compiled wordlist via
z Is best memorized with internet and VLE
synonym / antonym
z Use of online
z Requires dictation dictionaries which
overwhelm the traditional
ones in terms of word count
Vocabulary and related resources
online hardcopy
resour wordlist/ z Remember vocabularies
ces dictionary anytime anywhere with
mobile technology
z Online quiz (dictation)
with word database
z Repeated listening is z Utilize online radio / AV
needed broadcast via steaming
technology through internet
z Ability improves if
constantly “immerse” in z Upgrade teaching aids
that language. from cassette tape to digital
media with visual aid and/or
Listening z Best training aids speed control
possibly come from real-
life dialogues & online tapes
announcements from radio/AV and
native speakers programs CDs
z Closely related to oral

skill
z Idea can be presented z Distribution of
in a thousand and one distinctive articles, templates
ways with different tones & teaching aids via internet /
and styles VLE
z Learning via reading z Replace part of assess-
model articles with good ment with online quiz
rhetoric (especially MC questions for
lower grade)
z Practice makes perfect
Writing and z Discussion or Q&A
Internet as a Manual using online forum guided
Grammar platform for reading by e-mentors or e-tutors (as
discussion and and self-learning / knowledge
assessment (MC) marking anagement)
z Marking of students’
work by experienced
instructor, yet hardcopy
submissions are replaced by
electronic ones submitted
through VLE
Oral z Stress on interaction z Face-to-face classroom
gatherings with role-play
z Complex intonation practices
rules
z Recording of students’
z Existence of speech into digital form (e.g.
homophone mp3) and submit to
z Current Artificial digital face-to-face instructor via VLE.
Intelligence and speech audio/ guidance
recognition technologies speech
could not catch up with (supporti
the complexity of natural ve)
language
Fig. 3. Suggested blending model for Japanese study

4.1 Vocabulary
With respect to the vocabulary dimension, it is expected that the e-technology will
create synergy by speeding up the wordlist compilation and classification process for
students’ self-learning. The “huge information volume” and “high searching speed”
properties of e-resources match perfectly to the demanding wordlist compilation task.
Students can store the words in an electronic form instead of hand-written text on
small paper cards. It allows students update the personalized wordlists and review
them at anytime and anywhere. Students could also share or exchange wordlists in
VLE platforms, discussion forums, blogs or even personal wiki solutions [16] with
their fellows. Existing web-based systems such as Yahoo! Dictionary Service [17]
(Fig. 4) provide related synonym and antonym in terms of a large word counts and
related resources.
Fig. 4. Yahoo! Japan – Dictionary Service
Fig. 5. ii-Kanji: a mobile system Fig. 6. Kanji-Step: an online quiz system

for vocabulary learning with detail statistics
Mobile system such as ii-Kanji [18] (Fig. 5) facilitates kanji (Chinese character)
learning by enabling memorization and revision at anytime and anywhere. e-
Technology also enables students to have self evaluation through the online dictation
systems. Systems such as Kanji-Step [19] provide students with vocabulary quizzes
with words classified into categories and levels. As shown in Figure 6, statistics
would be generated after quizzes and it could act as clues for further improvement.
In the JLPT paper, candidates are always being asked to choose the right
pronunciation of a kanji. Students have to select the correct one from a set of similar
pronunciations (Fig. 7). The computerized grouping system suggested above will
certainly help students to solve the learning problems. To further facilitate the
learning process, we suggest incorporation of query system for similarly-pronounced
words into e-learning environments. Although such system is yet to be seen in public,
there should be no technical difficulty with free Japanese resources [20] and a server
side programming language such as PHP, ASP .NET or JSP.
Fig. 7. JPLT Paper 1: Writing-Vocabulary

Source: The 1997 Jap. Language Test Lvl 1, published by Society of Jap. Language Education,
HK
4.2 Listening
As mentioned earlier, the difficulties of listening skill training lie in “limited contact-
hour” and “asynchronous progress in class”. The use of e-resource could certainly
alleviate the current situation. First of all, the media itself is inherently superior.
Digital audio in wav, mp3 or wma formats allow random access, as opposite to the
sequential access method in traditional audio tape. Student could easily set bookmark
to appropriate phrase or word in which they consider more attention is needed.
Another advantage over traditional media is that students could adjust the playback
speed according to one’s personal listening ability. There are existing pedagogic tools
for audio indexing together with speed control for language learning [21]. With
classical signal processing techniques such as resampling and pitch-shift, students
could enjoy clear and slowdowned replay of listening material without acoustic
distortion. Popular media players such as PowerDVD [22] and GomPlayer [23]
already have the variable-play-speed features incorporated and are applicable to most
Audio/Visual (A/V) media.
Students can also enjoy listening of real-life programmes via streaming technology
on the Internet. Japanese mass media such as NHK [24] broadcasts free radio
programmes on the Internet. Students can refine their listening skill by immersing
themselves in real-life language and native Japanese dialogues. Students can also
learn jargons in specialized channels such as Daiwa Securities [25] and Nikkei
Business [26]. In addition, there are many Japanese learning programs packaged with
visual content and description such as the “ShinNihongo de Kurasou” programme (新
日本語で暮らそう) provided by NHK. With the advanced Internet technology, today
students can take more active role in self-learning at their own pace.
4.3 Oral
It is believed that the best way to enhance oral ability may be to maintain face-to-face
interactions between tutors and learners. As Yamazaki [27] suggests, despite the great
expectations of society for such an ideal speech translating system, there remains a
great number of difficult problems related to acoustic and linguistic phenomena that
need to be overcome. This is because spoken language is characterized by the use of
colloquial idioms, occasional omission of words and inversion of word order. It seems
that face-to-face interactions in the classrooms with role-play practices remain to be
the most effective way to polish the oral skill. Nevertheless, in reality, it is infeasible
to have intensive training between teachers and students because this involves a huge
investment in terms of budget and human resources. Based on our observations and
experiences, it is suggested that students may record their speech into a digital form
(e.g. mp3) and submit it to their instructor via email or VLE for feedbacks. This is
especially effective for those advanced level classes that the instructor may not have
enough time in the classroom to train up their students’ speech and presentation skills.
4.4 Grammar
In order to improve students’ writing and grammar skills, a combination of a blend of

e-technology and traditional tutoring is suggested. Nagata’s [13] experimental study
suggests that the computer’s metalinguistic feedback program can lead learners to
develop general grammatical competence in the use of particles. The program can
guide adult learners with explicit grammatical instruction and detailed, metalinguistic
feedback about errors. Instructors can also post some good articles on VLE
environment (Fig. 8) [5] for students’ self-learning. With on-line discussion forum,
instructors can also encourage their students to write Japanese diary and share their
writings with their fellow students. Online forum supporting alias and anonymous
identity may stimulate students’ motivation. It is expected that students would be
more willing to post their works and ask for comments. In addition, forum or chat
room can be set up allowing students to discuss the issues in relation to grammar,
culture or experience sharing. Instructors can also assign senior students to act as e-
helpers or e-mentors to help the junior students on their particular learning problems.
It can provide practical chances for those senior students to learn and also lead to
better utilization of resources. With the above suggestions, instructors can spend more
time on creating teaching materials and help those students with learning difficulties.
Fig.8. Japanese discussion forum on Blackboard (VLE system)
5 Conclusion
The effectiveness of blended learning depends on whether a teacher can match the
appropriate delivery media to existing teaching activities and to catch up with the e-
learning training. In this paper students are suggested using online dictionaries and
quizzes to assist vocabulary learning. The informative online resources not only save
time for knowledge management and retrieval but also free students from scheduled
and geographical restrictions. E-resources also benefit listening training by providing
acoustic features which are absent from traditional media. The use of media streaming
through the Internet allows students to access real life programmes and broadcasts
from Japan, which are perceived as ideal training materials for listening. For
grammatical training, we suggest the use of VLE and online forums with assistances
from e-helpers or e-mentors. Instructors may also employ the computer assisted
software with intelligent feedback systems which may help students solve the
sophisticated grammatical problems. Finally, with respect to oral skill training, the
preservation of face-to-face instruction with activities such as real life role-play shall
be maintained. As suggested by Chaudron [8], the interaction in the classroom
between peers is one of the most significant factors for successful learning of the
second language acquisition (SLA). However, recording of learning progress in a
digital form is also suggested to supplement with the classroom interactions because it
allows valuable clues of room for improvement.
To conclude, the use of e-technology and blended learning brings about effective
learning and cost effectiveness. However, like other revolutionary institutional
policies, ‘human’ is the ultimate factor which governs make or break. Despite the
availability of robust hardware (technology), we also need appropriate software
(instructors) to make things happen. Instructors must be well trained, both technically
and mentally, and accommodate themselves to the new-generation instructional

media. In addition, facilitators shall also adjust their teaching mode rested on the
motivation and feedbacks of their students, and be able to encourage and maintain
active and well-mannered learning atmospheres on the web-based learning.
6 Limitations and Recommendations for Future Research
This study has some weaknesses that future researchers might consider. The study
relied on information observed from a single organization. More extensive analysis
with larger sample size would serve to strengthen understandings and conclusions on
blended learning for language studies. Future research shall also examine the effects
of blended learning on students’ performance based on the empirical data and
analysis. More research into this area will contribute to formulate a better teaching
mode for educational practitioners.
References
[1] The 2006-07 Expenditure, The Government of Hong Kong SAR

http://www.budget.gov.hk/2006/eng/budget17.htm
[2] Thomson Job Impact Study - The Next Generation of Corporate Learning
http://www.netg.com/NewsAndEvents/PressReleases/top
[3] Blended Learning - Ask the Arthur – CIO
http://www2.cio.com/ask%5Cauthor/2001/questions/question232.html
[4] ATutor Learning Content Management System
http://www.atutor.ca/
[5] Blackboard – Educate. Innovate. Anywhere
http://www.blackboard.com/us/index.Bb
[6] Masterworks International - Distance Learning
http://www.masterworksinternational.com/faq.asp
[7] Jeff Snipes, “Blended Learning: Reinforcing Results”, Sept 2005
http://www.clomedia.com/content/templates/clo_article.asp?articleid=1070&zoneid=25
[8] Chaudron, C. “Second Language Classrooms: Research on Teaching and Learning.”,
Cambridge: Cambridge University Press, 2004.
[9] Cavana, R.Y., Delahaye, B.L. and Sekran, U., “Applied Business Research: Qualitative
and Quantitative Methods.”, Australia: John Wiley & Sons, 2001.
[10] The Japanese Language Proficiency Test (JLPT), Japan Educational Exchanges and
Services http://www.jees.or.jp/jlpt/en/index.htm
[11] Body & Brain Magazine
http://www.bodynbrain.com/01_magazine/news_view.asp?SeqNO=36
[12] The Japanese Language, http://www.wsu.edu/~dee/ANCJAPAN/LANGUAGE.HTM
[13] N. Nagata, “The Effectiveness of Computer-Assisted Metalinguistic Instruction: A Case
Study in Japanese”, Foreign Language Annals, 30(2), pp. 187-200, 1997.
[14] Wikipedia – Japanese Language – Politeness
http://en.wikipedia.org/wiki/Japanese_language#Politeness
[15] H. Ogata, and Y. Yano, "How ubiquitous computing can support language learning", In
Proceedings of KEST 2003, Honjo, Akita, Japan, pp. 1-6, 2003.
[16] List of wiki software, http://en.wikipedia.org/wiki/List_of_wiki_software
[17] Yahoo! Japan – Dictionary, http://dic.yahoo.co.jp/

[18] Today’s ii-Kanji, http://www.ii-kanji.com/
[19] Kanji-Step – Japanese Language Resource Center, http://www.kanjistep.com/
[20] EDICT - Japanese/English dictionary project (Monash University),
http://www.csse.monash.edu.au/~jwb/edict_doc.html
[21] D. Rossiter, G. Lam, B. Mak, “Automatic Audio Indexing and Audio Playback Speed
Control as Tools for Language Learning”, ICWL 2006, pp. 289-297.
[22] CyberLink – PowerDVD, http://www.cyberlink.com/
[23] GOM Player, http://www.gomplayer.com/
[24] NHK News, http://www.nhk.or.jp/news/
[25] Daiwa Internet TV, http://www.daiwatv.jp/
[26] NBOnline (Nikkei Business Online), http://business.nikkeibp.co.jp
[27] Yamazaki, Y, “Research activities on spontaneous speech translation”, Denshi Tokyo,
33, pp. 109-114, 1995.
Structured Blended Learning Implementation for an
Open Learning Environment
Jason K.Y.Chan, Ken C.K.Law
Department of Computer Science, City University of Hong Kong,

Tat chee Ave., Kowloon, Hong Kong,
jasonchan@cm13.mit.edu & cskckl@cityu.edu.hk
Abstract. This paper proposes a structured blended learning for providing e-

learning strategies adopted by the Open University of Hong Kong (OUHK).
The paper identified the factors that are likely to impact on the use of learning
technologies in future. By introducing the structured blended learning with the
concept of learning cycle, the problem of being lost in hypermedia environment
can then be solved. Knowledge can be delivered more effectively and
efficiently to meet student learning needs. A scenario example on the use of e-
learning technologies and user interface are illustrated. The implications of the
learning strategy adopted at OUHK will also be discussed.
Keywords: blended learning, learning preference, Open Learning Environment

(OLE), proactive planning, asynchronous and synchronous online learning.
1 Introduction
This paper proposes a structured blended learning mode for providing distance
education by the Open University of Hong Kong (OUHK) in the few years.
According to Iverson (1993), traditional methods of curriculum development usually
require four-to-seven year cycle in the process from draft originals to final revised
curriculum implementation. In the world of fast changing information technology
and ever changing applications, this development cycle will encompass an enormous
amount of changes. Such a period would also involve the assessment of current
strengths and weaknesses of the learning environment and the identification of new
educational trends. In Hong Kong, most higher education institution's strategic plan
adopted a 5-year development cycle. (OUHK, 2002; HKU, 2002; CityU, 2002) In
this paper, the factors that are likely to impact on the potential use of learning
technologies in the OUHK over the next few years will be described. In the first
section, we will provide a brief teaching and learning environment at the OUHK,
reviewing the factors that are likely to impact on the potential use of learning
technologies in the OUHK over the next few years. In the second section, we will
introduce the proactive plan about what learning technologies will be likely to be
employed at that time. In the third section, we will discuss the implications of this
change, followed by a brief conclusion in the fourth section.
102 Jason K.Y.Chan, Ken C.K.Law
1.1 Teaching and Learning
The OUHK was the first university, as a self-financing and non-profit making
institution, to offer open and distance education in Hong Kong. Since its inception in
1989, the University has striven continuously to achieve its mission of providing
flexible further education opportunities for adults aged over 17. The University has
offered more than 130 postgraduate, degree, associate degree and sub-degree
programs. (OUHK, 2005)
1.2 Blended Learning
The OUHK offers studies with flexible form of learning environment, for students to
choose, where and when, and also provides them with carefully structured materials
to guide them through the courses. The learning style offered by OUHK is a blended
learning mode via printed materials, CD-Rom, non-classroom face-to-face tutorial
sessions held in the evening and weekend and also online learning. Students can
decide on their study path and pace, making use of a variety of print and interactive
multimedia materials, tutorial support and also access to electronic library resources.
The OUHK provides classroom sessions around Hong Kong for tutorial lessons
through rental means. There are also trained tutors to conduct tutorial sessions,
tutorial discussions and course work assessments, and also responded to students'
queries by phone and e-mail. The adoption of Online Learning Environment (OLE),
students can access a comprehensive range of learning resources and personalized
services to help them study more effectively without the constraints of time and place.
(OUHK, 2005)
1.3 Readiness of Student with Information Technology
To use electronic means of communication, OUHK need to know if the students are
familiar and be able to access the necessary facilities and equipment. The OUHK
carried out surveys on students regarding their information technology readiness on a
regular basis. One of the surveys revealed that an increasing trend of PCs ownership
by students, access to the Internet, and willingness to use email for communication.
(OUHK, 2005)
Structured Blended Learning Implementation for an Open Learning Environment 103
Source: Chung, 2003
Another recent survey in July 2003 (Chung, 2003) indicates that: 99.0% of students
have PCs; 97.5% of students have access to the Internet; 78.5% use broadband, and
63.5% use email more than once a day; 78.0% of students prefer to use the Internet
for communication with the university, tutors and other students (8.0% do not prefer
to do so; the rest are neutral); and 84.0% of students prefer to receive information or
materials from the University’s Registry via email (8.8% do not prefer to do so; the
rest are neutral).
The results reveal that the majority of students are technologically prepared and have
access to information technology equipment. Fewer than 10% of our students do not
want to use electronic means as the primary form of communication with OUHK.
(Yuen, 2004)
1.4 Student Learning Preferences
When the students have access to the related facilities, another thing we need to know
is their willingness or preference to use learning technologies. Another OUHK
survey (Vermeer and Murphy, 2004) revealed that there is a strong student preference
for using multiple forms of media in learning.
Student Preferences for Using Different Forms of Media in Learning
Source: Vermeer and Murphy, 2004

OUHK students’ first preference is clearly face-to-face contact. Yet two forms of
online learning support -- informational course websites, and online discussion boards
-- still ranked very high, with over 50% of respondents considering them either
‘effective’ or ‘highly effective’. Since other data, both research-based and anecdotal,
show a highly variable commitment to the use of online learning tools in OUHK
courses, these positive results are significant. Equally significant is students’
disappointingly low opinion of course CD-ROMs: only 36% of respondents found
them effective or highly effective, and 26% considered them ineffective or highly
ineffective. (Vermeer and Murphy, 2004)
OUHK course studies still utilize two traditional forms of media, i.e. video and audio
tapes, as many students suggested the OUHK to provide online, streamed versions of
as Audio/Video content. Numerous respondents in fact suggested that the University
provide ‘video lectures’, ‘tutorial notes in PowerPoint’ and ‘highly interactive Flash
notes’. Another significant theme in respondents’ comments was repeated requests
for multiple forms of access to course content. The OUHK students want a varied
media mix, including overlapping provision of content using different media, and they
want it delivered using the latest technologies. However, the University need to
consider the key factors for the change involved in time, effectiveness and also
financially viable. (Vermeer and Murphy, 2004)
1.5 Insufficient Interaction and Guidance in Existing OLE
Successful e-learning environment involve interactivity between teachers and

students, between students and the learning environment, and among students
themselves, as well as active learning in the classroom (Sherry 1996). Multimedia
equipment such as graphics, video, animation, and sound, can add richness in contents
to materials, but they do not add interactions. An animation sequence or colorful
graphical scene attracts the learners’ attention, but does not engage them in making
decisions or immerse them in the program. Interactions make learning active rather
than passive, and they provide learners and the instructor with feedback. Interaction
is a cyclic process in which two actors alternatively listen, think and speak (Crawford,
2000). It is important to develop teaching materials with a variety of interactive
techniques.
Interactivity takes many forms; it is not just limited to audio and video, nor solely to
teacher-student interactions. It also represents the connectivity the students feel with
aides, facilitators and their peers. Garrison (1990) argued that the quality and
integrity of the educational process depends upon sustained two-way communication.
Without connectivity, e-learning degenerates into the old correspondence course
model of independent study. The student becomes autonomous and isolated,
procrastinates, and eventually drops out. Effective distance education should
approach Keegan's ideal of an authentic learning experience. However, the existing
web-based learning does not provide sufficient interaction means.
Moreover, the existing OLE connects hypermedia such as text, graphics, animation,
audio and video in a nonlinear manner. The major problem with hypermedia leads to
feel lost in learning hyperspace. Learners may get lost in complex hypermedia
connected webs. It is difficult for learners to find a way to organize the learning
material as a guided learning in an orderly manner. Besides, there is the
unpredictable nature of how readers link to hypermedia elements. If essential
information is located in a hypermedia link that is not structured, the learner will not
master the objectives in that lesson. Finally, a learning gap will exist between
learners and teachers, teachers and knowledge delivery, knowledge delivery and
learners. The interactions and communications between teachers and learners are not
sufficient and efficient. (Chan, 2003)
2. Proactive Planning
University policy makers, nowadays are required to have proactive thinking of how
the university will be lead, managed and structured the learning environment. They
also need to face many important issues and decisions concerning the impacts of
information technology on their institutions such as organization, governance,
management, and its relationships to students, faculties, and staffs. All of them will
require careful reevaluation, forecast and almost certain changes. (Duderstadt, Atkins
and Houweling, 2002)
2.1 Scenarios of Information Technologies in OLE
The OUHK imitated an e-learning project in January, 2004, which aimed to develop a
WEB-based Interactive Tutoring System (Webits) that enabled teachers and
students to interact effectively online, supported educational multimedia data
transmission in real time such as video, audio, text, and enhanced white-board data,
presentation of educational materials. The project was developed based on the
following scenario imagination and assumptions. Assume that technology
developments, that most people will have access to a wireless network with
bandwidth of at least one gigabyte per second within a few years. Most computers
will be equipped with webcam with face tracking technology as the basic standard
device. The developments in mobile computing technologies, the university server
will remind the students’ course schedule via SMS or MMS messages. Some course
introductions might be delivered in video and audio via cell phones. The new
communication means will blur the distinction between traditional and distance
education. The technologies have potential uses in both situations with little
noticeable differences. The main pedagogical issue is to understand where the new
technology will have real impact on learning effectiveness. Some of the technologies
will ease the constraints of time and distance by bringing the possibilities of face-to-
face tutoring to the students, in the home, on the street or anywhere, as long as the
students have access to a Internet connected device.
2.2 Combination of Asynchronous and Synchronous Online

Learning
The advantage of asynchronous OLE is flexible learning especially for global distance
education, as it does not require the learners or instructors to be online at the same
time. However, learners may get lost in complex hypermedia information webs.
Learners may find it difficult to orient to find their way to the learning topics. (Chan
and Law, 2004) It is concluded, apart from adapting the asynchronous OLE,
interactive tutoring systems will be introduced to provide synchronous interaction for
more efficient moderating of learners’ discussion and immediate feedback from
instructors.
The asynchronous and synchronous online learning platforms do not exclude each
other but provide complimentary ways of communications and interactions for
different teaching and learning purposes. The range of tools available in an
interactive tutoring system makes complex topics manageable. Complex topics can
be explained directly by using tools such as whiteboards, application sharing, text-
chat, real-time audio, and video-conferencing. These synchronous tools will be
combined with asynchronous tools such as video clips, text, images and animation.
Instructors and learners will be able to illustrate their ideas in both directions
interactively.
2.3 Learning Cycle in the OLE
The major challenge for distance education is to address the issues of dialogue across
the response and psychological distance between teachers and students. (Uys, 1999)
Chizmar and Walbert (1999) argue that pedagogy must drive the choice of
instructional technology. To provide successful teaching, teachers must be
considerate and design a series of steps that make connections with the students’ prior
understanding, actively process new information or practice skills, and set the stage
for demonstration of the learning outcomes. Teachers can adjust these steps in the
light of the ‘classroom’ experience as it unfolds. (Chan, Yim and Chen, 2005) And
hence, the blended learning model can be more structured by having a four-stage
Learning Cycle in the OLE platform as illustrated in Figure 1. The four stages are:
Pre-learning; Lecturing; Tutoring; and Assessment. We can see the flow of user
interfaces and how they are interrelated with the four learning stages to accomplish
the learning cycle.
Figure 1 Web-based Learning and Interactive Tutoring System with Learning

Cycle. (Chan, 2003)
2.3.1 Pre-learning
The activities in Pre-learning stage will capture the student's attention, stimulate their
thinking and help them access prior knowledge. The teaching materials at pre-
learning stage can be asynchronous. Figure 2 illustrated Pre-lecturing Stage with a
simulated screen shot on a topic.
Figure 2 Example of Pre-learning Stage on a topic.
2.3.2 Lecturing
Lecturing will be taken as a one-to-many stage. Students will be introduced to
concepts and skills/abilities using familiar material. This is to make it easier for
students to concentrate on the concept development. Students can input their queries
and comments during the virtual lecture. Figure 3 illustrated Lecturing Stage with a
simulated screen shot on a topic.
Figure 3 Example of Lecturing Stage on a topic.

2.3.3 Tutoring
Tutoring stage will give students the opportunity to expand and solidify their
understanding of the concept. Students will be involved in an analysis of their
exploration. Their understanding will be clarified and modified because of reflection
via synchronous interactive activities. In the tutoring stage, all participants connected
by web-cams to their computers will show their video images to the system and all
participants. Figure 4 illustrated Tutoring Stage with a simulated screen shot on a
topic, noticed the video-conference feature (with web-cam devices) to enable
discussions between tutor and students.
Figure 4 Example of Tutoring Stage on a topic.
2.3.4 Assessment
Finally, the learning cycle will close with an assessment stage that answers the
questions: “Was the instruction effective?”, and “What is the next step for the
learner?”. In this stage, the instructor will deliver a prepared assignment in multiple-
choice mode to the students to complete in a given period of time. The results and
statistics of the assignment or quiz will be sent to the participants immediately after
the given period. Figure 5 illustrated Assessment Stage with a simulated screen shot
on a topic.
Figure 5 Example of Assessment Stage on a topic.
3. Meeting the Learning Needs

It is important to point out that it is not necessary to apply the four-stage learning
cycle in all the courses. (Chan, Yim and Chen, 2005) The above stimulated
synchronous interactive online learning system should allow the instructor to use any
stage and in different order in real practice.
By introducing the concept of a learning cycle, the blended learning model will
become more structured. The problem of becoming lost in hypermedia information
web can then be solved. Knowledge can be delivered more effectively and
efficiently. The system design may be more creative by providing the following
features and arranging them in different learning stages:
z allow teachers to schedule their tutorials in a given period of time;
z provide video indexing function for teachers to correspond the lecture notes’
content with video playback;
z allow teachers and students to interact in a virtual face-to-face environment;
z allow teachers to display and make remarks on the files they uploaded; and
z allow teachers to show special items such as 3-D objects through the connection
to a visualizer.
On the other hand, as asynchronous e-learning tools, such as WebCT and Blackboard
were being used for several years, users may become familiar and comfortable with
the existing course delivery mode and would resist the introduction of synchronous
online learning platform. In certain cases, users may believe it will conflict with the
value of flexible learning.
However, as mentioned, the e-learning approach by the OUHK is a blended
learning mode via printed materials, CD-Roms, lecture-less face-to-face tutorial
sessions and OLE. In practice, instructors often give lectures in a one-to-many mode.
However, in tutorial sessions, it is often that students need guidance from instructors
in order to understand the teaching materials (in the printed text, CD-Roms or
websites), one-to-one interactive mode would be more desirable. OUHK adopted a
synchronous communication medium for the student to raise questions and obtain
immediate feedback and interaction with tutor. As suggested by McLuhan (1989) in
the work, quoted, “the medium is the message”.
Moreover, there is a need for a more flexible and practical schedule of tutorial
sessions since most of the OUHK students have full-time work during the day. It may
be difficult to attend the course in the evening, as many of them regularly need to
work overtime. With the synchronous online learning platform, the tutorial sessions
can be arranged with higher priority, such as starting at late evening to enable students
to cope with their overtime demand from employers. This is clearly more flexible for
OUHK students and their work life style.
4. Conclusion
By using a more structured blended learning platform with learning cycle, students
can gain the more guidance in learning and the University can provide more flexible
e-learning environment. However, “e-learning is not a cheap alternative to face-to-
face teaching” (Bates, 2002). But, for the OUHK, the combination of asynchronous
and synchronous online learning platform will reduce the need to have classrooms
around the city, (OUHK, 2005) with some cost saving in classroom rentals. This has
direct impact on the tuition fee reduction, to attract more students and helps to
increase enrolments. Although the “contact-hour course” is still labor-intensive
(Graves, Henshaw, Oberlin and Parker, 1997), it is still acceptable to the OUHK
which needs to provide mandatory face-to-face tutorial sessions. But the challenge is,
as Bates describes, “to develop a system that encourages teaching units to be
innovative and able to respond quickly to changes in subject matter, student needs,
and technology. At the same time, redundancy and conflicting standards and policies
across the institution must be avoided” (Bates, 2000). The University will need to
provide additional resources for tutor training and briefing sessions to students for
adoption of the new OLE, if the University is determined to have the full-scale phase
in of using information technology in the learning environment.
References
Anderson, T. (2003) ‘Getting the Mix Right Again: An Updated and Theoretical Rationale for
Interaction’, International Review of Research in Open and Distance Learning, 4(2).
Retrieved February 10, 2005, from http://www.irrodl.org/content/v4.2/anderson.html
Bates, A.W. (1993). Theory and practice in the use of technology in distance education, In
Keegan, D. (Ed.). Theoretical principles of distance education. London: Routledge.
Bates, A.W. (2000). Managing Technological Change: Strategies for College and University
Leaders. San Francisco: Jossey Bass.
Bates, A.W. (2002) National Strategies for e-learning in Post-Secondary Education and
Training. Paris: UNESCO/International Institute for Educational Planning.
Chan, J. K. Y. (2003). Web-based Learning and Interactive Tutoring System with Learning
Cycle, Thesis of Master of Science in Computing, City University of Hong Kong.
Chan, J. K. Y. (2005) Experience in developing interactive conferencing tool (voice chat),
Communications Technology Forum, Island Learning Centre, OUHK, January 15, 2005.
Chan, J. K. Y. and Law, K. (2004). A need for a fine tuned interactive online learning system
with learning cycle, presented in the conference of the 21st ICDE World Conference on
Open Learning and Distance Education, Hong Kong, February 18-22, 2004.
Chan, J. K. Y., Yim, P. Y., and Chen, L. (2005). Web-based interactive learning system and
learning cycle. China Educational Technology. 2005 (3).
Chizmar, J. F., and Walbert, M. S. (1999). Web-Based Learning Environments Guided by
Principles of Good Teaching Practice. The Journal of Economic Education, 30, 248-259.
Chung, L (2003). Survey on OUHK students’ use of IT -- 2003, unpublished report, Hong
Kong: The Open University of Hong Kong.
City University of Hong Kong. (2002). Strategic Plan 2003 to 2008: Meeting the Challenge of
Change. Retrieved March 20, 2007, from http://www.cityu.edu.hk/op/plan/plan_2003-
2008_01.htm
Crawford, C. (2000). Understanding Interactivity. Self published, 2000. Republished 2002, by
No Starch Press as The Art of Interactive Design.
Duderstadt, J. J., Atkins, D. E. and Houweling D. V. (2002). ‘Higher education faces a brave
new world’. Higher Education in the Digital Age, pp.3-22.
Graves, W., Henshaw, R., Oberlin, J. and Parker, A. (1997). ‘Infusing information technology
into the academic process’. In Peterson, M., Dill, D., Mets, L.A. and Associates (eds.)
Planning and Management for a Changing Environment. San Francisco: Jossey Bass, pp.
432-452.
Gunn, C. (1998). Virtual technologies in higher education: vision or reality? In Peters, M. &
Roberts, P. (Eds.). Virtual technologies and tertiary education. London: Routledge. pp. 134-
145.
HKU. (2002). The University of Hong Kong Strategic Development 2003-2008, Retrieved
March 20, 2007, from http://www.hku.hk/strategic-booklet/text/eng-html.htm
McLuhan, M. and Fiore, Q. (1989). The Medium is the Message. New York : Touchstone.
OUHK. (2002). Business Plan 2003–2006, Retrieved November 30, 2005, from
https://staff.intranet.ouhk.edu.hk/items/po/bp/index.htm
OUHK. (2005). The Open University of Hong Kong Staff Intranet. Retrieved February 13,
2005, from http://www.ouhk.edu.hk
Raskin, J. (2000). The Humane Interface. New Directions for Designing Interactive Systems.
Addison-Wesley.
Vermeer, R. and Murphy, D. (2004) ‘What students want – students get?’. Paper presented at
the 18th Annual Conference of AAOU, Shanghai, November 28-30, 2004.
Uys, P. M. (1999). Towards the Virtual Class: Technology Issues from a Fractal Management
Perspective. Proceedings of the ED-MEDIA 99-World Conference on Educational
Multimedia, Hypermedia & Telecommunications. Seattle: AACE. Retrieved February 10,
2005, from http://www.globe-online.com/philip.uys/www.globe-
online.com,philip.uys,edmedia1999.htm
Uys, P. M. & Siverts, S. A. (2001). Managing Technological Transformation in Higher
Education: A Southern African Perspective. Proceedings of the 22nd World International
Council for Distance Education Conference. Dusseldorf, Germany.
Vermeer, R. and Murphy, D. (2004). ‘What students want - students get?’. Paper presented at
the 18th Annual Conference of AAOU, Shanghai, November 28-30, 2004.
Wong, Wai-man. (1999) ‘Electronic Library Services for Distance Learners - Its Developments
in the Open University of Hong Kong.’ In [Proceedings of the] ICDE Librarians'
Roundtable, 11-12 October, 1999, The Open University of Hong Kong. Hong Kong: Open
University of Hong Kong, 1999, 82-85. ERIC ED 438 832.
Yuen, K. S. (2004). ‘Replacing postal mail by electronic mail for communication in distance
education: how is this achieved at the OUHK?’, Paper presented at the 18th Annual
Conference of AAOU, Shanghai, November 28-30, 2004.
The Disruptive Effect of Technology:
a University Case Study
Norah Jones
Centre for Excellence in Learning and Teaching (CELT),

University of Glamorgan, United Kingdom, CF37 1DL.
njones2@glam.ac.uk
Abstract. For many, e-learning and blended learning are seen as a technical
solution to improve teaching. It is commonly viewed as neutral - just another
tool in the lecturer's kit bag. This we believe is a naïve view and hides the
extent and complexity of change required at universities [8, 9]. Technology is
not just another way of delivering course content. Blended learning is
challenging our education practices and underlying epistemologies and theories.
The design of blended learning needs to be grounded in sound education theory.
We need to ensure that we blend technological and pedagogical advancements.
If we design programmes on-line and ignore education theory then we are in
danger of leaving learning to chance.
Keywords: blended learning, higher education
1 Aim
The aim of this paper is to explore the impact of embedding blended learning in a
case study university in the Higher Education sector in the UK. The paper provides an
organisational context, explores the definitional complexities associated with blended
learning; and the way in which blended learning challenges the hierarchies of
universities.
2 Introduction
E-College Wales (ECW) was established in 2001 at the University of Glamorgan as a

vehicle for the delivery of distance learning courses, from the University of
Glamorgan Business School and six further education colleges. The courses that were
developed for ECW were technologically enhanced versions of programmes that were
available by traditional means on-campus and new courses developed only for online
delivery. After three years of operation, ECW had recruited over 1,000 students onto
programmes, a considerable success in terms of e-learning programmes. The project
however disrupted many aspects of the university’s cultures.
The Disruptive Effect of Technology 115
When I agreed to manage this project I had assumed I would build on my

experiences. I quickly learned that this was not as simple as it had seemed; every
aspect of University life needed to be reconsidered and I found I was challenging
existing hierarchies and traditions[9, 10]. It became clear at the start of the project that
a range of staff, academic, technical, administrative and staff with new composite
skills from different departments across the University and across the Welsh further
education network needed to work together. Existing administrative structures were
problematic and hindered interdisciplinary arrangements. There were very few
examples of staff from so many different areas of the University working together in
one group; the boundaries between academic departments and support departments
were well protected by tradition and culture needed to change
3 The Blurring of Boundaries
The development of an e-learning environment led to the creation of multi-

disciplinary teams, including staff from Academic Registry, Learning Resources
Centre, Human Resources, Marketing, Student Information Systems, Information
Systems, Student Services, the partner colleges and the academic schools. It is clear
that the success of the project rested upon an integrated team involving all the
University's support departments working alongside the Business School from the
beginning. This is because in an e-learning environment the support is required at
the start and is immediately transparent to the e-learner when they log on-line [18].
Nunan et al. [16] highlight the importance of integration,
"Information technologies are bringing structural change to service

areas, causing a convergence of roles and functions between registry,
library, corporate services, production and teaching support and
student services" (p.72).
The blurring of traditional departmental boundaries has been particularly evident

in the development of modules on-line and represents an important change within
universities. This however is posing challenges, especially to heads of department
who may see this blurring as a threat to their power. Sloman [19, p.14] emphasises,
"Connectivity is especially powerful. Boundaries separating different organisations
and activities have become increasingly irrelevant (or blurred)". The new paradigm
has also created new relationships between further education partners and between
further education and higher education. Further education staff, and colleges, are
assuming new roles which will necessitate new agreements, funding models and
approaches to quality assurance. E-learning requires much higher levels of quality
assurance including issues of copyright; intellectual property rights need to be sorted
out at the beginning of the process in the development of on-line courses [12]. These
changes will also threaten the prevailing hierarchies and pre-conceptions about the
status of further education and the collaborative models were re-examined [5].
116 Norah Jones
It is clear that the impact of e-learning, and in the future mobile learning, will
require universities to re-think fundamentally their strategies, in a range of areas
including human resources, estates, pedagogy, quality assurance, funding,
management and commercial and educational partnerships. Inglis et al. [7, p.189]
confirm our findings:
“For most organizations, the transition to electronic delivery will represent a
significant shift. It will involve major changes to the organization: changes in
staffing, procedures, infrastructure, and most of all to the culture of the
organization”.
Information and communications technologies (ICT) are forcing major changes
in the location, development, methods, delivery, support, evaluation and timing of
education delivery. Although it provides real opportunities it also poses threats that
need to be addressed. Grasping the full potential of ICT will require a substantial
shift in human resources policies including recruitment, contracts, training and
development and innovative payment systems.
As far back as 1967, McLuhan [15] visioned 'the global village' and Hanna [6, p.
8] prophesises that "McLuhan's concept of the global village is about to come to life
for every person on the planet". E-learning and the Internet are seen as the ideal
medium to create a global village [7, 13, Evans and Nation 2000) and universities
are expected to be at the centre of the move towards the 'global village'. [20, p.39]
highlights this as follows:
" ..some of the biggest changes for universities will stem from further advances
in I.T… a capacity for interactive networking which will connect any university to a
global audience".
Communication technologies that are free from time or place constraints provide
new challenges to universities on how they should be organised. It is clear that
universities need to change to accommodate the impact of technology on learning.
Very early in the E-College project it became apparent that e-delivery offered
exciting opportunities for delivering to the ever more diverse backgrounds of
students which the University recruits in response to the Government’s aim of
extending participation to 50%. The challenge for the University would therefore be
to mainstream the managed learning environment with all of its attendant questions
of funding streams and HR issues.
Unless we adopt more flexible forms of delivery we will lose students to more
responsive universities including private universities and virtual universities. The
economic argument, although compelling at this time, is not sufficient to incorporate
ICT into universities. Lea and Nicoll [14, p.6] highlight "ICT is commonly
promoted through governments and the media as mere 'technical improvements to
the learning systems'…. However this view masks the extent and complexity of the
changes taking place, of the requirements for substantial institutional change and of
reconfigured practices and understandings of pedagogy". The value of e-learning
needs to be shown on pedagogical grounds. Laurillard [13, p.241] argues that the
delivery infrastructure should never be in the foreground; rather it should be
supporting the dialogue on learning.
At the end of the four year project, the University needed to make decisions about
the future of e-learning. The next section of the paper describes the ways in which the
University embedded technology into learning and teaching.
4 Moving from e-learning to blended learning
At the time when the ECW project was first initiated in 2001, elearning was at its
height. Although the idea of blended learning was starting to emerge, Universities
across the globe were focussing on offering courses completely online. The
development of large scales elearning projects such as the UKeU, dominated the
sector. Many large scale elearning projects failed, and one of the criticisms was that
these projects pursued a narrow concept of e-learning and failed to realise that
students required human contacts and support in their learning [11].
Thus the outcomes of the ECW Project led directly to the university establishing
blended learning as a strategic goal and creating a blended learning unit to support the
academic delivery of courses through Blended Learning. The Vice-Chancellor’s
vision for enhancement of learning and teaching placed e-learning amongst the
highest gaols of his strategy fro the University.
Glamorgan is committed to the delivery of a first class

learning environment incorporating the highest standard of e-
learning, tutor facilitation and use of cutting edge learning
facilities…
The focus of this activity was to move from distance approaches to online learning to
the development of on-campus blended learning delivery. The University recognises
the transformative effects of e-learning and is proceeding from a position of strength
which has led to a major commitment to blended learning across all faculties.
Although the work in E College Wales (ECW) has been at the forefront of e-learning
developments, colleagues across all academic schools are now involved in blended
learning innovations.
4.1 Definitional complexity

There are definitional complexities and ambiguities surrounding such terms as e-
learning and blended learning. Blended learning as a term has gained considerable
currency in both business training and educational contexts (Oliver and Trigwell,
2005). Despite its widespread use the way in which it is interpreted varies and
requires further exploration and analysis [1].
From the outset, it is important to understand that the terms ‘e-learning’ and
blended learning are used in many different and frequently confusing ways;
definitions cover any activity from the simple use of email and PowerPoint
presentations delivered on campus through to sophisticated multimedia simulations
for use in stand alone study at any location in the world. It should be noted that
blending different approaches to learning is not a new idea, in business schools
traditionally a variety of pedagogic approaches have been used, for example lectures,
seminars, tutorials, case studies, role play, residential week end course, adventure
training and action learning groups. The difference here is that Information
Technology and the development of Virtual Learning Environments (VLEs) are used
to support the learning process.
118 Norah Jones
4.2 Continuum of Blended Learning
At the University of Glamorgan we have adopted a continuum of blended learning in

which the ‘blend’ is a spectrum that moves from basic ICT usage to intensive ICT
usage:
The University has opted for a blended learning approach, whereby each Faculty is
expected to embed technology into the learning experience within their own frames of
reference/context. This decision reflects the impact of external drivers such as the
growing familiarity with technology by many of our potential clients, Government
and employers’ initiatives, competition from other universities and HEIs as well as
the University’s desire to enhance the learning experience of its students and
providing accessible higher education.
5 Overcoming Barriers
The ECW project identified a number of barriers which impacted on the development
of e-learning in the case study university. The next section provides the main issues
arising form the project; this includes technical, cultural and learning and teaching
issues. These issues will be analysed next; this forms the main agenda for change.
5.1 Technical Issues
As a result of ECW the Vice Chancellor of the University invested in a central e-

support unit based in its central IT department and a blended learning team of
educational experts were appointed to join the Learning and Teaching Central team
(CELT). The e-support team (eST) comprises staff who have a breadth of professional
and operational experience in e-learning and the use of information and
communication technology (ICT). Based in the central information systems
department and the learning resources centre, the team is proficient in developing and
supporting the deployment of technology to enhance learning across a variety of areas.
The eST is also responsible for the delivery of staff development activities and
support, such as the Blackboard training sessions and a series of ‘How to…’ guidance
documents.
The eST team offers:
Customer Support Services
One-stop-shop for all ICT and e-learning support — customer focused with
access to online solutions and other support teams. Experience includes customer
service centre set up, proactive student support and provision of support and
advice 24/7.
Facilitation & Publishing

Practical advice for utilising technology to enhance learning within pedagogically
proven frameworks. Includes developing online content from both an
instructional and publishing view point.
Multimedia Development
Experienced in providing a range of graphical, audio and animated e-learning
solutions, technical knowledge of software tools, development capability for
games, quizzes, interactive simulations and case-studies. Able to advise in the
following areas: accessibility, technical issues, creating e-learning and
multimedia, good practice guidance, layout and design.
Systems Development and Support

Develop maintain and support the Blackboard and bespoke virtual learning
environment (VLE) systems. Provide advice on system functionality and
development. Assess software capabilities and provide in-house solutions to
development issues.
The LRC eST offers:

eResources Management
The team’s main aim is to help staff integrate into their teaching — in the
classroom or online — the most appropriate existing learning resources from the
Learning Resource Centre’s collections and beyond, to create a resource-rich and
easy to use learning environment for students. Advice is provided on the
availability of learning resources in different formats, and on the options for
linking to external resources from Blackboard. Guidance is offered on the
copyright implications of using content and permissions can be obtained on your
behalf if required.
Media Production
This team manages and supports the LRC’s media production facilities and has
extensive experience of creating video, audio and photographic material.
Production can be undertaken for staff, in the studio or on location. If staff prefer
to develop their own material the team then provides equipment and advice on
how to get the best results.
120 Norah Jones
5.2 Learning and teaching issues

The Centre for Excellence in Learning and Teaching (CELT) comprises of blended
learning pedagogical advisors, research and evaluation staff and staff involved in
providing advice on teaching and learning. CELT is committed to ensuring that
blended learning will be not be driven by technology but by the needs of the
University, its staff and students. It is important to note that although face to face
teaching will be the norm for most students it is anticipated that there will be a
continued growth and development in the use of blended learning.
During 2006/07 we were engaged in an HEA project to benchmark our blended
learning activities. Part of this benchmarking involved a survey capturing the
experiences of teaching staff in blended learning. The most common pedagogical use
of technology by lecturers was to post lecture notes, this accounted for 79% of
respondents. The next most common usage (67%) was to use the VLE (BlackBoard)
to make announcements to students. Over half the sample (56%) used the VLE for
testing and assessment. There was also a growing percentage of staff (45%) who used
the discussion group features. One could arrive at a negative conclusion with the
highest usage on information storage but this was well received by students, they
wanted their lecture notes online and accessible outside of the lecture times. In
addition they valued the opportunity for discussions to be held in face to face
meetings. This however may reflect current students’ demands but not be typical for
future students.
What will students expect in the future? There are great changes being made in
both primary and secondary schools with regard to Information Technology (IT) and
in addition more and more young people are used to resource based information
gathering. We need to be aware of changes in the pre-university sectors, professional
bodies and in the work place with regard to IT. As blended learning becomes core
rather than peripheral and optional, it is important that all staff are engaged in
reflection on their curriculum development and the learning and teaching methods
best used to deliver the curriculum.
5.3 Cultural Issues

The blurring of traditional departmental boundaries continue to pose many challenges,
especially to heads of department some of whom see this blurring as a threat to their
power. Collaborating with staff who are outside the security of shared disciplinary
assumptions expose staff to new and different ideas and approaches.
The overall conclusion from Christensen’s analysis [2] is that it is extremely
difficult for established organisations to adopt and embrace disruptive technologies. A
small number of organisations manage it, but a much larger number fail in the attempt.
“If e-learning is a disruptive technology, then the next two decades will require a
dramatic restructuring of higher education. We would prefer to see a planned
transition in which universities planned to learn how to implement e-learning, than to
wait for universities to be put out of business by new organisations that have been
quicker to understand what e-learning can be used for” [14].
There is no doubt that changes in the methods of teaching through the use of
technology require a changed management structure to support the change. As Elton
[3, p.219] stresses ‘new ways of learning require new forms of institutional
management’.
It is clear that the impact of e-learning, and in the future mobile learning, will
require universities to re-think fundamentally their strategies, in a range of areas
including human resources, estates, pedagogy, quality assurance, funding,
management and commercial and educational partnerships.
6 Conclusion
It is very important that the technology is not incorporated into programmes

uncritically. Many students especially those aged between 18-25 will probably not
want entirely e-learning courses or to study at a virtual university. The key features
of university life for many young people are the social and recreational activities.
In addition for any age group sustaining motivation in a virtual environment is
problematic. As I have indicated earlier there are a variety of e-learning models
and including in this is blended learning which offers one solution so that on-line
learning enhances the best of face to face provision.
Nonetheless the advantages of blended learning are clear; we now have
immediate access to information sources from around the world, we can more
easily collaborate with others globally and we can keep in touch with our friends
and family. ICT has changed so many aspects of our lives and as the networks
continue to grow so will our dependence on them grow. Already we have seen how
ICT allows us to create simulations and allows for powerful visualisation and
software offers new forms of interactivity through computer mediated conference
facilities. Rogers [17, p.358] emphasises,
"When each student has his or her own portal to the world's
knowledge, the role of the teacher as giver of information will forever
change. It is not a matter of if but when this transformation will
occur".
Like Rogers I believe this transformation is imminent and lecturers need to be

prepared and developed to accommodate changes to their roles. The challenge for
the university is to use blended learning and in particular our experiences from
ECW as an opportunity to re-examine pedagogies and focus on the needs of the
learners. The needs of learners have changed due to financial pressures an
increasing number of students are now working and studying full time; students are
pressing for more flexible delivery to accommodate these demands. In addition the
Government’s agendas on life-long learning, widening access and increasing
participation rates to 50% has increased the need for universities to respond more
flexibly to the requirements of a more diverse group. Blended learning allows
students to use interactive materials flexibly at any time. Advances in technology
are converging with the requirement for universities to be more flexible. As with
all IT driven change, e learning has developed more slowly and less smoothly than
its advocates predicted, but with far greater consequences, ultimately, than most
122 Norah Jones
assume. E-learning is challenging our education practices. The design of blended

learning needs to be grounded in sound education theory. We need to ensure that
we blend technological and pedagogical advancements. If we design programmes
on-line and ignore education theory then we are in danger of leaving learning to
chance. Lewin is purported to have said that nothing was as practical than a good
theory, I would add that there is nothing worse than a theory misapplied! We need
to get the model right, become more adventurous in terms of timing, structure,
assessment etc. There is no doubt that unless we adopt more flexible forms of
delivery we will lose students to more responsive universities including private
universities and virtual universities. It is clear that technology disrupts but we
must learn from each other the important lessons. We cannot build knowledge
without sharing information and good practices.
References
1. Bonk, C. and Graham, C. (eds.) The Handbook of Blended Learning, Pfeiffer San
Francisco (2006)
2. Christensen, C.M. The Innovator’s Dilemma, Harper Collins New York (2000)
3. Elton, L. New ways of learning in higher education: managing the change, Tertiary
Education and Management, Vol. 5 (1999) 207-225
4. Connolly, M., Jones, N. and Turner, D. E-learning: a fresh look, Journal of Higher
Education Management and Policy, Vol. 18 (3) (2006) 135 -147
5. Connolly, M., Jones, C. and Jones, N. Managing Collaboration Across Further and
Higher Education: a case in practice, Journal of Further and Higher Education,
forthcoming 2007
6. Hanna, D. and Associates, Higher Education in an Era of Digital Competition, Atwood
Publishing, Madison (2000)
7. Inglis, A., Ling, P. and Joosten, V. Delivering Digitally, Kogan Page London (2002)
8. Jones, N. From Here to E-ternity. Professorial Inaugural Lecture, University of
Glamorgan (2004)
9. Jones, N. and O’Shea, J. Challenging Hierarchies”, Higher Education Vol. 48 (3) 379-
395 (2004)
10. Jones, N. and O'Shea, J. A Case Study in Managing Change: introducing e-learning into
a university, Management, Vol. 29 (8) 29-35 (2003)
11. Jones, N. and Peachey, P. The Development of Socialization in an On-line Learning
Environment, Journal of Interactive Online Learning, Vol. 3 Winter (2005)
12. Jones, N., Morgan, A. & Turner, D. The E-College and Quality Assurance: the
irresistible meets the immovable, Quality Assurance in Education", Vol. 10 (4) (2002)
229-237
13. Laurillard, D. Rethinking University Teaching, Routledge London (2002)
14. Lea, M. & Nicoll, K. (eds) Distributed Learning: Social Cultural Approaches to Practice,
Open University / Routledge Falmer London (2002)
15. McLuhan, M. The Medium is the Message, Penguin Harmondsworth (1967)
16. Nunan, T. Ricmor, G. and McCausland, H. 'E-learning Futures' in Rich, T. (2001)
Perspectives, Policy and Practice in Higher Education, Vol. 5 (3) (2000) 68-77
17. Rogers, C. Freedom to Learn, Prentice Hall New Jersey ( 1994)
18. Sloman, M. The E-Learning Revolution, CIPD London (2001)
19. Wilson, A. Strategy for Management Development in Scott, P. (eds,) Higher Education
Reformed, Falmer London (2000) 29-44
The Marriage of Rousseau and Blended Learning: An
Investigation of 3 Higher Educational Institutions’ Praxis
Esyin Chew1, Norah Jones1, David Turner2

1
Centre for Excellence in Learning and Teaching (CELT),
2
Education Department at Faculty of Humanities and Social Science,
University of Glamorgan, United Kingdom, CF37 1DL.
{echew, njones2, dturner}@glam.ac.uk
Abstract. This paper sets out the central problem of current blended learning
research that it does not have an appropriate focus on educational theory. The
blended learning praxis in higher education can be understood in terms of
Rousseau’s educational theory is explained. The research methods for
collecting qualitative data from 28 academics in 3 universities are explained
concisely. The analysis and discussion of institutional practices are used to
identify best practice for blended learning. The role of the educator and student
on holistic learning are examined. Overall, the author urges that the focal point
of blended learning research should not merely on the innovative technology
and instructional design issues; educational theories remain the fundamental
foundation for any educationalist as well as for any educational technologist.
Keywords: Blended learning, educational theory, educational technology,

higher education.
1 Introduction
“…blended leaning could become one of the most significant developments of the
21st century.” [27, p.26]
Blended Learning is a phrase which is increasingly being used in higher education.

However, Whitelock [29] claims that blended learning has not gained ground with
theorists but is embraced by practitioners. Practitioners of technology and education
show more interest in blended learning than educational theorists do. Boyle [6] further
explains that the design and development of blended learning solutions should be
pedagogically driven.
Blended learning represents a real opportunity to create learning experiences that
can compensate for shortcomings in face-to-face delivery of programmes or in
technology-mediated learning. There has been an increasing number studies on
blended learning, which have examined such aspects as its effectiveness and its
impact learning in modern higher educational [22, 16, 21, 24] and how it promotes the
innovative dialogues and practices in the learning environment [26, 5]. The increasing
interest shown by researchers in these topics has gone hand in hand with the
124 Esyin Chew, Norah Jones and David Turner
development of technology, especially the emergence of the internet, rich media

objects [26], mobile technology [28] and artificial intelligence [30, 7].
Loveless [18] asserts that technological development is a cultural artifact in the
experience of students and educators. Technology is a tool of cultural artifact to
educator or student who applies it. It is varying from one individual to another and
one discipline to another. The usage and effectiveness of the technology is merely
cultural variety. Dewey [11] argues that if the learner is not trained in the right use of
the tools, there is grave danger that he may deprave himself and injure others. Croft [9]
explains Dewey’s idea in the following statement:
“…individuals inappropriately acquainted with the technologies in contemporary use

risk deterioration of self and damage society itself…” (p. 302)
Otte [21] further emphasises that the tool, technology, should be thought of merely
as a means, and that pedagogical ends should be paramount. Matthews [19] suggests
an interesting metaphor, namely that an educator without the pedagogy end is like a
sailor without a rudder - blown around by whatever fashions and technology which
dominate the current educational direction.
In the blended learning context, the learning process may be insignificant if the
technology adopted does not perform consistently or if it is not coherent with the
educational considerations. The main focus of work on blended learning is the
convenience of educators and students, together with certain considerations related to
pedagogical issues. However, this results in two critical questions: Are such
pedagogical considerations underpinned by adequate educational theories or are they
merely common sense but labeled as “learning theories”? Does such theory speak to
the needs of educators or students from different disciplines?
Hence, there is a need to explore educational theory and its relationship with
blended learning. The educational philosopher, Jean-Jacques Rousseau was chosen in
this research because he is the earliest and influential educationalist who scattered the
traditional educational ideas as to the replacement of tabula rasa or banking concept
by investigation and reason [10]. Blended learning practice in three higher
educational institutions is studied in this research. The paper explains the research
methods used to obtain the qualitative data. The principles of blended learning form
the perspectives of both practitioners and theorists are analysed and discussed.
2 The Marriage of Rousseau and Blended Learning
The systematic integration of technology into education may only take place when the
technologist is able to understand educational theories and embed them to meet the
needs of educators and students [4]. Technologies and effective content development
methods are the major concern in most of the blended learning practices, at the
expense of the educational theories underneath. How blended learning practice is
driven by educational theories is rarely considered.
In general, current blended learning practices in higher education underline
Dewey’s [12, 25] and Vygotsky’s [8, 14] conceptions of social interaction, social
The Marriage of Rousseau and Blended Learning 125
constructivism and a guided learning environment. However, this paper uses the
theoretical framework taken from Rousseau, who claimed that the goal of education
should be to cultivate humanity’s natural tendencies [23, 13]. This overarching
philosophy can be linked to the development of holistic blended learning practice.
Rousseau's central idea is to reject the authority of the teacher and emphasis on
individual reasoning and reflection. The student must be developed in the
consciousness of complete freedom. The student must committed to the reasoning
willingly. The learning process is not instilling by an authoritarian teacher but
developing according to the dictates of human nature, not to the authoritative
instructions from educator. Rousseau also contended that the three great teachers of
man were nature, man, and experience, and that the second and third tended to destroy
the value of the first [10]. The challenge for the educator is to create such a free and
natural learning environment, but with certain constraints, without learner’s
recognising the constraint. The constraint is meant to protect the learner from
disruption from “man and experience”. This is the challenge for the educators.
This perception should be inevitably a major consideration in blended learning
design and development. Student-led instruction using educational technologies can
be used to create an open and free discussion space within a context. The
collaborative tools such as online discussion boards and wikis allow participants to
commit to open reasoning, open investigation and reflection on knowledge. The
educator or the editorial board plays the role of facilitator to create the constraint, for
instance to delete disruptive information or to suggest guidelines to the learner in such
an e-environment in a non-authority manner. The learner is the decision maker and he
or she can choose which information to agree with or to take issue with.
Rousseau's educational idea is to educate people to bring out their natural goodness,
self-esteem, independence, compassion and equality [15]. This is difficult aim and
cannot be realised in the face-to-face classroom. The process of education is
thoroughly developmental and it shall follow the growth of human ‘nature’. Blogs and
Wikis allow the learner to practise social interactions as well as to express their
candid views and reflections. Through the reasoning and investigation process,
learning is cultivated in a natural way. The learner will thus be fully developed not
only in relation to their knowledge but also in relation to their ethical development
because the education process is by internal nature and not by external forces.
Whatever is posted on the web would always be there and open for review and
criticism. The learner will be conscious of her/himself as an independent individual,
responsible for her/his own thoughts and actions, completely independent from others,
yet able to interact with others and direct her/his life by reason.
Before the time of Rousseau educationalists had opined that the education is a kind
of top (educator) - down (learner) instilling concept. Rousseau enlightens the later
educationists to “autonomy as an educational environment”. Hence, the aim of this
paper is to study Rousseau’s pedagogical practice, and to consider how it might be
developed in the context of blended learning. Based on this view of pedagogical
practice, it will go on to investigate three current higher education institutions which
are embedding blended learning into their teaching, both formally and informally.
3 Research Method
Anderson [2] defines education as a process and therefore a research method which is
flexible and process-oriented is needed. According to Adelman [1], case study
methodology can be used to identify and evaluate the flexibility of reality especially
in the variety and complexity of educational purposes and environments. With these
boundaries, case study method is an appropriate choice for the study of blended
learning research. The present study incorporated case studies with qualitative-
quantitative interactive continuum methodology [20] because such a method
integrates the strengths of both qualitative and quantitative strategies. First, the
arguments of Rousseau are studied. In order to study the practice in higher education
institutions, three universities were visited and observed. 28 academic staffs and 6
students from varying disciplines were observed and interviewed. Interviews lasted
between 40 minutes to 2.5 hours. Qualitative as well as quantitative data has been
collected from their teaching and learning experiences. The names of the interviewees
as well as the institutions are anonymous due to considerations of confidentiality and
ethics.
The principal criterion in the selection of exemplary higher educational institutions
was not “which HEI represent the totality?”, but rather, “which group of HEIs can
provide a better understanding of the research questions?” and “which group of HEIs
reflect strong, positive and constructive examples of the research interest?” Given
these criterions, a diverse group of HEIs and faculties were needed. For instance the
traditional old universities and the new universities upgraded from polytechnics, and
the contrasting nature of disciplines related to technology such as a Faculty of
Computer Science and Faculty of Education; or the Faculty of Information and
Communication Technology and the Faculty of Humanities and Social Sciences are
proposed to meet the criteria stated above. To maximize the findings in a case study, a
range of formal and informal data collection instruments were used. These included
recorded face-to-face interviews and site visits to the case study institutions, in
addition to direct observation during the site visits. The responses have been analysed
and discussed in the following section.
4 Discussion and Analysis
28 academic staffs were formally interviewed and their disciplines are summarized in
the table 1.
The major technologies and e-learning system used in these three universities were
investigated and observed. Overall, the effort on e-learning development is mainly
initiated at institutional level. However, awareness o, and commitment to, blended
learning varies from one institution to another. The discussion on this section focuses
on cross-institution comparison based on Rousseau's educational ideas. The data were
used to identify and analyse institutional policies and pedagogical issues in three main
areas: (1) Blended learning awareness and best practices which exemplify their
institutional policies, together with actual implementation; (2) The role of lecturer
after embedding blended learning (3) The value of blended learning in promoting
holistic learning. Table 2 illustrates the overview and the preliminary observations
from the four months of data collection.
Table 1. The Disciplines of the Interviewees

University Category University A University B University C Total
/ Disciplines (Old university (New university (New university
with 102 years with 6 years of with 14 years of
of history) history) history)
Science, Engineering, 6 8 3 17
Computing and IT
Art, Language, 4 3 4 11
Education, Business,
Humanities and
Social Sciences
Total 10 11 7 28
Table 2. Overview of the Preliminary Data Collection

University A University B University C
Blended Less awareness on Awareness on e- Awareness of blended
Learning blended learning as learning space than learning across the
Awareness well as e-learning. blended learning. university.
Blended Not an institutional- Made a commitment Made a commitment

Learning wide commitment. to web-based mediated to the adoption of
Practices The blended learning learning. Blended Learning
practices are up to the Aimed to promote the across the institution.
faculty and individual use of ICT and IT- A three-year project to
academic interest. intensive learning embed Blended
through innovative Learning across the
courses and learning University’s provision
support systems. with high publicity.
Support the web-based A clear model, named
leaning space continuum of e-
intensively with learning is used.
workshop and training.
VLE Elearning, an in- WEBLE, an in-house Blackboard,
Implemented house built web-based built virtual learning Questionmark
Across learning management space and courseware Perception, in-house
Institution system but not well for the academics and built rich media or
publicized to the students. web-based
academics and Known by most of the applications based on
students. lecturers and students. bidding process
Many lecturers and initiated by the
students are not aware academics.
of the existence of Known by most of the
such system. lecturers and students.
Technology Emphasis on the Emphasis on the Emphasis on both

or Pedagogy? traditional technological technological and
instructional methods. concerns. pedagogical concerns.
4.1 Blended Learning Awareness and Best Practices

Figures 1 and 2 depict the awareness of blended learning in these three universities.
From the interview, 63% of the interviewees were aware of the concept and definition
of blended learning, although one interviewee out of the 63% has a misconception
about the definition of blended learning:
Interviewee B3: In my opinion, blended learning is same as hybrid learning, I think

they are similar.
N o t Aw a re o f
B le n d e d
L e a rn in g , 1 3 ,
37%
Aw a re o f
B le n d e d
L e a rn in g , 2 2 ,
63%
Fig. 1. The Awareness of Blended Learning
16 15
14 13
12
9
10
No Of
8 6 7
Interviewee Yes
6 No
4
4 2
2 0
0
Univers ity Univers ity Univers ity Total
A B C
Fig. 2. The Awareness of Blended Learning by Institution

A large numbers of the academics in University B have never heard of the term
“Blended Learning” as their institutional policy aimed to promote the use of ICT and
IT-intensive learning. Less concern was focused on the pedagogical issues, as the
institutional ethos was shaped by the fact that the institution has a background as a
vocational college, and it only became a university few years ago. Most of the
academics who understand and agree on the benefits of teaching mediated by
technologies were from University A and C. University C has obviously made a
commitment to the adoption of blended learning across the institution’s provision,
driven by a three-year project with high publicity such as a well-design blended
learning website with exemplars of case studies, monthly seminars and a road show. It
has a clear institutional policy to embed blended learning across the campus
compared with the other two universities. Such practice is inspired by a continuum of
e-learning model shown in the figure 3. This blended learning model shows the
concise blend of an individual as well as institutional growing in blended learning.
The self evaluation state and the direction ahead are well-defined for anyone who
adopts this model.
Fig. 3. Continuum of E-Learning [17]
The clear model embedding across the University C has raised the awareness of the
academics as well as students on blended learning. Such awareness is the initial and
substantive move for the changes. Few of the positive responses from the academics
after using technology to complement the face-to-face classroom are clearly presented
in the following:
Interviewee C3: I have used Blackboard as support material and engaging

dialogue with students. I found that very helpful.
Interviewee C2: I am quite excited about the prospect to be able to use

Questionmark Perception…I would like to be able to use something like that to
give students formative feedback and summative feedback as well…I am quite
interested in the technology that can be interactive.
Interviewee C6: Yes, I started to use Blackboard more this year compared with
last year...at the moment it is easier for me to go through the lecture, it's all
there...If there is something that I forgot, they are going to be there, I know all
the information is there.
However, sufficient resources such as technical and IT staff as well as

educationalist, financial support from the management are the key factors for
embedding blended learning across the institution after awareness rising. These are
the major concerns raised from the academics:
Interviewee C3: I think you got to sort out this resource…there must be some
sense of resources available; the university is either bite the bullets and pay for it,
or forget all about this.
Interview B8: …depending on the implementation, depending on the management

wanted the lecturer to be as a designer or executor. It would be a good thing if he
is a designer and back-up by a group of technical team…For example the
lecturer is the director of a movie and back up by a team of people, this is
excellent. If he is the director and the cameraman, and also in charged of the
lighting yet worrying about the sound effect, he will kill himself at the end.
The idea of “director and support team” has precisely explained the successful
mechanism in blended learning. Interestingly University A has no clear institutional-
wide policy to embed blended learning however their awareness came from individual
research interest, the culture and facilities at the faculty level, and above all, from the
individual passion for enhancing the learning and teaching quality:
Interviewee A2: …when you get your hands on the digital one such as computer,
I think you can't go back anymore. I mean you just have to use it…it is a so
effective!
Most of the interviewees agree with the teaching and learning will be enhanced
when they are mediated by technology. However it emphatically plays as a
supplementary tool to compliment face-to-face rather than replacing it, technology
can never replace the face-to-face instruction. This point is precisely stated by the
following interviewees:
Interviewee B2: To me, there is always the main stream and supplementary in
education. And the main stream will never change, from the very beginning till
the end, the things which are changed are the supplementary and tools.
Regardless the technologies, or chalks or paper, the main stream would never
change.
On the other hand, these two elements can be also seen as a symbiosis or conflating
for better teaching and learning experience:
Interviewee A3: Blended Learning make use technology and also the humanity
values, face-to-face… the technology alone is not enough, with the human alone
is still also have certain constraint, as we are now in a technological world. So
we need to combine both.
University B aimed to offer the learning experiences that instill in students a

consciousness of their role in a rapidly changing and technological oriented world
with a strong sense of professional and social responsibility. This aligned with
Rousseau’s view on education is how to prepare a better individuals to construct a
better society, not how to teach or learn effectively. However, such ideal is too
theoretical and yet to be imparted to the academics and students practices. The current
implementation of blended learning across the university is still in the “unawareness”
and technological-focus stage.
4.2 The Role of the Lecturer after Embedding Blended Learning

One of the research questions is to obtain the interviewees’ opinion on whether the
role of lecturer will be or will not be changed by embedding educational technologies
into the teaching and learning practices. The below table describes the quantitative
findings:
D epending , 2,
5%
N ot C hanged,
13, 33%
C hanged , 25,
62%
Fig. 4. Will the Role of the Lecturer be Changed after Embedding Blended Learning?
25 academics opine that their role would be changed after embedding blended
learning, whereas 13 academics assert an opposite view. This is a debating issue from
the pedagogical perspective. There is an essential view offered from academic
discussed next:
Interviewee A5: With the advancement of the Internet, the information rich
society, nobody claim that we know everything. We have students who are
hardworking, they know more than us. The lecturers are no longer dictate, we
merely facilitate.
The role of the lecturer will be changed from merely knowledge deliverer to
knowledge facilitator. In facts, Rousseau against this kind of “top-down” knowledge
instilling process in education centuries ago and it should not have happened today.
The academics who argue that the role will not be changed fall into the group of
senior and experienced educator. They view technology-driven teaching as the
different form of instruction method without changing the main role as an educator:
Interviewee B2: ... Before the time of white board and marker, people were using
black board and chalk...You will use Power Point after such technology had been
invented. In the future, you may use other new things when they are invented.
However, these technologies are helping us to conduct our classes in a better
way.... who is the one who organise and conduct the class? The lecturer is..
The argument here is: what are the fundamental role(s) of a lecturer? According to
Rousseau, the role of the educator is to create an autonomy and nature learning
environment, not by any techniques to instill the knowledge to learner. The learner
learns from self reflection and reasoning, not to the authoritative instructions or
“knowledge transfer” from educator. Hence,
“It's depending the role in the past. If your role is the "baby-sitter" to the students
then you would change. But if your role isn't spoon feeding at the beginning then
it would not be changed. To me, my role never change in the past ten plus years. I
don't care there is Internet or no”. ~ Interviewee A8.
Relatively an experienced e-moderator concludes this with an insight view:
Interviewee C7: When we first started delivering e-moderating course, we have

a very firmly view, that if you are a good tutor offline and you would be a good
tutor online…you are a good tutor means you are a good tutor whatever the
medium is.
This view shows that good teaching is driven by pedagogy and it is not platform
dependant. The technology as a means, and that pedagogical ends should be
paramount.
4.3 The values of Blended Learning in Promoting Holistic Learning
N ot S ure , 4,
14%
Y es , 15, 54%
N o, 9, 32%
Fig. 5. Blended Learning help in Holistic Learning?
Blended learning does help in evaluating soft skills and promoting holistic learning
in the sense of not only assess then task accomplishment but the hidden and high
order thinking and communication skill. The following academics’ explicate such
perspective:
Interviewee C7: if I monitoring their progress using some kind of online

discussion forum or blog for their experiences, I can see who is participating and
who is giving leadership and in which the leadership is being challenged and
where the conflict is…so I think they are very powerful tools…not about the task
but about the individual and powerful learning tools.
This experience actually aligned with Rousseau’s cultivation of human kind natural
tendencies. It can be realised and enriched educational technologies such as blog and
wiki in order to stimulate the natural tendencies such as social interaction, knowledge
sharing, free and open discussion and etc. It is merely difficult to be realised in the
face-to-face classroom with space and time constraint:
Interviewee A7: If the number of students is not very big. If you have a very big
class you will not have the time to read through whatever the students' post.
Rousseau’s philosophy of education, therefore, is not related simply at particular

techniques that best ensure that the students in a big class will absorb information and
concepts. His goal was to produce an education that maximized individual human
potential rather than restricted it. From such basis, the educator shall make sure of the
current technologies such as blog and wiki to create an open and independent learning
environment to reflect and to develop the human potential such as critical thinking
and to throw out their ideas. Through the dialogues and ideas exchange, the
relationship between educator and learner are brought closer:
Interviewee A3: To if you have the wiki or blog, they become more interactive
in the sense, more creative and encourage students for throwing out their
ideas…
Interviewee B8: I join some of the students’ blog from Journalism

study. …Sometimes I do give some comments and I feel my relationship with
them is much closer. I think this is helpful as I know their life and what
happening around them, as well as their views at political issues…This blog is
unique as they really stimulate the critical thinking in life.
Time constraint indeed is above all the critical considerations for the lecturer:
Interviewee B5: I am not sure how far such technologies can help but one thing I
can sure is with such things the students would ‘troubling’ me more than the past.
Without the understanding of the educational theory and individual passion in

education, such obstacles would lead to frustration and the most, drop out from
blended learning.
5 Conclusion
The new technologies have the potential to be both helpful and harmful, as do the
educational systems. Educators and educational technologist cannot merely geared by
all sorts of e-learning technologies but left behind the educational theory. Instead,
quick responses and reasonably understanding of educational theories are required [3].
The heart of the blended learning praxis lays the need for a deep understanding of the
trends and educational theories, only through that understanding can emerge initial
interest and passion.
Rousseau’s educational theory was chosen for this research. His central idea is to
reject the authority of the teacher and emphasis on individual reasoning and reflection.
The student must committed to the reasoning willingly and in an autonomous

environment. The learning process is not instilling by an authoritarian teacher but
developing according to the dictates of human nature, not to the authoritative
instructions from educator. Thus, this research investigated the current praxis of
blended learning in higher education based on Rousseau’s idea. The qualitative data is
anlaysed and compared. The result reflected a clear institutional policy will promote
the awareness across the university, follow up individual interest and passion on
teaching and learning. Resources and support team in place is the critical successful
factor. The role of the lecturer will be enhanced if the educator does not realise the
values of “creating an autonomy and nature learning environment, not by any
techniques to instill the knowledge to learner” initially. In addition to this, blended
learning does help in holistic learning with bringing closer the relationship between
educator and learner with autonomous communication, and to stimulate the natural
tendencies without space and fixed-time constraint. In contrary, language barriers and
the abuse usage may cause frustration and above all, time constraint is the major
challenges for implementing blended learning.
As a conclusion, it is asserted that the educational theories remain the fundamental
foundation for any educationalist as well as educational technologist. With such
understanding and passion would triumph over the challenges mentioned above.
Blended learning researcher, therefore, is urged to meekly shift the focal point from
innovative technology and instructional design issues, to educational foundation, as it
is.
References
1. Adelman, C., Kemimis, S., Jenkins, D. Rethinking Case Study: Notes from the Second
Cambridge Conference. IN H., SIMONS (Ed) Towards a Science of The Singular. Centre
for Applied Research in Education, University of East Anglia, (1980) 45-61
2. Anderson, G. Fundamentals of educational Research (2nd Edition). RoutlegeFalmer
London (2004)
3. Aspy, D., N., Aspy, C., B. Toward Effective Advocacy for Humanistic Values, Vol. 37(2),
Journal of Humanistic Education and Development (1998) 85-95
4. Bailey, G., Pownell, D. Tecnology Staff-Development and Support Programs: Applying
Abraham Maslow’s Hierarchy of Needs, Learning and Leading with Technology, Vol.
26(3) (2006) 48-64
5. Barker, T. Computer Adaptive Assessment and Its Use in the Development of a Student
Model for Blended Learning, Blended Learning – Promoting Dialogue in Innovation and
Practice, University of Hertfordshire, First Annual Blended Learning Conference.
Learning in Higher Education and Workplace Learning Settings’ Hatfield (2006)
6. Boyle, T. A Dynamic, Systematic Method for Developing Blended Learning,
Education,Communication and Information, Vol. 5(3) (2005) 221-232
7. Chew, E., Jones, N, Generic Model of Computation for Intelligent Computer-aided
Assessment Progress (iCAP), University of Loughborough: Tenth International Computer
Assisted Assessment (CAA) Conference, 4-5th July, Loughborough (2006) 107-124
8. Cortazzi, M., Hall, B. Vygotsky and learning. Education Libraries Journal, Vol. 42 (1999)
17-21
9. Croft, R. S. What Is a Computer in the Classroom? A Deweyan Philosophy, Journal of
Educational Technology Systems, Vol. 22 (1994) 301
10. Cubberley, E. P. The History Of Education, Blackmask Online (2003)
11. Dewey, J. Democracy and Education: An Introduction to the Philosophy of Education,

The Macmillan Company New York (1922)
12. Ehrlich, T. Reinventing John Dewey's "Pedagogy as a University Discipline". The
Elementary School Journal, ProQuest Psychology Journals, Vol. 98 (1998) 489
13. James, J. D. Jean-Jacques Rousseau In: James, F. & Bradley, D. (ed.), The Internet
Encyclopaedia of Philosophy (2006) URL: http://www.iep.utm.edu/ [Accessed: 3.3.2007]
14. Jennifer, A., J. S., Monfries, M. Motivation to Learn in University Students: Links with
Vygotsky's Assisted Discovery. European Conference on Educational Research. Bath
(1995)
15. Johnston, I. Introduction to the Eighteenth Century and Rousseau's Emile. Malaspina
University-College, Media Relations & Publications Department. (1996)
16. Johnson, J., Tang, M. Integration of e-Management, e-Development and e-Learning
Technologies for Blended Course delivery, AACE Journal, Vol. 13(2) (2005) 185-199
17. Jones, N. Chapter 13:E-College Wales, A Case Study of Blended Learning. In: C. J. Bonk
& C. R. Graham (eds.). Handbook of blended learning: Global Perspectives, local designs.
Pfeiffer Publishing CA (2006)
18. Loveless, A., M. Where do You Stand to Get a Good View of Pedagogy?, Journal of
Technology and Teacher Education, Vol. 8(4) (2006) 337-349
19. Matthews, M. Knowledge, Action and Power. IN MACKIE, R. (Ed.) Literacy and
Revolution: the Pedagogy of Paulo Freire. Pluto Press London (1980)
20. Newman, I., Benz, C., R. Qualitative-Quantitative Research Research Methodology:
Exploring the Interactive Continuum, Southern Illinois University Press IL (1998)
21. Otte, G. Using Blended Learning to Drive Faculty Development (And Vice Versa), Sloan-
C Series, Vol. 6 (2005) 71-84
22. Ross, B., Gage, K. Global Perspective on Blended Learning: Insight from WebCT and Our
Customers in higher Education. In: C. J. Bonk & C. R. Graham (eds.). Handbook of
blended learning: Global Perspectives, local designs. Pfeiffer Publishing CA (2006)
23. Rousseau, J.-J. Emile, The Project Gutenberg EBook (2004)
24. Salmon, G., Lawless. Chapter 28: Management Education for the Twenty-First Century,
In: C. J. Bonk & C. R. Graham (eds.). Handbook of blended learning: Global Perspectives,
local designs. Pfeiffer Publishing CA (2006)
25. Simpson, D. J. John Dewey's Concept of the Student. Canadian Journal of Education, Vol.
26 (2001)
26. Smith, C. A Rich Media Rapid Prototype Approach to Developing Student Online
Resources, Blended Learning – Promoting Dialogue in Innovation and Practice,
University of Hertfordshire, First Annual Blended Learning Conference. Hatfield (2006)
27. Thorne, K. Blended Learning: How to Integrate Online & Traditional Learning, Kogan
Page London (2003) 18
28. Wagner, e., d. Chapter 4: On Designing Interaction Experiences for The Next Generation
of Blended Learning. In: C. J. Bonk & C. R. Graham (eds.). Handbook of blended
learning: Global Perspectives, local designs. Pfeiffer Publishing CA (2006)
29. Whitelock, D. Blended Learning: Forget the Name But What About The Claims? In:
Whitelock, D., & Mason, R. (eds), Bledned Learning. Special Issue of Education,
Communication and Information (2004)
30. Zhang, T., Z., Fu, Y., G., Shen, R., M. Improve Question & Answer System By Applying
Genetic Algorithm, Proceedings of the 3rd International IEEE Conference on Madune
Learning and Cybernetics, Shanghai (2004) 2317-2321
Developing an On-line Medical Curriculum
Management Platform
Medical Information Technology, Faculty of Medicine,

The Chinese University of Hong Kong, HKSAR
{jennyfang, franciswong, raymondchu}@mit.cuhk.edu.hk
Abstract. In the context of medical education, the current manual system of

managing and updating teaching activities is not only time-consuming but also
inefficient and inconvenient. In this paper, we present an account of the
development of a web-based Medical Curriculum Management Platform
(MCMP) that provides a flexible and comprehensive e-management
environment designed to assist in the management and facilitation of the
learning process from an administrative perspective. The system solves most, if
not all, problems related to the administration and integration of a web-based
multi-dimensional curriculum map. It also provides functions that cannot be
achieved manually, such as a bird’s eye view of the entire Faculty curriculum
map and students’ group/rotation roster. This overview can be accessed using a
simple tap, update and distribute approach, whereby various kinds of
information can be processed and posted with a minimum of effort.
Keywords: medical education, administration, e-management, curriculum map.
1 Introduction
This paper reports on the development of a web-based medical education

management environment that embraces, from an administrative perspective, an
innovative approach to the creation and management of complex organizational data
and the retrieval and use of this data in a simple way.
Medical education provides a particularly unique learning environment because of
its multi-faceted nature. In comparison with other disciplines, a medical curriculum is
extremely complex, with teaching and training activities scheduled in multiple
locations arranged across the various academic departments, hospitals and wards. For
selected groups of students, various teaching activities need to be arranged on the
same day but in different locations and time-slots. Since many members of the
medical teaching staff are also practicing doctors in the teaching hospitals, it is quite
common that changes to course schedules have to take place whenever urgent medical
cases arise. This necessitates regular updating of teaching schedules. Similarly,
student group arrangements need to be updated or changed according to the different
modules (or time periods) and teaching programs the students are undertaking.
The profiles of those involved in a medical education programme are many and varied,
and include not only the students and lecturers (who can be practicing doctors,
specialists, nurses and so on), but also university administrators and hospital staff.
The complexity of a curriculum programme involving such a diverse range of
stakeholders and constantly varying factors necessitates an extremely efficient
administration system, in which the role of the administrators is particularly crucial to
the successful management and implementation of the educational programme.
In the past, the arrangement and management of all teaching activities relied on a
somewhat outmoded manual system of updates, such as memos on notice boards,
mass emails or phone calls. Essentially this meant that all staff and students had to
regularly view all notices and announcements, in case there was something relevant to
their own programme of activities.
Due to the complexities of time and location, traditional updating methods also
made it difficult to establish clear and effective communication among departments.
If course coordinators, course designers and senior managers of faculty wanted to get
an overview of the entire curriculum arrangement, they had to obtain and consider
multiple documents from different departments. The difficulty of simply gathering the
relevant information meant that course content overlap often occurred despite the best
efforts of the responsible management. Under the traditional curriculum management
organization, lecturers basically taught their subject areas, and had little chance of
getting to know or understand what was being taught in other departments or in
different student years.
In order to try and tackle such problems as course content overlap and the
complexity of interdepartmental communication, the Faculty of Medicine decided to
implement a new curriculum commencing in 2001/02. The revised curriculum is
organized and delivered as one that is faculty-owned (not departmentally developed),
integrated (within and among years for basic and clinical sciences) and system-organ
based (not discipline-based). However, the success of the new curriculum approach
was significantly dependent on the contributions of the administrative staff in
facilitating the organizational arrangements necessary to ensure that staff and students
were not only in the right place at the right time, but also had access to essential
learning assets. The role of administration was extended to assume organizational
responsibility for the entire curriculum, and it was recognized that comprehensive
management support was essential. It was therefore decided that the development of a
web-based management system would be the most efficient way to proceed.
The challenges faced in the development of such a system were how to use
technology to:
• develop a robust curriculum management structure to provide the necessary
data/information via a central platform;
• provide easy accessibility to the data/information;
• make the organizational structure transparent to all users;
• improve the management of administration and learning assets (repositories of
learning objects/outcomes, teaching materials, timetables, students’ work and
assessment, and so on).
Although established environments such as WebCT and Blackboard have been

invaluable in enhancing the presentation modes that facilitate learning, and have
138 Jenny Fang, Francis Wong, Raymond Chu
consequently broadened our concepts of the possibilities of web-based learning

environments, their major focus is the provision of an e-learning environment and the
delivery of course materials at the individual course level. The new medical
curriculum was prefaced on the management of the curriculum across all year levels
and across multiple credentials. Current educational-support technology fails to
adequately cope with the challenges of maintaining, in an effortless way, the
necessary interactions and information essential to facilitating the creation and
management of a medical educational learning environment. [1]
Medical education embraces a wider learning community than that normally
envisaged in the development of online learning platforms. It is particularly unique in
that the notion of ‘classroom’ is extraordinarily dynamic. Medical students learn
within an environment that is constantly varying. Learning takes place in and around
lecture theatres, laboratories, tutorial rooms and hospital wards, as well as via on-site
experiences and a multitude of other exposures that all contribute to the understanding
of the discipline, and of the profession, of medicine.
The Medical Curriculum Management Platform (MCMP) was therefore developed
to cater for these diverse and demanding requirements. It was designed not only to
provide easy access to data for various users, but also to make the entire medical
curriculum transparent for the whole learning community. This system reflects a
unique approach an administrative point of view to the management of a medical
education curriculum.
2. Overview of the MCMP Model
The Medical Curriculum Management Platform was designed for the Faculty to
facilitate management of the new curriculum activities, as well as to embrace
information technology to reduce the administrative workload.
The MCMP was developed as a platform for students, lecturers and administrators
to share teaching and administrative information. The primary aim was to facilitate
the management of the personnel and resources involved in medical education
environment rather than to focus on the pedagogical programme of the degree. It is
not an e-learning system, and does not provide any features to create learning
planning, learning activities design, or learning flow control. However, it does
provide a link function to other e-learning systems provided by the university; as well
as a platform for student collaboration, and for teachers to distribute learning
materials and view students’ work.
The system serves five different types of user, namely, medical students;
professors, lecturers and tutors; adjunct tutors; administrators; and site administrators
(Fig. 1).
Fig. 1. System overview
The general functionalities of the system are extensive; some examples include:
• providing a web-based GUI for ease of interaction with the database;
• Providing users with an overview of all courses within the curriculum;
• providing a function for the administrator to create and post announcements
for specific student years or lecturers;
• providing a function to allow the administrators and lecturers to upload or
distribute teaching materials to specific students;
• allowing users to use the Form to Mail function to create a HTML form that
sends email to specific student groups, lecturers or administrators;
• providing a Teaching Event calendar and table to enable users to easily check
their teaching/learning schedules;
While the importance of the software analysis and design process is indisputable
with respect to ensuring that a constructed computer system realizes all its
functionalities, the design of the user interfaces is equally important to the success of
a web application as it largely determines the extent of user acceptance. As a
consequence, a great deal of time was spent in designing the user interface, with
special regard to the factors of usability, visualization, functionality and accessibility.
[2] The layout of each user interface was predefined in the design documents, which
were developed to ensure that all user interfaces were intuitive and easy to use with
help documentation.
Fig. 2. A administrator homepage

The MCMP interface is primarily divided into three modes:

¾ Administrator mode:
The administrator mode displays the GUI for the department administrators
and site administrators to manage the website and its contents. The content of
each user interface in this mode is dependent on the type, and the granted
access control, of the administrator. Each administrator can only access those
modules that they are allowed to use. A sample administrator page is shown
in Fig. 2.
¾ Lecturer mode:
The lecturer mode displays the GUI for lecturers to view their own schedules
and the teaching schedules of other years or panels. The interfaces are
designed to support lecturers in the management of their teaching activities
and their communication with students, administrators and other members of
staff.
¾ Student mode:
The student mode displays the GUI for students to allow them to access their
own schedules, logbooks, exam results, and other learning-related activities
(Fig. 3). The interface content for each student is dependent on their study
year and their learning group.
The general functionalities of the system are extensive; some examples include:
• providing a web-based GUI for ease of interaction with the database;
• providing users with an overview of all courses within the curriculum;
• providing a function for the administrator to create and post announcements
for specific student years or lecturers;
• providing a function to allow the administrators and lecturers to upload or
distribute teaching materials to specific students;
• allowing users to use the Form to Mail function to create a HTML form that
sends email to specific student groups, lecturers or administrators;
• providing a Teaching Event calendar and table to enable users to easily check
their teaching/learning schedules;
The flowcharts for lecturer and administrator access interfaces are decidedly similar,
but these users have more comprehensive access rights. Two typical functions
included in the administrative mode are the Teaching Event management and
Module/Panel Mark management features.
The Teaching Event management feature is designed to:
• show updated teaching events and provide an overview of the schedules of
each teaching group;
• provide an interface where administrators can easily create teaching events for
specific student groups in a particular frequency period (e.g., once only, once
per week, every weekend, once per module, etc.);
• allow administrators to modify event content and update variables (e.g.,
location, time, date and tutors) when schedules need to be refined.
The scope of this feature is captured in Fig. 4.
142
Fig. 3. System flowchart of student homepage

Fig. 4. Add Event(s) page for administrators

144
Fig. 5. Student exam report for administrators

Fig. 6. Layout of printed student report

The Module / Panel Mark management feature is designed to:

• assist administrators in the management of student results (the system allows
on-line input of results, upload of results using an upload function in Excel
format, as well as distribution of results to Faculty and students);
• provide on-line report viewing and report printing function (administrators
can print out the student examination report for faculty use).
The scope of this feature is captured in Fig. 5 and Fig. 6.
The Teaching Event management and Module/Panel Mark management features
are invaluable in reducing the workload of administrators and providing a
standardized framework for the centralization and presentation of data.
2.1 Development tools
The MCMP is a dynamic web application, and as such necessitates the

implementation of several server side programs in order for the system to retrieve data
from the central database server. We are using an MS SQL Database Server as the
central database server. The whole system was mainly developed using Adobe
Coldfusion Markup Language (CFML), combined with javascript, HTML and CSS.
CFML is a development tool for creating web-based applications and provides a set of
functions to assist the web developer to develop complex interactive web applications.
Common features provided within CFML include:
• dynamic, data-driven web pages;
• database connectivity;
• sophisticated data collection and processing;
• powerful report generation;
• email function;
• rich and engaging user interfaces;
• access to any back-end system.
Additional tools used by the development team included Macromedia Dreamweaver,
Flash and Firework.
2.2 Limitations of the system
Although extensive time was spent on the design and development of the system,
including the design of a comprehensive test plan and consequent exhaustive testing,
there are a number of factors that can affect the performance of the MCMP and which
need to be constantly monitored by the development and maintenance team. For
example:
• the accessibility of the GUI may vary depending on the client OS and browser
(beyond our control);
• the performance of the system can be affected by factors such as network speed,
configuration of the client machine, types of application and database server, and
the server configuration and tuning;
• the size of materials files to be uploaded and downloaded vary depending on the
limitations of network speed, server timeout setting and the uploading component
used.
3. Implementation
3.1 Student perspective
A medical curriculum is translated into a teaching programme by means of system

panel and skill modules, medical clerkship, surgical clerkship and rotation among
clinical departments. Students are divided into several core groups according to
clinical rotation, and then sub-divided into sub-groups under each rotation. Each
student therefore usually has more than five identities and needs to deal with group
members from different panels.
Before the launch of the MCMP, curriculum and course administration materials
were primarily circulated as paper handouts. In order to have an overview of their
daily schedules, students had to check handbooks from different panels and then
prepare their own timetables. As even those classes in the same panel or clerkship are
independently categorized into different sub-topics and listed in separate tables,
students experienced great difficulty in preparing their timetables at the beginning of
each school year. The MCMP relieves these difficulties by preparing a timetable for
each student dynamically according to their rotation period.
Students can check schedules and contact group members online via forums and
group email. Student logbooks can also be updated online. The online logbook was
especially designed to collect and store data relating to the procedures and
examinations that students have taken part in during their medical and surgical
clerkship. This data can be a useful reference for teachers and administrators when
monitoring student progress and suggesting further activities relevant for each student.
3.2 Lecturer perspective
Lecturers in the Medical School are not only teachers but also doctors in the hospital.
Their daily schedule is tight and the time that they have available for students is
limited. It is hard for lecturers to contact students after class as students disperse to
other classes or other panels. It is also difficult for lecturers to remain up-to-date with
the academic progress of their students.
The MCMP relieves these problems by providing an extensive web-based
collection of student curriculum information, such as student photographs, grouping
structures, contact information, logbooks, assignments and event schedules. Lecturers
can easily retrieve information about the students in their classes directly from the
web.
The MCMP’s online forum also provides a platform for lecturers and students to
communicate and keep in touch. This is important as both lecturers and students have
extensive teaching/learning and hospital responsibilities which make it hard to

schedule face-to-face appointments.
3.3 Administrator perspective
The MCMP recognizes that administrative staff play an important role in the
functioning of the whole curriculum by performing clerical tasks and data entry,
serving as a bridge between students and lecturers, and generally making sure that
work and communication occur efficiently and speedily.
In the past, student information was not disclosed inter-departmentally. This meant
that it was hard for administrative staff to access the information they needed when
arranging student events involving other departments. With MCMP, all student
information related to the curriculum is collected together in one platform, thus
enabling administrative staff to easily retrieve the information they need.
Instead of circulating memos, administrative staff can now simply post
announcements to the web using MCMP announcement tools. Announcements can be
posted to a specific audience (e.g. Year 2 and Year 3 students and teaching staff). If
amendments are needed, administrative staff can edit MCMP announcements online
after they have been posted. Through MCMP, administrative staff can also update
schedules and distribute exam results to a target audience in real-time. These tools
help administrators deal efficiently with their day-to-day work responsibilities and
simplify communication with lecturers and students distributed over a large number
of locations.
Feedback from administrative staff indicates that the MCMP assignment
submission function has also proved helpful in simplifying their work and saving time.
Administrators no longer need to email students individually or collect assignments
through a collection box, but can simply check online to ascertain whether students
have handed in assignments or not. If students need to be reminded of assignment due
dates, it is no longer necessary for administrative staff to telephone them using
possibly outdated contact lists, as announcements can now be posted or sent as emails
to selected individuals or groups of students.
3.4 Challenges
Although the MCMP can schedule all teaching events before the commencement of
the school term, re-scheduling due to class cancellation throughout the school year is
unavoidable due to the fact that lecturers are also doctors responsible for handling
emergency cases in the teaching hospital. Although use of the MCMP can ameliorate
this situation, sometimes hospital cases are so urgent that even if administrative staff
contact students individually through the system, class cancellation occurs too late to
properly inform students of the rescheduled class.
A tool such as the MCMP has many useful applications. However, the major factor
determining its acceptance is the user-friendliness of its interface design. Experience
has shown that end-users are reluctant to use a new system if the operation is
complicated or requires a lot of steps. [3] If the user is uncertain about what to do or
what the outcome will be, they will refuse to use the new tool and will very quickly
revert to a manual mode of operation. In order to prevent this early rejection, we must
clearly recognize the needs of the user and design the interface from a non-technical
user’s point of view. Not only do we need to provide a useful and efficient tool, we
also need to help educate the user in the use of that tool. Providing support workshops
to end-users allows us to obtain feedback and make system amendments where
necessary. A system can only be successful if it is accepted by its users.
4. Conclusion
The MCMP is an unique and innovative administrative platform that has increased the
efficiency of the medical department and decreased the workload and work hours of
both staff and students. The MCMP helps lecturers to communicate more easily with
students, administrative staff to work more efficiently. It also enables students to
organize and schedule their daily routines with less time and effort. The result is that
throughout the whole department less time and resources are wasted and more time
can be spent on teaching and studying. Although the MCMP was not developed to
provide new methods for e-teaching and e-learning directly – this requires initiatives
from the teaching staff themselves – however, it does provide support for e-learning
in as much as it is able to access any back-end system. What the MCMP has done is
to centralize curriculum information from different departments so that all users –
students, lecturers and administrative staff – can easily retrieve the information they
require online, rather than by making phone calls or sending memos to different
departments. The information collected in the MCMP is extensive and complex, it
positively enhances the learning environment and facilitates communication within
the Faculty. The MCMP provides an essential platform to underpin any educational
pedagogy.
5. References
1. Alexander, D., (2002). An accessibility audit of WebCT. Downloaded from

http://ausweb.scu.edu.au/aw02/papers/refereed/alexander/paper.html March 21, 2007.
2. The World Wide Web Consortium (W3C) - Introduction to Web Accessibility
http://www.w3.org/WAI/gettingstarted/Overview.html
3. Krug, Steve, (2006). Don’t Make Me Think: A Common sense Approach to Web
Usability, New Riders
Concepts of Blended Learning for Different Content
Types
Andreas Henrich and Stefanie Sieber
University of Bamberg, Faculty of Information Systems and Applied Informatics,

Chair of Media Informatics, D-96045 Bamberg, Germany
{andreas.henrich, stefanie.sieber}@wiai.uni-bamberg.de
Abstract. The immense effort for creating e-Learning content is one of the
limiting factors in the proliferation of e-Learning technologies. In the present
paper we therefore describe two concepts for Blended Learning that try to
achieve a good balance of creation as well as maintenance effort and the benefit
for the students. These two concepts are applicable for different classes of
courses. A text-based concept is described for courses dealing with fundamental
and mature contents and a recording-based concept for more advanced topics
with a high alteration rate. The concepts as well as their technical
implementation and evaluation results are presented.
Keywords: course types, LaTeX, recordings, moderate creation and

maintenance effort
1 Introduction
It has become a common practise to support learning with different kinds of electronic
systems, no matter if it comes to face-to-face study paths with full face-to-face
lectures or virtual study paths without any face-to-face phases. The spread of
electronic support reaches from just preparing digital learning material to providing
complex learning management systems (LMS) implementing elaborated didactic
concepts. There are many studies showing the advances of e-Learning as well as
evaluating e-Learning challenges and how to exploit its full potential, e.g. [6].
Of course a lot of research has been done in this area so far. Existing and
implemented concepts and studies cover all aspects of e-Learning. There are among
others theoretical concepts dealing with special issues, research reports on strategies
and concepts improving Blended Learning like [2], or methodologies for designing
higher-education by implementing complex systems, e.g. [1].
However a lot of educational institutions suffer from a lack of resources, concerning

financial and personnel support, and lots of known concepts and systems cannot be
set up or used due to these restrictions. In spite of that it should be possible for
smaller institutions to set up a quality system supporting teaching and learning in a
satisfying way.
The overall aim of this paper is therefore to present an approach that is well-
elaborated, combining traditional face-to-face teaching and evolving technical
possibilities, covering important factors of teaching and learning and most
importantly not having an immense need for additional resources. The technical basis
we used to implement our concept is an Open Source LMS, in our case Moodle1,
which is used university-wide2 and administrated by one person employed part-time.
The presented concepts should therefore be easily adaptable and practical for a lot of
institutions without special claims on resources.
The nature of Blended Learning always depends on various factors like scope of face-
to-face and virtual phases as well as obtainable working time. Since having a unique
concept for every single lecture cannot be efficient we were developing a concept to
transform plenty of practises, conditions and requirements to a few well-elaborated
but homogeneous forms.
Needless to say there are some general conditions Blended Learning should meet and
that therefore should be covered by every instance of the desired concept. Put another
way, we can define those conditions as general goals to be accomplished with our
concept in order to make it usefully applicable. Those goals have to be determined at
least from two sides, the lecturers' and the students' side.
Concerning the lecturers' side it is indispensable that lecture notes, respectively

provided learning material, can be created with little additional effort and maintained
with even less effort. In addition there should be a productive environment for
lecturers to absolutely eliminate dispensable time invested in the building process but
extend the time that is available for qualified supervision of students. Taking a look at
the students' side, an important demand is the request for printable material. Despite
the various advantages of online content, students have a general need for printable
material. Naturally students in Blended Learning courses require familiar advantages
of e-Learning as well. Another goal we added that cannot be accredited one of the
sides is the possibility to integrate the result into our university-wide LMS.
The concept we were developing should cover a wide, though specific span of
Blended Learning forms. To avoid restrictions due to special characteristics of study
paths and courses and to allow a detailed concept fitting the needs of all participants,
we decided to split our courses into two different types by the means of different
content types that are presented in the courses.
It is a practical experience that on the one hand there are courses having almost the
same content over the years without being obsolete, and on the other hand there are
courses that can hardly come through one semester without the need to be changed.
Depending on the subject, there are usually courses or course fragments containing
very fundamental and mature issues. The content of those courses is therefore almost
1
Official Moodle-Website http://www.moodle.org
2
Virtual Campus of the University of Bamberg is available at http://vc.uni-bamberg.de/moodle.
152 Andreas Henrich and Stefanie Sieber
static with a quite low alteration rate. Of course most subjects also have courses
covering more advanced and therefore often newer issues, e.g. actual research topics
and insights. Permanent monitoring of treated topics and subsequently a high
alteration rate are essential to uphold the quality of those courses.
Since it has been shown that delivering well-elaborated learning material itself is not
sufficient to provide high-quality teaching and that there are other factors like
monitoring the students' success and understanding that need to be concerned, we are
adding auxiliary components to our courses. There are various analyses on how to
ensure profound high-quality teaching, e.g. [4]. As mentioned before there are
different factors like time and resource restrictions for a lot of smaller educational
institutions that will not allow a widespread approach to cover all factors desirable.
Therefore we are trying to consider the most important factors like ensuring profound
understanding and supporting the students' ability to critical self monitoring and
review as far as possible by adding further components —such as self-test— to our
concept for both types of courses.
In the following two sections the differing concepts for the two types of courses as
characterised above are explained and reasoned. As finally shown in section 4 the
concepts turned out to be successful and can easily be adapted.
2 Text-Based Courses
Like outlined above, one type of courses are courses containing fundamental issues
setting up the base for following advanced courses. It is obvious that basics of a
specific well delved subject do not change very often or very fast so that the
presumption of almost static courses is realistic. The idea for this kind of courses is to
structure the subject matter like a book, to take the specific characteristics mentioned
into account. Accordingly it is feasible to provide the subject matter in different
digital forms.
To realise this concept we needed a building tool that is at best commonly known,
easy to use and of course providing the desired outputs. Conceivable outputs include
especially PDF- and HTML-documents. For this reason we evaluated a “single source,
multiple publishing” approach that builds different kinds of output from one XML
source [3]. An approach like this seemed to bring the brightest prospects.
Unfortunately creating and updating the text-sources directly in XML is pretty
inconvenient. The effort needed to maintain text itself is additionally amplified, e.g.
due to lots of formulae and graphics to be included. At the same time we also verified
the idea to provide the learning material in different digital forms by questionnaires
handed out to our students. The result strengthened the assumption that our web
server statistics allowed. Students not only definitely preferred printable content but
also saw no need for additionally offering the curriculum as interactive online content.
All these factors and experiences lead us to discarding the “single source, multiple
publishing” approach in the end and brought about the decision to offer PDF-
documents as learning material concerning the curriculum. The decision for PDF-
documents is motivated by the fact that the PDF-format is an established platform
independent format that has good prospects. PDF already offers mature possibilities
to provide comfortably structured and interactive documents. Besides, further
development is already taking place and in particular the interactivity options are
expected to be improved and increased in future versions. The following conclusion
to use LaTeX as building tool was just a logical step. LaTeX is a well known and
widespread format especially for academic writing and it also provides very good
integration of formulae and extensive possibilities to include graphics.
We employ a specific format that provides additional benefits, e.g. marginalia, which
are complemented to improve the structure and clearness of the text. The whole
material is also completed by a wide index and an extensive bibliography where
literature is directly linked if possible. Besides, we focus on adding lots of additional
links to benefit interactivity; there are anchors referring to further parts of the learning
material, concrete links to related parts of our LMS and a large number of links to
external websites. We are convinced that generating PDF-documents from LaTeX-
sources is a good choice because there is the possibility to have a really comfortable
interactive version of text. PDF-documents therefore can be used as online and print
version all at once, providing the benefits of both versions. Generally the PDF-
documents are uploaded to our LMS and provided to the students this way.
The concept to use the material of those courses embraces two phases. First, students
are working through the provided learning material on their own. This approach
replaces the normal face-to-face lectures and therefore, as assistance, a schedule
showing the optimal allocation of learning is handed out to the students. Second, to
supplement the self-organised learning there are weekly face-to-face meetings with
the lecturer. Mainly the material scheduled for the last week is discussed in these
meetings, but the discussion is also open for special issues or questions emerged.
To provide additional interactive parts to our text-based courses and encourage the
students' learning possibilities there are additional components supporting a broader
view on the content of the curriculum and expanding alternative ways for students to
learn and understand the essence of the subject. We are adding two interactive
components: Java Web Start applications and self-test tasks.
Java Web Start applications are an important part of our courses – especially where
a profound understanding of sophisticated concepts is desirable. We figured out that
there are several concepts in each course where such an application can provide useful
visualisation and additional understanding. Applications are uploaded to our LMS and
provided within the system. Due to the standalone nature applications are independent
from any system and can be used not only on our current LMS but on probably
different future versions too.
Fig. 1. Progression of a clustering applet visualising a clustering process
Depending on the content of the courses there is a variety of applications provided. To

name but a few, one course we are offering is Information Retrieval [5] and we are
e.g. providing applications on Recall and Precision graphs as well as applications to
explain the process of stemming methods or to combine understanding and
implementation of algorithms for pattern matching. As an example you can see the
time progress of an application visualising the clustering process of a specified
clustering algorithm in figure 1.
Self-test tasks represent the second component we chose to provide additional

options and interactivity for students. Self-test tasks constitute an instrument for
students to get an estimation of their current knowledge and understanding of
important concepts as well as a possibility to improve their current state of knowledge
by accomplishing the provided tasks. Self-test tasks are normally realised as a
sequence of multiple choice questions. The type of questions ranges from questions
on theoretical aspects to sophisticated computations.
Fig. 2. Integrated self-test task on signature trees
Other than Java Web Start applications self-test tasks are generated and provided
directly in the LMS. This approach brings a potential risk concerning possible
chances in the use of the LMS, but because our actual system offers a backup
functionality that exports selected data to XML we are willing to take the risk of
doing some migration work in case of a change of the system. Besides, the LMS
offers wide evaluation possibilities of test results, a factor that also confirms our
considerations.
Abiding with the Information Retrieval course there are, among others, self-tests on
principle Information Retrieval questions and models of Information Retrieval like
the vector space model or the binary independence retrieval model. In figure 2 you
can see a screen shot of a self-test task on signature trees. The specific problem is
searching for a precise signature in a given signature tree; there are 3 pre-formulated
answers to choose from.
The main goal of integrating and providing PDF lecture notes, Java Web Start
applications as well as self-test tasks as parts of one course in our overall LMS is
therefore to meet the requirements of students and lecturers all at once: providing well
elaborated and printable lecture notes that are easy to create and modify as well as a
balanced degree of interactivity to benefit students' success.
To round the concept for text-based courses we are providing weekly exercise lessons.
The exercises mainly consist of implementing the concepts presented and learnt
during the lecture-like parts of the course. We consider this as an important part of
our concept, respectively courses, because students learn to study and apply the
theoretical concepts practically.
As a further possibility students get the chance to collect bonus-points for the exam
concluding the course due to biweekly tests that are offered. In total there are 5
voluntary tests with tasks about different concepts that are part of the course and 4
points can be reached each time. The best 3 test results are added to the exam result if
the exam is initially passed. Since the best score in the exam is 90 points no matter if
bonus-points are considered or not students can markedly improve their exam results
by collecting bonus-points. Our LMS offers a good support for those tests. The
detailed test is provided on our LMS and students are first able to download the test
and build their solution wherever they want to. Second, they are able to deliver the
results directly in the system by an upload mechanism. Lecturers are then able to view,
judge and score the students' solutions also directly in the system, while students can
check their test results and score as well as a usually provided suggested solution after
the lecturers' scoring is completed. The thoughts behind those tests are that students
are really getting into the subject not just shortly before the exam but during the
whole semester because they need to understand the content of the course to process
the tests. Of course this might not apply to all students because it is a voluntary offer
but normally students are motivated to do the tests in order to improve their exam
results this way.
Looking at the whole concept at a glance we are trying to regroup the face-to-face
phases. That implies reducing face-to-face phases where they are not absolutely
necessary, e.g. to work through provided learning material, but extending face-to-face
phases where further supervision and contact can be useful, e.g. by additional
exercises..
3 Recording-Based Courses
In contrast to courses consisting primarily of fundamental and therefore static content,

there are courses containing current standards and systems as well as actual research
topics and results. Since we attempt to keep our courses up-to-date, these are
obviously courses that need to be changed more often. For this reason an appropriate
building and update strategy is needed to keep the effort within a manageable limit.
As a consequence we were looking for an approach that allows generating and
providing courses very easily with barely additional tasks apart from delivering the
face-to-face lecture itself. Obviously this brought us to Rapid e-Learning and we
started pursuing an appropriate Rapid e-Learning approach.
In contrast to text-based courses we were keeping the traditional way of face-to-face

lectures attended by students for recording-based courses. We chose to record the
lectures [7,8] and provide the recordings afterwards because the lecture has to be
delivered anyway and if it can be recorded in the same time there is apparently almost
no need for additional time and effort. The thoughts behind this concept are to provide
the lectures for students even after the face-to-face lectures were actually delivered.
This way, recordings can be used for post processing or exam preparations. Students
can run the recordings to repeat the whole curriculum or just parts they probably
missed or still have questions on. Of course no one is directed to use the recordings at
all, it is a voluntary offer. We had thoughts about a smaller number of students
attending the face-to-face lectures at the beginning because students could use the
recordings instead of taking part in the lectures but our thoughts were proven wrong.
Students are still attending the lectures and there is no noticeable smaller number
except for those missing due to inevitable conflicting dates with other lectures. So far
we see this as an advantage of our recordings since they are attenuating these
problems.
The requirements and preparations needed to accomplish this approach are neither
demanding nor expensive. First of all lecture notes are created as slides. There is no
need for any additional formatting or settings, and since slides are used as base to
deliver the lectures anyway the slides can be created as usual. Furthermore the plain
slides can be stored as PDF-documents so that they can be provided as additional
material for students to meet the requirement of printable material.
The actual recording process takes place when the face-to-face lecture is delivered.
The lecture is thereby delivered as usual but the lecturer records his lesson. The
technical equipping needed to run this system is imaginably low: A tablet PC
connected to a projector is used to display the slides and to provide additional
information and assistance on the slides as well as extra slides created off the cuff if
desired, we are using the Rapid e-Learning tool Lecturnity 3 . The only additional
requirement is an ordinary web camera. The camera records a video of the lecturer
while delivering the lesson, including the audio stream. Of course there are
restrictions concerning the quality of the recordings if recording is done with a simple
web camera, but since the recording result is integrated into and provided on our LMS
the restrictions correspond with the restrictions due to bandwidth and storage (in our
setting a recording of a 90-minutes lecture requires about 120MB).
Lecturnity is able to combine the recordings of the web camera and the recordings of
the slides. Furthermore thumbnails of the slides are displayed as table of content and
can be used to navigate through the video. The standard output format is a proprietary
format, but there are lots of standard outputs possible. It is e.g. possible to produce
Flash- or SCORM-files.
3
Official Lecturnity-Website http://www.lecturnity.de
Fig. 3. Screen shot of a recording available for students
To view the standard output format a free player is available. To make it even simpler,
we are providing the player on our LMS, therefore students only need to download
and install the player. The technical requirements to install the player are very low, so
that except for very few cases no additional costs occur for the students. There are
some students having problems due to video configuration, but the rate is below 1
percent. In total the lecturer only needs about 5 to 10 minutes of extra time for each
lecture to import the slides into Lecturnity, to export the final recording after the
lecture and to publish the recording in the LMS.
In figure 3 you can see a screen shot showing the result of the recording that is
available for students after the face-to-face lecture, viewed with the corresponding
player. As you can see the lecturer, the slides and the table of contents for navigation
are visible all at once.
The concept of recording-based courses is also supplemented with exercises and tests
to collect bonus-points for the exam. The realisation of these two features is basically
like in text-based courses.
3 Evaluation
After extending the scope of our Blended Learning concept, we were reorganising all
our provided courses and mapping each to the types described above. The resulting
distribution is quite balanced – there are 7 courses that we regularly provide, 2 of
them are basic courses and therefore provided as text-based courses, 4 of the courses
lie in the advanced field and are realised as recording-based courses, and 1 of the
courses has basic and advanced parts and is therefore realised as a hybrid form.
Evaluation concerning the lecturer-side is restricted to a self-critical review at the end

of each semester, showing that the deliberations concerning needed effort and
available time were generally correct. Since nobody but us is using the concept by
now there are no further evaluation results available.
Concerning the view of students, the two introduced course types are evaluated
against different criteria at the end of each semester. We are presenting exemplary
evaluation results of one text-based and one recording-based course that took place in
the last semester – the results are based on 25 evaluated questionnaires.
There are some general questions that are similar for text-based and recording-based
courses. Students were asked general ratings for the lecture, respectively the lecture-
like part, and the exercises; the scale for the general questions reached from 1 (very
good) to 6 (very bad). The grading for the lecture concept as a whole and the
associated exercises ranked from 2.1 to 2.6 for text-based courses and from 1.6 to 2.0
for recording-based courses.
The special focus for text-based courses was on questions about organisation and
structure of the learning material and the weekly face-to-face meetings. The scale
available reached from 1 (disagreement) to 4 (agreement). The learning material was
rated in different criteria, e.g. structure and understandability as well as personal
estimation on how good the learning material covered the underlying area, and how
helpful the learning material was for understanding concepts of that area; the average
values of ratings in this area reached from 3.1 to 3.8. Concerning the weekly face-to-
face meeting students basically had to rank the degree of help that was offered by
these meetings; the average value was 3.1 in this area.
The focus for recording-based courses was of course on rating the use of recordings.
The plain result was that all participants were using the recordings, some for
equalising missed lectures, some for post processing difficult parts and some for exam
preparation.
Drawing an overall resume all values are positively above average and students seem
to be comfortable with the concept of text-based as well as recording-based courses.
After mapping our Blended Learning courses successfully to the evolved concept, we
were thinking about further efforts and developments. Since we are participating in a
joint offer of a virtual study path4 it was obvious to transfer our existing concept to the
courses offered in this environment. We realised that the concept could easily be
adapted to virtual courses and there is actually almost no alteration needed.
Concerning the text-based courses all parts of the courses could be used just the way
they were, because the virtual courses are provided via our LMS too. The PDF lecture
notes can be provided as usual and the same applies to Java Web Start applications as
well as self-test tasks. Solely the weekly meetings to discuss concerns or questions
emerged for students while working through the learning material are dropped.
Instead of arranging face-to-face meetings the discussions are entirely moved to the
forums included in our LMS.
Transforming the recording-based courses slight changes or possibly additional time

is needed. In principle the recordings can be provided as usual and the same applies to
the additional lecture notes provided to students. The decisive advantage is that the
recordings make students of virtual courses feel like they are actually attending face-
to-face lectures. The problem is admittedly that the recordings have to be created
although there is no face-to-face class. In our case this problem does not occur
because the recording-based courses of the face-to-face study path are identical to the
recording-based courses of the virtual study path so that we can reuse the recordings
arising out of the face-to-face lectures. More concrete, the face-to-face course is given
in the winter semester and the corresponding recordings are used for the virtual study
path in the following summer semester. We are aware that this a special case and that
the additional time needed to create the slides and recordings without a corresponding
face-to-face lecture can turn out to be a problem due to time restrictions. Questions
and discussions normally covered during the face-to-face lecture are moved to forums
for recording-based virtual courses as well.
Both types of courses therefore proceed on the assumption that the lecturer spends
time on supervising the forums and answering questions arising in an appropriate time.
Obviously the face-to-face exercises have to be dropped for both kinds of courses
because there is no possibility for students to attend such classes. Nevertheless it is
possible to offer the tests to collect bonus-points for the exams as usual so that
students get for once an idea of practical issues related to the theoretical concepts
learnt before.
Detailed evaluations have shown that students are pretty satisfied with this approach.
Evaluations are taking place at the end of each semester and each proceeded course is
evaluated properly. The evaluation results presented below refer to two text-based and
one recording-based course of the last two semesters, that is about 40 fully completed
and evaluated questionnaires. We restricted the results to the last two semesters to
present the results as actual as possible. Evaluation criteria were on the one hand
criteria concerning the quality of supervision, e.g. supervision of content-related or
technical problems as well as organisational support. On the other hand criteria
4
The virtual study path is a joint offer of the University of Duisburg-Essen and the University
of Bamberg, further information is available at http://www.vawi.de.
concerning the content of the courses, e.g. the intermediation of knowledge,

comprehensibility or structure of the presented content were evaluated. The
evaluation scale allowed grading every criteria on a scale from 1 (low) to 7 (high),
additional space for comments on especially positive or negative features as well as
further comments was available.
Although the students admitted that the effort needed to pass the courses was above
average by tendency, we got very positive responses on average. All criteria
concerning the quality of supervision were ranked very high, at least 6.0 or above,
above all courses. There is no recognisable difference between recording-based and
text-based courses. The criteria concerning the content of the courses are rated little
worse but still very high relating to the evaluation scale. Concrete values vary from
5.5 to 6.7 for text-based courses and lie closely around 6.0 for recording-based
courses. Analysing the spaces for free comments it became obvious that especially the
provided recordings were appreciated when looking at the recording-based course.
There were a few students that had start-up difficulties because the learning material
for recording-based courses is organised as a structured collection of bullet items and
not continuous text, but the recordings and possibilities available by forums smoothed
these problems. Concerning the text-based courses lots of students commended the
systematic book-like structure of the learning material. Some students always go to
the bother of suggesting concrete improvements and as far as it is possible we are
trying to consider the suggestions for the next semester.
The most salient overall result was that a few simple but effective facts were
mentioned very positive across all courses. Extremely positive responses gained the
fact that the lecturer itself was actively attending the forum discussions and answering
questions of every kind. The short response time provided by lecturer and tutor is also
a circumstance that was appreciated a lot.
References
1. Andy Hon Wai Chun: The agile teaching/learning methodology and its e-learning platform.
In ICWL, pages 11–18, 2004.
2. Michael Derntl and Renate Motschnig-Pitrik: Patterns for blended, person-centered learning:
strategy, concepts, experiences, and evaluation. In SAC, pages 916–923, 2004
3. Max Dunn: Single-source publishing with xml. IT Professional, 5, 2003.
4. Joseph Fong, Irene S. Y. Kwan, Margaret Ng, Ivan Li, and S. K. Chan: An application-
oriented e-learning system with self-monitoring and adaptive exercises. In ICWL, pages
325–332, 2004.
5. Andreas Henrich and Karlheinz Morgenroth: Information retrieval as elearning course in
german - lessons learned after 5 years of experience. In TLIR, 2007.7.
6. Won Kim: Directions for web-based learning. In ICWL, pages 1–9, 2006.
7. Tobias Lauer, Rainer Müller and Stephan Trahasch: Learning with lecture recordings: Key
issues for end-users. In ICALT, 2004.
8. Thomas Ottmann, Stephan Trahasch, and Tobias Lauer: Systems support for virtualizing
traditional courses in science and engineering. In Quality Education @ a Distance, pages
73–82, 2003.
Blended Learning: Towards a Mix for SMEs -
Stakeholders and their Priorities
Sabine Moebs and Stephan Weibelzahl
National College of Ireland, School of Informatics, Mayor Street, Dublin 1, Ireland

{smoebs, sweibelzahl}@ncirl.ie
Abstract. While blended learning seems to be quite suitable for small and
medium sized enterprises (SMEs), current uptake of this learning method is low.
In this paper we propose a research design to examine the requirements for
blended learning in SMEs. It is based on a three-round ranking-type Delphi
study. Participants for the panels were carefully selected. Our method takes into
account that the area and the term of blended learning are discussed in very
different, partially contradicting connotations. For this purpose, we first provide
the background of the initial research question and describe our research design.
Next, we present preliminary results of the Delphi study and the steps in
preparation of round 2. Participants were selected for the online-Delphi and
grouped into panels of SME learners, trainers and providers of e-learning as
well as learners from large companies as a control group for the SME learners.
Keywords: Blended Learning, SME, Delphi study, Requirements
1 Introduction
Originating in the corporate training sector the term blended learning refers to the
provision or use of resources which combine e-Learning with more traditional
educational resources [1]. Our research combines an exploration of blended learning
and methods of learning operation in small and medium-sized enterprises (SMEs).
Blended Learning is often used as a buzz word with vague and varying meanings. It
has often been described as a way to get the best out of the two worlds of technology
enhanced learning and traditional classroom-based learning, benefiting from the
advantages of technology enhanced learning while compensating for its disadvantages
through additional in-class sessions [2, 3] and vice versa.. Blended learning is a
combination of any form of technology enhanced learning with face-to-face
instructor-led learning [3]. Blended learning has been proposed as a solution for
training needs in SMEs [4].
SMEs are often innovative, but under high economic pressure. This economic
pressure is a threat to ongoing learning activities although continuous training and
learning is necessary to stay competitive. Learning in the form of e-Learning is not in
high demand with SMEs although one could expect that it is highly suitable to the
learning demand at short notice [5] which is typical for SME learning. Research
suggests that blended learning can significantly improve learning satisfaction,
Blended Learning: Towards a Mix for SMEs 163
improve accessibility and increase participation [6]. It therefore has the potential to
alleviate a number of issues that arise in SMEs and act as constraints to the use of
blended learning, namely lack of social interaction during learning programs and
exchange of ideas. Customized in-house solutions used by multinational companies
(e.g., corporate LCMS or learning portal, content tailored to specific needs of
company) are usually not feasible for smaller organizations. Therefore SMEs rely on
what providers offer, i.e., instructor led training or off-the-shelf solutions. This is one
of the reasons why learners in SMEs are reluctant to use blended learning for their
learning and training needs [4].
2 Aspects of Blended Learning
Blended learning describes a learning environment that either combines teaching

methods, delivery methods, media formats or a mixture of all these.
In the literature the term is used to describe the integrated combination of
traditional off-line methods of learning with intranet, extranet web-based or internet-
based online approaches [7]. To accentuate the fact that the concept is learner
centered, blended learning can be described as a mix of delivery methods that have
been selected and fashioned to accommodate the various learning needs of a diverse
audience in a variety of subjects [8]. Blended learning aspects are covered in a variety
of research papers as well as in very practical instructions for blended learning. The
most prominent are briefly outlined here.
2.1 Dimensions of Blended Learning
To describe the variety of interaction Graham [9] introduced the so-called four
dimensions of interaction in face-to-face and distributed learning environments. The
four dimensions are space, time, fidelity and humanness. Space can range from live or
physical and face-to-face over mixed reality to virtual reality. The time dimension
develops from live synchronous with a very short lag time to asynchronous, which
has a long lag time. Fidelity reaches from a high level that is rich in senses, which
means it can incorporate sound, pictures, text and even fragrances, and the other end
of the dimension is using only one of the senses, e.g. text only. The humanness
dimension addresses the ratio of human interaction and machine interaction.
2.2 Frameworks in blended learning
Poor design of blended learning material can lead to much poorer learning results
in a blended environment compared to a single method delivery. Several authors
developed frameworks to react to this challenge.
Wenger and Ferguson [10] describe a framework to guide the design and
deployment of company trainings and courses. The framework reflects the idea that
most learning environments are blended already, considering that even a classroom-
164 Sabine Moebs and Stephan Weibelzahl
only course incorporates a variety of different learning modalities. Their approach

consists of three steps:
In a first step the learning ecology matrix is developed. The x-axis illustrates the
focus on the delivery of instruction that varies from “content delivery focus” to
“experience and practice focus” and the y-axis illustrates who controls the navigation
of the learning process varying from “guided navigation” to “learner self-navigation”.
In a second step four general learning modalities are included: studying, practicing,
teaching and coaching. These modalities do not refer to either classroom or e-learning,
but are rather applicable to both. In a last step the matrix is completed with distinct
instructional, learning and knowledge elements.
The learning ecology matrix aims at delivering a high quality learning experience
and at providing control over the learning experience for both, the learner and the
instructor. It strives at combining formal and informal learning rather than positioning
them as opponents. The social nature of learning has to be considered in all learning
elements. The aspect of cost-effectiveness is recognized, but merely in the sense that
any project aims at a combination of learning outcomes at a total minimum cost.
2.3 Success factors for blended learning
A variety of teaching methods, as well as a variety of different learners with

different preferences and needs determine success of blended learning. Several
success factors have been identified in the literature:
Design of the blend: A well designed blend of teaching methods can provide an
appropriate learning experience for most learners. The characteristics of the audience
have to be considered. This includes recognition of the amount of time they will have
to access the content, which includes connectivity issues [8, 11, 12].
Time flexibility: The flexibility in scheduling and format is critical to success.
Availability of the system enables them to study when they are ready to do so.
Mix of media and learning styles: The flexibility in media formats provides
optimum learning experiences based on personal preference. To select the right
methods and formats the learning styles and the education level of employees has to
be considered as well as the motivation of the learners [11, 13].
Student support: Response from tutors, subject matter experts as well as technical
or logistical support staff needs to be posted within 24 hours, which corresponds to a
rule of thumb for effective e-communication in general. The positive effect of a
timely response can be intensified by additional phone calls and face-to-face
conversations and will provide a sense that there are real people behind the online
environment [13].
Executive support: Blended learning needs executive support for the introduction
just as any other major change in a business environment. The decision to change to a
blended solution from the system that was in use before cannot be left to individuals
who are not in charge.
Content: The kind and quality of learning content is critical for success. Apart from
choosing the appropriate kind of content and the decision whether learning activities
are intended to inform people, develop skills, or build competencies, the consideration
of the time before information is out-of-date is of high importance [11].
2.4 Blended Learning in SMEs
The mix of ICT based training in combination with face-to-face interaction has been
identified as a success factor for blended learning in SMEs. A second success factor is
trust in the safety of the training environment, online as well as face-to-face. SME
learners prefer an informal and ad-hoc approach to learning which suits their busy
schedule. Lack of confidence in ICT on the other hand is an obstacle for taking on
blended learning by learners in SMEs as well as the lack of an immediate payback of
the training. Activity-based learning is clearly preferred over knowledge-based
learning. The size of the company has a strong impact on any kind of learning and
training activities which leads to small firms often lacking a lifelong learning culture
[14, 15]. A study of e-learning in continuing vocational training with emphasis on
SMEs came to the conclusion that there is not much information available on e-
learning in SMEs [16].
3 Statement of Purpose of Research
Blended Learning can combine the positive aspects of the two learning environments,
classroom based learning and e-Learning [2]. While blended learning seems to be
quite suitable for small and medium sized enterprises (SMEs), current uptake of this
learning method is low [4].
An on-line Delphi study and several in-depth interviews are used for data
collection. The study involves main stakeholders in blended learning in SMEs:
trainers, providers and researchers from the area of e-learning, blended learning and
lifelong learning, learners in SMEs will be compared with a control group of learners
from large companies.
This study will explore what is a good mix in blended learning - a mix of online
and face-to-face teaching - for learners in small and medium-sized enterprises (SMEs).
4 Delphi Study
There are different types of Delphi studies out of which a ranking-type Delphi is most
suitable for the research question. A pre-Delphi study provides a method to determine
the required type of study and other characteristics of the study. The quality of the
study is highly dependent on the selection of the participants and the analysis of the
data from the first round.
4.1 Pre-Delphi-Study
To determine whether a Delphi study would be a suitable approach and which type
would be most appropriate, we conducted a pre-Delphi [17] study. Initially we
extracted a set of questions from a literature review of the two areas blended learning
and learning in small and medium-sized enterprises as well as recommendations from

the Delphi technique [18].
In the pre-study we included a small group of potential study participants: a
representative from an SME, a provider, a trainer and a researcher.
At first we considered asking for feedback eventually leading to consensus on
different topic areas such as learning styles, different dimensions of interaction,
delivery modes and learning modalities. The pre-test showed that the research as
planned would not provide input from panelists about their priorities, but rather about
agreement or disagreement with the researchers’ perspective or previous research. It
became obvious that the study has to be open to the stakeholders’ selection of
important topics; otherwise we might run into the problem of simply confirming our
own ideas rather than eliciting the opinion of the panelists [18].
Moreover, we excluded the idea of using a mind map to collect initial input in the
first round, because it seemed likely that a number of participants might not be
familiar with the technology and therefore be distracted or even turned off from the
study topic itself. Instead we decided to develop an online form with a clear
navigation, a good usability that takes as much work from the participants as possible
and which is at best self-explaining how to use.
The pre-Delphi study provided us with a decision to conduct a ranking type Delphi
study and set the limit for the amount of issues to be asked from the panelists as an
input as well as the number of rounds that seem to be feasible without strongly
increasing drop-out rates from the participants.
4.2 Participants of the Delphi Study
The Delphi study involves a total of 50 participants with the sub-panels small and
medium sized enterprises (SMEs) from the IT industry, and from the tourism industry,
large companies, as well as trainers, providers and researchers from the areas blended
learning, e-learning and lifelong learning. SMEs were selected following the current
SME definition of the EU [19].
SME panels: Selection for the SME panels followed a general pattern for all SME
participants and an additional industry specific pattern. All SME participants were
selected following the pattern being employee of an SME, an interest in or experience
with blended learning or e-learning, internet access, management responsibility (team
leader, project manager, department head) and the number of years of experience in
the industry.
The participants from the IT SME panel have an average of 7.6 years experience in
the industry. They are managing directors, owners, manager, network technical staff
and software developers.
The tourism SME panel includes the following areas: hotels, B&B, tourism
information, travel agent and tourist guide or instructor (ski, snowboard, sailing, etc.).
The participants from the tourism SME panel have an average of 18.8 years
experience in the industry. They are owners, managing directors and project managers
and work for B&Bs, hotels, an outdoor events provider, travel agencies and tourism
information.
Large company panel: The large company panel participants were selected
following the pattern being an employee of a large company with more than 249
employees, experienced with blended learning or e-learning and having internet
access. Again we looked for team leaders, project managers and department heads for
this panel.
Trainer panel: The participants for the trainer panel were selected according to a
pattern [20] that required several years’ experience in blended learning, experience
with learners in SMEs and unrestricted internet access. The trainer panelists have an
average experience of 11.1 years in blended learning; the median is 8 years and years
of experience ranges from 5 to 22 years. The panelists are mainly trainers in the IT
sector, some in general education.
Provider panel: The participants for the provider panel were selected for their
experience in blended learning and the positions in e-learning companies. The
provider panelists have an average of 9.9 yrs experience in blended learning; the
median is 4 years and years of experience ranges from 4 to 16 years. The panelists’
positions are head of product development or education management, learning design
manager, technical staff, project manager and managing director.
Research panel: The participants in the research panel were selected for their
research area, work and publications in the areas of e-learning, blended learning and
lifelong learning. The research panelists have an average of 11.6 yrs experience; the
median is 10 years and years of experience ranges from 9 to 16 years. The research
topics are for instance evaluation of TEL, access to learning, digital learning styles,
social media, open educational research, business models in e-learning and e-learning
standards.
4.3 Ranking-Type-Delphi Study
To select a suitable Delphi application we turned to a taxonomy proposed by Day and

Bobeva [21]. Our Delphi study design can be described using their taxonomy. There
will be three rounds, one for discovery of issues and the two following rounds to
determine the most important issues and to rank them. The participants will be
heterogeneous since there will be five sub panels of participants with different
expertise, researchers and providers of e-learning, online and face-to-face trainers,
learners in SMEs and learners in big companies. Within the sub panels we aim at a
best possible homogeneity. The study will be conducted as an on-line survey and all
communication will be conducted electronically using e-mail, website and VoIP. We
aim at single-blind anonymity of the panelists while conducting the study. In addition
to these criteria we decided on a ranking type survey. The ranking-type Delphi aims at
finding an agreement between groups through a ranking of self-selected issues. The
ranking type Delphi study requires that the researcher focuses on three initial
decisions:
1. when to stop polling
2. how many issues to carry over to the next round
3. use of statistical techniques to support their conclusions.
Literature suggests that these answers have to be decided individually, depending

on the study design, number of participants, area of interest, etc. We follow Schmidt’s
[22] and Couger’s [23] examples and considered the results of the pre-Delphi.
The polling will stop after round 3. Initially the panelists are asked to list the 5-10
important aspects of blended learning. Participants have to add a description and a
rationale for putting the item on the list.
The total input from round 1 is consolidated into a list size short enough to be
accepted by the participants in the next round. The full list of all aspects, including
duplicated and synonyms can easily overstrain the participants and might result in
high drop-out rates. A too short list on the other hand can result in loss of information.
Where panelists use different terms for the same issue the researchers have to provide
a summary matching the different terms and one common description of the issue. In
the study an initial list of 225 items from round 1 was condensed to a list of 59 items
as input for round 2.
In the second round the panelists rank their “Top 20” issues, ties not allowed, out
of the consolidated list of round 1. The second round is aggregated into a list of “Top
20” items for each sub panel.
This 20-item panel specific list is presented in the third round as a list of the “Top
10” with ranks from ten to one. All other items on the list (11-20) are equally ranked
“0”. The panelists now rank their “Top 10” issues from the 20-item list.
The rank is calculated by combining the percentage of selection and relative
rankings by the individual participants. An approximation of the mean ranks has been
produced by multiplying each percentage of mention by its first-round rank. A
combined measure of the ranks in the second and third round provide a value for each
item and provide the final evaluation of each item on the 20 item list.
4.4 Results of Round 1 of the Delphi-Study
Round 1 collects a minimum of five and a maximum of ten aspects (items) of

blended learning considered most important for the topic from each participant. A
consolidated list with all the items from all participants will be given to the
participants in round 2.
In round 1 a list of approximately 200 unsorted items were collected. The
following list highlights the aspects that were mentioned by five participants or more.
The items are briefly described, reflecting the variety of meanings.
Accessibility: 24-hour accessibility of online parts, the option to work from home
or while traveling were mentioned was mentioned. Materials, tutor, IT and classrooms
have to be convenient as possible for the learner to encourage maximum participation.
The learner has to be able to decide, where and when to learn.
Time Flexibility: Trainers and participants should have the time to get to know the
system and the combination of online and face-to-face teaching and learning. Online
learning should enable to learn when it suits the learner and moreover enable
completely independent learning. It should enable participants to decide on suitable
learning times suitable to other activities and to use times in between normal work.
Online learning should support the learner to make best use of their own time.
Cost Efficiency: Blended learning as an option to keep the price of the training
solution as low as possible was mentioned. Participants also see a potential to find a
good mix by emphasizing the lower-cost elements of the blend (e.g. off-the-shelf e-
learning entities, on-line books, etc.). Some participants believe that blended learning
often means lower costs. The option to use course materials on a regional level is
expected to reduce travel costs. In general participants expressed the opinion that
online learning is more cost-efficient immediately as well as in the long-term.
Student Interaction: The ability to interact at different levels and through different
media should allow a more adaptive approach to learning. Participatory opportunities
for students to have a voice e.g. using VoIP was mentioned several times, but also
taking the student through a number of learning routes rather than a given sequencing
of learning materials.
Support Mechanisms: Personal support for every participant by mail, phone or chat
or mentoring is considered important. Collaboration tools are seen as possibilities to
greatly improve the team work that can be allocated and performed. The ability to
work in teams or virtual teams is perceived as a supporting function. To provide
ample opportunities for students to obtain help with specific problems was mentioned
as well as the need to explore topics that might be a little off the curriculum. The
online assessment is considered helpful for motivating the learner, because it gives
immediate feedback, but the social interaction in the classroom is also required.
Mix of methods and media: The use of different media, different learning media,
face-to-face and online are the characteristics mentioned most often. A selection of
media for specific parts of a course, e.g. test or interactive content that is read to the
learner or includes images and text as well as the classical reading of books, is
mentioned several times. The user should have printed/offline media supplementing
the online learning content. Learning with the suitable media and to make the right
choice seems crucial for this area. Online modules allow flexibility, but can be very
generic. Face-to-face phases in contrast can be very intensive and don't allow for
distraction. They are considered more restrictive.
Mix of learning styles: A mix of learning styles is expected to support learning
success. A basic mix of online and offline activities are expected as well as a mix of
different ways to present and teach. This mix is also expected to increase motivation.
Workplace-related learning: Learning must be relevant and useful to the learner,
otherwise it is just an exercise soon to be forgotten. The course needs to be relevant to
the skills / information gap that the organization has. Learning content has to be up-
to-date and important to the user.
Individuality: Instruction should be designed to adapt to the individual learner and
it should provide different kinds of learning experiences. The e-learning enables
learners to set an individual focus. Trainers or facilitators need to be able to deal with
different personalities and heterogeneous groups.
Knowledge Base Internet: The internet provides the most recent knowledge to
everybody in forums and there is no way to beat the internet as an easy to use
dictionary for any topic. Accessibility and easy search functions make the internet a
vast source of teaching materials and enables trainers to provide access to lots of
related material to build student enthusiasm. It provides a number of elements that we
can add into a blend, like blogs, wikis, mobile, podcasts etc.
Recognition of traditional learning: Formal class room training which follows a

specific training guideline tends to cover the topic in question in greater detail. From
experience this is suited to the more committed. Formal classroom/college is ideal
also for employee networking.
Self-Paced Learning: One needs to learn self-paced learning. Nowadays everything
is presented to students rather than letting them figure it out themselves. Provide
learning and practice experiences that are available over a continuum of time, versus
all within a short timeframe. Spaced learning and practice helps cement new
knowledge into long term memory, and provides additional cues for retrieving the
knowledge and skills under different circumstances. Course enables participants to
select order of topics and modules. The pacing of the learning process is placed to a
certain extent with the student and can suit their time needs and commitment. Self-
paced learning is suitable for shift workers and those on time constraints who may not
be able to attend a conventional timetable class or course. Blended Learning's main
benefit for students and employers is the flexibility to do the course at your own time
and pace.
Technology: Keep technology simple - If and when technology driven solutions
are part of blended programs, keep it as simple as possible. The switching between
mask/pages should be kept at a minimum. The learning environment has to be kept
simple. Many LMS have lots of different functions and features that an average
learner in an SME does not need. It is better to reduce the number of functions so that
learning of content plays the main role, not learning to use the system. Offer tools of
Social Software for more experienced learners. Make it easy to access and operate
any technology components of the blended solution (web page, on-demand course,
pod-cast, virtual classroom, etc.).
4.5 Getting from Round 1 to Round 2 of the Delphi Study
We used techniques of the coding phase in grounded theory [24] to analyze the
results of round one of the Delphi study. Constant comparison of the aspects collected
eventually revealed common properties, categories and eventually identified core
categories. We continually checked whether new categories or concise concepts
emerged.
Table 1. Consolidation of List Items from Round 1 to Round 2

R1 Æ Æ Æ Round 2
Activity Merge Merge Join
doubles synonyms similar
& word context
radicals
No. Items 225 176 145 59
The results were analyzed in three steps. First an alphabetical list of all items
revealed doubles. If the descriptions of the items actually described the same aspect
they were merged. In the next step the reduced list was checked for synonyms and
word radicals. In case they revealed more doubles in the descriptions, one of the
aspects was kept on the list. In the third step similar contexts or differently named
aspects with an identical or similar description were summarized and designated
umbrella terms.
4.6 Next Steps
A ranked, consolidated list is prepared from the results of the first round. In the
second round the panelists rank their “Top 20” issues, ties not allowed, out of the
consolidated list of round 1. The second round is aggregated into a list of 20 items for
each sub panel for the final round.
This 20-item list is presented in the third round as a list with the “Top 10” with
ranks from 10 to 1. All other items on the list (11-20) are equally ranked “0”. The
panelists now rank their “top 10 issues” from the 20-item list. The rank is calculated
by combining the percentage of selection and relative rankings by the individual
participants. An approximation of the mean ranks has been produced by multiplying
each percentage of mention by its first-round rank. A combined measure of the ranks
in the second and third round provide a value for each item and provide the final
evaluation of each item on the 20 item list. To visualize the results of round two and
three, concept maps [25] are provided to enable the participants to get a quick
overview of the results, although we are not using the complete process of concept
mapping. The statement maps, concept maps which locate the statements in a
coordinate plane of highest rank and percentage of agreement, will be prepared for
each of the sub panels.
5 Expected findings
This paper highlights the results from the first round of the ranking-type Delphi
study. The final results are expected to show which of the characteristics from the
total list of characteristics of blended learning from the first round are the most
important for the Delphi panel in total. It will also provide results for the individual
panels, how the selection process within the panel evolved and how the panel results
differ. The final results of the study will give an indication which topics to explore
towards a mix in blended learning for SMEs.
Acknowledgement
The work described in this paper is part of the Up2UML project [Project No. DE-
05-B-F-PP-146369] and is partially funded by the Leonardo da Vinci Agency /
European Commission.
References
1. Driscoll, M.: Blended Learning: Let's get beyond the hype (2002) Retrieved April 10 2006
from http://www-8.ibm.com
2. Bonk, C.J.; Graham, C. R.: Handbook of Blended Learning. Pfeiffer, San Francisco (2006)
3. Rosenberg, M.J.: E-Learning Strategies for delivering knowledge in the digital age.
McGraw-Hill (2001)
4. Hamburg, I.; Lindecke, C.; Terstriep, J.: eLearning - A chance for small and medium sized
enterprises (2005) Retrieved Apr 24, 2006 from http://eleed.campussource.de/archive/2/236/
5. Wood, D.; Watson, J.: Factors Limiting the Proliferation of E-learning within Small to
Medium Sized Enterprises. In: G. Richards (Ed.): Proceedings of World Conference on E-
Learning in Corporate Government, Healthcare, and Higher Education 2002. AACE,
Chesapeake (2002) 1049-1055
6. Daniel, B. K., Matheos, K.; McCalla G.: Blended Learning Approach for Technology
Enhanced Learning Environments. (2004) Retrieved March 23 2006 from
http://oltfile.qut.edu.au/
7. Garavan, T.N. & O’Donnell, D.: eLearning in Irish Organizations? Retrieved April 14, 2006
from www.cipd.ie (2003)
8. McSporran, M. & King, C.: Blended Is Better: Choosing Educational Delivery Methods
(2005) Retrieved March 20, 2006 from hyperdisc.unitec.ac.nz/
9. Graham, C.R.: Blended learning systems: definition, current trends and future directions. In:
Bonk, C.J. & Graham, C. R.: The Handbook of Blended Learning. San Francisco Pfeiffer
(2006)
10. Wenger, M.S. & Ferguson, C.: A learning ecology model for blended learning from Sun
Microsystems. In: Bonk, C.J. & Graham, C. R.: The Handbook of Blended Learning.
Pfeiffer, San Francisco (2006)
11. Bersin, J.: What works in Blended Learning. (2003) Retrieved April 12, 2006 from
http://www.learningcircuits.org/2003/jul2003/bersin.htm
12. Saunders, P. & Werner, K.: Finding the right blend for effective learning. (2004) Retrieved
March 15 2006 from http://www.wmich.edu/teachlearn/new/blended.htm#move
13. Serveau, K.: Examining an Online Corporate Study Group. In: G. Richards (Ed.):
Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare,
and Higher Education.. AACE Chesapeake (2004) 940-945
14. Stokes, A.: Using telemonitoring to deliver training to SMEs: a pilot study. In: Education +
Training, Vol. 43, No. 6. (2001) 317-324
15. Lawless, N., Allan, J., O’Dwyer, M.: Face-to-face or distance training: two different
approaches to motivate SMEs to learn. In: Education + Training, Vol. 42, No. 4/5. (2000)
16. Page, K.: A preliminary study on the current state of e-learning in lifelong learning.
Luxembourg Office for Official Publications of the European Communities (2006)
17. Haeder, M.: Die Expertenauswahl bei Delphi-Befragungen. In: ZUMA How-To Reihe, Nr.
5. (2000) Retrieved October 15 2006 from www.gesis.org
18. Linstone, H.A.; Turoff, M.: The Delphi Method – Techniques and Applications. (2002)
Retrieved March 10, 2006 from http://www.is.njit.edu/pubs/delphibook/
19. European Commission: The new SME definition: User guide and model declaration. (2006)
Retrieved April 24, 2006 from http://europa.eu.int/
20. Alexander, C.: A Pattern Language. Oxford University Press, New York (1977)
21. Day J.; Bobeva M.: A Generic Toolkit for the Successful Management of Delphi Studies. In:
Electronic Journal of Business Research Methods, Vol. 3, Issue 2 (2005) 103-116
22. Schmidt, R.C.: Managing Delphi Surveys using nonparametric statistical techniques. In:
Decision Sciences, Vol. 28, No. 3 (1997) 763-774
23. Couger, J.D.: Key Human Resource Issues in IS in the 1990s: Views of IS Executives
versus Human Resource Executives. In: Information & Management, Vol. 14 No.4 (1988)
161-74
24. Strauss, A., Corbin, J.: Basics of Qualitative Research: Grounded Theory Procedures and
Techniques. Sage Publications, Newbury Park London New Delhi (1990)
Supporting Self-Regulated Learning
in a Blended Course
Giuliana Dettori, Donatella Persico
Istituto per le Tecnologie Didattiche del CNR, Genova – Italy

dettori@itd.cnr.it persico@itd.cnr.it
Abstract. This study investigates the practice and development of self-

regulated learning abilities in a blended course for trainee teachers, where
traditional, face-to-face sessions alternate with online collaborative modules.
Based on data emerged from an end-of-course questionnaire and on the analysis
of the online interactions, the study suggests that the potential of the online
CSCL component of the course was especially valuable as concerns the social
aspects of SRL. The face-to-face sessions, instead, seem to contribute in
particular to the development of the cognitive sphere. This study suggests that a
balanced blend of presence and online activities may result synergetic from the
point of view of SRL.
Keywords: Self-Regulated Learning, Computer Supported Collaborative

Learning (CSCL), Interaction Analysis, Blended Learning
1 Introduction
Self-Regulated Learning (SRL) is a learner-directed process which transforms mental

abilities into operative capabilities, in relation to a task at hand [22, 23]. It allows
people to autonomously handle their own learning process and to face the needs to
learn and to get updated that arise, with increasing frequency, in professional carriers
and in everyday life [19]. Hence, self-regulation skills support lifelong learning by
making people active learners and by favouring the transfer of knowledge and
problem solving methods among different learning situations. For this reason,
developing students’ self-regulatory skills is currently considered a major goal in
education, and it is deemed as important as acquiring content knowledge.
It is widely accepted that self-regulation entails an active and conscious control
of one’s own activity from the point of view of meta-cognition, motivation and
behaviour, both individual and social [22, 23]. The same author also points out that
SRL entails the cyclical repetition of three phases, that is, planning or forethought,
performance (which includes execution monitoring), as well as evaluation of the work
done and achievements reached. Controlling cognition, motivation and behaviour
throughout these three phases allows the students to take advantage of their
knowledge and skills in the environment where learning takes place, progressing
towards the achievement of their personal learning objectives.
Concerning the development of self-regulation, the literature indicates that some

aspects, such as meta-cognitive knowledge and skills, generally improve as students
get older. It is also acknowledged, however, that the acquisition of SRL competence
is not automatic nor does it take place in a short time [2], but should be supported by
suitable teaching and practice. Developing SRL abilities is a complex task and
depends on many factors. The literature underlines the importance of creating and
structuring learning environments that allow and facilitate control of the essential
dimensions of learning [16]. The importance of receiving formative feedback [13] and
being encouraged to reflect and revise one’s work [20] is also stressed. The literature
suggests that SRL abilities need to be adapted when moving to different learning
contexts [6], as learners do not seem to self-regulate in the same way on all tasks.
SRL in Computer Supported Collaborative Learning (CSCL) has been
investigated by several authors [1, 7, 8, 14, 19]. From these studies, it appears that
CSCL environments are suited to practice SRL or even foster its development, both in
online and in blended courses, because of the great emphasis given to collaborative
activities and to the fact that in such courses meta-cognitive skills are often among the
explicit or implicit objectives of the learning process. Moreover, CSCL is essentially
based on textual interactions, and this seems to lead to a deeper reflection and
involvement, since the messages exchanged are recorded in the environment and
learners can access them at any time [12]. Finally, in CSCL learners are usually free
to choose when and where to work on a course. This gives them a perception of
freedom of choice, which is reported to encourage self-regulation [3].
The relationship between CSCL and SRL, however, has a twofold nature. If SRL
can be seen as a useful side-effect brought about by CSCL, it also turns out to be one
of its requirements. Effectively participating in CSCL, as a matter of fact, entails
possessing a good amount of self-regulation, not only for time management and work
organization but also for handling the transition from the classroom model of
instruction to that of distance, collaborative learning [1]. Moreover, a number of
meta-cognitive skills are necessary to take full advantage of this learning method. The
relationship between SRL and CSCL is therefore quite complex and worth further
investigation.
The study of SRL in blended learning, is even more necessary, especially since
there is not a single way to mix face-to-face and online activities [20]. Since in this
case learning always takes place in two different environments (one online and one
face-to face), SRL is possibly clearly put into play in more than one way; it is
therefore worth investigating if, and under what conditions, the support to SRL
provided by the online and face-to-face modules may result synergetic, or
independent or even in conflict with each other.
In order to help understand this relationship, we analysed the potential support to
SRL of a blended course for teacher training. We analysed the online component of
the course, by applying two complementary methods. The first consisted in asking
course participants to fill in an end-of-course questionnaire (the TELESTUDENTS
questionnaire developed by the TELEPEERS 1 European project). The aim of this
1
“Self-regulated Learning in Technology Enhanced Learning Environments at University
Level: a Peer Review”, Grant agreement 2003-4710-/001-001 EDU-ELEARN,
http://www.lmi.ub.es/telepeers/
176 Giuliana Dettori, Donatella Persico
survey was to investigate if the SRL-supporting features of the online course

component had been perceived by the participants as effective and useful in
regulating their own learning. The second was based on data obtained by carrying out
a quantitative analysis of the messages exchanged by a group of students in two
activities of the course. This small scale, more in-depth study provided us with
complementary information based on the observation of learning dynamics. Finally,
we considered if and to what extent the face-to-face component resulted synergetic
with the online one in determining the SRL potential of the whole course.
2 The course analysed
We analysed a course in Educational Technology, called TD-SSIS, run by the

Institute for Educational Technology (part of the Italian National Research Council)
within the Teacher Training School of the University of Genoa. The course objectives
were to promote the development of instructional design competence, with special
focus on the evaluation and selection of learning strategies, techniques and tools and
on the implementation of educational technology in the school context. This course
has been shaped differently over the past years, experiencing different blending
designs. In this paper, we focus on the course run in 2005, that addressed 94
prospective teachers of all disciplines. Most students (94,4% of the respondents) did
not have previous experience with online courses.
The course consisted of five blended modules lasting a total of twelve weeks,
with a face-to-face meeting at the beginning of each module to start up its activity. In
addition, the course featured two transversal activities, respectively devoted to
socialization and to meta-reflection on the course and its method, which were run
completely online, in parallel to the other modules. The five blended modules
consisted in:
1. a familiarisation game aimed at getting acquainted with each other and with the
software environment;
2. an activity devoted to online educational resources, including a phase of
individual work and one of peer review;
3. a role play where students were requested to spot strengths and weaknesses of a
WebQuest;
4. a case study focusing on a best practice project where several classes and schools
collaborated to develop a common artefact;
5. a conclusive activity promoting reflection on the course itself and on the learning
achieved.
The course adopted a social-constructivist learning model and employed Centrinity
FirstClass™ as Computer Mediated Communication (CMC) platform to support
collaborative activities. The main interaction modality was written and asynchronous,
but in some cases chats were also used. Guidance and facilitation was provided by a
team of 7 tutors plus the course coordinator. Online activities were carried out in
groups of about 10 students and took place within discussion areas set up on purpose.
3 Indicators of self-regulation from the questionnaire
The questionnaire used to collect data on trainees’ appreciation of the SRL features
was an adaptation of the TELESTUDENTS-SRL 2 . This questionnaire, which was
developed based on the SRL-characterization arising from the literature, especially
from the work of 22, 23], considers students’ perception and appreciation of several
aspects of a learning environment:
the support provided to the users in finding their way in the environment
(facilities of the interface and possibilities of personalization),
the support to planning the learning process (availability of planning tools
like calendars and progress reports, history, indication of prerequisite, etc.),
the support to execution (availability of materials in different formats, of
tools for collaboration and communication, of different learning paths and
levels of difficulties, of formative feedback),
the support to self-evaluation (evaluation tools, possibility to compare one’s
work with that of peers and with given models).
For each of these areas, the possibility to control cognitive, motivational, emotional
and social aspects was considered. All questions were to be graded on a 0 to 4 scale.
In the analysis of students answers, we took as positive the grades above 2, which
express an appreciation more than average.
The data gathered with this questionnaire come from 72 respondents (out of 94
students, amounting to 76,6% of the course participants). Its main outcomes are
reported and discussed below.
3.1 Cognitive and meta-cognitive aspects
The students expressed their appreciation in relation to the cognitive aspects as

concerns the possibility to:
establish personalised learning goals: mean 2,53, sd 0,96;
plan their own learning 2,16, sd 0,84;
switch to a new learning strategy if needed: mean 2,60, sd 0,84;
pace one’s learning activities: mean 3,44, sd 0,77;
learn from home 3,63, sd 0,59;
personalise the interface: mean 2,51, sd 1,19.
From these data, students appear to feel fairly in control of the cognitive aspects
related to planning, and in particular to take advantage of the freedom offered by the
environment in terms of time and work organization. The lowest average score was
given to the possibility to pace one’s learning activities. This relatively low rating was
probably due to the fact that most of the planning decisions had to be negotiated with
both the course designers and peers, and the learning pace was somewhat imposed by
the need to collaborate.
2 http://www.lmi.ub.es/taconet/tools.php
The study also investigated meta-cognitive aspects related to performance and

evaluation; the students expressed as follows their appreciation for the possibilities
offered:
encouragement to actively participate in learning: mean 2,79, sd 0,88
help to reflect on problem solving: mean 2,80 sd 0,98;
help to reflect on learning progress: mean 2,72, sd 0,95;
help to recognize the achievement of learning goals: mean 2,49, sd 0,92;
availability of appropriate feedback on work done: mean 2,37, sd 1.
In this case as well, the data are rather positive, especially as concerns the support
provided to meta-reflection. These data support the claim of several authors [10, 15]
that asynchronous written communication has considerable potential for reflection
since written messages are permanent, provide multiple viewpoints from group
members and force people to strive for clarity.
3.2 Emotional and motivational aspects
Concerning emotional and motivational aspects, the help provided by the

environment was deemed valuable as to:
re-establish a positive working attitude after difficulties: mean 2.67, sd 0,96;
keep up a positive working attitude: mean 2,40, sd 0,97;
work out strategies to keep up motivation: mean 3,43, sd 0,96.
have more confidence in their own abilities: mean 2,21, sd 1,06.
These data highlight a weakness of the online environment: the online help
facilities provided by the CMC platform used are rather poor and in some cases the
novelty of the approach generated anxiety and a sense of inadequacy. Both problems
were also reported by the tutors. The first was effectively counter-balanced by a
special conference devoted to technical help, where one expert was available to
provide support in a very short time. The second is typical of online activities and
hence unavoidable, but it can be alleviated by creating a comfortable social climate.
This was done by encouraging a friendly atmosphere in the online discussion areas,
that was further reinforced during the face-to-face meetings.
3.3 Social aspects
The social aspects concern the ability of course participants to interact with the
other members of the community in an efficient and effective way, and take
advantage of such interactions to improve their learning process. These aspects
include help-seeking, communication ability, effective collaboration, comparison with
given models or with peers’ performance and achievements, etc. The questionnaire
reported the following appreciation for the various social aspects of the online
activities:
working with peers: mean 3,04, sd 0,92;
communicating with peers: mean 3,00, sd 0,94;
comparing results with peers: mean 3,21, sd 0,84;
contacting and receiving help from tutors: mean 2,68, sd 1,01;
discussing work with tutors: mean 2,41, sd 1,03.

According to the above data, the students’ judgement about the potential of the
course to develop social SRL aspects appears particularly positive, especially
concerning contact with peers. Appreciation of this aspect also emerged from several
participants’ messages, where students expressed the opinion that the virtual
environment had much helped to create a sense of belonging to the course community.
They also pointed out that the use of the two delivering modalities had created an
opportunity to interact with more course mates, in comparison with other purely face-
to-face courses, in that the online collaborative activities had led them to interact with
peers they did not know before and would have not thought to approach in normal
conditions, and subsequently felt motivated to look for them during the face-to-face
classes.
3.4 Summary of questionnaire results
The data presented distinguish the three main components of SRL: the
cognitive/meta-cognitive one, the emotional and motivational, and the social
behaviour. This distinction derives from the literature on SRL, but it is clear from the
above discussion that the borders between them are quite blurred and that the various
aspects investigated within each of them are strictly intertwined. Nevertheless, it may
be useful to try a rough comparison of the support perceived by the students in
relation to the three components. If we compare the mean value of the above data we
obtain the histogram shown in Fig.1, which suggests that the social component of
SRL was perceived to be supported more than the cognitive/meta-cognitive
component, which in turn was deemed to be supported more than the
emotional/motivational one.
This general evaluation is confirmed by the data obtained by applying content
analysis to part of the messages exchanged in the course, that are presented in the
following section.
4 Indicators of self-regulation from interaction analysis
We resorted to interaction analysis in order to see if a different way to analyse the

course would confirm or not the questionnaire data. Content analysis of interactions
appears a suitable tool to this end, because it combines qualitative and quantitative
analysis of messages. This technique has been increasingly used by researchers to
gain insight about CSCL dynamics, taking advantage of the permanent nature of
computer conference transcripts [17].
4
3,5
3
2,5
2
1,5
1
0,5
0
cognitive and emotional and social aspects
metacognitive motivational
aspects aspects
Figure 1. Summary of the data from the end-of-course questionnaire
We analysed the messages exchanged by a group of eight students and one tutor in
the work areas of Modules 3 and 4, looking for expressions that could indicate the use
of SRL in some activities. Each activity lasted three weeks, for a total of six weeks
and 249 messages exchanged. The sample chosen had more or less the same features
of the whole cohort of students, as concerns ratio between males and females, mixture
of backgrounds, average grade earned in the final assessment. Both activities were
collaborative.
Two coders examined separately all students’ messages of the sample. One coder
was one of the course designers, while the other was an external rater. The chosen
unit of analysis was the message. After coding, the inter-rater reliability was
calculated. The percent agreement resulted to be 88% for the fist activity and 80% for
the second one.
The indicators of SRL we used are derived from the literature, checked against our
experience. They can be classified along three dimensions. The first dimension has to
do with the phases of the self-regulated learning process, that is, planning, monitoring
and evaluating. The second has to do with the distinction between self regulation in
individual work vs self-regulation in team work. The third distinguishes between
cognitive or meta-cognitive aspects and emotional or motivational ones. More details
about this distinction and the related indicators are reported by Dettori and Persico [9].
If a message contains reference to self-regulated actions, then we can deduce that
the author of the message, having taken those actions, has practised self-regulation to
some extent. For example, if one of the students proposes to his/her peers a given
workplan and asks for feedback, we can interpret the message as a planning action (in
that the author of the message must have done some planning to write the proposal)
and also as an attempt to socialise his/her own efforts, in that he/she is asking for
feedback and is not trying to impose his/her own ideas to the group. The opposite,
however, can not be claimed, because if self-regulation does not emerge from
students messages, this doesn’t mean that it did not occur, but simply that it was not
openly expressed. For example, a student might have done some planning in
individual work, but he/she might have decided that it was not relevant to tell his/her
peers and tutors about those plans. In conclusion, the interpretation of the results of
the study must be carried out bearing in mind that what has emerged from them may
just be the “tip of the iceberg” of a wider phenomenon.
The main results of the content analysis are reported in Fig. 2 to 4. The two
activities had the same duration, but both the total number of messages exchanged
and the percentage of messages that were found relevant to study SRL were higher in
Module 4 (a case study) than in Module 3 (a role play). This can partly be explained
by the nature of the two activities and by the increased familiarity among the group
members, but it can also suggest that self-regulation had improved over the course.
Most likely, all hypothesis contributed to determine these data. Module 3, being a role
play, proposed an inherent plan (who does what) that partially relieved students from
taking individual decisions on how to participate (this is confirmed by the data in Fig.
2, where indicators of planning events in Module 3 are significantly less than those of
Module 4). However, Module 4 shows a generally higher concentration of SRL-
related events, therefore supporting the idea that students self-regulated their learning
better in this phase, as concerns not only planning, but also monitoring and evaluation
tasks.
Message categorisation according to "process" Message categorisation "individual" vs "social"

model
70
30 60
25 50
20 40 Module 3
Module 3
15 30 Module 4
Module 4
10 20
5 10
0 0
Planning Monitoring Evaluation Individual Social
Figure 2. Coding results along the Figure 3. Coding results along the
categories of the process model individual vs social categories
Message categorisation: cognitive and

metacognitive vs emotional and motivational
40
30
Module 3
20
Module 4
10
0
cognitive/metacognitive Emotional/motivational
Figure 4. Coding results along the categories cognitive and meta-cognitive vs. emotional and
motivational
5 From online to blended learning
The above data mostly focus on the potential support to SRL of the online
component of our blended course. What about the support provided by the whole
course? Does the inclusion of some face-to-face activities increase, or decrease, or
leave unchanged, students’ possibilities to practice and improve SRL? If any change
is induced, what aspects are affected?
At first sight we might think that the presence of a face-to-face component would
decrease the global support to SRL, since students’ freedom to organize their learning
activities was obviously limited by the fact that the face-to-face activities had a fixed
schedule. SRL, however, is not much about being free to organise one’s learning
activity, but rather to be able to make good use of the amount of freedom allowed by
the learning situation. In this respect, some authors claim that total freedom in taking
decisions concerning one’s own learning is not really essential for self-regulation,
indeed learners need to have some minimal knowledge related to the learning activity
at hand in order to be able to make informed choices [18]. Therefore, some guidance
is necessary to allow learners to exercise control [11].
In the case of our blended course, the face-to-face classes at the beginning of the 5
modules had the function to start up the modules’ activity by providing a general,
content-related framework for the work to be carried out online. Giving introductory
classes in presence turned out to be particularly efficient, since the presentation of the
relevant content knowledge and the clarification of collective doubts on the topic
could be made more quickly than it would be possible online. Hence, the face-to-face
component of the course contributed to the SRL support on the cognitive/meta-
cognitive level, by providing the guidance necessary to help the students move at ease
through the online activity. Moreover, the face-to-face classes also served the purpose
to encourage and motivate the students who were less familiar with technology and
collaboration tools, therefore determining a positive perception of this aspect in
relation to the whole course.
In the authors opinion, on the other hand, the online component supported the
social level more than the face-to-face one. A course with about 100 students, and
with a dense schedule of activities, can in fact hardly provide frequent student-centred
and highly interactive activities. Since the lack of a suitable social experience appears
to be a primary source of self-regulatory dysfunctions [4], the online component of
the course served a purpose of great relevance to the overall quality of the learning
experience.
In order to better understand the relation between online and face-to-face in this
course, we will consider again some of the data coming from the end-of-course
questionnaire, and in particular the answers to questions aiming to distinguish
between the contribution given to the course by the different learning modes.
These data (Fig. 5) show that the overall course quality was positively judged by
the students, and that the online component of the course significantly contributed to
such quality, more than the face-to-face one. Consistently, the students found (Fig.6)
that learning collaboratively in the CMC environment was more important than
individual study of learning resources, which in turn was deemed more important than
the face-to-face sessions.
A further remark can be made about the students opinions on how much they felt
responsibility for their own learning. Again, the end-of-course questionnaire provides
a mean rating of 2,72 in a 0 to 4 scale (sd 1,06), which is quite satisfactory. Fig.7 also
gives an insight about students perceptions of motivation improvement along the
course. A significant increase in students motivation was also perceived by the tutors.
In conclusion, both components appear to have played different but important
roles in determining the SRL potential of the whole course. Each of them contributed
to overcome what could be seen as a weakness of the other component. Moreover, it
is important to recall what emerged from the students’ messages, mentioned in
Section 3.3. The work online with previously unknown peers, followed by face-to-
face sessions where they had an opportunity to meet, had led to the creation of a
warmer social atmosphere, which supported the emotional aspects of SRL more than
it would have been possible in a purely online or purely face-to-face course. Overall,
the balance of the online and face-to-face components was appreciated by the students
(mean 2,9; sd 0,94).
4,00 4,00
3,50 3,07 3,21 3,50
2,92 2,82 3,00
3,00 2,56 2,85
3,00
2,50 2,30
2,50
2,00
2,00
1,50
1,50
1,00
0,50 1,00
0,00 0,50
the course, the course the blended the face-to- the online 0,00
globally contents metthod face component working in the CMC face-to-face meetings individual study of learning
component environment material
Figure 5. Question “express your opinion Figure 6. Question “Rate the importance of
about the quality of the course and its the following modes of study in your
components (0=very low/4=very high)” learning process (0=very low/4=very high)”
4,00
3,50 3,12
2,94
3,00
2,48
2,50
2,00
1,50
1,00
0,50
0,00
in the initial phase in the intermediate phase in the final phase
Figure 7. Question “Rate your motivation during the course (0=very low/4=very high)”
6 Concluding remarks
The case study analysed suggests that blended courses can offer good opportunities
to foster students SRL skills. We must be cautious, however, to generalise these
positive data. The same outcomes can obviously not be expected from any blended
course, because the variety of possible ways to blend presence and online activities
affects the synergy between the two component. When these cover different parts of
the course program, for instance, or the face-to-face meetings have just an
organizational aim, the online activity can not take advantage of the cognitive
guidance provided by a face-to-face introduction. On the other hand, having only rare
face-to-face meetings or concentrating them at the beginning of the course, may fail
to create the positive social atmosphere induced by the alternation between online
collaboration and face-to-face interaction with peers.
Further research is therefore needed to study different course structures, analysing
the impact of components with different aims, organizations and lengths, as well as of
different combinations of them, in order to shed further light on the relationship
between SRL and blended learning.
References
1. Arbaugh, J.B. (2004) Learning to learn online: A study of perceptual changes between
multiple online course experiences. Internet and Higher Education 7, pp. 169-182.
2. Boekaerts, M. (1997) Self-regulated learning: a new concept embraced by researchers,
policy makers, educators, teachers and students. Learning and Instruction 7 n.2 , pp. 161-
186.
3. Boekaerts, M. (1999) Self-regulated learning: where we are today. International Journal of
Educational Research, 31, pp. 445-457.
4. Boekaerts, M., Pintrich, P. R., Zeidner, M., (2000). Handbook of self-regulation. Academic
Press., San Diego.
5. Bolhuis, S. (2003) Towards process-oriented teaching for self-directed lifelong learning: a
multidimensional perspective. Learning and Instruction 13, pp. 327-347.
6. Corno L. (2001). Volitional aspects of self-regulated learning. In Zimmerman B.J. , Schunk
D.A. (eds.), Self-regulated learning and academic achievement: theoretical perspectives (pp.
191-225). Lawrence Erlbaum Associates, Mahwah, N.J.
7. Dettori G.; Giannetti T. & Persico D. (2005). "CMC environments supporting self-regulated
learning". Proc. Internat. Conference m-ICTE 2005, Caceres (E), 7-10 June 2005, 379-383.
8. Dettori, G., Giannetti, T. & Persico, D. (2006). SRL in online cooperative learning:
implications for pre-service teacher training. European Journal of Education, vol.41 (3/4),
397-414.
9. Dettori G. & Persico, D. (2007). Using Interaction Analysis to reveal Self-Regulated
Learning in Virtual Communities. Proceedings of the International Educational Technology
Conference 2007, Crete, Sept. 2007, LNCS, Springer, Berlin, (in print).
10.Dillenbourg P. (1999), Collaborative-learning: Cognitive and Computational Approaches.
Oxford. Elsevier.
11.Jones, A. & Issroff, K. (2005) Learning technologies: Affective and social issues in
computer-supported collaborative learning. Computers & Education 44, pp. 395–408.
12.Kanselaar, G., Erkens, G., Jaspers, J. & Tabachneck-Schijf, H. (2001) Essay review:
Computer supported collaborative learning, Teaching and Teacher Education 17, pp. 123-
129.
13.Kramarsky B., Zeichner O. (2001). Using technology to enhance mathematical reasoning:

effects of feedback and self regulation learning. Educational Media International, 38 (2-3),
pp.77-82.
14.Lynch, R. & Dembo, M. (2004) The relationship between self-regulation and on-line
learning in a blended learning context. International review of Research in Open and
distance learning, 5, n.2,. Retrieved from http://www.irrodl.org/content/v5.2/lynch-
dembo.html in Jan. 2006.
15.Palloff, R. & Pratt, K. (1999) Building learning communities in cyberspace: Effective
strategies for the online classroom. San Francisco: Jossey-Bass Publishers.
16.Rosario P., Nuñez Perez J.C., González-Pienda J.A. (2004). Stories that show how to study
and how to learn: an experience in the Portuguese school system. Electronic Journal of
Research in Educational Psychology, vol.2, n.1, pp. 131-144, http://www.investigacion-
psicopedagogica.org/revista/index.php3.
17.Rourke L., Anderson T., Garrison D.R. & Archer W. (2001). Methodological Issues in the
Content Analysis of Computer Conference Transcripts, International Journal of Artificial
Intelligence in Education, vol. 12, 8-22.
18.Schraw, G. (1998) Promoting general metacognitive awareness, Instructional science 26, pp.
113-125
19.Van Den Boom, G., Paas F., Van Merrienboer, J. J.G. & Van Gog T. (2004) Reflection
prompts and tutor feedback in a web-based learning environments: effects on students’ self-
regulated learning competence, Computer in Human Behaviour 20, pp. 551-567.
20.Vye N. J., Schwartz D. L., Brasford J. D., Barron B. J., Zech L (1998). SMART
Environments that support monitoring reflection and revision. In D. J. Hacker, J. Dunlosky,
A. C. Graesser (Eds.), Metacognition in educational Theory and Practice. Laurence
Erlbaum Associates, Mahwah, NJ.
21.Yoon, S.-W. & Lim D. H. (2007). Strategic blending: strategic blending to improve learning
and Performance. International Journal. on E-Learning, 6 (3), 475-489.
22.Zimmerman B. J. (1998). Developing Self-fulfilling cycles of academic regulation: an
analysis of exemplary instructional models. In Shunk and Zimmermann (eds) Self-regulated
learning. From teaching to Self-reflective practice, The Guildford Press, New York, 1-19.
23.Zimmerman B. J. (2001). Theories of self-regulated learning and academic achievement: an
overview and analysis. In B.J. Zimmerman & D.A. Schunk eds., Self-regulated learning and
academic achievement: theoretical perspectives. Lawrence Erlbaum Associates, Mahwah,
NJ., pp. 1-37.
A Study on the Impact of the Use of an Automatic and
Adaptive Free-text Assessment System during a
University Course
Diana Perez-Marin, Ismael Pascual-Nieto, Enrique Alfonseca, Eloy Anguiano, and

Pilar Rodriguez
Computer Science Department, Universidad Autonoma de Madrid,

C/ Francisco Tomas y Valiente, 11, Cantoblanco, 28049, Madrid, Spain
{Diana.Perez, Ismael.Pascual, Enrique.Alfonseca, Eloy.Anguiano,Pilar.Rodriguez}@uam.es
Abstract. Students should be given the opportunity of being assessed with

computers not only with objective testing such as multiple-choice questions or
fill-in-the-blanks items, but also with open-ended questions. Moreover, given
the new possibilities that adaptation brings to e-learning, the benefits of
incorporating adaptation into the assessment should also be tested. Recently, the
automatic and adaptive free-text scorer called Willow has been developed. In
this paper, we present the first study done on the impact of the use of Willow
during a course by student volunteers of our home university. The type of
students who use this system, how they use it, the impact of its use on their final
scores and the students and teacher degree of satisfaction have been analyzed. It
is concluded that students like and find useful Willow because it helps them to
review concepts and, the teacher likes the idea of having a system that provides
more exercises and feedback to the students.
Keywords: free-text scoring, e-assessment, e-learning, Natural Language

Processing
1 Introduction
Assessment is essential to learn. However, in the traditional educational environment,

teachers do not usually have enough time to accomplish a formative assessment or
assessment for learning (to give feedback about the results of the evaluation) and
summative assessment or assessment of learning (to give the numerical score as result
of the evaluation) [1]. Thus, the formative aspect is sometimes neglected. As a
possible solution to this problem, automatic computer based tests are being used since
many decades ago, starting a new area known as Computer Assisted Assessment
(CAA) with objective testing such as fill-in-the-blank exercises and Multiple Choice
Questions (MCQ).
However, some pedagogues consider that objective testing questions do not always
measure the higher cognitive skills of the students [2]. In answer to this problem, in
the mid-sixties, CAA was extended with the possibility of automatically assessing
free-text answers [3].
A study on the impact of the use of an automatic and adaptive free-text assessment 187
The advances in Natural Language Processing (NLP) have made possible a

favorable progress of this field. In fact, statistical and NLP techniques, Information
Extraction (IE), clustering, and integrated-approaches are being used to tackle this
problem with increasingly better results [4]. However, the automatic assessment of
students’ answers has not been fully solved yet, due to the complex nature of the task
proposed. Many of the current systems are limited to restricted domains, need to be
trained before they can be used, and do not take into account information of the
students to select the questions and to modify the evaluation [5].
On the other hand, it is more and more common to find applications that are
adapted to the users’ models. Moreover, the popularization of e-learning has brought
a new perspective to education, in which it is not the teacher the one that controls the
tutoring process, but the students who learn at their own pace, and according to their
knowledge background and personal features. An Adaptive Hypermedia (AH) course
presents the contents and modifies the possibilities for navigating the lessons, in a
way that is adapted to the students’ models.
Recently, the Willow system [6] has been developed. It is the first free-text
Adaptive CAA system (ACAA) able to process students’ answers written in Spanish
or in English in an automatic and adaptive way. It uses techniques from the NLP and
AH fields. Besides, it is also able to generate from the students’ answers, their
conceptual model (i.e. the structure of concepts and relationships among them). In this
paper, we present a study on the impact of use of Willow when it has been used by a
group of students in our home university. In particular, the type of students who use
this system, how they use it, the impact of its use on their final scores and the
students’ and teacher degree of satisfaction will be analyzed. It is concluded that
students like and find useful this kind of systems because it helps them to review
concepts (particularly in the days before the exam) and, teachers like the idea of
having a system that provides them more exercises and feedback to the students.
The article is organized as follows: Section 2 describes the Willow system; Section
3 details the experiment carried out, focusing on the most relevant items under study
(type of students, how they use the system, scores achieved and degree of
satisfaction); and, finally Section 4 ends with the main conclusions and lines of future
research.
2 Willow
Willow is an automatic and adaptive free-text CAA (ACAA) system1. It gets as input
a short answer (one or two paragraphs) written by a student in Spanish or in English
in free text (without templates or any kind of restriction) and a set of correct answers
written by the teachers in free text or references (again without templates or any kind
of restriction) and produces as output: a numerical score, the processed answer and
the correct answers of the teachers. Figure 1 shows an example of its interface when
asking a question to the student.
1
Available on-line at http://orestes.ii.uam.es:8080/ateneaAdaptativa/jsp/loginAtenea.jsp.
188 Diana Perez-Marin et al.
Fig. 1. A question delivered by the system according to the settings provided.
The main aim of Willow is not to substitute the teacher but to allow the students to
have more training available before exams. Its core idea is to compare the student’s
answer to a set of correct answers called the references. The more similar a student’s
answer is to the references, the higher the score the student achieves. The references
can be written by several teachers (each teacher writes a reference per question, so
that finally, there are at least three different references per question to cover as much
as possible paraphrasing in the students answers) or they can be the best students
answers of the last year training.
The system is underpinned by a combination of Natural Language Processing
(NLP) and Adaptive Hypermedia (AH) techniques. The NLP techniques are included
in the wraetlic NLP tools 2 , including stemming (to transform each word to each
canonical form, e.g. cars to car), closed-class words removal (to ignore meaningless
words such as prepositions or determiners), term identification (to group multiwords
and get a better knowledge of the students’ concepts, e.g. to consider Operating-
System instead of Operating and System as two independent words), Word Sense
Disambiguation (WSD) (to identify the sense in which a polysemous word is being
used) and Latent Semantic Analysis (LSA) (to capture the semantic similarity
between the words) [7].
The comparison between the student’s answers and the references is performed by
the statistical module called ERB. This module uses a modified version of the Bleu
algorithm [8]. This algorithm does not require any training or special processing to
the texts. It accepts as input the student’s answer and the references in plain text and
produces as output a score of the similarity between them. However, the texts can be
processed before entering the ERB module with the techniques selected in the
2
Available at www.ii.uam.es/˜ealfon/eng/download.html
configuration step in order to make them more comparable. Besides, the score given
by ERB can be combined with the similarity score given by the LSA module [7].
The optimum combination found is stemming, removal of closed-class words, LSA
and statistical techniques achieving up to 56% average Pearson correlation between
the scores given by the teachers and Willow’s scores to the same set of questions.
The AH techniques comprise the management of static students’ models including
a set of features such as language or age. Teachers are asked to choose which features
should be considered in each new collection of questions and, to write several
versions of the statement and references for each question according to the features
chosen. That way, Willow is able to present the right statement of the question and
assesses it using the right versions of the references for each student’s model.
A dynamic procedure that selects the next question to ask depending on how the
student has answered the previous questions and the difficulty level of the questions
have also been incorporated in Willow. Each question has been associated a difficulty
level from 0 (easy) to 2 (difficult) by the teacher. Next, each student is assigned a zero
level of difficulty in all the topics of the course. During the assessment session as the
students answer questions of different topics chosen according to their difficulty
levels, the values are modified to adjust the level of difficulty of the questions to the
level of knowledge that each student has in each topic addressed in the collection.
In particular, students are promoted to a higher level of difficulty in a topic when
they pass a certain percentage (whose value is configurable) of the questions in the
level in which they are. Besides, when they fail a certain percentage (also
configurable) of the questions, they will be dropped to the lower level (if they have
not passed all the questions of that level and they are not in the lowest level).
Besides, to foster reflective thinking students are not permitted just to give an
answer in blank. Moreover, whenever a student fails a question, it is not directly
provided the teachers’ references. Instead, a set of clarification questions is presented:
− The first clarification question (CQ) just asks for more information about the
question (to distinguish if the student knew the answer to the question but did not
bother to answer it or, indeed, if the student did not know the answer).
− The second CQ asks about a certain concept that is related to the question.
− The third CQ is a yes/no question in which the system presents a statement
extracted from the references and the students have to say whether it is true or false.
There is an automatic negation module that is in charge, of randomly changing the
verb to a negative form or using the antonyms of certain adjectives, so that students
do not learn that they just have to always say that the statement is true as it is
copied from the references.
A sample set of clarification questions would be as follows:
Willow’s question What is an operating system (OS)?
Student’s answer It is a process with threads.
Willow’s 1st CQ Please, explain your previous answer more.
Student’s answer It is the first application executed in the computer.
Willow’s 2nd CQ Please, explain more about application.
Student’s answer The OS is an application that serves as interface between
the hardware and the user.
Willow’s 3rd CQ Is it true that Unix and Windows are examples of OSs?
Student’s answer Yes.
To sum up, once the answer has been introduced in Willow, it is applied a set of
NLP techniques so that it can be compared against the references that have also been
processed by the same set of references. Next, if the question has been passed, a new
question is presented of the same level of difficulty if the student has not passed the
percentage of questions necessary to go up to a higher level of difficulty. On the other
hand, if the question is failed, the set of clarification questions starts and the student is
asked until s/he passes the question or, the question is marked as failed to be asked in
the future.
3 Experiment
During the course 2006-2007, an experiment was carried out with a group of students
of the Operating Systems subject of the Telecommunications Engineering career. The
goal of the experiment was to study how students and, which type of students use
Willow during the course when they are given free access to the system, accessible
from any computer connected to Internet at any time.
The spirit of the experiment was of blended learning. That is, to complement the
traditional teaching with the new technologies (in this case, the use of Willow after
class). Besides, to avoid that students think that it was extra homework, the use of
Willow was voluntary and individual. In fact, the motivation given by the teacher to
the students was that it will count positively in the final score of the students who
were between failing and passing the subject. Additionally, we also explained the
students that the questions introduced in the system were gathered from previous year
exams and, that they contain the references (correct answers) written by the teachers.
Willow was set up to use stemming, removal of closed class words and ERB. LSA
was not used because although it provides a higher correlation, it also needs a big
corpus to train the system and, we did not have any available at the time of
experiment. Nevertheless, the improvement in the correlation when using LSA is
quite low (54% average Pearson correlation without LSA and 56% average Pearson
correlation with LSA).
The Operating System course in Willow consists of 5 topics, with 4 questions per
topic. The teacher only chose for this course, the language as feature. Thus, each
student had to register in the system to give the information about his or her personal
data and language. Next, we asked the students to try to complete 4 questions per
week (i.e. a topic per week) and, finally to fill in a satisfaction questionnaire that was
also on-line.
3.1 Type of students
From the 59 students enrolled in the course, 24 logged into the system (i.e. 40.7%)
summing a total of 172 answers provided to the system. From these 24 students, it can
be considered that 9 (37.5%) completed the use of Willow as they finished at least
half of the total number of answers in the system. We expected this value (less than
50% of the students completing 10 questions or more) as in a preliminary
questionnaire in which we asked students how many hours they study Operating
Systems per week, all of them answered less than 5 hours as they had too much work
with other subjects. Therefore, we were unsure whether they would do any activity at
all, that was not compulsory.
In fact, as one of the goals of the study was to analyze which type of students are
willingly to use automatic and adaptive free-text scorers, at the beginning of the
experiment, we fixed two features to classify the students of the class: skill and
interest. Both of them were taken in a range from 0 (complete lack of this feature) up
to 1 (complete existence of the feature). Skill was measured according to the
numerical scores given by the teacher in the exam of the course (i.e. a student who
achieves the maximum score in the exam will have 1 in skill). Interest was measured
according to the number of questions answered in Willow (i.e. a student who has
answered the 20 questions will have 1 in interest).
Fig. 2.A plot of the skill and interest of the students who did not complete the use of Willow.
Fig. 3.A plot of the skill and interest of the students who completed the use of Willow.
Once the experiment finished, we implemented an automatic classifier of types of

students that scales the number of questions answered in Willow and the score
achieved in the exam to the values of skill and interest. These values are plotted for
the students who did not complete the use of Willow in Figure 2 and for the students
who completed it in Figure 3. It can be seen that students who used Willow had a
higher level of skill than students who did not complete the use of Willow. However,
it should also be noted the cases in which students with low interest has been able to
achieve a high score and, the case of students with high interest but unable to achieve
a high score despite their effort.
Regarding students who failed the last year the subject and had to course it this
year again, we were unsure about whether they would use the system more than other
students (as they wanted to make sure that this year they will pass the subject) or, if
they would not use the system (as they are students with low level of interest). In
total, 9 students from the 59 were in this situation (retakers). From them, only 2 used
Willow. That is, they did not show a higher trend of using this kind of support
systems than non-retakers students.
3.2 How the students use the system
Willow keeps a detail log of all the actions that the students perform with the system.
Table 1 shows a summary of the logs gathered. A session is each time that a student
has logged in the system. It can be seen how only six students have used an end-of-
session condition different of just closing the application and, in average, they have
answered eleven questions per session. It is also interesting to observe how these
students have exploited the Willow’s personalization features as nearly all of them
have changed the text area size and the font size.
Table 1. Summary of the logs gathered in the experiment of the use of Willow.
Fig. 4. Number of questions answered week by week from October 2006 to January 2007 by
the students who used Willow.
Another interesting fact is how the number of sessions before important exams
dramatically increases as can be seen in Figure 4. Week 1 corresponds to the first
week of the experiment (October 16th to 22nd). It is considered that the number of
students who entered the system the first weeks is just because they were curious
about the system they have been told in class. Next, the second peak in the graph is
around the weeks 5-7 given that the first exam was the November 28th in the seventh
week of course. After that, despite students have been advised to complete at least
four questions per week to have a complete training before the final exam, there is
less use of the system until the end of December - beginning of January when students
are again reviewing, in this case, for the final exam at the end of January. Thus, it can
be seen how the system is used by the students to get more training before their
exams.
3.3 Impact of Willow’s use
First of all, it is important to keep in mind that Willow is not a system to teach new
concepts but just to review concepts (formative assessment). In order to validate this
supporting function, we have examined the results achieved by the students (both
using and not using Willow) in the final exam of the course.
From the 59 students enrolled in the course, 51 (86.44%) took the final exam. If we
just focus on the students who completed the use of Willow, all of them took the final
exam. Furthermore, the two highest scores in the class are among the students who
completed the use of Willow and, the retakers who used Willow, passed the subject
while most of the retakers who did not use Willow failed again the subject.
In fact, the mean score (in a scale from 0 to 10) of the 42 students who took the
exam without having used Willow is 4.6 (1.6 deviation), whereas the mean score in
the same scale of the 9 students who took the exam having used Willow is 5.2 (1.5
deviation). That is, in average, students who reviewed with Willow got more than half
point in the final score than students who did not review with Willow.
3.4 Degree of satisfaction
During all the experiment, students have stated how much they like the idea of
using an automatic system to reinforce concepts even when they did not have much
time to use it. In the middle of the course, we asked the whole class to fill in a
voluntary questionnaire in which two items about the use of Willow were included (I
have used Willow and I think that... and I have not used Willow because...).
From the sixteen students who answered the questionnaire, six students gave an
answer to the first item. In general, all of them agreed that Willow is a very
interesting system that gave them the possibility of getting more training before their
exams. Furthermore, students did not only state that they think that it is useful for
them but, also, for other subjects and, that they would recommend it to their
colleagues. Ten students answered that they have not used Willow yet and, the reason
was unanimous: lack of time as the amount of compulsory work from the practical
part of the subject is too high. However, they were all also aware that they need to
study more, if they want to pass the theoretical part of the subject.
At the end of the course, we passed another voluntary questionnaire to gather the
final conclusions of the students about the use of Willow and, how they have enjoyed
using it. From the five students who answered the questionnaire, it can be stated that:
− All the students have Internet at home and they prefer to use Willow from their
home.
− They regard the dynamic adaptation in the order of the questions as very useful and,
like that the difficulty of the questions starts in a low level.
− All the students find very easy to use Willow and useful to review concepts.
− 80% of the students say that the system lives up their expectative, feel satisfied
with its use and would recommend its use (even in other subjects).
− The other 20% claimed that they cannot say that, because they do not like to be
presented again and again questions that cannot pass. This feeling of repetition
could be explained because there were only 4 questions per topic.
− All the students highlight the possibility of having immediate feedback as very
helpful.
− The teacher has also supported the use of Willow during all the course. In fact, he
considers very interesting the opportunity of giving extra training to their students
with a system that also gives them immediate feedback.
4 Conclusions and future work
It can be concluded that students and teachers enjoy and find helpful the use of an
automatic and adaptive free-text scoring system. Furthermore, that all students who
used Willow to review (answering more than 12 questions) took the final exam and
passed it with higher scores than students who did not use it.
However, it has also been observed how students tend to dedicate less than 5 hours
per week to study Operating Systems at home. Therefore, despite they consider that
they have to review more, and we recommended them to answer 4 questions per
week, most of them only used massively the system in the days before exams.
In the future, as it has been concluded that using Willow helps the students, we
intend to make compulsory the use of Willow in one subject (not related to computers
so that we also studied the impact of Willow on non-technical students) and to force
them to organize their study so that they do not leave all the review in the previous
days of the exam. Finally, it is also planned to improve the classification of students
to analyze if the use of Willow can motivate and keep engaged a student with low
interest (albeit at the beginning s/he did not want to use the system) up to a medium or
high level interest.
Acknowledgments. This work has been sponsored by Spanish Ministry of Science

and Technology, project no. TIN2004-03140.
References
1. Gardner, J., ed.: Assessment and Learning, SAGE Publications (2006)

2. Birenbaum, M., Tatsuoka, K., Gutvirtz, Y.: Effects of response format on diagnostic
assessment of scholastic achievement. Applied psychological measurement 16 (1992)
3. Page, E.: The imminence of grading essays by computer. Phi Delta Kappan 47 (1966)
4. Valenti, S., Neri, F., Cucchiarelli, A.: An overview of current research on automated essay
grading. Journal of Information Technology Education 2 (2003) 319–330
5. Perez, D.; Alfonseca, E. & Rodriguez, P. Can computers assess open-ended questions?,
Revista Novática 183 (2006)
6. Perez-Marin, D., Alfonseca, E., Rodriguez, P.: On the dynamic adaptation of computer
assisted assessment of free-text answers. In proceedings of the Adaptive Hypermedia
Conference, LNCS 4018, Springer-Verlag (2006)
7. Perez, D., Gliozzo, A., Strapparava, C., Alfonseca, E., Rodriguez, P., Magnini, B.:
Automatic assessment of students’ free-text answers underpinned by the combination of a
bleu-inspired algorithm and latent semantic analysis. In: Florida Artificial Intelligence
Research Society conference, FLAIRS-2005, American Association for Artificial
Intelligence (AAAI) Press (2005)
8. Papineni, K., Roukos, S., Ward, T., Zhu, W.: Bleu: a method for automatic evaluation of
machine translation. Research report, IBM (2001)
A Wiki-Based Collaborative Learning Design and Its
Effect in Secondary Math Studies
Liming Zhang1, Chan Lam Wong2

1
Faculty of Education, University of Macau, Macao SAR, PR China
2
PuiChing Middle School, Macao SAR, PR China
1
lmzhang@umac.mo, 2pcmsricky@yahoo.com
Abstract. This paper promotes use of wiki in order to facilitate the

collaborative learning in secondary math studies. An experiment was presented
in this paper to analyze the correlation between the students’ posts in the wiki
environment and their math achievement. A questionnaire was designed after
the experiment to collect the opinions from the student view. The experiment
results are positive and promising. It is concluded that after school collaborative
practice towards the homework could be one of the solutions to improve the
students’ math learning.
Keywords: Collaborative Learning, Wiki Technology, Math Learning
1 Introduction
The concept of collaborative learning has been widely studied and advocated
throughout the professional literature [5]. Collaborative learning is a method of
teaching and learning in which students make a team and work together to explore the
answer for a significant question or create a meaningful project [11]. The students are
responsible for one another's learning as well as their own. Thus, the success of one
student helps other students to be successful.
Math is one of the most difficulty subjects in the secondary studies. Many
researchers have been seeking for new pedagogies to improve the student
performance in math [10]. Collaborative learning is one of the methods emphasized in
the studies. Benefits from a collaborative environment include [3]:
z Interpersonal Development. Students learn to relate to their peers and other
learners as they work together in groups. This can be especially helpful to
students who have difficulties with communication skills. They can benefit from
structured interactions with others.
z Actively Involving Students in Learning. Each member has opportunities to
contribute in small groups. Students are apt to take more ownership of their
material and to think critically about related issues when they work as a team.
z More Opportunities for Personal Feedback. Because there are more exchanges
among students in small groups, students receive more personal feedback about
their ideas and responses. Such feedback is often not possible in large-group
A Wiki-Based Collaborative Learning Design and Its Effect in Secondary Math Studies 197
instruction, in which one or two students exchange ideas and the rest of the class
listens.
Proponents of collaborative learning also claim that the active exchange of ideas
within small groups not only increases interest among the participants, but also
promotes critical thinking [5]. Some researchers concluded that there is persuasive
evidence that collaborative teams achieve at higher levels of thought and retain
information longer than students who work quietly as individuals [6]. The shared
learning gives students opportunities to engage in discussion, taking responsibility for
their own learning, and thus become critical thinkers [14].
However, some mathematicians doubt whether collaborative learning could be used
in math classrooms without considering the major purpose of math instruction, which
is to help students learn to think mathematically [4]. They think that under the
collaborative learning class, teachers cannot share their insights with students or warn
them against pitfalls, and at the same time students cannot learn enthusiastically from
their teacher [16]. How to take the advantage of the collaborative learning and
meanwhile to release the mathematicians’ mind of doubt becomes one of our research
topics.
Carroll [2], in his classic paper, hypothesized that learning should be viewed as a
ratio of time spent in the learning activity to the actual time needed to learn. Many
empirical investigations [10] indicated that higher rates of learning are related to the
amount of time students spend actively engaged in their academic tasks. Coursework
may represent a way to increase academic learning time.
Researchers have also investigated the effects of math coursework on math
achievement, examining the impact of the numbers (amount) and the types (content)
of math courses [8]. Sebring [13] noted that the quantity of math coursework had an
effect on math achievement. Gamoran [18] reported that students performed better on
standardized math achievement tests if they took more math courses; and math
coursework particularly differentiates math achievement among students. It comes to
us that a collaborative learning based coursework might be an effective way to
improve the student math performance.
A wiki-based collaborative learning is studied in this paper. In the learning
environment, a teacher guided coursework was designed which is close to the
homework in the style. The coursework could be done after school in a collaborative
way in accompanying with the traditional teaching in the classroom. The homework
still should be done by the students individually. The advantage of the collaborative
learning is emphasized in this design, and at the same time the students’ active
learning time is effectively extended. The experiment showed positive result as
expected.
A wiki is a serve-based collaborative tool that allows any authorized user to share
information by writing new content, adding to existing content, and editing or
commenting on content. There are some applications of wiki technology reported in
the areas, including student writing [9], knowledge management systems [12], and
graduate course work design [1]. Few applications could be found in the secondary
math studies.
This paper aims to investigate an effective and efficient learning strategy as a
supplement in the traditional secondary math studies.
198 Liming Zhang, Chan Lam Wong
2 Overview of Wiki Technology
'Wiki' is the Hawaiian word for 'quick'. Leuf and Cunningham [7] define a wiki as "a
freely expandable collection of interlinked Web pages, a hypertext system for storing
and modifying information-a database where each page is easily editable by any user
with a forms-capable Web browser client".
A wiki enables documents to be written very collaboratively, in a simple markup
language using a web browser. A single page in a wiki is referred to as a "wiki page",
while the entire body of pages, which are usually highly interconnected via hyperlinks,
is "the wiki". A wiki is actually a very simple, easy-to-use user-maintained database
for creating, browsing and searching information [15].
The technical attributes of wiki technology includes [12] :
z Wikis run over the World Wide Web and can be supported by any browser.
z The technology is governed by an underlying HTTP protocol that determines
client and server communication.
z Wikis are able to respond to both requests for data (GET) and data submission
(POST), in a given Web front, based on the HTTP protocol.
From a functional dimension, three major attributes of wiki technology were
offered in the same paper:
z Any member of a wiki community can edit any pages in that community's wiki
Web site. The editing of wiki pages does not require any additional functions in
the Web browser.
z Members of a wiki community can build and develop "meaningful topic
associations" by creating numerous links between wiki pages. The linking of
wiki pages is simple to do.
z Originally the technology was not meant to engage casual visitors; rather, the
technology was designed to enable users to regularly update the wiki pages in a
collaborative fashion, thereby continuously changing the nature of the wiki Web
site.
Ward Cunningham [19] who developed the first wiki web site shares his design
principles at his web site, including the following:
• Open - Should a page be found to be incomplete or poorly organized, any
reader can edit it as they see fit.
• Incremental - Pages can cite other pages, including pages that have not been
written yet.
• Organic - The structure and text content of the site are open to editing and
evolution.
• Mundane - A small number of (irregular) text conventions will provide access

to the most useful page markup.
• Universal - The mechanisms of editing and organizing are the same as those of
writing so that any writer is automatically an editor and organizer.
• Observable - Activity within the site can be watched and reviewed by any
other visitor to the site.
Wiki has the following features and functionalities [17].

• Ability to edit a page
• View recently changed pages
• ‘History’ feature to view or roll back to previous versions of a page
Most wikis also provide the user with a set of navigation or utility tools such as
[17]:
• ‘Discuss’ offline changes or proposed changes to a page
• A ‘backlinks’ function (view all the pages that link to the page
currently displayed)
• Search function
McKiernan [17] lists eleven wiki advantages including the capability of
incorporating the knowledge of experts, peers and other professionals, the means to
annotate evolving issues, and a way of increasing communication efficiency and
productivity. He also lists seven wiki disadvantages including installation of wiki
engine software, Issues of legal liability, privacy, reputation, and security, and
Content accuracy, balance, comprehensiveness, and consistency, and reliability.
3 Research Method
An experiment was presented in this paper to analyze the correlation between the
students’ posts in the wiki environment and their math achievement. The math topic
factorization was selected as the research topic partly because it is easy to be
described using the wiki tool. One class in Form 8 was selected as the experiment
group. There were 54 students in the class. The wiki environment was designed and
developed by their math teacher himself. A questionnaire was designed and
distributed to the students after the experiment. There were 14 questions in total,
including the frequency and the purpose of the students to visit the wiki environment.
All the students completed the questionnaire.
3.1 Wiki Environment Design and Development
There were two stages in this study. In the first stage, a wiki environment was
designed and developed. MediaWiki is selected as a development tool in this research.
MediaWiki is a free server-based software, that is licensed under the GNU General
Public License (GPL). It is an extremely powerful, scaleable software and a feature-
rich wiki implementation, that uses PHP to process and display data stored in its
MySQL database [15]. Pages use MediaWiki's wikitext format, so that users without
knowledge of XHTML or CSS can edit them easily. When a user submits an edit to a
page, MediaWiki writes it to the database, but without deleting the previous versions
of the page, thus allowing easy reverts in case of vandalism or spamming. MediaWiki
can manage image and multimedia files, too, which are stored in the filesystem.
A typical wiki allows any visitors to add, remove, edit and change content without
the need for registration. In order to manage and evaluate the contents posted by the
students in the study, part resource program codes were amended. Only the students
who registered in the course were allowed to login to the proposed wiki environment.
Other visitors can browse the wiki pages, however, editing and posting are not
permitted. Figure 1 illustrates the first wiki page in the proposed wiki environment.
Fig. 1. The first wiki page in the wiki environment

The contents of the wiki pages were designed based on some important and
difficult problems discussed in the class. In most cases, the teacher provided certain
exercise topics in the wiki pages, which were closely related to their homework. The
students could either try to answer the questions if they though they were able to, or
post further questions to discuss with other students. If a solution was not correct, or
not perfect, it could be further amended by other students or by the teacher, if
necessary. The students worked on the wiki pages individually at home through
Internet. The discussions and the whole solutions posted in the wiki pages could help
them to finish their homework by themselves. Figure 2 shows the solutions of some
math exercises posted by the students.
The exercise topics posted by the teacher in the wiki pages were not the homework
itself; however, they were in the similar styles. Though neither the students were
requested to provide the solutions to the posted exercises, nor the answers were
recorded as part of the students’ final scores, there were still quite a lot students
getting involved in this learning activities. The main reasons claimed by the students
later included that the discussed topics were closely guided by the teacher and were
helpful for them to finish their homework.
3.2 Experiment Design
The second stage of the study was to investigate the correlation between the students’
participating in the wiki based collaborative learning and their math achievement.
Pearson correlation was used to test whether there was significant correlation between
the above two parties.
Fig. 2. An example of the wiki pages posted by the students
The students’ math achievement in the semester priori to the experiment was used as
pretest result. The experiment lasted two and a half months. A posttest was conducted
after the experiment. A questionnaire was also provided to the students to collect their
opinions towards the collaborative learning experience.
4 Data Analysis and Discussion
The students’ pretest and posttest data were analyzed in two aspects.
i) The correlation between the students’ pretest achievement and their posting
amount in the wiki environment.
Table1 illustrates the statistics results of the students’ posting amount based on their
pretest achievement. Table 2 shows the correlation results conducted by SPSS.
Table 1. Statistics results of the students’ posting amount and their pretest achievement
Pretest Posts in the wiki environment
Number
Scores
of the Average Standard
Range Frequency
Students Frequency Deviation
(0~100)
30-40 2 34 17.00 22.63
40-50 2 27 13.50 3.54
50-60 4 38 9.50 6.66
60-70 17 540 31.76 40.17
70-80 13 229 17.62 31.23
80-90 7 397 56.71 54.68
90-100 9 681 75.67 83.59
Table 2. Pearson Correlation between the students’ posting amount and their pretest
achievement
Posting Amount
Pretest Pearson
.369(**)
Achievement Correlation
Sig. (2-tailed) .006
N 54
** Correlation is significant at the 0.01 level (2-tailed).
The result from Table 2 shows that there is a significant correlation between the
students’ pretest achievement and their posting amount in the wiki environment. It
indicates that the better the students’ math performance before, the more their
interesting in posting to the wiki environment.
ii). The correlation between the students’ posting amount in the wiki
environment and the students’ posttest achievement.
Table 3. Statistics results of the students’ posting amount and their posttest achievement
Posttest Posts in the wiki environment
Number
Scores
of the Average Standard
Range Frequency
Students Frequency Deviation
(0~100)
30-40 0 0 0 0
40-50 2 15 7.5 2.5
50-60 1 16 16 0
60-70 12 240 20 32.7
70-80 10 247 24.7 38.2
80-90 12 462 38.5 43.2
90-100 17 966 56.8 66.3
Table 4. Pearson Correlation between the students’ posting amount and their posttest
achievement
Posting Amount
Posttest Pearson
.355(**)
Achievement Correlation
Sig. (2-tailed) .008

N 54
** Correlation is significant at the 0.01 level (2-tailed).
Table 3 illustrates the statistics results of the students’ posting amount based on
their posttest achievement. The result from Table 3 shows the students’ achievement
in the posttest was improved.
Table 4 shows the correlation results conducted by SPSS. It shows that there is a
significant correlation between the students’ posting amount in the wiki environment
and the students’ posttest achievement. It indicates that the more the students’ posting
amount in the wiki environment, the better their posttest achievement.
In the study, the students participating to the experiment were classified into three
groups based on their posting amount to the wiki environment. They were named as
frequently posting group, question-asking group, and silent group.
z In the frequently posting group, the students actively got involved into the
learning. Most posts submitted by them were solutions to the questions,
though sometimes the answers were not fully correct. Most students in this
group were top students in the class.
z In the question-asking group, the students usually posted some questions
requiring additional explanation to the solutions already posted by other
students. It showed that those students could not fully understand the
solutions posted by others, but they would like to make an effort towards
understand them. The math level of the students in this group was mostly in
the middle of the class.
z There were a few students who were relatively quiet in the wiki environment
during the whole experiment period. They just posted very few questions.
Most of the students’ math level in this silent group was low in the class.
In the questionnaire, we investigated the effects of the proposed wiki based
collaborative learning from the students’ view.
z The students from the frequently posting group claimed that they were very
excited when using the wiki because there was a kind of competition
between them. They would like to be the one who posted the correct answer
first. But they also claimed that they didn’t think that the wiki had much
effect on their math achievement.
z The students from the question-asking group claimed that although they
didn’t post as many as the ones from the frequently posting group, they got
much benefit from the question answers posted by others. It helped them to
finish their homework more smoothly. They claimed that the wiki was of
great positive effect on their posttest math achievement.
z The students from the silent group claimed that although they didn’t post
many in the wiki, they followed others’ posts in the whole experiment period.
It helped them to understand the math contents better. They also admitted
that their posttest performance was better than before and they were more
confident in math after the experiment.
5 Conclusion
The promising experimental result through wiki environment shows that the
collaborative learning seems to be one of the effective and efficient ways to improve
the students’ math achievement in secondary school. The web-based after school
collaborative environment could become a bridge connecting the knowledge students
learned in class with their own practice. The design of the after school collaborative
learning in accompanying with the traditional teaching in the classroom could not
only release the mathematicians’ doubt, but also emphasize the advantage of the
collaborative learning itself. This is in consistency with the learning theory
constructivism.
The experiment was designed and conducted in high school in Macao with the
specific topic factorization. Nevertheless the method and results may be extended to
topics and students of wide range and without geographical restriction.
References
6 Bold, M., “Use of Wiki in Graduate Course Work”, Journal of Interactive Learning
Research, 17, 1, (2006) 5-14.
7 Carroll, J.B. ,A model of School Learning, Teachers College Record, 64, (1963) 723-733.
8 Educational Broadcasting Corporation, Concept to Classroom – Workshop cooperative and
collaborative learning, Available at:
9 http://www.thirteen.org/edonline/concept2class/coopcollab/index.html. (20/03/2007).
10 Ding, M., Li, X., Piccolo, D. and Kulm, G. “Teacher Interventions in Cooperative
Learning Math Classes”, The Journal of Educational Research, 100, 3, (2007) 162-175.
11 Gokhale, A.A. , “Collaborative Learning Enhances Critical Thinking”, Journal of
Technology Education. 7, 1, (1995).
12 Johnson, R. T., and Johnson, D. W., “Action Research: Cooperative Learning in the
Science Classroom”, Science and Children, 24, (1986) 31-32.
13 Leuf, B. and Cunningham, W., “The wiki way. Quick collaboration of the web”,
Addison-Wesley, (2001).
14 Ma, X. and McIntyre, L.J., “Exploring the Differential Effects of Math Courses on Math
Achievement”, Canadian Journal of Education, 28,4, (2005) 827-852.
15 McPherson, K., Wikis and Student writing, Teacher Librarian, 34, 2, (2006) 70-72.
16 Melia, M. C. and Rosenberg, M.S., “Effects of Cooperative Homework Teams on the
Acquisition of Math Skills by Secondary Students with Mild Disabilities”, Exceptional
Children, 60,6, (1994) 538-548.
17 Panitz, T., “A Definition of Collaborative vs Cooperative Learning”, Deliberations on
Learning and Teaching in Higher Education, Available at:
http://www.city.londonmet.ac.uk/deliberations/home.html (19/03/2007).
18 Raman, M., Ryan, T., and Olfman, L., “Designing Knowledge Management Systems for
Teaching and Learning with Wiki Technology”, Journal of Information Systems
Education, 16, 3, (2005) 311-320.
19 Sebring, P.A., “Consequences of Differential Amounts of High School Coursework: Will
the New Graduation Requirements Help? ”, Educational Evaluation and Policy Analysis,
9, (1987) 258-273.
20 Totten, S., Sills, T., Digby, A., and Russ, P., “Cooperative learning: A guide to research”,
New York: Garland, (1991).
21 Wikipedia--the Free Encyclopedia, Available at: http://en.wikipedia.org/wiki/Main_Page.

(20/03/2007).
22 Wu, H., “The Math Education Reform: Why You Should Be Concerned and What You
Can Do”, The American Mathematical Monthly, 107, (1997) 946-954.
23 McKiernan, G., “Wikis: Disruptive Technology for Dynamic Possibilities”, Available at:
http://nl.wikipedia.org/wiki/Hoofdpagina (20/03/2007).
24 Gamoran, A., “The Stratification of High School Learning Opportunities”, Sociology of
Education, 60, 3, (1987) 135-155.
25 Cunningham, W., “Wiki Design Principles”, Available at
http://c2.com/cgi/wiki?WikiDesignPrinciples (20/03/2007).
Implementation of an E-Learning System – Learning
Design and Learning Objects Management and
Evaluation through Standardization of Resources
Hugo Rego, Tiago Moreira, Francisco José Garcia
University Of Salamanca, Plaza de la Merced s/n

37008 Salamanca, Spain
hugo_rego05@yahoo.com,thm@mail.pt,fgarcia@usal.es
Abstract. AHKME e-learning system main aim is to provide a modular and

extensible system with adaptive and knowledge management abilities for
students and teachers. This system is based on the IMS specifications
representing information through metadata. Metadata is used to satisfy
requirements like reusability, interoperability and multipurpose. The system
provides authoring tools to define learning methods with adaptive
characteristics, and tools to create courses allowing users with different roles,
promoting several types of collaborative and group learning. It is also endowed
with tools to retrieve, import and evaluate learning objects based on metadata,
where students can use quality educational contents fitting their characteristics,
and teachers can of use quality educational contents to structure their courses.
The metadata management and evaluation play an important role in order to get
the best results in the teaching/learning process.
1 Introduction
In the era we live in, information and its transformation into knowledge became
crucial that’s why standardization became important because it provides a semantic
representation of knowledge through ontologies in which concepts are clearly and
unambiguously identified, also providing a set of semantic relation types which allow
representing meaning by linking concepts together [15][5].
In order to develop our system, Adaptive Hypermedia Knowledge Management E-
learning system (AHKME), we had to choose from among several existing
technological standards and specifications the ones that best fit our needs in order to
reach our objectives of multipurpose, independence of the learning domain,
reusability and interoperability of resources and courses, since several standards and
specifications have been developed to structure pedagogical contents and to allow the
characterization of a wide variety of learning environments [20]. AHKME uses both
knowledge representation and management based on metadata described by
specifications where teachers can create, evaluate, import and retrieve quality
educational resources, and students aquire knowledge through quality learning objects
(LO), as well as through the most appropriate learning technique based on their
Implementation of an E-Learning System 207
characteristics and learning style, available learning activities, instructional design

and LO characteristics.
The goals of AHKME and main contributions are: the LO management and quality
evaluation; Use of the IMS specifications to standardize all the resources of the
platform; the interaction of subsystems through the feedback between them allowing
the platform to adapt to the students and teachers characteristics and to new contexts.
In this paper we will start to present an analysis of e-learning current approaches
and a standards and specifications comparative analysis to find the best to develop our
system. Then we describe the system and focus on applications that provide the LO
management and evaluation. Finally we present the conclusions and future work.
2 Current Approaches
In order to start developing AHKME we have analysed several e-learning systems,

though this analysis has to be an evaluative methodical process of several aspects,
where we have to consider the system’s main target, if it is an enterprise or
educational environment and a training or educational implementation, the students’
pre-knowledge and IT skills, the platform’s support in terms of resources and
infrastructures, and the criteria basis should be quality.
An e-learning platforms/systems analysis can be done empirically, which is a more
technical analysis considering the platforms’ features, tools and potentialities, in
terms of availability and quality (9-available, x–not available). So, we have analysed
Blackboard, WebCT, IntraLearn, Angel, Atutor, Moodle, Sakai and DotLRN like
shown on Table 1, in order to identify strong points and weaknesses, so we could try
to use them in the development of our platform [4][7][10].
From Table 1 we found that almost all the platforms have good administrative and
communication tools, compliance with standards, high implementation level and good
documentation, though they have some problems regarding LO management, sharing,
reusability and quality evaluation, resources adaptation to the students’ characteristics,
among others. From the comparison of commercial and freeware/open-source
platforms we found that the commercial ones have more difficulty integrating with
other systems and supporting different kinds of pedagogies and of course the costs.
These weaknesses are mainly traduced in problems regarding interoperability, re-
usability and quality of resources, learning domain independence, extensibility of the
platforms, meeting some of our goals already presented. In order to solve these
problems we have decided to develop an open source platform focused on these issues.
3 Standards and Specifications Comparative Analysis
One of the biggest difficulties of e-learning systems/platforms is in structuring content

and information using nowadays pedagogical models so they can reach a wider range
of educational systems and obtain a greater teaching quality, that is why standards
were developed for. As we know the use of standards have become very useful
because automatically makes everything cross systems providing this way common
208 Hugo Rego, Tiago Moreira, Francisco José Garcia
knowledge. The use of a standard helps to achieve more stable systems, reduces the
development and maintenance time, allows backward compatibility and validation,
increases search engine success, among many other know advantages [18].
Table 1. Analysis of e-learning systems

Tools/Features Platforms
Comercial Open Source
IntraLearn
WebCT
ATutor
Moodle
Angel
Sakai
.LRN
BB
Technical Aspects
Interoperability/integration 9 9 9 9 9 9 9 9
Standards and specs. Compliance (1) (6) (1) (2) (3) (1) (1) (1) (6) (6)
(2) (3) (1) (4) (5) (6) (2)
Extensibility x x x x 9 9 9 9
Adaptation and Personalization
Interface custom. and personalization 9 9 9 9 x 9 9 9
Choose interface language 9 9 9 9 9 9 x 9
Students previous knowledge x x x x x x x x
Courses and resources adaptability x x x x x x x x
Administrative
Student manage. / monitor. tools 9 9 9 9 9 9 9 9
Database access mechanisms x x 9 9 9 9 9 9
Admin. workflows quality & functio. 9 9 9 9 9 9 9 9
Tracking users 9 9 9 9 9 9 x x
Resources Management
Content authoring and editing 9 9 9 9 9 9 9 9
LOs and other types of content mng. x 9 x x x x x x
Templates to aid on content creation x 9 9 9 9 9 9 9
LO search and indexation x x x x 9 x x x
Evaluation of quality of resources x x x x x x x x
Learning objects sharing/reuse x x x x 9 x x x
Communication
Forum 9 9 9 9 9 9 9 9
Chat 9 9 9 9 9 9 9 x
Whiteboard 9 9 x 9 9 x x x
Email 9 9 9 9 9 9 9 9
Audio and video streaming x x x 9 x x x x
Evaluation
Self assessments 9 9 9 9 9 9 9 9
Tests 9 9 9 9 9 9 9 9
Inquiries 9 9 9 x x 9 x x
Costs H H H H N N N N
Documentation 9 9 9 9 9 9 9 9
SCORM-(1);IMS-(2);AICC-(3);LRN-(4);Section 508-(5);Some IMS Specifications-(6);High–H;None–N
Having detected the main problems of current e-learning approaches, we’ve started
to analyse several aspects of several standards and specifications to choose the one(s)
that would best fit our needs, like presented in Table 2.
We have analyzed the IMS Specifications [12], AICC, SCORM [1] and Dublin
Core [8], and from this analysis we’ve chosen the IMS specifications, since they
allow most of the issues we’ve analyzed and that we consider important for our goals.
Table 2. Standards and specifications comparative analysis

Features IMS AICC SCORM Dublin Core
Metadata 9 9 9
Learner Profile 9
Content Packaging 9 9 9
Q&T Interoperability 9
DR Interoperability 9 9
Content structure 9 9 9
Content Communication 9 9
Learning Design 9
Simple Sequencing 9 9
Accessibility 9
XML 9 9 9
Bindings
RDF 9 9
Implementation handbooks 9 9 9
Learner registration 9
4 AHKME Description
AHKME, presented on Figure 1, is an e-learning system that is divided in four

different subsystems: Learning Object Manager and Learning Design subsystem,
Knowledge Management subsystem, Adaptive subsystem and Visualization and
Presentation subsystem.
Fig. 1. AHKME’s structure

These subsystems were structured taking into account the following: First we have
the LO creation and management process, followed by the course creation process
through learning design (LD). In parallel the Knowledge Management subsystem
evaluates LOs’ and courses’ quality that then pass through an adaptive process based
on the students’ characteristics to be presented to them.
These subsystems are web applications that were developed using Asynchronous
JavaScript And XML (AJAX) to create interactive web applications [3], HTML and
CSS for the Web pages’ design, PHP (PHP: Hypertext Preprocessor) to run on server
side to make the manipulation of XML files, Javascript to implement mechanisms in
Web forms and .NET and C to implement several software agents. All of these
subsystems use XML as standard for file storage and knowledge representation which
allows content interchange between different applications and platforms, facilitating
content publishing [6]. All LO management and LD subsystem tools include a
mechanism that packages the generated information, at the level of LOs, courses as
well as at the level of the adapted courses. We will now focus on the parts of this
system that provide the LO management and evaluation through metadata.
4.1 LOM and Learning Design Subsystem
The Learning Object Management and Learning Design subsystem is mostly used by
teachers where they can develop, search, retrieve, import and analyze resources and
also create courses. We will now describe the tools and features of this subsystem and
how they are related with the IMS specifications.
4.1.1 LO Manager
The Learning Object Manager tool, presented on Figure 2, allows teachers to

define/create metadata to describe LOs through IMS Learning Resource Metadata
specification which is based on the IEEE Learning Object Metadata (IEEE LOM) [11]
standard that allows the knowledge management and representation through LOs.
Fig. 2. LO Manager Architecture

This tool structures all information in a XML manifest, that gathers all XML files
with LO’s metadata and resources facilitating the learning contents’ management. It
allows the creation of general metadata that can be associated to any LO and the
creation of packages with their manifests and LOs and their storage in a MySQL
database, enabling their management. All these files and packages pass through a
validation process to check their conformance with the specifications, and the
communication between tools and databases is based on the XML Document Object
Model.
The LOs are not static in the repositories, but they are in constant evaluation by the
KM subsystem. After the evaluation it may be needed to change the LOs’ cataloguing
or the way they are related with others to get better associations letting these changes
to be reflected until the packages’ creation, taking into account the user’s wishes,
granting a higher level of flexibility.
The main advantage of using the IMS specification for LOs is that through the
association of descriptive tags, we can better index, find, use and reuse them.
4.1.2 LD Editor
The subsystem’s part referring to the Learning Design (Figure 3) provides a tool
where teachers can create and structure courses using level A of the IMS LD
specifications defining courses’ activities, sequence, users’ roles, student or staff, and
metadata. It generates a XML manifest gathering all the courses’ XML files, LOs,
metadata and resource files. With the usage of XML files information can be reused
in the construction of other courses facilitating the learning information portability
[13].
Fig. 3. Learning Design Tool architecture

This tool also provides the package creation with courses integrating them in a data
repository, to reach a more efficient management and, also, communicates with the
KM subsystem to evaluate the courses created. After the evaluation this tool allows
the courses’ restructuring allowing the user to interact in the learning design process.
4.2 Knowledge Management Subsystem
We’ve decide to create a subsystem which main objective is to assure quality to the
information in the platform through the LOs’ and courses’ evaluation, to get the best
courses and the best resources to reach to the best learning/teaching process.
We will now describe how the learning object evaluation is processed.
4.2.1 LO Evaluation
The quality of learning resources is becoming an aspect with great importance on e-

learning environments, since when e-learning systems emerged there was a massive
production of resources without taking into account their quality. Vargo, et. al states
that a systematic evaluation of learning objects must become a valued practice if the
promise of ubiquitous, high quality Web-based education is to become a reality [19].
To archive an optimal evaluation of LOs, it’s necessary to consider quality criteria
from different kind of categories, for this reason the criteria with the respective
weight presented on Table 3 were proposed [16].
Table 3. Evaluation criteria categories and matching with the IMSLRM educational category
Eval. criteria Weight IMSLRM Ed. Description
categories elements
intended end user role; Criteria that can evaluate, for example,
Psychopedagogical 30% typical age range; if the LO has the capacity to motivate
difficulty the student for learning;
learning-resource type; Criteria to evaluate if the LO helps to
context; typical archive the unit of learning objectives,
Didactic-curricular 30%
learning time; etc;
description
semantic density; Criteria to evaluate the legibility of the
Technical-aesthetic 20%
language LO, the colors used, etc;
interactivity type; Criteria to evaluate LOs accessibility
Functional 20% interactivity level among other aspects to guarantee that it
doesn’t obstruct the learning process;
The final evaluation is the sum of all category classifications multiplied by their
weight and has following rating scale: 0=not present; 1=Very low; 2=Low;
3=Medium; 4=High; 5=Very High.
With these criteria, we’re developing two different tools to evaluate LOs’ quality.
One tool allows teachers and experts to analyze, change and evaluate LOs through a
Web application and after the individual evaluation, all the persons involved gather in
an on-line forum to reach to the LO final evaluation [16].The other tool is an
intelligent agent that automatically evaluates LOs which architecture is shown in
Figure 4.
Fig. 4. Architecture of the agent

The agent starts to import the LO to evaluate and others already evaluated, then
applies data mining techniques (decision trees) to the LO educational characteristics
defined in the IMSLRM specification to calculus its final evaluation and agent stores
it on a database and on the annotation element described in the specification.
For this evaluation we matched the IMSLRM educational category elements and
the evaluation criteria categories like presented on Table 3. We have just considered
the educational category because it has almost all LOs’ technical and educational
aspects we found important.
With these two tool LOs are constantly being availed of their quality, playing an
important role in the reusability of the LOs for different contexts.
5 Integration with Other Systems
The objective of integration with other systems is to give an opportunity for a LMS
to benefit from this Learning Design Back-Office system, as well as to give a Front-
End to AHKME as presented on Figure 5.
Fig. 5. Presentation Architecture

Our objective is to benefit from all the LMSs’ strong points already mentioned on
our analysis adding the tools we have developed by merging/integrating systems
being possible depending on the LMSs’ integration tools. For example if you have an
open source system it can be directly integrated or if you have Blackboard it can be
done through building blocks.
6 AHKME LOM Vs Similar Tools
We have also done an analysis of key features of some metadata tools along with
AHKME’s LO Manager (LOM Editor [14], ADL SCORM [2], Reggie [17] and EUN
[9]). To make this analysis we have defined a set of tasks like the ones described on
Table 4 and tested if the different tools supported them.
The analysed tools provide functionalities for meeting specific requirements like
XML validation and metadata files creation, lacking important issues like: educational
orientation, by not providing a list of available educational metadata; require that the
person who edits metadata must know XML; functionalities regarding the user’s
needs to characterize several learning environments; resources’ management.
Table 4. Comparative analysis between AHKME LOM tool and similar tools
AHKME
SCORM
Reggie
Editor
LOM
LOM
ADL
EUN
Task
Creation of new metadata files 9 9 9 9 9

Modification of data in metadata files 9 9
Support any educational metadata standard, specification 9 9
Modification of structure of metadata files 9
Validation in terms of data values 9 9 9
Validation of structure of metadata 9
Support of the XML 9 9 9
Packaging of LOs metadata 9
Evaluation of LOs metadata 9
LO Search and Indexation 9
Allow metadata document management 9
So, AHKME LOM distinguishes itself from the others by introducing an

abstraction level to the user from the technical aspects in terms of the XML language
and is more focused on the user needs, by facilitating the metadata annotation of the
LO through a metadata automation process and the search and retrieval of the LO, for
the user to reuse the LO in another scenarios. Because of AHKME’s LO quality
evaluation, the user may choose the best LOs that best fit his educational scenario.
7 Conclusions
In this article we have presented how AHKME uses metadata for learning resource
management and evaluation through the IMS specifications, which use combine
potentialities of metadata and XML, providing LOs’ description through metadata
allowing them to be catalogued, localized, indexed, reused and interoperable.
The main AHKME contributions are: the LO management and quality evaluation;
the usage of the IMS specifications to standardize all the platforms’ resources in order
to reach learning components interoperability and compatibility; All subsystems
interaction through feedback allowing the platform to adapt to the students and
teachers characteristics and to new contexts; Being a multipurpose system it can be
applied to several kinds of matters, students and learning strategies in both training
and educational environments; As modular and open source system allows developers
to add new modules and extend the system or integrate it with already develop e-
learning systems and tools; through knowledge management the content has
continuous evaluation, granting quality to all resources in the platform for teachers
and students to use.
In terms of future work, we will include in the learning design tool, the level B of
the IMS LD specification that allows the inclusion of properties and general
conditions. In the adaptive subsystem we will add some functionality according to the
IMS Question and Test Interoperability and Enterprise specification. In the
knowledge management subsystem we will add the feature of course quality analysis,
through the development of a standardization knowledge model and evaluation tools.
Acknowledgments. This work has been partly financed by Ministry of Education and
Science as well as FEDER KEOPS project (TSI2005-00960).
References
1. ADL: Sharable Content Object Reference Model (SCORM)® 2004 3rd Edition - Overview
Version 1.0. Advanced Distributed Learning (2006)
2. ADL SCORM Metadata Generator (2005): http://www.adlnet.org
3. AJAX: Asynchronous JavaScript And XML (2007).
http://www.w3schools.com/ajax/default.asp,
4. ANGEL (2005): http://www.angellearning.com/
5. Berners-Lee, T., Hendler, J., Lassila O.: The Semantic Web. Scientific American, 284(5),
pp.34-43 (2001)
6. Bray, T., Paoli, J., and Sperberg-MacQueen, C.M.: Extensible Markup Languaje (XML) 1.0,
3rd Edition, W3C Recommendation (2004)
7. Colace, F., De Santo, M, Vento, M.: Evaluating On-line Learning Platforms: a Case Study,
HICSS’03 – Hawaii International Conference on System Science (2002)
8. Dublin Core Metadata Initiative (2005): http://dublincore.org
9. EUN (2005): http://www.en.eun.org/menu/resources/set-metaedit.html
10. Graf, S., List, B.: An Evaluation of Open Source E-Learning Platforms Stressing
Adaptation Issues. ICALT 2005 - The 5th IEEE International Conference on Advanced
Learning Technologies (2005)
11. IEEE LTSC Working Group 12: Draft Standard for Learning Object Metadata. Institute of
Electrical and Electronics Engineers, Inc. (2002)
12. IMS Specifications, IMS Global Learning Consortium, Inc. (2004)
http://www.imsglobal.org/specifications.cfm
13. Koper, R., Olivier, B., Anderson, T.: IMS Learning Design Information Model - Version
1.0 Final Specification. IMS Global Learning Consortium, Inc. (2003)
14. LOM Editor (2005): http://www.kom.e-technik.tu-darmstadt.de/~abed/lomeditor
15. Mendes, M.E.S., Sacks, L.: Dynamic Knowledge Representation for e-Learning
Applications. In Proceedings of the 2001 BISC International Workshop on Fuzzy Logic and
the Internet, FLINT'2001, Memorandum No. UCB/ERL M01/28, USA, Aug., 176-181
(2001)
16. Morales, E., García, F. J., Moreira, T., Rego, H., Berlanga, A.: Units of Learning Quality
Evaluation. In SPDECE 2004 Design,Evaluation and Description of Reusable Learning
Contents. Proceedings (Guadalajara (Spain), October 20-22, 2004). CEUR Workshop
Proceedings Vol. 117. http://ceur-ws.org/Vol-117. ISSN 1613-0073 (2004)
17. Reggie Metadata Editor (2005): http://metadata.net/dstc
18. Totkov, G., Krusteva, C., Baltadzhiev, N.: About the Standardization and the
Interoperability of E-Learning Resources. CompSysTech’2004 - International Conference
on Computer Systems and Technologies (2004)
19. Vargo J., Nesbit J.C., Belfer K., Archambault A., Learning object evaluation: computer-
mediated collaboration and inter-rater reliability. International Journal of Computers and
Applications, 25 (3) (2003)
20. Wiley, D.: Connecting learning objects to instructional design theory. In D. Wiley (Ed.):
The Instructional Use of Learning Objects,10 May 2003 (2003)
Visualisation of Learners’
Contributions in Chat Conversations
Stefan Trausan-Matu1,2, Traian Rebedea1,

Alexandru Dragan1, and Catalin Alexandru1
1
“Politehnica” University of Bucharest,
Department of Computer Science and Engineering,
Splaiul Independetei nr. 313, Bucharest, Romania
2
Research Institute for Artificial Intelligence of the Romanian Academy
Calea 13 Septembrie nr.13, Bucharest, Romania
trausan@cs.pub.ro, traian@createit.ro, alexd18@yahoo.com, cata@ew.ro
Abstract. In this paper is presented a novel dialogistic, socio-cultural

perspective and an associated software tool, which provide structured
visualisation and analysis means of Computer Supported Collaborative
Learning chat conversations. The implemented tools use knowledge-based
techniques and are based on Bakhtin’s dialogistic paradigm. They visualize the
threading of topics and utterances in the conversation and the contributions of
the participants in collaborative learning during instant messenger chats.
Natural language processing based on the WordNet lexical ontology and
semantic distances are used for detecting topics in the chat and their threading.
The experiments with the developed application were performed with students
at a course on Human-Computer Interaction in Bucharest Politehnica
University.
Keywords: Computer Supported Collaborative Learning, Dialogism, Chat

Conversations, Ontologies, Natural Language Processing
1 Introduction
In recent years, in conjunction with the omnipresence of the Internet and to the
increasing number of collaborative tools like discussion forums and instance
messaging (chat conversations), Computer Supported Collaborative Learning (CSCL)
became an extending and promising way of learning on the Internet, which could
supplement traditional classroom learning. However, CSCL needs particular
supporting tools, for example for tackling and reviewing complex dialog threads in
collaborative learning in small groups using chat conversations. The paper describes
such a tool, that offers visualisation means to analyse the threading of dialog topics
and the contribution of each learner in a chat conversation.
A theoretical background for developing tools for supporting CSCL using chat
conversations is the socio-cultural paradigm, stating that knowledge is socially built
in communities [7] (including the case of small groups of students that learn together).
218 Stefan Trausan-Matu et al.
This new paradigm is imposing itself not only due to technology advances but also
because the individual cognition perspective of classical artificial intelligence (stating
that knowledge should be considered as being in the mind of individuals) did not
fulfil all its expectations [4,14]. However, knowledge-based technology, combined
with natural language processing, has some important applications (e.g. in text mining)
and we should not throw away its potential facilities. Therefore, the approach
presented in this paper is integrating both the knowledge-based (ontology-based),
cognitive paradigm and the socio-cultural one.
Learning paradigms have also changed in a similar way, from Computer-Assisted
Instruction and Intelligent Tutoring Systems to Computer-Supported Collaborative
Learning (CSCL) [4,7]. As a consequence, learning is now conceived as discourse
building, as Sfard remarked: “rather than speaking about ‘acquisition of knowledge,’
many people prefer to view learning as becoming a participant in a certain discourse”
[6]. The way learning is considered has implications on the nature of the computer
tools designed to support it. For example, the tools described in this paper, which
offer the possibility of visualising the discourse in chat conversations, are based on
Bakhtin’s dialogistic theory [1,2], which may be seen as extending Vygotsky’s socio-
cultural ideas [12]. Knowledge-based processing techniques and the lexical ontology
WordNet (http://wordnet.princeton.edu) are used for the identification, delimitation
and visualisation of the inter-animation of the voices of the learners. In addition, an
assessment of the competence of each learner is provided.
There are chat environments for CSCL containing facilities like whiteboards and
explicit referencing. Such an environment is ConcertChat [3], used in this paper.
There are also applications that use natural language processing for abstracting (e.g.
speech acts identification [9] and summarization [10]) or knowledge extraction from
chats and forums. However, these facilities are limited, and one assumption of the
research whose results are presented here is that the limitations are due to the
neglecting of the socio-cultural paradigm.
The experiments for validating the developed application were performed with
students from the final year at the Computer Science Department of the Bucharest
Politehnica University, at a course on Human-Computer Interaction. For performing
the chat conversation, the ConcerChat was used.
The paper continues with a section introducing the socio-cultural and Bakhtin’s
dialogism paradigms. The third section discusses the knowledge-based ideas used in
the present approach. The next section contains the description of the visualisation
tools. The paper ends with conclusions and references.
2 A Dialogical, Socio-Cultural Paradigm of Learning
The socio-cultural paradigm is based on the work of the Russian psychologist Lev
Vygotsky, who emphasized the role of socially established artefacts in
communication and learning [12]. Mikhail Mikhailovici Bakhtin brought a lot of
details to the ideas of Vygotsky, analysing the role of language and discourse, and
especially of speech and dialog. Bakhtin focuses on the idea of dialogism, making it a
fundamental philosophical category, named dialogistic: “… Any true understanding is
dialogic in nature.” [11]. Moreover, Lotman considers text as a „thinking device” [13],
determining that: “The semantic structure of an internally persuasive discourse is not
finite, it is open; in each of the new contexts that dialogize it, this discourse is able to
reveal ever new ways to mean” [2].
In forums and chat conversations, group knowledge arises in discourse and is
preserved in linguistic artifacts, whose meaning is co-constructed within group
processes [5], and has a dual nature. Communities of voices, in parallel to the trend
towards unity, have an additional differential, unmerged, character: “The intersection,
consonance, or interference of speeches in the overt dialog with the speeches in the
heroes’ interior dialogs are everywhere present. The specific totality of ideas,
thoughts and words is everywhere passed through several unmerged voices, taking on
a different sound in each” [1]. This dual nature of community and individuality of
voices is expressed by Bakhtin also by the concept of polyphony, that he considers the
invention and one of the main merits of Dostoevsky novels [1]. The relation of
discourse and communities to music was remarked also by Tannen: “Dialogue
combine with repetition to create rhythm. Dialogue is liminal between repetitions and
images: like repetition is strongly sonorous” [8].
In chat conversations, different voices are obvious recognized. However, starting
from Bakhtin’s ideas, in our approach the concept of voices is not only limited to the
number of participants in the chat. A voice is, from this perspective, something said
by a participant in a given moment and that it may be reflected in many subsequent
utterances. Also, each utterance may contain an unlimited number of voices.
3 Knowledge-Based Text Processing
Ontologies like WordNet or FrameNet (http://framenet.icsi.berkeley.edu) are very

successful inheritors of knowledge representation research in artificial intelligence.
They are semantic networks or frame structures built starting from human experience
and, in fact, they are ways of sharing experience. Any collaboration using natural
language, any discourse needs to start from a common vocabulary, a shared ontology.
The word “ontology” is used in philosophy to denote the theory about what is
considered to exist. Any system in philosophy starts from an ontology, that means
from the identification of the concepts and relations considered as fundamental.
Ontologies capture fundamental categories, concepts, their properties and relations.
One very important relation among concepts is the taxonomic one, from a more
general to a more specific concept. This relation may be used as a way of “inheriting”
properties from the more general concepts (“hypernyms”). Other important relations
are “part-whole” (“meronym”), “synonym”, “antonym”.
Viewing knowledge bases as ontologies determines important advantages for
developers of knowledge-based systems. First of all, an ontology is developed as a
coherent framework for the reality and therefore it facilitates knowledge acquisition
and machine learning. A new concept is easy to add in such a framework by finding
one or some more general concepts and defining some differences between the new
concept and the more general ones.
Ontologies are very important in text mining. For these kind of applications they
offer the substrate for semantic analysis and, very important, the possibility of
defining a measure of semantic closeness, based on the graph with concepts from
ontologies as nodes and their relations as arcs. This semantic closeness is very
important in text analysis for example in the retrieval of texts that do not contain a
given word, but they contain a synonym or a semantically related word.
4 Visualization of Users’ Competences
The approach presented here integrates Bakhtin’s socio-cultural ideas with

knowledge-based natural language processing for the visualisation of the
contributions of each learner. The procedure consists in the identification of the topics
discussed in the chat, the separation of the contributions of each participant to a topic
(the voices) and, eventually, the measurement and visualisation of these contributions.
4.1 Identification of Chat Topics
The chat topics are identified in several ways in the present approach. A first method
id to detect the list of concepts (words) that appeared most frequently in the
conversation, by using statistical natural language processing methods. Accordingly,
the importance of a subject is considered related to its frequency in the chat. The first
step in finding the chat subjects is to strip the text of irrelevant words (stop-words),
text emoticons (e.g. “:)”, “:D”, and “:P”), special abbreviations used while chatting
(e.g. “brb”, “np”, and “thx”) and other words considered of no use at this stage.
Fig. 1. A fragment of a chat for a Human-Computer Interaction course, using the ConcerChat
facilities of referencing
The resulted chat text is then tokenised and each different word is considered as a
candidate concept in the analysis. For each of these candidates, WordNet is used for
finding synonyms. If a concept is not found on WordNet, mistypes are searched. If
successful, the synonyms of the suggested word will be retrieved. If no suggestions
are found, the word is considered as being specific to the analyzed chat and the user is
asked for details. The last stage for identifying the chat subjects consists of unifying
the candidate concepts discovered in the chat.
In addition to the above method for determining the chat topics, a surface analysis
technique is used. Observing that new topics are generally introduced into a
conversation using some standard expressions such as “let’s talk about email” or
“what about wikis”, a simple and efficient method is used for deducing the topics in a
conversation by searching for the moment when they are first mentioned.
The process of identifying a pattern in an utterance is done using the synset for
each word that has already been extracted from WordNet. This technique will be
improved in a future version of the application by using machine-learning methods
for detecting the patterns specific to the introduction of new topics. Another option is
to consider the extension of the simple patterns described above to more complicated
parsing rules.
4.2 The Graphical Representation of the Conversation
The graphical representation of the chat was designed to permit the best visualization
of the conversation, to facilitate an analysis based on the polyphony theory of Bakhtin,
and to maximize the straightforwardness of following the chat elements. For each
participant in the chat, there is a separate horizontal line in the representation and each
utterance is placed in the line corresponding to the issuer of that utterance, taking into
account its positioning in the original chat file – using the timeline as an horizontal
axis. Each utterance is represented as a rectangle aligned according to the issuer on
the vertical axis and having a horizontal axis length that is proportional with the
dimension of the utterance. The distance between two different utterances is
proportional with the time passed between the utterances. Of course, there is a
minimum and a maximum dimension for each measure in order to restrict anomalies
that could appear in the graphical representation due to extreme cases or chat logging
errors.
The relationships between utterances are represented using coloured lines that
connect these utterances. The explicit references that are known due to the use of the
ConcertChat software are depicted using blue connecting lines, while the implicit
references that are deduced using the method described in this paper are represented
using red lines. The utterances that introduce a new topic in the conversation are
represented with a red margin.
The graphical representation of the chat has a scaling factor that permits an
attentive observation of the details in a conversation, as well as an overview of the
chat. The different visual elements determined by our application – such as utterances
in the same topic, topic introducing utterances and relationships between topics – can
be turned on and off in the graphical representation by use of checkboxes.
At the bottom of the graphical representation of the conversation, after the line
corresponding to the last participant in the chat, there is a special area that represents
the importance of each utterance, considered as a chat voice, in the conversation (see
figure 2). How this importance is determined is presented in a further section.
Fig. 2. The threads of references in the chat
4.3 Discovering the Implicit Voices
Considering each chat utterance as being a voice that has a certain importance in the
conversation, it is obvious that each utterance generally contains more than a single
voice, as it includes the current voice and probably at least one referring voice. As we
are working with ConcertChat transcript files, we acknowledge the voices that are
explicitly pointed out by the chat participants during the conversation, using the
software’s referencing tool. Nevertheless, because users are seldom in a hurry or
simply not attentive enough, part of the utterances do not have any explicit references.
Thus, it is necessary to find a method for discovering the implicit references in an
utterance; in this way, we shall identify more relationships between the utterances in
the chat.
The method proposed here is similar to the one presented above for determining
the introduction of new chat topics. We are using another list of patterns that consists
from a set of words (expressions) and a local subject called the referred word. If we
identify that an utterance matches one of the patterns, we firstly determine what word
in the utterance is the referred word (e.g. “I don’t agree with your assessment”). Then,
we search for this word in the predetermined number of the most recent previous
utterances. If we can find this word in one of these utterances, then we have
discovered an implicit relationship between the two lines, the current utterance
referring to the identified utterance.
We have also implemented two empirical methods, which provide very good
results when utilizing any chat software. One of these empirical methods is based on
the following fact: if between three utterances there are two explicit relationships
from the first to the second and from the second to the third and the second utterance
is a short agreement or disagreement, then between the first and the third utterance
there exists an implicit relationship. For example, consider the following example,
where there are explicit references between A and B, respectively B and C, it is

clearly we have an implicit relationship between A and C. In the last utterance, we
have influences from both A and B:
A – I think wikis are the best

(…)
B – I disagree REF A
(…)
C – Maybe we should talk about them anyway REF B
4.4 Determining the Strength Value of an Utterance
Starting from existing references within the analysed conversations, both those
explicit, offered by the used chat environment, as well as those implicit determined by
the program using the previously presented methodology, one could assemble a
conversation graph. This graph may be used both for determining the strength value
of each utterance in the chat considered as a separate voice, as well as for
emphasizing certain subjects (threads) of the conversation.
The importance of an utterance in a conversation can be calculated through its
length and by the number of key (important) words. Another approach was also
investigated: an utterance is important if it influences the subsequent evolution of the
conversation. Using this definition as a starting point, we may infer that an important
utterance will be that utterance which is a reference for as many possible subsequent
utterances.
Even if this approach could be extended to include the types of subsequent
references (implicit or explicit, agreements or disagreements), in the present case we
have preferred a more simplistic approach, without making allowances for the types
of references to the utterance.
Consequently, the importance of an utterance can be considered as a strength value
of an utterance, where an utterance is strong if it influences the future of the
conversation (such as breaking news in the field of news). When determining the
strength of an utterance, the strength of the utterances which refer to it is used. Thus,
if an utterance is referenced by other utterances which are considered important,
obviously that utterance also becomes important.
As a result, for the calculation of the importance of every utterance, the graph is
ran through in the opposite direction of the edges, as a matter of fact in the reverse
order of the moment the utterance was typed. Utterances which do not have
references to themselves (the last utterance of the chat will certainly be one of them)
receive a default importance – taken as the unit. Then, running through the graph in
the reverse order of references, each utterance receives an importance equal to that of
the default plus a quota (subunit) from the sum of the importance of the utterances
referring to the current utterance. Another modality to calculate could be 1 plus the
number of utterances that refer to the present utterance, but this choice seemed less
suitable.
By using this method of calculating the importance of an utterance, the utterances
which have started an important conversation within the chat, as well as those
utterances which begin new topics or mark the passage between topics, are more
easily emphasized. If the explicit relationships were always used and the implicit ones
could be correctly determined in as high a number as possible, then this method of
calculating the importance of a voice would be successful.
4.5 Assessing the Competencies of the Learners in the Conversation
In order to determine the competences of the chat users, we first searched the most
important topics in the analyzed chat conversation. The generated graphics evaluate
the competences of each user, starting from the list of subjects determined as
explained above and using other criteria such as questions, agreement, disagreement
or explicit and implicit referencing. The graphics are generated using a series of
parameters like: implicit and explicit reference factors, bonuses for agreement,
penalties for disagreement, minimum value for a chat utterance, penalty factors for
utterances that agree or disagree with other utterances as these utterances have less
originality than the first ones.
Fig. 3. The evolution of the competence degree
During the first step of the graphics generation, the value of each utterance is
computed by reporting it to an abstract utterance that is built from the most important
concepts in the conversation determined as described above. When constructing this
utterance, we take into account only the concepts whose frequency of appearance is
above a given threshold. Then, all the utterances in the chat are scaled in the interval
0 – 100, by comparing each utterance with the abstract utterance. The comparison is
done using the synsets of each word contained in the utterance. Thus, this process
uses only the horizontal relations from WordNet. An utterance with a score of 0
contains no words from the concepts in the abstract utterance and an utterance with a
score of 100 contains all the concepts from the abstract utterance.
On the Ox axis the graphics hold all the utterances in the chat and on the Oy axis
the value attributed to each participant in the conversation, representing each user’s
competence (see figure 3). Accordingly, for each utterance, at least the value of a user
competence is modified – the value for the user that issued that utterance.
For each utterance in the chat, the values of the users’ competences are modified
using the following rules:
1) the user that issued the current utterance receives the score of the utterance,
eventually downgraded if that utterance is an agreement or disagreement in relation to
a previous utterance (in order to encourage originality);
2) all the users that are literally present in the current utterance are rewarded with a
percentage of the utterance value, considering that they have some merit in the value
of this utterance, as being mentioned in the text of the utterance encourages us to
think so;
3) the issuer of the utterance explicitly referred to by the current utterance is
rewarded if this utterance is an agreement and is penalized if the utterance is a
disagreement;
4) the issuer of the utterance explicitly referred to by the current utterance that is
not an agreement or a disagreement, will be rewarded with a fraction of the value of
this utterance; and
5) if the current utterance has a score of 0, the issuer will receive a minimum score
in order to differentiate between the users that actually participate in the chat and
those who do not.
All the percentages and all the other factors used for computing the competence of
each user are used as parameters of the process and can be easily modified in the
application interface. The process described above builds competence function
graphics for each participant in the chat. At the start of the process, each user has a
null competence. It should be mentioned that the competence of a user is not a strictly
increasing function, as users are penalized for utterances that are in disagreement with
the other users’ opinions.
5 Conclusions
The paper presents an application that visualizes the voices (following Bakhtin’s ideas)
of the participants on forums or chat conversations, similarly to music scores. In
addition, some other diagrammatic representations are used for viewing the influence
of a given speaker’s voice.
The application may be used for inspecting what is going on and in what degree
learners are implied in a forum discussion or a chat conversation. Moreover, the
competence of each participant may be measured, that means that learners may be
assessed in collaborative learning on the web.
The application uses the WordNet ontology. Knowledge acquisition for concepts
that are not present in this ontology is provided through dialogs with the user of the
analysis system and by caching the results. Natural language technology is used for
the identification of discussion topics, for segmentation and for identifying implicit
references.
Further work will consider more complex semantic distances (than only
synonymy). Machine learning techniques will be used for the identification of
discourse patterns. New rules for the identification of implicit links are now under
development.
Acknowledgments. The authors wish to express their appreciation to the members of

the Virtual Math Teams research project at Drexel University, whose voices are
present in different ways in the paper. The research presented here has been partially
performed under a Fulbright Scholar post-doc grant (awarded to Stefan Trausan-
Matu), the EU-NCIT EU excellence centre and the CNCSIS project K-Teams. Any
opinions, findings, or recommendations expressed are those of the authors and do not
necessarily reflect the views of the sponsors.
References
1. Bakhtin, M.M., Problems of Dostoevsky’s Poetics, Ardis, (1973)

2. Bakhtin, M.M., The Dialogic Imagination: Four Essays, University of Texas Press, (1981)
3. Holmer, T., Kienle, A., Wessner, M. “Explicit Referencing in Learning Chats: Needs and
Acceptance,” in Innovative Approaches for Learning and Knowledge Sharing, First
European Conference on Technology Enhanced Learning, EC-TEL 2006, Nejdl, W.,
Tochtermann, K., (eds.), Lecture Notes in Computer Science, 4227, Springer, (2006) 170-
184
4. Koshmann, T., Toward a Dialogic Theory of Learning: Bahtin’s Contribution to
Understanding Learning in Settings of Collaboration, in C.Hoadley and J. Roschelle (eds.),
Proceedings of the Computer Support for Collaborative Learning 1999 Conference,
Stanford, Laurence Erlbaum Associates, (1999).
5. Schegloff, E., Discourse As An Interactional Achievement: Some Uses Of 'Uh huh' And
Other Things That Come Between Sentences, in Tannen, D. (ed.), Georgetown University
Roundtable on Languages and Linguistics 1981; Analyzing Discourse: Text and Talk,
Georgetown University Press, Washington D.C. (1981)
6. Sfard, A., On reform movement and the limits of mathematical discourse, Mathematical
Thinking and Learning, 2(3), (2000) 157-189
7. Stahl, G., Group Cognition: Computer Support for Building Collaborative Knowledge,
MIT Press, (2006)
8. Tannen, D., Talking Voices: Repetition, Dialogue, and Imagery in Conversational
Discourse, Cambridge University Press, (1989)
9. Trausan-Matu, S., Chiru, C., Bogdan, R., Identificarea actelor de vorbire în dialogurile
purtate pe chat, in Stefan Trausan-Matu, Costin Pribeanu (Eds.), Interactiune Om-
Calculator 2004, Editura Printech, Bucuresti, (2004) 206-214.
10. Trauşan-Matu, S., Stahl, G., Sarmiento, J., Polyphonic Support for Collaborative Learning,
in Y.A. Dimitriadis et al. (Eds.): CRIWG 2006, Lecture Notes in Computer Science 4154,
Springer, (2006) 132 – 139
11. Voloshinov, Marxism and the Philosophy of Language, New York Seminar Press, (1973)
12. Vygotsky, L., Mind in society, Cambridge, MA: Harvard University Press, (1978)
13. Wertsch, J.V., Voices of the Mind, Harvard University Press, (1991)
14. Winograd, T., Flores, F., Understanding Computers and Cognition, Norwood, N.J.: Ablex,
(1986)
HECACEJ: B-Learning Tool for Static Content
Creation in Joomla!1
Angel Mora-Bonilla, Domingo López-Rodríguez,

Enrique Mérida-Casermeiro, Salvador Merino-Córdoba
Department of Applied Mathematics,

University of Málaga, Málaga, Spain
{amora,dlopez, merida}@ctima.uma.es,smerino@uma.es
Abstract. In this work we have reviewed the concepts of Learning

Managementystem (LMS), Learning Content Management System (LCMS) and
Content Management System (CMS). Their main features are studied, and a
comparison among them is presented, along with a summary of some well-
known integrations between LMS and CMS. Another important point in this
work is presenting HECACEJ, a B-learning tool designed to create static
content in Joomla! This tool is a specific content manager for publishing the
material for a subject, which does not need help from the administration menu
of Joomla! (difficult to use by a non-expert professor) and is able to publish
these contents in Joomla! main menu. This tool eases the publication of B-
learning contents (class notes, past exams, professors' data, etc.) in a static web
page that can be accessed from the main menu of the platform Joomla!, which
provides facilities for manipulating the contents but it was not developed ad-
hoc for our proposal.
Keywords: B-learning, LMS, CMS, Joomla!, integration of tools
1 Introduction
The group FERMAT is formed by professors of Mathematics who wanted to join all
the subjects relative to Mathematics Applied to Telecommunications in a single point.
Since 2002, students in Telecommunications Engineering School have a meeting
point in Fermat website (www.fermat.uma.es). Every subject is studied and worked
within a B-learning framework.
In a first version, FERMAT website [1] was a web page 2 maintained by several
professors spending a lot of work and time. It became necessary to use a CMS tool in
order to optimize resources maintenance and to take advantage of new learning tools.
1
This research was partially supported by Spanish PIE04/030 and UMA N 008 DGI projects.
2
Old FERMAT web (http://campusvirtual.uma.es/fermat/) was translated to six languages and
it has been recognized as the second winner of the award for the III Inter-university
competition in research and teaching in the web (http://www.campusred.net/certamen).
228 Angel Mora-Bonilla et al.
It is very well-known that e-learning presents big advantages on classical distance

teaching, since besides allowing flexibility of timetable, it allows the students to
interact and to instantaneously access to on-line resources and multimedia: text,
graphics, videos, audio and animations, among others. We propose a blended learning
model [2] for our subjects: normal classes are mixed with contents that we upload to
the FERMAT web.
The professor, as an element to favour learning, should design a succession of
motivational environments in which student's activity is developed in order to obtain
the pursued purposes. When we are in the B-learning model, the good design and
variety of these environments is much more important than in the classical model of
teaching.
To design the learning environments in e-learning, the professor has diverse
software tools. Among the most used ones we have:
• Learning Management System (LMS): also known as Platform or Virtual
Campus, it consists on programs that allow to automate the administration of
formative actions, it registers users, organizes a catalogue of courses,
manipulates user's data, carries out reports, allows to provide and to modify
contents, controls and evaluates student's participation, etc. It also offers tools
for the interaction professor-student as well as professor-professor and
student-student (mail, forum, chat, video-conference, ...).
• Content Management System (CMS): This type of programs [3] aims at
simplifying the creation and administration of contents. Basically this is
achieved by separating the creation of contents and their presentation. The
program takes charge of locating titles, text, graphics, links to the material, etc.
This type of programs has a wide incidence in periodic publications.
• Learning Content Management Systems (LCMS): They have their origin in
LMS, but they are focused to the educational environment, also allowing the
creation and management of content. This system type allows to create, to
publish, to store and to administer educational resources, being their main
users the professors. It is the appropriate tool to design learning situations in a
B-learning environment.
Thus, it can be deduced that in an environment of e-learning, we should have a tool

of the type LCMS, or a LMS and a CMS that should also be integrated, so that the
designs and contents, created by the latter, can be presented by means of the former.
In this paper, we are interested in facilitating the design and management of B-
learning tools in an university environment, so, besides the integration necessity, we
should facilitate their use for professors that are not expert in computer languages,
which not all content managers allow. For this reason, free content managers,
allowing this adaptation, are needed.
2 Learning Content Management System
In this section we intend to describe and compare three types of management systems:
LMS, CMS and LCMS.
2.1 Learning Management System
We have already indicated how this type of programs automates the administration of
formative actions (users' registration, creation of a catalogue of courses, management
of user's data, etc), gives the professor methods to distribute contents and to observe
the participation of the students, as well as it allows the students to access to
interaction mechanisms (forums, chats, mail, etc).
A LMS is an application residing in a web server in which formative activities are
developed. All the LMSs consist of a `learning environment' to which students and
professors access and of an àdministration environment', where usually only
professors and coordinators will access and where one will be able to enable courses,
to register the students, to import contents, to enable or to disable services, to obtain
statistics of the course, etc. However, a LMS will not include authoring possibilities
(creation of contents). In [4] the author remarks the key of the increasing interest in
LMS platforms: `The core purpose of the LMS system proposed in this paper is to
help learners pursue proactive and self-oriented education by allowing learners to pro-
actively configure their own content'.
Free LMSs and commercial LMSs exist. The most used ones are the free ones
since this not implies that they have an inferior quality. In addition, they are more
complete than the commercial ones. Nevertheless, some institutions prefer
commercial ones since they are configured and supervised by the company. Among
free LMSs we distinguish the most used ones: Claroline [5], Dokeos [6], Moodle [7].
There exist many other free platforms as: Bazaar, Docebo, Eledge, Ilias, DattLNR,
Ganesha, OpenUSS, Sakai, Spaghetti, ..., while among commercial LMSs we can find:
WebCT [8] and Blackboard [9]. In general, all of them perform their functions
appropriately, highlighting the platform Moodle due to its quick and extensive
installation in the educational community, and it is the chosen by the University of
Málaga for the installation of the Virtual Campus. Also, Moodle allows to compile
certain mathematical expressions written in Latex, what makes it very appropriate for
technical subjects.
In [10] the author remarks the process involved in the development of this learning
management system. The author discusses the process of building a LMS evolving
from the first experience in a subject with some basic learning material. The tools that
a LMS system provides to the professors, normally is enough to translate the material
to a LMS and to immediately develop new learning material for the students. In our
experience, we move to Moodle the subjects and develop a lot of learning
questionnaires, lessons, forums, cites, glossaries with hundreds of definitions added
by the students, wikis, tutorials, books, etc. In [11] the authors explain all these
learning tools developed in their subject.
2.2 Learning Content Management System
LCMSs [12] are software applications whose objective is to manage educational

content. In general, they combine the courses management capacities of a LMS with
creation and storage capacities of CMS. This way, LCMSs constitute an unique
platform that allows students to personalize the resources and contents.
Contents can be re-used and published in several formats, being stored in a

centralized database which can be accessed in many different forms. Data can be
visualized in different ways, what allows to adapt them to diverse groups of students.
In general, contents use XML tags and follow several standards like AICC and
SCORM, allowing to export designed materials.
A good example of LCMS is Atutor that is easy to use. The demo version can be
downloaded from [13].
2.3 Differences Between LMS and LCMS
The fundamental difference between both of them [14] is the objective that they
pursue. While LMSs focus their attention on the administrative management and the
student's performance (manages students' data and qualifications, programs events),
LCMS platforms focus in contents management, allowing their adaptability and
controlling the browsing among them. In the first case, the student tends to feel like
observed and controlled, while in the second he/she feels that the important thing is
the process of learning.
However, Brooks et al. [15] remarks that `Most popular learning content
management systems provide poor collaboration support for learners'.
2.4 Content Management Systems
CMSs [3] are a group of tools, supported by a database located in a web server and,
optionally, one series of client programs that allow the access to those tools. These
tools allow to manage in an uniform, accessible and comfortable way a dynamic
website with periodic upgrades and on which several people can act, each one with a
certain purpose.
In [16] Sehring and Schmidt summarize the use of CMS: Ìnnovative information
systems such as content management systems and information brokers are designed to
organize a complex mixture of media content -- texts, images, maps, videos, etc'.
From the client's point of view, it is a dynamic website, with an uniform
appearance, with design centered in the user and which, therefore, allows to easily
carry out the tasks it has been designed for.
In summary, a CMS has two main functions:
• To facilitate contents creation: To create content becomes as easy as to fill in a
form. There exists a single source for all them.
• To manage the presentation of those contents: It facilitates the publication of
contents in several formats starting from an unique source, adds meta-data to
contents in order to facilitate the browsing among them in multiple facets
(timeline, categories, author, ...). It also manages the contents and their
maintenance (creation, presentation, upgrade ...).
Their use is relatively recent and we can say that, at user level, they have extended
since 1999.
There exist CMSs of different types, according to the contents to manage: Forums,
Blogs, Wikis, Websites or Portal type systems. Those managing websites are the most
interesting due to their modularity. We could also classify CMSs according to the
license type: in some cases the source code is protected and it cannot be modified,
while in other cases free access is allowed.
Using a CMS, users can elaborate and manage their webs obtaining dynamic pages
with plenty of functionality. Thus, a CMS is useful due to:
• Inclusion of new functionalities in the WEB. With a CMS, new modules can
be included without supposing many changes. So the web is able to grow and
to adapt to future necessities.
• Maintenance of a great number of pages. Mainly, if they are maintained by
several people with access permissions to different areas.
• Re-use of objects and components.
• Interactive pages. The static pages arrive to the user just as they are stored.
However, the dynamic ones are generated according to the user's petitions.
The CMS achieves this by connecting with a central database where all the
data of the website is stored.
• Visual changes of the WEB. CMSs facilitate style changes, by using the
existing standard CSS to design the style of presentation of the contents.
• Consistency of the WEB. The WEB presents an uniform aspect according to
the style selected.
• Control of accesses to the WEB. A CMS allows to manage the access to
different areas of the WEB to diverse groups of users.
In general, open code CMSs are more appropriate since they allow to adapt and to
modify themselves to our necessities. A great quantity of free CMSs exist (Action
Applications, Apache Lenya, ASP Nuke, DoorSoft, ...) but we would like to highlight
Mambo, Joomla! and Xoops, which are the most extended.
We have chosen Joomla! [17] due to its great diffusion, good integration with
Moodle and its ònline' management, that is, via Internet and by means of the HTTP
protocol. Joomla! works in Linux platforms as well as under Windows. The user does
not need to know the HTML language or to access a WebMaster to upgrade
something. Joomla! manages contents by means of the relational database MySQL.
In [18] the authors pose some problems of the use of CMS: `However, learners'
experiences of these systems are almost invariably static, with information being
delivered regardless of their background or knowledge. Due to variation between
learners, it is suggested that these web-based distance-learning systems would benefit
from the capability of adapting their content to meet individual needs'.
In the same sense, the use of Joomla! by a FERMAT professor, even with
computer science knowledge at user level, is complicated and justifies the
development of a specific application that facilitates the use of Joomla! in our
framework. Actually, FERMAT joins ten subjects and was developed using Joomla!
but the process of maintenance of the information contained in each subject was
excessively increasing. Joomla! provides facilities for manipulating the contents but it
was not developed ad-hoc for our proposal.
2.5 Integrations of CMSs with the Platform Moodle
Once Moodle is chosen as the LMS to use, we should elect a CMS that can be
integrated with it. The integration of a CMS with Moodle implies that Moodle can be
accessed from the CMS (without necessity of identifying us if we have already made
in CMS, and if we have an account in Moodle).
The official page of Moodle [7] informs us that Moodle integrates officially with
Mambo, Xoops and many other CMSs. Also, Joomla! work-team has carried out the
integration Moodle-Joomla! which can be downloaded from [19].
(b)
(a)
Fig. 1. (a) HECACEJ integrated in the main menu of Joomla!, (b) Past exams table created by
HECACEJ, an example of static content created by HECACEJ and associated to one of the
subjects which were also created by HECACEJ.
3 Our Context
The group FERMAT3 is formed by several professors of the Department of Applied

Mathematics in the University of Málaga in Spain. These professors formed the group
with the aim of making some resources of Applied and Theoretical Mathematics
available to the students of this University.
Thus, in order to achieve better results, a web-page (www.fermat.uma.es) was
created and all available resources were included in it. This website was developed
with the help of the CMS Joomla!
The contents of this page are all necessary resources to study several subjects
(Calculus, Algebra...) in the Engineering School of Informatics in the University of
Málaga: professors' data, tutorships, exams, class notes... for each of the subjects.
3
‘Foro Experimental de Recursos de Matemáticas Aplicadas a las Telecomunicaciones’, in
English: Experimental Forum of Resources of Mathematics Applied to Telecommunications.
Information is shown in different web pages, which are called `Static Content' by
Joomla!, meaning that they do not require constant updates. The information shown in
those pages is conceived to last for a long time.
The current problem, which has motivated the development of the tool presented in
this work, is the difficulty that carries the creation of a subject (page with the program
of the subject, tutorships and listings of the professors, etc.) in Joomla!, which, as
mentioned before, is the educational platform used by FERMAT to maintain the
communication with the students.
Every time a professor wants to create a subject page, he/she had to enter into the
complicated (for a non-expert user) administration menu of Joomla! and insert `by
hand', using a Web editor, every content being a part of the subject, including designs,
that is, colors, font shapes and sizes, tables which are used to represent the contents.
Obviously, it is not useful to create a page design every time for each of the
subjects, when, actually, this design is going to be always the same, or very similar.
But, perhaps, the most time-wasting process is the creation of the static contents,
since the professor has to repeat some listings of the corresponding professors' data,
create links associated to the documents uploaded to the server, build tutorship tables...
for every subject.
These drawbacks motivated the development of a Contents Meta-Manager, able to
collect all the information only once, save it in the database and show it in a Web
page every time that it is necessary.
Fig. 2. Main menu of HECACEJ: Subjects, Professors and Passwords.
4 HECACEJ: A Meta-Manager for Static Content Creation
As commented before, the platform Joomla! has a drawback for our proposal: its
complicated administration menu for a non-expert user. The professors must browse
the administration menus of Joomla! to introduce a news for the subject or search a
previous news related a subject, etc. For instance, if a professor must change the
timetable of the subject it is necessary to search in a lot of news stored for all the
subjects.
We wanted to create a tool that could substitute the administration menu of
Joomla!, and that could automatically create (with a more friendly interface) the
different subjects, that is, the contents of each subject. HECACEJ4 is a tool with an
educational purpose, associated to the platform Joomla!, used to create static web
contents.
This tool is a Contents Meta-Manager, which means that is associated to a main
CMS and operates àbove' it. In our case, the main Contents Manager is the
administration menu of Joomla!. In other words, HECACEJ is a tool that manages the
contents of the platform Joomla! directly, without needing to use the administration
menu, since it works in a higher level. The integration between the platform and
HECACEJ can be observed in Fig.1.
HECACEJ has a very intuitive and easy-to-use interface (see Fig. 2.) which, step
by step, leads us by several menus to collect all necessary information about the
subject we are creating, and finally updates automatically Joomla! internal data base
with all contents and therefore updates the external view of these data, making it
public in the web page of FERMAT (see Fig. 1. (a)).
By entering in the subjects menu, we will be able to create a new subject or to
choose one of those already existent, see Fig. 4. By selecting one of them, we will
have a blackboard where to locate important news related with the subject (evaluation
method, suspension of classes, schedule modification, practical classes, etc.). After it,
we can give the description of all the aspects of the subject with an uniform format:
program and class notes, professors, exams, presentations, available questionnaires,
problems, practical classes, suggested bibliography (see Figure 5).
Fig. 3. Tutorships table created by HECACEJ.
Fig. 4. Subjects menu used by HECACEJ to create new or modify existing subjects.
4
HECACEJ - `Herramienta Educativa para la Creación Automática de Contenidos Estáticos
en Joomla', which is the translation in spanish of B-learning tool for automatic static content
creation in Joomla!. A full demo of HECACEJ is available at www.satd.uma.es/amora/hecacej
In Fig. 1. (b), we can observe an example of static content created by this tool, a
table representing a repository of past exams. Each cell of the table is a link to the
corresponding exam. If the professor has released the solved exam, the student will be
able to open it with one click.
(a) (b)
Fig. 5. (a) Dialog where to introduce the program of the subject and possible class notes, (b)
dialog to modify or create questionnaires.
But not only subjects can be created. In addition, this tool has a menu to collect
and store professors' information, see Fig. 6. Every authorized professor, can insert
his/her own data (including tutorships...), which will be stored and later published in
every subject page he/she belongs to, as shown in Fig. 3.
The integration of HECACEJ in Joomla! is rather easy. By using the
administration menu of Joomla!, a link to the tool main page has to be created. Once
this is done, we return to Joomla! main page and will find the link to the tool. Just
clicking on it takes us to the main window of HECACEJ, Fig. 2.
Fig. 6. Professor menu of HECACEJ.

5 Conclusions
In this work, the concepts of Learning Management System (LMS), Learning Content
Management System (LCMS) and Content Management System (CMS) have been
reviewed.
The main features of each one of these systems are studied, and a comparison
among them is presented, along with a summary of some well-known integrations
between LMS and CMS (e. g., Moodle - Joomla!).
One important contribution of this work is to present HECACEJ, a B-learning tool
designed to create static content in Joomla!. This tool provides a way to manage
contents without the help of the administration menu of Joomla!, and is able to
publish these contents in this main menu. Thus, this tool eases the publication of B-
learning contents (class notes, past exams, professors' data...) in a static web page that
can be accessed from the main menu of the platform Joomla!
References
1. Ángel Mora, Mérida, E., et al.: Fermat: E–learning of mathematics. Current developments in
technology assisted education - m-ICTE 2006 - http://www.formatex.org/micte2006sp/
(2006) 1263 – 1267
2. Martin Oliver, K.T.: Can blended learning be redeemed? E–Learning 2 (2005) 17 – 26
3. William H. Dutton, P.H., Park, N.: Social and institutional factors shaping e-learning in
higher education: case study of a university-wide course management system. (European
Conference on E Learning).
4. Kim, S.: Web based learning application for lms. LNCS 4113 (2006) 1164 – 1169
5. Claroline. (url: http://www.claroline.net/)
6. Dokeos. (url: http://www.dokeos.net/)
7. Moodle. (url: http://www.moodle.org/)
8. WebCT. (url: http://www.webct.com/)
9. Blackboard. (url: http://blackboard.com/)
10. Richardson, J.: Building an effective LMS. LNCS 2783 (2003) 27 – 36.
11. Ángel Mora, Mérida, E., López, D.: Development of a virtual learning community for the
subject numerical methods under Moodle. Current developments in technology assisted
education- m-ICTE 2006 - http://www.formatex.org/micte2006sp/ (2006) 361– 365
12. Duc, P.T., Haddawy, P.: A modular approach to e–learning content creation and
maintenance. LNCS 3143 (2004) 217 – 224
13. ATutor. (url: http://www.atutor.ca/atutor/demo.php/)
14. Greenberg, L.: LMS and LCMS: What’s the difference? (Learning Circuits.
http://www.learningcircuits.org/2002/dec2002/greenberg.htm)
15. Christopher Brooks, R.P., Greer, J.: Awareness and collaboration in the ihelp courses
content management system. LNCS 4227 (2006) 34 – 44
16. Sehring, H.W., Schmidt, J.W.: Beyond databases: An asset language for conceptual content
management. LNCS 3255 (2004) 99 – 112
17. Joomla! (url: http://www.joomla.org/)
18. Zakaria, M.R., Moore, A., Ashman, H., Stewart, C., Brailsford, T.: The hybrid model for
adaptive educational hypermedia. LNCS 2347 (2002) 580 – 585
19. Integration Moodle-Joomla!
(url: http://forge.joomla.org/sf/frs/do/viewSummary/projects.joomoodle/frs).
MAAS – Mobile Administrative and Assessment System
Apple W P Fok, Horace H S Ip, Chilli C K Chan

Centre for Innovative Applications of Internet and Multimedia Technologies (AIMtech)
Email: {applefok@cs.cityu.edu.hk, cship@cityu.edu.hk, ckcchan@cs.cityu.edu.hk}
Abstract. Researches focus on studies of using mobile technologies to

strengthen teaching and learning by offering free, boundless access. Less
attention has been paid to non or less educational activities which may also
affect the teaching and learning effectiveness. Consider key technical,
managerial and organizational issues of the deployment of mobile devices for
school administration and assessments, in this paper we will introduce the
design and development of a Mobile Administrative and Assessment System
(MAAS) to facilitate school teachers performing routine administrative tasks
and collecting valuable formative assessments records.
Keywords: Mobile System, Formative Assessments, School Administration
1. Introduction
Understanding students’ learning behaviors and needs is the most important stage
before we decide what and how to select and organize instructional materials. To
collect and analysis students’ behaviors and performances continuous to bother
teachers not only simply because of the assessments process, but more importantly,
some of the behaviors and performances can hardly be collected and recorded in a
systematic and effective way.
Mobile technologies can make a significant impact in teaching and learning [1,
2]. High and dynamic interactions between the learners, teachers and educational
resources in an indoor or outdoor environment are achieved and benefited from the
development of short-range wireless technologies. Mobility has provided an emerge
capability to learning technology. A great demand for an ‘anytime’ and ‘anywhere’
learning environment by combining mobility, is resulted by information explosion in
the 21st century. Wireless technologies provide the interactive communication
between students and/or teachers, access to highly informative contents from internet,
look for knowledge with keyword, and take part in an activity anytime and anywhere.
A mobile learning environment can be constructed by wireless equipments such as
PDA, that may improve performance over that come from traditional learning. New
learning activities or models for active learning can be also created under these
wireless equipments in mobile learning environment [3]. As children view mobile
devices as another version of their favourite Gameboytm or Nintendotm, educators
believed that mobile learning helped to promote the participation of students in
238 Apple W P Fok, Horace H S Ip, Chilli C K Chan
learning. They believed that mobile learning could help to change the attitude and
behavior of the students, and could provide them with a greater motivation for further
learning. [7] even believed that mobile technology helped family learning.
Mobile devices are personal, accessible, flexible and allow for collaborative
solutions. They can help to realise collaboration. They allow sharing, exchanging,
organizing and managing information easily. Everybody does not need to be together
or fix in one place can communicate with each other and receive and response quickly.
This indicated that mobile technology could provide the opportunities for
collaborative learning and improve communication between learners and teachers.
Collaboration between teachers and non-teaching staff in school environment where
educational events occur in close relations to less educational events is described as a
team working process where one exchanges information, expands on them, changes,
modifies or discards them together. Schools can make use of this mobile environment
not only improving the learning but also benefiting to other operations at schools,
such as administrative work and assessments. Taken the advantages of using handy
size-wise mobile device like PDAs in this study, teachers can perform not only daily
routine school administrative works effectively and also conduct formative
assessments and record keeping and analysis efficiently.
Consider key technical, managerial and organizational issues of the deployment

of mobile devices for school administration and assessments, in this paper we will
introduce the design and development of a Mobile Administrative and Assessment
System (MAAS) to facilitate school teachers performing routine administrative tasks
and collecting valuable formative assessments records.
2. MAAS System Architecture
Mobile Administrative and Assessment System (MAAS) is an information

management system for student assessments, attendance and discipline records.
MAAS consists of two components, a front-end PDA application and a backend
database management system. Figure 1 illustrated the MAAS system architecture. On
the server side, MAAS provides web services including user management, student
management, assessment management, assessment result, attendance checking, assign
and check homework and student discipline records. Front-end PDA application is
divided into two parts. The core part of the application runs on PDA devices and the
web service interface is held on a server. PDA devices connect to the server through
WiFi wireless network and exchange data with database by the web service interface
created on MS IIS (Microsoft Internet Information Server). Attendance lists,
assignment checklists, assessments rubrics and administrative records can be
uploaded/downloaded through wireless connection to/from the server.
Fig. 1. MAAS System Architecture
Front-end PDA application includes two programs: i) the Assessment program that
Create, save or import student grouping (specialized to contents in group assessment),
and record students’ performances; ii) the Administrative program that facilitates
checking and monitoring of students general behaviors in school include attendance
(i.e. records of student attendance in school and activities), and discipline (i.e. records
of homework submission, misbehaviors, and good behaviors). Figure 2 shows the
MAAS web services and Figure 3 shows the MAAS PDA applications. Taken the
advantages of the mobility of PDA devices, these tasks can be accomplished
effectively through wireless connection anywhere, anytime in the school. During the
assessments, all assessment data is stored in the database on PDA in file format.
Teachers can check and update the records easily.
Fig. 2. MAAS Web services

Fig. 3. MAAS Front-end services
3. MAAS Multi-facet functionalities
To facilitate the school administrative tasks through collaboration, MAAS provides

distinctive services for three distinct user groups: school system administrator, school
administrative staff and teachers. Although they share the same underlying platform
and basis functionalities, like rubric design and display, view rubric results and
analysis, check and retrieve reports and report summary, different group touches
different sets of data and have multi-level user access rights to the data and the
analysis results. To facilitate different user groups, MAAS provides different sets of
features. Figure 4 illustrated the functionalities of different user groups and the data in
relations to the access rights.
To make the administrative work more efficient for teachers and reduce their
workload from these administrative tasks, MASS provides useful features in six
categories related to the administrative work in schools. These six categories are User
Management, Student Management, Assessments and, Attendance, Discipline and
Behavior Records. Through these features provides by MAAS, teachers can perform
assessments using a small mobile device, e.g. PDA or smart-phone, in a convenient
way, with high accuracy, no location limitation and instantly. It enhances the way of
assessment from the traditional paper-based process and makes the process more
effective. Moreover MAAS analyses the statistics of assessments and, attendance,
discipline and behavior records to gives out meaningful and useful results for school
operations. Administrative staff can get a clearer picture about students to make an
accurate and appropriate school planning and change the daily school operations in an
efficient and effective way such as attendance taking and student report generating.
Fig. 4. User Management
System Admin Admin Staff Teachers

User Management
System admin, admin staff and √ √ ×
teacher information management
Student Management
Class, Student, Subject management √ √ ×
Assessments
Form Management √ × √
Form browsing, Result Reporting, √ √ √
Result Statistics
Attendance and, Discipline and
Behavior Records
Attendance records management √ × √
Homework records management √ × √
Stationary records management √ × √
Behavior records management √ × √
Statistics √ √ √
Reporting √ √ √
Fig. 5. MAAS Functionalities of different user groups and the data in relations to access rights.
User Management
MAAS includes User Management feature which is simple and effective. School
administrators and administrative staff can easily manage user accounts through this
feature. In User Management, user account information can be added, displayed
clearly, modified with well organised form or removed by a single click then confirm.
This feature releases the burden from the heavy workload of the annoying
management of user accounts. Figure 5 shows the screen layout of User Management
in MAAS.
Fig. 6. User Management in MAAS

Student Management
To support system administrators and administrative staff to manage student accounts

and related information, e.g. class and programme information, Student Management
provides efficient tools for adding, editing, and locking different user accounts. These
features reduce the stress of administrators and administrative staff for managing over
thousand student accounts in a school. The details of student accounts can be create
shown, changed and deleted by a few simple steps without any pre-requested
technique. Also MAAS can manipulate class and programme, e.g. physics or biology
stream programmes, with the same advantages. Student Management from MAAS
smooth out the complicated tasks of managing student accounts then administrators
and administrative staff can stay away from these repetitive jobs. Figure 6, 7 and 8
show Student, Class and Program Management respectively.
Fig. 7. Student Management in MAAS
Fig. 8. Class Management in MAAS

Fig. 9. Program Management in MAAS
In addition to the analysis results in form, class, MAAS provides the analysis
result in student level. Senior admin staff can obtain statistical overviews in form and
class levels. It helps them to set the goal of school year plan, make the direction of
school development and design the school year plan. On the other hand, MAAS
enhances the work and reduces the workload for general admin staff. The analysis
results in student level from MAAS gives the general admin staff a hand with a
complex and heavy workload task, student reporting. On the other hand, strengthen
the communication with teachers, admin staff can inform and update teachers ad hoc
or latest information received when accidental events happened. Updated records and
alerts will be sent to teachers through MAAS.
Through three levels quantitative analysis, the class level, student level, and
strand level, MAAS provides student performance analysis for teachers. Student
performance status can be obtained from different sources, including the assessment
scores, participation in activities, and even students’ general behaviors. It uses data
mining techniques off-line to cluster and understand students’ behaviors so as to
predicate the trends of each student. To develop diverse assessment types and formats
to better align with the curriculum, theories and practice of learning and teaching,
teachers can design and share different types of assessment rubrics as formative
assessment tools to collect and evaluate students’ learning progress.
A teacher is allowed to check and update the information and view the analysis of
student(s) in his/her class, and also the summary records of other classes. Summative
Assessments and Comparison with other classes or a grade level can give teachers
clearer observations on their students, so as to adjust their teaching plans and
strategies accordingly.
To create a pleasant and collaborative working environment that strengthens the

communication between teachers and school administrative staff, MAAS not only
supports the recording services such as assessment records, attendance records,
homework submission records and, discipline and behaviors records, but more
significantly, provides alerts or notifications of records updated, continuous
misbehaviors of one particular students and accidental events. While teachers are
attending a class, s/he will also receive relevant information as if s/he needs. For
instance, a class teacher needs to be informed if one of his or her class student is
required an early leave because of instant sickness.
4. MAAS Supports/Responses to Different Types of Assessments
In order to enhance the effectiveness of schools, the Education and Manpower Bureau
(EMB) of Hong Kong introduced the School Management Initiative Scheme in 1991
[4]. This scheme offered a school-based management framework for improving
school management and performance for the provision of quality school education.
School-based management involves decentralization of decision-making from the
Education and Manpower Bureau to schools. The devolution of responsibilities
enables schools to streamline their administrative procedures for their goals and end-
vision. To support efficient retrieval of students’ data for internal as well as external
review Apart from the academic performance, another key aspect of student learning
emphasises student’s proper behaviours namely “student discipline”. Schools have to
be strictly observed in handling students with discipline problems. Student discipline
has to be concerned by all teachers at schools. Teachers have to play a role in
reinforcing positive behaviour (e.g. politeness, punctual and responsible etc) and
handling misbehaviour (e.g. running, smoking, fighting, foul languages etc) of
students [5].
Outcome-based approaches continuously dominant the world wide education
system and stress the importance of assessments. Empirical studies stated that
“assessment for learning” is one of the principles of facilitating learning and teaching.
It is an integral part of the learning/teaching cycle. To improve the student’s learning,
a diversity of assessment modes has to be used [6]. Formative assessments should be
used to provide effective formal or informal feedback (e.g. corrective, diagnostic) that
motivates and improves learning. In-class-behaviours Assessment accesses qualitative
work based on the purpose and process of learning, not academic outcomes. For
instance, observation may be used to assess participation in discussion, performance
in laboratory session, and project work for interdisciplinary thinking [4]. And school-
based assessments will address the diversified needs of students and evaluate
students’ competency levels from multiple perspectives such as arithmetic, linguistic,
and musical etc.
Through Assessment Form Management in MAAS, school administrators and

teachers can add, browse and delete assessment forms and subject information
required for assessments. Four rubrics design templates support the assessments
record in Grading or exact Marking with one or two levels descriptions (Figure 9).
MAAS provides the ability to school administrators and teachers for managing this
information in a single and simple menu. Assessment form details can be reviewed by
school administrators, administrative staff and teachers through MAAS. MAAS
collects the assessment results of students to generate the statistics. This statistics
given by MAAS provides administrative staff supportive evidences for planning the
school development. On the other hand, MAAS not only gives the statistical
assessment results of students but also the student performance ranking from the
assessment results. This information from MAAS helps teachers to have a clear
overview to student performance and let them design their teaching plans according to
different abilities of classes or even individual students. These teaching plans are
customized for corresponding classes or students; enhance the teaching process and
benefit students most. Nevertheless MAAS generates reports for teachers and
administrative staff from the assessment results to let them have knowledge about
student performance and report generation is also a heavy load and repetitive task.
With MAAS, teachers and administrative staff can reduce their workload from this
task. Figure 9 and 10 show the rubric templates and an example as an assessment
rubric. The results of these rubrics will be recorded and analysis as Class, Student
Assessment Statistics and Student Performance Ranking respectively (Figure 11).
Fig. 10. Four Rubric Assessment Templates

Fig. 11. Example of an Assessment Rubric
Class Assessment Statistics

Student Assessment Statistics
Student Performance Ranking

Fig. 12. Class, Student Assessment Statistics and Student Performance Ranking
To make the grading process more efficient, grouping feature is designed. Teachers
can form and adjust different groups according to either performances or discipline
reasons. Figure 12 shows the grouping features.
Fig. 13. Grouping Features of MAAS
Behavior Records and Analysis
To observe and understand students’ daily non-academic performances, several

behavioral records are used including the Attendance Records, Homework
Submission Records, and Discipline Records.
MAAS lets teachers perform student attendance taking anytime they need
through mobile devices like PDAs. Collaboration among teachers to student
attendance taking can be achieved by MAAS. For late students, teachers can instantly
update their attendance record during the lesson through MAAS. Therefore MAAS
provides teachers an up-to-dated record of student attendance and helps teachers and
administrative staff to monitor students effectively. Also MAAS records student
attendance records and generates attendance statistics from these records. These
records from MAAS can give teachers a summary on the attendance of students and
have a facility to find out students with attendance problems. Then by the summary
reporting function in MAAS, corresponding arrangements can be quickly taken for
students with attendance problems. The student attendance statistics reports in class
level included in MAAS can helps administrative staff to adjust the school operations
and change the design of school plan according to this supporting statistical data. It
makes the administration in schools more efficient and effective. Figure 13 shows
Student Attendance Record and Student Attendance Summary respectively.
In behavior record feature in MAAS, homework submission of students can be

recorded. Homework submission record is a daily time-consuming task. MAAS can
simplify and enhance the efficiency of this task by that teachers assign homework
through MAAS and record the submission of students with PDAs anytime and
anywhere. Then the student submission information is uploaded to MAAS. Accurate
and up-to-dated homework submission records can be kept by the anytime and
anywhere update from MAAS. In the global view of homework submission,
administrative staff can use this statistics for analysis and find out student problems.
This useful statistics then helps the design an appropriate strategy to tackle these
problems in schools. Figure 14 shows the features that support the assign homework
cycle including Homework assigned, Homework Submission Record and Homework
Submission Statistics.
Student Discipline Control is an important task in School Management. Teachers

are always under stress from student discipline problems. MAAS provides teachers a
convenience method to record student discipline records through PDAs. These
records are instantly updated to MAAS and the system often keeps the most up-to-
dated records. The mobile nature supported by MAAS creates a collaborative
environment for teachers to perform student discipline record task. The stress from
student discipline can be shared among teachers through MAAS. Individual student
summary record in MAAS can point out students with discipline problems to teachers
and alert teachers to take appropriate actions. Also MAAS gets student discipline
records and produces a summary of student discipline records. It is very useful
information to administrative staff for the modification to the current school planning
and operations. According to MAAS student discipline statistics, customized school
plan and operations can enhance the school development and benefit to schools
significantly. Figure 15 and show Student Discipline Record, Student Discipline
Statistics in MAAS respectively.
Fig. 14. Student Attendance Record and Student Attendance Summary

Fig. 15. Features support the assign Homework cycle

5. Evaluation and Discussion
Taking the advantages of advanced technologies that provide any where, any
time access through the web, incorporate a portable mobile device (i.e. PDA), the
MAAS acts as i) a convenience data collector ; ii) an effective data transmitter; and iii)
an interactive communication channel. To strengthen the proof of the advantages from
MAAS, an implementation study has been conducted in one Hong Kong Secondary
School in January 2007. Using two different sets of rubrics in English and Chinese
languages for the subjects Physics and Chinese Culture, teachers are required not only
to try out the system functionalities, but also compare the differences and evaluate the
effectiveness of using such a device. All system features are tested and used for two
different classes for two different assessments, science laboratory test and Chinese
Oral Presentation. Both assessments taken in this evaluation have been completed
smoothly and teachers can familiarize with the operations of MAAS. By the mobile
nature of MAAS, teachers can walk around the classroom to inspect student work and
access student performance at the same time (Figure 16). All the students observable
learning behaviors are measured and records through MAAS. The time spends on
students’ learning is increased. With the support from the rubric description, the
assessments can be done more effectively and precisely. In addition, the mobile nature
of MASS, gives an opportunity for collaboration between teachers. The centralized
administrative and assessment work can be distributed to and completed by teachers
at any parts in schools instantly. Therefore teachers can share these administrative and
assessment jobs. The workload of teachers can be reduced.
MAAS faces the same problem that a limited screen size on PDA
limits/constrains mobile PDA applications nowadays. It is really challenging to design
a rich yet concise interface. The selection of essential elements to be shown on the
limited screen that can not only provide sufficient information, but also serve the
functionalities that enhance the effectiveness of accomplishing a task required
thoughtful plan and sacrifice. To minimize the navigation path, the general operation
flow would be changed. For instance, a teacher used a class list for attendance
checking. Names of students are shown on one page. However, on the PDA, not only
affected by the font size, but also the check-box or text box for information input, it is
not pleasant to read all the information at once. Alternatively, default values such as
“Present” can be set beforehand and teachers can simply change the stage to absent or
an “Absent” page can be designed for simply class number inputs.
Fig. 16. Student Discipline Record and Student Discipline Statistics in MAAS
Most mobile devices are difficult to handle with, especially during the inputting
of characters. Thus in MAAS, we overcome this problem by serving predefined items
that users can simply click onto interact with the system, and utilizing some skills
such as enlarging buttons when they are about to be clicked. On the other hand, to
highlight and discriminate the usage of rubrics and different rubrics items, special
effects are required. The marked/graded rubrics will be highlighted and a summary of
the rubric usage will also be provided.
Fig. 17. MAAS in classes (Physic and Chinese)
6. Conclusion and Future Work
In this paper, we have introduced the design and development of a Mobile

Administrative and Assessments System (MAAS) that helps to reduce the workload
of teachers from the administrative and assessment. Furthermore, MASS helps create
a collaboration environment for teachers to complete school administrative and
assessment work successfully. Recently, Mobile Administrative and Assessment
System (MAAS) has been setup at two Hong Kong schools, one Secondary School
and one Primary School (Fig. 17). To fully optimize the use of MAAS, analysis
reports and report summaries will be designed. To further evaluate MAAS
performance and effectiveness, different use cases studies including different
assessment rubric designs that support multiple subjects and operations in class, and
enhance the readabilities and extensibilities of different components of the system
will be conducted. In response to the current mobile devices trend, the interface
design would be adjusted to 320 x 240 that requires not only item representation
changes, but also the navigation flow should be adjusted and changed. This in turns
may affect users’ usage pattern and styles. Apart from the evaluation of the system
performances and features provided, we also emphasis the applicable use of mobile
devices in school environment. What and how mobile devices can be use to facilitate
and enhance teaching and learning effectiveness? Pedagogical uses of mobile devices
to strengthen e-education are our on-going research work.
References
1. Perry D. (2003) Handheld Computers (PDAs) in School. Coventry, U.K.: Becta (for
DfES)
2. Zurita, G., & Nussbaum, M. (2004). A constructivist mobile learning environment
supported by a wireless handheld network. Journal of Computer Assisted Learning, 20,
pp 235-243.
3. C.Y. Chang, J.P. Sheu and T.W. Chan, “Concept and design of Ad Hoc and Mobile
classrooms,” Journal of Computer Assisted Learning (2003) 19, 336-346
4. EMB – School Administration Guide.
http://www.emb.gov.hk/index.aspx?nodeID=681&langno=1 accessed 21st January,
2007
5. Student discipline. http://www.emb.gov.hk/index.aspx?nodeID=1740&langno=1
accessed 21st January, 2007
6. Principles of facilitating learning and teaching.
http://www.emb.gov.hk/index.aspx?nodeID=1690&langno=1 accessed 21st
January, 2007
7. DfES. (2005) The e-Strategy ‘Harnessing Technology: Transforming learning and
children’s ervices’. Retrieved from http://www.dfes.gov.uk/publications/e-strategy/
Large-scale Computer-Assisted Assessment in
Computer Science Education
New Possibilities, New Questions
Mario Amelung, Michael Piotrowski, and Dietmar Rösner
Otto-von-Guericke-Universität
P.O. Box 4120, 39016 Magdeburg, Germany
{amelung,mxp,roesner}@iws.cs.uni-magdeburg.de
Abstract. Since 2003 we have successively introduced the use of e-learning

and computer-assisted assessment (CAA) components into all of our courses,
namely online multiple-choice tests, electronic submission of assignments, and
automatic testing of programs. We originally did not intend to make major
changes to the courses; our primary motivation was just to make them more
efficient and more effective by freeing teachers from administrative burdens
and by offering more flexibility and interactivity for students. After several
semesters of usage we have noticed, however, that the courses have changed
much more radically than originally envisaged. The electronic support of face-
to-face courses offers many new possibilities, but it also opens up new
questions. This paper describes our system and our experience, and discusses
some of the questions we have encountered.
1 Introduction
In computer science education, lectures are typically accompanied by exercise courses

or tutorials. These courses are essential for the learning effect since they provide op-
portunities for students to solidify the knowledge acquired in lectures and to apply it
to practical problems.
We were, however, dissatisfied with a number of aspects of the traditional way of
teaching, practicing and assessing in undergraduate computer science courses at our
university: They offered only relatively little motivation for students and allowed only
restricted conclusions about the students’ performance during the course, whereas the
administration of the courses required a lot of work. Especially unsatisfactory was the
handling of programming assignments: The traditional way of handing in programs
on paper and discussing them on the blackboard is only viable for very small
programs, and some problems are hard to detect without actually running a program
and testing it. Students thus only gained little practical programming experience. In
fact, there were serious complaints that even some of our second and third-year
students still had insufficient programming skills.
These problems prompted us to develop a number of e-learning components (de-
scribed below) to support the teaching of face-to-face courses. We did not want to
258 Mario Amelung, Michael Piotrowski, and Dietmar Rösner
abolish face-to-face courses, but rather make them more efficient and, hopefully, also
more effective.
Since winter semester 2003/2004 we have successively introduced the use of e-
learning components into all of our courses. This includes online multiple-choice
tests, electronic submission of assignments, and automatic testing of programs.
After several semesters of usage we have noticed, however, that the processes
within the exercise courses have changed much more radically than we expected, and
we strongly feel that now, with many technical aspects solved, with a mature
implementation, and after several semesters of usage, it is time to reflect on our
experiences.
The structure of this paper is as follows: Section 2 gives an overview of our e-
learning environment, briefly explaining some technical details of the automatic
program testing module to provide the necessary background for the subsequent
discussion. In section 3 we describe the structure of our exercise courses, before and
after the introduction of electronic support. Section 4 discusses the new possibilities
offered by the electronic support of face-to-face courses and the questions raised by
some of these possibilities. Section 5 summarizes the discussion.
2 The Learning Environment
Instead of using a separate learning management system (LMS), which would require
additional training and administration, our approach is based on modules that extend a
general-purpose content management system (CMS) with e-learning functionality, in
our case the open-source CMS Plone1. We designed, implemented and deployed a
number of Plone modules. These modules—collectively called eduComponents—
provide specialized content types offering the following functions (see also [1,2]):
− ECLecture: Course information and registration.

− ECQuiz: Electronic multiple-choice tests.
− ECAssignmentBox: Electronic submissions for assignments and support for
manual assessment and grading.
− ECAutoAssessmentBox: A version of ECAssignmentBox with automatic checking
and assessment of programming assignments with immediate feedback.
These components can be used separately or in concert, and they can be combined
with other Plone products (e.g., discussion boards, glossaries, and bibliographies) to
create tailor-made learning environments. We are using the eduComponents modules
to support face-to-face courses, but they can also be used in pure online courses.
ECQuiz supports the creation and delivery of multiple-choice tests (see also [3]).
Multiple-choice tests are especially useful as formative tests to quickly assess the
performance of all students of a class without the need for extra grading work.
The assessment of essay-like student submissions offered by ECAssignmentBox is
semi-automated, meaning that the teacher does the assessing but is aided by the tool
1
http://plone.org
Large-scale Computer-Assisted Assessment 259
during the entire process of grading students’ work and giving feedback. Therefore
ECAssignmentBox defines a specialized workflow for student submissions.
Because automatic testing of programming assignments is especially interesting in
computer science education, we will describe our approach in more detail: ECAuto-
AssessmentBox allows students to submit solutions for programming assignments via
the Web at any time during the submission period. Submitted programs are auto-
matically tested and students get immediate feedback on whether their programs are
syntactically correct and—more importantly—then yield the expected results.
Fig. 1. ECAutoAssessmentBox automatically checks submissions to programming exercises

and immediately offers feedback (student view).
When a student submits a program, it is first sent to ECSpooler, a Web service

which manages a submission queue and several backends. Backends provide testing
for a specific programming language, usually in conjunction with the corresponding
compiler or interpreter. Currently implemented are backends for Haskell, Scheme,
Erlang, Prolog, Python, and Java. Backends can also be used to implement different
approaches for testing: For example, we have implemented one backend for Haskell
which compares the output of the student solution with the output of a model solution
for a set of test data, and, as an alternative, we have implemented another backend for
Haskell which uses QuickCheck [4] for testing based on formal specifications of
properties required for a correct solution. With the appropriate backends, the system
can also be used to check submissions in other formal notations or to analyze natural-
language assignments (we have already experimented with style checking and
keyword spotting). The results of the tests performed by the backend are immediately
returned and are displayed by ECAutoAssessmentBox (see Fig. 1).
There are quite a number of systems which provide facilities for automatic testing
of programs (e.g., TRAKLA [5], SchemeRobo [6], CourseMarker [7], AT(x) [8], or
Praktomat [9]). Our approach differs from most other systems in its architecture,
which clearly separates frontend, spooler, and backend, offering a high degree of
flexibility and enabling a variety of frontends/backends to be used. Our system also
concentrates on testing and does intentionally not offer advanced tutoring functions.
The eduComponents modules are freely available as open-source software licensed
under the terms of the GNU Public License.2
3 Course Structure
Since winter semester 2003/2004 we have been gathering experiences with eduCom-
ponents for online multiple-choice tests, electronic submission of assignments and
automatic testing of programs in our exercise courses. During the last semester
(winter 2006/2007) our learning environment was actively used by over 200 students
at our institution.
Before we introduced CAA—and in particular the automatic testing of
programming assignments—into our courses, the typical procedure was as follows:
Once a week students were given assignments; they then worked through the
assignments and prepared their solutions for the classroom session of their exercise
group (each group comprising about 12 to 24 students) on paper. For each
assignment, the tutor called on a student to present his or her solution. The
presentation was given at the blackboard and typically involved writing the solution
onto the board. This mode of writing and copying was time-consuming and error-
prone. Given the time constraints, only a limited number of solutions could be
presented and discussed. It was also not clear whether the presented programs would
actually work.
Now, the exercise courses use the following process: Students get access to online
exercise sheets (see Fig. 2) and submit their assignments using ECAssignmentBox or
ECAutoAssessmentBox during the submission period, which typically ends several
hours before the classroom session.
Submissions for programming assignments are automatically tested. Depending on
the tests defined by the teacher for the assignment, submissions must at least be
syntactically correct; typically they must also yield the expected results or comply
with a formal specification.
2
Available from: http://wwwai.cs.uni-magdeburg.de/software/. The automatic testing
components are not yet publicly released, but are available on request.
Fig. 2. View of a typical online exercise sheet
The automatic testing of programs is not intended to replace the testing of

programs by the student with the appropriate compiler or interpreter. The motivation
is primarily didactic: Our experience shows that quite a number of students tries to
avoid writing and testing programs and would rather submit non-working sketches.
The automatic testing of programs enforces the requirement that programming
submissions must be running programs. But the student program can also be tested
with input unknown to the student, thus uncovering unhandled edge cases; depending
on the settings, the student may then submit a corrected version. For teachers,
automatic testing provides help in judging the acceptability of programs, since they
can easily see which programs run, and which do not. To prepare for the course,
teachers then review the submissions, looking for recurring problems or outstanding
solutions. Based on this information and on previous student performance they also
select the students who will have to present their solutions in class.
Students have to present their solutions as before, but during the classroom session,
the assignments and the presented solutions are displayed using a data projector. This
removes the need to copy solutions to the blackboard. If the solution and the student’s
presentation have been satisfactory, the submission is moved to the corresponding
workflow state (e.g., graded).
4 Experiences
4.1 Effects on Students and Teachers
The paper-based system allowed students to get points for assignments they had not
actually completed, whereas they now have to submit written solutions for the assign-
ments electronically before the classroom session. For programming assignments with
automatic testing the demands for students’ solutions are much more explicit and
rigid with respect to correctness, quality, and clarity: Sketchy notes and vague ideas
on a piece of paper are no longer sufficient.3 Students thus also have to try to make
sure that their solution is working correctly. Consequently the intensity of work
needed for the exercises has effectively increased with the new system.
For teachers using automatic testing of programs, the most significant effect is that
the effort for designing assignments has increased. This is something that other users
of automated program testing systems have also reported (e.g., [9]). Once there is a
collection of assignments to choose from, this problem is alleviated, but automatic
testing always requires problems and tasks to be formulated much more formally and
precisely. This is necessary to enable automatic testing and in order to avoid
misunderstandings, which could result in students trying to solve a different problem
than the one the teacher had in mind and then getting puzzled about the reactions of
the automatic testing system.
4.2 New Opportunities
The fact that all assignments and tests are now managed as electronic documents in a
CMS results in substantial improvements of the learning environment and offers
many new opportunities. Major aspects include:
− Student solutions of assignments are available online and can easily be presented,
discussed, or modified during classroom sessions.
− Alternative solutions of assignments can easily be made accessible as a portfolio
for further self-study or for exam preparation.
− New types of assignment are feasible, for example, peer reviewing of programs of
other students (cf. below).
− Personalized assignments with additional tasks are possible; these could be
designed to help with problems observed in prior assignments.
3
While it would have theoretically been possible to enforce these requirements in the paper-
based system, the workload for teachers would have been unacceptable.
Additional advantages result from the data collected, for example the number of
tries needed before a correct solution is found, or the number of submissions for an
assignment. Cumulative statistics could help to detect a variety of problems; they
could also be used to assess the quality and the difficulty of assignments.
From the teachers’ point of view, for both programming and essay-like
assignments, reviewing larger numbers of student submissions has proven to be
feasible because the submissions are collected at a central location, so that they can be
browsed and inspected before the classroom session. Specific problems observed in
the submissions can then be addressed. Since all submissions are now also available
online in the classroom session, solutions can easily be presented and compared;
faulty solutions to programming assignments can be corrected and immediately
tested. The time spent formerly to write sketchy solutions onto the blackboard is now
free for discussion.
4.3 New Questions
Not all of the new possibilities are completely novel, but they are much easier to
realize and easier to actually use with the help of the new technical capabilities.
Some of the questions are inherent in many other computer applications as well.
Having records of information (in our case: assignments and log data) raises questions
of access, privacy, and prevention of misuse.
Some other questions have more to do with pedagogical considerations. These
often involve the need for decisions about clear rules for all participants in the
learning processes. An example could be the issue of personal attendance to
classroom sessions: Some of our students felt that there was no need for personal
attendance to the classroom session after they had submitted working solutions to the
programming assignments. However, for pedagogical reasons, we insisted on
personal attendance and participation in discussions—but other decisions could be
made. Technically, using ECAutoAssessmentBox and ECAssignmentBox in a pure e-
learning context would, of course, also be possible.
New possibilities often come with new demands: For example, ECQuiz makes it
easy to set up multiple-choice tests and to get an automatic evaluation for a large
number of students. The challenge for the teacher is, however, to design tests that
require more than the recall of previously memorized knowledge and instead explore
more demanding cognitive dimensions in Bloom’s taxonomy [10], such as
application, analysis, or evaluation.
Open vs. Testable Assignments Obviously, to enable automatic testing, program-

ming assignments have to fulfill certain requirements. For example, the students’
solutions have to contain a top-level function with a certain name and a certain
signature, i.e., the order of the arguments and their types and type of the result are
specified in the assignment.
However, sometimes the task consists in finding a good representation for the
problem and its solution, so that, for example, the return type should be left open.
Interactive programs are another class of tasks that have more degrees of freedom and
that are harder to test automatically.
In our experience, teachers get quickly used to the automatic testing facilities and
their requirements and then habitually consider automatic testability when designing
programming assignments. This presents the danger that assignments are discarded if
they cannot be tested automatically. It is therefore necessary to consider the learning
objectives and to critically evaluate the appropriateness of assignments for certain
objectives.
For an assignment with an interactive Prolog program for playing tic-tac-toe we
therefore decided to not use automatic testing but to assign the submissions
anonymously to other students for critical evaluation and peer review. This ensures
that the students can be creative in designing their solutions, and students get more
actively involved; cf. [9] for a report of a successful use of peer review in computer
science education. The possibility to create this type of assignment is of course also
based on the availability of all submissions as electronic documents.
Policy Issues The new learning environment available with eduComponents not only
allows, but somehow demands decisions about policy issues. An example is the issue
of plagiarism: Since all student submissions are available electronically, it is easily
possible to automatically check for suspected plagiarism. However, this raises a
serious policy issue: How much effort should be invested in plagiarism detection?
When do we run the risk of wasting energy for an “arms race” between plagiarism
detection and more sophisticated techniques of hiding plagiarism? Is there a chance to
instead suppress plagiarism by stimulating a spirit among students that mastering
challenges is rewarding and that handing complete solution to a peer for copying is
counterproductive and not an act of solidarity?
A related policy issue is team work. On one hand, being able to cooperate is
essential for success in the job, e.g., as a project member in the software industry. On
the other hand, students must also be able to successfully work on problems on their
own, so free riding in a team should be avoided. How can this conflict be resolved? Is
it possible to support both goals? Shall individual work on assignments be enforced
(running the risk of plagiarism) or shall group submissions be allowed if declared as
such? What are the consequences? Do we rely on students to cope with attempts in
free riding on their own? Can we rely on the final exams, which have to be passed
individually, as “hour of truth”?
Control and Privacy The fact that all students submissions and log data about the
number of attempts, submission times, etc., are available and storable for a long time
as electronic documents, allows for much more control of students’ working habits,
results and overall performance. But do we really want to exercise such a type of
control? Do we really want glass box students? Or, in other words: What is more
effective in the long run, a teaching system that rewards positive outcomes and that
stimulates engagement, creativity and striving for excellence, or a teaching system
that invests much energy in exercising control and in the punishment of unwanted
behaviors? And an even more fundamental question could be: What is our general
conception of students?
The logging in ECAssignmentBox and ECAutoAssessmentBox was intended to
allow evaluation of the effects of this system for the learning process and
investigation into related questions. Astonishingly, there were hardly any concerns
articulated from our students that they would feel uncomfortable with this kind of data
collection. The only exception was the question posed by some students why a record
of the previous submissions is stored when they submit an improved solution. This
feature had been introduced to investigate the usefulness of the automatic feedback.
4.4 Unexpected and Unintended Usage
ECAssignmentBox and ECAutoAssessmentBox have been designed and implemented

as lightweight solutions. They support either direct typing of answers into a text field
and the uploading of assignments from a file, but they intentionally do not offer
sophisticated editor functionality.
Nevertheless, there were unanticipated usages of the system. Some students used it
as a kind of “online storage” to work on essay-like assignments: They started to work
on an assignment from one computer, used the submission feature to store an
intermediate version, and later continued to work on the same assignment from a
different computer. This resulted in a large number of spurious superseded
submissions.
Other students abused ECAutoAssessmentBox as a Web-based interpreter to solve
programming assignments. We therefore introduced a parameter for teachers to
restrict the number of possible re-submissions for automatically tested programming
assignments. This limitation also enforces a secondary learning objective: We expect
that our students are able to use the native programming environments and
interpreters for the various programming languages and to leverage them instead of
submitting untested programs.
5 Conclusions
At the end of each semester we ask our students to complete a questionnaire on their
experience with the learning environment. The questions cover three areas: The use of
electronic submissions in general, their effect on the students’ working habits, and the
usability of the eduComponents. The results in all three areas have consistently been
very positive. Students especially value the reporting and statistics features, which
help them to track their learning progress. Furthermore, students find it helpful that
their assignments are stored centrally, and can quickly be accessed for discussion in
the course or for later review. Students also report that they work more diligently on
their assignments because teachers can now easily access and review all of their
submissions.
A seemingly minor change in the organization and technical basis of exercises—
i.e., introducing that all assignments and all solutions of students are electronic
documents in a CMS—resulted in significant changes in the learning environment and
changed learning processes much more fundamentally than expected in the beginning
of the transition to the new system. When we started using CAA and other e-learning
components we had the primary motivation to relief teachers and students from
administrative burdens by automating certain processes and supporting others. Our
experience is, however, that the change in the way how assignments are submitted has
lead to many other changes due to the new possibilities offered by the system. But the
new opportunities also pose new demands for both teachers and students.
Although the workload for students has increased there is a broad acceptance of the
new system and students would welcome its use in other lectures as well. We interpret
this as a positive reaction on the new opportunities and as an indication that students
accept the higher intensity of their own engagement because they experience and
appreciate an improved return on investment for their learning outcomes.
References
1. Amelung, M., Piotrowski, M., Rösner, D.: EduComponents: Experiences in e-assessment in

computer science education. In: ITiCSE ’06: Proceedings of the 11th annual conference on
Innovation and technology in computer science education, New York, NY, USA, ACM
Press (2006) 88–92
2. Rösner, D., Amelung, M., Piotrowski, M.: E-Learning-Komponenten zur Intensivierung der
Übungen in der Informatik-Lehre – ein Erfahrungsbericht. In Forbrig, P., Siegel, G.,
Schneider, M., eds.: 2. GI-Fachtagung Hochschuldidaktik der Informatik. Bonn, Germany,
GI-Verlag (2006) 89–102
3. Piotrowski, M., Rösner, D.: Integration von E-Assessment und Content-Management. In
Haake, J.M., Lucke, U., Tavangarian, D., eds.: DeLFI2005: 3. Deutsche e-Learning
Fachtagung Informatik der Gesellschaft für Informatik. Bonn, Germany, GI-Verlag (2005)
129–140
4. Claessen, K., Hughes, J.: QuickCheck: a lightweight tool for random testing of Haskell
programs. In: ICFP ’00: Proceedings of the fifth ACM SIGPLAN international conference
on Functional programming, New York, NY, USA, ACM Press (2000) 268–279
5. Laakso, M., Salakoski, T., Korhonen, A., Malmi, L.: Automatic assessment of exercises for
algorithms and data structures – a case study with TRAKLA2. In: Proceedings of the 4th
Finnish/Baltic Sea Conference on Computer Science Education, October 1-3, 2004, Koli,
Finland, Helsinki, Finland, Helsinki University of Technology (2004) 28–36
6. Saikkonen, R., Malmi, L., Korhonen, A.: Fully automatic assessment of programming
exercises. In: ITiCSE ’01: Proceedings of the 6th annual conference on Innovation and
technology in computer science education, New York, NY, USA, ACM Press (2001) 133–
136
7. Higgins, C., Hegazy, T., Symeonidis, P., Tsintsifas, A.: The CourseMarker CBA system:
Improvements over Ceilidh. Education and Information Technologies 8(3) (September
2003) 287–304
8. Beierle, C., Kulaš, M., Widera, M.: A pragmatic approach to pre-testing Prolog programs. In
Seipel, D., Hanus, M., Geske, U., Breitenstein, O., eds.: Applications of Declarative
Programming and Knowledge Management. 15th International Conference on Applications
of Declarative Programming and Knowledge Management, INAP 2004, and 18th Workshop
on Logic Programming, WLP 2004, Potsdam, Germany, March 4-6, 2004, Revised Selected
Papers. Heidelberg, Springer (2005) 294–308
9. Zeller, A.: Making students read and review code. In: ITiCSE ’00: Proceedings of the 5th
annual SIGCSE/SIGCUE ITiCSE conference on Innovation and technology in computer
science education, New York, NY, USA, ACM Press (July 2000) 89–92
10. Bloom, B.S., Engelhart, M.D., Furst, E.J., Hill, W.H., eds.: Taxonomy of Educational
Objectives, Handbook 1: Cognitive Domain. Longman, White Plains, NY, USA (1956)
Exploratory Research on an Affective e-Learning Model
Liping Shen1, Enrique Leon2, Victor Callaghan2, Ruimin Shen1
1 – Computer Science Department, Shanghai Jiaotong University, China

2 - Computer Science Department, University of Essex, UK
lpshen@sjtu.edu.cn, {eeleon, vic}@essex.ac.uk, rmshen@sjtu.edu.cn
Abstract. This paper explores how emotion evolves during the learning process
with the longer term aim of developing learning systems that are able to
recognize and respond appropriately to emotions exhibited by learners. We
undertook this research by designing and building an experimental prototype of
an emotion aware learning system conducting experiments and studying the
relationship between emotion and learning. We report on our initial results
which not only indicate there is a usable relationship between affect and
learning, but by using the emotion states in Russell’s affective model, we have
been able to make some significant progress towards experimental validation of
Kort’s learning spiral model, which has not been empirically validated to-date.
Keywords: e-Learning, affective computing, emotion-aware.
1. INTRODUCTION
Background
Technology is changing our lives at a breathtaking rate, no more so than in the world
of education and e-Learning. The evolution of e-Learning can be traced from its roots
in Computer Aided Instruction, through Intelligent Tutor System and Web-based
Learning, to the Smart Classroom, Mobile Learning, Pervasive Learning and
Personalized Learning technologies of today. To date, in these developments, there
has been a bias towards the cognitive and relative neglect of the affective. Of course
nobody denies the role of ‘affect’ or emotion in learning. Certainly teachers know that
it plays a crucial role in motivation, interest, and attention. Research has
demonstrated, for example, that a slight positive mood does not just make you feel a
little better but also induces a different kind of thinking, characterized by a tendency
towards greater creativity and flexibility in problem solving, as well as more
efficiency and thoroughness in decision making[7]. These findings suggest emotion
may be an important factor in learning and point to new advances in understanding
the human brain not just as a purely cognitive information processing system, but as a
system in which both affective functions and cognitive functions are inextricably
integrated with one another.
268 Liping Shen et al.
Related Work
The extension of cognitive theory to explain and exploit the role of affect in learning
is, at best, in its infancy [12]. Kort [9] has proposed a four quadrant learning spiral
model in which emotions change while the learner moves through quadrants and up
the spiral, yet it has not been empirically validated.. He also proposed 5 sets of
emotion that may be relevant to learning, but, no follow-on studies into these basic
emotion sets for learning was reported. The Affective Computing Group at MIT’s
Media Lab is investigating the interplay of emotion, cognition, and learning as part of
its “Learning Companion” project. This project is developing an ‘affective
companion’ prototype that will provide emotional support to students in the learning
process, assisting them by helping to alleviate frustration and self-doubt [1]. Studies
carried out by the AutoTutor Group have provided evidence for a link between
learning and the affective states of confusion, flow and boredom [4]. For user emotion
modeling, Russell’s two-dimension ‘circumplex model of affect’ [14], where
emotions are seen as combinations of arousal and valence, is widely referenced. The
OCC [11] model has established itself as the standard appraisal model. This model
specifies 22 emotion categories based on valenced reactions to situations constructed
either as being goals of relevant events, as actions of an accountable agent, or as
attitudes of attractive or unattractive objects. Conati and Zhou are using the OCC
theory explicitly for recognizing user emotions in their educational game Prime Climb
[3]. Katsionis and Virvou have adapted OCC theory to model students’ emotions
while they learn in an educational game [8]. Beyond education applications, there is
also relevant work underway such as that by Hanjalic and Xu who represent and
model video content (in their case, movies) with emotion tags to support
personalization that can be used for applications such as the automatic generation of
‘video highlights’ or personalized recommendations for video films [6].
2. TECHNOLOGY PLATFORMS
The work reported in this paper is based on the integration of an emotion detection
system used to augment the operation of a cutting-edge intelligent environment test-
bed in Colchester known as the iDorm (intelligent Dormitory), with a massive e-
Learning test bed in Shanghai. The pervasive e-Learning platform (Fig1) was
developed by the School of Network Education of Shanghai Jiao Tong University
[17]. It delivers fully interactive lectures to PCs, laptops, PDA, IPTV and mobile
phones. It also includes a number of, what are called, "smart classrooms". The lecture
material can be accessed by students both in real-time (i.e. live) or from an archive
(within minutes of the lecture finishing). There are more than 15000 Students in
Network Education College, most being part-time students. They have different
backgrounds with dynamic knowledge structures. Given such diversity, it is important
to provide personalized learning services and to create learner profiles for students the
system has harnessed data mining technologies [16].
The intelligent Dormitory (iDorm) [2] is a cutting edge test-bed, based at Essex
University, for pervasive computing taking the form of a digital home The operation
of the iDorm is orchestrated by intelligent agents and the ooccupants of the iDorm
utilize a variety of networked services, including e-Learning (the iDorm is University

based, and occupants are frequently learners). Thus iDorm and the Smart Classroom
share much in common. As part of the iDorm work, Leon et al. have developed a real-
time emotion detection system, which achieved an 85.2% correct recognition rate in
experiments involving three emotional categories, (neutral, positive, and negative), on
8 subjects [10]. This approach comprises an eXperimental Vital-sign-based Emotional
State Transmitter (X-Vest), a finger clip with built-in sensors providing physiological
signals (heart rate - HR, skin resistance -SR, blood volume pressure -BVP, gradient of
skin resistance –GSR, and speed of the changes in the data -CS). It recognizes
affective changes using a combination of Auto Associative Neural Networks
(AANNs) and sequential analysis (UK Patent 0611762.6).
Figure 1. Pervasive eLearning Platform in Shanghai
3. AFFECTIVE LEARNING MODEL
3.1 Rational & R&D Strategy
Russell and Kort’s models share a common axis: the emotional state. If, during
learning, emotion is found to change in a consistent manner then this would provide a
means to study how learning behaviors relate to emotion (and vice-versa). At a simple
level this might be employed to provide teachers with feedback on a learner’s
emotional state (useful for remote learning where there are no visual cues). Moreover
if, during learning, the transition between emotional states on Kort’s model displays
some kinds of loops then this would indicate a tighter coupling between Russel and
Kort’s models, opening the possibility for the theory associated with these well
established models (e.g. Kort’s affective learning spiral) to be applied to emotion-
aware e-Learning systems.
Thus our experimentation focused on gathering data to explore the affective
evolution during learning and the relationship between Russell and Korts Models.
3.2 Affective Learning Models
As Picard [13] stated, “Theories of affect in learning need to be tested and evolved.
However, there is still very little understanding as to which emotions are most
important in learning, and how they influence learning. To date there is no
comprehensive, empirically validated, theory of emotion that addresses learning”, so
as a first step, we will use our prototype to fix the user emotion space using Russell’s
‘circumplex model’. We will then use the emotion states (personalized to the user)
detected during learning process to empirically validates Kort’s ‘Learning Spiral
Model’. The following is the description of these models and our rationale for
exploring the relationship between these two models.
3.3 Russell’s Circumplex Model of Affect
Figure 2. Russell’s circumplex model of affect
In Russell’s circumplex model of affect (Fig2), emotions are distributed in a system

of coordinates where the y-axis is the degree of arousal and the x-axis measures the
valence, from negative to positive emotions [13]. This model focuses on subjective
experience which means emotions within these dimensions might not be placed
exactly the same for all people. In fact, Figure 3 is the author Russell’s own
dimensional model of emotion.
Whilst Russel provides a comprehensive set of emotions, these are not well
matched to our more focused application of learning, and are too numerous for self-
assessment tests; therefore we have chosen a carefully selected subset and additions to
explore a basic emotions for learning, namely, interest/curiosity, engagement,
confusion/comprehension, frustration, boredom and hopefulness/optimism. At this
stage it is not clear that we have the optimum set for our needs, rather this is a starting
point and undoubtedly this may evolve or take many investigations before it is well
established.
3.4 Kort’s Learning Spiral Model
Kort [9] has proposed a four quadrant learning spiral model in which emotions change
while the learner moves through quadrants and up the spiral (Fig3). In quadrant I the
learner is experiencing positive affect and constructing knowledge. At this point, the
learner is working through the material with ease and has not experienced anything
overly puzzling. Once discrepancies start to arise between the information and the
learner’s knowledge structure, they move to quadrant II, which consists of
constructive learning and negative affect. Here they experience affective states such
as confusion. As the learner try to sort out the puzzle but fails, he might move into
quadrant III. This is the quadrant of unlearning and negative affect, when the learner
is experiencing states such as frustration. After the misconceptions are discarded, the
learner moves into quadrant IV, marked by unlearning and positive affect. While in
this quadrant the learner is still not sure exactly how to go forward. However, they do
acquire new insights and search for new ideas. Once they develop new ideas, they are
propelled back into quadrant I; thus, concluding one cycle around the learning spiral
of Kort et al. As learners move up the spiral, they become more competent and
acquire more domain knowledge.
Figure 3. Kort’s Learning Spiral Model

3.5 Rationale for Exploring Relationship between Russell and Korts Models
Russell and Kort’s models share a common axis: the emotional state. If, during
learning, emotion is found to change in a consistent manner then this would provide a
means to study how learning behaviors relate to emotion (and vice-versa). At a simple
level this might be employed to provide teachers with feedback on a learner’s
emotional state (especially useful for remote learning where there are no visual cues).
Moreover if, during learning, the transition between emotional states on Kort’s model
displays some kinds of loops, then this would indicate a tighter coupling between
Russel and Kort’s models, opening the possibility for the theory associated with these
well established models (e.g. the affective learning spiral) to be applied to emotion-
aware e-Learning systems.
Thus our initial experimentation has been focused on gathering data to explore the
affective evolution during learning and the relationship between Russell and Korts
Models.
4. PRELIMINARY EXPERIMENTS AND RESULTS
The preliminary experiment was carried out in the intelligent inhabited environment,
iDorm2.
4.1 Methods
The participant was a female visiting scholar who lived and worked in the iDorm2.
During the experiment, she wore the X-Vest which provided the valence value and
raw data from 5 biosensors. Data from the X-Vest was collected every 2 seconds. As
Skin Resistance (SR) is a very good indicator of arousal [12], we used the raw SR
data to linearly evaluate and track, in real-time, the arousal value. A low level of SR
denotes high arousal and vice versa. We observed that the subject’s skin resistance
can vary by as much as a factor of ten between morning and evening. To settle this
diurnal SR variation problem, we introduced a dynamic normalization (averaged over
the previous 5 minutes). The participant was asked to conduct the experiment twice a
day for 5 days, wearing the X-Vest and collecting arousal and valence data while she
was learning. In this preliminary experiment, the learning process and learning
materials are not pre-designed, i.e. the subject and learning materials are selected by
the participant herself. Each learning session lasted at least 30 minutes. The arousal-
valence data was displayed, in real time on a colored four quadrant diagram. Each
time the system detected a change of emotion, a multi-choice dialog was triggered,
listing six basic emotions, from which the participant had to select the nearest match
to her current emotion.
All the raw data, arousal, valence, and self reports were recorded together with
time tag in a data file for further study and analysis.
Figure 4. Skin Resistances Experimental Data

4.2 Results
Experimental data was gathered from 9 learning sessions and 1 TV session

(unstructured entertainment). Each session lasted at least 40 minutes with 4 sessions
including a self-reporting function.
Arousal Results
From the data, we observed the following interesting characteristics about how
arousal evolved during learning process:
1. During a single learning session, the arousal remains relatively stable. The standard
deviation was found to be around 100 K-Ohms (Fig 4b, c).
2. During the TV session, the arousal varied greatly and the standard deviation was as
large as 846 K-Ohms (Fig4a). This is consistent with the unstructured nature of the
material.
3. Arousal was not only the result of learning, but was influenced by other factors
such as physical exertion. For example, the participant reported that she was more
aroused to learn when she walked to and fro (which she usually does when she
feels tired or sleepy). The recorded SR data revealed this phenomenon (Fig 4d).
4. From Figure 4b, c, d, it can be seen that when the participant was learning, the
arousal was usually moderate, i.e. not too high or low.
Self-reporting Results
Russell’s two-dimension model of affect focuses on subjective experience; as such,
emotions within these dimensions might not be placed in exactly the same for all
people as it relies on personalities and the diversities of language (i.e. understanding
and expression of words differs greatly on culture and self-experience) and what’s
more, one emotion does not have a single fixed value in the Russell’s space [15].
Thus, to work on Russell’s model, we need to locate the experimenter emotions
within this space. As explained earlier, we adopted a set of 6 basic learning emotions
to locate the participant within the Russell’s space. From the self-report data (Table
1), for this participant, we observed confusion and engagement were the two most
frequent emotions her learning, whereas the frustration and boredom rarely occur. The
standard deviations are all very large, so we have chosen to use an 80% confidence
interval for each emotion. Figure 5 gives the emotional valence-arousal space of the
participant.
Table 1. The means, standard deviation and confidence intervals of 6 basic emotions
Emotion Mean Standard 80% Confidence Probability
deviation intervals (%)
Interest valence 0.655 0.484 (0.537, 0.773) 19.1
arousal 208.276 252.428 (146.751, 269.8)
Engagement valence 0.918 0.277 (0.867, 0.97) 32.2
arousal 171.265 216.681 (131.042, 211.489)
Confusion valence -0.569 0.5 (-0.654, -0.484) 38.1
arousal 74.845 334.884 (17.831,131.859)
Frustration valence -0.667 0.577 (-1, -0.038) 2.0
arousal -199.667 70.002 (-275.876, -123.458)
Boredom valence -1 0 -1 0.7
arousal -333 76 (-409, -333)
Hopefulness valence 0.917 0.289 (0.803, 1) 7.9
arousal -122.250 208.722 (-204.400, -40.1)
Figure 5. Participant’s Emotional Space
Affective Loop Results

Kort has suggested that learning behavior would manifest itself in a spiral-like
form i.e. a series of linked cycles separated in time. From our data we observed three
loops across the 4 quadrants (the red, green and purple loops) during a 15 minutes
learning process (Fig6). In addition, like all real-life processes, they are not idealized
forms, rather a noisier (e.g. our recognition rate is around 85%) and less smoothly
formed geometry. Even at this early stage of our work, these results suggest that there
is an approximately spiral nature to this data, although clearly we need more data and
better visualization to confirm this. We believe that learning loop depends on the
learning material and learning activity but, again, these need further study to validate.
However, we hope these initial results will prove encouraging to others who have
speculated on this relationship and hopefully will motivate more detailed work on this
aspect.
Figure 6. Affective Loop during Learning Process

5. DISCUSSION
Whilst our research is still ‘work in progress’, even at this early stage we have
uncovered some interesting results, such as:
• During a single learning session (up to 40 minutes), the arousal is relatively
stable
• People usually learn best in a state of moderate arousal.
• Arousal is not only the result of learning, but is effected by other factors.
• The participant’s emotional space was compatible to that of Russell’s model.
• Our experimental data reveals some kinds of learning loops which, to some
extent, validates Kort’s model (although more experiments and analysis are
needed)
The results we have reported in this paper are of preliminary experiments that,
whilst very encouraging, are still very coarse and need further refinement. In
particular we flag the following issues for additional research:
• Kort’s learning spiral model is restricted to a constructive approach and it needs
to be broadened out to include other ‘types’ of learning process, for example,
conceptualization and identification [5].
• The learning material used to evaluate this model needs to be more formally
designed to reveal learning behaviours, be more diverse and representative.
• There are factors, other than learning, that could influence emotion; for example,
who people are learning with; what they are learning; where they are learning and
so on. It may be that combining these variables at the right degree is the key to a
better affective learning model.
• To simply use skin conductivity as the sole indication of arousal is too crude.
There needs to be some investigation as to how more reliable arousal can be
obtained from physiological signals. Likewise, ideally valence would be
continuous than discrete.
• Our current experiments are based only on one participant; clearly, to make the
results more reliable, we would need to have a bigger and more controlled sample.
As should be clear from our discussion, this paper is work-in-progress and we are
reporting results from the first phase of a much longer term research program. Our
immediate aims are to refine the arousal analysis, design structured learning material,
and gather data from more participants. After that we plan to develop the affective
learning model combining affective information with wider learner profiles and the
existing Shanghai architecture. Finally we aim to deploy it in the Shanghai e-Learning
platform and evaluate it with real learners. We will look forward to report on this
work as it moves from research to real deployment over the coming years.
6. ACHNOWLEDGEMENTS
This is a collaborative research project conducted between the Digital Lifestyles

Center, University of Essex, UK (http://digital-lifestyles.essex.ac.uk) and e-Learning
Center, Shanghai Jiaotong University, China (www.dlc.sjtu.edu.cn ). During the
formation of the research ideas and models, we have enjoyed many useful discussions
with our colleagues. We would especially like to thank Chris Fowler for his valuable
guidance on the role of arousal in learning, Graham S. Clarke for his inspiring ideas,
Malcolm Lear for the biosensor technical design and support, my student Chenping
Lu for his programming work, and Michael Gardner for his strong support which has
been important in making this collaboration possible.
REFERENCES
1. Burleson W., Picard R. W. et al, A Platform for Affective Agent Research, Workshop
on Empathetic Agents, International Conference on Autonomous Agents and Multiagent
Systems, New York, NY, July 2004.
2. Callaghan V, Clark G, Colley M etc. ,Inhabited intelligent environments, BT
Technology Journal – Kluwer,Vol.22, No.3, July 2004
3. Conati, C. & Zhou, X. (2002) “Modeling students’ emotions from cognitive appraisal in
educational games”. In S.A. Cerri, G. Gouarderes and F. Paraguacu (Eds.) : Intelligent
Tutoring Systems 2002, LNCS, 2363, pp. 944-954, 2002.
4. Craig S. D. , Graesser A. C., Sullins J. etc., Affect and learning: an exploratory look into
the role of affect in learning with AutoTutor, Journal of Educational Media, Vol. 29, No.
3, October 2004
5. Fowler C.J.H., Mayes J.T., Learning Relationships from Theory to Design, Association
for Learning Technology Journal, 1999, Vol. 7 No.3, 6-16.
6. Hanjalic A., Xu L.Q., Affective Video Content Representation and Modeling, IEEE
TRANSACTIONS ON MULTIMEDIA, VOL. 7, NO. 1, 2005
7. Isen A M: ‘Positive affect and decision making’, in Lewis M and Haviland J (Eds):
‘Handbook of emotions’, Guilford, New York, (2000).
8. Katsionis G., Virvou M., Adapting OCC theory for affect perception in educational
software, Proceedings of the International Conference on Human-Computer Interaction
2005 (HCI 2005), Las Vegas, USA.
9. Kort B,Reilly R,Pcard R.An Affective Model of Interplay between Emotions and
Learning: Reengineering Educational Pedagogy-building a Learning Companion.
Proceedings of IEEE International Conference on Advanced Learning
Technologies,2001.
10. Leon E., Clarke G., Callaghan V., A User-independent Real-time Emotion Recognition
System for Software Agents in Domestic Environments, to appear in: Eng. App. of
Artificial Intelligence, Int’l Journal of Intelligent Real-Time Automation, summer 2006.
11. Ortony A., Clore, G. L. and Collins, A., the Cognitive Structure of Emotions, Cambridge
University Press, Cambridge, UK, 1988.
12. Picard R W and Scheirer J: ‘The Galvactivator: a glove that senses and communicates
skin conductivity’, Proceedings from the 9th International Conference on Human-
Computer Interaction, New Orleans (August 2001).
13. Picard R W, Papert S, Bender W, etc., Affective learning — a manifesto, BT
Technology Journal • Vol 22 No 4 • October 2004
14. Russell, J.A. (1980) A Circumplex Model of Affect, Journal of Personality and Social
Psychology, 39:6, pp. 1161-1178, American Psychological Association.
15. Russell J. A., Core affect and the psychological construction of emotion, Psychological
Review, 110, 145–172. 2003
16. Shen L.P., Shen R.M., Ontology-based Intelligent Learning Content Recommendation
Service, International Journal of Continuing Engineering Education and Life-Long
Learning, v 15, n 3-6, 2005, p 308-317
17. Shen R.M., Yang F., Han P., A Dynamic Self-organizing E-Learner Communities with
improved Multi-Agent Matchmaking Algorithm IN:AI2003, Dec.3-5, 2003,Perth,
Australia
Design and Development of Blended Learning through
LMS
K. P. Hewagamage, S.C. Premaratne, K.H.R.A. Peiris
e-Learning Centre, University of Colombo School of Computing,

Email: {kph, scp, rap @ucsc.cmb.ac.lk}
Abstract. Design and Development of Blended Learning is a challenging task

than it is anticipated when we consider features and facilities in a Learning
Management System (LMS). In this paper, we present what those challenges
and how we have solved both technical as well as social problems in
implementing a blended learning environment for the University of Colombo
School of Computing (UCSC). It is very hard to achieve what we plan without
all stakeholders participation with positive attitude. ICT awareness is very
important for the success of blended learning but it is not everything for its
success. More learning happens when students start engaging in activities
which overlap both the classroom as well as the virtual learning environment
(VLE). We also present our experience in activity mapping as a way to extend
the classroom environment
Keywords: blended learning, VLE, LMS, activity mapping, learning activities
1. Introduction
In a self-evaluation of undergraduate programs, UCSC (University of Colombo

School of Computing) discovered several problems which are affecting negatively for
the internal student courses. In the analysis of these problems, the faculty decided to
use e-learning based methodology as a part of enhancing learning environment of
internal students. The e-Learning Centre of UCSC was given the responsibility of
carrying out this initiative of setting up the blended learning environment. However,
this project faced so many challenges than anticipated in the very beginning.
In this paper, we are presenting our experience in developing a blended learning
environment for undergraduates’ courses conducted by the University of Colombo
School of Computing (UCSC). The faculty and students are ICT literate community
since their main discipline is computer science. Therefore, the background of faculty
and students is enough to undertake the responsibility in participating e-learning
based activities. Technical infrastructure of the UCSC and technical knowledge of
users (staff and students) in the organization are not enough to achieve objectives of
the e-learning framework. Those two factors are like front wheels of a vehicle and we
need 4 wheels with good set of tires for a smooth and steady journey. Irrespective of
power and capabilities, obstacles are unavoidable in this journey and we should be
prepared to face these challenges.
280 K. P. Hewagamage, S.C. Premaratne, K.H.R.A. Peiris
We first describe on our vision of improving the relevance and quality of

undergraduate education, which we have quoted as IRQUE project of the
organization. In this project, we have carried out several evaluations with respect to
education process, initial capabilities of students including their soft skills, final
outcome and output of the degree program, and staff capabilities and weaknesses.
Before building a blended learning environment, such evaluation is very important to
identify the background and foundation.
e-Learning Centre (eLC) was established in order to assign the responsibility of
carrying out e-learning integration of all the academic and professional services of
UCSC. A specialized staff was recruited in planning and development of virtual
learning environment (VLE) for all internal undergraduates. However, many of issues
faced in this project didn’t have ready made solutions and we had to carry out
research and development work in finding suitable solutions. In the section 3 of the
paper, we will summarize initial research and development work related to the
blended learning environment.
We identified the VLE is the heart of a blended learning environment and its
proper integration with the classroom teaching is the key to initiate the change. It is
important to consider the VLE as an extension of classroom activities rather than the
supplementary system to access learning resources. At the same time, VLE should
provide facilities to implement collaborative learning activities among students in the
classroom. Some of these activities will be initiated and carried out independent of the
teacher/lecture of the classroom.
In this paper, we describe challenging issues we have encountered in this endeavor
together with our solutions so far in building blended learning environment in Section
5. Finally we conclude the paper by summarizing the work presented together with
our future plan in enhancing our blended learning environment in the Section 7.
2. Improving Relevance and Quality (IRQUE)
What is the quality and relevance in higher education study programs? There are
different interpretations about them but simply they are some sort of indicators about
the study programs and its applicability in the industry. The quality can be measured
using student performance in the subject evaluation and their ability to apply
subjective knowledge in problem solving. The relevance is usually measured with
respect to their ability provide solutions at very beginning of first employment.
Generally, there is a gap between employer’s requirement and graduates capabilities.
If graduates have enough soft skills, then they can reduce the gap through a simple
training program. However, if graduates cannot demonstrate their capabilities to
reduce the gap, they will be unemployed or underemployed. At the same time, these
two attributes are not completely independent ones and they are dependable on each
other.
The most important process which is required to enhance the education process in
which teachers and students interact each other is the transformation of the
knowledge. However, we identified under the IRQUE evaluation that teacher student
ratio has adversely affected due to the increase of intake by 400% in recent batches of
under progress. Teachers are delivering lectures to large number of students (around
120 – 160 students) at once and there is no dialog between teachers and students.
Both parties find difficulties to involve in any sort of dialog in front of all these
students. All lecture materials (usually power point slides) are given in advance of
lectures. There is no feedback about lecture notes from students to teachers. Reading
those lecture notes is not interactive and students are helpless in their learning
process.
In this evaluation, we observed the failure rate and drop out rate are also
increasing. The lecturer and students ratio is not so bad but the interaction between
two parties seems to be poor. Although students have enough computer lab access
facilities together with Internet, electronic learning resources are not provided through
this infrastructure. Library does not have enough copies of recommended reading
materials and no digital versions of those recommended reading materials are
provided to access through the local area network of the organization.
It was also not possible to give attention to different levels of students when there
are around 160 students in a lecture hall. These students cannot be grouped according
to their skills and provide different lecturing due to administrative and resource
constraints in the organization. Most of students have language difficulties and they
need a special attention due to poor understanding contents/factors taught in the
lectures.
3. Establishment of e-Learning Centre
UCSC is a computing faculty of University of Colombo (UoC) and its faculty wanted
to find out IT solutions to minimize bad effects in this process. The use of information
and other technologies are not a proper integration of IT and education. Sometimes, it
could diversely affect the education process. For example, the use of power point
slides in the lecture halls reduces student active role in the process. Students are very
keen and enthusiastic to have more and more notes and nice illustrations on visually
projected screens but many students fell in sleep and may not properly absorb the
required knowledge at the end of teaching sessions.
When the UCSC is formed as an autonomous faculty of UoC, the importance of e-
Learning and its future role is identified [4,7] and a separate centre was established to
provide the service of e-Learning to students and staff while promoting R&D work in
this area. A coordinator was appointed among academic staff members who have
experience of practical difficulties and challenges faced in the academic education
process. A separate staff was recruited for the system administration as well as
content development in e-learning centre. Initially the focus was towards the distance
learning initiative of the UCSC. However, as mentioned above, after IRQUE
evaluations, the importance of proper integration of technology with the education
process is highlighted specially due to sudden increase in the intake of students (from
40 to 160 students).
4. Learning Management System (LMS)
The importance of learning management system was identified as a significant

requirement. Several systems both commercial and open source products were
evaluated [2] considering various factors such as network resources and
infrastructure, resource availability for sustainability (i.e. funds allocated from the
institute budget), customization of the product for specific requirement of the
organization, external acceptance of similar academic organization, etc. We first
evaluated several commercial products WebCT, Blackboard and Theducation (a
Swedish product localized to English), but they were not selected specially due to two
reasons, the customization of the product and the cost of product which was based on
the number of users of the system. However, when we evaluated Open Source
Products, those two critical issues in the commercial products were not major issues.
When we evaluated the community around these products, Moodle (Modular Object-
Oriented Dynamic Learning Environment) seems to be acceptable FOSS product with
a word wide user group. At the same time, the computing resources required for the
deployment of product are something which is commonly used in a web server (i.e.
Apachie, PHP, MySQL). After several testing, eLC obtained the approval from the
academic board of study to use the Moodle LMS as the learning management system
to implement the blended learning environment of UCSC.
5. Social Resistance for the Blended Learning
When the term e-Learning was used among the academic community as a way to
enhance education process, there was some resistance among some faculty members
just like the way the computerization or ICT enable environment is introduced for any
working environment. At the beginning, it was a surprise for us since UCSC is a
computing faculty which advocates its theoretical & engineering development as well
as integration of computing with other working environment. However, every human
teacher will have the common fear of ICT, when he/she did not properly understand
the potential of the ICT in education. Hence, the very first challenge of eLC in its
blended learning initiative was a social constraint due to the various interpretation of
e-Learning and its applicability of e-Learning.
Negative Arguments against the Blended Learning Initiative:

• Technology adversely affects students’ active status in the classroom [this
argument was based on the use of power point slides in a classroom teaching.
When students are not taking down notes, student attention may vary and some of
them will start engaging other activities like chatting, sleeping etc.]
• Accessibility of technology during classroom [all students do not possess laptop or
any portable devices when lectures are delivered]
• Imbalance of Number of Computers: Students ratio and its adverse affect in
allocating enough time to access LMS or Internet.
• Increase in the teachers’ workload
• Teachers’ unawareness of facilities and methodologies [Many of senior staff have

highly academically qualified researchers but they do not have a special training
about teaching methods]
Positive argument for the Blended Learning Initiative;
• Possible solution being tested [6,5]
• Technology infrastructure has already been established
• ICT-awareness is above the required level (UCSC is a computing faculty)
Action taken to promote the blended learning:
• Special demonstrations are held about LMS to senior staff.
• Several workshops were carried out to educate the academic staff (Workshops are
carried out with respect to small groups since it was difficult to get all staff
committed to a single time slot).
• A LMS Administrator was appointed to provide on demand help for academic staff
• All research assistances at eLC are advised to work closely with respective
teachers (senior academic) in setting up relevant courses in the LMS.
These actions didn’t give 100% solution to social issues in implementing a blended
learning environment. However, it was a satisfactory beginning.
Student training and evaluation of the service:
Documentation doesn’t solve all problems faced when a virtual learning environment
is introduced for the blended learning. A demonstration followed by a proper training
is very crucial even before the starting formal academic courses. Training is usually
conducted as hands on workshops for small groups at a time. In this training, LMS
features are introduced illustrating how students can participate in the blended
learning activities.
6. Technical Challenges to introduce LMS
When a new system is installed, it is natural to face many technical problems which
are not unforeseen during the testing stage. Especially when a system is being used
more than 100 students at the same time. Therefore, it takes some period of time for
the system to become stabilize. At the same time, we are using an open source
product, it is very important to keep a good eye on possible bugs in the system and
carry out regular update with the system. At the same time, it is not advisable to
experiment with an online running system which has real users. Proper software
engineering approach must be utilized in the maintenance of the system. Therefore we
sat up an identical offline copy of the same system, for testing when system updates
are posted. The LMS admin evaluates updates/patches of the system with the offline
system, before updating them with an online real system.
Taking the back up is a critical requirement since it is hard to justify when what
will be happened. It could be a technical failure of hardware or software but eLC must
have to accept the damage if something is happened. Taking the backup is not enough
and it doesn’t give any validation or verification that the backups can be restored. We
found the offline testing version is very useful to verify these backup taken.
Taking a backup once in a month was also not enough; we had to develop a policy
for taking backups. Our policy was based on following factors:
• Time between backups is fixed to one day [Incremental approach for the backup
taking was implemented]
• Backups are supposed to take when LMS activities are low.
• Teachers were instructed to maintain their course backup [Many teachers were not
happy with this request but we wanted to convince it is a responsibility of teacher]
One of the big threats is Internal hacking of LMS by students. The direct hack
through server as well as hacking through the LMS is another issue. Sometimes,
friendly hacking is done by their friends. We usually ask students to update their
profile with the personal email address but students are usually very careless. They
also do not get any official email address and it also affected the official
communication. Simply, LMS login and other system accesses must be linked
through a central system of the organization in order to avoid some of technical
failures.
Network congestion
A computer is multi-tasking equipment and students (as well as other users) usually
use a computer for several activities at the same time. Many of these applications use
network/Internet bandwidth in addition to the processor and memory of the machine.
A typical example is that the user may open a web browser with several windows to
access different web sites, a messenger application, streaming application for audio or
video content or an online game. When a student logs into the LMS with all these
active applications, accessibility is not quick as they want. Depending on the
resources, the users must select relevant applications without disturbing to their own
as well as others in the laboratory (for example, when a student starts downloading a
big file, it affects everybody in his/her network). In addition to all these things, many
student machines in computer labs are infected by viruses and it creates unnecessary
traffic in the network system.
Therefore, it is very important to establish a resource access policy when the
blended learning environment is being introduced. This policy could be a guideline to
reduce the problems that users face when they access the virtual learning
environment. It is not very difficult to introduce a policy but people hardly follow
these policies and keep complaining the system and the network.
7. Activity Design for Blended Learning:
This is the hardest part when the virtual learning environment (VLE) is introduced to
enhance the classroom environment. It is not possible to implement all classroom
activities through the VLE but some activities can be enhanced by overcoming
limitations in the classroom environment. At the same time, a new set of activities can
be designed and developed using the features of VLE, and if properly used, the
efficiency and effectiveness of the whole education process can be enhanced
significantly. We discuss mapping of some selected classroom activities for the
blended learning based on our experience in this section.
i. Classroom Activity: Power point based lectures
Description:
A lecturer usually conducts one or two hour lectures in a lecture room for 40 – 160
students at a time. (Depending on the classroom, some lectures use a microphone
otherwise students find very difficulty to hear the lecturer’s narration). Lecturer may
ask some questions but students hardly answer when there is a large number. Students
can ask clarification from a teacher but it is not commonly practiced in the classroom
environment. Students may get a printed note set in advance and he/she may take
down some notes based on the explanation in the lecture session.
If a student looses his attention, he may find difficult to understand what is
presented. A teacher may find very boring exercise when he/she has to repeat basic
details. Sometimes, the lecturer may late and he may not give the full description due
to time constraints in the schedule of classrooms and/or the lecturer has to complete
more content with respect to syllabus.
Blended Design:
Lectures are recorded based on the curriculum without any time constraints. Based on
the topics and sub-topics in the syllabus, recorded videos (lecturer’s face and power
point slides synchronized) are segmented for short durations. A segment size may be
15 – 30 minutes (It was recommended to keep segment size 15 minutes if possible).
However, it was not possible to distribute these video materials through LMS due
to their size and available bandwidth in the network.
Instead of normal lecturer, these video presentations were used in the classrooms
and after every 15 minutes, the lecturer or tutor, stops playing the video presentation
for a discussion or questions. Effectiveness is high but time management became a
problem to cover the syllabus. Therefore, these resources are made available from the
library.
ii. Classroom Activity: Question and Answer ( Q&A) Session
Description:
A lecturer usually spends last 5 -10 minutes in every lecture to discuss difficult things
that students cannot understand and answer some of their questions related to the
session. All students do not get this opportunity and many of them do not want to ask
questions in front of others since they are not sure about the question.
Blended Design:
The course page, in the LMS, is structured with respect to syllabus and a discussion
forum is added at the end of every section. In the classroom, when a lecturer
completes a section, he/she reminds students to ask relevant questions under this
discussion forum. If there are no questions in a section, the teacher will ask a random
question to initiate the dialog. In the classroom environment, when a student asks a
question, only a teacher/tutor answers this question. However, in the discussion
forums, other students also started answer questions by promoting collaborative
learning environment which was not easy to establish in a normal classroom.
Fig. 1. Course Page in the Moodle LMS - Based on the sections in the syllabus of the course,
lessons and other resources are added
iii. Classroom Activity: Giving References for Topics
Description:
It is a common practice that a lecturer gives references during a session. This could be
a reference of recommended reading materials, a web resource or a personal note. In
reality, the students find various difficulties to follow up these references and there
will be no dialog between teacher and students about these references.
Blended Design:
The teacher can add all these references as resources under the relevant topic in the
course page of LMS. He/she can scan relevant pages of recommended reading pages
and upload as resources in this course page. Students can download them for offline
reading without waiting to access materials from the library.
iv. Classroom Activity: Revision Exercises (MCQ practice questions)
Description:
During lecturers or after lecturers, students usually get exercises as practice questions.
Sometimes they may get answers to verify themselves. However, there is no feedback
from lecturers or tutors directly. Question type also vary from the multiple choice
questions (MCQ) to structured or essay type questions.
Blended Design:
At each section of course page, there is a practice question covering most important
key questions. However, the question type is limited to MCQ and closed type
questions. It was possible to provide essay type questions but the feedback was not
possible to automate.
v. Classroom Activity: Open Discussion of selected topics of interest
Description:
In reality, open matters related to a particular subject is rarely discussed during a
lecture or after a lecture. Occasionally, some students meet a lecturer to discuss these
types of issues outside the classroom. However, they are never heard by other
students in the same class/group except the lecturer mentioned them in the next
session when he meets all students.
Fig. 2. Open Discussion among Teachers and Students

Blended design:
If a student has an open issue or question, he/she can post it in a discussion forum of
relevant section or news forum of the course. Posting to a news forum bring the
attention of all students and teachers in the course. Teachers can also initiate
interesting dialog by posting open issues of the subject (See Fig. 2).
vi. Classroom Activity: Formative Assessment
Description:
According to curriculum, all courses offer around 40% of marks from the formative
assessment and the rest of marks from summative assessment at the end of semester.
This formative assessment is usually based on the assignments given during the
course period. Some of these assignments are individual ones and others are group
work. In reality, there will be several administrative difficulties teachers as well as
students faces when they are supposed to submit assignment work. For example,
printers in the lab may be difficult resource to use near the deadlines, and some
students later complain they submit on time but no administrative assistants to accept
the submission just before the deadline.
Blended Design:
The distribution of assignments and submission of assignments were easy to manage
or handle when those activities are channeled through the LMS. It saves lots of
resources (when 160 students have to submit soft copies only) and makes it efficient
since the teacher and assistants do not want to keep the manual record of submissions.
Students can directly submit the report from the LMS. The deadline can be easily
implemented without causing administrative problems.
vii. Classroom Activity: Tutoring
Description:
When lecturers are difficult to understand, tutorial classes are conducted for small
groups by tutors. More exercises are usually discussed in these classes and their
success depends on the skills of tutors. Students have mix opinions on these classes
and attendance is poor or irregular.
Blended Design:
Interactive e-learning lessons are designed as additional learning resources for all
students. They are usually developed by instructional designers at the e-Learning
Centre according to teachers’ (Subject Mater Experts) explanation of learning content.
Instruction designer will use different presentation to simplify the learning content.
Figure 3 shows a sample image of such interactive lesson.
viii. Classroom Activity: Group Work
Description:
When a group work is given, it is important that all members discuss the given work
and distribute among all members the work. In reality, all members are not active and
sometimes some members complain that they do not get a good opportunity to
participate in these activities. At the same time, there are passengers in groups, who
get full marks without doing anything. Lecturers usually use lot of adhoc techniques
to allocate marks among group members but they are not very successful.
Blended Design:
When the group work is given, students are categorized as groups within the course of
LMS. All students are supposed to participate in the discussion only through group
forums which are not visible to other students. When they have classroom meetings,
the group leader has to write minutes of meeting in the group wiki where others can
make comments. Teachers use this information when they allocate marks among
group members.
Fig. 3. Interactive Learning to support tutoring in the blended learning
ix. Classroom Activity: Collaborative Construction of Artifact
Description:
Many assignments are group work and each group must submit the group report on or
before the given deadline. Sometimes these reports are copied from past reports in the
library. The collaboration of group work is not visible among participants. Teachers
find some practical difficulties when they correct these reports and allocate marks
among group members.
Blended Design:
Group reports are required in a form of a wiki. Teacher can see its development
progress and can also find out their collaboration. Interestingly, teacher can give his
comments before the final deadline which is not an option in the normal classroom
environment.
x. Classroom Activity: Sharing personal notes
Description:
Some talented students prepare their own notes based on the lectures and syllabus.
They are not shared and sometimes not visible for teachers. If teacher can identify
these talented role players, it is very easy to promote collaborative learning model in a
classroom environment.
Blended Design:
All students in the course are allowed to maintain their own blog or journal as their
personal notes. Teacher may examine such blogs if they get more attention from other
students (based on the LMS statistics). When a teacher wants to give a
recommendation or additional marks, he/she will consider these notes.
8. Summary and Future work
In this paper, we have descried our experience in developing a blended learning

environment for internal undergraduates of University of Colombo School of
Computing. The requirement of blended learning was clear when the batch size of
student intake was increased by 400%.
When e-learning is introduced to enhance the classroom learning, it is possible to
get several resistances from both teachers and students. Some of resistances are due to
lack of knowledge about potential capabilities and others are due to visibility of
activities. These social resistances from the environment can be controlled by
assisting and training teachers and students.
The virtual learning environment (VLE) given through the learning management
system (LMS) is the backbone in introducing blended learning environment. Open
source LMSes are feasible products to develop VLE and our experience given in this
paper is based on the Moodle LMS. There are several technical challenges which
must be carefully and properly handle when any LMS is introduced for the blended
learning. Some of challenges are due to other factors such as network congestion as
stated in this paper.
In order for the success of blended learning, it is very important to use VLE to
extend classroom activities rather than supplementary solutions. Therefore, classroom
activities must be properly mapped to a blended design. In this paper, we have listed
10 such selected activities to describe our experience. However, those mapping could
be further improved to enhance the effectiveness and efficiency in the blended
learning environment.
According to feedback from teachers and students, we are planning to enhance the
activity mapping from classroom environment to VLE. Our immediate plan is to
develop a catalog of activities which can be mapped from the real classroom
environment to a blended learning environment. The activity mapping will be
developed like design patterns [3] for teachers and their assistants when they develop
want to develop blended learning environment. We hope this mechanism will be
useful in extending classroom activities to integrate the virtual learning environment.
References:
1. K. P. Hewagamage, V. K. Samaranayake, A. R. Weerasinghe, and G. I. Gamage,

“Facing Challenges of an External Degree Program using ICT” A Case study of
University of Colombo School of Computing”, in the proceedings of Digital Learning
2005 (DL2005),
2. K. P. Hewagamage, K.H.R.A Peiris, W.A.U.C. Weerakoon, “Evaluation and Adaptation
of Learning Management System (LMS) for Higher Education”, in the proceedings of
Digital Learning 2006 (DL2006), Bangkok, Thailand, May 2006
3. “Web services for blended learning patterns” Derntl, M.; Mangler, J., Advanced
Learning Technologies, 2004. Proceedings. IEEE International Conference 30 Aug.-1
Sept. 2004,
4. “Software Engineering Education Applying a Blended Learning Strategy”, Bunse, C.;
Grutzner, I.; Ochs, M.; Peper, C.; Steinbach-Nordmann, S. Software Engineering
Education and Training, 2005. CSEE&T 2005. Proceedings. 18th Conference on, Vol.,
Iss., 18-20 April 2005,
5. “Designing for the changing role of the instructor in blended learning” Danchak, M.M.;
Huguet, M.-P. Professional Communication, IEEE Transactions on, Vol.47, Iss.3, Sept.
2004,
6. “The instructional effects of on-line tests on the large-scale IT courses” Nakano, H. etc.
Information Technology Based Higher Education and Training, 2005. ITHET 2005. July
2005.
7. “Enhancing the student project team experience with blended learning techniques”,
Reichlmayr, T. Frontiers in Education, 2005. FIE '05. Proceedings 35th Annual
Conference, Oct. 2005.
An Access Monitoring Tool Based on Cookies for
Course Management Systems
Raquel Hijón-Neira, Ignacio López-López, Ángel Velázquez-Iturbide,

Francisco Domínguez-Mateos
Universidad Rey Juan Carlos

28933 Móstoles, Madrid, Spain
{raquel.hijon,ignacio.lopez,angel.velazquez,
francisco.dominguez }@urjc.es
Abstract. In this paper, we describe an application developed in our university

in order to gain insight of students’ interactions with an e-learning platform.
Access to this information can be very useful to teachers, who can rethink the
way they present their material based on students’ usage patterns. The
application uses cookies as the basis to identify different users. It offers a very
wide range of queries as well as diagrams to represent students’ interactions
with the platform. The application is portable and can be easily installed at any
other type of website that we need to analyze.
1 Introduction
It can be expected that the widespread use of Course Management Systems (CMS),
both for on-line learning and for blended learning, will affect the way teachers
evaluate students. On the one hand, different ways of evaluation can be developed.
On the other hand, teachers may use information about the on-line activity of students
to evaluate their work. We are interested in the design and implementation of tools
and applications to gather information about students’ activity for both purposes. This
information may also lead to discovery of learning and usability patterns [1] that may
help us to rethink our teaching, or maybe the way we offer the information to the
students [2, 5].
We are working and exploding the different ways that the information of students’
accesses to CMSs can be obtained. In [3] we explained the steps that were needed,
and its technical requirements. We also found out that there are a wide range of tools
that could generate some of this information but none were specifically designed for
education and e-learning.
In order to fill this gap, we are addressing several alternative ways of obtaining
information about students’ activity. Firstly, we developed an ad-hoc application that
collects information about students’ accesses to the web site in a dedicated database.
Secondly, we analyzed log files using a standard application working on the server.
Thirdly, more subjective studies were made by means of survey analysis [4] and e-
questionnaires [6]. Finally, our most recent approach has been the development of an
An Access Monitoring Tool Based on Cookies for Course Management Systems 293
application that uses cookies to analyze students’ interactions with our CMS, visually
providing several graphical representations of it (Fig.1 shows the architecture).
Fig.1. Tracking Tools assessing URJC CMS
In the rest of the paper we describe the application based on cookies. In section 2
we explain our objectives. In section 3 we include an evaluation of existing tools,
their pros and cons. In section 4 we give an overview of the functionality, user
interface and technologies of the application. In section 5, we describe in more detail
its underlying Data Base Description and Cookies Functionality. In section 6, the
analysis of the students’ interaction with our tool can be seen. Finally, we describe
our conclusions and future work.
2 Objectives of the Application
Since we had developed another tool that visualizes nominal accesses, we did not
need the personal identification of students entering into the CMS, but a lively and
compact tool like this one that allowed us to treat the information from the accesses
quickly, online and on real time like it does. The access monitoring tool based on
cookies that we are describing implements the following requirements:
• Our necessity was the acquisition of information from the students interacting
with our CMS, making a distinction between its different pages and parts, such us
forums, assignments, theory, etc.
• It should provide factual information (in a table format) as well as statistical (in a
graphical format) in real time and from any computer connected to the Internet.
• It should be shared under a GPL license, and therefore, distributed and upgraded
by the user to meet future requirements for free.
294 Raquel Hijón-Neira et al.
• It should store non personal data from the students accessing the site (operating
system, browser, screen resolution, IP and country of origin).
• It should be able to identify visitors of the CMS in each visit they do to the site.
• It should track the route followed by each user in each visit.
• It should have user-friendly interfaces that allow teachers to easily monitor
interactions with the CMS.
• It should provide printable versions of the queries made, selected and grouped
under different criteria.
• It should allow the selection of a time period by a calendar option, and
combinations of it, such us: complete days, months or academic years.
3 Evaluation of Existing Applications
A wide range of tools that apparently cover the functionality required was analyzed
and tested. In the following table (see Table 1), a comparative analysis is shown,
where files of the chart refer to the following products:
A. Nedstat [14] European leader in web analysis, it also allows the creation of
customized reports.
B. Onestat [15] World leader in statistics of the web sites.
C. Google Analytics [12] offers information on how the user that found the site,
interact with it.
D. Estadisticasgratis [10] offers traffic analysis based on the users profile.
E. Estadisticas-gratis [11] complete statistical service for free, real time service and
24 hours a day.
F. Developers4web [9] Detailed statistics very easy to use.
G. Akstat [7] uses markers to measure the web site.
H. Logdy [13] very new service, very intuitive and easy to use.
I. CQ Counter [8] has an easy to use interface and an extraordinary service.
J. XITI [18] offer basic indicators of the web site traffic.
K. Weboscope [19] allows multiple web site analyzers.
L. Contaweb [16] contains a little element that inserts itself in the web pages that are
going to be analyzed.
In columns, it is evaluated the most relevant aspects for us, namely:
1. The group of applications that serve our CMS have been developed under the
same technology: PHP, MySQL, JavaScript and HTML. So, it is important, that
at least, the new application uses the same database, since they have to share
tables.
2. Does it use cookies? We studied if they were using this technology, sometimes it
was hard to know, since they do not provide us with the source code.
3. Does it do the tracking from analysis of log files?
4. Does it offer detailed time visit analysis by different parameters such us by hour,
day, week, month, year, etc?
5. Does it show the kind of operative system the visitors use?
6. Does if offer the type of browser the visitors use to connect to the CMS?
7. Does it offer the information about the screen resolution the visitors have?
8. Does it show the color deepness of users?
9. Does the application bring the plug-ins installed?
10. Does it offer visits´ geographical location?
11. Does it show sources of accesses to the Web?
12. More Common Routes are shown in detail?
13. Does it support local hosting?
14. Does it support multi-language?
15. Does it have a GPL license?
16. Is it a free version? If not, what is the price (in €) per year for an average website
of 20 pages? (When their product offers a free and a commercial version it
appears both data).
17. Does the system include publicity?
18. Has the application been designed to deal with e-learning and/or CMS?
Table 1. Comparative chart of similar products
3 – The application has the characteristic

2 – The application does not have the characteristic
? – By the information offered we can not evaluated it
As we can see, none of them has been created to deal with CMS. Furthermore, the
free versions some offer are a very weak representation of the features the commercial
product has, moreover, these versions support a whole lot of publicity. Thus, either
we could not know or they did not use cookies but two of them (D and E), therefore
three were not suitable since we wanted to use them in our approach. So, even when
paying a fortune, all the requirements are not supported by any of them.
4 Description of the Tool
The user interface that the teachers use to query access statistics is shown in Fig. 2.
1
7
4
2
3 5
6
Fig.2. Our Application at the Rey Juan Carlos University
In the area 1, a menu allows selecting general facilities, such us: summary of visits,
change of password, create another (teacher) user, pages analyzed, obtain PHP code
(to allow each new page added into the site to be tracked), summary of the visits,
statistics which is the one selected in the previous figure, help about how to use the
application, information about it, suggestions and log off.
If we focus on the part named ‘Statistics by Number of Visits’ (area 2), we can
query statistics for a range of dates, select the pages or subjects we want to examine,
and summarize the info for different criteria.
The graphical format of the results (either a class of graph or a table) can also be
selected (area 3) for this query as well as queries in areas 5 and 6. Currently, the
following graphs are supported: pie graph, bar chart, line graph, and table.
If we click in ‘Most Common Routes’ (area 4), it is shown the most visited
sequence of pages. As a parameter of the query, the user is required to give the
number of clicks of the sequence.
If we click in ‘Number of Visits per page’ (area 5), it is shown the number of visits
for a range of dates.
If we select ‘Statistics by User’ (area 6), it is shown the user information for a
previously selected period of time, grouped by one of the following options: operating
system, browser, screen resolution, and country of origin.
It is mandatory the selection of a range of dates (area 7). There are options to select
a week, month, academic year, as well as a specific date from a calendar that is
displayed to enter dates.
The application uses cookies to identify the students accessing the site. It was
developed with PHP and database is in MySQL. Finally, some JavaScript code allows
implementing some special features.
The application is available under a GPL license, so it can be distributed and
installed in any website. It has a self-installation feature that automatically identifies
the different parts of the CMS to be studied and creates the database to store accesses.
6 Data Base Description and Cookies Functionality
6.1 Data Base Description
The Data Base has been developed in MySQL, due to the compatibility with the rest
of data bases that form the CMS. The structure is formed by five tables that fulfill the
data storage requirements (see Fig. 3).
In the table 'Users', there are all the data related to the users that have visited at
least once our CMS. The attributes are: Users identification, ip address, Operating
system, browser, screen resolution and country of origin.
In the table 'Pages' are stored all the pages that are visited in the CMS. The page
tuple is included the first time a student enters a page. The attributes are: page
identification and page name.
In the table 'Visits' are stored all the visits that have occurred on the CMS. Each
time a student enters a new tuple is created. The attributes are: visit identification,
user identification, page identification, previous visit identification (if it exits), day,
month, year, week and time.
In the table 'Administrator' are stored the information related to the users of the
Statistics tool, which are the teachers. The attributes are: user name, password, cookie
value, state (identify if the user is authenticated or not, that is to say, if it is a teacher
or a student).
Fig.3. Entity-Relationship model
The table 'Countries' contains the ip sub rank corresponding to each country. The
attributes are: identification of register (primary key), integer value that correspond to
the beginning of the ip sub rank, integer value that correspond to the end of the ip sub
rank, two fields for different codifications of the country (with two and three
characters each) and the country name.
6.2 Cookies Functionality
As we have been seeing, the users’ identification is being made by cookies. In Fig. 4
there is a description of the process. When a visitor enters into any of the web pages
the CMS has, the application validates if the user present the user cookie, in case
he/she does, the visit is stored in the corresponding table since the user has been
authenticated. In case the user does not present this cookie, the application checks if
he/she presents the accept cookie (which is the cookie that is used to check if the user
accepts cookies), if this cookie is presented the user information is stored, as well as
the user cookie and the visit information.
Fig.4. Cookies Functionality
If the user does not present the accept cookie, the application checks if the user has
been redirected , in case she/he has already been redirected, it has been checked that
the user does not accept cookies; Finally, if the user has not have been redirected, the
accept cookie is saved and he/she is redirected to the home page.
7 An Example of Analysis of Student’s Interaction with the Tool
We are going to place here a set of examples of the variety of information the teachers
can obtain from the website.
We have asked for the number of visits the Software Engineering subject in the
fourth course of Computer Science Engineering has had in November of 2006, see the
results in Fig. 5. We have selected the information representation in a horizontal bar
chart and also in a table form. It can be corroborated that the two days that students
most connected to the site where November 10th and 17th which were those Fridays
after practice class when they most needed to make downloads and check information
in the CMS.
In Fig. 6 we have asked for the country of origin of visits accessing the subject
pages in our CMS. We have chosen this time a pie chart representation as well as a
table form. It can be observed that 98% of the visits are from Spain, but the teacher
can check that there are a 2% from 6 different countries in this month, which
corroborates that our 2 students that this semester are in Finland and Sweden
respectively are working on the subject.
In Fig. 7 we have asked the tool for the route most followed in the CMS for three
clicks, and we have found out that it is the path that leads to the theoretical material of
the subject Programming Methodology in the second course of Computer Science
Engineering at our University.
Fig.5. Subject Engineering visits in Fig.6. Country of origin of visits to

November 2006 subject pages
Fig.7. Route most followed for 3 clicks Fig.8. Summary of visits to the CMS
Moreover, if a summary of the traffic web site want to be seen, a screen with the
general information of the interactions can be obtained (see Fig. 8). In this chart it can
be seen when the first and last visit occurred, number of different visitors, total
number of visits (visits to any page of the site, counted separately), unique visits (is
considered when a user enters, goes to different pages and disconnect; it counts a
unique visit even he/she watches many pages in the same 'session'), average of visits
in the period, average of unique visits and average of different visitors per day.
8 Conclusions and Future Work
We implemented a tool to track students’ access to our CMS from a different point of
view, the use of cookies. The tool allows teachers to control access to different parts
of the CMS, such as forum, theory, assignments, etc. Moreover, all these
characteristics can be tracked by course and a certain period of time. On the other
hand, the information gathered can be modified and presented to the user (teacher) as
needed; this is to say that there is a wide range of visualization possible, such as pie
graph, bar chart, and line graph or table format. Furthermore, the information can be
easy and nicely printed.
Regarding the technical aspects of the CMS site and in order to improve them and
be up to date, we needed the information our tool provides about: operative system,
screen resolution, browser and so on.
We also needed the identification of different visitors and to be able to identify
which paths they follow when entering the site. Furthermore, in how many clicks, that
we can also get to know when a certain page is difficult to find.
Finally, we wanted to be able to share this application with the community of
university teachers that have the same needs of getting to know what is happening
with the interactions of their students into the CMS they share and use.
We have informally used the tool as teachers and, most importantly, we made a
first usability study to assess it [20]. The teachers’ reactions were positive and the
study provided us with a list of specifications to improve its functionalities. After
these improvements are implemented, we plan to use in real situations by teachers
using our CMS.
9 Acknowledgements
This work is supported by the research project TIN2004-07568 of the Spanish

Ministry of Education and Science.
This work was supported by projects TIN2004-07568 of the Spanish Ministry of
Education and Science and S-0505/TIC/000230 of the Autonomous Region of
Madrid.
References
1. Brusilovsky, P.: ‘Adaptive hypermedia. User Modeling and User-Adapted Interaction’,

Ten Year Anniversary Issue (1/2) (2001) 87–110.
2. Castro, F., Vellido, A., Nebot, A., Minguillón, J.: ‘Detecting atypical student behaviour on
an e-learning system’. In: I National Symposium of Technologies of the Information and
the Communications in the Education, SINTICE’2005, Granada, Spain (2005) 153–160.
3. Hijón, R., Velázquez, A. ‘Steps Towards the Analysis of Students’ Activity in E-Learning
Platforms’. Proceedings of E-Learn 2005 World Conference on Corporate, Government,
Healthcare & Higher Education. Vancouver, Canada, October 24-28, 2005. Association
for the Advancement in Computing in Education. AACE Press. 738–743
4. Hijón, R., Velázquez, A. Web, Log Analysis and Surveys for Tracking University
Students. International Conference Applied Computing 2006. International Association for
Development of the Information Society. 561–564.
5. Sheard, J., Albrecht, D.W., Butbul, E.: ViSION: Visualization student interactions online.
In Treloar, A., Ellis, A., eds.: Proc. of the Eleventh Australasian World Wide Web
Conference Gold Coast, QLD, Australia, Southern Cross University, Lismore, NSW,
Australia (2005) 48–58.
6. Urquiza, J., Calvo, R., Hijón, R., Velázquez, A. Generation, Management and Publication
of XML-Based E-Questionnaires. International Conference Applied Computing 2006.
International Association for Development of the Information Society. 153–160.
7. http://www.akstat.com/website/index.php?lang=es_ES (accessed March 21, 2007)
8. http://cqcounter.com/ (accessed March 21, 2007)
9. http://www.developers4web.com (accessed March 21, 2007)
10. http://www.estadisticasgratis.com (accessed March 21, 2007)
11. http://www.estadisticas-gratis.com (accessed March 21, 2007)
12. http://www.google.com/analytics/es-ES (accessed March 21, 2007)
13. http://logdy.com (accessed March 21, 2007)
14. http://www.nedstat.es/web/nedstates.nsf/pages/index. (accessed March 21, 2007)
15. http://www.onestat.com/es/index.html (accessed March 21, 2007)
16. http://sourceforge.net/projects/contaweb/ (accessed March 21, 2007)
17. http://trancos.escet.urjc.es/urjc/ (accessed March 21, 2007)
18. http://www.xiti.com/es/ (accessed March 21, 2007)
19. http://www.weboscope.com/ (accessed March 21, 2007)
20. Hijón, R., Urquiza, F., Dominguez, F., Velazquez, A. Assessing the Usability of a
Cookies-Based Access Monitoring Tool for a CMS. The 7th IEEE International
Conference on Advance Learning Technologies (ICALT’07). Accepted.
Learning News Writing Using Emergent Collaborative
Writing Technology Wiki
Will Wai-kit Ma
Department of Journalism & Communication, Hong Kong Shue Yan University, Hong
Kong SAR, PR China
wkma@hksyu.edu
Allan Hoi-kau Yuen

Faculty of Education, The University of Hong Kong, Hong Kong SAR, PR China
hkyuen@hkucc.hku.hk
Abstract. There is always a call for studies on blended media, which is one of
the effective or pragmatic approaches to be used in particular instructional
contexts. In the learning of writing, a review of prior studies suggest that
revision as one of the key process to quality writing. However, current
approaches do not provide relevant processing capability to such an area. Until
recently, Wikis have been an emergent technology to effectively capture
collective intelligence of individual users through writing, for example, as
shown in the successful case of Wikipedia. In turn, this study attempted to
explore if Wikis technology would mediate individual learning experience, in
particular, in the process of journalistic writing. In a field study of a student-
written Wiki developed by student journalists for an online shared resources of
original news reporting, to complement traditional classroom teaching, it was
found that quality of student written work was significantly correlated to
revision behvaior, supported to prior claims. Three modes of implementation
strategies were suggested in order to boost learners’ motivation to make
relevant and quality revision to their work.
Keywords: Wiki, Writing Processes, Pedadgogy, Instructional Design
1. Introduction
In many disciplines, educational practitioners have always been finding ways on how
to motivate students to write more and to write better. Although many approaches,
including most e-learning systems, have been proposed to support teaching and
learning, they do not change much from the instructor lead design. Systems are
basically designed for course management (e.g., list of enrolment details), one-way
delivery of learning material (e.g., downloading of presentations), or dissemination of
information (e.g., announcement or calendar). On the other hand, two-way interaction
communication tools, such as chat room or discussion forums, are always low in
usage; even with administrative interventions (e.g., giving scores for participation). It
is wonder if these tools could help provide a better learning experience for the
learners. Current approaches ignore the importance of learners’ participation in the
304 Will Wai-kit Ma & Allan Hoi-kau Yuen
learning process. Although they do achieve some promising results, instructors find
they spend too much time and effort to manage a course while student learners only
come to download required files. The use of these approaches need more pedagogical
guidance before satisfactory results will be achieved. Most importantly, these current
approaches are irrelevant to the pedagogical issues in writing.
Recently, we have found that the emergent technology Wiki, which has a unique
feature to collaborative writing and community building, is worth more study to
investigate its potential to complement teaching and learning. Wiki is defined and
described as “a type of website that allows users to add, remove, or otherwise edit
and change all content very quickly and easily.”([1]). A Wiki is basically a set of
linked web pages designed as an online database, created through the incremental
development by a group of collaborating users, and Wiki is also the software in itself
used to manage the set of all the web pages created ([2]). Wikis provide a fast,
permanent and searchable platform for knowledge record that facilitates end-users
management by them. Wikis provides knowledge at a centralized access point while
decentralizes the knowledge production process to all users located far apart.
Would the employment of Wikis provide learners with a new learning experience
during the learning writing process?
Therefore, the objective of this paper is to explore Wikis as a learning medium to
learning writing, in the context of news writing. The arrangement of the paper is as
follows: In Section 2, we review the effects of learning with media. In Section 3, we
review the news writing processes. In Section 4, we evaluate the pedagogical
relevance of Wikis in learning writing. In Section 5, we suggest a model framework
to investigate the effects of Wikis as a learning medium towards learning news
writing. Then, we present the method we use in this study. The analysis of the
empirical result findings are shown in Section 7. Discussions are presented in Section
8.
2. Learning with Media
The key to use media in learning is “to focus on the pedagogical potential of the
media to enhance broader learning and teaching aims”, rather than using it to develop
a limited range of skills [3]. Supported by other empirical studies (e.g., [4]), media is
as separate or as “add-on” in schooling in recent years. The increase of quantity of
media use in the classroom in the absence of effective pedagogies is not well
embedded in the curriculum. In a review of learning with media, Kozma ([5])
distinguished media by cognitively relevant characteristics of their technologies,
symbol systems and processing capabilities (p.180). However, the most obvious
characteristic of a medium is its “technology and the primary effect of a medium’s
technology is to enable and constrain its other two capabilities: the symbol systems it
can employ and the process that can be performed with it” (p.181). As media
characteristics, and the instructional design that employed them, interacted with
learner and task characteristics, they influenced the structure of mental representations
and cognitive processes. Therefore, to assess the main effect of the medium, learner
and task should be fixed. Or else, the medium might have different effects on
Learning News Writing Using Emergent Collaborative Writing Technology Wiki 305
different learners, or different tasks characteristics. Specifically, media characteristics

affected the structure, formation, and modification of learners’ mental models (see
Figure 1).
Task
Learner
Media Mental Models

-Cognitive relevant characteristics
-Symbol systems
-Processing capabilities -Structure
-Formation
-Modification
Figure 1. Effects of Learning Media on Mental Models
3. Journalistic Writing and the Importance of Revising
Specifically, in journalistic writing, Ward ([6]) suggests several generic steps,

including: (1) To identify and find news and/or information which will attract and
interest the key audience/readers; (2) To collect all the materials needed to tell the
story/provide the information; (3) To select from the collection the best material; and
(4) To present that material as effectively as possible (p.30). Blundell ([7]) also
pointed out some similar steps in featured story writing, including gathering relevant
raw material, refining the main theme, shaping and advancing story ideas, keeping
related material together, and handling key story elements. They show that
journalistic writing has its own generally accepted procedures, which needs to be
observed and adopted, while not like those leisure writing.
Writing quality emphasizes on purpose, focus, information, and structure ([8]). To
improve quality of writing, prior studies of writing instruction recommended process
writing as an instructional intervention, where text can be developed through
planning, drafting and revising in form of a problem solving processes (e.g., [9]).
Revising is a core process in writing and has been a concern in many prior studies.
For example, Pogner ([10]) studied the text production processes including text
production (drafting a document), feedback (reader commentary) and revision
(revising the text) from a socio-cognitive approach where peer interactions in the
social context affect the text production processes in the discourse community of
engineering. That is, revising may not only a cognitive process of the author, but also
a social process involving the interactions from the peers or readers. Moreover, to
examine an effective model in writing, Roundy and Thralls ([11]) suggested writing
to include two sections: section one encourages reflection on all aspects of the
message, leading revision while section two informs error-analysis and further
revision. Therefore, revision is suggested again, as the key to effective writing.
To further understand best practices in writing, Halpern ([12]) employs an informal
survey of 125 writers in business, industry and government and identifies six key
processes: invention (to generate ideas and information for any assignments),
adaptation for audience (to adapt the writing for different audiences), clarification of
purpose (to clarify purpose of the writing), organization (to organize the material
effectively), control of voice or persona (to control the tone or voice of the
communication), and polishing (to polish the drafts of different forms). Polishing here
does not only refer to the use of better language, but also may include the revision of
any of the previous steps in purpose, organization, tone, etc. On the other hand, it is
also important to be informed on how to make revision, for example, through enough
feedback. Empirical studies found that providing feedback constitutes an integral part
of the learning process in writing (e.g., [13]) and suggested that low learners were
those who did not receive any teacher feedback and felt general uncertainty in their
work.
4. Wikis and Their Relevance to Pedagogy
What are Wikis? How are they relevant to improving journalistic writing?
Wikis, hence, have the unique design features that complement traditional teaching
on the writing processes:
(1.) Learner-centre system design: Unlike most e-learning systems that instructors
are the only persons to delivery teaching material, Wikis allow all users to add
content;
(2.) Facilitating the drafting process: Wikis require only Internet connection and a
browser, no additional applications are required. Wikis adopt only very simple
and limited (mundane design principle) tag language to make adding Wiki pages
an efficient process;
(3.) A complete support to revision in the writing process: The core concept of Wikis
is to encourage users adding content while enriching the content in the long run
(organic design principle). Wikis allow anyone to edit (revise) any pages without
restriction (flat structure design principle). Wikis record down all the edits so
that anyone can check for prior changes in order to further improving the content
(open design principle).
Meeks ([14]) referred to pedagogy as, “the ways in which an instructor designs the
material and social aspects that she, her students, and their tools inhibit as they
accomplish a curriculum” (p.1). In the particular context of journalistic writing, Wikis
seems very relevant to complement traditional teaching. It is therefore worthwhile to
further study the medium to find out the most effective way of use of the medium to
support teaching and learning.
In fact, the media industry is one of the first disciplines to grasp the full potential
of Wikis. Former reporter has implemented Wikis since 2004 for a social networking
web site to allow its users to submit, choose and rank content on news stories
(www.digg.com) ([15]). Wikis have also helped collected a multi-layered body of
knowledge and linked together disparate bits of information in one place, in an
example of advertising application ([16]). Examples show that news reporting could
be an appropriate application and promising area of implementing Wikis to manage
the relevant knowledge ([17]), for example, Wikinews (http://www.wikinews.org)
and USC Online Journalism Review Wikis (http://www.ojr.org/ojr/wiki/). Wikis have
been found employed in a number of mainstream media news sites, for example, Los
Angeles Times and its Wikitorial – invite visitors to rewrite the newspaper’s editorials
using Wikis.
5. Model Framework and Hypotheses Development
We suggest a model framework to investigate the effects of Wikis as a learning

medium towards learning writing, specifically in the context of news writing (see
Figure 2).
Task
(News
Writing)
Learner
(Writing
Self-efficacy)
Media Mental Models
(Wiki Usage; (Writing Performance)
Editing Behavior)
Figure 2. Model Framework of Writing Performance Mediated by Wikis
According to Kozma’s review on learning with media, task, learner, and the
medium are the three core factors affecting the formation, structure and modification
of learners’ mental models. Here in this study, we define our learning task as news
writing. Hence, it is the same for all learners in the study. Moreover, we measure the
writing performance as a proxy for the learning effects on learners’ mental models.
For the learners, they may have different background, domain knowledge and
confidence that may affect their final performance (e.g., [18], [19], [20]). In news
reporting process, it involves the interview of eyewitnesses, the collection of
background information, the selection and organization of evidence and material, the
drafting of the news reporting, and the editorial review on the accuracy, reliability and
completeness of the news story. In this complicated journalistic process, it is expected
that the one with a higher self-confidence in writing may have a higher chance to
accept the task as a challenge and to complete the difficult task with determination.
On the other hand, the one with a lower self-confidence in writing may more hesitate
to accept even an ordinary writing task. They will try to avoid the problem, not to face
the problem directly, spend less time on the task, finally, may not be able to complete
the task. Even if they complete the task, the quality of work does not truly reflect their
competence in the area. Therefore, we postulate,
H1: Individual learner’s writing self-efficacy would have a direct and positive
relationship to his or her writing performance.
Revision has been suggested to be an important step in the writing process. In view
of the journalistic process, including interview eyewitnesses, collecting background
information, organizing key story elements, etc., journalistic writing is no easy task. It
is logical to expect that a piece of quality news reporting cannot be made at the first
time. It probably requires many more rounds of revision to finally achieve the
reporting goals. It is therefore,
H2: Individual learner’s revision behaviour would have a direct and positive
Although the core processing capability of Wikis is the support of the editing
process, there may also be other benefits of using Wikis. Learners add content to
Wikis. Regular usage of learners and the organic grow of content in Wikis gradually
form a virtual community of Wikis users. They login the platform, they read news
headlines, news articles, they add their own, they revise their own work, and they add
links for key terms while linking independent articles together. Therefore, all these
social activities may also have effects on individual mental representations and
cognitive processes. We postulate,
H3: Individual learner’s usage behaviour would have a direct and positive
6. Methodology
6.1 Background
A student-written Wiki (named, HKNews at http://hknews.syc.edu.hk) has been setup

for the Department of Journalism & Communication at a private local university in
Hong Kong. HKNews Wiki is open to the public; however, its primary aim is to
support teaching and learning by allowing student reporters to contribute original
news reporting. Up till February 27, 2007, there were 3720 news reporting articles in
HKNews Wiki site, and a total of 47,675 edits (12.82 edits per article), with a total of
1,938,340 view counts in the whole site. There are 1,480 registered users, of which 3
(or 0.20%) are System Operators (Sysops).
6.2 Data Collection
Every Tuesday afternoon, the department arranges industrial leaders from society to
offer seminars to all the students and staff in the department. We targeted our study in
Year One students who enrolled in the course JOUR100, a compulsory introductory
course to all Year One students and they have not used HKNews Wiki before. Stage
1: Survey on writing self-efficacy - On October 9, at a class, 75 of them completed
and returned at class (One of them was absent from class). Stage 2: Content analysis
on news reporting on HKNews Wiki site – We chose October 10 seminar event and
the corresponding news reporting for analysis. This event was chosen because this
was the third seminar that student reporters already had two times of basic hands-on
experience on using Wikis and had solved most of the technical problems.
6.3 Measure
Stage 1 - Survey: For the survey, writing self-efficacy measurement scale was
adopted from Jacobs et al. (2005, ibid) and wordings were modified to reflect news
reporting tasks. Stage 2 – Content analysis: For the content analysis, we analyzed the
personal profile page, the article page and the history page. For the personal page,
student reporters included mainly the headlines of their written news articles. Some
might inserted some graphics on the page. The article was graded by their instructor,
who was a senior faculty member in the department and has more than thirty years
working experience in TV broadcasting, radio and publication industry. The score
became the final grade, ranged from 0 to 100 marks.
7. Data Analysis
Data analysis started with a review on the background and general Wikis’ usage
behaviour of the subjects. Then, the student-written Wiki pages would be analyzed to
give an understanding of how learners made use of the Wikis to write and present
their work. Lastly, the three hypotheses were examined using regression analysis.
7.1 Analysis of Subjects
75 student reporters completed the survey and returned at class where one was absent
from the class. Analysis of subjects is shown below (see Table 1).
Table 1. Descriptive Analysis of Subjects

Items Statistics Items Statistics
Gender Male: 19 (25.3%) Wiki’s Usage Mean: 4.24
Female: 56 (74.7%) Frequency (SD: 1.03)
(never to always,
1-7)
Age Mean: 19.39 Wiki’s Usage Mean: 98.05 min.
(Last Week Total) (SD: 55)
Computer Mean: 2.03 Wiki’s Usage Mean: 31.76 min.
Knowledge (SD: 0.66) (on average, every (SD: 31.39)
(very little to expert, time)
1-4)
Familiarity of Wikis Mean: 2.84
(very little to expert, (SD:0.72)
1-4)
Internal consistency of the 26-item writing self-efficacy scale is examined using

Cronbach’s alpha value. The scale shows a Cronbach’s alpha value of 0.96, exhibited
a valid internal consistency, with above the common threshold for exploratory
research ([21]). The results of the 26-item writing self-efficacy survey are shown
below (see Table 2).
Table 2. Descriptive Analysis of Writing Self-efficacy

Writing Self-efficacy 26-items: Mean: 56.76 (SD: 12.987) to 72.25 (SD: 16.206)
(Cronbach’s α=0.96) Composite: Mean: 63.82 (SD: 9.81); Min.: 13.85, Max.: 81.92
7.2 Analysis of Student-written Wiki Pages
Out of the 75 student reporters, 57 of them had included news headline and the
news reporting on the Oct 10, 2006 event. An analysis of these 57 articles is listed
below (see Table 3).
Table 3. Descriptive Analysis of News Reporting of Oct 10 Event on HKNews Wiki

Formatting Analysis Formatting Analysis
Author’s Name No: 30 (52.6%); Bulleted list Nil
Yes: 27 (47.4%)
Length (no. of Min.: 160; Max. 1040 One idea per No: 8 (14%)
words) Mean: 423.86 (SD: 148.15) paragraph Yes: 49 (86%)
Highlighted Nil Inverted No: 11 (19.3%)
keywords pyramid style Yes: 46 (80.7%)
Sub-headings No: 23 (40.4%) Photos, Nil
Yes: 34 (59.6%) Reference
sources
7.3 Analysis of Editing Log of Oct 10 News Reporting
Analysis of the news reports found that, 14 (24.6%) student reporters uploaded the
news reporting onto the HKNews Wiki site without any further edits while the others
edited ranged from 1 time to 10 times. On average, there were 2 times of editing
updates for each news report (SD: 2.10); (2) Edits time span: Moreover, there is also a
time span between the first published version to the final edited version. Two-third
(N=38, 66.7%) of the news reports finished all the editing process on the same day,
while the remaining 19 (33.3%) ranged from 1 to 21 days. On average, it takes 2.11
days (SD: 4.17) for student reporters to feel comfortable about the report and not to
edit any more; (3) Contributors’ identity: Wiki users may register and login the
system to both browse around and making changes in the Wiki environment. They
might also not login but were capable to do all the steps; however, the IP address of
the computer would be listed instead. It was found that the mean number of account
login in editing the news reports was 1.95 (SD: 2.15); while IP address editing was
1.04 (SD: 1.55); (4) Timeliness of news reports publication: The timeliness of the
news reports can be measured by the time span between the date of the event and the
date of the report published on Wiki. Analysis of the time span of the 58 news reports
found that it takes an average of 3.46 days (SD: 4.13), which means that there is a
lagged time for the news appeared for the readers. One third of the reports appeared
on the same day or 1 day after the event (N=21, 36.8%); another one quarter
published two days after the event (N=15, 26.3%); the remaining (N=21, 36.8%)
published from 3 days to 23 days after the event. An analysis of this editing log is
listed below (see Table 4).
Table 4. Descriptive Analysis of Editing Log of Oct 10 News Reporting on HKNews Wiki
Editing Log Analysis
(1) Number of edits No edits: 14 (24.6%)
N.B. Total number of revision, except first posting Max.: 10, Min.: 1
Mean: 2 (SD: 2.10)
(2) Edits time span 0 days: 38 (66.7%)
N.B. Time span between the first edit through the final Max.: 21 days, Min.: 1 day
edit Mean: 2.11 days (SD: 4.17)
(3) Contributors’ identity Account login: 1.95 times (SD: 2.15)
N.B. Registered and use account login vs. no login but IP: 1.04 (SD: 1.55)
shown IP address of editor’s computer
(4) Timeliness of news reports publication 0-1 day: 21 (36.8%)
N.B. time span between the date of the event and the 2 days: 15 (26.3%)
date of the report published on Wiki 3-23 days: 21 (36.8%)
Mean: 3.46 days (SD: 4.13)
7.4 Regression Analysis
The writing performance was examined by the grades. Linear regression analysis
found that total number of edits (β=0.462, p<0.001) was significant factor that
predicted the score of the news reporting (supporting H2). The model explained 22.6
per cent of the variance of the score, significant at p<0.005 (Adjusted R-square:
0.178). However, Wiki’s knowledge and writing self-efficacy was not found
significant to predicting writing performance (H1, H3, not supported).
8. Discussions
In this paper, we analyzed the shortcomings of current approaches to support teaching

and learning: instructor lead design and irrelevant to learning writing. We then
examine the potential of Wikis as a learning medium in the context of journalistic
writing. Supported by prior studies in writing literature, the unique design feature of
Wikis in supporting revision seem relevant and important to providing learners a
unique experience in practicing and learning journalistic writing. Empirical results of
this study found that number of editing of a news reporting is significantly correlated
to the writing performance, supporting the theoretical assumption.
On the other hand, it is important to discuss more of the various instructional
designs to encourage more revision and discussion of the issues under different
circumstances where revision takes place. We hereby suggest the following
framework in the implementation of Wikis to support learning writing (see Figure 3).
Individual Group Large Scale

Authorship Authorship Collaboration
Figure 3. Fusion Modes of Implementation Strategies for Wikis
While revising is the key, it is suggested that the three writing modes: (1)
Individual authorship; (2) Group authorship; and (3) Large scale collaboration, be
introduced in the curriculum, either at the same time or throughout different
semesters, in order for a learner to fully benefit from using Wikis to create a unique
learning experience.
(1.) Individual Authorship: This is important to keep a learner the responsibility to
his or her own learning as he or she owns the writing piece. We need to design a
mechanism to motivate the learner to self-reflection on his or her writing and do
the editing on his or her own. For example, for a writing task, the instructor may
suggest a schedule for all learners so that they draft the outline the first week,
they finish the first draft the second week, and they make three additional
revisions on the three following weeks. This is actually a practice in composition
writing and is the basis for process writing. The key is to develop a regular
pattern so that learners will continue the revision process.
(2.) Group Authorship: Learners work in groups may have a chance for an intense
interaction among peers. Form learners into small groups, request them to
complete a writing task and develop a mechanism for them peer review each
others work. Wikis provide a platform transparent to all. Anyone can edit any
page. Therefore, Wikis make it an efficient tool for a group of people working
together. In this instructional design, there is a diffuse responsibility on who
owns a writing piece. Learners work together and hence, learners have a chance
to learn from a different perspective on how to edit a writing piece. Wikis
provide a history log so that anyone can compare any two prior versions. This
social interaction expands the cognitive process of any individual learner alone.
(3.) Large Scale Collaboration: This design does not care any more about
authorship. The focus is on the writing piece. Anyone can create a new writing
piece while any other one can edit and enrich the content. Therefore, learners do
not write because of the ownership of the writing piece, but are based on the
topic that they are interested. A learner can learn as much as he or she wishes by
his or her commitment to a specific topic. On the one hand, a learner learns by
his or her own study and writing practice in the topic; on the other hand, he or
she also learns from other learners interested in the topic.
There are several limitations in the study. The domain knowledge about journalistic
writing is contextualized and specific; the results may not directly be generalized into
other disciplines. Further studies in other writing context may add better
understanding of the issue. Moreover, the sample size is small and the time frame is
limited, these are also the limitations of the study. In future research, we can study a
bigger sample size of subjects, choosing several instead of one writing piece over a
period of time that would probably provide us with a rich explanation to the learning
process. This is especially true for a learner to fully reflect his or her mental
representation into measurable learning output.
9. Conclusion and Future Works
In this paper, we analyzed the design features of Wikis, specifically the editing
capability, in relation to the writing process using a field research on a group of
student journalists on learning journalistic writing. We examined the usage pattern
and found significant relationship between revision and writing performance. This
result is supported by prior studies in writing while showing that Wikis as an effective
learning medium relevant to the learning of writing.
However, some problems still need to be solved, such as how to motivate learners
from revising their work, how to improve peer interactions to provide basis for quality
revision, or how individuals react differently within the medium. We will continue
our research on these problems and provide implementation guidance for educational
practitioners fully utilize Wikis as an effective learning medium.
References
[1] Wikipedia:Wiki, Retrieved March 18, 2007, from http://en.wikipedia.org/wiki/Wiki,

2007 (March 18).
[2] Leuf, B. and Cunningham, W. The Wiki Way: Collaboration and Sharing on the Internet,
Addison-Wesley, Reading, MA, 2001.
[3] Gaskell, A. "Editorial," Open Learning (20:2), 2005, pp. 109-111.
[4] Condie, R. "ICT and Learning: Scottish Schools and Pupils," Proceedings of the Heads of
E-learning Forum (HELF), University of Strathclyde, 2005.
[5] Kozma, R.B. "Learning with Media," Review of Educational Research (61:2), 1991, pp.
179-221.
[6] Ward, M. Journalism Online, Focal Press, Oxford, UK, 2002.
[7] Blundell, W.E. The Art and Craft of Feature Writing, Plume, New York, 1988.
[8] Davis, K.W. The McGraw-Hill 36-Hour Course in Business Writing and
Communication: Manage Your Writing, McGraw-Hill, New York, 2005.
[9] Gersten, R. and Baker, S. "Teaching Expressive Writing to Students with Learning
Disabilities: A Meta-analysis," Elementary School Journal (101), 2001, pp. 251-272.
[10] Pogner, K.-H. "Writing and Interacting in the Discourse Community of Engineering,"
Journal of Pragmatics (35:6), 2003, pp. 855-867.
[11] Roundy, N. and Thralls, C. "Modelling the Communication Context: A Procedure for
Revision and Evaluation in Business Writing," Journal of Business Communication
(20:3), 1983, pp. 27-46.
[12] Halpern, J.W. "What Should We Be Teaching Students in Business Writing?" Journal of
Business Communication (18:3), 1981, pp. 39-53.
[13] Jacobs, G., Opdenacker, L. and Van Waes, L. "A Multilanguage Online Writing Centre
for Professional Communication: Development and Testing," Business Communication
Quarterly (68:1), 2005, pp. 8-22.
[14] Meeks, M.G. "Wireless Laptop Classrooms: Sketching Social and Material Spaces,
Kairos, 9(1)," (20 07: March 12), 2004.
[15] Bulik, B.S. "Media Morph: Digg.com," Advertising Age (77:8), 2006, pp. 19.
[16] Oser, K. and Kerwin, A.M. "Media Morph: Wiki," Advertising Age (76:48), 2005, pp.
38.
[17] Dorroh, J. "Wiki: Don't Lost That Number," American Journalism Review (27:4), 2005,
pp. 50-51.
[18] Bazerman, C. "Physicists Reading Physics," Written Communication (2:1), 1985, pp. 3-
23.
[19] Bandura, A. "Self-efficacy: Toward a Unifying Theory of Behavioural Change,"
Psychological Review (84:2), 1977, pp. 191-215.
[20] Bandura, A. Social Foundations of Thought and Action, Prentice-Hall, Englewood Cliffs,
NJ, 1986.
[21] Nunnally, J.C. and Bernstein, I.H. Psychometric Theory, McGraw-Hill, NY, 1994.
Description Patterns
in Learning Design Authoring Systems
Lei Xu1, Li Zheng1, Jing Liu1, Yintao Liu1, Fang Yang2
1 Department of Computer Science and Technology, Tsinghua University,

Beijing, China
2 Department of Foreign Languages, Tsinghua University,
Beijing, China
Abstract. Learning design authoring is one of the core functions of a learning

management system. As the types of elements to edit inside learning design
vary, it can be time-consuming to supply the environment for editing each type
of element individually. This paper explores the similarities among the editing
of these different types of elements, and proposes an approach to accelerate the
development of such a complicated system and to achieve better adaptation for
changes of concept model or requirements. The basic idea is to unify the editing
environments for different types of elements, by introducing description
patterns. A demonstrative implementation utilizing AJAX technology is
presented and its potential in authoring systems for other forms of e-learning
material, such as content package proves to be feasible and practical.
Keywords: Learning Design, LMS, XML, Description Pattern.
1 Introduction
1.1 Emerging of learning design within e-learning research
In the development of e-learning industry, the concept of learning design has become
increasingly a popular issue.
Similar to content package and simple sequencing, learning design can be an
approach to organize various educational resources. But it does more than that.
Besides physical materials, it takes activities and roles participating in the learning
process into account. Therefore it concerns not just simplex activities associated with
reading materials but also group work like discussion, role plays and etc. Being
capable of organizing learning scenarios like the ones within collaborative learning, it
gets learners effectively involved in the learning process.
Although the idea of ‘Design of learning’ has already existed implicitly in the
traditional teaching context, when it comes to the digital and web environment, new
challenges occur in the application of an idea which is not so fresh.
In reality, the design of learning lies in the lecture notes of an instructor, and
maybe partially in his mind. But e-learning systems cannot utilize such a form of
design. It is thus necessary to transform the teaching plan to a form which can be
316 Lei Xu et al.
easily understood by computers. That is why methods of binding the design of

learning to digital content are proposed, and one of them is the IMS 1 -LD 2
specification, which has been commonly accepted by the e-learning research
community.
1.2 Digital binding of learning design
To some extent, the IMS-LD specification[5] can be recognized as a language which

describes how different roles perform activities within environments(including
learning objects and services), and how these three categories of building blocks are
coordinated into a learning flow, within the method element which contains both play,
act, and role-parts[6]. Such an approach to separate building block repositories and
the flow control mechanism, make the consisting elements both reusable and sharable.
E-learning systems that supporting the IMS-LD specification can easily exchange
learning designs in different levels.
The undergoing project of the national standards on learning design also takes the
IMS-LD specification as the major reference. During our research, we also keep the
conformance to IMS-LD as the top agenda.
The IMS-LD way of binding is tree-structured, preferably using XML. So the
essential part of job of a typical learning design authoring system is to enable the
editing of such a tree structure, which will be analyzed in details in subsequent
sections. The complexity of such a task is mainly determined by the following factors:
1. number of element types within the tree;
2. numbers of attributes possessed by each type of element;
3. numbers of attribute types possessed by each type of element;
4. number of dependency links among attributes and other elements.
As the four factors above increases, the entire job can drive the developers crazy,
because there can be numerous types of attributes to take care of, and countless
property sheets to build up, let alone the potential changes in the concept model,
which can result in major changes in the authoring system which is even harder to
accomplish.
To avoid such inconveniences, we propose the description patterns to be used which
can save most of the trouble.
1.3 What is Description Pattern
A description pattern generally supplies information on one of the attributes of its

target. By obtaining a set of necessary description patterns of an element, an authoring
1 Abbr. for Instructional Management Systems, now commonly mentioned as the IMS Global
Learning Consortium is a non-profit standards organization concerned with establishing
interoperability for learning systems and learning content and the enterprise integration of
these capabilities. Official site: http://www.imsglobal.org/
2 IMS Learning Design is a specification for a metalanguage which enables the modeling of
learning processes. The specification is maintained by IMS Global Learning Consortium.
system will know how to present the property sheet for it, allowing the author to view
and alter the value of it.
For example, for a Role-part element inside a learning design we may find three
description patterns telling us that
1. it has an attribute named ‘title’, of string type, and is optional, and
2. it references a role inside the learning design, named ‘role-reference’, and is
a must, and
3. it must refer to a component, which can be a learning activity, or a support
activity, or an activity structure, or an environment.
So a description pattern is simply a way of describing versatile attributes of an
element. As the types of attributes of the target element vary, there should be various
patterns to describe them. Elements inside a learning design may be quite different
and complicated in different aspects, but they do share a large collection of common
types of attributes, which is why it is possible to simplify and accelerate the
development of the authoring system using description patterns.
2 Related Work
As for now, there are several Learning Design authoring tools, like RELOAD3 Editor,
LAMS4. Reload Editor provides users an appropriate user interface model to allow
easy creation of learning designs [7], and is quite a convenient LD packaging tool that
runs as a Eclipse based local application. LAMS includes innovative design features
that put it at the forefront of current tools for activity management [7], utilizing
Adobe® Flash® technology.
These projects focus on how to build an application that provides better support for
IMS-LD and an intuitive and friendly user interface. The purpose of our work is not
just develop yet another Learning Design authoring tool, which would appears
unnecessary, instead, we explore ways to accelerate the developing process of such
tools as it is a commonly accepted fact that “it is not trivial to provide full support” [9]
for IMS Learning Design. A Learning Design authoring system acts as the test
environment for the approaches we have proposed and the potentials of those
approaches are not just limited to Learning Design authoring systems.
3 Use Case Analysis with respect to Description Pattern
This section explores the essential functions of a learning design authoring system, in
the context of which the description patterns are utilized. The basic use cases of a
3
Reusable eLearning Object Authoring and Delivery, a project that develops tools to support
the learning technology interoperability specifications such as IMS and SCORM, more
details can be found at http://www.reload.ac.uk.
4
Learning Activity Management System, more information can be found at
www.lamsinternational.com.
318 Lei Xu et al.
typical learning design authoring environment can be roughly divided into the
following 3 stages:
3.1 Tree structure navigation
This means the expanding and collapsing of tree nodes. So the required description
should indicate if a requested tree node, namely a concerned learning design element
is expandable and if so, what the child nodes are.
Therefore, a type of description pattern is necessary to give the exact information
about descendants of the target. And for that matter, we call it Descendants
Description Pattern.
3.2 Tree structure editing
Normally a learning design authoring environment is expected to allow users to modify the
structure of the tree, which includes four major actions: creating new child nodes under a parent
node, moving an existing node elsewhere, duplicating a node, and deleting a node. Therefore,
to supply for the information we propose four types of description patterns:
1. Descendant Creation Description Pattern

This pattern is expected to tell the authoring system what types of elements
can be created as the child node for the target node. So the author can only
select from a certain collection of types to create new nodes.
2. Relocation Description Pattern
This pattern is expected to tell if the target node can be moved up or down,
and if it can be detached from its current parent and attached to a new parent.
It determines if the author’s moving action can be performed successfully.
3. Duplication Description Pattern
This pattern is expected to tell the system if the target can be duplicated, and
if there is a unique identification issue to address, usually by altering certain
attribute of the target to distinguish it from the original one.
4. Deletion Description Pattern
This pattern is expected to tell the system if the target node can be deleted. If
so, how the deletion might affect other remaining elements.
For example, if a learning activity is deleted from the Activities repository; all its
references go invalid. So this type of description should be able to perform certain
post-deletion actions to assure the consistency of the learning design structure.
3.3 Tree node attributes editing
In this context, the required descriptions can get much more complicated as the types
of attributes of all the elements vary. In spite of this, there are still some common
fields that all these patterns share:
1. The target element it describes;
2. The method to get the attribute value;

3. The method to set the attribute value;
4. Whether the attribute is read-only;
5. What is the default value for the attribute;
6. The title of the attribute, which is displayed within the authoring interface.
Then we explore the frequently used patterns in a simple-to-complex manner.
3.3.1 Simple field description

This category includes the following attribute description patterns:
Number (Integer, float point, or percentage)
This pattern indicates the corresponding attribute is a number, and its value should
fit certain constraints, like minimum value, maximum value, precision and etc. So
such a pattern is preferred to have value validation capabilities.
Format-free text or string
This pattern doesn’t have many constraints, if any, it could be no more than one
like ‘maximum characters’ and is easy to ensure. One example is the attribute ‘title’.
Special format string
Like phone number, URI, date and duration, text should conform to certain regular
expression, so such patterns are also expected to have validation capabilities. Besides,
it should be able to convert string to an appropriate object, for example, converting a
date string to a Date object, as the target element object may require.
3.3.2 Static enumeration

This category includes:
Boolean
This is almost the same as numbers, as only two values are possible (true or false),
but may require different presentation approach, which could be radio button while
the ones above normally lies in a text field.
Vocabulary
This is some special enumeration specified for certain type of element attribute.
For example, the ‘structure type’ attribute of an activity structure can be ‘select’ or
‘sequence’ only. Such a pattern should be assigned the valid value set (vocabulary)
according to the target type at runtime.
3.3.3 Dynamic collection

This is the most complicated attribute description pattern which is meant to handle
the component reference situation. As mentioned above, the mechanism of reference
is widely used within the learning design information model. It works like this: the
actual component is stored in a certain repository and once needed, it is only
necessary to supply its identity (usually a unique identifier), not a clone of the
component itself. So what component can be referenced depends on the accepted
types and the contents of the repository, which is determined by the learning design
itself, not by some statically predefined vocabulary.
In this case, the pattern should be able to supply the information on:
1. The types of elements can be referenced, and
2. The possible value set for a chosen type, which is determined at runtime.
320 Lei Xu et al.
Take the role-part element for instance: a first pattern tells that the role-part
references a role, the corresponding description pattern should tell the authoring
system only one type of element can be references - role, and the value set for the
‘role’ category is the elements stored under the ‘roles’ element, with a tag of
‘Learner’ or ‘Staff’; a second one may tells that it refers to a component that
represents the behavior of the role, the accepted types are learning-activity, support-
activity, activity structure, unit-of-learning and environment, and for each type the
pattern is expected to supply the possible value set.
4 Interpreting and Presenting Description Patterns
This section introduces how the classification of patterns will be used to build the
authoring environment, especially the property sheets for various elements inside a
learning design.
All the description patterns are supposed to be stored in the form of configurations,
and will be loaded to the system for runtime retrieval.
Using the Descendants Description Pattern, we can easily enable the tree-structure
navigation, which is the premise of all the editing actions.
With Descendant Creation Description Pattern, Relocation Description Pattern,
Duplication Description Pattern and Deletion Description Pattern, the authoring
system can guide the author throughout the process of adding child nodes for a parent,
moving, duplicating or deleting an existing node.
The attribute description patterns, which contain a lot of subcategories, are
intended for the authoring system to build the attribute sheet for a learning design
element which the author wishes to view and edit. The process is shown in figure 1.
Figure 1 Generating attribute sheet for an element

For a requested element, the system first obtains the description patterns for it
(arrown), and then it builds the input field for each pattern, and assembles all the
fields to form the entire property sheet (arrowo). The most difficult job is to create an
appropriate input field for each description pattern.
For example, to present a text attribute like ‘identifier’ to the author, the system
may dynamically create a text filed in the property sheet with the title illustrating
what the attribute is. And for a Boolean description pattern, it creates a group of two
radio buttons or a selection box with two options.
In the case of a component reference, it shows the current referenced value, which
is followed by a set of controls to alter the value: a selection box holding all the
acceptable types to reference, and a query button. After the author specifies one
category and presses the query button, the system retrieves all the possible
components fitting into the requested category, and displays them in a list for
selection. In this way, the author is able to change the component references.
After the author modifies some of the attributes, the system passes each modified
attribute value to the corresponding description pattern (arrowp) and tells them to set
the new values (arrowq). Data flow in this process is reversed in contrast to the
rendering process. And if some patterns refuse the new values for validation or
consistency reasons, the system will explain to the author why the updating has failed
(arrowr).
5 Benefits of Description Patterns
By introducing description patterns, the learning design authoring system is not

concerned about what type the requested element belongs to, instead all it needs to do
is retrieve all the necessary description patterns, render these patterns to the interface
and pass the input from the author to these patterns to update the attributes of the
element. To the authoring system, all elements become unified and separated
processing for each type of element is no longer necessary. Furthermore, the
description pattern approach has more advantages, two of which are as follows:
5.1 Better Adaptation for changes
As the description patterns wraps up all the attributes of elements, the influence that
potential changes in the future can produce on the editing environment is greatly
reduced. Here we take two major possible changes to consideration.
5.1.1 Information Model Adjustment

It is possible that the learning design concept may change in the future, or the
specification itself get improved or extended, which would result in the changes in the
binding tree structure: new types of element added, attributes of existing element
changed, etc. To assure the authoring system still function normally, it is only
necessary to add description configuration for new type of element, or modify the
configuration for changed attributes.
322 Lei Xu et al.
5.1.2 Multiple Authoring Policy Control

Although the information model of learning design is relatively stable, the authoring
policies required by users may vary a lot. For example, some system administrators
may think that it is not a good idea to allow authors to modify the identifier of an
element, so he requires the identifier attributes to be removed from view. To satisfy
this, we just need modify the description pattern configuration so that all descriptions
about identifiers are removed. Then the identifier attribute would disappear from the
authoring interface, without modifying the authoring environment code.
An author may argue that the selection box holding the type of role element a role-
part can refer to should be ‘Learner’ and ‘Staff’ rather than one whole category ‘role’.
To make it work, we just need to modify the description configuration for the ‘role
reference’ attribute to accept two categories: Learner and Staff, and the value set for
the ‘Learner’ category is the roles with the ‘Learner’ tag stored inside the underlying
learning design, and for the ‘Staff’ category, the ones tagged ‘Staff’. Still the
authoring environment code is left untouched.
As we can see, using description patterns make it easy to achieve customizable
authoring environment per author that best fits his habits. And if the author does not
wish his learning design to be modified by others, we can assign a complete read-only
description pattern configuration to his works once viewed by others. The description
patterns provide us a flexible and easy way to control multiple authoring policies.
5.2 Reusability
The mechanism of description patterns is not just intended for learning design
authoring; instead it provides us with a way of modeling an authoring framework for
XML-bound educational resources. Because the use cases we have analyzed above is
not only dedicated to learning design itself, other forms of educational resources that
has a conceptual tree structure can also take most of the convenience offered by
description patterns The only extra effort needed is that some new types of attribute
description patterns may need to be defined, and the authoring system needs to be
extended to be aware of how to render these new patterns. But the basic logic of
description and presentation totally fits. As the follow-up implementation proves, it is
easy to be ported to content package or LOM editing environment.
6 Implementation and Application
This section focuses on how we implemented a learning design authoring system

using description patterns and how it goes after the actual system is put into use.
6.1 System Framework Overview

The learning design authoring environment is designed as part of a LMS, which
adopts a layered architecture [1](figure 2).
Figure 2 Generating attribute sheet for an element

The first part that the learning design authoring system concerns is how the design
described by XML is parsed and converted to a series of objects that represent
different element within the learning design information model. Such a mapping
process happens between the Public Service Layer and the Educational Service Layer,
or more precisely, the XML and the Learning Design module. Though we do not care
how it works, the objects obtained are the targets that our description patterns
working on.
After the learning design object are ready to use, the authoring environment which
lies in the Educational Application Layer, get all the description patterns for each
object the author may view or edit. These patterns, act as the bridge between the
presentation interface and the background objects. Information on the tree structure
and the attribute set of an element pass on in the form of description pattern to tell the
interface how the learning design environment is rendered; and after the author makes
adjustment to the design, the patterns fetch the changes and apply them to the
corresponding objects so that the author’s thoughts can be persistent in the system
after the objects are formatted and written back into XML, as shown in figure 2. The
following two major modules accomplish this job.
6.2 Java back-end

The task is mainly produce proper description patterns for different types of learning
design elements. For the sake of flexibility and convenience, configurations for these
patterns and elements are stored in XML format. Figure 3 shows a simple example of
how a description pattern is specified by means of XML. In this case, the class
324 Lei Xu et al.
LearningDesign has an attribute called identifier, which belongs to the type 'identifier-
text' and shall be randomly generated at runtime.
Figure 3 Simple attribute configuration specified in XML

The dynamic-object case is shown in figure 4, as for a role-part, a role need to be
referenced and the role should be selected out of all the roles stored inside the
containing learning design. The options here specify the possible categories where the
dynamic objects come from, and for the role reference, they are learner and staff.
Figure 4 Dynamic-object attribute by XML configuration

Thanks to the Java reflection mechanism, the configuration can be well understood
by the system, and the required description pattern is easily generated by the name of
the Class the actual object belongs to, and the descriptions can easily get and set fields
inside the objects. The job of parsing the configuration and forming corresponding
Java objects is mainly done by Caster XML, which can marshal5 almost any "bean-
like" Java Object to and from XML [8].
6.3 AJAX front-end
As the interface presented to the author is determined by the active description pattern,
and all controls for input and output are dynamically created, instead of interpreting a
pile of server pages with static fields in them. Once there is an action performed in the
front-end, the back-end receives the request and generates all necessary description
patterns and sends them back in a certain data format rather than web pages. After
receiving and parsing the data, the front-end dynamically changes the user interface
according to the description. The back-end is not involved at all. As the user agent is a
web browser, this is just a good use case of the AJAX technology, which features in
5
It is the action of converting a stream (sequence of bytes) of data to and from an Object [8].
asynchronous client-server communication and dynamic web interface control by

JavaScript and CSS. XML is used to transfer information on description patterns, and
that is the common language both the back-end and front-end speaks. An example of
such XML documents is shown in figure 5.
Figure 5 The content of XML on description patterns for a role

On receiving the responding XML, the front-end will render the user interface (figure
6) to allow the author modify the attributes.
Figure 6 Interface rendered for Role editing according to description response

The case for dynamic object is a bit more complicated, first the interface displays
all the valid categories, and then query on a certain category should be enabled.
Figure 7 shows how it works when selecting environment reference for an activity.
Figure 7 Dynamic object interface for environment reference selection
6.4 Implanting into other authoring systems

As we expected, after the learning design authoring system is implemented, a similar
one for editing Content Package is implemented by means of description patterns. As
most of jobs are accomplished in the learning design system development stage, the
latter one was built and set up rapidly, which shows the amazing power of the
326 Lei Xu et al.
description patterns in building similar authoring systems for XML-bound educational

resources. The case remains while applying it to LOM editing environment.
6.5 Application
Currently, our LMS with support for LOM, Content Package and Learning Design, is
serving as a on-line teaching platform for our department, including C++/C
programming, Webpage Design, and is hopefully to be integrated to the whole on line
learning platform of Tsinghua University, as the system provides the teachers with
more effective ways to guide the students in the learning process, and enables more
interactions outside the physical classroom.
Conclusion and Future Work

As the elements within a learning design vary greatly, it is difficult to supply an
authoring system for them. But by means of description patterns, we can efficiently
perform repetitive tasks such as attribute retrieval and modification. These patterns
encapsulate the differences between various types of elements and make them look
the same to the rendering front-end. This approach of design pattern proves to be
feasible in learning management system. However, it can hardly describe complicated
dependencies among elements, or attributes containing logical conditions. In such
cases, specialized authoring environment should be considered to balance
commonality abstraction and difference handling. This issue will be addressed in our
follow-up studies.
References
[1] Wang, X., Zheng, L., Yang, F.(2004), “An Implementation of Learning Objects
Management System”, Advances in Web-Based Learning – ICWL 2004 Lecture Notes in
Computer Science, Vol. 3143, (2004)393-399.
[2] Liu Jing, Zheng Li, Yang Fang (2005), “Information management in e-learning system”,
Advances in Web-Age Information Management. 6th International Conference, WAIM
2005. Proceedings (Lecture Notes in Computer Science Vol.3739) 275-83 2005
[3] Alistair Inglis (2005), “Using conceptual mapping as a tool in the process of engineering
education program design”, Journal of Learning Design, Vol. 1 No. 1, 45-55
[4] James Dalziel(2003), “IMPLEMENTING LEARNING DESIGN: THE LEARNING
ACTIVITY MANAGEMENT SYSTEM (LAMS)”, ASCILITE 2003
[5] IMS (2003). IMS Learning Design v1.0 Final Specification, retrieved March 28th, 2006
from http://www.imsglobal.org/learningdesign
[6] Ann Jeffery and Sarah Currier (2003), “What Is IMS Learning Design?” retrieved January
14th, 2007, from http://www.cetis.ac.uk/lib/media/WhatIsLD_web.pdf
[7] Sandy Britain (2004), “A Review of Learning Design: Concept, Specifications and Tools”,
retrieved February 12th, 2007, from
http://www.jisc.ac.uk/uploaded_documents/ACF83C.doc
[8] Castor XML documentation: retrieved September 15th, 2006 from http://www.castor.org/
[9] CoperCore documentation: retrieved April 9th, 2007 from
http://coppercore.sourceforge.net/
M-Learning: A Pedagogical and Technological Model
for Language Learning on Mobile Phones
Yannick Jolliet
HEC-INFORGE, University of Lausanne, Switzerland

yannick.jolliet@unil.ch
Abstract. The mobile learning (m-Learning) is a rapidly evolving area which

remains to be further explored, in particular about the possibilities offered by
mobile phones which have now become commodities. Is it possible to leverage
this fantastic base of 2 billion mini-computers people are regularly carrying
with them to allow a mobile learning experience, anywhere, anytime? We
answer the question on the basis of a “use case” about learning of a language
(that is particularly well suited to this medium), and we develop an approach for
such a service. We highlight not only the feasibility but mainly the pertinence
and the added-value of such an approach, this from a pedagogical as well as
from a technical standpoint. Finally, the development perspectives of mobile
technologies are impressive and encourage us further studying and designing
such personalized m-Learning solutions.
Keywords: mobile learning, m-Learning, e-Learning, LMS, IVR, mobile phone,
pedagogy, language learning, use case.
INTRODUCTION
The objective of this article is to present a pedagogical and technological approach to

support language learning via a mobile phone. To the opposite of other studies using
advanced devices such as « PDAs – Personal Digital Assistants » or « Tablet PCs »,
we will focus here on technologies widely available throughout the world
(specifically second generation of mobile phones), with the perspective to allow m
Learning for “everybody”.
We start by positioning the context of mobile-learning (m-Learning) in the general
field of flexible learning. We then define typical characteristics of a target market for
an m-Learning service focused on language learning.
Finally, we will analyse how such a service could be structured and setup, from a
pedagogical as well as from a technological stand point.
328 Yannick Jolliet
CONTEXT
Definition and Characteristics of m-Learning

In order to correctly position m-Learning versus other various flexible learning
approaches, we refer to the Brown model (figure 1). It positions m-Learning as a
distant learning method via an electronic media, highlighting its mobile specificity
which distinguishes it from online e-Learning
It clearly appears that m-Learning is complementary to other learning approaches,
whether they are electronic or not. It is in this perspective that we position our
reflexion: how to define an added-valued learning service, complementary – and not
substitutive – to other learning approaches.
m-Learning distinguishes itself by some characteristics and key advantages,
namely its ubiquity and possibilities of learning anywhere/anytime (« just-in-time
delivery »). Classical mobile phones (not « PDAs) have some great strength: they are
typically common and relatively cheap devices, small, light and with a long
autonomy. However, in parallel they do also suffer from some serious weaknesses
and limitations: typically very small screens and difficulty to input data for example.
Figure 1: Sub-components of flexible learning, Brown (2004)
Context and Approach

If multiple studies have been conducted about the usage of mobile technologies to
support or foster learning, the majority are focused on using complex and expensive
devices such as PDAs or Smartphones (phones with PDA features). However, it is
known these devices are also perceived as dedicated to professionals and difficult to
use (Attewell 2004).
On the other side, rare are the studies focusing on learning via standard mobile
phones, especially outside of Asia. Device-related constraints highlighted above are
often cited to justify the lack of pertinence or viability of such approaches. In
addition, studies performed are closer to traditional e-Learning but applied on small
screen, which probably explains also the low success rate.
M-Learning: A Pedagogical and Technological Model 329
From our perspective, we believe “standard” mobile phones are such widely
available devices they represent a fantastic opportunity, and therefore it is important
to rethink our pedagogical strategies and our usage of technology with its inherent
constraints, in order to offer a new form of m-Learning service. This way, and
through time, m-Learning will tend to gradually distinguish itself from conventional e
Learning.
Our approach will therefore be not to simply adapt pedagogical content to the
telephone medium, but rather to use mobile phones’ possibilities -in synergy with
other technologies- to develop an effective and innovative mobile learning
experience.
m-Learning Potential
There are today more than 2bio mobile phones throughout the world! Never a
technology penetrated the world so rapidly, widely and deeply. Rising speed is
increasing, mainly lead now by China and India but also by Brazil. « Informa »
predicts 2.7bio mobile phones in 2010, with 50% in Asia-Pacific region (source
ZDNet Research, see Reference).
The mobile is –and will become more and more– a commodity and a mass-media.
Below, we will focus on those rapidly growing markets in Asia-Pacific, but our
reflexion applies to more developed markets.
We have seen multiple telephony generations (2G, 2.5G, 3G), each offering a
higher information flow rate, but also leading to the creation to more and more rich
and complex phones. Studies show that by 2010, 50% of users will still be using 2G
technology, and only 25% will have upgraded to 3G (source ZDNet Research, cited).
Figure 2: Mobile telephony evolution in Asian markets (sources: multiple)

Figure 2 represents the evolution of theses multiple dimensions for the fast-
growing Asia-Pacific market: market size (diameter), technology maturity (2G, 3G)
and device type. Note that even by 2009, PDAs & Smartphones share remains
minimal compared to classical mobiles phones.
330 Yannick Jolliet
All these factors drive us to think there is an important technical foundation,

rapidly growing, a real platform, for a mass market m-Learning, at the condition of
leveraging classical technologies, meaning 2G mobile phones.
In parallel of these technical advancements, we see also a profound society
mutation related to how we learn. We hear more and more about a « learning society
» and real-time learning, through people’s life but also through the day, anywhere,
anytime.
An important investment in personal development and training has become
necessary to remain competitive in the global knowledge economy. m-Learning can
naturally bring interesting solutions to answer part of this need, allowing to use “lost”
moments and to choose where and when to learn something.
A PERTINENT M-LEARNING OFFERING
Learning needs obviously differ, but the context of these rapidly growing regions
which ambition to become key players in the worldwide economy, encourages
students and managers to learn foreign languages, in particular English.
Actually, learning English is on one side a key factor for professional success and
an important differentiator criterion for individuals, and on the other side a mean for a
country to integrate the global economy. To highlight only one example: next
Olympic games in Beijing in 2008 have encouraged Chinese government to launch a
program to encourage and subsidized English learning, in particular for jobs related to
tourism (see internet site in Reference).
Similarly, m-Learning techniques obviously apply to multiples learning subjects,
but we will focus in this paper on the case of language learning, and this via a 2G
mobile phone technology. This specific “subject-technology” couple seems to us to be
the most pertinent one to illustrate our point.
A language learning service via m-Learning

If a learning service via mobile phones offers some distinctive advantages (mobility,
choice of time and place, potentially vocal interaction, etc.), it still remains subject to
multiple constraints, some having been already mentioned (small screen, difficulty of
reading long texts, data storage difficulty when even possible, multimedia limitations
on first generation mobile phones, etc.) It is therefore important to set realistic
objectives and to structure the learning process in order to take into account mobile
phones’ possibilities and limitations.
Researches about m-Learning

Multiple researches have been conducted these last years in the m-Learning field, and
they have served as a basis for our reflexion, especially:
• The « m-learning » European project sponsored by the «Information Society
Technologies Programme» (see References).
• The worldwide MOBIlearn project, lead by Europe, which explores the different
mobiles pedagogical approaches (see References).
• The INLET project which attempted to proposed an introduction to the Greek

language via mobile phones during the Olympic games of 2004.
• Prototypes realised by Regan which have demonstrated that m-Learning could
effectively support language learning (Regan 2000).
• A mini Italian training via SMS, for foreigners, which was rather successful (see
References).
• The English CTAD company’s service, exclusively based on a vocal learning via
mobile phones.
• The basic service of English language training via mobile phones, set up by the
BBC in China.
Concrete Proposition for a m-Learning Service

Taking into account what precedes, our proposition to define an effective learning
experience via mobile phones is articulated around the following elements:
• A learning of the key 1’000 words of a languages (level: between surviving and
autonomy), in roughly 50 modules of 20 new words each, grouped by themes like
transportation, food & beverages, health, etc.
• Learners can choose anytime any module in the list.
• They learn through scenario/role playing, which requires interaction with the
system as well as with other learners.
• Each learner becomes member of a community and interact with others via text
message (SMS) or vocal conversations.
• A learning scenario is divided into very small modules (« learning objects »),
authorising very short sessions (2-3 minutes) and a great flexibility.
• An entire scenario can be played in 2 days.
• Learning instructions, including key vocabulary, are sent prior to the course by
email (and are also available on an internet site for download or vocal play back).
• After each module the scenario and the key points are summarised and available
the same way.
• An interactive vocal help system is available at any time, completed by a coaching
system between learners community members.
We immediately highlight some key characteristics which will be detailed further
down. On the pedagogical side: the fun & role play approach and the interaction
within the community. On the technological side, the complementarities of media,
and the value of an interactive vocal service.
APPROPRIATE PEDAGOGY FOR M-LEARNING
Pedagogical Dimensions, Tools & Approach

If it is clear that reading of long text on mobile phones is uneasy if not impossible, the
textual dimension is still possible through usage of SMS (or even MMS = multimedia
SMS). It is even very effective as messages are pushed to the recipient, who could
respond/interact with the system, for example by sending a SMS with the right answer
to a question. Finally, this technology is simple to use and largely adopted by phone
332 Yannick Jolliet
users. On recent phones, equipped with navigators to read Web or WAP pages (web
pages designed for mobile phones), or even equipped with Java client, we could
potentially think to go further and send references or multiple paragraphs texts.
The speech dimension is also fundamental, and probably evens the corner stone of
a good m-Learning service. Phones are designed to transmit voice, and they allow
therefore -without any major resistance- the listening of texts of a certain length, or of
grammatical rules, or of voice help. Reciprocally, phone allow users to comfortably
speak with either a system which can record him, allowing the learner to listen back
to himself and compare with an ideal pronunciation; or it could allow an interaction
with other learners from the community, or even with a physical professor.
As per the writing, it is clear that phones are not so adapted because of their
technological limitations. However, they can allow some interesting options: for
example, via multiple choices questionnaires on SMS, to choose between different
possible spellings, or to input and spell correctly few words.
Some pedagogical tools in an m-Learning context offer valuable possibilities:
• List of some key words, or the “word of the day”, pushed via SMS to develop
vocabulary.
• Multiple choices questionnaire “true/false”, “fill-in the blanks” exercises with
responses via SMS.
• Listening to a text read by vocal service, followed-up by a quiz to practice
understanding.
Low / Medium Interactivity High Interactivity
Reading Listening Speaking Voc & Grammar SMS Writing Combines Listening, Speaking,
SMS (MMS) IVR IVR and IVR (MMS) SMS (WAP) Reading, Voc & Grammar
- Usual concepts - Top 1200 words - Top 100 verbs - Advanced sentences - 4 Verb tenses
Autonomy
News download with Quizz News hearing with Quizz Listen to a sentence, Grammatical rule of the day Text download with Scenario based learning as we have
record yourself and listen download (SMS or MMS) or mistakes. Identify imagined it (number of participants still need
to your recorded voice listen to it all with Quizz mistakes and send how to be tested) with a riddle to solve.
many mistakes there are. Combines IVR and SMS
Receive the corrected text
Communication
- Usual concepts - Top 600 words - Top 50 verbs - List of 100 basic sentences - 2 verb tenses (present + other)
Music Text download with Novel hearing (100 words) Listen to a sentence, Listen to sounds and identify Text download with Connect 2 persons wishing to talk English
Quizz with Quizz record yourself and listen right word mistakes. Identify together
to your recorded voice mistakes and send how
many mistakes there are.
Receive the corrected text
- Basic concepts (my name, etc? - Top 100 words - Top 10 verbs - List of 25 basic sentences
Every day simple Every day simple Listen to a word or a Word / Verb of the day Word download with 3 Connect 2 persons through SMS for English
Survival
questions download (e.g. questions listening (e.g. sentence, record download (SMS or MMS) or different ortographs. message exchanges
how much does it cost, how much does it cost, yourself and listen to listen to it all with Quizz Indicate whether #1, 2 or or
where is station,...) with where is station,...) with your recorded voice 3 is the right one Connect an English teacher with a customer
Quizz Quizz for a 10 minutes course
Figure 3: m-Learning: matrix of pedagogical tools & technologies, related to the different
learning levels
(source: Eisenhart, Dufour, Loroch, Jolliet)
• Recording of a vocal sequence with a play back to practice accent.
• Grammatical explanation via vocal service.
• Interactive scenario involving multiple participants for a “real-life” experience (see
chapter 4.2).
• Reference services with added value, such as dictionary (definition, synonyms, and
antonyms) or translation service (word, phrase), etc. These services might be
effectively delivered using simple technologies such as SMS or vocal systems (see
section « Technology » below).
The approach we defend is to rethink the learning sequence, and to structure a totally
new course to leverage mobile technologies possibilities.
Figure 3 summarizes different learning possibilities as potential m-Learning services.

It positions multiple pedagogical tools & techniques for each learning level, and
shows options and multiple and diverse.
Scenarios and Role-Play

Constructivist learning theories highlight that learners interpret information and the
world through their own perception of reality: we learn though observation and
interpretation, and we transform the information into personal knowledge (Coopers
1993).
Student learns better when they can put learning into context. m-Learning facilities
a personalized learning as it permits to learn (and collaborate) in any place at any
time, allowing learning to be in context. Josefson’s researches on nurses’ training also
show that knowledge is mainly acquired through experience and practice, and not
only through mental faculties.
In this perspective, m-Learning seems to offer interesting possibilities because of
the mobility and freedom it offers, and because of its possible contextualization,
especially when comparing to traditional computer e Learning.
It is known that fun and games are fundamentals and powerful elements of the
learning process. Multiple studies prove this fact, for example Dempsey’s ones which
highlight how much simulations, adventure and problem solving contribute to the
learning process (Dempsey and al. 2002). 82% of participants to a study on this
subject feel that pedagogical mobiles games can help them in the learning of reading
and writing (Mlearn 2004 conference). Finally, games stimulate curiosity, willingness
to learn and to persevere (Ko, 2002), and allow experimentation with an immediate
feedback (Roubidoux and al. 2002).
Our approach of m-Learning proposes a situational immersion through role play,
and introduces a suspense element with a sequenced scenario. The story allows a
better attention and a higher fidelity rate, so a more efficient learning.
Concretely, with our approach for a language learning course via m-Learning,
courses are structured around mini-scenarios that learners will play with other
participants, and with interactions of a vocal computer. « Script », text and vocabulary
are shared upfront with students (via a web site or email, or even via fax or mail,
depending on people’s technical equipment). Figure 4 shows an example of such a
script: it describes the participants-actors, the story with keywords highlighted and a
vocabulary list to learn.
Anybody can listen the story played by a vocal computer in order to practice
listening and understanding, and when participants are ready, they indicate it via
SMS. The system assembles then a group of participants and notifies them by SMS in
order to initiate the interactive session; the computer could play one or multiple roles
depending of people’s availability.
334 Yannick Jolliet
Figure 4: example of a role-play script for English lesson via m-Learning (source: Eisenhart,
Dufour, Loroch, Jolliet)
In our scenario above, Susan calls Tim and plays her role; he eventually correct
her, or ask the system to pronounce again the sentence to check her accent, then he
continues with his own text; and so on, in a real-time dialog. At any point in time,
participants can interact with the system, either via a specific key on the phone or via
a vocal command (see section « Technology » below) to obtain help, such as
clarification on a word or on the scenario itself.
The system can introduce a complicating factor in the form of an unknown
participant (played actually by the system) which gives wrong information and
complicates the initial scenario (actor in black in the example of script above). This
element of surprise forces students to understand what is happening and to solve
together an enigma, adding some suspense to the learning experience.
The fun aspect allows to move from a sometime fastidious and scholar learning
experience, to a stimulating social experience. And we see that phone is an effective
medium for such interactivity and for a learning experience in community.
Community and Networks

From a long time we know that learning could be reinforced by collaboration with
peers (Vygotsky 1978). In fact, the teaching practice evolves largely toward a more
collaborative and participative approach. From a directive « teacher-centric » [1-n]
model; we progressively move to a « learner-centric » approach (Ally 2004). The
professor then becomes a coach and an animator who facilitates a more personalized
and community based learning, where each student is a source of knowledge and a
support for the others ([n-n] model). With mobile technologies, the learner becomes a
true actor who controls the learning process and takes decisions aligned with his
cognitive state.
The socio-cognitive vision would also add that learning takes place in a social
context, and that interactions and communication are as important as content of the
learning process. Collaboration and information sharing with peers is a powerful
mean to acquire and experiment new learning, and this is an essential element for an
e-Learning approach.
Phone is by nature a communication medium and without any doubt richer than
email or text read on a computer; as such, it authorizes an effective learning
experience. And this is why, despite current technical limitations, it is pertinent to use
mobile phones in the context of e-Learning (versus PDAs or laptops).
A learning community, dynamic and well leveraged, effectively contributes to the
learning experience through multiple aspects. For example, support between members
as well as the feeling to belong to a group contribute to reassure learners and increase
their perseverance and fidelity to the program.
Expertise sharing within the community allows a rich learning experience, of the
[n-n] type. Anyone can in fact act as an « expert » and share with the network an
answer or an explanation, put things in context or give an example (typically via the
community site on the Internet, cornerstone of the learning network).
Furthermore, interaction with other real persons offers a human dimension to the e-
Learning, which is often missing with other « Computer Based Training » (CBT)
approaches.
Finally, emulation between participants is also a success factor: an individual score
based on success (e.g. answers to quiz, responses to lessons) and participation (help to
others, information sharing) could be published on the web site. Winners of such a «
Top100 » would gain access to new modules, or to other services for free (translation,
etc.). Won points show progresses made, and are steps toward the final objective
(which could be a certification).
TECHNOLOGY FOR M-LEARNING
We encounter too often resistances against the m-Learning based on standard mobile
phones, mainly because of its technical limitations which at a first look seem too
important.
But in fact, vocal technologies progressed immensely and became very structured,
allowing creating m Learning services and vocal applications without “reinventing the
wheel” each time. Without entering into technical details, we will briefly describe
336 Yannick Jolliet
here how technology could allow the implementation of an m Learning service as

described above.
Multimedia
We mentioned it -and we will come back to this point-: to be really effective, the
approach has to be multi-channels and multimedia. The phone becomes the principal
mean to interact and to learn, but it has to be supported by other technologies when
necessary: namely a web site to allow easy administrative management, but also
efficient collaboration among members. Other medium such as emails or even paper
allow keeping references of courses (scripts, vocabulary, grammatical rules, etc.).
These media can even include hyperlinks to easily jump to an electronic content by
entering only a simple code (either on the web site or on the phone via SMS) (see
PaperLink technology in reference).
Multi-levels Interactive System

In order to implement a dynamic m-learning solution, it is important to setup a truly
automated and « intelligent » system, a complete platform orchestrating interactions
between users and with the system. Such a “server” should allow a rich interaction via
multiple communication channels, and a management of vocal dialogs but also text or
visual messages. The architecture remains classical, with three logical layers:
• A presentation layer to support the interaction with users: a graphical interface
(Web, Wap) for web browser, a textual interface for phones (SMS) or computer
(email), or even – and mainly- a vocal interface through an « Interactive Voice
Response » (IVR).
• An application layer to manage information and content logic, as well as to manage
the learning process content and to personalise the content: it is the true
orchestrator of the system and the heart of the « Learning Management System ».
• Finally, a data warehouse, containing all « learning objects » (content, vocabulary,
grammatical rules, pre-recorded messages, scenario, etc.) which have to be
dynamically assembled by the above layer.
Figure 5 describes such a technical architecture, well adapted to an interactive and
dynamic m Learning service. All these technologies are existing today and could be
assembled to create such an integrated system (e.g. by using architectures and tools
such as the ones of VoiceObjects, see references).
Interactive Voice Response (IVR)

Standards and technologies in the field of voice recently reached a maturity state to
allow industrialized development of powerful interactive voice applications. We are
far from the old fragmented solutions to be assembled: SMS server, answering
machine, basic navigation via phone keys and pre-recorded messages, etc. Today,
integrated solutions cover the entire spectrum of rich vocal application development
and allow to dynamically creating complete personalized vocal interfaces, similar to
what is done on the internet portals (e.g. « My Yahoo »). We speak then about « vocal
pages » and of Vocal Content Management System (vCMS).
Such systems are based on voice-specific XML standard (« VoiceXML ») which
allows describing, coding, managing and playing vocal sequences. « Text to Speech »
technologies are today absolutely Figure 5astonishing; at light-years of artificial play

back we used to hear only few years ago. It is now extremely difficult to differentiate
the machine from the human, as machine’s voice is now really warm and modulated,
and flowing naturally.
Figure 5: Suggested rich technical architecture for an Integrated

Symmetrically, voice recognition technologies have gone a long way and reach
now 99% recognition rate in quiet environments. And this is precisely where we face
a bottleneck with mobile technologies: ambient noise, often important, adds up to the
poor mobile phones’ microphone and transmission quality: bottom line the voice
338 Yannick Jolliet
recognition cannot be really effective in such a case. However, recognition of

predefined keywords is absolutely possible and effective, opening an interesting field
of possibility to interact with the user who could easily navigate through “vocal
pages”, similarly to browsing the web. Of course, interactions and options selection
can also be made using phone’s keys.
Finally, we should highlight two interesting aspects this type of architecture offers:
anonymity and easy payment. It is essential to preserve participants’ anonymity, and a
fortiori when these one interact with each others. Here, any relation between
participants always go through the system which manages basically a three parts (or
more) conversation, and which can directly and dynamically intervene in the
pedagogical relationship or the on-going scenario.
The payment of such an m-Learning service is largely facilitated by the fact that all
interactions and lessons (conversations and SMS) happen via a unique over-taxed
phone number, and thus the phone operator can very easily manage the billing of the
service and the retribution of the partner.
So, such a technical architecture authorizes the truly dynamic generation of
personalized vocal pages (taking into account user profile, his level, etc.), a relatively
rich interaction with the user (via SMS, keys or vocal commands) and a granular
management of the different reusable « learning objects » (voice, word/text, tests,
etc.). Theses contents are dynamically assembled depending of the context (number of
learners online, personal profiles, course level, etc.) to allow a personalization of the
pedagogical scenario, and a rich learning experience.
CONCLUSION AND PERSPECTIVES
With this article, we tried discussed the viability of an effective learning experience
via simple mobile technologies. We demonstrated the pertinence and certain
possibilities for learning a language using second generation mobile phones.
Adapting pedagogy to the technology is pertinent in this case as it allows
capitalising on devices and habits largely adopted, without trying to replace preferred
users systems.
It is obvious that extremely rapid progresses of communication technology, as well
as their wide spread around the world, offer at short term dramatically increased
pedagogical possibilities. To cite only few of them, current or available in a very near
future, we would highlight: media photo & video or phones equipped for video
conference, instant messaging, pushed emails, internet browsing, embarked or
downloadable applications (typically in Java)… But even more: content developed in
Flash-lite which allows rendering interactive and animated multimedia (see Linsalata
2005), high-speed phones (3G) or geo-localisation via integrated GPS chips…
In conclusion, it is clear that numerous of rich perspectives exist down the road,
and still remain to be explored in order to position added value m-Learning services.
This especially in the mobile learning field, coupled with other teaching forms /
media in a truly integrated approach (« blended learning »). Finally, the number of
mobile devices will soon be greater than PCs, which promises m-Learning an
important role in tomorrow’s teaching.
ACKNOWLEDGMENTS
We would like to thank here F. Eisenhart, A. Dufour & V. Loroch for their
contribution to our thinking process about some of the aspects developed here.
REFERENCES
Articles and Books

Coopers PA (1993). Paradigm shifts in designing instruction: from behaviourism to cognitive to
constructivism. Educational Technology, 33(5), p12-19.
Brown (2004), The role of m-learning in the future of e-learning in Africa. In D Murphy, R
Carr, J Taylor and W Tatmeng Distance education and technology: issues and practice.
Hong Kong: Open University of Hong Kong Press, p197–216.
Roubidoux MA, Chapman CM, Piontek ME (2002), Development and evaluation of an
interactive web-based breast imaging game for medical students. Academic Radiology,
9(10), p1169–1178.
Ko S (2002). An empirical analysis of children’s thinking and learning in a computer game
context. Educational Psychology, 22(2), p219–233.
Berson MJ (1996). Effectiveness of computer technology in Social Studies: a review of the
literature. Journal of Research on Computing in Education, 28(4), p486–499.
Dempsey JV, Haynes LL, Lucassen BA, Casey MS (2002), Forty simple computer games and
what they could mean to educators, Simulation and Gaming, 33(2), p157–168.
Regan, M (2000). Mobile learning, http://sll.stanford. edu/projects/mobile learning.
Vygotsky (1978), Mind in Society. Cambridge MA: Harvard University Press.
Conferences
Ally (2004). Using learning theories to design instruction for mobile learning devices, in A
book of papers from MLEARN 2004, published by Jill Attewell and Carol Savill-Smith, p5.
Attewell (2004). Mobile learning and social inclusion: focusing on learners and learning, in
Learning with mobile devices, research and development, a book of papers, edited by Jill
Attewell and Carol Savill-Smith, p3.
Hitz (2004). PaperLinks – Linking printouts to mobile devices, in A book of papers from
MLEARN 2004, published by Jill Attewell and Carol Savill-Smith, p5.
Internet sites
ZDNet Research blog, 29.9.2005 12:10AM, http://blogs.zdnet.com/ITFacts/?p=9065.
Linsalata (2005), White Paper sponsored by Macromedia, Addressing growing handset
complexity with software solutions, in IDC Analyze the Future, www.idc.com.
« m-Learning » : European project sponsored by the «Information Society Technologies
Programme», www.mlearning.org.
« MOBIlearn » : worldwide research & development project, lead by Europeans,
www.mobilearn.org.
Mini-Italian training course via SMS : realised by the company Albatros in collaboration with
the CRMPA, in the context of the cited European m-learning project, www.mlearning.org.
VoiceObjects, Voice Application Management Systems, www.VoiceObjects.com.
China and English language learning: “Beijing Launches English-Learning Programs”
(http://www.china.org.cn/english/2001/Jul/16399.htm), “Olympic Bidding Success Spurs
English Language Fever in China”
(http://english.peopledaily.com.cn/200107/29/eng20010729_76042.html)
Author Index
Alberts, Philip P. 53 Lam, S. 9

Alexandru, Catalin 217 Law, Ken C. K. 101
Alfonseca, Enrique 186 Lee, Kenneth K.C. 88
Amelung, Mario 257 Lee, P. W. R. 79
Anguiano, Eloy 186 Leon, Enrique 267
Liu, Jing 315
Barker, Philip 42 Liu, Yintao 315
López-López, Ignacio 292
Callaghan, Victor 267 López-Rodríguez, Domingo 227
Chan, Chilli C. K. 237
Chan, F. T. 79 Ma, Will Wai-kit 303
Chan, Jason K. Y. 101 Mérida-Casermeiro, Enrique 227
Chew, Esyin 123 Merino-Córdoba, Salvador 227
Chong, Melody P.M. 88 Moebs, Sabine 162
Choy, M. 9, 30 Mora-Bonilla, Angel 227
Chu, Raymond 136 Moreira, Tiago 206
Murray, Linda A 53
Dettori, Giuliana 174
Domínguez-Mateos, Francisco 292 Pascual-Nieto, Ismael 186
Dragan, Alexandru 217 Peiris, K.H.R.A. 279
Perez-Marin, Diana 186
Fang, Jenny 136 Persico, Donatella 174
Fok, Apple W. P. 237 Piotrowski, Michael 257
Fong, Joseph 19, 30 Poon, C.K. 9
Premaratne, S.C. 279
Garcia, Francisco José 206
Griffin, Darren K. 53 Rebedea, Traian 217
Rego, Hugo 206
Henrich, Andreas 150 Rodriguez, Pilar 186
Hewagamage, K. P. 279 Rösner, Dietmar 257
Hijón-Neira, Raquel 292
Hsueh, Hsiang-Yuan 66 Shen, Liping 267
Hua, Jing-Shiuan 66 Shen, Ruimin 267
Huang, Shi-Ming 66 Sieber, Stefanie 150
Stephenson, Julia E. 53
Ip, Horace H. S. 237
Trausan-Matu, Stefan 217
Jones, Norah 114, 123 Turner, David 123
Jolliet, Yannick 327
Velázquez-Iturbide, Ángel 292
Kim, Won 1
Author Index 341
Wang, F.L. 9, 30 Yang, Fang 315

Weibelzahl, Stephan 162 Yu, Y.T. 9
Wong, Chan Lam 196 Yuen, Allan Hoi-kau 303
Wong, Francis 136 Yuen, L. 9
Xu, Lei 315 Zhang, Liming 196

Zheng, Li 315

Blended Learning For Programming Courses A Case ST

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Blended Learning For Programming Courses A Case ST

Uploaded by

Copyright:

Available Formats

Joseph Fong Fu Lee Wang (Eds.

Organized by: The Hong Kong Web Society

Sponsored by: City University of Hong Kong

This workshop focuses on e-Learning as a supplementary learning to classroom

Joseph Fong (City University of Hong Kong)

Web-based Logging of Classroom Teaching Activities for Blended Learning 19

Blended Learning for Programming Courses:

Blended Learning with Webs, Wikis and Weblogs 42

Discovery of Educational Objective on e-Learning Resource:

Blended Learning: Experiences of Adult Learners in Hong Kong 79

An Observational Study on Blended Learning for Japanese Language

Structured Blended Learning Implementation for an Open

The Disruptive Effect of Technology: a University Case Study 114

The Marriage of Rousseau and Blended Learning: An Investigation of

Concepts of Blended Learning for Different Content Types 150

Blended Learning: Towards a Mix for SMEs –

Supporting Self-Regulated Learning in a Blended Course 174

A Study on the Impact of the Use of an Automatic and Adaptive

A Wiki-Based Collaborative Learning Design and Its Effects in

Implementation of an E-Learning System – Learning Design and Learning

Visualisation of Learners’ Contributions in Chat Conversations 217

HECACEJ: B-Learning Tool for Static Content Creation in Joomla! 227

MAAS – Mobile Administrative and Assessment System 237

Large-scale Computer-Assisted Assessment in Computer Science

Exploratory Research on an Affective e-Learning Model 267

Design and Development of Blended Learning through LMS 279

Learning News Writing Using Emergent Collaborative

Description Patterns in Learning Design Authoring Systems 315

M-Learning: A Pedagogical and Technological Model for Language

Author Index 340

School of Information and Communication Engineering

Abstract. Blended learning is known roughly as combining the

Keywords: blended learning, blended learning methodology, e-

The recent rapid advances in information technology, including the Internet

2. A Precise Definition of Blended Learning

As a first approximation, blended learning may be defined as a combination

There are at least three key dimensions to consider in classifying learning:

4. Formal, course-scheduled e-learning. This type of e-learning has been

3. Methodology for Creating a Blended Learning Program

The scope of a blended learning program, as already discussed, would

Key issues to be carefully analyzed follow.

• Cost issue: To add the e-learning element into a traditional course or

classroom, the classroom can be put to good use. Good scheduling of a

It is important that there be some pre-determined set of measures and a

The emergence of e-learning, and the realization of the strengths and

Acknowledgments. This research was supported by the Korean Ministry of

1. Bersin & Associates, “Blended Learning: What Works?”, May 2003,

M. Choy, S. Lam, C.K. Poon2, F.L. Wang, Y.T. Yu and L. Yuen

Department of Computer Science, City University of Hong Kong

Abstract. Learning computer programming is known to be difficult for many

Keywords: Computer programming, automated system, learning.

Thanks to the booming development of the Internet, there is currently a trend

2 The Programming Assignment aSsessment System (PASS)

PASS was first developed in our university in 2004 as an executable prototype

2.1 System functions and design

Fig. 1. Relationships among core entities in PASS.

Submissions monitoring and analysis.

Problem sharing in problem repository.

3 Approaches to Blended Learning using PASS

The teaching and learning of computer programming is a challenging job. A well-

A Programming Exercise of Solving a Quadratic Equation

Write a program to solve a quadratic equation. The general form of a quadratic

Test Cases at the Beginner Level of Difficulty