Professional Documents
Culture Documents
8-5
Plenary Sessions
Online Management Education: Quality Assurance – Guaranteeing
Quality Learning Outcomes
1,3
U21Global, 2Hong Kong University and U21Global
Waseda University
Presentations
hlange@u21global.edu.sg, spinks@hkucc.hku.hk, steo@u21global.edu.sg
Session 1-3
that time was not well regarded in that particular con- This paper explores quality assurance and related
text. issues with respect to online education.
However, U21Global has not been daunted by this Over the last decade, online education has rapidly
particular challenge, and quality remains as the key developed gaining a foothold into many educational
driver of its approach to education. With over 130 jurisdictions. For example, according to sources in the
MBA graduates, it has been encouraged by the excel- USA1, worldwide there are more than 200,000 online
lent learning outcomes achieved as demonstrated by courses, and it is estimated that online participation in
the successes of its graduates and students. these courses is rising by 20 percent year on year. In
This paper discusses the challenges to online 2004, the growth was 33 percent, and in that year, 2.3 Session 4-6
higher education programs in establishing and main- million students in the USA took at least one online
taining quality. It draws on the example of U21Global course.
and examines how U21Global has both established This growth has not simply been an experiment of
and continues to maintain quality programs and learn- a new learning technology, but an economic impera-
ing outcomes. It explores the internal and external tive. Educational demand is growing rapidly reacting
influences on both quality and quality assurance. to the economic development of a globalised world.
Specifically, the paper discusses the quality issues Traditional educational providers are straining under
associated with the key components of online educa- the demand, and new ways of delivering education
tion. The issue of pedagogy is also paramount and in are emerging in response.
this context the learning design element of However, the development of online education has
Session 7-9
U21Global’s programs is highlighted as an example not had a smooth path in terms of quality and quality
of a pedagogically rich online education environment assurance, and some of the early forms of online edu-
which incorporates many of the quality assurance pro- cation were justifiably criticized. The notion that
cesses, and also sets new benchmarks for online edu-
cation.
The question of whether quality assurance systems 1
www.distance-learning-college-guide.com
225
Session 8 Misc.
e-learning could offer scalable educational opportuni- compliance models to attest their quality. For these
Plenary Sessions
ties more economically than campus based education organizations, the emphasis is on quality assurance by
is largely discredited at this time, excepting if peda- others.
gogy suffered. Mature organizations have moved beyond this,
To commence the discussion, the next section cov- and have developed internal quality assurance pro-
ers a framework for understanding quality assurance cesses which align with their strategic intent. These
issues within the context of online education, and this organizations have well developed reputational
is followed by a discussion of each of the main ele- effects based on their internal QA systems which have
ments of this framework. U21Global’s internal quality been tested over time, more through market and pub-
assurance processes are examined in then examined, lic accountability rather than through regulatory bod-
Waseda University
and concludes with a discussion of the challenges ies. While regulatory bodies oversee and audit from
Presentations
faced by a global educational provider in quality time to time, it is the public acceptance and account-
assurance. ability of these highly reputed educational providers,
based on experience and outcomes which matters in
2 A framework for understanding quality mature.
issues in e-learning Most educational institutions probably lie some-
Spinks (2006) outlines a three dimensional frame- where in the middle of the two extremes outlined
work in which to understand the issues associated above, and benchmark themselves against the current
with quality assurance in education. The three axes best practice, while adapting external QA standards to
describe maturity of the organistaion/education, the their internal processes. The concept of QA and matu-
DLI Review & Report
geographic reach of the education, and the delivery rity is expressed neatly in Australian Department of
mechanism. This is best seen in Figure 1: Framework Education, Employment and Workplace Relations
for Understanding QA Issues (2008, p. 4) which sets out higher education quality
Spinks poses the question as to whether quality assurance principles for the Asia Pacific Region. This
assurance (QA) systems should be specific to the is summarized in Figure 2 below:
maturity of the organization (dimension 1), location
and/or culture (dimension 2) and delivery method
(dimension 3), or moreover whether QA systems
should be specific to the organization, location or
delivery mechanism, or whether there are universals
Session 1-3
Global
E-learning DIMENSION 3:
DELIVERY
Immature
DIMENSION 1:
Session 4-6
Whether QA systems for education should differ to be local, as there are many country and regional
depending on the maturity of the organization has accreditation agencies (for example, regional agencies
already been catered for in current systems. As we within the USA and country specific systems).
review the evolution of QA systems, we see that at We generally find that in practice, local or regional
one extreme, new and untested education providers QA systems tend to entrench local content, locally
tend to have more reliance on external regulation and trusted delivery systems and local cultural norms.
226
Distance Learning and
the Internet Conference 2008
Further, we find that quality benchmarking in such student support, including the availability of academic
Plenary Sessions
systems tends also to be local – further entrenching as well as non-academic assistance.
the “localness” of the QA system and the resulting There is less evidence from practice about the
educational system. For example, local systems tend most acceptable QA systems for online education, and
to encourage local issues, problems and case studies it is this particular dimension of the Spink’s QA
to be used as the appropriate examples within educa- framework, namely educational delivery, that we
tional content, and tend to encourage a continuation explore further in the following section of this paper,
of the styles of educational delivery and language of and in this context, review the experience of
instruction and pedagogy that has persisted in the U21Global.
past. In other words, local QA systems tend to be
3 Accreditation and Quality Assurance
Waseda University
inward looking.
Presentations
Adaptation to change is not encouraged as the Systems
benchmarking is local, and hence a move to a glo- The typical accreditation processes and quality
balised curriculum for example, will require a dra- assurance systems involve the assessment of an edu-
matic change both in the QA systems as well as the cational institution’s mission; organizational struc-
education itself, rather than a system which encour- ture; resource development and capacity; curriculum,
ages changes which incorporate continuous improve- pedagogy and assessment; faculty and faculty sup-
ment and innovation. For example, after years of port; student support and learning outcomes.
debate in Malaysia, and resistance to system change, In this context, possibly the most important of
Malaysia’s educational QA system has in 2007 under- these, based on modern QA norms, is whether the
Session 1-3
may find it more difficult to satisfy individual country Given that learning outcomes and the effective-
or regional accreditation which is more likely to be ness of these towards economic production and
locally focused as discussed above. capacity building are the ultimate determination of the
quality of the educational programs, we argue that the
2.3 Delivery Mechanism system of quality assurance and related standards
The final dimension covers the issue of whether or should be the same, regardless of the educational
not there should be different QA systems education delivery mechanisms.
which is delivered through a face to face classroom
media, and also specific QA for online education. 3.1 Quality Assurance of e-learning – the Session 4-6
This raises the question of whether there are interna- Similarities
tionally acceptable QA systems for e-learning, and When the network of prestigious, research inten-
whether these should be the same or different to those sive university network, Universitas 21, established
used to assess physical campus based education. U21Global, the fully online graduate management
Some systems make it clear that the principles of school, it established both quality standards which
QA, such as providing evidence for the effectiveness U21Global had to achieve, and also established an
of learning and ensuring that this data feeds back to external quality assurance agency to undertake the
enhance the development of the education, apply to QA to assess whether the standards had been
all modes of delivery, whether face-to-face (F2F), dis- achieved. The standards set were in simplicity – that
tance, or online. There are, however, good arguments U21Global provide education which is at least as
Session 7-9
to the contrary. Online education has particular needs good as that provided by the universities in the
that should be addressed more rigorously than in F2F Universitas 21 network. In this case, Universitas 21
education. These include the more obvious character- made a judgement that the type of QA was similar to
istics such as attention to IT availability and security, that required by any of its universities, that embedded
but also those that need to be accorded a higher prior- in the processes and systems of its universities.
ity in online delivery than in F2F delivery, such as We see other examples in the US and UK. For
227
Session 8 Misc.
example, AACSB the private accreditation agency has in virtual teams, where the engagement is measured in
Plenary Sessions
a well developed set of QA criteria and processes qualitative terms rather than just in terms of numbers.
used to assess the quality of business schools. It uses
the same processes for fully online institutions, such 3.2.2 Student Administrative and Library
as U21Global, as it does for those institutions which Support:
provide only campus based education. In the UK, the Two factors are important with respect to adminis-
Quality Assurance Agency, has decided not to prepare trative support and library of any quality educational
separate guidance on the quality assurance of e-learn- program. However, they become critically important
ing. Similarly, the eight US regional accreditation in the case of e-learning. The first is the ease of
agencies have jointly stated that the well-established access to the support structures, and for external stu-
Waseda University
essentials of institutional quality are applicable to the dents studying online, the access needs to be at times
Presentations
emergent forms of learning. that suit their flexible study requirements. Second, the
support must be effective. The provision of support is
3.2 Quality Assurance of e-learning – the one thing, but if it does not meet the needs of a
Differences mobile student body study at flexible hours, then it
However, when we look at the learning design adds little value. It is noted that campus based univer-
itself, we find that there are some differences in the sities don’t usually have to deal with this2. Further, if
way that e-learning programs should be quality students are residing within many different time zones
assured relative to campus based education. For then support must meet the various time demands of
example, we find that it is in the pedagogy and design the students. In conjunction with this, academic pres-
DLI Review & Report
and support structures that key differences occur. ence (in a virtual sense) assists in providing added
There are some important key areas where we security in terms of the student support mechanism.
believe differences are evident in high quality e-learn- Hence, the closer the student feels to the professor
ing programs. Four of these are learning design and who is facilitating the virtual class, then the more
delivery; student support services; technology sup- assured the student will be about the support struc-
port, availability and reliability; and identification. tures in place. In fact, the integration of the adminis-
trative support and academic support is one way to
3.2.1 Learning Design and Delivery: close the gaps in the student support structures and to
Distance programs by their very nature are more ensure a more holistic supportive environment for stu-
learner centric, and while in the past distance pro- dents.
Session 1-3
grams have tended to also be lonely educational expe- The positive value of the presence of professors in
riences, technological advances have ensured that physical classrooms is never challenged and despite
there is now the possibility of far more interaction very little research evidence there seems to be univer-
between the professors and students in online educa- sal agreement that it is valuable. However, there has
tion than in the past. As students of distance education also been very little investigation of the value of the
programs have flexible study hours, they usually presence of faculty in the virtual classroom. Bedi and
require more discipline around their study program Lange (2007) is an exception, and examines the role
than required of campus based students. This in com- of faculty interaction in e-learning programs, and
bination with the student centric nature of the learning empirically test its value in terms of both the students’
Session 4-6
design means that students take more responsibility learning outcomes and the students’ perceptions of
for their learning. Students in such environments educational value. Bedi and Lange find that learning
therefore need greater orientation to their study pro- outcomes and student perceptions of quality are found
gram including information about timelines and to be positively related to the presence of faculty in
schedules, the learning processes and expectations, the online classroom.
course materials, and support (both technical and
administrative). The design of learning in high quality 3.2.3 Technology - Support, Availability and
e-learning programs needs to capture the imagination Reliability:
of students to encourage high levels of motivation and In an online program, the availability and reliabil-
participation, and to avoid high withdrawal rates, and ity of the underlying learning management system is
Session 7-9
inactivity, often associated with distance programs. obviously of particular importance in the delivery of
As students don’t meet face to face, then there also
needs to be more opportunities made within the learn-
ing environment for students to engage with each 2
However, it is recognized that this is changing and that the
other. Hence, the design of any program should incor- so called Y generation has different needs including more
porate student networking online including working flexible campus based offering of services.
228
Distance Learning and
the Internet Conference 2008
quality education. Of equal importance is the avail- tice in online education. Plagiarism is easily checked
Plenary Sessions
ability and reliability of technical support for students in the online environment as usually all assessment is
in a timeframe that fits their study patterns. As such, in digital form, making the application of plagiarism
higher standards of system access and reliability are software and checks relatively simple.
usually required by online education providers. On
the access side, systems need to cater for differences 4 U21Global – a Case Study
in bandwidth, for example, to ensure that students We now look at these issues in the context of
from less developed countries are not disadvantaged. U21Global and its quality assurances processes. First,
Moreover, systems need to be protected from infec- we examine briefly the evolution of online education
tion and hacking, although this is true of all university and U21Global’s place in that, and following this we
Waseda University
systems. A distinct difference between an online edu- discuss U21Global’s educational environment and its
Presentations
cational provider and its counterpart on campus, is the quality assurance.
close relationship between technology and pedagogy.
In e-learning, the technology enhancements are often 4.1 The Evolution of Online Education
driven by the pedagogical features that are sought U12Global was established to meet some of the
within the learning design, or more generally, it is the challenges to education envisaged in the late 1990s by
pedagogy that drives the technology changes, and not the Universitas 21 network of universities. Such chal-
the other way around. The interrelationship between lenges included the rapidly growth in demand for
technology helpdesk and student services support is education, and the necessity of the provision of edu-
also required to ensure no gaps in the service avail- cation to fuel associated economic growth. The grow-
Session 1-3
work submitted by the student has integrity. With this nomic prosperity is demonstrated in Figure 3 below,
is mind, learning design of the online education which plots higher education gross enrolment ratios
should support a process of learner revelation – so against GDP for a number of selected countries (2004
that the fellow students and the professor in the vir- data).
tual class get to know each of the students, their writ- From Figure 3, we see that there is a positive link
ing styles, presentation styles, cultural aspects of their between tertiary education enrolment and economic
work, etc. This is something often difficult to pick up growth. Universitas 21’s objective of broadening
in the physical classroom, as so many students “hide” access to quality education in the developing world
or keep a low profile during class. In a campus based was also an objective for the establishment of Session 4-6
education, the first time that a piece of work is U21Global. Access to many learners at any age was
received is often weeks after the start of the class, and possible, including working adults with families, and
there is no guarantee that this piece of work is that of whose means did not allow them to travel offshore to
the student who submitted it. In an online learning gain the education that they needed. Through
environment, this may be an even greater challenge. U21Global, Universitas 21 could offer borderless
Hence, the pedagogy or learning design should education in a truly global way.
encourage and reward continuous assessment and
engagement providing opportunities for students to
reveal their styles, characters, etc – all of which can
be used to identify their work. Hence, there should
Session 7-9
229
Session 8 Misc.
Cross-country Comparison of GER and GDP/Capita, 2004 ity. A possible conclusion from the early experience is
Plenary Sessions
Higher 80
Education
USA
that the quality assurance of content is only a small
Gross
Enrollment Australia part of the quality considerations of online delivery of
2
Ratio (1) (%) R = 0.58
60
Korea education. As argued above, this suggests an equiva-
lence of QA processes for both campus based educa-
Lebanon Japan
Singapore tion and online education – as also concluded by QA
40 Philippines
Jordan
agencies in the UK and US. As such, QA processes
Thailand
Qatar Hong Kong
for online education should involve serious consider-
20
Iraq
Iran Malaysia
ation of the pedagogical approach, course content,
Indonesia China UAE
and assessment, together with those components that
Waseda University
India Oman
together make up a quality education, such as
Presentations
0
1,000 10,000 100,000
resources, learner support, staffing and technology.
GDP/Capita, PPP ($)
cally questionable, with several failing to live up to operates. This begins with the development of its edu-
their business plans in the aftermath of the dot com cational environment.
bubble. The alignment between technological innova- U21Global offers a pedagogically rich, learner
tion and pedagogical innovation, or more correctly, centric environment for its students. This is best seen
the lack of alignment was one critical issue for the in Figure 4, which shows the integrated nature of the
failures in most cases. For example, early online various components of the education provided. The
development was made without much regard in some pedagogy itself is importantly embedded throughout
cases for pedagogy, following instead the distance all aspects of the model.
education model that left learners alone to fend for
Session 4-6
230
Distance Learning and
the Internet Conference 2008
Plenary Sessions
The
Instructivist The journey by students is the fact that professors get to
Professional
Environment
Environment know their students well as the subject progresses.
The
e-text,
Multimedia Authentic assessment
The small class size of at most 40 students ensures
Constructivist
Environment
Textbooks Work project Transfer of
Knowledge (Networking;
The
this. More importantly, they get a strong sense of each
Alumni)
Communicative student’s culture, style, language, tone of writing, for
Individual assignments Environment
Team work, Situated
enquiry, Faculty
example. This makes identification of students – a
moderation
The
Emails, Bulletin
Boards, Voice over, key issue in quality assurance - much easier at the
The ip, Chat rooms
Supportive
Evaluative
Environment
Networking
meetings
time of individual project assessment and grading, as
Environment
online libraries, Career
by that stage, the professor is familiar with the stu-
Waseda University
Self-assessment, Formative
assessment, Peer
counselling, Student Admin
support, Technical support, dents’ work.
Presentations
assessment Learning support, Financial
Authentic assessment,
Evaluation-based QA
support
The influence of the professional environment out-
lined in Figure 4 within U21Global comes from a
Source: Teo, S. (2003). Towards a total learning environment number of specific design features in the pedagogy.
for the distanced adult learner. World Conference on First, the content itself is rich with many real life
E-Learning in Corp., Govt., Health, & Higher Ed. 2003(1), examples and application of the theoretical concepts
1213-1216. to business and management related issues in practice
and across the globe. Second, case studies are used
Figure 4: U21Global’s Educational Environment extensively throughout the subjects for both class dis-
cussion activities as well as for project work. Third,
Session 1-3
(both of which are assessed formally), but also development of managerial skills and capabilities is
through chat rooms, section cafes and other informal also a feature of U21Global’s program design, as well
media set up specifically for the class. as the specific objectives. The extent to which this is
To cater for the global student body with students achieved by U21Global is examined in Spinks, Lange
from over 70 countries represented, discussion board and Chan (2006) which finds that as students progress
activities which cover the full 12 weeks of a subject through their studies with U21Global, there is evi-
must be asynchronous. This has distinct advantages dence of soft skills development as represented first
for student learning often lost in the “time starved” by student teamwork grades and also peer review
discussions within physical classrooms. It also has scores, and secondly by students themselves4. The Session 4-6
advantages for the professor facilitating the virtual development of soft skills are stated learning objec-
class. Advantages to learners arise as they have time tives of U21Global’s Masters level programs and it is
to consider the particular issues and to collect thought noted that many authorities in the area regard such
and responses to the debate. It is particularly advanta- development as critical outcomes of good MBA pro-
geous for learners for which English is a second lan- grams. Such examination is also important as one of
guage. All students engage in the discussions, as they the criticisms often levelled at online education is the
are assessable, and this improves learning effective- lack of development of such competencies. The argu-
ness for at least two reasons. First, it helps develop ment is usually that as there is no face-to-face discus-
the learner themselves, and second, all students learn
from each other. Hence, there is a strong element of
Session 7-9
231
Session 8 Misc.
sion in e-learning programs, there can be no develop- was an explicit agreement amongst national QA sys-
Plenary Sessions
ment of higher level competencies like teamwork, tems that they would respect the decisions of any
leadership, negotiations skills and networking. other national or international system.
However, such comments have usually lacked any This is a real challenge for quality assurance agen-
empirical support, and fail to take into account the cies as well as for the educational provider itself.
development of different levels of e-competencies. Similarly, the OECD/UNESCO guidelines of 2005 on
The foregoing discussion correctly suggests that the provision of Cross Border Education do call for
U21Global places significant emphasis on internal more interaction and communication between QA
quality assurance. The elements of this system have authorities. There has been some evidence of initia-
been considered alongside the criteria that both inter- tives to move beyond regional and country quality
Waseda University
national and national quality assurance bodies have assurance towards international or at least multina-
Presentations
identified as important for online education. tional benchmarks. See for example, the move by
At a broader strategic level, U21Global also con- Asia Pacific quality assurers to agree on a common
siders issues such as branding, reputation and market- set of standards within a common framework.5 As dis-
ing through their careful development of internal QA cussed above, new organistaions require external reg-
systems and culture, and through the careful choice of ulatory authorities to ensure quality standards. Given
external QA systems. this argument, then it is reasonable to conclude that
The selection of external QA systems is an inter- without any agreement on the regulation/quality
esting one for a global educational provider like assurance of online education, then it would hardly be
U21Global, and challenges associated with this are surprising to find a large variation in the quality of
DLI Review & Report
discussed in the following section. online providers. Such variation could have those
It is the experience at U21Global that taken whose education is pedagogically rigourous and
together, both internal and external QA systems can effective at one end of the spectrum to those who
help to build an institutional culture in which the offer awards to those applying without much rigour or
excitement of building a quality online educational effort in between!
environment is shared by all staff, in contrast to the In part to overcome this type of challenge,
typically more negative views of QA overviews. U21Global has sought international accreditation
Furthermore, online educational providers that can through both EFMD and AACSB, the two most prom-
effectively implement both systems in their institution inent international quality assurance agencies for
will be able to not only derive value in the form of management education.
Session 1-3
the provider? If so, then should we expect an online port structures which are of more particular concern
institution offering education to a diverse interna- in online education.
tional student body, to be quality assured by every
nation in which it has learners? For a large organiza-
tion like U21Global, this would mean being quality 5
Refer in particular
assured by 40 or so different countries, unless there http://www.brisbanecommunique.deewr.gov.au.
232
Distance Learning and
the Internet Conference 2008
References
Plenary Sessions
Australian Department of Education, Employment
and Workplace Relations (2008), Higher
Education Quality Assurance Principles for
the Asia Pacific Region, 18 February 2008.
http://www.brisbanecommunique.deewr.gov.
au/NR/rdonlyres/F7C48BD9-DA8D-4CFD-
8C6A-914E001E2E39/23073/FinalQAPrinci
ples.pdf
Bedi, K. & Lange, H. (2007). The impact of faculty
Waseda University
interaction on the learning experience and
Presentations
outcomes of online learners. In ICT:
Providing choices for learners and learning.
Proceedings ascilite Singapore 2007. http://
www.ascilite.org.au/conferences/singapore07/
procs/bedi.pdf
Spinks, J. A. (2006) “Emerging Issue in the Quality
Assurance of e-learning Programmes and
Institutions”, keynote address to the
International Conference on Distance
Session 1-3
for the distanced adult learner. World
Conference on E-Learning in Corp., Govt.,
Health, & Higher Ed. 2003(1), 1213-1216.
Williams, J. B. (2005). “Managing retention through
e-learning”. Asian Management Review,
October-December, 2005, 54-59.
Session 4-6
Session 7-9
233