1

An experiment in blended learning:

can education take place without lectures?
Kevin A. Jones1 and Ravi S. Sharma2

1
1
School of Computer Engineering
Nanyang Technological University
Republic of Singapore
2
UC Business School
University of Canterbury
New Zealand

Abstract—Lecturing is synonymous with formal teaching and particularly with higher education, notwithstanding the
overwhelming evidence of the availability and learning effectiveness of other models of instructional design. In a large, public,
technical university located in Singapore, all core courses utilize the lecturing model. However, in one of its schools, an opportunity
presented itself for conducting a course using the blended learning model and to eliminate entirely the use of the lecturing model.
Such a course is a realization of the “teach less, learn more” philosophy espoused by its strategic leadership. It also is a step closer
to a reinvention of the university experience. The results of the four year experiment described in this paper are conclusive: such a
radical learning approach does enhance the learning outcomes of the students. On the other hand, its drawbacks are the increased
effort required of both the students and their teacher, and the inability for some students to come to grips with the multi-modal
blended learning approach because it is at odds with their predominantly lecturing educational experience.

Keywords — educational technology, online learning, instructional design.

I. INTRODUCTION

LECTURING • the transmission of knowledge by a teacher to multiple learners via verbal and visual
signals of the teacher’s devising • is the traditional, widely-used, and possibly instinctive way of
perpetuating knowledge; people are comfortable and accepting of the lecturing model, especially in an
educational setting [1]. Probably for this reason, lecturing is the pre-dominant model of instructional
design in formal learning systems, particularly institutions of higher education [2]. The educators therein
are recognized by the institutional hierarchy as authoritative creators of new knowledge, and that their
raison d’etre is to pass on that new knowledge to the learners. Lecturing and the university seem to have
a symbiotic fit: the professors have the knowledge that their students want, and lecturing has a long history
across civilizations as an effective mode for passing on that knowledge.
Since the Second World War, there has been a proliferation of publications on instructional design and
educational technology [3]. Given the dominance of the lecturing model, one would anticipate that a high
proportion of those publications should affirm its continued use. To the contrary: the proportion of
publications proposing new models of instructional design to replace lecturing is much higher than that
affirming its continued usage. Just how significant is the imbalance in favour of new instructional design
 2

models may be illustrated thusly: in a sampling of 303 relevant IEEE articles2, only 14% affirm
“traditional” lecturing, while another 10% advocate “enhanced” lecturing. The rest endorse entirely new
models.
This clear pronouncement from the educational research community has spurred the emergence of
instructional design models other than lecturing, though more in secondary education [4], and less in
higher education [5]. Even so, the deployment of these new models in the classroom is nearly always as
complimenting or enhancing the lecturing model as if lecturing is the central ingredient of any and all
teaching [6]. The central question is, can any course particularly a university course be successfully
conducted using models of instructional design other than lecturing; that is, use no lecturing whatsoever?
The context of this paper is the Nanyang Technological University (NTU), a large, public university in
Singapore with a focus on science and technology. Its engineering faculty was empowered to adopt any
teaching practice of their choosing; not unexpectedly, it chose the lecturing model, and continues
accordingly to the present day. The authors secured financial support and approval from NTU’s academic
leadership for conducting an experiment to use an instructional design model other than lecturing in an
existing course [7]. If such an instructional design could be accepted by the learners while maintaining
the same degree of academic rigour as with lecturing, then it would be a leap forward to realizing the
national educational philosophy of “teach less, learn more”. Furthermore, freed from the constrictions
inevitably manifest by the lecturing model, of set schedules and venues, and a marginalizing dogmatist
majority, NTU could project its degree offerings to a larger market through the virtual space and
collaborations with other like-minded universities in the MOOC paradigm.
The experiment plan was to convert the instructional design of a semester-long, sophomore course on
software engineering course, from the traditional lecturing to a blended learning mode with the caveat
that lecturing is not retrofit into the new model. The schedule was to run the blended learning module on
four separate semesters from 2008 to 2012. Accounting for the novelty of the blended learning
instructional design, and NTU’s unfamiliarity with delivering an online course, the plan was to pilot run
for the first three years, and then launch the fully operational (canonical) version in 2012. During the
semester breaks, design and performance considerations were to be reviewed and all resultant
enhancements implemented. The canonical version was to be evaluated for learner satisfaction, academic
rigour, and teacher satisfaction.
Why did the authors consider investigating blended learning in the first place? Primarily, blended
learning has generated considerable interest in the educational community with substantial practice-based
evidence that it is a successful instructional design [cf. 4, 11, 12]. Its only difficulty is its varying
definitions probably due to its newness, commonplace vernacular, and the rapid expansion of its “install
base”. The most common definition of blended learning • a mix of face-to-face (F2F) and eLearning

2
The authors conducted a metadata analysis in August 2013 of IEEE journals and magazines from 1945 to present in IEEEXplore Digital Library that are
further filtered subjectively to compare the lecturing model against others in the context of a learning experiment.
 3

instructional designs [13] both having varying definitions • is used for this experiment. The definitions
adopted for this research is that F2F is a humanistic and managed interaction (learners together and
controlled by the teacher), and eLearning is a virtualistic and scaffolded encounter (learner solitarily
explores material with guidance from the teacher). Importantly, technology is not a definitive aspect of
either, for example, an online webcast with a teacher can be F2F, while reading a textbook can be
eLearning. This distances blended learning from the teaching dualism conflict of ‘traditional versus 21st
century’, and espouses the “approach whereby both F2F and online learning are made better by the
presence of the other” [14]. Fig. 1 illustrates the definitive characteristics of F2F and eLearning as
adopted in this experiment.
This paper describes the tasks and challenges faced in converting the course instructional design from
lecturing to blended learning, and subsequently deploying it. Section II covers the experiment approach,
and in particular, the conversion method from lecture-based to blended learning. Section III reviews
learning theories in order to draw on pedagogy that supports our approach. Section IV outlines the design
and development of the new instructional design. The final section concludes with implications and
suggestions for further research in Section V. An annotation of major findings may be found in the
document submitted as “Supplementary Material”.

II. EXPERIMENTAL APPROACH

An earlier-formulated six-step blended learning conversion method [8] was applied to the experiment.
The method is purposefully generic, except for step 1 (specific to the blended learning model), and step
5 (dependent on the pedagogic norms at NTU). The method’s steps are as follows:
· Step 1: Establish the mix ratio – This sets the relative percentage of the two constituent parts in the
blended learning mix. The ratio not only underpins the work breakdown planning of the experiment,
but it situates the course on the F2F-eLearning continuum, which can be the characteristic of the
most interest in management briefings on the new instructional design. Fig. 1 renders the idea of
F2F-eLearning mix ratio with varying percentage values of each.
· Step 2: Reconstruct the pedagogy – This rationalizes the essential principles and strategies for
setting the right conditions for the intended learning. It is both an opportunity and a necessity. An
opportunity because there are a continuum of new discoveries and developments in learning science
and instructional theory that should be taken into consideration in any new course. A necessity
because a new instructional design demands understanding for its effective conditionalization,
especially with respect to its conditions of learning.
· Step 3: Brainstorm the learning activities – This crafts the purposeful sessions that perpetuate the
selected learning theory and instructional design models from the previous step. These learning
activities classify pedagogically and in-practice to either the F2F or eLearning models. They duly
account for the class size, learning space, timetabling, and available educational technology.
 4

Liberated from the confines of the lecture format, countless innovative renditions of new learning
activities may be conceptualised.
· Step 4: Map the learning path – This assembles the basic structure of the entire course using the
learning activities from the previous step. The learning path can be segmented into three temporal
dimensions: semester, weekly, and daily. The semester path includes one-time-only learning
activities like the welcome briefing, midterm assessment, and final assessment, and the aggregate
of all weekly paths. The weekly path includes particular learning activities, and all the daily paths
(except in this course, daily paths are not applicable because no consecutive activities are scheduled
on any single day).
· Step 5: Run pilot and review performance – This is the test run of the new (blended learning)
instructional design. Changing the instructional design is not a routine endeavour in any educational
institution, especially for the case of the design being the first of its kind. It is not advisable to
downplay the exactitude of a new instructional design for a course. In NTU, it is mandatory to
advise the students of their participation in an educational experiment as part of the Institutional
Review Board’s purview of research ethics.
· Step 6: Incorporate enhancements and launch – This is the inauguration of the course with its new
instructional design being fully operational. This should be considered only after conducting at
least a year of a test run review and enhancement. Even after the course is fully operational,
enhancing individual learning activities and incorporating new pedagogy is perpetual.

The detailed conduct of these six steps form the basis of the next three Sections of the paper, steps 1
and 2 for Section III, steps 3 and 4 for Section IV, and steps 5 and 6 for Section V.
The experiment’s workload was expected to be similar to creating a new course from scratch. The
learning resources of the lecture model would not migrate to the eLearning model. They would have to
be not mere digital versions but virtualizations that are very expensive time-wise to build [9]. There were
no other faculty available to assist in the experiment, so the entire development effort for the experiment
rested with the teacher delivering the course. In a similar experiment where lectures were also replaced
in their entirety, a team of four persons was assigned to the project [10]. Hence, the four year duration of
the experiment, of which the first three repetitions were essentially pilot runs, was in hindsight, rather
ambitious.
While this experiment was without precedence in NTU, similar ones have been conducted in other
universities. These served as a reality check and expectation baseline for this experiment. The closest in
overall context and conduct is the University of Rijeka’s blended learning information science course
with lectures significantly reduced [15]. In Technical University of Madrid’s computer science
undergrads contrasting the performance of blended learning against a distance learning model [16]. The
closest experiment in terms of eliminating lectures is in the University of Utah, lectures for electrical
 5

engineering students were replaced with structured sequences of 5 minute videos, F2F reviews, and small
group work [17]. Finally, there was the earlier referenced experiment of eLearning totally replacing
lecturing for a foundational psychology course in North Dakota State University [10].

III. LEARNING THEORY AND INSTRUCTIONAL DESIGN

A. Step 1 – Mix ratio

There is no optimal ratio of F2F to eLearning; the instruction designer may conditionalize the mix
however he/she sees fit. The mix ratio set for this experiment is 60% F2F and 40% eLearning, derived
from an analysis of the learner’s course timetable, study culture, and workload. The aggregate of the F2F
activities could not exceed the university norm of six hours scheduled in the learner’s weekly timetable.
is one hour preparation for every one hour of lecturing — the undergrad’s unofficial and accepted study
culture — could not be exceeded by the eLearning activities in the blended learning model. Finally, a
standing dictum of SCE management is to optimize the learner’s workload, in particular to teach less. As
eLearning is proclaimed to be as good, if not better in learning as lecturing [cf. 10, 18, 19], it is reasonable
to replace some of the class work with eLearning. Accordingly, the class work is reduced to five hours;
see Table I for the final tally of the course’s assigned time per week, 5 hours of F2F and 3.5 hours of
eLearning. With the final emphasis being greater for the F2F model, the Dean of the college provided
the senior management support for the experiment3.

B. Step 2 – Rethink the pedagogy

The course pedagogy is decomposed into two distinct foundations: learning theory and instructional
design. Learning theory is a hypothesis of how learning occurs, based on a rigourous interpretation of
behavioural, cognitive, social, and epistemological foundations and research. Instructional design is the
realization of instructional theories, strategies, and procedures into formal educational interventions
typically supported by technology. While distinct, the theory and design do not exist in strict isolation.
They are linked by articles of good learning, that is, principles or conditions that inform sound learning
theories, and occurrences or events in a particular model of instructional design that result in solid
learning. Gagné [20], the acknowledged leader in this field, identifies an ordered set of nine learning
conditions and another set of learning events universal to all educational interventions that promote good
learning, and asserts that there is a one-to-one correspondence between the two sets. The correlation of
conditions and events is shown in Figure 2.
To rethink the pedagogy, one must understand how people learn (learning theories), establish the pivotal
learning strategies (realizing the learning theories in the context of the course), associate the learning

3
During an informal event for faculty at the College of Engineering, the first author discussed the blended learning format in his subject. Although
supportive in principle, the incumbent Dean emphasized that the delivery could not be all eLearning. He shared his vision that a key aspect of the university
experience was the traditional F2F discourse between the professor and the student, and in the management’s opinion, it is absolutely essential for NTU to
retain that for some part of the four years of the degree process. This is not dissimilar to the one recently espoused by Dr. Condoleezza Rice of Stanford on
the subject of MOOCs.
 6

conditions and events with the strategies, and then establish the instructional design models that best
support and project the learning strategies and events.
Of all the mainstream learning theories, the one that underpins the majority of the teaching and learning
of engineers in NTU is cognitivism [21]. This theory postulates that learning is an internal process of the
mind and the end-result of mental processing. The significance of this seemingly obvious statement is
clearer when juxtapositioned to behaviourism, the older theory that cognitivism supersedes.
Behaviourism asserts that learning is behaviour modification brought about by conditioning using positive
or negative reinforcement, and meaningful feedback. It is a dominant theory, but it is unable to explain
complex learning, including the permeation of knowledge from precept to concept, and the formulation
of ideas that are not directly reinforced. Cognitivism explains these by envisioning learning as a pervasive
network of memory units containing various sorts of information. The learner is provided with facts and
concepts pertaining to a new “increment” of knowledge, and continuously relates that with previous
memorized increments through the employment of innate mental functions. With time, enough
knowledge is accrued until the appropriate extent of knowledge is reached. Resultantly, cognitivism
belongs to the category of traditional learning because it commits to one version of knowledge,
transmission of that knowledge by an authority, and uniform internalization of that knowledge by learners.
The criticism of cognitivism is that it does not cater for creativity, learner differences, and cross-
disciplinary thinking. Metaphorically, a learner’s knowledge of a discipline is a silo, and to understand
the full extent of the discipline, the learner must fill the silo to capacity with facts, scenarios, and
examples. The aim of this experiment is to go beyond the traditional learning with a more progressive
theory.
A progressive learning theory that supports creativity, learner differences, and cross-disciplinary
thinking is constructivism [21]. It postulates that knowledge is created by the learner, and is a perception
of reality based on the learner’s prior knowledge and extrapolation of meaning. This perception is not a
subversion or substitution of reality, but a conditionalized interpretation. The learner is provided with
facts and concepts (as in cognitivism) but the learning environment is systematized to activate learning
transfer, that is, an application of knowledge to new scenarios and not its mere accrual (as in the ‘silo’
metaphor). A different metaphor applies to constructivism: knowledge is a stone given by the teacher to
the learner who carves it into a sculpture that is a perception of personal reality. All learners start with
similar stones, so each fashions a sculpture of similar dimensions but with perceivably unique aesthetics,
and taking different times to complete. In constructivism, learners can transfer their knowledge once
sufficient fluency (optimization and normalization of cognitive connections in their new knowledge) has
been applied, and their knowledge is neither too contextualized nor too abstract. In other words, transfer
can be conducted by novices (by trial and error) as well as subject matter experts (by adaptive inference
and patterning). Constructivism is underpinned by metacognition, where the learner conducts a reasoned
exploration of the thinking and learning involved with the objective of improving his/her learning.
 7

Finally, constructivism is adaptive to solitary as well as social contexts involving peers, teacher, and the
macrocosm of the learning environment. The use of a case study approach to understand deeper
constructs and contexts of a socio-technical system is an example [22]. The knowledge that is constructed
in the social environment contains the added dimensions of diversity of experiential and contextual
history, and the synergistic interplay of language. This is the kind of learning that the teacher in the
experiment intends to provide the learners.
A closing consideration with respect to learning theory concerns blended learning. As the chosen
instructional design for the course, its learning theory must be included in the dialog for integrity sake.
This is challenging because the learning theory supporting blended learning is elusive. However, there is
a promising candidate: variationism or learning by discernment or contrast [23]. This theory stipulates
that a learner contrasts a new experience against a previous one, and derives knowledge about the new
experience from the perceived differences between the two. Accordingly, this learning theory is included
into the rethought pedagogy.
Learning is best served by drawing from all of the above learning theories, and not restricting itself to
just one. All have relevance in the kind of learning scenarios expected in higher education. However,
the emphasis of the new instructional design is squarely on constructivism. So, the teaching and learning
strategies for the experiment promote mostly the constructivism learning theory, and to a lesser degree,
behaviourism, cognitivism, and variationism. The formulated strategies are:
· Build a competency-based foundation – The learner creates an integral knowledge baseline
comprising new facts and foundational concepts. Early in the course, the learner’s preconceptions
about the subject are activated and if necessary corrected; it is essential for the learner to be aware
of and subsequently abandon incompatible prior knowledge.
· Facilitate knowledge fluency – The learner’s easily and effectively retrieves prior knowledge and
correlates it to all immediate learning. This does not imply that the learner is an expert. It does
mean that the learner organizes his/her knowledge with some intuitive associations of relative
meanings, patterns, and situational metadata.
· Emphasize learning in teams – Teams of learners are formed early in the course to facilitate their
organic growth in effectiveness and productivity. The team size is optimized to ensure individual
members have a reasonable amount of workload, while having sufficient available time to sustain a
healthy degree of discourse with the other members. In other words, the team arrangement is
intended to leverage the unique strengths of its members via interaction, and then share equally the
undertaking of the work products.
· Sponsor learner metacognition – The learner sets his/her learning goals, and monitors his/her
progress in achieving them. The intention is for the learner to learn how to learn using the new
instructional design; this includes learning teamwork. It also involves the active guidance of and
 8

monitoring by the teacher to incentivise the learner.

· Encourage learning transfer – The learner develops a culture of thinking out of the box, and there
being no single right answer. In addition to espousing quantity, productivity of the learners also
embodies creativity and risk. The intention is to provide less standard examples and solicit more
application by the learners of foundational knowledge in unexplored scenarios to create new
knowledge. Such new culture requires continuous reinforcement as the imperatives of conformity
and right-wrong have been drilled into the learners throughout their entire schooling [24].
· Engage with contrasts – The learner employs informative contrasts for developing concepts and
stimulating interest. In this experiment, these contrasts explore a longitudinal rather than a true-
false perspective of the subject material, and are both theoretical and real-life. The learner is not
required to make a relative value judgment; however, he/she is required to understand the relative
situated utilizations. This supplements the ‘foundation’ and ‘fluency’ strategies.
· Enhance the marking paradigm – The proportion of the student’s course mark coming from the final
examination is decreased, and from continuous assessment is increased. More of the assessment is
formative and less is summative. This strategy is cogent with the metacognition and team strategies.
These strategies align with one of the most cited • over 10,000 citations • handbooks on learning [25]
and correspond in varying combinations with all of Gagné’s learning conditions, for example,
‘reinforcement’|’providing feedback’ informs the ‘fluency’, ‘team work’, and ‘transfer’ strategies. See
Table II for a complete specification of the correspondence.
The final activity in rethinking the pedagogy is to select suitable instructional design models for F2F
and eLearning models. There are numerous instructional design models [26], all with strengths as well
as weaknesses; how then does one choose? The authors recommend that the models meet the following
criteria: long history (which usually means that the supporters outweigh the detractors), continued backing
by the model’s founder, workable in F2F and eLearning encounters, and apply to as many of the learning
strategies as possible. The chosen instructional design models chosen are active learning, collaborative
learning, and problem-based learning (PBL); see Table II for the correlation of learning theory to
instructional design model.
The active learning model encompasses all learning theories, and has the learner performing actions
that cohere to his/her learning process [27]. It can be as simple as pausing during a lecture for the learners
to conduct a review and clarification of notes with a partner, or as complicated as a heavily scripted role-
playing scenario. In fact, active learning encompasses such a wide variety of actions, that in practice, it
is easier to define by exclusion (what it is not). For example, it is not a lecture. Active learning is often
affiliated with F2F, but there is no stipulation that it cannot be affiliated with eLearning. Also, the mere
existence of activity does not mean active learning; the activity must relate to the learning intended by
the teacher. For example, homework is clearly active, and should promote learning, but will not if the
actions are not consistent with the curriculum. In the classroom, actions are harmonized with the desired
 9

learning via direct intervention of the teacher. For this outside the classroom, the teacher relies on
scaffolding. Active learning can be conducted with the learners as individuals or in teams. The computer-
based classroom is an ideal environment for the use of active learning, for which several experiments
have been conducted [28].
The collaborative learning model promotes cognitivism, constructivism, and variationism, and has the
learners in groups working towards a common objective [29]. Collaborative learning can be active, and
active learning can be collaborative, so how are the two differentiated? By target! Active learning targets
the individual learner; even when in small groups, the learning is directed at and confirmed for
individuals. On the other hand, collaborative learning explicitly targets the group, and the learners are
not considered as existing separately from the group. This leads to the contention of accountability of the
group members. If the learners in a group are accountable only to themselves, that is, they share their
knowledge but are assessed individually, this is cooperative learning. If the group members are
accountable to each other because they are obliged to participate equally and as equals, this is
collaborative learning. Nonetheless, how to mark a collaborative group is still being debated; the most
recent research indicates that formative not summative assessment be used, and that the learners in the
group focus on their objective and not their reward (which is typically a mark) [30]. Being common
practice in engineering education, collaborative learning is applied to the University of Rijeka blended
learning experiment [15], and it is also applied extensively in the use of online and multimedia learning
resources for undergrad courses, for example a course wiki [31].
The PBL model also promotes cognitivism and constructivism, and has the learners in groups, working
in a classroom environment towards a common objective, but with two differentiations: very precise and
immutable process, and specialized role of the teacher [32]. The process is initiated by the release to the
group of an ill-structured problem. The group develops or refines the problem, conducts interdisciplinary
research and analysis, evaluates the alternatives and arrives at an answer, and lastly, provides their
solution to other peer groups in the learning session. The teacher’s role is subject matter mentor • not
expert nor authority • and metacognitive coach. PBL is “apprenticeship for real-life problem solving”
[33]. In other words, the learning value emerges from the group’s approach to interdisciplinary
exploration, so the success of PBL depends greatly on the quality of the problem and the coaching acumen
of the teacher. Originally applied to graduate medical education, PBL is now used extensively in
engineering education for network design, embedded systems, mobile applications, and communication
systems [34]. PBL is also applied in the University of Rijeka blended learning experiment [15].

IV. COURSE DESIGN AND DEVELOPMENT

A. Step 3 – Brainstorm the learning activities

A learning activity is a set of physical and mental actions tailored to promote learning in accordance
with a particular instructional design model and within a reasonably fixed amount of time. It is a
 10

manifestation of an instructional design model, and is structured, scripted, sequenced, and managed or
scaffolded. Clearly, the pedagogical and contextual • as in the university ethics and undergraduate
experience • success of the converted course depends largely on the integrity of formulated linkages
from learning theories through strategy and instructional design, to learning activity. Regrettably, there
are no tools or quality assurance processes to boost the rigour of the migration process of “theory to
activity”. It is dependent exclusively on the experience, creativity, thoroughness, and eruditeness of the
teacher involved.
There is an abundance of learning activities • for instance, book report, case study, debate, essay,
journal, modeling, one-minute paper, panel discussion, pop quiz, practice, presentation, role play, thought
experiment, and web search • in numerous combinations appropriate for higher education learners.
Unquestionably, the learning activities must be monitored in their inceptive runs to determine which are
under performing or are in conflict with the contextual environment. One dependable way of increasing
the performance of any learning activity is to design it to espouse as many as possible of Gagné’s learning
conditions and events while not incurring incoherence.
After much deliberation and “what-if?” analysis, eight learning activities were developed for the
experiment. In the following alphabetically-order outline, each activity starts with a descriptive tag of
“estimated hours duration | learning strategy | design F2F or eLearning | realized instructional design
model | purpose of activity”, followed by a narrative describing the activity as it appears in the canonical
version of the course. The estimated durations are on a weekly basis, and correspond to those for F2F
and eLearning in Table I. The more complicated learning activities are explained with Unified Modeling
Language (UML) Activity diagrams.
· Bedrock – 2 | foundation and contrasts | eLearning | active learning | constructing the knowledge
base of the course content, one topic at a time. The activity is a contiguous online sequence
conducting the following actions (in order): engage learner, multiple choice question (MCQ) quiz
of prior knowledge, explore topic, survey textbook, and quiz of new knowledge. Learners scoring
9 or 10 in the prior knowledge quiz are placed into the master stream where the topic content and
exercise is more challenging, and the reward is slightly higher. The remainder follows the
journeyman stream where the topic content is more basic and the confirmation assessment is a repeat
of the earlier-taken prior knowledge quiz. The contrasts are built into the sequence, where the
learner is steered to one of two streams (journeyman, master) depending on the degree of prior
knowledge of the topic, and offered up-to 12 kinds of media (including slide, speaker note, handout,
textbook, chapter questions, journal paper, and video) representing the topic content. The teacher
tracks the performance of every learner for the entire sequence. Following Gagné’s learning events
closely, the online sequence flow is: attention, objective, prior learning, stimulus, guidance,
performance, assessment, and retention. See Figure 3.
· Conspectus part one – 1 | fluency and team | eLearning | active and collaborative learning | clarifying
 11

and reinforcing the newly established knowledge base. After constructing an initial knowledge base
for a topic, the learners join their team to share their understanding of the topic and build consensus
on any uncertainties or conflicts. Next, the team answers a question concerning an informative
aspect of the topic, and crafts a presentation to share that answer with the cohort; another team is
working concurrently but in unanimity on the same problem. Finally, each learner individually
submits a written summary of the topic that is not greater than 250 words. The teacher uses these
submissions to identify any deficiencies in the topical learning, and addresses them later in the
review activity. This delay in teacher’s feedback is offset by the immediacy of the team’s; such is
the strength of collaboration. The learning events for this activity are: prior learning, guidance,
performance, feedback, and retention.
· Conspectus part two – 1 | fluency, transfer, marking, and contrasts | F2F | active learning |
regularizing each learner’s newly constructed knowledge base. One or more learners from the team
present the problem and their answer to the cohort. Immediately after, the other team with the same
problem gives their presentation. The cohort evaluates the soundness of each answer against their
own understanding of the topic; in other words, the two teams are not competing with each other,
but are being contrasted with the understanding of the cohort. After each team pair is peer evaluated,
the teacher provides supplemental assessment of the session. The learning events for this activity
are: attention, objective, prior learning, stimulus, performance, feedback, assessment, and retention.
In Figure 4, parts one (commented as “eLearning”) and two (as “F2F”) are merged. Although it
seems simpler to execute than the bedrock activity, the conspectus is quite complex in its intended
learning and assessment.
· Project – 2 | transfer, team, and contrasts | F2F | active and collaborative learning | applying prior
knowledge from multiple disciplines to resolve a large-scale realistic problem. The learners work
in teams to build software subsystems. Concurrently, the teams integrate their subsystems into a
single coherent system. This re-envisioned software engineering project requires the learners to
apply hard skills (modeling, programming) and soft skills (negotiation, critical thinking) in a multi-
disciplinary encompassment. The teacher works with the learners to guide their learning and
manage the overall construction of the software system. This activity espouses all nine of Gagné’s
learning events.
· ReflectionIndividual – 0.1 | metacognition | eLearning | active learning | cultivating introspection
by the learners concerning their progress. The learner submits a written value judgment of their
learning and the factors influencing his/her learning. The teacher tracks the learners’ submissions
and intervenes privately with any learner whose progress appears impeded. The learning events
are: attention, objective, and prior learning.
· Review – 1 | marking and contrasts | F2F | active learning | providing meaningful feedback to learner
 12

of his/her achievement. The learner undertakes a 10 question MCQ using an audience response
system, and the teacher follows with planned and ad-hoc exercises to clarify any misunderstandings
apparent from the quiz and the earlier conspectus part one topic summary submission. The learning
events are: attention, objective, stimulus, performance, feedback, and assessment.
· Seminar part one – 1 | fluency, team, and transfer | F2F | PBL | constructing new knowledge about
the topic. At the commencement of the session, the team is assigned a challenging topical problem
not seen before, and proceeds to analyse, answer, and present their solution within the period. Each
presentation ends with a brief peer Q&A followed by the teacher’s wrap up. Gagné’s learning
events in this activity • prior learning, guidance, performance, feedback, and retention • are more
intensified due to the time pressure.
· Seminar part two – 0.5 | fluency | eLearning | collaborative | building a collective online knowledge
repository for the cohort. One learner from every team uploads their team’s problem and solution
into the course wiki. Thereafter, every learner peruses the wiki, and challenges the soundness of
other study teams’ posted solutions and/or defends the soundness of his/her team’s own solution.
The teacher tracks the comments, and supplements the online discourse to provide additional
learning considerations. The learning events are: prior learning, feedback, and retention. In Figure
5, parts one (commented as “F2F”) and two (as “eLearning”) are merged.

B. Step 4 – Map the learning path

The learning path in the weekly dimension summarizes Gagné’s learning conditions and events into
three phases: initialize, take possession, and assure quality. Initialize provides the extent of curricula
content promoted for learning. Take possession has the learner enhancing his/her fluency and building
retention. Assure quality has the learner transferring his/her knowledge to the course project, reflecting
on his/her learning, and receiving final feedback to confirm the completeness of learning. The weekly
learning path model (Figure 6) is an adaptation of the Spiral diagram [35]. Each trirant represents one
phase, and each revolution, one week’s content. The numbers inside the revolution represent one week.
One revolution begins from the vertical axis, traversing left, down, and around until it returns to vertical
axis. When a revolution ends, another begins immediately thereafter. In this experiment, there are 13
revolutions in the ‘weekly model’, one for each week in the course.
The curricula content is organized and learning activities sequenced according to the weekly learning
path. The teacher apportions the content to the model in weekly chunks that cognitively “flow” from one
to the next, and that are not too much to overwhelm the weaker learners, and not too little to disappoint
the stronger learners. From an analysis of their characteristics, the weekly learning activities and phases
are related as follows: bedrock (in initialize), conspectus and seminar (in ‘take possession’), and project,
review, and reflectionIndividual (in ‘assure quality’); as shown in Figure 7. The “trident” symbol
indicates the activity comprises several actions and is detailed in an earlier diagram.
 13

The learning path milestones for the semester dimension, that is, first day of classes, midterm break,
last day of classes, and final exam, are all fixed by NTU. For this experiment, only the necessities of the
experiment • orientation, enhanced assessment, and the team reflection activity • need be super-
imposed onto the semester learning path; this is shown in Figure 8. There is no weekly learning path
during the midterm exams and the team reflection and project assessment. There is a study period
separating the last day of classes and the final exam.
· ExamFinal – 2 | marking and transfer | F2F | active and collaborative | solidifying the learner’s new
knowledge in two ways. Firstly, its weighting is only 40% of the course total mark, far below NTU’s
anecdotal norm of 60% or more, thus decreasing the anxiety that it would otherwise generate.
Secondly, its question set is bound to a single, life cycle style scenario that is released three weeks
prior to the exam date. By design but not direction, this becomes a group learning project, the last
of the course. The learners study the scenario, formulate possible emergent questions and the
answers to them, and then (optionally) discuss with the teacher for feedback; they set their goals and
control the process. The learning events are identical to the conspectus.
· ExamMidterm – 2 | marking and contrasts | F2F | active learning | providing feedback on the quality
of learning midway. There are two contrasting exams • MCQ and short text-diagram answer •
each taking one hour for conduct. The learners receive their marks quickly by virtue of an automated
scoring function in the technology. The learning events are: feedback, assessment, and retention.
· Orientation – 1 | foundation and metacognition | F2F | active learning | engaging and preparing the
learner for the innovative learning and new instructional design. It is conducted on the Friday before,
and repeated on, the first day of classes to reinforce the learning. The learning events are: attention,
objective, and guidance.
· Project assessment – 2 | transfer and marking | F2F | active learning | introducing the learner to the
IT commercial process. The entails a demonstration of the software system by the learners that is
judged by a team of teachers including an external examiner. All nine learning events are in action.
· ReflectionTeam – 1 | team and metacognition | F2F | collaborative learning | building consensus on
the performance of the new instructional design in the course. In a team setting, the learners engage
in a free-form and privileged-platform discussion on the new instructional design, progressive
pedagogy, and impacts of these on the learning experience. From the discussions, the teacher garners
lessons learned and implements the appropriate changes to the course between semesters. The
learning events are: attention, objective, prior learning, guidance, performance, and feedback.

C. Technology
In this experiment, the learning activities utilize commercial, off-the-shelf (COTS) educational
technology readily available in NTU and some custom-built software. This is primarily to avoid the
technology itself from becoming a significant obstacle to other teachers performing the experiment on
 14

their own courses [36]. The custom-built software is programmed with user-friendly java script and
HTML. The learning technologies used are as follows:
· Audience response system (COTS) – Its hardware is RF transmitters and receiver, and its software
integrates with PowerPoint (ppt) and Excel (xls) office applications, respectively to capture and
score learner responses. Different instances of ppt-xls are required every week for these sessions:
orientation, seminar, and project (for attendance taking), ‘conspectus part two’ (for peer scoring),
review and midterm first quiz (for answering MCQ), and midterm second quiz (for displaying each
question and its elapsed time). To heighten student engagement, evocative pop culture images
comprise the backgrounds in the slides. Over 250 slides are now in service.
· AcuLearn (COTS) – This creates and delivers video recordings of on-campus F2F sessions as one of
the 12 kinds of media offered in the bedrock activity.
· Blackboard (COTS) – This manages and delivers the 5GB of course content to the learners. The
native functions of adaptive release, building blocks, connect, discussion board, email, and group
management are used extensively.
· Content management software (custom built) – This provides breadth-wise and depth-wise access to
the content. The navigation is by week (numbered tab), then by active link to the bedrock learning
activity and its content items. In Figure 9, the selection is tab “1” for “Week 1”. Access is
unrestricted calendar-wise: the learner may access material from before or after the current week
which is the highlighted reference.
· Course management software (custom built) – This webpage is the front-end of the course, providing
essential information and access to content, schedule, and wikis. The navigation is by aspect (titled
tab), then by active link to aspect item. In Figure 10, the selection is tab “Introduction”. All aspects
are available from day one to the learners.
· Help online software (custom built) – This is the scaffolding for the new instructional design to give
learners full self-sufficiency. It includes downloadable material on learning theory, the new
instructional design planning and proposal, user guides, fixes to common software faults, and weekly
progress marks. In Figure 11, the selection is tab “How do I study”.
· Learning Activity Management System (LAMS) (COTS) – This sequences a wide variety of learning
actions and controls every learner’s progress through the sequence. Different instances of sequences
are required every week for the bedrock and conspectus part one activities. Over 30 sequences are
now in service. Each sequence is rendered as a digraph, where the nodes are activities and the edges
are one-way paths. Figure 12 shows the bedrock activity’s sequence.
· Messenger (COTS) – This provides instant rich media messaging for real time mentoring of learners
doing the activities bedrock and conspectus part one, which are eLearning and require an online
solution.
 15

· Streaming video (COTS) – This provides online viewing of videos from public sites such as
YouTube, and private sites such as Blackboard, as one of the 12 kinds of media offered in the bedrock
activity. This software must be capable of handling multiple simultaneous requests for viewing of
any sized video file.
· Wiki (COTS) – This manages and delivers the cohort’s online knowledge repository, with a different
instance for the conspectus part one and seminar part two activities. Two critical functionalities for
this software is keeping a history of versions, and handling multiple edit requests with no information
loss.

D. Steps 5 & 6 – Review pilot performance and launch blended learning course
To recap, the objective of this research was to convert the instructional design of a semester-long,
sophomore course on software engineering course, from the traditional lecturing to a blended learning
mode. This is a significant goal to achieve because the viability of innovative instructional design within
the higher education community in Singapore is woefully under-reported.
Achievability of the development goal depends on the efficacy of the new instructional design, and
possibly the culture of the learners. The efficacy of blended learning has been investigated in numerous
reports of which the majority found that it is measurably more effective than, and even online learning
alone offers a modest advantage over, lecturing [cf. 11, 12, 37]. In other words, the experiment is not
starting from a position jeopardized by a dubious instructional design model. Nevertheless, its novelty
has risk that is associated with the acumen of the teacher and his/her learners. In other words, the
development goal depends not on the blended learning model in general, but on the quality of the
particular instructional design implementation, and its teacher and students. The only instrument
available to the authors to track this effect is the ‘student feedback on teaching’ (SFT) index.
Some studies suggest that culture has an influence on how a person learns [cf. 38, 39]. Generally, the
computer engineering cohort at NTU is culturally homogeneous: over 98% of its learners are Asian, of
which about 70% are from Singapore, and the rest are foreign nationals mostly from China, India,
Indonesia, Malaysia, and Vietnam. The referenced studies identify a strongly engrained cultural influence
on ethnic Asian learners: deference to teacher authority, anticipation of following concrete learning paths,
and control of learning being the sole purview of the teachers. In other words, with the acquiescence of
the Asian teaching ecosystem, the Asian learners are “hand-held” throughout their formal education. As
a result, the teacher in the experiment needs to establish an overt presence in any online activity, and also
be cognisant that such compensatory effort may be totally foreign and thus “lost” to the Asian learner.
To this end, Internet messaging (IM) service with the teacher is instituted as the presence-provider in all
eLearning because of its ease of use and wide-spread familiarity with today’s undergrads.
The findings’ data is collected from two periods: lecturing period (2007-08, the last academic year
before the experiment), and experiment period (2011-12, the canonical version of instructional design).
 16

Each period is partitioned into two semesters: ‘semester one’, for the re-takers and special programme
cohort, and ‘semester two’ for the normal programme cohort. Correspondingly, the learner count for
‘semester one’ is always less than for ‘semester two’. The two semesters of each period are subjected to
an archetypical test-retest reliability confirmation. There is also a reduction of entrants from 2007 to
2012, but this is believed to be a societal softening of interest for the engineering fields that is irrespective
of the experiment.
Quantitative data (grades) collection is continuous across both periods; the qualitative data
(questionnaires) is only from the experiment period. Data reliability is improved by two necessary
concessions. First is normalization of the fundamental bias in all grades • they are subjective with a
veneer of empiricism cast by the grader’s expertise and authority • by using only those generated by the
teacher; values moderated by the SCE committee are not used. Second is optimization of the consistency
and reliability of the questionnaires by conducting a three year field trial, and eliminating their “snap-
shot” isolation and contextlessness. The learners are apprised in the ‘orientation’ learning activity, of the
experiment data collection process being contiguous for the 13 weeks, and have real-time access during
the course to all data as it is tabled.
In 2010, NTU’s newly established Institutional Review Board approved the experiment. In accordance
with the agreed terms of that approval, during the ‘orientation’, the learners sign a form agreeing to
participate, and to the use in the approved manner of all data generated in the experiment. This required
that every change in the instructional design and learning activities was to be recorded in an internal
design document titled ‘Proposal for Converting CPE207 from Lecture-Based to Blended Learning’.
Major findings of the field experiment have been recorded in a document submitted as “Supplementary
Material” to this paper.

V. IMPLICATIONS AND CONCLUSIONS

Though blended learning may appear to the uninitiated as too ahead of its time • both technologically
and pedagogically • for an Asian learning ecosystem to make meaningful use of, the authors conclude
otherwise.
Significant among these is the conclusion that Asian learners, typified by undergraduate engineering
students at NTU, do not need lecturing to learn. Findings 1 to 10 clearly indicate that almost all the 2nd
year computer engineering learners accept the particular blended learning with no lecturing. They are
motivated to learn more, they “feel” they have the necessary support and visibility concerning their
learning progress, they “feel” they are learning better, their grades are more consistent and modestly
higher, and they consistently attend all the F2F activities. The learners recognize and value the additional
control, flexibility, and freedom they have in their learning. In comparison to lecturing, the learners in
the particular blended learning are exceptionally satisfied. Only 5% of the learners feel that lecturing is
 17

an indispensable instrument in their learning! The implication of this alone should be reason enough for
any higher education institution (including global ones) to explore other instructional design models.
Another significant conclusion is that blended learning is entirely transferable to Asian higher
education. Findings 1-10, 14, and 18 show better academic performance, more satisfied learners,
substantial demand of time and effort to both develop and operate blended learning, same breadth but
more depth of learning. These findings exactly mirror all those “western” higher education studies on
blended learning and elimination of lecturing referenced in this paper. Clearly, a blended learning
instructional design is equally at home and entirely relevant in both oriental and occidental learning
ecosystems.
When designed effectively, blended learning offers a significantly gratifying teaching experience as
evidenced by findings 13, 17, and 18. Having learners “feel” they are learning is definitely satisfying for
the teacher as it fulfills the primary goal of the profession. However, an additional and more profound
satisfaction for the teacher emerges from his/her interactions with the learners, especially if these reach
aspects of the subject that are too challenging for most learners to grasp. The authors believe that the
highest compliment a learner can pay his/her teacher is to initiate a valid dialog on some difficult issue
that the teacher has to “think about”. Why? Because teachers are learners themselves! Blended learning
provides the opportunity of advanced interaction to a much greater extent than lecturing because more
learners have a heightened awareness of the course content when present in the F2F activities. Most
learners do not prepare in advance of taking their lectures [2], but it is a distinctly different proposition
with blended learning! The learners complete their eLearning, make the cognitive connections to prior
knowledge, overhaul their knowledge fluency, and even attempt a transfer or two, all in advance of the
F2F activities. From this state, it is just a matter of teaching acumen to deftly elicit and fully exploit this
knowledge from such primed learners.
The university must lead a blended learning initiative as can be seen from findings 4, 11, and 14-16. A
large body of research on the relationship of educational innovation and the host organization
substantiates the necessity for the organization’s encompassing endorsement and support. The sphere of
influence of the teacher is much more limited than the college’s. It is also confusing for the learners if
their learning experience is different than their peers for the same degree qualification. The learners and
teachers equally need from the school, regardless of the extent of its autonomy in the university, a coherent
vision with process and resources support [40]. The purview of the school includes the sequence and
duration of periods in the weekly timetable, class size, classroom allocation, technical support, teaching
assistance, visibility at year coordinator briefing, final exam weighting, and coordination of innovation in
all years of the degree programme, all being factors in the performance of an instructional design. The
experiment is a case in point. ‘Seminar’ and ‘project’ periods were sequenced before ‘conspectus part
two’, and for all F2F, class sizes were too big and the one hour duration was too small. Notwithstanding,
the most significant letdown is the university’s refusal to enact a moratorium on the SFT for the duration
 18

of the experiment. The 3-point drop may not seem significant, but in an excessively data-driven
institution it could be very damaging to an untenured teacher’s employment. Given the isolation and
vulnerability of the teacher due to the university’s lack of support, the experiment was in hindsight, a
pyrrhic victory.
Asian undergraduate learners are studious and committed enough to perform successfully in eLearning.
Findings 6 and 17 (learner initiative in final exam scenario learning, and the corrective action when
struggling) indicate that there is more to the Asian undergraduate learners than what the published
research suggests as being conformist and unable to be self-directed. Although the typical Asian learners
are “spoon-fed” and “hand-held” throughout their entire formal education, it is possible that this is more
a consequence of nurture rather than nature! For this possibility, the authors are seeded with a new, higher
appreciation of the potential of Asian undergraduates.
The path in blended learning is organic not linear. To satisfy bureaucratic requirements, the learning
paths in NTU’s undergraduate degree programmes all are basically linear in nature. In the particular
blended learning, it is evident that the learner has significant latitude in not only when and how he/she
will partake in learning sessions, but what content he/she will focus on; see findings 10, 13, 17 and 18.
This might seem disconcerting at first, because of the perceived risk that the learners may go off on a
tangent and miss the point being made. But, so long as the learner knows the learning objectives for the
course and is able to demonstrate that they achieve them, it should not matter when, how, and why they
got there! In other words, the learner has control over his/her personal learning journey in which every
step is decided upon when it needs to be taken, all the while keeping the end point in view; this journey
is organic, not linear [41].
The authors submit there are several open questions that can point the direction of further research in
innovative instructional design including blended learning. What is the real scholarship of Asian learners,
are the findings of this experiment transferable to other disciplines, which is the best direction for Asian
higher education to take in innovating instructional design, and how can the Asian higher education
ecosystem maximize the potential of blended learning so that it is a part of the new education practices
of the 21st century? Are there significant dilemmas [42] that must be avoided? Such research will be
confronting contentious areas such as lifelong learning, knowledge and content curation, conservativism
and societal norms in learning, intransigent beliefs in myths, and the domination of accreditation.
One area of particular interest technologically is a repository of learning resources that can accept
entries from post graduate students, other teachers, and even industry thinkers and leaders. Aided by an
inference engine to apply ontological and pedagogical benchmarks to the entries, such a repository would
be a powerhouse in maintaining the connection of education and real-world, a connection that future
generations of learners would only find more difficult to make. Notwithstanding the onslaught of MOOCs
and initiatives that question the value of a college education, the academic community needs to keep in
 19

mind that education is a changing journey, not a prescribed destination. "If we teach today as we taught
yesterday, we rob our children of tomorrow." [43]

REFERENCES

[1] B.G. Charlton, “Lectures are an effective teaching method because they exploit human evolved 'human nature' to improve
learning – Editorial”, Medical Hypotheses, vol. 67, pp. 1261-5, 2006.
[2] G. Gibbs, “Twenty terrible reasons for lecturing”, SCED, Occasional Paper No. 8, Birmingham, 1981.
[3] R.A. Reiser, “A history of instructional design and technology: Part 1: A history of instructional media”, Educational
Technology, Research and Development, vol. 49, no. 1, pp. 53-64, 2001; and, Part 2: A history of instructional design,
Educational Technology, Research and Development, vol. 49, no. 2, pp. 57-67, 2001.
[4] M.B. Horn and H. Staker. (2011, Jan.). The Rise of K-12 blended learning. Clayton Christensen Institute for Disruptive
Innovation, CA. [Online]. Available: http://www.christenseninstitute.org/.
[5] A.C. Applebee, K. McShane, S.D. Sheely, and R.A. Ellis, “Balancing act: How can universities recognise the scholarly
nature of eLearning development for university teachers?” in Proceedings ASCILITE, Brisbane, 2005, pp. 17-25.
[6] R.B. Barr and J. Tagg, “From teaching to learning – A new paradigm for undergraduate education”, Change: The
Magazine of Higher Learning, vol. 27, no. 6, pp. 12-26, 1995.
[7] Ministry of Education (2008, May). Teaching Excellence Fund, 2008-2010, Group 2 Project 7, Development of a Scalable
and Extensible Blended Learning Model for NTU Engineering Students.
[8] K.A. Jones, “Recipe for Asia Pacific 21st century university classroom (Cover story)”, Innovation, vol. 11, no. 1, pp. 39-
41, 2012.
[9] A.L. Díaz and B.F. Entonado, “Are the functions of teachers in e-Learning and face-to-face learning environments really
different?”, Educational Technology & Society, vol. 12, no. 4, pp. 331–343, 2009.
[10] W.S. Maki and R.H. Maki, “Learning without lectures: A case study”, Computer, vol. 30, no. 5, pp. 107-108, 1997.
[11] J. Watson. (2008, January). Blended learning: The convergence of online and face-to-face education. Promising Practices
Series, North American Council for Online Learning [Online]. Available:
http://www.inacol.org/research/promisingpractices/NACOL_PP-BlendedLearning-lr.pdf.
[12] B. Means, Y. Toyama, R. Murphy, M. Bakia, and K. Jones. (revised 2010, September). Evaluation of Evidence-Based
Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies. U.S. Department of Education,
Office of Planning, Evaluation, and Policy Development, Policy and Program Studies Service [Online]. Available:
http://www.ed.gov/about/offices/list/opepd/ppss/reports.html.
[13] C-H. Liu, “The comparison of learning effectiveness between traditional face-to-face learning and e-learning among goal-
oriented users”, 6th International Conference on Digital Content, Multimedia Technology and its Applications (IDC),
Seoul, Korea, 16-18 August 2010, pp. 255-260.
[14] D.R. Garrison and N.D. Vaughan, Blended learning in higher education: Framework, principles, and guidelines. San
Francisco, CA: John Wiley & Sons, 2008, p. 5.
[15] N. Hoic-Bozic, V. Mornar, and I. Boticki, “A blended learning approach to course design and implementation", IEEE
Transactions on Education, vol. 52, no. 1, pp. 19-30, 2009.
[16] F. Alonso, D. Manrique, L. Martinez, and J.M. Vines, “How blended learning reduces underachievement in higher
education: An experience in teaching computer sciences”, IEEE Transactions on Education, vol. 54, no. 3, pp. 471-478,
2011.
[17] C. Furse, “Lecture-free engineering education”, IEEE Antennas and Propagation Magazine, vol. 53, no. 5, pp. 176-179,
2011.
[18] K. Chassie, “The allure of e-learning”, IEEE Potentials, vol. 21, no. 3, 2002, pp. 33-35.
[19] S. Ruth, “Is e-learning really working? The trillion-dollar question”, IEEE Internet Computing, vol. 14, no. 2, pp. 78-82,
2010.
[20] R.M. Gagné, The conditions of learning and theory of instruction, 4th ed. New York, NY: Holt, Rinehart & Winston,
1985.
[21] E. Mechlova and M. Malcik, “ICT in changes of learning theories”, in 10th IEEE International Conference on Emerging
eLearning Technologies and Applications, Stará Lesná, The High Tatras, Slovakia, 8-9 November 2012, pp. 253-262.
[22] R.S. Sharma, A. Siddiqui, Aijaz, A. Sharma, R. Singh, R. Kumar, S. Kaushal and S. Banerjee, Siddhartha, “Leveraging
knowledge management for growth: a case study of Tata consultancy services}, Journal of Information Technology Case
and Application Research, vol. 9, no. 4, pp. 29—65, 2007.
[23] M. Oliver and K. Trigwell, “Can ‘blended learning’ be redeemed?”, E-Learning, vol. 2, no. 1, pp. 17-26, 2005.
 20

[24] M.L. Derwent, “Why are so few students becoming engineers?”, IEEE Communications Magazine, vol. 28, no. 12, pp.
54-55, 1990.
[25] J.D. Bransford, A.L. Brown, and R.R. Cocking (Eds.), How people learn: Brain, mind, experience, and school,
Washington, D.C.: National Academy Press, 2004.
[26] C.M. Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, Volume II.
Mahwah, NJ: Lawrence Erlbaum Associates Inc, 1999.
[27] C.C. Bonwell and J.A. Eison, Active learning: Creating excitement in the classroom. Washington, DC: School of
Education and Human Development, George Washington University, 1991.
[28] K.E. Holbert and G.G. Karady, “Strategies, Challenges and Prospects for Active Learning in the Computer-Based
Classroom”, IEEE Transactions on Education, vol. 52, no. 1, pp. 31-38, 2009.
[29] D.W. Johnson, R.T. Johnson, E.J. Holubec, and P. Roy, Circles of learning: Cooperation in the classroom. Alexandria,
VA: Association for Supervision and Curriculum Development, 1984.
[30] J.W. Strijbos, “Assessment of (computer-supported) collaborative learning”, IEEE Transactions on Learning
Technologies, vol. 4, no. 1, pp. 59-73, 2011.
[31] W.T. Tsai, W. Li, J. Elston, and Y. Chen, “Collaborative learning using wiki web sites for computer science undergraduate
education: A case study”, IEEE Transactions on Education, vol. 54, no. 1, pp. 114-124, 2011.
[32] W. Stepien and S. Gallagher, “Problem-based learning: As authentic as it gets”. Educational Leadership, vol. 50, pp. 25-
28, 1993.
[33] J.D. Ward and C.L. Lee, “ A review of problem-based learning”, Journal of Family and Consumer Sciences Education,
vol. 20, no. 1, Spring/Summer, pp. 16-26, 2002.
[34] J.E. Mitchell, B. Canavan, and J. Smith, “Problem-based learning in communication systems: Student perceptions and
achievement”, IEEE Transactions on Education, vol. 53, no. 4, pp. 587-594, 2010.
[35] B.W. Boehm, “A spiral model of software development and enhancement”, Computer, vol. 20, no. 5, pp. 61-72, 1988.
[36] P.A. Ertmer, “Addressing first-and second-order barriers to change: Strategies for technology integration”, Educational
Technology Research and Development, vol. 47, no. 4, pp. 47-61, 1999.
[37] R.M. Bernard, P.C. Abrami, Y. Lou, E. Borokhovski, A. Wade, L. Wozney, P. A. Wallet, M. Fixet, and B. Huang, “How
does distance education compare with classroom instruction? A meta-analysis of the empirical literature”, Review of
Educational Research, vol. 74, no. 3, 2004, pp. 379–439.
[38] P. Ramburuth and J. McCormick, “Learning diversity in higher education: A comparative study of Asian international
and Australian students”, Higher Education, vol. 42, pp. 333-350, 2001.
[39] C. Ziguras, “Educational technology in transnational higher education in South East Asia: The cultural politics of flexible
learning”, Educational Technology & Society, vol. 4, no. 4, pp. 8-18, 2001.
[40] A.C. Applebee, K. McShane, S.D. Sheely, and R.A. Ellis, “Developing a blended learning community at the University
of Sydney: Broadening the comfort zone” in Proceedings ASCILITE, Brisbane, 2004, pp. 58-66.
[41] K. Robinson (filmed February 2010, posted May 2010). “Bring on the learning revolution”, TED2010, [Online].
Available: http://www.ted.com/talks/sir_ken_robinson_bring_on_the_revolution.html.
[42] R.S. Sharma and S. Bhattacharya, Knowledge dilemmas within organizations: Resolutions from game theory, Knowledge-
Based Systems, Volume 45, 2013, Pages 100-113.
[43] J. Dewey, Schools of Tomorrow, New York: EP Dutton & Co, 1915, p. 18.
 21

Supplementary Material for Jones & Sharma – FIGURES AND TABLES

F2F eLearning
humanistic virtualistic
mostly verbal mostly visual
synchronous, communal individual, asynchronous
fixed time-space anytime-anywhere
managed scaffolded

30:70 50:50 60:40

FIGURE 1. Blended learning mix comprising F2F and eLearning.

TABLE I
COMPARATIVE WEEKLY BREAKDOWN BY LEARNING SESSION BY DURATION
Model Timetable Periods
1st lecture 2nd lecture 3rd lecture Tutorial Laboratory
old: Lecture class work – 1hr; class work – 1hr; class work – 1hr; class work – 1hr; class work – 2hr;
homework – 1hr homework – 1hr homework – 1hr homework – 1hr homework – 1hr
new: Blended F2F – 1hr; F2F – 1hr; eLearning – 1hr F2F – 1hr; F2F – 2hr
learning eLearning – 1hr eLearning – 1hr eLearning – 0.5hr

Conditions of learning Events of learning

1-attention
2-objective
3-prior learning
4-stimulus
Learning Instructional
theory 5-guidance
design
6-performance
7-feedback
8-assessment
9-retention
FIGURE 2. Bridging learning theory and instructional design
with learning conditions and events.

TABLE II
CORRESPONDENCE OF LEARNING THEORY TO STRATEGY TO INSTRUCTIONAL DESIGN
Established Learning Theory Learning Strategy Chosen for Most Pertinent Instructional Design
Course Events Model
Cognitivism Build foundation 2,3,4
Cognitivism, Constructivism Facilitate fluency 3,7,9
PBL
Constructivism Emphasize team learning 5,6,7 Active Collaborative
Constructivism Emphasize learning transfer 6,7,9 learning learning
Behaviourism, Constructivism Enhance marking paradigm 2,6,8
Variationism Engage with contrasts 1,4,6
Behaviourism, Constructivism Sponsor metacognition 1,2,3
 22

Bedrock
12 kinds of
master media avail
«setup»
Precondition: received orientation One media for topic selected
Invariant: teacher tracks «body» summative;
entire activity {weight Launch, study, and analyse content score=0..1
= y} Feedback usefulness of media
Take
«test» master
Loop condition: 2 exercise
video
{optional} {weight
Set Take [•9]
=all}
expectation prior score?
and knowledge Take textbook survey
{weight motivate quiz
=all} [<9] {optional}
formative; «setup» Take
textbook One media for topic selected journeyman
all=x+y read prior quiz
{weight «body»
= x} Launch, study, and analyse content formative; same
Feedback usefulness of media questions as
knowledge quiz
«test»
Loop condition: 2 12 kinds of
journeyman
media avail

FIGURE 3. Bedrock (UML Activity diagram).

Conspectus parts one and two Precondition: completed bedrock activity

order preset
teacher calls Invariant: teacher tracks entire activity

eLearning F2F F2F

Answer Peer
eLearning Contrasting
question, critique
answers
Deliberate, and craft scoring
presentation
and build media
consensus {individual, {anonymous,
{teamwork, classroom}
on topic classroom}
anywhere}
{teamwork,
anywhere} eLearning
Craft and post summary in wiki
{individual, anywhere}

FIGURE 4. Conspectus (UML Activity diagram).

Seminar parts one and two Precondition: completed bedrock activity

Invariant: teacher tracks entire activity
Receive Hand out as
problem learner teams Teacher joins discuss-
statement enter classroom ion as mediator, mentor

eLearning
F2F F2F eLearning · Critique other
Brainstorm solutions
Present Post solution
problem and solution · Defend own
in wiki solution
solution
{Individual, {individual,
{Teamwork, classroom} {individual,
anywhere} anywhere}
classroom}

FIGURE 5. Seminar (UML Activity diagram).

 23

3-Assure 1-Initialize
quality

4 3 2 etc.
1

2-Take possession
FIGURE 6. Model of weekly path phases.

Weekly Learning Path assure quality

take possession

bedrock Do review for

Do conspectus for week’s topic
week’s topic
«parallel»
Continue project
Do bedrock for employing topics
week’s topic
Do seminar for
LAMS platform week’s topic Do reflection
individual

FIGURE 7. Weekly learning path (UML Activity diagram).

Semester Learning Path Midterm break Do Classes end

Do
reflection
midterm team
exam#2
text and
«parallel» «parallel»
Do diagram Distribute Do
learning Do weekly Do weekly final exam final
activity learning learning scenario exam
orient- path Do path
ation midterm
Loop exam#1 Loop Do project
condition: 6 MCQ condition: 5 assessment

FIGURE 8. Semester learning path (UML Activity diagram).

FIGURE 9. Content Management display.

 24

FIGURE 10. Display of Course Management (top half of page) and Wiki launch (lower half).

FIGURE 11. Online Help

 25

FIGURE 12. LAMS sequence in the editor view of the software.

TABLE III
MEAN AND OTHER GRADE STATISTICS FOR FOUR DIFFERENT COHORTS OF LEARNERS
Year Sem Size Mean Var Std Dev Critical t Std Err Lwr Lim Upr Lim
2007 One 84 64.51 156.1 12.49 1.989 1.363 61.8 67.22
-08 Two 150 65.93 61.57 7.847 1.976 0.641 64.66 67.19
2011 One 58 67.72 41.05 6.407 2.002 0.841 66.04 69.41
-12 Two 73 67.86 127.9 11.31 1.994 1.333 65.76 71.70

TABLE IV
LETTER GRADE DISTRIBUTION FOR FOUR DIFFERENT COHORTS OF LEARNERS
Year Sem Letter Grade
A A- B+ B B- C+ C D+ D F
2007 One 1.19% 11.90% 23.81% 27.38% 19.05% 7.14% 3.57% 2.38% 0% 3.57%
-08 Two 0.67% 9.33% 26.67% 32.67% 17.33% 7.33% 4.67% 0% 0% 0.67%
2011 One 1.72% 13.79% 31.03% 25.86% 17.24% 6.90% 3.45% 0% 0% 0%
-12 Two 4.17% 18.06% 44.44% 20.83% 5.56% 0% 1.39% 1.39% 1.39% 2.78%

TABLE V
SUMMARY OF SFT FOR TWO PERIODS
Year Activity Size Index Average
2007 Lecture 50 88.11
87.66
-08 Tutorial 23 86.69
2011 Conspectus Pt2 31 84.43
84.79
-12 Seminar 18 85.43
 26

120.0%

100.0%

80.0% 11S1
Attendance
60.0% 11S2
07S1
40.0%
07S2
20.0%

0.0%
1 2 3 4 5 6 7 8 9 10 11 12 13
Weeks
FIGURE. 13. F2F activities’ attendance readings for four different cohorts of learners.

TABLE VI
RANKING OF LEARNING ACTIVITIES BY LEARNERS

Conspectus wiki
Conspectus F2F

Seminar wiki

Practice final
Seminar F2F

Lab realistic
Review F2F
preference
Order of

Bedrock

project

exam
1st 26.15% 23.08% 9.23% 13.85% 13.85% 0.00% 7.69% 1.54%
2nd 31.25% 21.88% 6.25% 10.94% 17.19% 0.00% 9.38% 3.13%
3rd 13.85% 23.08% 21.54% 12.31% 3.08% 3.08% 16.92% 6.15%
4th 9.38% 21.88% 17.19% 14.06% 15.63% 4.69% 14.06% 3.13%
5th 10.77% 7.69% 15.38% 10.77% 10.77% 9.23% 21.54% 13.85%
a
Index 65.37% 64.71% 37.11% 37.76% 37.85% 5.57% 35.28% 11.75%
a
The Index value in Tables VI-VII is the sum of weighted percentages of preferences, that is,
“1st + 4/5 * 2nd + 3/5 * 3rd + 2/5 * 4th + 1/5 * 5th”

TABLE VII
RANKING OF EDUCATIONAL TECHNOLOGIES BY LEARNERS
Lecture recordings

Enhanced graphic
Clicker (audience

user interface

Enhanced ppt
Multimedia
preference

Web video

classroom
response)
Order of

LAMS

Wiki

IM

1st 35.38% 7.69% 16.92% 24.62% 3.08% 4.62% 1.54% 3.08% 1.54%
2nd 26.15% 21.54% 13.85% 18.46% 6.15% 4.62% 0.00% 7.69% 1.54%
3rd 15.38% 18.46% 16.92% 7.69% 7.69% 9.23% 7.69% 15.38% 1.54%
4th 9.23% 7.69% 18.46% 10.77% 12.31% 18.46% 6.15% 15.38% 1.54%
5th 3.13% 12.50% 10.94% 14.06% 7.81% 21.88% 12.50% 12.50% 4.69%
Index a 69.86% 41.58% 47.73% 51.12% 19.10% 25.61% 11.12% 27.12% 5.25%
 27

TABLE VIII
LEARNER’S ASSESSMENTS OF IN-PROGRESS MARKS AND HELP ONLINE SOFTWARE
Survey Question Yes No
Does the weekly posting of in-progress marks positively influence your learning progress? 58.90% 30.14%
Do you feel that the subject online Help! provides good support for the learning experience? 66.15% 33.85%

TABLE IX
LEARNER’S ASSESSMENTS OF THEIR EXPERIENCE WITH BLENDED LEARNING
MCQ Response Set 11S1 11S2 Avg
I know I'm learning better, and can see an improvement in my performance. 20.4% 27.69% 24.1%
I feel that I am learning better, but didn't notice much difference in my performance. 28.6% 33.85% 31.2%
I might be learning better, but I'm not sure. 46.9% 32.31% 39.6%
I am not learning better than I was with lectures. 4.1% 6.15% 5.1%

TABLE X
RANKING OF WHAT LEARNERS LIKED BEST ABOUT BLENDED LEARNING
Distilled Response b Qty
freedom and control of learning process 17
flexibility in when and how much time for learning 14
continuous learning with hands-on practice, regular review, and reinforcement 12
freedom of choice available from the many references 9
richness and contrasts of online experience 9
working in groups in the seminar 6
achievement of greater depth of learning 5
imposition of deadlines to enforce proactive studying 5
presentation & peer assessment in the conspectus 5
interactivity of the F2F sessions 4
chunking of the syllabus into weekly topics 3
summarisation of weekly topic in the conspectus 3
guidance in learning priorities from the prior knowledge quiz 3
motivation to learn by virtue of teacher’s interest 2
weekly sequence of learning activities 2
lowered anxiety with step-by-step learning instructions 2
feedback and mentoring by teacher 1
b
The responses in Tables X-XII are distillations of the singular open responses into a short list of
conjoint themes. The inherent subjectiveness of this process is lessened by repeating the distillation
separately from the first, and then normalizing the two independent sets of distillates into one.
 28

TABLE XI
RANKING OF WHAT LEARNERS WOULD IMPROVE IN THE BLENDED LEARNING
Distilled Response b Qty
nil, I like it the way it is 12
allocate more time for F2F activities and assignments 9
provide more quizzes and example solutions 9
increase feedback, guidance, and evaluation 8
improve or replace peer evaluation in conspectus 5
provide additional online learning resources 5
guidance on resources, online platform & time management 5
re-incorporate lecturing into teaching 3
replace F2F conspectus 3
improve operation and GUI for wiki 2
include online explanation with quiz answer 1
improve quality of the lecture recording videos 1
decrease size of study team in seminar 1
change conduct of online activities from individual to group 1
replace project in lab 1
improve questions in conspectus 1
include inter-group discussion of solutions in seminar 1
re-arrange weekly learning sequence 1
start introducing blended learning to 1st year students 1

TABLE XII
RANKING OF TECHNOLOGY THE LEARNERS WOULD ADD INTO THE BLENDED LEARNING
Distilled Response 2 Qty
nil 33
video conferencing for teaching, Q&A, and consultation 11
online meeting and discussion forum 7
video series on producing UML models 3
upgrade LAMS to be compatible with iPad and other devices 2
interactive drawing platform in F2F activities 1
photo gallery for UML drawing examples 1
SMS question or fun fact of the day to engage students 1
SMS querier (custom SCE-built software for audience texting) 1
texting function for clickers 1
serious gaming learning software 1
better performing wiki software 1
mobile app for launching online activities 1
video recording of seminars 1
adaptive release function for a video player 1

TABLE XIII
USAGE OF IM BY THE LEARNERS
2011-12 Quantity
Size Utilization
semester User Message
One 58 16 35 27.59%
Two 73 7 20 9.59%
 29

TABLE XIV
LEARNERS’ CORRECTIVE ACTIONS TO STRUGGLING WITH THEIR LEARNING PROGRESS
MCQ Response Set 11S2
Seek extra teaching from the teacher 9.64%
Participate in the school student-lead tutorial group for CPE207 0.00%
Seek tutoring from my fellow classmates and friends 31.33%
Work it out on my own without external intervention 46.99%
Do nothing 1.20%

Kevin A. Jones received his M.Eng. degree in Software Engineering jointly from Royal Military College of
Canada, and Queen’s University in 1994, and his B.Sc. degree in Honours Math and Physics from Royal Military
College of Canada in 1978. He is currently pursuing the Ph.D. degree in Information Studies at NTU. Kevin has
spent over 10 years as a senior lecturer of Software Engineering in SCE, seven years as lead architect and
business development consultant for PeopleSoft implementations in ASEAN, and 21 years in service as an
instructor and senior engineer in electronics and communications. He has authored several technical papers on
interactive digital media and innovative teaching. Current research interests are software design and modeling,
and innovation in adult education. He serves as an external examiner for a prominent Polytechnic institution in
the area. He is the NTU member of the National Infocomm Competition Organizing Committee, and the Challenge Director for
virtualXgame contest. For the past ten years, he has coached the International Collegiate Programming Competition teams from
NTU.

Ravi S. Sharma is currently an Associate Professor of Information Systems at the University of Canterbury
Business School. His teaching, consulting and research interests are in digital business strategies and
transformations. He has (co-) authored over 100 technical papers and his work has appeared in leading journals,
conferences, trade publications, and the broadcast media. He has also lead-authored graduate level texts on
KM Tools and Digital Marketplaces. He received his Ph.D. in management sciences from the University of
Waterloo and is a Chartered Engineer (UK) and a Fellow of the Institution of Engineering & Technology. He
also sits on the editorial or advisory boards of several professional and non-profit organizations.