You are on page 1of 23

Information Systems Research informs ®

Vol. 19, No. 1, March 2008, pp. 26–47


doi 10.1287/isre.1070.0141
issn 1047-7047  eissn 1526-5536  08  1901  0026
© 2008 INFORMS

Using Self-Regulatory Learning to Enhance


E-Learning-Based Information Technology Training
Radhika Santhanam
Gatton College of Business and Economics, University of Kentucky, Lexington, Kentucky 40506,
santhan@uky.edu

Sharath Sasidharan
Lewis College of Business, Marshall University, Huntington, West Virginia 25755
sasidharan@marshall.edu
Jane Webster
Queen’s School of Business, Queen’s University, Kingston, Ontario Canada K7L 3N6,
jwebster@business.queensu.ca

T echnology-mediated learning methods are widely used by organizations and educational institutions to
deliver information technology training. One form of technology-mediated learning, e-learning, in which
the platform is the tutor, is quickly becoming the cost-effective solution of choice for many corporations. Unfor-
tunately, the learning outcomes have been very disappointing. E-learning training makes an implicit assumption
that learners can apply a high level of self-directed learning to assimilate the training content. In contrast, based
on perspectives from social cognitive theory, we propose that instructional strategies need to persuade learn-
ers to follow self-regulated learning strategies. We test our ideas with participants who were trained through
e-learning to design a website. Our findings indicate that participants who were induced to follow self-regulated
learning strategies scored significantly higher on learning outcomes than those who were not persuaded to do
so. We discuss our findings, and suggest that the interaction among information technology features, instruc-
tional strategies, and psychological learning processes offers a fruitful avenue for future information systems
training research.
Key words: e-learning; laboratory experimentation; information technology training; self-regulatory learning;
social cognitive perspective; pretraining scripts; website development; self-efficacy
History: Ritu Agarwal, Senior Editor; H. Raghav Rao, Associate Editor. This paper was received on August 19,
2005, and was with the authors 8 months for 3 revisions.

Introduction et al. 1997, Santhanam and Sein 1994, Venkatesh 1999,


Almost every employee today has to be skilled in Yi and Davis 2003).
using information technology (IT); therefore, organi- In recent years, IT training is increasingly being
zations continue to invest significantly in IT training delivered through electronic means such as tech-
for their employees, and universities offer many IT nology-mediated learning (TML) methods instead of
training courses for students (Agarwal and Ferratt through face-to face interactions between learners and
2001, Bureau of Labor Statistics 2004, Homer and trainers. TML as a training vehicle is growing rapidly
Povar 2004, Piccoli et al. 2001). Given the extensive because it is seen as a cost-effective way to deliver
deployment of IT training in corporations and institu- training at convenient times and at remote locations
tions, it is not surprising that information systems (IS) to large numbers of employees and students (Zhang
researchers expend substantial efforts to identify the et al. 2004). A study by the Sloan Consortium finds
most effective training methods and strategies (e.g., that educational institutions use TML extensively:
Agarwal et al. 2000, Bostrom et al. 1990, Compeau 81% offer at least one fully online or blended course,
et al. 1995, Compeau and Higgins 1995a, Johnson and and 34% offer complete technology-based degree pro-
Marakas 2000, Olfman and Mandviwalla 1994a, Lim grams (Allen and Seaman 2005). In practice, however,
26
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 27

TML has not provided the benefits that were orig- future work in both learning through IT and learning
inally anticipated; corporations, educational institu- about IT.
tions, and IS researchers are thus very motivated to We focus our attention on the specific TML envi-
search for ways to improve TML effectiveness (Alavi ronment, referred to as e-learning, where the learner
and Leidner 2001, Gupta and Bostrom 2005, Moller interacts primarily with the IT platform rather than
2002, Olfman et al. 2006, Sasidharan and Santhanam with other learners or instructors (Jonassen and
2006). Reeves 2001, Jones and Paolucci 1999, Zhang et al.
IS researchers emphasize the exigency of conduct- 2004). As we elaborate in the next section, the instruc-
ing research on TML for several reasons. First, this tional strategy in this type of training infrastruc-
research can build on and contribute to the cumu- ture is anchored on self-directed and independent
lative knowledge developed by IS research on the learning. The implicit assumption is that learners are
interaction of IT and human problem solving and able to meet the demands expected of the instruc-
learning processes (Alavi and Leidner 2001, Leidner tional strategy, apply high levels of learner con-
and Jarvenpaa 1995). Second, because IT training is trol, and self-direct their learning. Reports indicate,
an integral component of IS research, existing find- however, that learners are not able to apply the
ings on how best to develop and test IT skills could anticipated high levels of learner control, are not
contribute to our understanding of whether IT infras- motivated to learn, and tend to use inadequate learn-
tructures are effective IT training platforms (Gupta ing strategies (Bell and Kozlowski 2002, Brown 2001,
and Bostrom 2005, Olfman et al. 2006). It can also Rossett and Schafer 2003). Therefore, we argue that
help us research IT software aspects that provide new if e-learning is to become an efficacious IT train-
opportunities for the control and pacing of user learn- ing method, instructional strategies should be mod-
ing. Finally, enhancing IT skills of students through ified to include interventions that persuade learners
TML is a research topic of growing interest to IS aca- to follow self-regulated learning (SRL) strategies. In
demics as evidenced by research articles, specialized the next section, we describe how the social cogni-
journals, and special interest groups on IT education tive perspective on self-regulation can help design
(Hardaway and Scamell 2005, Leidner and Jarvenpaa interventions that modify instructional strategies in
1995, Markus 2005). Consequently, there is a need to e-learning-based training environments. We then
study the role of TML as an effective IT training deliv- report on an experiment in which we provided
ery mechanism (Gupta and Bostrom 2005, Olfman manipulations to encourage learners to self-regulate
et al. 2006, Salas and Canon-Bowers 2001, Zhang et al. while learning to use Website-development software.
2004). We conclude the paper by discussing the implications
We believe that IS researchers, with their multi- of the study for practice and future research.
disciplinary focus and ability to integrate social and
cognitive processes with technology affordances, are
uniquely qualified to study TML. In this research, we Research Framework
address the role of TML as an IT training mecha- In this section, we first explain the different terms used
nism in the following manner: (1) We draw on a key in reference to TML. We then review TML research
learning-through-technology framework (Alavi and and describe the learning-through-technology frame-
Leidner 2001) that has not received much empirical work proposed by Alavi and Leidner (2001). Finally,
attention; (2) we examine one specific TML environ- we propose that e-learning-based training could be
ment and the underlying instructional strategies in improved by paying attention to self-regulation, and
order to identify methods that can improve IT train- elaborate on how it could be applied by proposing
ing outcomes; and (3) we propose and test interven- specific hypotheses to be tested.
tions based on social cognitive (SC) theory (Bandura
1991) that can induce learners to self-regulate their Characteristics of TML and E-Learning
learning and enhance their IT skills via TML training. TML refers to an environment in which IT is used
Combined, this approach provides a foundation for to mediate/support teaching and course delivery and
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
28 Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS

includes a variety of learning environments: differ- 2001). Technology-based learning, or learning from
ent functions and features of IT may be selectively computers, encompasses both distance learning and e-
applied, instructional packages may be bundled in learning because technology is used in both as the pri-
various ways, and learners may be provided with dif- mary medium to deliver course content (Gupta and
ferent levels of control (Benbunan-Fich 2002, Bostrom Bostrom 2005, Jonassen and Reeves 2001, Jones and
2003, Jonassen 2004). E-learning describes a TML Paolucci 1999). In distance learning, the teacher and
environment in which a single user interacts with learner are separated in space and time, but they can
technology and attempts to self-direct and complete a communicate with one another; in many implementa-
training course (Zhang et al. 2004). tions, learners can communicate with other learners as
In Figure 1, we list a few commonly used terms and well. Among TML contexts, e-learning provides max-
corresponding descriptions related to e-learning. Vir- imal control to learners so that they can learn in an
tual learning is a broad term to refer to computer- independent manner. However, as seen in Figure 1, in
based environments with a wide range of resources e-learning the learner may be unable to communicate
made available to learners (Anohina 2005, Piccoli et al. with the instructor or with other learners.

Figure 1 E-Learning-Related Terms and Descriptions

Virtual Learning
• An encompassing term denoting computer based instructional environments that are relatively open systems
with a wide range of resources that learners can use to interact with other learners and instructors. Some researchers
refer to this as Web-based learning or online learning if educational content is transferred via the Web browser and
computers connected to the intranet.
• Delivery of educational content via a Web browser over the public internet, a private intranet, or an extranet.
• Training via computers connected to the World Wide Web.
Technology-Mediated Learning (TML)
• A term used in instructional environments where information technologies are used to support course delivery
and to manage the teaching and the learning process. It can refer to a broad set of applications such as learning labs,
teleconferencing systems, computer-mediated communications, collaborative learning systems, etc. Though some
IS researchers refer to this as e-learning, most disciplines make a distinction between TML
(learning with computers) vs. technology-based learning (learning from computers). When information technology is used
primarily to support the learning process but course content is not necessarily delivered via computers, it tends to be
referred to as TML.
Technology Based Learning/Training (TBL/TBT)
• A term used when the delivery of course content is via computer technology. This is thus referred to as learning
from computers. Within this are two groupings of terms: distance learning and e-learning.
Distance Education/Learning
• Separation of teacher and learner in space and/or time
• Provision for two-way communication
• Educational institution typically provides certification on course completion
• May involve some classroom teaching as well
• May also include learners engaged in group learning
E-Learning
• User engaged in self-paced learning with learner in control. Learning package is delivered or transacted through
electronic means, sometimes referred to as computer microworlds.
• If e-learning is conducted through stand-alone computers, it is generally referred to as
Computer-Based Training/Computer-Assisted Instruction.
• If e-learning is delivered through the internet, intranet (or extranet), or a Transmission Control Protocol/Internet Protocol
network, it is referred to as
Internet-based training/learning.
• If e-learning involves use of communication technologies to facilitate discussions among learners, it is referred to
as collaborative e-learning.

Note. Figure adapted from Anohina (2005). Descriptions obtained from Anohina (2005), Alavi and Leidner (2001), Piccoli et al. (2001), Jonassen (2004), and
Keegan (1996).
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 29

An e-learning infrastructure typically consists of Hiltz 2003, Richardson and Swan 2003). This points
a database-centric learning content-management sys- to the importance of paying attention to instructional
tem that lists the course catalogs (from which the strategies in specific TML environments, and to the
learner can choose a specific course), registers the special challenges in contexts where learners have to
learner, and manages the interaction between user and learn independently without opportunities to collab-
system (Bostrom 2003). A typical IT training course, orate with other learners. As is evident from Figure 1,
such as an introductory course on website develop- e-learning, the technology examined in this study,
ment, might consist of several sequential learning represents such an environment where learners do
modules, with each module providing instructional not get opportunities to interact with other learners.
information on one topic related to designing a web- A review of TML research in IS indicates that, sim-
site. The first module may consist of general infor- ilar to other disciplines, IT skill development has
mation on websites, the second on features of the not been given much attention (Alavi 1994; Alavi
specific software; the third may introduce procedures et al. 1995, 2002; Coppola et al. 2002; Leidner and
to develop a website, and so on. Advanced courses Fuller 1997; Sasidharan and Santhanam 2006). Most
could comprise a dozen such modules. of these studies test TML in collaborative learning
contexts and, as in other disciplines, conclude that
Prior Research on TML using TML methods does not automatically lead to
The general challenges of TML-based training have superior learning outcomes: attention must be given
been addressed by researchers in other disciplines, to instructional strategies and specifics of the TML
but little attention has been paid to IT skill develop- context. Hence, as articulated by Alavi and Leidner
ment courses (e.g., Bernard et al. 2004, Jonassen 2004, (2001), learning outcomes can be increased by paying
Jones and Paolucci 1999). Factors impacting distance attention to the interaction of three key factors in spe-
learning effectiveness have been examined even from cific TML environments, namely, information technol-
the days of correspondence courses, with recent stud- ogy, instructional strategy, and learners’ psychological
ies focusing on technology-mediated environments processes. Except for a study by Piccoli et al. (2001),
and their implications for the new roles of teach- little research has addressed these factors and their
ers and students (Berge and Mrozowski 2001, Keegan impact on learning outcomes.
1996, Noffsinger 1926). A meta-analytic review of over Based on Alavi and Leidner’s (2001) framework,
200 research studies on distance learning concludes, we argue that, because IT features in e-learning often
among other findings, that learning outcomes can be do not permit learners to interact with other learn-
enhanced by paying attention to specific features of ers or instructors, the IT platform becomes the dom-
the instructional environment and to the proper use inant mode of communication with the learner. The
of instructional strategies (Bernard et al. 2004). Other accompanying instructional strategy relies on self-
reviews of TML also point to the importance of pay- directed learning with the assumption that learners
ing attention to instructional strategies and to the spe- can independently regulate their learning and absorb
cific features in a TML environment. A review that the training content. In other words, the conditions of
summarizes research findings comparing TML con- training are established to evoke a high level of self-
texts involving groups of learners versus individual directed learning: the training design is based on an
learners concludes that TML courses are not supe- objectivist learning model, which assumes that learn-
rior overall, but that group learning with TML has ers learn best in an isolated and intensive manner
more favorable effects than individual learning from by regulating their own learning (Gagne 1977, Gagne
TML. This suggests that feedback, the social con- et al. 1992, Leidner and Jarvenpaa 1995). But such a
text, instructional strategies, and interaction among design imposes a very high burden on the learner: to
learners can maximize learning outcomes (Lou et al. be motivated and focused on learning without any
2001). Unfortunately, many commercial technologies guidance from human instructors or other learners.
are based on an individual learning model, not on Even in e-learning environments where learners can
a collaborative learning model (Benbunan-Fich and interact with other learners and instructors, learners
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
30 Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS

complain that they find it difficult to take on the Vygotskian, constructivist, volitional, phenomenolog-
responsibility for directing their own learning (Piccoli ical, information processing, and social cognitive. All
et al. 2001). Empirical observations attest that learn- of these perspectives tend to view SRL as something
ers situated in e-learning environments do not ade- purposive that involves the use of specific strategies,
quately self-direct their learning, nor do they exercise but they differ on the factors they view as being rel-
high levels of learner control, which might explain evant to a learner’s use of SRL. For example, oper-
the higher drop-out rate (Allen and Seaman 2005, ant theorists emphasize the role of external factors,
Bell and Kozlowski 2002, Brown 2001, Zhang et al. suggesting that learning responses are ultimately con-
2004). Therefore, as described below, we propose that trolled by external reward/punishment contingencies
the instructional strategy, or the sequence of activities such as verbal coaching and reinforcement. Informa-
that systematically exposes learners to experiences to tion processing theorists emphasize the role of three
help them acquire knowledge, must be modified to types of memory (sensory, short term, and long term)
enhance the effectiveness of e-learning-based training. in the SRL process. Learning strategies in this per-
spective emphasize learners’ ability to cluster bits
Self-Regulation and E-Learning of information into larger units and develop their
The interaction between IT features and instructional capacity to process information. In contrast with the
strategies necessitates that the learner exhibits an external factors highlighted in the operant theory, the
e-learning strategy that includes high learner control, social cognitive perspective takes an agency perspec-
self-discipline, and self-motivation. We propose that tive and highlights individuals’ roles and their own
learning outcomes will be enhanced if instructional ability to enact self-regulation. It states that learners
strategies are modified to include interventions that by themselves can self-regulate through their specific
instruct learners to follow self-regulatory learning use of strategies toward learning, and emphasizes
strategies that include, among other things, encour- that several factors, such as learners’ self-efficacy
aging learners to believe that they can learn through beliefs, their motivation to learn, and their learning
e-learning training, enhancing their motivation to goals and strategies, must be addressed in combi-
learn, formulating appropriate goals for the course, nation (Kauffman 2004, Pintrich and DeGroot 1990,
and devising methods for organizing course con- Schunk 2001, Zimmerman 1989). We focus on this
tent. In other words, instructional strategies should perspective because it takes a collective approach to
include interventions that instruct learners to apply addressing SRL, and the SC theory has been used fre-
self-regulation in their learning. quently in IS research.
Self-regulation refers to a general skill that keeps
people focused on a task, helps them monitor their The Social Cognitive Perspective on SRL
task-completion progress, and explains success in a The salient aspects of the SC perspective include,
broad range of phenomena, for example, management among other factors, learners’ motivations, their out-
of chronic illnesses, training for sports, treatment of come expectancies, and their perceived self-efficacy
obsessive behaviors, and learning in academic set- beliefs (Schunk 2001, Zimmerman 2000). Learners’
tings (Bandura 1991, Boekaerts et al. 2000). Partic- perceived self-efficacy beliefs (i.e., beliefs about their
ularly in academic environments, researchers have capabilities to learn) are essential factors that affect
found that students who self-regulate their learning all phases of self-regulation, and are formed based on
reach higher academic achievements irrespective of prior observations or prior performance or through
their courses of study (Pintrich and DeGroot 1990, some form of persuasion (Schunk and Ertmer 1999,
Zimmerman et al. 1992, Zimmerman and Schunk 2000; Zimmerman et al. 1992). The SC perspective
2001). SRL involves strategies by which learners also emphasizes the situation-specific nature of SRL;
actively engage in learning and apply intentional that is, learners may not engage in self-regulation
efforts to manage and direct their learning activities. equally in all learning environments (Schunk 2001).
Zimmerman and Schunk (2001) review various the- For example, research indicates that, although high
oretical perspectives on SRL, including the operant, achievers tend to use SRL strategies, they may not
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 31

apply them in every situation (Schunk and Ertmer single components of learning may not be enough;
2000, Zimmerman and Martinez-Pons 1990). There- instead, a collective (i.e., multifaceted) approach is
fore, SC theorists emphasize the situational task goals useful because learners who discover a lack of pro-
of the learner, such as completing a specific home- gress through metacognitive monitoring also need
work assignment, and situation-specific self-efficacy some motivational regulation to continue their learn-
beliefs, such as beliefs in their abilities to solve fraction ing efforts (Pintrich and DeGroot 1990, Zimmerman
problems in arithmetic (Schunk 2001, Zimmerman and et al. 1992).
Schunk 2001). Although some SRL strategies may gen- Most individuals use some level of self-regulation,
eralize across settings, learners must know how to but they differ on the quality and extent to which they
adapt to situation-specific domains and feel competent apply it in specific contexts (Zimmerman 2000). This
in doing so. depends on their knowledge of SRL strategies, their
In line with the situation specificity of SRL, fore- decisions to use known strategies, and their ability
thought is described as an important period that to use these strategies skillfully (Schunk and Ertmer
occurs before learning and sets the stage for action. 1999). Thus, adopting an SC perspective considers not
Motivations and preparations during forethought in- only learners’ choices of cognitive strategies, but also
fluence learners’ level of engagement with the learn- their fears, doubts, confidence, self-beliefs, and sense
ing task and impact learning outcomes (Schunk and of personal agency in specific performance contexts
Ertmer 2000). During learning, outcomes can be im- (Zimmerman 1995, Zimmerman and Martinez-Pons
proved by providing performance-based feedback 1988). Based on our adoption of the SC perspective
and facilitating learners’ metacognitive monitoring on SRL, we believe that because learners’ use of SRL
(Bell and Kozlowski 2002, Schmidt and Ford 2003). has to be activated in a specific situation (such as
But addressing only metacognition or other isolated while undergoing e-learning-based training), context-

Figure 2 Phases and Beliefs in Self-Regulatory Learning

Performance or
volitional control

Forethought
task analysis Self-reflection
motivational and adaptation
beliefs

Phase Structure and Sub-Processes of Self-Regulation

Cyclical self-regulatory phases

Forethought Performance/volitional control Self-reflection

Task analysis Self-control Self-judgment


Goal setting Self-instruction Self-evaluation
Strategic planning Imagery Causal attribution
Seft-motivation beliefs Attention focusing Self-reaction
Self-efficacy Task strategies Self-satisfaction/affect
Outcome expectations Self-observation Adaptive defensive
Instrinsic interest/value Self-recording
Goal orientation Self-experimentation

Note. Adapted from Zimmerman (2000). This article was published in the Handbook of Self-Regulation, edited by M. Boekarts, P. Pintrich, and M. Zeidner,
Attaining self-regulation: A social cognitive perspective, 13–39, copyright Elsevier, 2000.
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
32 Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS

specific interventions that seek to enhance learner note taking. Similarly, scripts could also be used to
motivation and that instruct learners to follow SRL influence these activities by instructing learners to
strategies could improve learning outcomes. apply the cognitive strategies of self-directing, record-
ing, note taking, etc. As learning progresses, learners
Development of Hypotheses should engage in metacognitive activity and evaluate
We draw on a process model of SRL (shown in Fig- whether their strategies are working and, if neces-
ure 2) to design an instructional strategy that can help sary, adapt. Either of two methods can be effective
to persuade learners to apply SRL during e-learning in accomplishing this: (1) learners could self-evaluate
training (Zimmerman 2000). In the model, SRL can and reflect of their own volition or (2) they could
occur through three cyclical phases: forethought, voli- be provided with external evaluations and feedback
tional control, and self-reflection. Although this is a (Zimmerman and Kitsantas 1997). Therefore, one way
process model suggesting how SRL could be affected, to help learners reflect on their strategies could be
it can be observed from Figure 2 that the model via external feedback emphasizing the success of
includes the multifaceted aspects of SRL. In the fore- the learners’ self-regulatory strategies. The feedback
thought phase, the learners’ learning goals, motiva- could be used to set goals for the rest of the training
tion to learn, self-efficacy beliefs, and learning plan session and enhance learners’ motivation. This type of
should be addressed. In the performance/volitional feedback and positive evaluation is an integral aspect
control phase, the learner focuses on the learning of self-regulation that reinforces the use of SRL strate-
task, applies cognitive strategies like note taking, gies and sustains motivation and self-efficacy beliefs
and organizes study materials, thereby self-directing (Schunk 2001, Schunk and Ertmer 1999). Therefore,
their learning. During self-reflection, learners conduct we propose two hypotheses:
metacognitive monitoring of their learning progress
and adapt their strategies. Hypothesis 1A. In an e-learning-based IT training
One could use this model to design an instruc- session, presession interventions designed to increase the
tional strategy such that a pretraining intervention learner’s use of self-regulatory learning strategies and
occurs at the forethought phase, when learners get accompanying beliefs will enhance learning outcomes.
engaged in task analysis through which they under- Hypothesis 1B. In an e-learning-based IT training ses-
stand the learning plans and goals for the train- sion, interventions that provide positive feedback to learners
ing session. Hence, before the learner begins the on their use of self-regulatory strategies will enhance learn-
e-learning tutorial, trainers could provide a task anal- ing outcomes.
ysis for the training session by describing learning
plans and goals for the session. During the fore- Learner Characteristics
thought phase, learners must also develop motiva- Researchers propose that certain dimensions of SRL
tional beliefs toward training and, once again, trainers can be influenced by stable dispositional traits: if a
could influence these motivational beliefs. This could learner has a higher ability to set goals, to apply meta-
be achieved if trainers were to provide learners with cognitive strategies, and to self-regulate, etc., those
scripts that inform them about the various modules in skills could help the learner apply SRL and achieve
the e-learning course, the goals and elements of each higher learning outcomes (Ford et al. 1998, Schmidt
module, and a list of what they should have learned and Ford 2003, Winnie 1995). In IS research, individ-
by the end of each module. The scripts could also be ual learner traits are noted as influencing training
designed to boost learners’ motivational beliefs, out- outcomes (Agarwal et al. 2000, Bostrom et al. 1990,
come expectations, and interest in learning through Webster and Martocchio 1992). Studies on distance
e-learning. learning also indicate that learner characteristics may
When learners start on the actual learning task, influence learning outcomes and should be consid-
they should engage in performance/volitional con- ered (Bernard et al. 2004). Therefore, it is worthwhile
trol activities such as attention focusing and self- to examine whether individual characteristics could
recording, and self-observational strategies such as influence e-learning-based training outcomes.
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 33

Because the focus of this study is on SRL and Hypothesis 2C. In an e-learning-based IT training
the training interventions attendant on SRL, we were session, self-efficacy for self-regulatory learning will be pos-
interested specifically in examining learner character- itively associated with learning outcomes.
istics that might relate to SRL or to behaviors in a new
learning environment. As shown in Figure 2, individ- Research Method
ual goal orientation could influence performance and
To test our hypotheses, we chose a laboratory exper-
volitional control. Beaubien and Payne (1999) describe
iment as the suitable approach. We performed a
goal orientation as an important individual motiva-
pilot test before we conducted the experiment (see
tional variable that explains how people develop com-
Appendix 1).
petence in new learning and performance situations.
Although first understood as a two-factor structure, Facilities, Target IT Application, and Participants
goal orientation is now described as comprising three Using an e-learning platform used in training em-
factors: learning orientation, performance approach, ployees and students, we conducted the experiment
and performance avoidance orientations (VandeWalle at a large public university. The IT training pack-
1997, Zweig and Webster 2004). For the purposes of age provided a course on website design and devel-
this study, the learning orientation factor is relevant opment with Microsoft FrontPage software. In this
because e-learning training represents a new learning course, learners proceeded through several modules
task. People with high learning orientations are more
as they underwent training in the conceptual ele-
likely to find new learning tasks challenging and to
ments and procedures used to set up websites. The
view learning performance as indicative of mastery
training laboratory was similar to an IT training room,
(Brett and VandeWalle 1999, Fisher and Ford 1998,
with 16 standalone computers in individual cubicles.
VandeWalle 1997). Therefore, we propose the follow-
Each participant, wearing a headset, completed the
ing hypothesis:
course independently, without interference from other
Hypothesis 2A. In an e-learning-based IT training learners. Training was interactive and delivered by
session, learning orientation will be positively associated audio and video components. The participant sequen-
with learning outcomes. tially activated and completed the learning modules.
Self-efficacy beliefs are important motivational For our experiment, we selected eight consecutive
beliefs in SRL (see Figure 2). In IS training research, modules from the course, providing enough informa-
self-efficacy beliefs are identified as a critical predictor tion content to develop a website with linked pages
of learning outcomes (Agarwal et al. 2000, Colquitt and internal and external links. Each screen within
et al. 2000, Compeau and Higgins 1995b) and, as a course module displayed a typical FrontPage inter-
described earlier, task-specific self-efficacy beliefs are face with a box that described the screen layout and
considered to be relevant (Bandura 1997). In this menu options. The participant was asked to choose
study, learners were trained through a computer- one of the menu options, to type text, and to exe-
based program, so it can be expected that individ- cute appropriate actions. The participant viewed the
uals’ self-efficacy beliefs regarding learning through results. If satisfied, the participant hit the forward
computers may influence learning outcomes. In addi- button and proceeded to the next screen, but could
tion, individuals differ in their ability to self-regulate move backward or forward within the module at any
learning, because this ability is developed over a life- time. After a module was completed, the participant
time through social and other influences (Schunk and moved on to the next one. Undergraduate business
Zimmerman 1997). Users may understandably have students participated over four semesters and were
differing self-efficacy beliefs regarding their ability to given course credit for volunteering. Students who
self-regulate learning and this could influence learn- had web-design experience were excluded, but were
ing outcomes. Hence, we propose the following: given course credit through other means. We next
Hypothesis 2B. In an e-learning-based IT training describe our experimental procedures and then pro-
session, computer-learning self-efficacy will be positively vide details on the interventions and measures used
associated with learning outcomes. in this study.
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
34 Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS

Experimental Procedures about 90 minutes, and the final testing through com-
Immediately after students volunteered for the study, prehension and development of a website took about
they were given a background information survey 30 minutes.
that measured their individual learner traits and de-
mographics (see Appendix 2). They were then as- Interventions and Measures
signed to one of the training sessions. Prior to starting
Background Information Survey. We created a
the e-learning training, participants received either a
survey that gathered background information (in-
treatment or control script. During the training ses-
cluding demographics such as age, gender, and com-
sion, they received feedback in the form of another
puter experience) and measured individual learner
treatment or control script. Thus, the combination
traits. Measures of learner traits included learning
of treatment or control scripts prior to and dur-
orientation, ( = 085), computer-learning self-efficacy
ing training resulted in four different conditions:
measure ( = 092), both provided by Zweig and
treatment-treatment (T1 -T2 ), treatment-control (T1 -C2 ),
Webster (2004), and a self-efficacy for SRL mea-
control-treatment (C1 -T2 ), and control-control (C1 -C2 ).
sure ( = 087) from Zimmerman et al. (1992) (see
We refer to the scripts provided prior to training as
pretraining scripts and those provided during train- Appendix 2).
ing as midpoint scripts (see Figures 3 and 4). Pretraining Scripts. The pretraining treatment
At their scheduled training session, participants script was based on the self-regulation model pro-
were randomly assigned to receive either the treat- vided by Schunk and Zimmerman (1998) (Figure 2).
ment (T1 ) or the control (C1 ) pretraining script. After pilot testing and modifications, the final pre-
They were given eight minutes to read the pretrain- training treatment script (T1 ) provided task analysis
ing scripts and then they completed manipulation and learning goals for the session (see Figure 3a).
check measures on motivation to learn and computer- We asked participants to take notes, pay attention,
learning self-efficacy beliefs, described below. Next, and stay focused. We also attempted to boost their
participants received training through four modules motivational beliefs by influencing their self-efficacy
of the e-learning system. As a pretext to provide par- beliefs and their outcome expectations from the learn-
ticipants with feedback, we asked them to answer five ing exercise (e.g., with statements such as “You are a
easy-to-answer multiple-choice questions on website very capable learner”). The pretraining control script
design. The purpose was not to determine partici- (C1 ) was about the same length to control for the
pants’ learning scores, but to demonstrate that our amount of information presented to the participants
feedback was based on their performance. Therefore, (Webster and Martocchio 1995). It provided only gen-
after completing four modules, participants answered eral information on communication technologies (see
these questions and received feedback from the mid- Figure 3b).
point script (treatment [T2 ] or control [C2 ], randomly
assigned). After reading this script, they continued Midpoint Scripts. One group of participants re-
through the next four modules. At the end of the ceived the midpoint treatment script (T2 ), which eval-
eight modules they were asked to answer questions uated their performance and asked them to focus on
testing their declarative knowledge. Next they were learning goals, to pay attention, and to monitor their
asked to complete a hands-on performance task (see learning progress, all elements of SRL. Consistent
Appendix 3): to develop a website in about 20 min- with other research on feedback (e.g., Martocchio and
utes (the time determined in the pilot study). Webster 1992), all participants in this group received
We collected data through 16 experimental ses- positive feedback, regardless of their actual perfor-
sions. Participants spent an average of about 2 hours mance, to control for differential effects. The other
and 20 minutes completing the assignment: the back- group received a midpoint control script (C2 ) that did
ground information survey took about 20 minutes, not provide any SRL-specific information or feedback
the learning session with the midpoint feedback took (see Figures 4a and 4b, respectively).
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 35

Figure 3 Pretraining Scripts


(a) Pretraining Treatment Script (T1 )
Welcome to this session where you will learn to design a website using Microsoft FrontPage. FrontPage is a user-friendly software, and
you will find that designing a website is a simple and easy task. It is extremely important to learn the skills necessary to design a website
because it is a critical skill that is highly valued by employers. By attending this session, you will learn the skills to set up your own
personal website and show it to prospective employers.
Your goal in this session is to learn methods and concepts needed to design a website, and then apply these skills to develop a simple
website. You will learn through the computerized e-learning system at ∗∗∗∗∗ called ∗∗∗∗ . Remember that you must pay close attention and
stay focused on the material presented on the screen. There will be many explanations, and the system will guide you in learning. You
will find that it is easy to follow and understand. We are very confident that you will find that learning through this e-learning system is
easy, and perhaps even more structured than learning from a classroom lecture. Several business students have used it before; they found it
to be useful, and have given positive comments.
Most important, by being part of the business college and taking business courses, you are among a select set of students in the university
who have met the criteria to be admitted to attend business courses. It means you have a relatively higher GPA and are capable and smart.
We therefore believe that you are a very capable learner and will be able to focus, pay attention, and understand the material conveyed
to you. You have shown that you can set your goals for this session and apply yourself to the task of learning to design a website. We
think that you will be successful in learning to use FrontPage.
The information will be conveyed to you through eight simple, short, and consecutive modules. In the first module, you will understand
what is meant by a website and learn about the different components in a website. Then, you will learn what the FrontPage screen looks
like: i.e., the various menu options, the toolbars, and buttons on the screen. You will also learn how the application window is divided
into different areas and the names/uses of these areas. In the subsequent modules, you will understand the notion of a template, create
a webpage, and add more pages to the site. You will learn procedures to enter text on a webpage, format text, apply themes, and create
hyperlinks to other webpages and websites. Each module is typically focused on one or two important aspects related to designing a
website. Please pay attention to the material presented in each module and understand the main ideas.
Note that it is very important for you to pay close attention and stay focused on your task. Please be patient; read and follow the instructions
correctly. After every module, you should reflect and try to recall the main elements learned in the module. If necessary, visualize what you
learned, recall the main operations, and think how you would execute it. For example, recall the main procedures you would follow to
change the color of a page. Hence, you must reflect on what you learned and monitor your learning progress. It is critical to be very
focused and understand everything that is being described. Because you are capable learners, we are confident that you will be able pay
deep attention, process the information provided by the system, and monitor your learning progress.
You can pace your own learning. Hence, use your own discretion and, if you need to, go back over the material. Please feel free to take
notes as you would in any class and write down whatever you need. When you see a red arrow on the screen, it implies that you should
point/choose the option indicated by this arrow. Because you are quite capable, we think you can manage your learning and that you will
find the instructions fairly easy to follow. At the end of this session, you will have learned how to create a simple website using FrontPage.
Enjoy and have a successful learning experience!

(b) Pretraining Control Script (C1 )


Welcome to this session where you will learn some fundamental issues relating to designing a webpage. The software that you will learn
is Microsoft FrontPage. Today, you will learn through the computerized online learning system at the ∗∗∗∗∗ called ∗∗∗∗∗ . In this system, a
computer e-learning system will teach and guide you in the procedures required to design a webpage. The ∗∗∗ is a typical technology-based
training (TBT) software that is finding increased use in academic and business circles.
The American Society for Training and Development (ASTD) defines technology-based training (TBT) as “the delivery of content via
Internet, LAN or WAN (intranet or extranet), satellite broadcast, audio- or videotape, interactive TV, or CD-ROM.” TBT in turn is considered
to encompass the dimensions of Computer Based-Training (CBT) and Web-Based-Training (WBT). CBT refers to the use of computers in the
instruction and management of the teaching and learning process and includes both Computer Assisted Instruction (CAI) and Computer
Managed Instruction (CMI). WBT refers to the delivery of educational content through a Web browser over the public Internet, a private
intranet, or an extranet. Typically, they provide links to other learning resources such as references, email, bulletin boards, and discussion
groups.
A related component of TBT is the concept of Distance Learning or Distance Education. ASTD defines Distance Learning as the “educa-
tional situation in which the instructor and students are separated by time, location, or both.” Courses are delivered to remote locations via
synchronous or asynchronous means. The former refers to real-time, instructor-led learning with direct and simultaneous communication
between participants. Associated technologies will include whiteboards, audio or videoconferencing, Internet telephony, or two-way live
broadcasts. The latter has a time-lag component in the interaction between the instructor and the student. The popular self-paced courses
taken via the Internet or CD-ROM, online discussion groups and email are examples of the asynchronous mode.
The use of technology in training spans three decades and can broadly be classified into three distinct phases. The initial phase was in
the eighties where the primary driver was multimedia technology that provided rich audio, video, graphical, and animated content. The
second phase in the nineties saw the development of the Internet as the primary vehicle for the electronic dissemination of training and
learning. The development of efficient Web browsers, hyper-text markup languages, and media players propelled the Internet as a viable
training medium. The third phase commenced at the turn of the century with the explosive increase in bandwidth access and availability.
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
36 Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS

Figure 3 (Cont’d.)
This, coupled with advances in Web-design technologies and high quality streaming media, enabled the development of real-time, low-cost,
highly effective online training and learning environments.
This ∗∗∗∗ is a typical ∗∗∗ software and we expect that at the end of this session, you will have understood some of the elements that go into
creating a webpage using FrontPage. Please be patient, read the instructions, and follow them correctly. You can pace your own learning
process. Hence, use your own discretion and go backwards if you need to. Please feel free to take notes as you would in any class and
write down whatever you need to. When you see a red arrow on the screen, it implies that you should point/choose the option indicated
by this arrow.

Dependent Variables. Learning outcomes on IT instruments, we relied on Zimmerman’s (1994) iden-


software training are measured with a declarative tification of four dimensions to SRL: motives, meth-
knowledge test (i.e., a written test to evaluate learn- ods, performance outcomes, and social-environmental
ers’ conceptual understanding), and a hands-on task- resources. The motives dimension deals with learn-
performance test that evaluates their procedural ers’ motivations to learn and complete the course.
knowledge and ability to use the software (Yi and The methods dimension encompasses the application
Davis 2003). In this study, the declarative knowl- of SRL strategies. The performance outcomes dimen-
edge test included multiple choice and fill-in-the- sion deals with the monitoring of learning goals and
blank questions that were pretested in the pilot study. closely corresponds to metacognitive activities. The
The hands-on performance task required participants social-environmental resources dimension deals with
to develop a website. These tests were graded by a learners’ access to peers and teachers who can pro-
graduate student who was blind to the conditions and vide guidance (this dimension was not applicable for
trained on this task (see Appendix 3). our training context).
We conducted two types of manipulation checks,
Manipulation Checks on SRL. Researchers have one with our participants and the other with a sepa-
noted both the lack of standard instruments to rate group. We did not want to ask our participants
assess whether participants follow SRL strategies directly about their self-regulatory learning strate-
and the problems with developing such instruments gies (such as their methods for learning or moni-
(e.g., Winnie and Perry 2000). In developing our toring their learning goals) because doing so could

Figure 4 Midpoint Scripts


(a) Midpoint Treatment Script (T2 )
Dear Student,
Excellent effort and progress!!! You are making good progress toward the goal of learning to develop a website using FrontPage. This
midpoint evaluation shows that you have understood the fundamental concepts of what constitutes a website, what is meant by a hyperlink,
and how to present information on a webpage. Your scores are excellent compared with other students who have learned through this
e-learning system. Because you are learning on your own without any human instructor, this evaluation shows that you are quite capable
of paying attention to the material, understanding, learning, and monitoring your learning progress. Please continue to do so, and focus
on this learning task. In the next few modules, you will continue to learn other aspects of designing a website such as learning to create
internal and external links from your page and to apply themes to your website. Please continue with the good progress you have made
so far, and remember to reflect on the content learned in each module. Continue to pay attention and stay focused on the task and you
will achieve the goal of learning how to design a website. GOOD JOB!!
(b) Midpoint Control Script (C2 )
Dear Student,
Web-based training (WBT) is an innovative approach to distance learning in which computer-based training (CBT) is transformed by
the technologies and methodologies of the World Wide Web, the Internet, and intranets. Web-based training presents live content, as fresh
as the moment and modified at will, in a structure allowing self-directed, self-paced instruction in any topic. WBT is media-rich training
fully capable of evaluation, adaptation, and remediation, all independent of computer platform. Web-based training is an ideal vehicle for
delivering training to individuals anywhere in the world at any time. Web browsers that support 3-D virtual reality, animation, interactions,
chat, and conferencing, and real-time audio and video will offer many training opportunities.
Instructional designers and training analysts are learning more about how to write and produce WBT. Therefore, in the future you will
see a variety of WBT course offerings that will be distributed over the public Internet and private intranets. In the next few modules, you
will learn more about webpage design.
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 37

cue them to use these methods. Consequently, we for each of these methods from Kanfer et al. (1994)
measured only the motives dimension for our main and Pintrich and DeGroot (1990). To assess the perfor-
study participants. Specifically, we twice measured mance outcomes dimension (student-initiated efforts
their states of computer-learning self-efficacy (Zweig to set specific goals and monitor progress), three items
and Webster 2004) ( = 092) and motivation to learn were adapted from Schunk and Ertmer (2000) and
(Hicks and Klimoski 1987) ( = 088): once at the Schmidt and Ford (2003) (see Appendix 4B).
beginning of training after the participants had read
the pretraining scripts, and once during the training Results
session (see Appendix 4A). We expected that these Of the 134 participants who volunteered for the exper-
measures would indirectly indicate self-regulatory iment, 10 were eliminated because of their prior Web-
learning strategies because self-efficacy beliefs and development experience and six because of technical
motivation are described as being important compo- problems (their computers froze or their audio did
nents in the application of SRL strategies as per the not work). For the remaining 118 participants, the
social cognitive perspective. It is to be noted that these average age was 21.6, the average GPA was 3.3, and
manipulation checks are state measures of computer they were almost equally divided between males and
learning self-efficacy and motivation to learn and may females (see Table 1a). All had experience with pop-
have quite different effects on learning performance ular software packages like Word and PowerPoint,
than the corresponding trait measures that are con- but none had experience with Web-development soft-
sidered to be relatively stable dispositional variables ware. As shown in Tables 1a and 1b, we found no
(George 1991). significant demographic differences among the train-
The second manipulation check was conducted on a ing groups. A factor analysis and validity checks
separate group of participants. These were measured of individual learner traits showed a high correla-
at the midpoint, before participants received the mid- tion between learning orientation and self-efficacy
point scripts. The results from this group were not for self-regulation, and a lack of discriminant valid-
included in the main study and, thus, we could assess ity. Therefore, it was decided to drop the learning
the methods and performance dimensions without orientation measure and conduct the factor analy-
concern for cueing their behaviors. That is, these
experimental sessions were conducted solely to per-
Table 1a Demographic Information
form manipulation checks on whether participants
who received the treatment script (T1 ) engaged in rel- Demographic
measures T1 -T2 T1 -C2 C1 -T2 C1 -C2 Overall
atively more SRL methods and performance strate-
gies than those who received the pretraining control Mean age (SD) 215 (1.7) 218 (2.0) 218 (1.7) 214 (1.5) 216 (1.7)
script (C1 ). For this separate group of participants, Mean GPA (SD) 33 (0.4) 33 (0.4) 32 (0.4) 33 (0.5) 33 (0.4)
Number of males 14 15 13 15 57
seven experimental sessions similar to the main study Number of females 22 10 12 17 61
were conducted. After participants completed four
Note. Treatment-treatment (T1 -T2 ), treatment-control (T1 -C2 ), control-
modules, they were given the manipulation test ques- treatment (C1 -T2 ), control-control (C1 -C2 ).
tionnaire (see Appendix 4).
Consistent with Schunk and Ertmer (1999), we
Table 1b Tests of Demographic Differences Between Conditions
developed this questionnaire based on Zimmerman’s
work. Specifically, we chose methods found to be Experimental conditions
highly significant in SRL (Zimmerman and Martinez- Demographic measures T1 -T2 vs. C1 -C2 T1 -T2 vs. T1 -C2 T1 -T2 vs. C1 -T2
Pons 1986, 1988): organizing and transforming (stu-
p values of differences
dent-initiated overt or covert rearrangements of
in means test
instructional material to improve learning), and Age 040 047 046
rehearsing and memorizing (student-initiated efforts GPA 047 045 023
to memorize material by overt or covert practice). Note. Treatment-treatment (T1 -T2 ), treatment-control (T1 -C2 ), control-
Based on these descriptions, we adapted three items treatment (C1 -T2 ), control-control (C1 -C2 ).
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
38 Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS

Table 1c Factor Loadings Table 2a Manipulation Check: Motives

Items Factor loadings Experimental conditions

SR7 0888 0133 0091 −0003 Mean score (SD) Differences


SR6 0871 0122 0139 0015 Manipulation check Cronbach’s Treatment Control in means
SR10 0859 0265 −0033 −0129 measures alpha n = 61 (n = 57) (t-tests)
SR9 0820 0015 0160 0099
SR3 0774 0245 0211 −0154 After pretraining script
SR2 0741 0090 0101 −0039 Motivation to learn 078 6.37 (0.57) 6.11 (0.81) p < 003
SR4 0599 0216 0244 0221 Computer learning 076 6.09 (0.59) 5.93 (0.77) p < 011
CSE6 0166 0875 0136 0041 self-efficacy
CSE7 0227 0874 0114 0035 After midpoint script
CSE5 0086 0870 −0058 −0238 Motivation to learn 084 6.24 (0.66) 5.96 (0.90) p < 003
CSE3 0295 0752 0203 0164 Computer learning 085 6.06 (0.72) 5.90 (0.79) p < 013
CSE2 0087 0659 0372 0258 self-efficacy
SR5 0245 0459 0441 0303
SR1 0239 −0068 0760 −0152
CSE1 0055 0496 0720 −0004 Table 2b Manipulation Check: Methods
SR11 0448 0271 0519 0013
CSE4 0165 0467 −0081 0646 Experimental conditions
SR8 0466 0281 0096 −0622 Mean score (SD) Differences
Notes. Trait variables: CSE, computer-learning self-efficacy; SR, self-efficacy Manipulation check Cronbach’s Treatment Control in means
for self regulated learning. Those items in bold indicate items used for sub- measures alpha n = 16 (n = 17) (t-tests)
sequent analysis.
Organizing and 070 5.85 (0.78) 5.35 (0.83) p < 004
transforming
Table 1d Means, Standard Deviations, and Intercorrelations Rehearsing and 071 5.48 (1.00) 4.84 (1.19) p < 005
memorizing
Cronbach’s
Mean SD alpha CSE SR HO DC Note. Checks were conducted with a separate group of participants.

CSE 555 098 090 081


SR 544 108 092 0394∗∗ (0.80) Table 2c Manipulation Check: Performance Outcomes
HO 2941 441 NA 0056 0022 NA
Experimental conditions
DC 644 262 NA −0016 0044 0446∗∗ NA

Notes. Trait variables: CSE, computer-learning self-efficacy; SR, self-efficacy Mean score (SD) Differences
for self regulated learning. Dependent variables: HO, hands-on performance; Manipulation check Cronbach’s Treatment Control in means
DC, declarative knowledge; NA, not applicable. Diagonal values enclosed in measures alpha n = 16 (n = 17) (t-tests)
parentheses indicate the square root of the AVE. Performance outcomes 083 5.94 (0.79) 5.33 (0.84) p < 003

Correlation is significant at the 0.05 level (two-tailed).
∗∗
Correlation is significant at the 0.01 level (two-tailed). Note. Checks were conducted with a separate group of participants.

sis again. In Table 1c, the factor analysis of individ- struct being larger than the correlation between the
ual learner traits of computer-learning self-efficacy constructs. The Cronbach alpha tests of reliability are
and self-efficacy for self-regulation are provided. It also respectable.
is seen that seven of the 10 items in the measure The results of our manipulation checks are shown
of self efficacy for self-regulation loaded together in Tables 2a–2c. As shown in Table 2a, participants
whereas five of the seven items in the computer- who received the pretraining treatment scripts scored
learning self-efficacy scale loaded together and they higher on a state measure of computer-learning self-
had acceptable factor loadings. These seven and five efficacy beliefs and motivation to learn, suggesting
items, respectively, were used in our final analy- that our pretraining treatment scripts affected the
sis. As seen in Table 1d, the measures of individual motive dimension of SRL. Tables 2b and 2c illustrate
learner traits of computer-learning self-efficacy and that those participants who received the pretrain-
self-efficacy for self-regulation satisfy tests of conver- ing treatment script reported following a significantly
gent and discriminant validity, with the square root greater amount of SRL strategies (both methods and
of the average variance extracted (AVE) of each con- performance outcomes) compared with those who
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 39

received the pretraining control script. Note (as per We dropped the learning-orientation measure be-
theory) that all learners indicated following some cause of validity issues in relation to the other indi-
level of SRL; however, it is the quality and extent vidual traits, so we could not test Hypothesis 2A.
of SRL that is important (and the participants in the To test Hypotheses 2B and 2C concerning the effects
treatment group indicated that they applied a rela- of individual learner traits of computer learning self-
tively higher level of SRL). efficacy and self-efficacy for self-regulation on learn-
When we conducted tests of hypotheses, we found ing outcomes, we examined the covariates in the
that the dependent measures of declarative knowl- MANCOVA and in individual ANCOVAs. These indi-
edge and hands-on task performance were correlated vidual learner traits were not significantly related to
(r = 045, p < 00001). We first conducted an overall the learning outcomes.1 Thus, we did not find sup-
MANCOVA (in which the individual learner charac- port for Hypotheses 2B and 2C concerning the effects
teristics were entered as covariates). With a significant of individual learner traits on learning outcomes.
MANCOVA (Wilks’ Lambda = 0887, p < 005), we
then conducted a Dunnet’s t-test, a recommended test Discussion
for simultaneous comparison of treatment condition Based on Alavi and Leidner’s (2001) framework,
scores (see Table 3). The mean dependent measure we examined the interaction of features of informa-
scores and standard deviations for each of the four tion technology, psychological learning processes, and
conditions are shown in Table 4 and graphed in Fig- instructional strategies embedded in an e-learning-
ure 5. The scores for the treatment-treatment (T1 -T2 ) based IT training environment. Because learners may
condition are the highest, followed by treatment- not be exercising the high levels of self-directed learn-
control (T1 -C2 ), control-treatment (C1 -T2 ), and control- ing strategies required in this training set-up, we
control (C1 -C2 ) conditions. developed instructional strategies that included inter-
For Hypotheses 1A and 1B concerning the train- ventions to induce learners to apply higher levels
ing conditions, the T1 -T2 condition differed signifi- of self-directed learning strategies. Our tests showed
cantly from the C1 -C2 condition. As shown in Tables 4 that when the instructional strategy included such
and 5a, the pretraining treatment script was effective interventions that taught learners to self-regulate,
in improving the learning outcomes. Hence, Hypoth- learners applied more self-regulatory learning strate-
esis 1A was supported, suggesting that presession gies, leading to enhanced learning outcomes. We also
interventions would enhance learning outcomes. Fur- found that during the training session, providing
thermore, the midpoint scripts resulted in higher de- positive feedback to learners on their use of SRL
pendent measure scores for the treatment groups, sup- reinforces the use of SRL strategies, suggesting that
porting Hypothesis 1B as shown in Table 5b. It can learners must be reminded and motivated if they are
be seen from Tables 5a and 5b that the differences to continue using SRL strategies. Results show that
in scores between the treatment and control groups
are higher on the declarative knowledge test than the 1
We also conducted a hierarchical regression analysis using the
hands-on performance test, but both are significantly T1 -T2 and C1 -C2 conditions. On each of the dependent vari-
ables (declarative knowledge and hands-on performance) we first
different.
regressed the treatment conditions, then the individual learner
trait measures, and then the interaction terms between individ-
Table 3 Differences in Learning Outcomes for Experimental ual learner trait measures and treatment conditions. On declara-
Conditions tive knowledge, the addition of the individual learner traits did
not account for significant additional variance (R2 change = 004,
Experimental conditions
p < 085), nor did the addition of the interaction terms (R2 change =
T1 -T2 vs. T1 -T2 vs. T1 -T2 vs. 005, p < 019). Similarly, with hands-on performance, the addition
Learning outcomes C1 -C2 T1 -C2 C1 -T2 of individual learner traits did not account for significant additional
variance (R2 change = 001, p < 077), nor did the addition of the
Difference in means (p values) interaction terms (R2 change = 001, p < 080). Thus, the results do
Declarative knowledge 0002 0071 0006
not indicate interactions between the individual learner traits and
Hands-on performance 0040 0395 0254
training interventions.
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
40 Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS

Table 4 Learning Outcomes by Condition

Experimental conditions

Treatment-treatment Treatment-control Control-treatment Control-control


Learning outcomes (T1 -T2 ) n = 36 (T1 -C2 ) n = 25 (C1 -T2 ) n = 25 (C1 -C2 ) n = 32

Mean outcome score (SD)


Declarative knowledge (Max = 15) 7.67 (2.50) 6.40b (2.18) 5.76a (2.31) 5.63a (2.87)
Hands-on performance (Max = 35) 30.56 (4.14) 29.56 (3.64) 29.16c (4.23) 28.12b (5.36)

Note. Means with superscripts a, b, and c differ from the corresponding treatment-treatment (T1 -T2 ) condition at significance levels
of 0.01, 0.05, and 0.1 respectively.

the group instructed to follow SRL before and dur- entation and self-efficacy for self-regulation measures
ing training achieved the highest learning outcomes, were correlated, perhaps because each of them taps
whereas the group that received no instructions to into different forms of regulation. Although learning
self-regulate learning either before or during train- orientation taps into regulation relating to achieve-
ing received the lowest learning scores. Our results ment motivation, and self-efficacy for self-regulation
showed that all learners apply some level of SRL, but taps into academic motivation, they did not emerge
e-learning necessitates a higher level of SRL. Interven- as distinct measures and could not be tested simulta-
tions like those examined in this study could induce neously. We tested self-efficacy for self-regulation and
learners to apply the higher levels of SRL needed in computer-learning efficacy; these did not have direct
e-learning-based training contexts, thereby leading to or interactive effects on learning performance. One
higher learning outcomes. explanation can be found in the theory itself, which
Based on past IS training research, we hypothe- states that SRL is very situation specific and has to
sized that individual learner traits relating to SRL be activated in any given context, pointing to the
may influence learning outcomes. Our results did not importance of interventions. Learners who had higher
support this premise. We found that the learning ori- self-efficacy beliefs for self-regulation and computer
Figure 5 Graphs of Learning Outcomes
learning may not all have activated their self-efficacy
beliefs in the new e-learning context. Another expla-
Hands-on performance
31.0 nation for the lack of significant effects for individual
30.5 learner traits can be found in the debates over the
30.0
29.5
29.0 Table 5a Learning Outcomes for Pretraining Conditions
28.5
28.0 Experimental conditions
Differences
27.5
Outcome measures for Treatment Control in means
27.0
pretraining conditions n = 61 n = 57 (t-tests)
26.5
T1- T2 T1- C2 C1- T2 C1- C2
Mean outcome score (SD)
Declarative knowledge Declarative knowledge 715 (2.44) 568 (2.62) p < 0001
8.0 Hands-on performance 3014 (3.94) 2861 (4.87) p < 003
7.5
7.0
6.5 Table 5b Learning Outcomes for Midpoint Conditions
6.0
5.5
Experimental conditions
Differences
5.0 Outcome measures for Treatment Control in means
4.5 midpoint conditions n = 61 n = 57 (t-tests)
4.0
T1- T2 T1- C2 C1- T2 C1- C2 Mean outcome score (SD)
Note. T1 -T2 , Treatment-treatment; T1 -C2 , treatment-control; C1 -T2 , control- Declarative knowledge 689 (2.59) 596 (2.60) p < 003
treatment; C1 -C2 , control-control. Hands-on performance 2998 (4.21) 2879 (4.69) p < 007
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 41

role of individual traits and their effects on learning that SRL is one such learner skill. This study advances
performance. Researchers have argued that there is a the body of IT research on social cognitive theory
marked difference in the effects of the distal trait vari- (Compeau and Higgins 1995a, Johnson and Marakas
ables when compared to the proximal state variables 2000, Yi and Davis 2003) to encompass SRL. Therefore,
of the same construct (George 1991). Our findings, our study helps contribute to a cumulative theoreti-
along with other prior findings, signal conflicting find- cal foundation for IT training both from a TML and a
ings on the relationship between trait-like individual social cognitive research perspective.
differences and learning performance, suggesting that
we must investigate more closely the influence of trait- Implications for Research and Practice
like measures of individual differences relating to self- This study introduces the concept of self-regulation
regulation and learning performance. to IS research and shows that it is relevant to train-
Our findings that individual traits relating to SRL ing programs. Our findings are based on a strong
did not have an influence on learning outcomes, but experimental design, with data collected over four
that interventions did, point to the value of encourag- time periods and training conditions crossed at the
ing learners to follow SRL strategies when taking part midpoint. As with any experimental study, however,
in e-learning training. Results show that, unlike other this research has several limitations. We used self-
IS training environments where individual learner reports on the use of SRL as a manipulation check
traits play a relatively important role, in e-learning- rather than interviews, observation, or verbal proto-
based training environments, instructional strategies cols, because self-reports provide a method by which
may play a more critical role than pre-existing differ- to compare the use of SRL among the groups. Fur-
ences in individual learner traits. thermore, although research suggests that training
Though the pretraining scripts were fairly straight- outcomes should be evaluated using several dimen-
forward and simple to implement, we found strong sions, we focused only on learning outcomes due to
learning effects, consistent with other research manip- our primary interest in improving the effectiveness
ulating pretraining interventions (e.g., Webster and of e-learning. Consequently, this research should be
Martocchio 1995). According to the social cognitive extended to other outcome variables.
perspective, people engage in some degree of SRL, Another limitation is the use of student partici-
but what truly matters is the quality and extent to pants. Although we used college students as partic-
which they apply these strategies in a specific con- ipants, our findings can be considered to be fairly
text. Messages such as those provided in the pre- realistic: the students were learning the topic for the
training scripts seem to persuade the learner to apply first time and website design is an important skill
SRL strategies in the given training situation. From for this group. This research, however, should also
an organizational standpoint, once an e-learning tech- be extended to employees in the workplace. SRL
nology platform has been purchased and an IT course may be more relevant to employee training because
chosen as the target content, interventions in the of its capacity to counteract work-related distrac-
instructional strategy such as those tested in this tions; however, our study does not address desktop-
study may be the most suitable means to increase based training, only training conducted in controlled
learning effectiveness. But it is also worthwhile to environments. Reports indicate that many organiza-
research whether persuasive messages such as those tions, including the American federal government,
used in this study could be incorporated as part of the use e-learning as the primary method to deliver tech-
e-learning technology infrastructures so that learners nology training to employees (e.g., Hasson 2005) and
can be encouraged to apply SRL strategies when they research should be extended to this setting.
undergo e-learning courses. We applied paper-based interventions to influence
Prior IS research has concluded that the use of TML- self-regulatory strategies, but we believe that these
based methods for IT training may require the identi- interventions could be designed as an integral part
fication of special skill sets to achieve success (Piccoli of e-learning platforms. There are several major play-
et al. 2001). In this study, we identify and demonstrate ers in the corporate IT e-learning market who offer
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
42 Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS

a variety of IT training courses covering hardware, study indicates that learners benefit by self-regulating
software, programming languages, Web development, their learning in new situations and, therefore, an
networking, and database administration, and also interesting corollary would be to examine whether the
offer certification courses (e.g., Learning Steps 2007, extent of use of self-regulatory strategies may explain
Skill Soft 2007, Thomson Course Technology 2006). some of the conflicting training outcomes observed in
When we examined these training platforms, we prior IS training research (e.g., Davis and Weidenbeck
found that they provide course content in a very pro- 1998, Olfman and Mandviwalla 1994b, Santhanam
cedural manner (i.e., they instruct the learner to oper- and Sein 1994).
ate commands and menu options one by one). They Another avenue for future research is to utilize
provide messages before the start of training that out- Alavi and Leidner’s (2001) framework in other con-
line the course and inform the learners about the texts. In this study, we applied the framework to one
course objectives. Thus, it could be argued (as per Fig- specific TML environment—e-learning—but it could
ure 2), that these systems provide learning goals for be applied in a similar manner to improve learning
the training session but do not instruct the learner on outcomes in other TML environments. For example, in
any other aspect of SRL strategy. collaborative TML environments, the expected learn-
We believe that these commercial training tools ing process involves cooperative learning with other
could be designed such that they deliver persuasive learners. But research findings indicate that not all
messages to learners to improve e-learning outcomes. learners use the communication tools to cooperatively
That is, one approach would be for vendors to add learn and improve learning outcomes in these TML
instructions such as those used in the study to the exist- environments. Cooperative learning theories suggest
ing pretraining instructions that are provided to the that learners can learn effectively and improve learn-
learner. Several commercial tools also provide features ing outcomes only if instructional strategies include
such as text boxes for trainers to add their own instruc- exercises such as group projects and case studies
tions; another approach, therefore, would be for train- that require a high degree of interdependence and
ers to add messages similar to those used in this study. learner interaction (Cohen 1994). IS research could
These training tools could also provide feedback dur- therefore determine the appropriate conditions, i.e.,
ing the training session as was done in this study. Thus, the types of IT group projects, and other modifica-
if these commercial tools could deliver, through the tions to instructional strategies that can induce learn-
technology component of the e-learning platform, sim- ers to use the communication features in collaborative
ilar messages to those we provided via paper-based learning environments, not only to communicate, but
scripts, we believe it could lead to more SRL and bet- also to interact share ideas, and thus develop higher
ter learning outcomes. Future research should examine learning outcomes.
the software, programming, and other technical issues Our finding that individual trait differences relat-
in the design of e-learning tools that can provide such ing to self-regulatory skills did not significantly affect
messages at the beginning and during the training ses- learning performance opens new research avenues.
sion, and also identify how they could be personalized Researchers could extend the examination of the rela-
to each learner. tionship between traits and learning performance by
One avenue for future research on SRL is to deter- using state measures (e.g., state anxiety) that might
mine if users’ SRL skills could be developed via train- play a mediating role (Chen et al. 2000). Particularly in
ing programs. After users are trained to use SRL learning and training contexts, relationships between
skills, users could participate in e-learning, and re- trait-like individual-difference constructs and learn-
searchers could then determine whether the learn- ing performance seem to vary across performance
ers applied their newly developed skills. If they did episodes, with positive effects in some cases but not in
not, what situational reminders might be needed? others. For example, researchers have found a mixed
Although we specifically focused on IT training in this relationship between performance orientation and the
study, the proposed interventions could be useful in regulation of cognitive learning strategies. Some find-
other e-learning-based training courses as well. This ings suggest no relationship (e.g., Ford et al. 1998,
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 43

Middleton and Midgley 1997) and others suggest Table A.1 Pilot Study Results on Impact of Pretraining Interventions
an interactive relationship (Schmidt and Ford 2003). Training outcomes Training outcomes
A more detailed theorized examination of relation- with pretraining with pretraining
ships among more distal traits, learning performance, treatment scripts control scripts
and more proximal state-like constructs is necessary Mean outcome score (SD) 9 (1.84) 8.08 (1.82)
(Chen et al. 2000). Furthermore, as described earlier, Observations 34 35
self-regulatory competencies, such as academic writ- Differences in means (t-test) p < 002

ing and mathematical reasoning, may differ across scripts. The pretraining treatment script encouraged par-
domains (Schunk and Zimmerman 1997, Zimmerman ticipants to follow self-regulatory strategies, whereas the
and Kitsantas 1997). Therefore, another avenue for control script provided general information on communica-
research is to determine whether self-regulatory traits tion technologies. We also measured participants’ individ-
for e-learning tasks differ from other academic self- ual differences in goal orientation and self-efficacy beliefs.
We recruited participants who were randomly assigned
regulatory skills. If so, could we develop a measure to a group that received either the pretraining treatment
to test these traits and determine which employees script or the pretraining control script. When participants
can or cannot learn effectively from e-learning train- reported for training, they completed a background ques-
ing programs? tionnaire that measured their traits. Participants then took
Our study informs IT training practice. Almost the e-learning course on FrontPage, which consisted of four
modules. We measured their learning outcomes with a con-
every employee needs IT skills, and when new IS are ceptual understanding test. Table A.1 illustrates that the
implemented, corporations increasingly conduct train- group that received the pretraining treatment script exhib-
ing via e-learning. Unfortunately, e-learning-based IT ited higher learning outcomes.
training has yielded disappointing results. Based on Based on these pilot results and feedback from partici-
our findings, organizations might use simple inter- pants, we elected to change our experimental materials and
procedures. First, we modified and fine-tuned the question-
ventions before and during training to improve learn- naires and scripts. We determined that if we were going
ing outcomes. For those vendors designing e-learning to provide midpoint feedback, participants must undergo
infrastructures, our results suggest that these inter- training with eight course modules, not four. We modified
ventions could be imbedded in the technology plat- the pretraining treatment scripts to be clearer, more specific,
form to make learning more effective. Finally, based and to reflect the goals for these eight modules. We devel-
oped midpoint scripts.
on social cognitive theory, our study findings alert IS We conducted Pearson correlation tests between the indi-
researchers to an important concept that could be piv- vidual traits and the comprehension scores. Although the
otal in explaining the learning and use of information correlation between computer learning self-efficacy and the
systems: self-regulation. training outcome score was positive (Pearson correlation =
0.110, p = 053), it was not significant. The correlation
between self-efficacy for self-regulatory learning and learn-
Acknowledgments ing outcomes was also not significant (Pearson correlation =
This work was supported in part by an award to the first
−0.09, p = 059). We felt that we may not have seen
author from the Kentucky Science and Engineering Founda- significant correlations because our sample size was rel-
tion as per Award Agreement #KSEF-148-502-03-80 with the atively small. Also, because all the participants were of
Kentucky Science and Technology Corporation. The authors similar backgrounds and experience, their scores on these
thank the reviewers and the associate and senior editor for measures showed low variance. We also decided that to mea-
the many insightful comments that helped to improve this sure individual differences in traits we must administer the
research study and shape this paper. They also benefited background questionnaire well before the training session
from feedback provided by Adrienne Olnick Kutzschan of (i.e., when participants volunteered for the experiment). We
Queen’s University and research seminar participants from felt that this timing would be a better measure of their trait
Temple University and Arizona State University. variables. Therefore, we made these changes and planned to
collect a larger set of data for the main experiment.
Appendix 1. Description of the Pilot Test
After a few tests to check that the infrastructure was Appendix 2. Questionnaire Items for
working and that typical students could interact with the Measuring Individual Traits
e-learning system, we conducted our pilot test. For the (Anchored on seven-point Likert scales; anchors: strongly
pilot test, we developed pretraining treatment and control disagree to strongly agree.)
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
44 Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS

Learning Orientation (2) FrontPage creates a homepage called Index.htm and


1. The opportunity to learn new things is important to me. two associated folders called Images and .
2. The opportunity to do challenging work is important (3) Which of the following enables you to create a new
to me. webpage from a template?
3. I prefer to work on tasks that force me to learn new 1. page templates
things. 2. work pane
4. If I don’t succeed on a difficult task, I plan to try 3. template bar
harder the next time. 4. content pane
5. In learning situations, I tend to set fairly challenging 5. task pane
goals for myself. (4) When a hyperlink is created, the destination of the
6. I am always challenging myself to learn new concepts. hyperlink is encoded as a .
7. The opportunity to extend my range of abilities is (5) Bookmark links can connect a page in a website to a
important to me. location on:
1. a page in another website
Computer Learning Self-Efficacy 2. a page in the same website
1. I feel confident using a computer to learn about and 3. the same page
apply new concepts. 4. 2 and 3
2. Using a computer is an efficient way for me to learn 5. 1, 2, and 3
new things. (6) In FrontPage, bookmark links are displayed using a
3. I could apply new concepts that I learned from a com- .
puterized training program. (7) A navigation bar is a set of text or button hyperlinks
4. I don’t feel that I could learn new skills from a com- typically used to:
puterized training program.∗ 1. access pages in another website
5. It would be easy for me to become skillful at tasks 2. test the links in a website
learned from a computerized training program. 3. access pages in the same website
6. I would be comfortable using a computerized training 4. give a pictorial view of the links
program. 5. none of the above
7. I could successfully use a computerized training pro- (8) The navigation bar uses the as labels for the
gram. hyperlink.
Self-Efficacy for Self-Regulation (9) To ensure that the navigation bar contains hyperlinks
1. I am able to finish homework assignments by dead- to only the webpages linking to that webpage, which of the
lines. following Link Bar Properties options should be selected:
2. I am able to study even when there are other interest- 1. parent level
ing things to do. 2. global level
3. I am able to concentrate on school subjects. 3. child pages under global level
4. I am able to take class notes of class instruction. 4. child level
5. I am able to use the library and the internet for infor- 5. none of the above
mation for class assignments. (10) The same formatting can be applied to other text on
the same page using the option.
6. I am able to plan my schoolwork.
7. I am able to organize my schoolwork. Appendix 3B. Web Development Activities and
8. I am able to remember information presented in class Scores (Maximum of 35 points)
and textbooks. Now you are ready to design your own website. Please
9. I am able to arrange a place to study at my resi- click on “Start” at the bottom left corner of your screen, and
dence/home without distractions. select “Programs,” “Microsoft FrontPage.” This will open
10. I am able to motivate myself to do schoolwork. Microsoft FrontPage for you. Please design a simple website
11. I am able to participate in class discussions. based on the activities listed below. You have 20 minutes

This item was reverse coded. to complete this exercise.

Appendix 3A. Declarative Knowledge Activity 1 (Maximum of 11 points)


Test Questions Create a personal home page named index.htm or
(1) The HTML button is used for: default.htm; 2 points.
1. converting the text material to source code Enter 3–4 lines of text related to you containing the phrases
2. displaying how a page will appear in a browser “∗∗∗∗ College of Business” and “hobbies.” [e.g., My name
3. generating text material from the source code is XYZ and I am an accounting student at the ∗∗∗∗ College
4. ensuring validation of JavaScript with source code of Business. My hobbies include reading, swimming and
5. none of the above hiking   ]. 3 points.
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 45

Format the text in Times New Roman. 1 point. Appendix 4B Additional Manipulation Check
Provide the text material with the heading “My Home Measures
Page” formatted in bold, Arial. 2 points.
Apply a theme to this webpage. 3 points. Methods Measures of SRL (Used with a Separate
Group of Participants)
Activity 2 (Maximum of 9 points) Organizing and Transforming
Create a webpage named hobbies.htm. 1 point. 1. I tried to put important ideas into my own words.
Enter 3–4 lines of text related to your hobbies. 1 point. 2. I used what I have learned in the past to try to succeed
Format the text in Times New Roman. 1 point. at learning the material.
Provide the text material with the heading “My Hobbies” 3. I tried to connect information to things that I already
formatted in bold, Arial. 2 points. know.
At the very end of the text material, on a new line, enter
the phrase “To Home Page.” 1 point. Rehearsing and Memorizing
Apply a suitable theme to this webpage. 3 points. 1. I tried to recall the key ideas from earlier in the
tutorial.
Activity 3 (Maximum of 15 points) 2. I went over things I had learned in prior modules.
From your home page, convert the phrase “∗∗∗∗ College 3. I worked on the Web-design procedures as they were
of Business” into an external hyperlink pointing to being presented even though I did not have to.
http://∗∗∗∗ .∗∗∗ .edu/. 5 points.
From the same webpage, convert the phrase “hobbies” into Performance Measures of SRL (Used with a
an internal hyperlink pointing to your hobbies page. Separate Group of Participants)
5 points. 1. I tried to think through each topic and decide what I
From you hobbies webpage, convert the phrase “To Home was supposed to learn.
Page” into an internal hyperlink pointing to your home 2. I was aware of the progress of my learning with
page. 5 points. respect to my goals for this session.
3. I thought about what things I needed to do to learn.
Appendix 4A. Questionnaire Items for
Manipulation Checks References
(Anchored on 7-point Likert scales; anchors: strongly dis- Agarwal, R., T. W. Ferratt. 2001. Crafting an HR strategy to meet
the need for IT workers. Comm. ACM 44(7) 59–64.
agree to strongly agree.)
Agarwal, R., V. Sambamurthy, R. Stair. 2000. Research report: The
Motives Measures of SRL (Used with evolving relationship between general and specific computer
self-efficacy – An empirical assessment. Inform. Systems Res.
Main Study Participants) 11(4) 418–430.
Computer Learning Self-Efficacy Alavi, M. 1994. Computer-mediated collaborative learning: An
1. In this session, I feel confident in using a computer to empirical evaluation. MIS Quart. 18(2) 150–174.
learn about and apply new concepts. Alavi, M., D. Leidner. 2001. Research commentary: Technology
mediated learning – A call for greater depth and breadth of
2. Using a computer in this session is an efficient way
research. Inform. Systems Res. 12(1) 1–10.
for me to learn new things.
Alavi, M., B. Wheeler, J. Valacich. 1995. Using IT to reengineer busi-
3. I could apply new concepts that I will learn from this ness education: An exploratory investigation of collaborative
computerized online training program. tele-learning. MIS Quart. 19(3) 293–312.
4. I don’t feel that I could learn new skills from this com- Alavi, M., G. M. Marakas, Y. Yoo. 2002. A comparative study of dis-
puterized training program.∗ tributed learning environments on learning outcomes. Inform.
5. It would be easy for me to become skillful at tasks Systems Res. 13(4) 404–415.
learned from this computerized training program. Allen, I., J. Seaman. 2005. Growing by Degrees: Online Education in
6. I am comfortable using a computerized training pro- the United States, 2005. The Sloan Consortium, Needham, MA.
gram in this session. Anohina, A. 2005. Analysis of the terminology used in the field of
virtual learning. Educational Tech. Soc. 8(3) 91–102.
7. I could successfully use a computerized training pro-
Bandura, A. 1991. Social cognitive theory of self-regulation. Organ.
gram such as the one in this session. Behav. Human-Decision Processes 50(2) 248–287.
Motivation to Learn Bandura, A. 1997. Self-Efficacy: The Exercise of Control. W. H.
1. In this session I will exert considerable effort in learn- Freeman, New York.
ing the material on FrontPage. Beaubien, J. M., S. C. Payne. 1999. Individual goal-orientation as a
predictor of job and academic performance: A meta analytic
2. I am motivated to learn the material on FrontPage pre-
review and integration. Proc. 14th Annual Meeting of The Society
sented in this session. for Indust. and Organ. Psych., SIOP, Bowling Green, OH.
3. I am trying to learn as much as I can about FrontPage Bell, B. S., S. W. Kozlowski. 2002. Adaptive guidance: Enhancing
from this session. self-regulation, knowledge, and performance in technology-

This item was reverse coded. based training. Personnel Psych. 55(2) 267–306.
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
46 Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS

Benbunan-Fich, R. 2002. Improving education and training with IT. Gupta, S., R. Bostrom. 2005. Research framework for collaborative
Comm. ACM 45(6) 94–99. e-learning in an end-user training context. Proc. 11th Americas
Benbunan-Fich, R., S. R. Hiltz. 2003. Mediators of the effectiveness Conf. Inform. Systems. Assoc. Inform. Systems, Atlanta.
of online courses. IEEE Trans. Professional Comm. 46(4) 298–312. Hardaway, D. E., R. W. Scamell. 2005. Use of a technology-mediated
Berge, Z. L., S. Mrozowski. 2001. Review of research in distance learning instructional approach for teaching an introduction
education: 1990 to 1999. Amer. J. Distance Ed. 15(3) 5–19. to information technology course. J. Inform. Systems Ed. 16(2)
Bernard, R. M., P. C. Abrami, Y. Lou, E. Borokhowski, A. Wade, 137–147.
L. Woznoy, P. A. Wallet, M. Fiset, B. Huang. 2004. How Hasson, J. 2005. E-learning: A progress report. Federal Computer
does distance education compare with classroom instruction? Week, FCW Media Group, Falls Church, VA, August 29.
A meta-analysis of the empirical literature. Rev. Educational Res. http://www.fcw.com/print/11_34/news/90465-1html.
74(3) 379–439. Hicks, W. D., R. J. Klimoski. 1987. Entry into training programs
Boekaerts, M., P. R. Pintrich, M. Zeidner. 2000. Handbook of Self- and its effects on training outcome: A field experiment. Acad.
Regulation. Academic Press, San Diego, CA. Management J. 30(3) 542–552.
Bostrom, R. 2003. Facilitating learning through technology. Proc. 9th Homer, J., D. Povar. 2004. The 2004 state of the industry
Americas Conf. Inform. Systems. Assoc. Inform. Systems, Atlanta. report: Press release from the American Society for Train-
ing and Development. Training and Development (March).
Bostrom, R., L. Olfman, M. K. Sein. 1990. The importance of learn-
http://www.astd.org/.
ing style in end-user training. MIS Quart. 14(1) 101–119.
Johnson, R. D., G. M. Marakas. 2000. The role of behavioral mod-
Brett, J., D. VandeWalle. 1999. Goal orientation and goal content as
eling in computer skills acquisition: Toward refinement of the
predictors of performance in a training program. J. Appl. Psych.
model. Inform. Systems Res. 11(4) 402–417.
84(6) 863–873.
Jonassen, D. H., ed. 2004. Handbook of Research on Educational
Brown, K. 2001. Using computers to deliver training: Which
Communications and Technology. Lawrence Erlbaum Associates,
employees learn and why? Personnel Psych. 54(2) 271–296.
Mahwah, NJ.
Bureau of Labor Statistics. 2004. BLS releases 2002–2012 employ-
Jonassen, D. H., T. C. Reeves. 2001. Learning with technology:
ment projections. http://www.bls.gov/news.release/ecopro.
Using computers as cognitive tools. D. H. Jonassen, ed. Hand-
t04.htm.
book of Research for Educational Communications and Technology.
Chen, G., S. M. Gully, J. Whiteman, R. N. Kilcullen. 2000. Exami- Macmillan, New York, 693–719.
nation of relationships among trait-like individual differences,
Jones, T. H., R. Paolucci. 1999. Research framework and dimensions
state-like individual differences, and learning performance.
for evaluating the effectiveness of educational technology sys-
J. Appl. Psych. 85(6) 835–847.
tems on learning. J. Res. Comput. Ed. 32(1) 17–27.
Cohen, E. G. 1994. Restructuring the classroom: Conditions for pro-
Kanfer, R. A., L. Philip, T. C. Murtha, B. Dugdale, L. Nelson. 1994.
ductive small groups. Rev. Educational Res. 64(1) 1–35.
Goal setting, conditions of practice, and task performance:
Colquitt, J., J. LePine, R. Noe. 2000. Toward an integrative theory of A resource allocation perspective. J. Appl. Psych. 79(6) 826–835.
training motivation: A meta-analytic path analysis of 20 years
Kauffman, D. 2004. Self-regulated learning in web-based environ-
of research. J. Appl. Psych. 85(5) 678–707.
ments: Instructional tools designed to facilitate self-regulated
Compeau, D., C. A. Higgins. 1995a. Application of social cognitive learning. J. Educating Comput. Res. 30(1/2) 139–162.
theory to training for computer skills. Inform. Systems Res. 6(2)
Keegan, D. 1996. Foundations of Distance Education, 3rd ed. Rout-
118–143.
ledge, London, UK.
Compeau, D., C. A. Higgins. 1995b. Computer self-efficacy: Devel-
Learning Steps. 2007. Online computer training courses and
opment of a measure and initial test. MIS Quart. 19(2) 189–211.
distance learning tutorials. http://www.learningsteps.com/
Compeau, D., L. Olfman, M. K. Sein, J. Webster. 1995. End-user courses.php.
training and learning. Comm. ACM 38(7) 24–26.
Leidner, D., M. Fuller. 1997. Improving student processing and
Coppola, N., S. Hiltz, N. Rotter. 2002. Becoming a virtual profes- assimilation of conceptual information: GSS supported collab-
sor: Pedagogical roles and asynchronous learning networks. orative learning vs. individual constructive learning. Decision
J. Management Inform. Systems 18(4) 169–189. Support Systems 20(2) 149–163.
Davis, S., S. Wiedenbeck. 1998. The effect of interaction style and Leidner, D., S. Jarvenpaa. 1995. The use of information technology
training method on end-user learning of software packages. to enhance management school education: A theoretical view.
Interacting with Comput. 11(2) 147–172. MIS Quart. 19(3) 265–291.
Fisher, S., J. K. Ford. 1998. Differential effects of learner effort and Lim, K. H., L. M. Ward, I. Benbasat. 1997. An empirical study of
goal orientation on two learning outcomes. Personnel Psych. computer system learning: Comparison of co-discovery and
51(2) 397–420. self-discovery methods. Inform. Systems Res. 8(3) 254–272.
Ford, J. K., E. M. Smith, D. A. Weissbein, S. M. Gully, E. Salas. 1998. Lou, Y., P. C. Abrami, S. D’Apollonia. 2001. Small group and indi-
Relationships of goal orientation, metacognitive activity, and vidual learning with technology: A meta-analysis. Rev. Educa-
practice strategies with learning outcomes and transfer. J. Appl. tional Res. 71(3) 449–521.
Psych. 83(2) 218–233. Markus, L. M. 2005. Innovations in information systems education.
Gagne, R. M. 1977. The Conditions of Learning. Holt, Rinehart, and Paper presented as part of the Special Interest Group on Edu-
Winston, New York. cation. Proc. 11th Americas Conf. Inform. Systems. Assoc. Inform.
Gagne, R. M., L. J. Briggs, W. W. Wager. 1992. Principles of Instruc- Systems, Atlanta.
tional Design, 4th ed. Hartcourt Brace Jovanich, Fort Worth, TX. Martocchio, J. J., J. Webster. 1992. Effects of feedback and cognitive
George, J. M. 1991. State or trait: Effects of positive mood on proso- playfulness on performance in microcomputer software train-
cial behaviors at work. J. Appl. Psych. 76(2) 299–307. ing. Personnel Psych. 45 553–578.
Santhanam, Sasidharan, and Webster: Using Self-Regulatory Learning to Enhance E-Learning-Based IT Training
Information Systems Research 19(1), pp. 26–47, © 2008 INFORMS 47

Middleton, M. J., C. Midgley. 1997. Avoiding the demonstration of Thomson Course Technology. 2006. Teach with technology. http://
lack of ability: An underexplored aspect of goal theory. J. Edu- www.course.com/technology/.
cational Psych. 89(4) 710–718. VandeWalle, D. 1997. Development of a work domain goal orienta-
Moller, L. 2002. Introduction to the special issue of online training tion instrument. Educational Psych. Measurement 57(6) 995–1015.
in an online world. Quart. Rev. Distance Ed. 3(1) 7–8. Venkatesh, V. 1999. Creation of favorable user perceptions: Explor-
Noffsinger, J. S. 1926. Correspondence Schools, Lyceums, Chautauquas. ing the role of intrinsic motivation. MIS Quart. 23(2) 239–260.
Macmillan, New York. Webster, J., J. J. Martocchio. 1992. Microcomputer playfulness:
Olfman, L., M. Mandviwalla. 1994a. Conceptual vs. procedural soft- Development of a measure with workplace implications. MIS
ware training for graphical user interfaces: A longitudinal field Quart. 16(2) 201–226.
experiment. MIS Quart. 18(4) 405–426. Webster, J., J. J. Martocchio. 1995. The differential effects of software
Olfman, L., M. Mandviwalla. 1994b. An experimental analysis of training previews on training outcomes. J. Management 21(4)
software training manuals. Inform. Systems J. 5(1) 19–36. 757–787.
Olfman, L., R. P. Bostrom, M. K. Sein. 2006. Developing training Winnie, P. H. 1995. Inherent details in self-regulated learning. Edu-
strategies with an HCI perspective. D. Galletta, P. S. Zhang, cational Psychologist 30(4) 173–187.
eds. Human-Computer Interaction and Management Information Winnie, P. H., N. E. Perry. 2000. Measuring self-regulated learn-
Systems: Applications. M. E. Sharpe, Armonk, NY, 258–283. ing. M. Boekarts, P. Pintrich, M. Zeidner, eds. Handbook of Self-
Piccoli, G., R. Ahmad, B. Ives. 2001. Web-based virtual learning Regulation. Academic Press, San Diego, CA, 532–564.
environments: A research framework and a preliminary assess- Yi, M. Y., F. D. Davis. 2003. Developing and validating an observa-
ment of effectiveness in basic IT skills training. MIS Quart. tional learning model of computer software training and skill
25(4) 401–426. acquisition. Inform. Systems Res. 14(2) 146–169.
Pintrich, P., E. DeGroot. 1990. Motivational and self-regulated learn- Zhang, D., J. L. Zhao, L. Zhou, J. F. Nunamaker. 2004. Can
ing components of classroom academic performance. J. Educa- e-learning replace classroom learning? Comm. ACM 47(5)
tional Psych. 82(1) 33–40. 75–79.
Richardson, J. C., K. Swan. 2003. Examining social presence in Zimmerman, B. J. 1989. A social cognitive perspective of self-regu-
online courses in relation to students’ perceived learning and lated academic learning. J. Educational Psych. 81(3) 329–339.
satisfaction. J. Asynchronous Learn. Networks 7(1) 68–88. Zimmerman, B. J. 1994. Dimensions of academic self-regulation:
Rossett, A., L. Schafer. 2003. What to do about e-dropouts: What if A conceptual framework for education. D. Schunk,
it’s not e-learning but the e-learner? Training and Development B. J. Zimmerman, eds. Self-Regulation of Learning and Perfor-
(June) 40–46. mance, Issues and Educational Applications. Lawrence Erlbaum
Associates, Hillsdale, NJ, 3–21.
Salas, E., J. A. Canon-Bowers. 2001. The science of training: A
decade of progress. Annual Rev. Psych. 52(10) 471–499. Zimmerman, B. J. 1995. Self-regulation involves more than
metacognition: A social cognitive perspective. Educational Psy-
Santhanam, R., M. Sein. 1994. Improving end-user proficiency:
chologist 30(4) 217–221.
Effects of conceptual training and task variation. Inform. Sys-
tems Res. 5(4) 378–399. Zimmerman, B. 2000. Attaining self-regulation: A social cognitive
perspective. M. Boekarts, P. Pintrich, M. Zeidner, eds. Handbook
Sasidharan, S., R. Santhanam. 2006. Technology-based training: of Self-Regulation. Academic Press, San Diego, CA, 13–39.
Toward a learner centric research agenda. D. Galletta, P. Zhang,
Zimmerman, B. J., A. Kitsantas. 1997. Developmental phases in
eds. Human-Computer Interaction and Management Information
self-regulation: Shifting from process goals to outcome goals.
Systems: Applications. M. E. Sharpe, Armonk, NY, 247–257.
J. Educational Psych. 89(1) 29–36.
Schmidt, A. M., J. K. Ford. 2003. Learning within a learner control
Zimmerman, B. J., M. Martinez-Pons. 1986. Development of a struc-
training environment: The interactive effects of goal orientation
tured interview for assessing student use of self-regulated
and meta-cognitive instruction on learning outcomes. Personnel
learning strategies. Amer. Educational Res. J. 23(4) 614–628.
Psych. 56(2) 405–429.
Zimmerman, B. J., M. Martinez-Pons. 1988. Construct validation of
Schunk, D. H. 2001. Social cognitive theory and self-regulated
a strategy model of student self-regulated learning. J. Educa-
learning. B. Zimmerman, D. Schunk, eds. Self-Regulated Learn-
tional Psych. 80(3) 284–290.
ing and Academic Achievement: Theoretical Perspectives. Lawrence
Erlbaum Associates, Hillsdale, NJ, 125–151. Zimmerman, B. J., M. Martinez-Pons. 1990. Student differences
in self-regulated learning: Relating grade, sex, and giftedness
Schunk, D. H., P. A. Ertmer. 1999. Self-regulatory processes during to self-efficacy and strategy use. J. Educational Psych. 82(1)
computer skill acquisition: Goal and self-evaluative influences. 51–59.
J. Educational Psych. 91(2) 251–260.
Zimmerman, B. J., D. H. Schunk. 2001. Theories of self-regulated
Schunk, D. H., P. A. Ertmer. 2000. Self-efficacy and academic learn- learning and academic achievement: An overview and analy-
ing: Self-efficacy enhancing interventions. M. Boekaerts, P. R. sis. B. J. Zimmerman, D. H. Schunk, eds. Self-Regulated Learn-
Pintrich, M. Zeidner, eds. Handbook of Self-Regulation. Academic ing and Academic Achievement: Theoretical Perspectives. Lawrence
Press, San Diego, CA, 631–650. Erlbaum Associates, Hillsdale, NJ, 1–37.
Schunk, D., B. Zimmerman. 1997. Social origins of self-regulatory Zimmerman, B. J., A. Bandura, M. Martinez-Pons. 1992. Self-
competence. Educational Psychologist 32(4) 195–208. motivation for academic attainment: The role of self-efficacy
Schunk, D., B. Zimmerman. 1998. Self-Regulation of Learning and Per- beliefs and personal goal setting. Amer. Educational Res. J. 29(3)
formance: Issues and Educational Applications. Lawrence Erlbaum 663–676.
Associates, Hillsdale, NJ. Zweig, D., J. Webster. 2004. Validation of a multidimensional
Skill Soft. 2007. About us. http://www.skillsoft.com/about/default. measure of goal orientation. Canadian J. Behavioral Sci. 36(3)
asp. 232–243.