You are on page 1of 17

ARTICLE

DECEMBER
JOURNAL OF2003
MARKETING EDUCATION

Virtual Communities and the Assessment of


Online Marketing Education

James W. Peltier, William Drago, and John A. Schibrowsky

Online education has created a “virtual community” learn- tionships: customer to provider and customer to customer
ing environment. Effective assessment of this new learning (Hagel and Armstrong 1997; McWilliam 2000). Termed vir-
environment is paramount to providing quality education and tual communities, the core benefit of these interactive infra-
may provide insights to effective management of virtual com- structures is that they provide a computer-mediated meeting
munities in the business world. A model of online education place where customers can pursue common goals and inter-
effectiveness is proposed and then empirically investigated. ests (Bressler and Grantham 2000; Rheingold 2000). Strate-
Dimensions included in this model are student-to-student gically, the multidimensional nature of virtual communities
interactions, student-to-instructor interactions, instructor requires Web site managers to consider a host of potential
support and mentoring, information delivery technology, success criteria, including the number and quality of
course content, and course structure. Measures of these customer-customer interactions, site structure and content,
dimensions were then analyzed and found to be significant customer support services, information delivery technology,
predictors of the variance in students’ evaluations of the and customer-provider communications (Bickart and
global effectiveness of the online educational experience. Schindler 2001; Hagel 1999; Mathwick 2002; Rothaermel
and Sugiyama 2001).
The e-commerce industry is not alone in its quest to under-
Keywords: virtual; communities; assessment; online; mar- stand how virtual communities can be used to enhance inter-
keting education active relationships and share of customers. Globally, insti-
tutions of higher learning are beginning to adopt teaching
technologies and infrastructures that use the unique and col-
The paradigm shift in the management of customer rela- laborative capabilities of the Internet (Lang and Zhao 2000).
It is estimated that nearly half of all U.S. colleges and univer-
tionships spawned by the advent of the Internet has forever
changed how buyers and sellers interact in the global market- sities provide Internet-based educational offerings, serving
place (Peltier et al. 2002). Fundamentally, the emerging nearly 2 million students in the United States in 2002, with
media technologies used in e-commerce, in terms of both growth estimates as high as 5 million by 2006 (Wechsler
information and resource exchange, have also altered the way 2002). The explosive growth in Internet-delivered education
marketers develop relationships and how consumers partici- can be attributed to a variety of factors, including the
pate in them (Baker, Buttery, and Richter-Buttery 1998; Hol- increased number of working/nontraditional students seek-
land and Menzel-Baker 2001). Of particular interest is the ing degrees (Allen 1997), time management flexibilities
ability of interactive media to transform the communication associated with participating in asynchronous learning envi-
process from a solely “marketer-dominated” to an “on- ronments (Shea, Motiwalla, and Lewis 2001; Sullivan 2001),
demand” channel of information exchange in which custom- the logistical advantages gained through the elimination of
ers are allotted some degree of control (Klein 2003). travel to and from education sites (Aron 1999; Carpenter
Although e-commerce is still in its infancy, marketers are 1998; Donoho 1998), and the important fact that learning via
beginning to recognize that the Internet is not merely a new
James W. Peltier is Arno Kleimenhagen Professor in the Department of Mar-
channel of distribution or another mode of communication.
keting at the University of Wisconsin–Whitewater. William Drago is a pro-
Instead, it represents a transition to a new business model fessor of management in the Department of Management at the University of
requiring a different set of relationship-oriented activities Wisconsin–Whitewater. John A. Schibrowsky is a professor of marketing in
(Deighton 1997; Wind and Rangaswamy 2001). In response, the College of Business at the University of Nevada at Las Vegas.
many e-marketers are creating community infrastructures for Journal of Marketing Education, Vol. 25 No. 3, December 2003 260-276
the purpose of strengthening two types of interactive rela- DOI: 10.1177/0273475303257762
© 2003 Sage Publications

260
JOURNAL OF MARKETING EDUCATION 261

virtual communities results in employees and/or potential keting Education published two special issues on distance
new hires who have the ability to interact effectively via tech- education–related topics that underscored the need to better
nology (Benbunan-Fich et al. 2001; Phillips 1998). understand how information technology and marketing edu-
Although the use of the Internet in higher education and cation can come together to create quality learning experi-
the formation of virtual learning communities are proliferat- ences for students (Karns and Pharr 2001b; McAlister
ing, critics contend that distance education adversely affects 2001b). During the past 3 years, no less than 10 articles have
the nature of the student-to-instructor and student-to-student appeared in the Journal of Marketing Education on the topic
interactions, thereby undermining the quality of the educa- of online education.
tional experience (Rahm and Reed 1997; Sonner 1999). In a recent Editors’ Corner in the Journal of Marketing
Opponents of online education are bolstered by the fact that Education, Karns and Pharr (2001a) noted, “The use of tech-
Web-based course instruction often has a higher drop-out rate nology is a burgeoning trend in all higher education, and elec-
than that found for traditional in-class instruction (Aron tronic commerce is an important sector of the economy. It is
1999; Roblyer 1999). Although not established empirically, vitally important for marketing educators to explore this
the higher drop-out rate could be due to a variety of reasons, topic” (p. 3). They went on to state that integrating technol-
including a preference for face-to-face communications over ogy and distance learning in marketing education “is an
virtual interactions and that Web-based environments may important aspect of our future in marketing education.” Guest
stifle some students who excel in traditional teaching envi- editor for the Journal of Marketing Education, Debbie
ronments (McGrath 1997-98), frustrations students often McAlister (2001) concurred that “although innovation is
have with learning and applying new technologies taught in marketing courses around the world, less attention
(McCorkle, Alexander, and Reardon 2001; Siegel 1996), or has focused on the role and prospects of for innovation in
the inability of some faculty members to transition to an marketing education.” She further noted, “Educational tech-
online and interactive learning environment (Greco 1999; nology tools are increasingly common, although relatively
Lee 2002). little research has examined their impact and effectiveness”
As with the paradigm shift in how buyers and sellers inter- (p. 167).
act via e-commerce sites (Mathwick 2002), interactive edu- Without question, the marketing education community is
cators will need to find ways to enhance virtual relationships well suited to be a leader in online education. Not only have
with and between their students (Leonard 1999; Looi and marketing departments in U.S. universities and colleges been
Ang 2000). Undeniably, the use of the Internet as an instruc- at the forefront in developing curricula on Internet marketing,
tional medium has a significant effect on the role that educa- e-marketing courses, interactive marketing, and the like, mar-
tors play in the learning experience (Atwong and Hugstad keting academics have been leaders in conducting research
1997). At issue is whether online learning and e-commerce on how the Web-based interactions create mutually beneficial
technologies will merge in a way that is conducive for im- buyer-seller relationships.
proving the quality of distance learning programs and the In this article, we employ an interactive marketing
experiences students have with others in their virtual learning approach to the development and improvement of online edu-
community (Abernathy 1999; Karns and Pharr 2001a; Lang cation. Viewing online education from an interactive market-
and Zhao 2000; Rheingold 2000; Young 2001). ing perspective results in a unique approach to the study of
Unfortunately, little is understood on how to best plan, online education. It focuses on online education as a service,
implement, and evaluate interactive Web-based courses based on the use of virtual communities to enhance the ser-
(Kaynama and Keesling 2000; Sweeney and Ingram 2001). vice experience. It also brings attention to the issues of online
Particularly lacking is research seeking to understand the dif- distribution, personalization and customization, one-to-one
ferential needs and learning styles of online students in gen- marketing, and interactive marketing communications
eral (Drago, Peltier, and Sorensen 2002; Maharg 2001) and (Benbunan-Fich et al. 2001). In doing so, we take a marketing
marketing students in particular (Celsi and Wolfinbarger orientation in examining the issues surrounding the develop-
2002; Kaynama and Keesling 2000; Ponzurick, France, and ment of online education by focusing on the needs and per-
Logar 2000; L. J. Smith 2001; Sweeney and Ingram 2001; ceptions of the customers (i.e., students).
Ueltschy 2001; Young 2001). Because of the importance of In the research reported here, we provide insight into the
treating students as “customers” (Haynes and Hunts 2000), types of interactive relationships online education consumers
conceptual and empirical research is needed to find ways to are seeking in their pursuit of knowledge. Although our
enhance the online educational experience of this important research focus in this study is on improving the quality of
target audience (Eastman and Swift 2001; Lee 2002). online MBA programs, the ideas we present are transferable
With an increasing number of marketing departments to a broader educational experience and also have relevance
offering distance education courses and programs, online to understanding other e-marketing relationships. Special
education has become a topic of considerable importance and attention is given to understanding the role that virtual com-
interest to marketing educators. In 2001, the Journal of Mar- munities play in enhancing student-to-instructor and student-
262 DECEMBER 2003

to-student relationships, and to the types of information con- Undeniably, technology is transforming how businesses
tent, course structure, and delivery systems that consumers operate, how marketing educators teach, and how students
desire in the relationship. Because the theory of virtual com- learn (McCorkle, Alexander, and Reardon 2001). However,
munities is underdeveloped (Mathwick 2002), especially in many questions remain regarding how technology can be best
the context of interactive and online education (Maharg integrated into today’s rapidly changing educational environ-
2001), research assessing the quality of cyber-technology ment. The traditional approach of the professor at the black-
and distance education is warranted (Long et al. 2000). board and students in their seats is no longer appropriate for
In accomplishing these objectives, we briefly introduce all learning environments (D. H. Smith 1996). Ultimately, the
literature from three areas to help provide a foundation for advancing technologies associated with the online delivery of
our model: distance education and technology, virtual com- graduate business education will require educators to move
munities, and assessing the educational experience. We then away from an instructor-centered model to one of student
introduce a model of the various dimensions of effectiveness empowerment (Gibson 1996; Markel 1999).
of the online education experience. Next, we present the find- Unfortunately, the technological capabilities necessary to
ings of an empirical investigation of these measures and their transition from traditional to online education is often too
ability to account for variance in students’ perceptions of the much for many marketing instructors to handle (McCorkle,
effectiveness of their online educational experiences. Finally, Alexander, and Reardon 2001). As such, instructors’ experi-
we discuss the results of the study, provide insights to the ences in traditional classroom educational settings may not
effective management of virtual communities in educational be well suited for meeting the needs of students learning at a
and other business settings as they apply to teaching online distance (Kelley 1987). Like the paradigm shift in how buyers
graduate marketing courses, and suggest directions for future and sellers interact via e-commerce, interactive educators
research. will need to find ways to best maximize the student-educator
relationship. In particular, the highly technical and interactive
nature of the online learning environment may require many
THEORY DEVELOPMENT
marketing instructors to adapt their teaching styles to match
Distance Education and Technology the varied needs of their online learners (Eastman and Swift
2001; Lee 2002).
Although graduate business education delivered online is
in the early stages of its life cycle, distance education has been Virtual Communities
in existence for many years. Distance education has been
Clearly, online education has transformed teaching and
defined in many ways, two of which are relevant for the cur-
learning not only in marketing education but across all busi-
rent study. Carter (2001) defined distance education as a
ness disciplines. The teacher is no longer the center of all
“structured learning environment in which the teacher and
knowledge, nor is the student a passive recipient of infor-
students are separated by time and geographical place with
mation. This shift creates new challenges and opportunities
some form of technology being used for the transmission of
for those educators involved with online education. Instead,
all teaching and learning” (p. 250). Moore (1990) described
an interactive process has evolved in online education
distance learning as all of the arrangements for providing
whereby participants are assigned and often share new roles
instruction and transmitting educational materials through
to the mutual benefit of all parties in the virtual community
print or electronic media to geographically dispersed students
(Pye 1999). Hagel and Armstrong (1997) emphasized the
in a place or time different from that of the instructor.
relationship-building aspects of virtual communities through
Both of these definitions highlight the importance of tech-
their view that this type of relational infrastructure provides
nology in the delivery of the learning experience and the
an opportunity to bring together people with similar interests,
transfer of knowledge within marketing curricula (Karns and
unrestrained by time and space. Through such community
Pharr 2001a). Distance education has expanded in recent
relationships, information content and intracommunity com-
years because of Internet technologies that have allowed for
munications are exchanged and members learn from each
the delivery of multiple types of information content and
other.
communication that can be accessed from different locations
Virtual communities have been conceptualized in a num-
(O’Leary 2002). Emerging interactive technologies offer
ber of different ways. Early work by Karp, Stone, and Yoels
considerable promise for bridging the gap between knowl-
(1977) focused on three elements that define communities:
edge acquisition, skill development, and real-world applica-
sustained social interaction, shared attributes and values, and
tions (Fitzgerald 1995). Common among these e-commerce
a delineated geographic space. Building on this work,
technologies are discussion-based virtual communities; digi-
Lawrence (1995) set forth three organizational conditions
tal libraries; electronic publishing; e-mail; and structural plat-
that need to be in place to define a community: sustained
forms for organizing the integration of technologies, course
social interaction, community standards, and membership
content, and human capital (Lang and Zhao 1999).
rules. Brown and Duguid (2000) contended that virtual
JOURNAL OF MARKETING EDUCATION 263

commuities are social worlds in which individuals interact that encompasses universal concepts of a quality educational
through various Internet tools, exhibits, and information doc- experience.
uments. Moreover, virtual communities are maintained Although there is a strong consensus that teaching and
through a continual stream of discourse, communicated via learning are multidimensional constructs, there is little agree-
computer-mediated technologies, which are introduced, ment on the nature and number of dimensions that accurately
commented upon, and shaped by members of that community and completely measure the learning experience (Boex 2000;
(Murphy 1997). Because virtual space limits the ability for Patrick and Smart 1998; Shevlin et al. 2000). Virtually no
face-to-face encounters, the community is responsible for studies have addressed this issue for online and interactive
evaluating the quality and reliability of the information and education. There is growing agreement, however, that online
discussions that exist in that space (Fombrun 1996; student customers can provide feedback on what they are
Rheingold 2000). looking for in an interactive relationship (Houston and
Although research is sparse on the topic, there is a growing Bettencourt 1999; Sweeney and Ingram 2001) and that their
consensus that to be successful, virtual communities need to evaluations of the experience are reliable, reasonably valid,
address a number of important relational criteria, including and relatively uncontaminated by outside factors (for a
customer-to customer interactions, provider-to-customer review of teaching evaluations, see Marsh 1987; Wachtel
interactions, information content, up-to-date cyberspace 1998). This is important in that the need for accountability is
technology, and community infrastructures, all of which must becoming a major issue in higher education (Ballantyne,
come together under the guidance of an effective manager of Borthwick, and Packer 2000).
the virtual space (Bickart and Schindler 2001; Hagel 1999; While the teaching effectiveness literature is far too exten-
Hagel and Armstrong 1997; Mathwick 2002; Rothaermel sive to review here, a short list of relevant findings suggests
and Sugiyama 2001). that teachers and learning environments are most effective
The learning environment experienced by students when various criteria are met: clear instructions are given
through online education meets the definitions of a virtual (Fraser 1986); interpersonal rapport in the learning environ-
community. Sustained interaction between students and ment exists (Lowman and Mathie 1993); instructor feedback
between students and the instructor occurs throughout the is provided (Marsh 1987); the learning environment encour-
time period of the course; standards or policies as well as ages choice and independent thought (Feldman 1976); qual-
membership rules are developed explicitly by the adminis- ity course content and learning experiences are stressed
tration of the educational program and the instructor for (Entwistle and Tait 1990); a challenging and motivating aca-
the course and implicitly by students through their previous demic environment is provided (Ramsden 1991); clear and
experiences in the online program. Furthermore, all com- useful lectures are available (Boex 2000); there is respect for
munication is heavily dependent on computer-mediated tech- learners (Patrick and Smart 1998); instructor-student inter-
nologies. The instructor or facilitator can thus be seen as a actions are emphasized (Brightman, Elliot, and Bhada 1993);
manager of this educational virtual community, and an eval- the instructor shows strong organizational skills (Nelson and
uation of the overall effectiveness of the online course Lynch 1984); and there is effective use of teaching technolo-
can also be seen as an evaluation of the effectiveness of the gies such as multimedia lectures, presentation software, and
virtual community experience as perceived by the student. interactive learning tools (Ballantyne, Borthwick, and Packer
Using an entertainment analogy, the educator in an online 2000).
environment is less of an actor and more of a director or pro-
ducer of the experience. Employing a virtual learning com- MODEL DEVELOPMENT
munity perspective to online education helps to frame the
ways in which this new technology can enhance marketing The preceding discussion melded the concepts of dis-
education. tance education, information technology, and virtual com-
munities. Although online education can be practiced in a
Assessing the Effectiveness variety of ways, we present the following definition of online
of the Educational Experience education:
The meteoric rise in online education and virtual commu-
nities has not been accompanied by research that explores the Online education uses Internet-based technology to deliver
multiple types of information content and communication,
types of interactive experiences students are looking for or
accessible from learner-selected locations. Minimally, the
that investigates the best ways to evaluate the quality of these
cyber classroom must contain some type of structural plat-
relationships (Lee 2002; Drago, Peltier, and Sorensen 2002; form (e.g., Lotus Notes, Blackboard, etc.) for organizing the
Looi and Ang 2000); research has been particularly lacking in integration of technologies and course content. In more
marketing education (Celsi and Wolfinbarger 2002; sophisticated forms, the online education environment pro-
Kaynama and Keesling 2000; Mathwick 2002). Prior to pre- vides a forum for engaging in discussion-based virtual com-
senting our online education model, we examine literature munities and other forms of electronic communications
264 DECEMBER 2003

The model is presented in Figure1. The model posits that


each of these dimensions is positively related to perceptions
Instructor of the overall quality of the online learning experience for
Support & graduate business education. Below is a discussion of each of
Mentoring the six dimensions.
Student-to- Instructor- Student-to-Student Interactions
Student to-Student
Interaction Interaction In an online educational setting, the learning process is not
Overall the top-down knowledge delivery method associated with
Perceived traditional seat time classes. Instead, marketing educators in
Effectiveness an online environment are more likely to rely on peer-to-peer
Information
learning in the form of group discussions and team projects
Course (Eastman and Swift 2001). In this collaborative learning envi-
Delivery
Structure
Technology ronment, the discussion forum is the primary tool for inter-
activity in the online classroom. Although some marketing
Course
educators contend that there is no replacement to face-to-face
Content discussions offered in traditional classes, a growing research
stream suggests that discussion forums may be used to capi-
talize on advantages not apparent in face-to-face tutorials
(Sweeney and Ingram 2001). For example, students may feel
FIGURE 1: Proposed Online Education Model
freer and more open in online discussions, are more likely to
speak out, and develop a feeling of trust for other members in
the virtual education community (Figallo 1998; Hiltz 1986;
Sullivan 2001). Moreover, because other students have
between class members and with the instructor, and pro- access to their responses, online students might be motivated
vides access to digital libraries and electronic publishing to work harder to produce high-quality work (Harasim 1990).
technologies. In the virtual space, community members must play a key
role in assessing the quality and reliability of the communica-
A review of the distance education, virtual communities, tions and content they encounter (Fombrun 1996). In this
and teaching effectiveness literature highlighted at least six way, member participation in the discussion forum is a criti-
possible dimensions of teaching effectiveness as they apply cal ingredient in weaving the fabric of the virtual community
to an online educational setting: (1) student-to-student inter- (Rheingold 2000). A failure to achieve high-quality student-
actions, (2) student-to-instructor interactions, (3) instructor to-student interactions in the electronic discussion forum will
support and mentoring, (4) information delivery technology, likely result in an inferior educational experience for mem-
(5) course content, and (6) course structure. In addition, early bers in the virtual learning community (Karuppan and
on in the development of the program, the college’s distance Karuppan 1999; Ueltschy 2001).
learning committee had clearly laid out requirements for Like those offered in a commercial environment, virtual
instructors wishing to participate in the online program, and communities in online education offer students the opportu-
all faculty members participated in a 3-day training seminar. nity to build close relationships with other students and that
Although these expectations were not research tested at that are not constrained by time and space (Hagel and Armstrong
point, the following specifics were mandated: (1) all courses 1997). Establishing and nurturing relationships with class-
needed to have substantial opportunities for students to inter- mates is important in an asynchronous environment
act with each other (student-student interactions), (2) course (Benbunan-Fich et al. 2001), especially for collaborative dis-
content needed to be approved, (3) the role of the instructor as cussions and team projects (Eastman and Swift 2001). Since
the educator remained critical and training in this area was team projects and learning groups are often used as a way to
provided, (4) clear expectations and learning objectives were build relationships and complete assignments, students’ per-
required for each module (course structure), (5) CD ROM ceptions of the appropriateness of the size and constitution of
technology was mandated (information delivery technol- their individual learning groups is crucial. It has been argued
ogy), and (6) rapid responses to student requests were viewed that the optimal size of these learning groups, and the result-
as a key driver to course satisfaction. Consequently, the ing impact on the quality of the interactions, is curvilinear
model was based on a combination of the literature review (Rothaermel and Sugiyama 2001). In an online education
and the requirements imposed on the faculty members who environment where group projects and discussion forums are
participated in the program. We then tested the model used, a curvilinear relationship would suggest that too many
accordingly. participants would limit the close-knit relationship of the
JOURNAL OF MARKETING EDUCATION 265

group (Brown and Duguid 2000), and too few would limit the over, given the importance of maintaining a strong virtual
number of valuable insights that would be generated community, the instructor must get involved with motivating
(Rothaermel and Sugiyama 2001). The size of the groups is students to participate in interactive discussions with other
especially important when a considerable amount of e-mail- students (Aron 1999; Sullivan 2001). As a consequence, the
based interactions are required between group mates. instructor is seen as having a supportive, almost pastoral role
in the learning process (Tricker et al. 2001).
Instructor-to-Student Interactions
Consistent with commercial-based online virtual com-
As the designer and manager of the virtual community, the munities, the need for guidance, warmth, rapport, and moti-
instructor has two types of communication roles, those activi- vational skills are a high priority for online educators. This
ties associated with mentoring, supporting, and motivating guidance would logically include such activities as interme-
the community (see next section) and those associated with diation, developing the rules of the game (e.g., when activities
communications with individual students (Benbunan-Fich are due, how they are delivered and recorded, etc.), and facili-
et al. 2001). Both of these communication roles highlight the ties management (Hagel and Armstrong 1997). The manage-
importance of interactive teaching styles (Roach, Johnston, ment style of the instructor could have a substantial impact on
and Hair 1993). In contrast to the numerous student-to- the quality of online education. Too little management of the
student interactions found in the virtual discussion forum, virtual learning community by the instructor could lead to
the majority of individual student-to-instructor communica- chaos, while too much site management might negatively
tions will be in the form of off-site communications, most affect the intangible level of trust, relational bonding, and
commonly via e-mail, and to a lesser extent, telephone. knowledge exchange associated with virtual communities
Rothaermel and Sugiyama (2001) speculated that off-site (Brown and Eisenhardt 1998).
communication may be a stronger factor in explaining a In addition to the expected support function of online
member’s connectedness to the virtual community than are instructors, interactive educators will also have to instill con-
on-site interactions. fidence in their students that they are competent and capable
Critics of cyberlearning warn that this mode of communi- tutors (Sweeney and Ingram 2001). Although as yet untested
cation may eliminate crucial personal interactions between in an online educational setting, the general teaching effec-
instructors and students (Navarro and Shoemaker 1999). tiveness literature has shown that perceptions of an instruc-
Because the asynchronous nature of online education places tor’s charisma (Shevlin et al. 2000), intellectual capabilities
logistical demands on instructors (Tricker et al. 2001), many (Boex 2000; Brightman, Elliot, and Bhada 1993), ability to
faculty members have a difficult time adjusting to the fact that explain (Marsh 1987), stimulation of student interest
students want them to be “on-call” all the time (Greco 1999). (Feldman 1976), and assessment skills (Fraser 1986) are all
Although as yet relatively unexplored in a virtual education associated with positive course evaluations.
setting, online learners desire many things regarding their In summary, each of the three dimensions discussed above
interpersonal interactions with their faculty, including fre- was based on the interactive nature of the online learning
quent communications with easy-to-access instructors (Baily experience. Student-to-student interactions represented
and Coltar 1994), quick and timely feedback (Bishop 2000), communications between students via the discussion forum,
detailed technical support (Shea, Motiwalla, and Lewis e-mail, and telephone; student-to-instructor interactions rep-
2001), and meaningful feedback with respect (Abernathy resented personalized and tailored communications; and
1999; Patrick and Smart 1998). Meeting these student-to- instructor support and mentoring addresses interactions
instructor needs will likely lead to higher satisfaction with the related to managing the virtual community. In the next three
learning experience (Eastman and Swift 2001). sections, we explore online learning in terms of course con-
tent, structure of the virtual community, and information
Instructor Support and Mentoring delivery technology.
The “instructor in front of the class” and “students in their Course Content
seats” model of teaching is no longer appropriate for a high-
tech learning environment (D. H. Smith 1996). Instead, Web- As with commerce-based virtual communities (Hagel
based learning environments demand a transition from the 1999; Hagel and Armstrong 1997), it is unlikely that the qual-
instructor’s role as the authority figure to that of one voice ity of the learning experience in an online educational setting
among many (McGrath 1997-1998). Unfortunately, many can rely solely on member-contributed communications. The
faculty members have resisted this transference of power perceived value of instructor-delivered content (e.g., Atwong
from the teacher to students (Kitchen and McDougall 1998- and Hugstad 1997; Benbunan-Fich et al. 2001), Internet-
1999). This may account for some of the student dissatisfac- driven content (e.g., Lang and Zhao 2000), and assigned
tion with online education. It is in this role as online mentors learning and assessment activities (L. J. Smith 2001) is likely
and moderators that good “teachers” stimulate, guide, and to affect students’ perceptions of the learning experience. In
challenge their students (Sherron and Boettcher 1997). More- that students have some control over the pace of their learn-
266 DECEMBER 2003

ing, with the exception of unfolding events and fluid informa- eral consensus that Web sites have little value unless they are
tion needs, it is critical that the planning and placement of structured in a logical and easy-to-follow manner (Chen and
most of the course content be completed ahead of time Wells 1999; Nielsen 2000; Stevenson, Brunner, and Kumar
(Abernathy 1999). 2000). The existence of an easy-to-use and clearly laid-out
There is some support in the e-marketing world that virtual course structure in an online learning environment in terms of
members who feel that they are receiving valuable informa- Web formats and computer-based infrastructures is espe-
tion content by participating in the community are more cially important (Eastman and Swift 2001; Karuppan and
inclined to engage in commercial transactions (Chen and Karuppan 1999). Web-based course infrastructures that are
Wells 1999; Fombrun 1996) and use that content to drive rela- difficult to understand and use create the need for ongoing
tionships in that community (Kozinets 1999). Extrapolating technology guidance and negatively affect the clarity of the
these arguments to an online educational community would learning environment (Palloff and Pratt 1999). Consequently,
suggest that course content could increase student participa- the need for a user-friendly structure in educational offerings
tion in the community, spark the introduction of new ideas for cannot be ignored (Ballantyne, Borthwick, and Packer 2000;
further discussion, and strengthen community relationships. Tricker et al. 2001).
Consistent with the belief that learning is a multidimen-
Information Delivery Technology
sional construct, perceptions of the effectiveness of what con-
stitutes a high-quality content-based relationship is multi- Distance learning technologies are transforming the way
faceted as well (Drago, Peltier, and Sorensen 2002). In in which educators convey information to students
particular, perceptions of the quality of the design, delivery, (McCorkle, Alexander, and Reardon 2001). Online discus-
and assessment of course content in both a traditional and sion forums and other electronic forms of communication
online educational setting have been investigated in terms of and information access are common components used in
topical relevance (Ballantyne, Borthwick, and Packer 2000; Web-based courses. The primary purpose of the material
L. J. Smith 2001); students’ perceptions of outcome-oriented delivery format is to “simulate” the classroom experience as
activities, including tests, exercises, exams, projects, and close as possible in terms of what instructors provide stu-
similar assessment techniques (Lang and Zhao 2000; dents. Of importance, lecture and learning materials con-
Ramsden 1991; L. J. Smith 2001); depth and clarity of lecture veyed to students using this technology should have real sub-
content (Atwong and Hugstad 1997; Eastman and Swift stance and not simply present lecture outlines (Eastman and
2001; Entwistle and Tait 1990); up-to-date information Swift 2001; Karuppan and Karuppan 1999). As a conse-
(Tricker et al. 2001); and the speed and quality of instructor quence, the ability to convey clear and valuable lectures, in a
feedback (Drago, Peltier, and Sorensen 2002; Fraser 1986; format that interests students, is likely to lead to a more satis-
Marsh 1987). fying learning experience for students (Smart, Kelley, and
Conant 1999).
Course Structure
In summary, a review of the literature identified a total of
One of the least studied aspects of online education and six dimensions of the online educational experience. The first
virtual communities is the structural element of the learning three dimensions involved the communication aspects of
experience. Although not well-defined in virtual communi- online learning, while the final three dimensions were con-
ties of either commercial or educational Internet sites, two cerned with the substantive aspects of online education. Next,
types of course structure are discussed here: course expecta- we developed the measures associated with these six dimen-
tions and course infrastructure. Although seemingly unre- sions and evaluated their ability to predict the overall effec-
lated, both of these online structural elements address the tiveness of the online educational experience.
ease with which students understand what is expected of them
and how these expectations are communicated via the elec-
METHOD
tronic course room.
Teaching effectiveness literature for traditional classes has Research Setting
shown that students need and want clear structural guidance
in terms of expectations related to topical coverage, required The study was conducted at a large midwestern university.
workload, and evaluation criteria (Boex 2000; Brightman, U.S. News and World Report recently ranked the university’s
Elliot, and Bhada 1993; Entwistle and Tait 1990; Ramsden online MBA program in the Top 25 programs in the United
1991). For an online learning environment, it is especially States. In delivering online education to students, all classes
important to communicate these expectations in advance via contained five common elements: (1) use of the Internet to
course syllabi (Karuppan and Karuppan 1999). transfer information and to provide links to cyber materials;
Virtual communities can be evaluated in terms of how they (2) Lotus Notes structural platform for conveying course
are configured in various computer-mediated communica- expectations, daily instructions, e-mail, syllabi, course
tion structures. In the e-commerce community, there is a gen- schedule, testing, online grading, and so forth; (3) use of CD-
JOURNAL OF MARKETING EDUCATION 267

ROMS to present audiovisual lecture materials; (4) an inter- ment of the enjoyment of the experience was needed. Consis-
active discussion room; and (5) students and faculty never tent with the principles outlined by Churchill (1979), the
formally meet face-to-face. Additional materials, techniques, three dependent measures loaded on one dimension in a pre-
and procedures are course and instructor specific. liminary factor analysis, suggesting that they are measuring
The university was interested in identifying the key some overall dimension of effectiveness of the online course.
dimensions associated with enhancing the online MBA expe- A summed measured was then created.
rience. As part of this process, the business college wanted to
Data Collection
survey current online MBA students regarding their educa-
tional experiences. This particular setting was ideal for study- The questionnaire was distributed to all students enrolled
ing virtual communities. Since much of the graduate educa- in the college’s online MBA courses taught during the spring
tion experience deals with interacting with teachers and other and summer semester. For a course to be considered for the
students to build communities of learning, the online educa- study, it had to be primarily Web based and taught exclusively
tion experience needed to make sure that virtual communities at distance. Courses where students were required to come to
of learning were present. It is important to note that online class on campus at anytime were excluded. In the case of the
educational formats are not identical across all universities courses considered here, it should be noted that CR-ROMs
that use the Internet to deliver all or portions of a particular were used to deliver the lecture material, but all courses
class. As such, universities and, specifically, marketing included substantial Web-based content associated with
departments using different methods of delivery and commu- them. For example, they employed digital libraries, sample
nication tactics would not necessarily evaluate the success of advertisements, pertinent Web sites, online video, along with
their programs in exactly the same way. With this said, the six virtually all communication via the Web.
dimensions that we addressed in the model would likely still A total of 18 online courses were taught in this time period
apply to some degree in most online educational courses and to 548 students. The management, marketing, finance/busi-
programs. ness law, economics, and office systems departments offered
courses. Course size ranged from a low of 15 students to a
Questionnaire Development
high of 59 students. Average class size was 30.4 students.
Based on a review of the literature on distance education, Surveys were made available to students during the last
virtual communities, and teaching effectiveness, a survey was two weeks of each class through “LearningSpace,” a course
developed by the College’s Graduate Studies Committee. In management software product used by all the online courses
the tradition of the Domain Sampling Model (Nunnally in the MBA program. E-mail reminders were sent to students
1978), a pool of questions designed to cover the various asking them to fill out these questionnaires. Students were
aspects of each of these literature bases was developed. Some assured anonymity and that all surveys would be electron-
questions were generated from the Purdue Rating Scale ically delivered directly to the graduate office. A total of 299
(Remmers 1960) and from surveys used in other online pro- questionnaires were returned, for a response rate of 54.6%.
grams, while others were designed by the committee to tap
areas not addressed by secondary sources. The questions
ANALYSIS OF THE MEASURES
were designed to be answered by using a 5-point Likert-type
scale, with 1 indicating strongly agree and 5 indicating The 47 questions pertaining to the various aspects of
strongly disagree. the courses were evaluated according to the measurement
The questionnaire was pretested to identify any confusing principles outlined by Churchill (1979) and Peter (1979).
or misleading questions. Minor changes were made, and the First, the items were subjected to a principal components
instrument was completed. The final questionnaire contained analysis employing varimax rotation to identify the major
47 questions pertaining to specific aspects of the courses. commonalities in the data associated with the design and
Three questions were designed to tap the students’ evalua- execution of online MBA education. The resulting factor
tion of the overall effectiveness of the course: (1) I would rec- analysis is displayed in Table 1. Twelve of the questions with
ommend this course to friends/colleagues, (2) I have learned low and/or multiple factor loadings were dropped from the
a lot in this course, and (3) I have enjoyed taking this course. assessment.
These dependent measures are useful for a number of rea- An analysis of the eigenvalues was used to determine the
sons. First, referral likelihood is an important measure in the appropriate number of factors to use in the study. A total of six
services marketing literature for measuring relationship factors were identified with eigenvalues greater than 1.0. In
strength. Second, previous work in teaching effectiveness lit- addition, there was a significant break in the scree plot after
erature relies on measures of “amount learned.” Third, six factors. The six factors accounted for a total of 70% of the
because initial criticisms pertaining to online education sug- variance.
gested that students would not “enjoy the experience” of tak- Next, the factors were labeled based on the questions that
ing courses outside of the traditional classroom, some assess- loaded on each dimension. The resulting factor names were
268 DECEMBER 2003

TABLE 1
PRINCIPAL COMPONENTS FACTOR ANALYSIS

Instructor Student-
Support to- Instructor-
and Course Course Student Information Student
Question Mentoring Content Structure Interaction Technology Interaction

My instructor played an important role in facilitating learning. .775


My instructor helped me apply theory/applications to solve problems. .756
My instructor explained difficult material clearly. .716
My instructor readily maintained rapport with this class. .705
My instructor contributed to the discussions in this course. .675
My instructor was actively helpful when students had problems. .674
The instructor provided feedback on assignments and/or exams. .626
My instructor motivated me to do my best work. .619
My instructor identified major or important points in the course. .580
I have interacted with my professor in this course. .543
My instructor emphasized relationships between and among topics. .514
In this course, many methods were used to involve me in learning. .484
This course supplied me with an effective range of challenges. .748
Course materials stimulated my desire to learn. .726
The course emphasized understanding of concepts and principles. .709
This course effectively challenged me to think. .709
Course assignments were interesting and stimulating. .690
This course was up-to-date with developments in the field. .633
Student evaluation techniques such as projects, assignments, and
exams were related to the learning objectives of this course. .558
There were a sufficient number of applied-learning experiences. .477
The structure of the modules was well prepared and organized. .716
Projects/assignments were clearly explained. .673
Existing classroom technology was used effectively in this course. .609
I understood what was expected of me in this course. .566
The group work contributed significantly to this course. .825
Group size was appropriate for course purposes. .743
Student interaction was an important learning component. .675
This course provided an opportunity to learn from other students. .660
I had sufficient opportunity to interact with other students. .578
The content of the CD-ROMs contributed toward learning. .861
CD-ROMs contained information not covered in the text. .781
Information in the CD-ROMs was effectively communicated. .693
I felt free to ask questions throughout this course. .673
The instructor responded to my questions in a timely manner. .659
The instructor was easily accessible to me. .616
I felt free to express and explain my own views. .602

Percentage of variance 19.770 15.273 13.235 8.681 7.075 6.160


Cumulative percentage of the variance 35.043 48.278 56.959 64.034 70.194

consistent with a priori expectations and included instructor in Table 2. The following discussion describes each of these
support and mentoring, course content, course structure, constructs.
student-to-student interaction, information delivery technol-
ogy, and student-instructor interactions. Instructor Support and Mentoring. The 12 items included in this
The individual items associated with each measure were measure addressed the performance of the instructor in the
subjected to an item-to-total correlation analysis. Items with course. They included questions pertaining to teaching meth-
low item-to-total correlations were eliminated. The coeffi- ods, identifying key points, explaining concepts, facilitating
cient alphas for final measures were computed to estimate the learning, and motivating students.
Course Content. This measure dealt with the subject matter of
reliabilities. They ranged from .835 to .951, indicating that
the course. Eight individual items dealt with subject cover-
the measures were highly reliable. The specific items making
up each of the measures and the coefficient alphas are shown
JOURNAL OF MARKETING EDUCATION 269

TABLE 2
ONLINE EDUCATION MEASURES

Instructor Support and Mentoring Coefficient alpha = .9513


1. My instructor played an important role in facilitating learning in this course.
2. My instructor helped me apply theory/applications to solve problems.
3. My instructor explained difficult material clearly.
4. My instructor readily maintained rapport with this class.
5. My instructor contributed to the discussions in this course.
6. My instructor was actively helpful when students had problems.
7. The instructor provided sufficient feedback on assignments and/or exams.
8. My instructor motivated me to do my best work.
9. My instructor identified major or important points in the course.
10. I have interacted with my professor in this course.
11. My instructor emphasized relationships between and among topics.
12. In this course, many methods were used to involve me in learning.

Course Content Coefficient alpha = .9250


1. This course supplied me with an effective range of challenges.
2. Course materials stimulated my desire to learn.
3. The course emphasized understanding of concepts and principles.
4. This course effectively challenged me to think.
5. Course assignments were interesting and stimulating.
6. This course was up-to-date with developments in the field.
7. Student evaluation techniques such as projects, assignments, and exams were related to
the learning objectives of this course.
8. This course included applied learning and problem-solving experiences.

Course Structure Coefficient alpha = .8358


1. The structure of the modules in the Lotus Notes environment was well prepared and organized.
2. Projects/assignments were clearly explained.
3. Existing classroom technology was used effectively in this course.
4. I understood what was expected of me in this course.

Student-to-Student Interaction Coefficient alpha = .8368


1. The group work contributed significantly to this course.
2. Group size was appropriate for course purposes.
3. Student interaction was an important learning component of this course.
4. This course provided an opportunity to learn from other students.
5. I had sufficient opportunity to interact with other students in this course.

Information Delivery Technology Coefficient alpha = .8676


1. Information in the CD-ROMs for this course was effectively communicated.
2. CD-ROMs for this course contained information not covered in the text.
3. The content of the CD-ROMs for this course contributed toward learning.

Instructor-Student Interaction Coefficient alpha = .8812


1. I felt free to ask questions throughout this course.
2. The instructor responded to my questions in a timely manner.
3. The instructor was easily accessible to me.
4. I felt free to express and explain my own views throughout this course.

Effectiveness of the Online Educational Experience Coefficient alpha = .9304


1. I would recommend this course to friends/colleagues.
2. I have learned a lot in this course.
3. I have enjoyed taking this course.

age, rigor, appropriateness of the student evaluation tech- Student-to-Student Interactions. This measure concerned the
niques, and applied learning experiences. degree to which the course encouraged student group work
Course Structure. This measure involved the structural aspects and interacting with other students. A total of five ques-
of the course. Four items covered the structure of the mod- tions covered items ranging from value of group work to the
ules, along with clearly defined expectations and learning degree to which the course encouraged learning from other
objectives. students.
270 DECEMBER 2003

Information Delivery Technology. This measure focused on the TABLE 3


content of the instructor-designed CDs that were an integral REGRESSION ANALYSIS
part of each course. Three questions pertaining to the value
added by the CDs were included. Standardized Standard
Student-Instructor Interactions. The four items included in this Variable Coefficient Error t-value Significance
measure dealt with communications between the student and Constant 6.582 .106 61.822 .000
instructor. They included questions about the degree to which Course Content 1.942 .107 18.205 .000
the student felt free to ask questions and express his or her Instructor Support and
views, along with questions about the accessibility and Mentoring 1.523 .107 14.281 .000
responsiveness of the instructor. Course Structure 0.888 .107 8.330 .000
Information Delivery
Finally, a measure of the students’ evaluation of the effec- Technology 0.731 .107 6.852 .000
Student-to-Student
tiveness of the online educational experience was developed Interactions 0.671 .107 6.292 .000
using three global effectiveness measures from the survey Instructor-Student
instrument. These three items were highly correlated, sug- Interaction 0.346 .107 3.246 .001
gesting that they were measuring the same thing. As such,
they were combined into a summative overall effectiveness R = .849
R 2 = .720
index. This index included the summed responses from ques- 2
Adjusted R = .714
tions pertaining to the amount of learning, enjoyment of tak-
NOTE: Dependent variable = effectiveness of the online educational
ing the course, and the likelihood of recommending it to experience.
friends and colleagues. The coefficient alpha for the three-
item measure was .930, indicating the measure is highly
reliable.
DISCUSSION
REGRESSION RESULTS This study reports a number of important findings for mar-
A regression analysis was conducted to determine the rela- keting educators interested in online education. Based on a
tive importance of the six measures discussed above to review of the literature on distance education, virtual com-
account for variance in students’ evaluation of the effective- munities, and teaching effectiveness, six dimensions of the
ness of the online educational experience. Due to the poten- online education experience were identified. The resulting
tially high multicollinearity between the measures, factor factor analysis confirmed that these six dimensions were, in
scores from the principal components analysis discussed fact, separate and important factors associated with virtual
above were used as the independent variables. The three-item online education communities as perceived by students.
measure of students’ evaluation of the effectiveness of the Three of the factors dealt with interpersonal/communication
online educational experience was used as the dependent issues, including student-to-student interactions, student-
variable. To determine the relative importance of the individ- instructor interactions, and instructor support and mentoring.
ual items, a regression was conducted. Table 3 displays the The three other factors dealt with course design and included
results of the regression analysis. course content, course structure, and information delivery
All six dimensions significantly affected the global effec- technology. Unquestionably, the results highlight the multidi-
tiveness measure and were included in the final model. The mensional nature of virtual learning communities.
order of the importance of the variables in explaining the vari- While all the factors provide insights into the design and
ance in students’ evaluation of the overall effectiveness of the execution of online programs, the instructor support and
online educational experience during the course was as mentoring dimension is by itself interesting. In particular, it
follows: underscores the importance of the instructor as an active par-
ticipant in online courses (and, perhaps the importance of
1. Course content managers as active participants in the virtual communities
2. Instructor support and mentoring they manage). Based on the questions contained in this
3. Course structure dimension, learning support by the instructor requires a mix
4. Information delivery technology of facilitative learning, providing direction and explanations,
5. Student-to-student interaction maintaining rapport with and between classmates, actively
6. Student-instructor interactions participating in discussions, motivating, and providing effec-
tive feedback.
The model accounted for 72% of the variance in the It should also be noted that many of the survey questions
dependent variable. used in this study were obtained from secondary sources,
such as the Purdue Rating Scale. While the findings suggest
JOURNAL OF MARKETING EDUCATION 271

that the questions were excellent predictors of the overall reliable but also comprehensive. It provides an excellent
effectiveness of online programs, more research is needed to starting point for future research in the area of online educa-
determine if these questions can be generalizable to other tion and other virtual communities. It should be noted that the
educational and virtual communities settings. The excep- effectiveness questions used to determine global effective-
tions are in the Information Delivery Technology and Course ness of the virtual learning experience refer directly to
Structure factors, where references are made to “modules in courses taught in an educational environment. This may
Lotus Notes” and to the specific use of CD-ROMs in the affect their relevance to assessment of virtual communities in
course delivery. Certainly, there is other courseware available the business world. Further research is necessary to deter-
and other forms of information delivery technology that can mine the comprehensiveness of these factors or dimensions
and are used to deliver courses and information to students. outside the educational domain.
As such, marketing educators interested in designing and Second, the regression analysis indicated that all six
assessing online educational offerings will want to customize dimensions significantly contributed to the variance
the measures for their specific situations. explained by the model. This provides additional evidence of
The factors or dimensions uncovered by this study act as a the validity of the measures and implies that each factor
guide for those responsible for the development and delivery affects the effectiveness of the online education experience in
of online courses and training programs. While these factors different ways. As such, each factor is important in its own
were identified for virtual “learning communities,” they are right and merits further investigation. For example, the
likely to be similar for other types of virtual communities. As impact of site structure and content on online learning and
such, marketing educators teaching Internet marketing, data- other virtual communities needs additional research. In addi-
base marketing, or other applications of virtual communities tion, the temporal and interactive combinations of these
will find this information valuable. These findings can be dimensions for virtual communities need to be investigated.
incorporated into class lectures and discussions pertaining to Lessons learned here can be used in other virtual community
the use of virtual communities as an Internet-based marketing research.
tool for developing dialogue with and between current and Finally, the regression analysis determined the relative
future customers, as learning support and value-adding vehi- importance of each variable in accounting for the variance in
cles, as tools to increase consumer-to-consumer interactions the effectiveness of the education experience. Recall that the
and manager-consumer interactions, as a framework for the most important factor was Course Content, followed by
development of site content and structure, and for evaluating Instructor Support and Mentoring, Course Structure, Infor-
the technology used in customer relationship management mation Delivery Technology, Student-to-Student Interaction,
(CRM) programs for consumer and business settings. While and Instructor-Student Interaction. This leads to a number of
these findings are valuable, future research is needed to deter- implications for online marketing education.
mine how generalizable these dimensions are to virtual com- The number one factor in determining the effectiveness of
munities in the business world. the online education experience was the content of the course.
The study also developed reliable measures of the online This is both surprising and encouraging for faculty members
education experience. This is an important step in the devel- who have complained for years that the degree to which a stu-
opment of assessment and training tools for online education dent “liked” an instructor rather than course content or some
and other virtual communities. Each of the measures suggests other more pertinent characteristic of teaching drives student
a variety of activities and skills that are necessary in the devel- evaluations. In fact, the three substantive aspects of online
opment of successful online courses, for example, designing education (Course Content, Course Structure, and Informa-
the course content of online courses, structuring the course tion Delivery Technology) were ranked as the first, third, and
and its activities, facilitating student-to-student interactions, fourth most important factors, while only Instructor Support
motivating students in a non-face-to-face environment, inter- and Mentoring made it into the top four factors. In online
acting with students via written rather than verbal communi- courses, the design and inclusion of well-developed materials
cations, and mastering the requisite technology. These skills can do a lot to facilitate learning, to assist in the application of
and activities are also important for training marketing fac- theory to solve problems, to explain difficult material, and so
ulty members to make the transition from traditional class- forth. In addition, it appears that students might credit these
room teachers to online facilitators and mentors. At some materials to the abilities of the course instructor in absentia.
point in time, these online educational skills and activities This supports the services marketing research that suggests
may become requisite for all faculty members. the importance of tangible cues and makes intuitive sense
The regression analysis provided three significant find- since the students have few other observable signals on which
ings. First, the model was able to account for 72% of the vari- to make the evaluation. Although the credit is often given to
ance in students’ evaluation of the effectiveness of the online the educator, the process can largely occur without any “real”
education experience. This suggests that the model is not only interaction on the part of the instructor. Future research
272 DECEMBER 2003

should explore the extent to which these issues are more teria used by the WebCT rubric and the model developed in
related to course content/design as opposed to instructor sup- this study, important differences exist in the specifics items
port and mentoring in online educational settings. employed. We highly recommend that marketing depart-
Furthermore, the results suggest that instructor-to-student ments consider using both peer evaluations based on the
and student-to-student interactions are the least important WebCT Exemplary Course Project rubric and student evalua-
factors in the evaluation of the overall effectiveness of online tions based on the criteria we have identified with this empiri-
courses. While many universities emphasize the importance cal research. In this way, peer and student evaluations can be
of these interactions as an essential part of the “educational considered when assessing he effectiveness of online courses
experience,” the results are not necessarily counterintuitive. It and programs.
seems reasonable that many students who seek out online
courses may go into the process with less need for highly
CONCLUSION
interactive learning environments or perhaps lower expecta-
tions. This suggests that marketing faculty members who This study starts where the Journal of Marketing Educa-
have a difficult time relating to students in face-to-face inter- tion’s April 2001 special issue on online learning (Karns and
actions might fair better in an online education setting. How- Pharr 2001) left off. The research reported here provides sig-
ever, it should be noted that both instructor-to-student and nificant findings for marketing educators and others inter-
student-to-student interactions were statistically significant ested in developing, evaluating, and teaching students about
predictors of the overall effectiveness of the online course. virtual communities. Six dimensions of online learning vir-
More research is needed in this area to determine if the order- tual communities were identified. Measures of each of the
ing of importance remains constant across a variety of online dimensions were developed and purified. One of the major
educational settings. contributions of this research is the development and confir-
Also, this ordering suggests that those interested in online mation of the online assessment model.
teaching are likely to need to learn new skills. For example, The relative importance of the dimensions in accounting
while little is known about the best way to structure an online for the variance in students’ perceptions of the effectiveness
course as compared with a traditional seat time course, it is of the online experience was reported. Employing a student-
clear that differences do exist. Similarly, it appears that the customer focus, this information provides marketing educa-
technology is an important driver of the effectiveness of the tors with direction in terms of designing and teaching online
online education experience. As such, the online educator courses. The results suggest that the most critical driver is
must become familiar with the various technologies and course content followed by course support and mentoring.
determine how to get the maximum value out of each avail- Marketing educators have long been interested in ways to
able form. More research is needed in this area to determine enhance marketing curricula and courses (cf. Atwong and
whether this ordering remains the same for undergraduate Hugstad 1997; Benbunan-Fich et al. 2001; Celsi and
marketing courses as well as executive and continuing educa- Wolfinbarger 2002; Eastman and Swift 2001; Kaynama and
tion courses and modules. Keesling 2000; McCorkle, Alexander, and Reardon 2001;
Many marketing faculty members are accustomed to Ponzurick, France, and Logar 2000; L. J. Smith 2001;
working on the content and structure of their courses during Sweeney and Ingram 2001). This study continues in that
the semester rather than prior to it. With an online course, the tradition by offering insights into effective teaching of online
content, structure, and technology have to be determined and MBA-level courses. Table 4 provides a list of recommenda-
finalized before the course begins. This requires a consider- tions specific to online teaching for each of the six dimen-
able change in the approach to teaching. Educators must plan sions based on insights generated from the 12 different online
on rearranging their schedules to include substantial graduate marketing courses at the university used in this
between-terms time commitments. Those marketing faculty study.
members who traditionally use this time for personal pur- The findings presented here are especially important for
poses, travel, and research activities will need to make adjust- marketing academics who are interested in the use of virtual
ments. During the semester, faculty members have to be con- communities as a marketing communication and education
cerned with learning support, student-to-student interactions, tools. The approach of viewing online education from a vir-
and communicating with individual students. tual communities framework allows marketing academics to
Marketing educators should consider the findings here blend traditional curriculum assessment with current
complementary to the WebCT Exemplary Course Project thoughts pertaining to the use of the Internet as a marketing
(2003). The project provides a rubric for evaluating the Web- communication tool. The highlights of the study for those
enhanced and Web-based courses by outside reviewers (e.g., interested in virtual communities include the following:
peers). The criteria employed include course design, interac-
tion and collaboration, technology, assessment, and learner 1. The development of reliable measures of virtual commu-
support. While there is substantial overlap in terms of the cri- nities.
JOURNAL OF MARKETING EDUCATION 273

TABLE 4
GUIDELINES FOR MARKETING EDUCATORS TEACHING ONLINE COURSES

Online Course Effectiveness Recommendations

Course Content Students have various preferred learning styles—a range of learning tools should be used. Giving
students several options for assignments/projects is often helpful.
CD-ROMs and other material-delivery technologies need to be up-to-date, including Web linkages to
interesting information.
Step-by-step tutorials for quantitative materials are important (e.g., SPSS tutorials).
Marketing simulations work very well online.
In that online students have business experience, projects and exercises that address real-world
application receive high marks.
Emphasizing important concepts and principles is important, but the instructor also needs to challenge
students to think.
Instructor Support and Mentoring The professor as mentor is very important. Online students are dependent on the instructor to facilitate
learning through this delivery method.
Communication is vital. Frequent contacts via e-mail, phone, and live online chats are valued by
students.
Setting up a class e-mail list (i.e., list server) so that general messages can be sent to the entire class
is absolutely necessary. The instructor can then communicate efficiently to the entire class to provide
encouragement, to give course updates, and to provide feedback to all when a question is submitted
by an individual student that may be of interest to all.
Professors need to be active participants in class discussions in the virtual community. Even little
comments show involvement.
Detailed feedback to student work is needed. A detailed answer key is well received by students,
including how points were assigned.
Course Structure Use multiple media—including CD-ROMS, chat rooms, Web links, PowerPoint audiovisual lectures, and
any of a host of other materials that have been developed by others.
Ease of delivery is vital. Clearly lay out expectations, assignment/project directions, due dates, and so
forth.
The technology used to deliver the course needs to be easy to use. The students need quick access to
a help desk for troubleshooting.
Prepare a training video showing students how the course works, especially in the use of the various
programs.
Introduce each learning module with an overview session of what will be covered in the module and
how learning of the materials will be evaluated.
It is helpful to maintain some consistency in developing the modules of the course so students have an
idea of what is expected after completing the first module.
Information Delivery Technology The quality of the materials that students receive as part of the course greatly affects their perceptions
of the overall quality of the course. The online program discussed in this article uses CD-ROMS to
deliver audiovisual materials. These materials must be developed with up-to-date technology and
high quality in mind.
Although not necessary for all lectures, prerecorded audiovisual materials that actually have the profes-
sor talking on screen work well.
College faculty members use E-teach, which simultaneously allows the professor to be seen and/or
heard; PowerPoint slides; and a scrolling outline that lets students move back and forth through the
presentation.
Course material offered should provide additional information to students, not just reflect the same con-
tent that can be obtained from a textbook.
Student-to-Student Interaction Group learning is critical to the success of online classes—students isolated from others tend to feel
like they are taking a correspondence course.
Encourage students to e-mail one another—especially group members.
Through class e-mail and/or the discussion room, allow students to discuss problems encountered,
solutions, sources of information, and so on that do not violate learning goals.
Require and grade student participation in discussion-based activities. This is one of the more critical
components to online learning.
Student-Instructor Interactions It is vital that the instructor be easily available and open to students.
Turnaround time to e-mailed questions should be within 24 hours, sooner if possible. Frustration sur-
faces if students cannot proceed due to lack of contact.
Encourage students to contact you for any reason and provide multiple ways to do so.
Live chat sessions (e.g., MSN Instant Messenger) are highly successful.
NOTE: SPSS = Statistical Package for the Social Sciences; MSN = Microsoft Network
274 DECEMBER 2003

2. The recognition of the multidimensional nature of virtual ving virtual communities with different cultures and perhaps
learning communities. more diverse cultures is unknown. Furthermore, while the
3. The identification of multiple drivers of virtual communities’ response rate of 55% is impressive, one must be concerned
effectiveness and that each factor affects effectiveness in a with the 45% of students who did not respond to the survey.
different way.
Also, the data may suffer from data loss and sample selection
4. The three most important factors in virtual community effec-
problems that are inherent in end-of-the-semester evalua-
tiveness are content, manager support, and virtual commu-
nity structure. tions. As noted by Becker (1997), students who fail to com-
5. The importance of the virtual community manager in the plete the course will not be assessed but may have some
success of a virtual learning community. strong opinions concerning the effectiveness of the learning
environment they witnessed while in the course.
Virtual communities can be used for a variety of marketing In addition, the information delivery technology that we
activities ranging from information dissemination to cus- examined here focused on the most common form of elec-
tomer educational activities to word-of-mouth communica- tronic lecture presentation delivery currently being used in
tions. Marketing educators teaching Internet marketing, data- online education, CD-ROMs. As other technologies emerge
base marketing, or other applications of virtual communities as alternatives to deliver electronic course materials via the
will find this information valuable. These findings can be Internet, and as students begin to adopt faster information
incorporated into class lectures and discussions pertaining to delivery technologies (e.g., from dial-up, to digital subscriber
the use of virtual communities as an Internet-based marketing line [DSL}, and beyond), CD-ROMS will likely be replaced
tool for developing dialogue with and between current and by on-demand student-server connections. These new forms
future customers, as learning support and value-adding vehi- of information delivery technology will need to be investi-
cles, as tools to increase consumer-to-consumer interactions gated in terms of their impact on the overall effectiveness of
and manager-consumer interactions, as a framework for the online courses.
development of site content and structure, and for evaluating This study represents an important first step in providing
the technology used in CRM programs for consumer and greater insight into the effective development and use of vir-
business settings. tual communities. The results of this study aid marketing edu-
While this research represents an important step forward cators in two ways. First, it provides information to help them
in the development and evaluation of online learning and vir- become better online educators and to develop better online
tual communities, it raises many additional research ques- courses and programs. Second, this article provides insights
tions. One question is whether there are temporal relation- pertaining to the use of this new educational and business
ships between the dimensions. For example, does instructor model in marketing settings. It provides insights for market-
support affect perceptions of course content? This and related ing educators interested in incorporating information about
interrelationships can be investigated with the use of struc- virtual communities into marketing classes. Marketing stu-
tural equation modeling. Also of importance, can the results dents need to be provided with the knowledge of the effective
be generalized to other online education settings and other management of virtual communities in the business world
virtual communities? Although we borrowed theory and and how they can be used to build communication networks
practice from the e-marketing world, much of what was with and between customers, motivate purchases, increase
learned can be applied and explored further in commercial spending, and strengthen relationships. It is also hoped that
contexts. Replications and extensions of the research pre- the results of this study will motivate marketing educators to
sented here are needed. In addition, how do the dimensions continue to do research to develop a better understanding of
differ from those of traditional classes? How does the order of virtual communities.
these differ in online settings compared with traditional
classes? What are the skills that make for successful online REFERENCES
educators, and do they differ from those of the traditional
Abernathy, Donna J. 1999. www.online.learning. Training & Development
classroom? Finally, how do the online courses compare with
53 (9): 36-42.
traditional courses when it comes to accomplishing the hier- Allen, Charlotte. 1997. The virtual university. Washington Post Magazine 10
archy of learning objectives (Bloom 1956) ranging from (August): 16-34.
knowledge acquisition up to evaluation skills? Comparisons Aron, Laurie Joan. 1999. Online U. Across the Board 36 (8): 63-66.
of online and traditional classes are needed to better under- Atwong, Catherine T., and Paul S. Hugstad. 1997. Internet technology and
the future of marketing education. Journal of Marketing Education 19
stand the differences between the two methods of instruction.
(Fall): 44-55.
There are several limitations to this study that should be Baily, E. K., and M. Coltar. 1994. Teaching via the Internet. Communication
noted. First, the study represented one online program at one Education 43:184-93.
university that continues to serve a mostly regional market in Baker, M. J., E. A. Buttery, and E. M. Richter-Buttery. 1998. Relationship
terms of its virtual learning community. How generalizable marketing in three dimensions. Journal of Interactive Marketing 14
(Fall): 47-62.
its findings are to other programs in different locations ser-
JOURNAL OF MARKETING EDUCATION 275

Ballantyne, Roy, Jill Borthwick, and Jan Packer. 2000. Beyond student evalu- Gibson, Chere Campbell. 1996. Toward emerging technologies and distrib-
ation of teaching: Identifying and addressing academic staff development uted learning: Challenges and change. American Journal of Distance
needs. Assessment & Evaluation in Higher Education 25 (3): 222-336. Education 10:47-49.
Becker, William E. 1997. Teaching economics to undergraduates. Journal of Greco, JoAnn. 1999. Going the distance for MBA candidates. Journal of
Economic Literature 35 (September): 1347-73. Business Strategy 20 (3): 30-34.
Benbunan-Fich, Raquel, Hector R. Lozada, Stephen Pirog, Randi Priluck, Hagel, John. 1999. Marketplace: Net gains: Expanding markets through vir-
and JoesphWiesenblit. 2001. Integrating information technology into the tual communities. Journal of Interactive Marketing 13 (Winter): 55-65.
marketing curriculum: A pragmatic paradigm. Journal of Marketing Hagel, John, and Arthur Armstrong. 1997. Net gains: Expanding markets
Education 23 (April): 5-15. through virtual communities. Boston: Harvard Business School Press.
Bickart, Barbara, and Robert M. Schindler. 2001. Internet Forums and influ- Harasim, Linda M. 1990. Online education: Perspectives on a new environ-
ential sources of consumer information. Journal of Interactive Marketing ment. New York: Praeger.
15 (Summer): 31-40. Haynes, Deborah C., and Holly Hunts. 2000. Using teaching evaluations as a
Bishop, Ann Peterson. 2000. Communities for the new century. Journal of measurement of consumer satisfaction. Consumer Interests Annual
Adolescent & Adult Literacy 43 (5): 472-78. 46:134-39.
Bloom, Benjamin S. 1956. Taxonomy of educational objectives: The classifi- Hiltz, Starr R. 1986. The virtual classroom: Using computer-mediated com-
cation of educational goals, by a committee of college and university munication for university teaching. Journal of Communication 36 (2):
examiners. New York: Longman’s Green. 95-104.
Boex, L. F. Jameson. 2000. Attributes of effective economics instructors: Holland, Jonna, and Stacey Menzel-Baker. 2001. Customer participation in
An analysis of student evaluations. Journal of Economic Education 31 creating site brand loyalty. Journal of Interactive Marketing 15 (4): 34-
(Summer): 211-26. 45.
Bressler, Stacey E., and Charles E. Grantham. 2000. Communities of com- Houston, Mark B., and Lance A. Bettencourt. 1999. But that’s not fair! An
merce. New York: McGraw-Hill. exploratory study of student perceptions of instructor fairness. Journal of
Brightman, Harvey J., Merwyn L. Elliot, and Yezdi Bhada. 1993. Increasing Marketing Education 21 (August): 84-96.
the effectiveness of student evaluations of instructor data through a factor Karns, Gary L., and Stephen Pharr. 2001a. Editor’s corner. Journal of Mar-
score comparative report. Decision Sciences 24 (1): 192-99. keting Education 23 (April): 3-4.
Brown, John Seely, and Paul Duguid. 2000. The social life of information. , eds. 2001b. Integrating technology and distance learning in market-
Boston: Harvard Business School Press. ing education. Journal of Marketing Education 23 (1).
Brown, Shone L., and Kathleen M. Eisenhardt. 1998. Competing on the edge: Karp, David A., Gregory P. Stone, and William Yoels. 1977. Being urban: A
Strategy as structured chaos. Boston: Harvard Business School Press. social psychological view of city life. Lexington, MA: D. C. Heath.
Carpenter, Rebecca. 1998. The anywhere MBA. Canadian Business 71 Karuppam, Corinne, and Muthu Karuppum. 1999. Empirically based guide-
(October): 62-67. lines for developing teaching materials on the Web. Business Communi-
Carter, Alex. 2001. Interactive distance education implications for the adult cations Quarterly 62 (3): 37-45.
learner. International Journal of Instructional Media 28 (3): 24-26. Kaynama, Shohreh, and Garland Keesling. 2000. Development of a Web-
Celsi, Richard L., and Mary Wolfinbarger. 2002. Discontinuous classroom based Internet marketing course. Journal of Marketing Education 22
innovation: Waves of change for marketing education. Journal of Mar- (August): 84-89.
keting Education 24 (April): 64-73. Kelley, M. E. 1987. Course teams and instructional design in Australian dis-
Chen, Qimei, and William D. Wells. 1999. Attitudes toward the site. Journal tance education. Distance Education 8 (1): 106-20.
of Advertising Research 39 (5): 27-37. Kitchen, David, and Douglas McDougall. 1998-1999. Collaborative learning
Churchill, Gilbert A. 1979. A paradigm for developing better measures of on the Internet. Journal of Educational Technology Systems 27:245-58.
marketing constructs. Journal of Marketing Research 16 (February): 64- Klein, Lisa R. 2003. Creating virtual product experiences: The role of
73. telepresence. Journal of Interactive Marketing 17 (Winter): 41-55.
Deighton, John. 1997. Commentary on exploring the implications of the Kozinets, Robert V. 1999. E-tribalized marketing? The strategic implications
Internet for consumer marketing. Journal of the Academy of Marketing of virtual communities of consumption. European Management Journal
Science 25:347-51. 17 (3): 252-64.
Donoho, Ron. 1998. The new MBA. Training 35 (October): D4-D9. Lang, Karl, and J. Leon Zhao. 2000. The role of electronic commerce in he
Drago, William, James W. Peltier, and Don Sorensen. 2002. Course content transformation of distance education. Journal of Organizational Com-
or instructor: Which is more important in online teaching? Management puting & Electronic Commerce 10 (2): 103-27.
Research News 25 (6/7): 69-83. Lawrence, Thomas B. 1995. Power and resources in an organizational com-
Eastman, Jacqueline K., and Cathy Owens Swift. 2001. New horizons in dis- munity. Academy of Management Journal, Special Issue Best Paper Pro-
tance education: The online learner-centered marketing class. Journal of ceedings, 251-55.
Marketing Education 23 (April): 25-34. Lee, June. 2002. Faculty and administrator perceptions of instructional sup-
Entwistle, Noel, and Hilary Tait. 1990. Approaches to learning, evaluations port for distance education. International Journal of Media 29 (1): 27-45.
of teaching, and preferences for contrasting academic environments. Leonard, David C. 1999. The Web, the millennium, and the digital evolution
Higher Education 19:169-94. of distance education. Technical Communications Quarterly 8 (Winter):
Feldman, Kenneth A. 1976. Grades and college students’ evaluations of their 9-21.
courses and teachers. Research in Higher Education 18 (March): 69-111. Long, Peter, Tony Tricker, Margaret Rangecroft, and Peter Gilroy. 2000. Sat-
Figallo, Cliff. 1998. Hosting Web communities: Building relationships, isfaction with distance education. Total Quality Management 11 (4-6):
increasing customer loyalty, and maintaining a competitive edge. 530-36.
Chicester, UK: Wiley. Looi, C. K., and D. Ang. 2000. A multimedia-enhanced collaborative learn-
Fitzgerald, Gail. 1995. The effects of an interactive videodisc training pro- ing environment. Journal of Computer Assisted Learning 16:2-13.
gram in classroom observation skills used as a teaching tool and as a Lowman, John, and Virginia A. Mathie. 1993. What should graduate teaching
learning tool. Computers in Human Behavior 11:467-79. assistants know about teaching? Teaching of Psychology 29 (2): 84-88.
Fombrun, Charles J. 1996. Reputation: Realizing value from the corporate Maharg, Paul. 2001. Negotiating the Web: Legal skills learning in a virtual
image. Cambridge, MA: Harvard Business School Press. community. International Review of Law Computers & Technology 15
Fraser, Barry J. 1986. Classroom environment. London: Croom Helm. (3): 345-60.
276 DECEMBER 2003

Markel, Mike. 1999. Distance education and the myth of the new pedagogy. Rheingold, H. 2000. The virtual community: Homesteading on the electronic
Journal of Business & Technical Communication 13:208-22. frontier. Cambridge, MA: MIT Press.
Marsh, Herbert W. 1987. Students’ evaluation of university teaching: Roach, Scott S., Mark W. Johnston, and Joseph F. Hair. 1993. An exploratory
Dimensionality, reliability, validity, potential biased, and utility. Journal examination of teaching styles currently employed in marketing educa-
of Educational Psychology 11:253-388. tion. Journal of Marketing Education 15 (Fall): 32-38.
Mathwick, Charla. 2002. Understanding the online consumer: A typology of Roblyer, M. D. 1999. Is choice important in distance learning? A study of stu-
online relational norms and behavior. Journal of Interactive Marketing dent motives for taking Internet-based courses at the high school and
14 (Winter): 40-55. community college levels. Journal of Research on Computing in Educa-
McAlister, Debbie T. 2001a. The editor’s corner. Journal of Marketing Edu- tion 32 (Fall): 157-71.
cation 23 (December): 167-68. Rothaermel, Frank T., and Stephen Sugiyama. 2001. Virtual Internet commu-
. 2001b. Innovation in marketing education. Journal of Marketing nities and commercial success: Individual and community-level theory
Education 23 (3). grounded in the atypical case of TimeZone.com. Journal of Management
McCorkle, Denny E., Joe F. Alexander, and James Reardon. 2001. Inte- 27 (3): 297-314.
grating business technology and marketing education: Enhancing the dif- Shea, Timothy, Luvai Motiwalla, and David Lewis. 2001. Internet-based dis-
fusion process through technology champions. Journal of Marketing tance education: The administrator’s perspective. Journal of Education
Education 23 (April): 16-24. for Business 77 (2): 112-17.
McGrath, Carolyn. 1997-98. A new voice on Interchange: Is it talking or writ- Sherron, Gene, and Judith V. Boettcher. 1997. Distance learning: The shift to
ing? Implications for the teaching of literature. Journal of Educational interactivity. Boulder, CO: Cause.
Technology Systems 26:291-97. Shevlin, Mark, Philip Banyard, Mark Davies, and Mark Griffiths. 2000. The
McWilliam, Gil. 2000. Building stronger brands through online communi- validity of student evaluation of teaching in higher education: Love me,
ties. Sloan Management Review 41 (Spring): 43-54. love my lectures? Assessment & Evaluation in Higher Education 25 (4):
Moore, Michael G. 1990. Contemporary issues in American distance educa- 397-405.
tion. New York: Gannon. Siegel, Carolyn F. 1996. Introducing marketing students to business intelli-
Murphy, Ian P. 1997. Web “communities” a target marketer’s dream. Mar- gence using project-based learning on the World Wide Web. Journal of
keting News 31 (14): 2-3. Marketing Education 18 (August): 14-24.
Navarro, Peter, and Judy Shoemaker. 1999. The power of cyberlearning: An Smart, Denise T., Craig A. Kelley, and Jeffrey S. Conant. 1999. Marketing
empirical test. Journal of Computing in Higher Education 11 (1): 29-54. education in the year 2000: Changes observed and challenges anticipated.
Nelson, Jon P., and Kathleen A. Lynch. 1984. Grade inflation, real income, Journal of Marketing Education 21 (December): 206-16.
simultaneity, and teaching. Journal of Economic Education 15 (Winter): Smith, David H. 1996. Developing a more interactive classroom: A continu-
21-37. ing odyssey. Teaching Sociology 24 (1): 64-75.
Nielsen, Jacob. 2000. Designing Web usability. Indianapolis, IN: New Smith, Lois J. 2001. Content and delivery: A comparison and contrast of elec-
Riders. tronic and traditional MBA marketing planning courses. Journal of Mar-
Nunnally, Jum C. 1978. Psychometric theory. 2nd ed. New York: McGraw- keting Education 23 (April): 35-44.
Hill. Sonner, Brenda S. 1999. Success in capstone business course: Assessing the
O’Leary, Mick. 2002. E-global library advances the virtual library. Informa- effectiveness of distance learning. Journal of Education for Business 74
tion Today 19 (3): 19-21. (March–April): 243-47.
Palloff, Rena M., and K. Keith Pratt. 1999. Building learning communities in Stevenson, Julie S., Gordon C. Brunner II, and Anand Kumar. 2000. Webpage
cyberspace: Effectiveness strategies for the online classroom. San Fran- background and viewer attitudes. Journal of Advertising Research 40 (1):
cisco: Jossey-Bass. 51-62.
Patrick, Jeff, and Roslyn M. Smart. 1998. An empirical evaluation of teacher Sullivan, Patrick. 2001. Gender differences and the online classroom: Male
effectiveness: The emergence of three critical factors. Assessment & and female college students evaluate their experiences. College Journal
Evaluation in Higher Education 23 (2): 165-78. of Research & Practices 25:805-18.
Peltier, James W., John A. Schibrowsky, John Davis, and Donald Schultz. Sweeney, Jillian C., and Deborah Ingram. 2001. A comparison of traditional
2002. Interactive psychographics: Cross-selling opportunities in the and Web-based tutorials in marketing education: An exploratory study.
banking industry. Journal of Advertising Research 42 (2): 7-22. Journal of Marketing Education 23 (April): 55-62.
Peter, J. Paul. 1979. Reliability: A review of psychometric basics and recent Tricker, Tony, Margaret Rangecroft, Peter Long, and Peter Gilroy. 2001.
marketing practices. Journal of Marketing Research 16 (February): 6-17. Evaluating distance education courses: The student perception. Assess-
Phillips, Vicky. 1998. Online universities teach knowledge beyond the books. ment & Evaluation in Higher Education 26 (2): 165-77.
HR Magazine 43 (8): 120-28. Ueltschy, Linda. 2001. An exploratory study of integrating interactive tech-
Ponzurick, Thomas G., Karen R. France, and Cyril M. Logar. 2000. nology into marketing curriculum. Journal of Marketing Education 23
Delivering graduate marketing education: An analysis of face-to-face (April): 63-72.
versus distance education. Journal of Marketing Education 22 (Decem- Wachtel, Howard K. 1998. Student evaluation of college teaching effective-
ber): 180-87. ness: A brief review. Assessment & Evaluation in Higher Education 23
Pye, Joe. 1999. Perspectives of ICT in professional development and educa- (2): 191-211.
tion. Information Services & Use 19 (4): 307-12. WebCT Exemplary Course Project. 2003. http://webct.com/exemplary.
Rahm, Diane, and B. J. Reed. 1997. Going remote: The use of distance learn- Wechsler, A. 2002. Online learning lasts an ever-widening net. Times Union,
ing, the World Wide Web, and the Internet in graduate programs of public March 3.
affairs and administration. Public Productivity and Management Review Wind, Jerry, and Arvind Rangaswamy. 2001. Customerization: The next rev-
20 (4): 459-71. olution in mass communication. Journal of Interactive Marketing 15
Ramsden, Paul. 1991. A performance indicator of teaching quality in higher (Winter): 13-32.
education: The course experience questionnaire. Studies in Higher Edu- Young, Mark R. 2001. Windowed, wired, and webbed: Now what? Journal of
cation 16:129-50. Marketing Education 23 (April): 45-54.
Remmers, H. 1960. Manual of instructions for the Purdue Rating Scale of
Instruction. West Lafayette, IN: Purdue University.

You might also like