Online Courses as Effective Learning

Environments or "Digital Diploma Mills":

The Importance of Collaborative Learning
 Keynote Address, Telelearning ‘99
 Montreal Canada November

 Copyright, Starr Roxanne Hiltz

 New Jersey Institute of Technology
Two views of Online courses
 A means of cutting the costs of “delivering”
educational materials to students, akin to
previous use of instructional television. Post
materials on the web; collect assignments; can
handle thousands of students.
 OR: A means of improving the quality of learning
opportunities, by supporting learning
communities in an anytime/anywhere
environment (Learning Networks “TLN” or
“ALN”)- require small classes mentored by
skilled faculty members
Popular press: negative images of
online courses
 Article title: “Wiring the Ivory
Tower: But will online courses lower
standards?”
 Business Week, August 9, 1999

 “No dorms, no sports fields, NO

COSTLY PROFESSORS”
Cites UNext:
 Will spend $1 million a course for
video streamed lectures by “stars”
 Use part time instructors to answer
email and grade assignments
 NOT what is meant by LN!!
AFT/NEA Commissioned report
on Distance Learning
 Asks, “What’s the difference” between
traditional and distance courses;
argues there is no proof that
distance/online is as good or better
Some Questions about “Differences”
This Talk Will Cover:
 Is online collaborative learning superior to
the mass distribution of materials via the
Web?
 How do both models compare to traditional
college level courses, in terms of process and
outcomes?
 What are some guidelines for maximizing the
quality of research to answer these
questions?
Background
 Over the last 15 years, NJIT has constructed a
series of computer-mediated communication
systems tailored to support “anytime/
anywhere” interaction among students and
instructors called “Virtual Classroom” [TM]
 Used first in a variety of individual courses
and then for full degree programs; and
 Developed various evaluation instruments and
approaches.
 TM
Virtual Classroom Project at
NJIT
 1993-1996 produced, delivered and evaluated
26 courses comprising the undergraduate
majors in Information Systems and Computer
Science, with Sloan Foundation support
 Continuing: “From Virtual Classroom to
Virtual University” 1997-2000
 Expand the innovation to other schools
and departments and graduate degree
programs
8
Theories for Studying Online Courses
 Pedagogical Theories: Objectivist (passive) vs.
Constructivist (active, collaborative) learning
 Media Effect theories (e.g., Media Richness,
Media Synchronicity)
 Group Interaction Theories (e.g., Adaptive
Structuration; Poole & DeSanctis)
 …LN’s are a social technology through which a
group may choose to faithfully or unfaithfully
appropriate the structures provided by the
technology, heuristic, environment, etc…
Premises of the NJIT studies:
 Online courses provide unique
opportunities to support
collaborative (group based) learning
 Collaborative learning is crucial to
the effectiveness of online learning
environments
 Some VARIETIES OF COLLABORATIVE
LEARNING
 Seminar: Students as Teachers
 Peer Writing Groups (Constructive Criticism)
 Group Projects
 Case study discussions
 Web “treasure hunts,” compilations
 Debates
 Construct an exam
 Networked classes
The Research Process
… a series of interlocking choices in which
we try simultaneously to maximize
several conflicting choices…

Key choices
 generalizability with respect to populations
 realism for the participants

 precision in control and measurement of variables

McGrath, 1982
Methodological Tradeoffs
 Large sample surveys can maximize generalizability
 Laboratory Experiments can maximize precision of
control
 Field experiments can maximize realism
 Qualitative Methods can maximize depth of
understanding ( the “why”)
 “Triangulation” or multiple methods maximize overall
validity of results
 Replication of results in different settings provides
generalizability
Questionnaire Data (1993- 1996
completed project)
 Total Responses (sometimes partial):
 Virtual Classroom + video: 698
 VC + FtF: 463
 No VC: 268
 Question Form: Please compare online
classes to your previous experiences with
“face to face” college - level courses. To
what extent do you agree with the following
statements...
14
Taking Online Classes is More
Convenient (73% agree)
37
40 36

35
30
25
Percentage

17
(%)

20
15
10 7
4

5
0
Strongly A. Agree Neutral Disagree Strongly D.

N = 624

15
Having the computerized conferencing system available
provided better access to the professor(s). (65% agree)

42
45
40
35 29

30
Percentage

25
(%)

17
20
15
8
10 4

5
0
Strongly A. Agree Neutral Disagree Strongly D.

N = 622, Mean= 2.2

16
Did use of the system increase the quality
of your education? (58% agree; +22%
Same)

30 27

25 22
19

20
Percentage
(%)

15 12

10 7 7
5

5
0
1 2 3 Unsure 5 6 7

Definitely Yes = 1 N = 617

Definitely Not = 7

17
“Virtual Classroom Overall”
Index Items included:
 1. VC Increases Quality
 2. Better learning
 3. Learned More
 4. Would not take another VC
 5. Would have gotten more from a
traditional class
 6. VC increases efficiency of learning
Chronbach’s Alpha = .85
18
Correlations with VC Overall
(Pearson’s)
 All significant at least at .01 level;
n= about 523
 Collaboration Index .30
 Better Access to Professor .46
 Felt more involved in taking active
part .54

19
Conclusions based on field trials
 LNs DO tend to increase access to educational
opportunities, efficiency (speed of completion)
of the degree, and quality of educational
experience as subjectively reported
 This is conditional on a number of factors, including active
participation by the student, and communicating/
collaborating with classmates
 No Significant Differences in Course Grades
between traditional and LN sections

20
Replication: Similar results from
 U. of Illinois, Drexel, SUNY, U. of Central
Florida, etc.
 E.g. Drexel: 95% felt they had better access to Prof
 Vast majority of students report that LN courses
are “better” in terms of access and learning
 Student evaluations strongly correlated with
amount on online interaction with professor and
with other students
BUT:
 In terms of the relationship between
Collaborative Learning and the effectiveness
of LNs, “correlation is not causation”
 Triangulation: One course in the project,
Computers and Society, was selected for a field
experiment
 Effects of Computer-Mediated Communication on
Learning, Performance and Satisfaction: A
Comparison of Groups and Individuals Solving
Ethical Scenarios, A thesis by: Raquel Benbunan
TASK SELECTION RESEARCH METHODS

 Discussion and solution of computer

ethics scenarios
 Decision-making task, completed by writing a
report on analysis and recommendations for
action
 Key knowledge in the course and on the final
exam

23
EXPERIMENTAL DESIGN
Distribution of Subjects across Conditions

MANUAL ONLINE

IM 44 subjects IOL 42 subjects

INDIV.

GFtF 28 subjects GOL 22 subjects

GROUPS (5 groups) (5 groups)

24
Mean Final exam scores on ethics,
(GPA as covariant)
Manual On-line

Individuals 71.83 64.64 68.23

Groups 73.97 70.19 72.08

72.90 67.42

25
 Length of the Reports

Mean Words by
Condition
Manual Online

Individuals 380 462 421

Groups 390 756 573

386 609 481

Model F = 8.98 p = .0001 ***

Teamwork Effect F = 11.60 p = .0009 ***
Online Effect F = 21.10 p = .0001 ***
Interaction Effect F = 10.02 p = .002 **

** = Significant at p < .01; *** = Significant at p<.001

Triangulation: Faculty Views
 Semi-Structured Interviews with 20 NJIT Faculty

 Those faculty who utilized group/collaborative

learning and obtained active participation in
discussions online tend to perceive that students
learn more online than in the traditional
classroom; those who did not make online
discussions and group work a central course
activity did not
 Replication: SUNY faculty- 47% felt online
students learned more; 46% saw no difference;
only a few thought the classroom performed better
Faculty Workload and Satisfaction- 100
SUNY faculty (1999)
 Preparation time: 43% said “much
more” and 41% said “more” than
traditional classroom
 Teaching time: 25% said “much more”
and 39% “more”
 Teacher-student interaction: 52% more
or much more; 31% the same
 Would they do it again: 99 yes
Summary and Conclusions:
 Each of the three NJIT studies has its methodological
weaknesses, but taken together they solidly support
the conclusions that
 1.L N courses can be as effective or more effective than
traditional classrooms, in terms of access and learning
outcomes.
 2. Collaborative learning designs are more effective for
online learning than individuals working alone with
materials posted online.
 Data from other universities replicate the findings.
 What’s the Difference? It’s the pedagogy,
stupid!!
A Challenge We Share
 Answer the critics with data: Replicate,
Triangulate, INNOVATE

 For More Info:

 Roxanne’s home page:
http://eies.njit.edu/~hiltz
 WWW.ALN.ORG