You are on page 1of 10

Received: 9 May 2017 Revised: 30 November 2017 Accepted: 25 February 2018

DOI: 10.1111/jcal.12258

ORIGINAL ARTICLE

Effectiveness of learning in online academic courses compared


with face‐to‐face courses in higher education
Tal Soffer | Rafi Nachmias

School of Education, Tel Aviv University, Tel


Aviv, Israel Abstract
Correspondence This study examined the effectiveness of 3 online courses compared with the same 3 courses in a
Tal Soffer, School of Education, Tel Aviv face‐to‐face (F2F) format, which had the same characteristics (e.g., the same instructor and final
University, P.O.B 39040, Ramat Aviv, Tel Aviv exam content and place). Effectiveness was examined by utilizing a wide range of variables,
69978, Israel.
Email: talsofer@post.tau.ac.il
including 2 objective measures (N = 968): grades and completion rate; and 9 subjective measures
(N = 360): 7 measures include instructional aspects (course structure, learning content, lessons
watched, assignments, communication), engagement, and satisfaction. Findings indicate signifi-
cant differences between online and F2F courses in most of the examined variables. Students
in the online courses reported better understanding of the course structure, better communica-
tion with the course staff, watching the videos lessens more, and higher engagement and satisfac-
tion. Students in the F2F courses reported better contribution of the learning content. Students'
final grades were higher in the online courses, and no differences were found in the completion
rate. The findings suggest that in many of the examined effectiveness aspects, online courses are
as effective as, or more effective than, F2F courses. Interpretations of the findings are discussed.

KEY W ORDS

course effectiveness, learning outcomes, online learning, online courses, students' perceptions

1 | I N T RO D U CT I O N curricula in higher education, there is an ongoing debate about the


quality and effectiveness of such courses compared with face‐to‐face
Online courses are not a new phenomenon in higher education. How- (F2F) courses (Allen & Seaman, 2015; Hannafin, Hill, Oliver, Glazer, &
ever, in recent years, we have witnessed a rapid growth in online aca- Sharma, 2003) and a lack of compelling quantitative evidence of learn-
demic courses, which are offered by higher education institutions ing effectiveness (Lack, 2013). Indeed, calls were made to explore and
(Toven‐Lindsey, Rhoads, & Lozano, 2015). New technologies offer identify if and how the formats of online courses are different in their
advanced learning platforms that provide suitable and easier environ- effectiveness in regard to students' learning and satisfaction
ments for implementing online courses along with cost savings (Crawford‐Ferre & Wiest, 2012; Nguyen, 2015), in order to better
(Farinella, Hobbs, & Weeks, 2000; Garbett, 2011; Kim & Bonk, address teaching and learning processes (Lim, Morris, & Kupritz,
2006; Pape, 2010), which make them more attractive to university 2007; Song, Singleton, Hill, & Koh, 2004).
administrators and students. In fact, students are much more likely The current study adds to the existing literature in two ways: First,
to experience an online learning environment now than they were in it examines differences between three F2F and online courses across
the past (Tichavsky, Hunt, Driscoll, & Jicha, 2015), with 32% of all various disciplines, which provides a relatively large sample size that
enrolled students in 2011 having taken at least one online course conclusions can be inferred from, and second, it expands the scope
(Allen & Seaman, 2014). Indeed, decision makers at academic institu- of the term “effectiveness”. Specifically, effectiveness of learning in
tions consider online academic courses to be a critical long‐term strat- this study was examined by a large scope of variables: learning out-
egy (Allen & Seaman, 2015; Parker, Lenhart, & Moore, 2011) for comes (achievements and completion rate), perceived instructional
improving teaching and providing access to a wide range of audiences aspects (e.g., course structure, course content learning contribution,
(Macfadyen & Dawson, 2010; Roby, Ashe, Singh, & Clark, 2013). How- assignment assessments, and communication perspectives), perceived
ever, along with the expansion of online courses within academic engagement, and satisfaction.

J Comput Assist Learn. 2018;1–10. wileyonlinelibrary.com/journal/jcal © 2018 John Wiley & Sons Ltd 1
2 SOFFER AND NACHMIAS

2 | LITERATURE REVIEW opportunities for instructor‐student and student–student interaction


in online learning environments (Dixson, 2012; Maki & Maki, 2007;
Various variables are used in the literature for comparing the quality Swan et al., 2000; Young & Norgard, 2006). It is important to provide
and effectiveness of online courses to F2F courses. Specifically, various means to facilitate such communication, including discussion
although some focus on pedagogical and instructional aspects such forums, chat rooms, email, and synchronous meetings (Wallace,
as course design, students' interactions, instructor presence, satisfac- 2004). Indeed, research has demonstrated that instructor‐student
tion, and motivation, others compare the courses in regard to educa- and student–student communication are strongly correlated with
tional outcomes such as tasks and final grades (Baker, 2010; Balkin, higher student engagement with the course (Dixson, 2012) and have
Buckner, Swartz, & Rao, 2005; Crawford‐Ferre & Wiest, 2012; Dixson, a significant and positive impact on students' learning and satisfaction
2012; Lack, 2013; Maki & Maki, 2007; Osman, 2005; Sebastianelli, (Marks, Sibley, & Arbaugh, 2005; Sher, 2009). Communication also has
Swift, & Tamimi, 2015). The following literature review will therefore an effect on students' motivation and learning outcomes (Du, Havard,
focus on these aspects. & Li, 2005; Lam, Cheng, & McNaught, 2005; Sargeant, Curran, Allen,
Jarvis‐Selinger, & Ho, 2006; Tu, 2005) as well as on collaborative learn-

2.1 Instructional aspects' effectiveness in online


| ing (Balkin et al., 2005; Tee & Karney, 2010). In contrast, a lack of
proper communication might lead to a sense of isolation and an insuf-
courses versus F2F courses
ficient sense of community (Song et al., 2004).
2.1.1 | Course structure Previous studies indicate the importance of a strong instructor
A suitable learning environment with a well‐established course design is presence for effective online courses. Instructor presence involves fac-
crucial to the quality and effectiveness of online courses. Indeed, it was ulty giving feedback to students regarding academic performance and
found that students perceived course design as affecting the success of activities, answering questions, and encouraging discussion about
an online learning experience (Song et al., 2004). In order to improve course‐related and other topics (Dixson, 2012). It also fosters a sense
course design, studies emphasize the need for a variety of instructional of connectedness and of being part of an online learning community
methods that address various learning styles as well as multiple (Sebastianelli et al., 2015; Wallace, 2004).
methods for content learning and delivery such as synchronous and
asynchronous classes (Osman, 2005; Gaytan & McEwen, 2007; Dixon, 2.1.3 | Tasks and assignments
2010; UDI Online Project, 2010; Crawford‐Ferre & Wiest, 2012). Integrating interactive activities and inquiry‐based learning pedagogies
Studies have also stressed the importance of consistency and within online courses may stimulate students to use higher order skills
modularization in course structure (Swan et al., 2000; Young & Norgard, and increase their engagement and satisfaction in the course (Chen,
2006). For example, Bruckman (2002) recommended the use of a well‐ Lambert, & Guidry, 2010; Du et al., 2005; Duderstadt, Atkins, &
established, consistent structure, both across and within courses, in Houweling, 2002). Lim et al. (2007) found that the most effective
order to achieve a standardized format. Indeed, higher levels of consis- learning activities for learners are instructional activities such as group
tency in course structure were found to be related to higher levels of and individual projects, discussion activities, and class assignments.
students' satisfaction (Shea, Swan, Fredericksen, & Pickett, 2002). Previous studies indicated that students in online courses spend much
Additionally, Peltier, Schibrowsky, and Drago (2007) claim that more time learning and performing tasks compared with F2F courses
course content is the most significant driver in students' perception of (Maki & Maki, 2007; Robertson, Grant, & Jackson, 2005).
the quality of their learning experience. Thus, course content should
be challenging and delivered in a way that motivates the learner (Drago,
Peltier, & Sorensen, 2002; Jones & Kelley, 2003; Sebastianelli et al.,
2.2 | Students' engagement and satisfaction with
2015). In this case, the video lectures, as part of the course design, have online courses versus F2F course
an important role in presenting the information in an attractive and con- Students' satisfaction is one of the key elements in determining the
sistent manner and may have an effect on learning outcomes (Squires, quality of online learning. High satisfaction leads to higher motivation
1999). Digital video technology enables students to interact with video and persistence in learning (Bekele, 2010; Pintrich & Schunk, 2002).
lectures in a non‐linear way—to select parts or play single video lectures Online courses are perceived with high satisfaction among students
with minimal search time, according to their pace. This may enhance (Russell et al., 2016; Shea, Fredericksen, Pickett, Pelz, & Swan,
students' engagement and improve learning effectiveness (Zhang, 2001) due to the variety of interactive and engaging learning envi-
Zhou, Briggs, & Nunamaker, 2006). Finally, navigation in the course ronments they offer, which encourage satisfaction among the partic-
structure and ease of use were perceived by students to be beneficial ipants. Students in online courses perceive learning to be more
to their learning in online courses (Palmer & Holt, 2010). convenient and to provide more autonomy and control than F2F
courses. Specifically, the flexibility in terms of time and location
2.1.2 | Communication allows for a certain degree of self‐paced study, which gives students
Interaction is at the core of the learning experience and is widely cited more control over their learning (Northrup, 2002; Polloff & Pratt,
as a defining characteristic of successful learning in both traditional 2001; Rodriguez, Ooms, & Montañez, 2008; Young & Norgard,
and online learning environments (Picciano, 2002; Wanstreet, 2006). 2006). Researches show that students in online courses may be more
However, this aspect is more meaningful in online courses due to the satisfied and motivated in the following conditions: when the instruc-
absence of physical meetings. As such, it is important to create tors provide clear objectives and requirements for the course, when a
SOFFER AND NACHMIAS 3

collaborative learning environment is created, when a high level of 3.1 | Research questions
interaction between instructors and students is offered, and when
The research questions were as follows:
meaningful feedback to the participants is provided (Lao & Gonzales,
2005; Swan et al., 2000; Toven‐Lindsey et al., 2015). Research found 1. Are there differences between online and F2F courses in
that students' satisfaction could be influenced by consistent course students' perceived learning effectiveness (instructional aspects,
design, in a course and across courses, and the provision of technical engagement, and satisfaction)?
support and flexibility (Bekele, 2010; Young & Norgard, 2006). This 2. Are there differences between online and F2F courses in
issue should be carefully examined, because in many cases, online students' achievements (outcomes and completion rate)?
learning requires more commitment and involvement on the part of
3. Are there differences between course subjects in the aforemen-
the students, the lack of which could lead to lower satisfaction
tioned differences between online and F2F formats.
(Kuo, Walker, Belland, & Schroder, 2014).

3.2 | Research field


2.3 | Learning outcomes in online courses versus F2F
During the 2013 academic year, the university decided to implement a
courses
process for converting F2F courses into online courses. For this pur-
Learning outcomes are the most popular measure for evaluating pose, the courses were chosen from a special program called “The
course effectiveness. Positive learning outcomes refer to improved Complementary Studies Program,” which is intended for undergradu-
learning as measured by the completion rate and grades received ate students at the university and offers students the opportunity to
by students throughout a course (e.g., quizzes, homework assign- study subjects that fascinate them. The courses are characterized by
ments, final exam, project test scores, student engagement with a wide range of participants from various disciplines, they are two
the class material, and reduction in withdrawal or failure; Nguyen, credit hours each, and they are taught during one semester.
2015). Lack (2013), in his meta‐analysis of online learning status, This study examined the first three courses that were converted.
shows that, overall, hybrid and online learning are more or less as Two courses (referred to in this research as Course A and Course B)
effective as F2F learning and that most studies had mixed results were studied in the humanities faculty, and the third course (referred
when comparing outcomes between the different course formats. to in this research as Course C) was studied in the medicine faculty.
Specifically, although some studies found no significant differences Each of the three courses was taught once in the F2F format and once
between the course formats in learning outcomes (Lim et al., in the online format (a total of six courses).
2007; Neuhauser, 2010), others found that students reported higher The pedagogical model of the F2F courses was based on weekly
achievements and performance in online learning (Burkhardt, Kinnie, meetings in class, which were accompanied by a course website on
& Cournoyer, 2008; Connolly, MacArthur, Stansfiled, & McLesslan, Moodle learning management system, which mainly included the lec-
2007; Lim, Kim, Chen, & Ryder, 2008; Navarro & Shoemaker, tures presentations and additional materials related to the course sub-
2000). Moreover, similar results were found in studies, which exam- jects. Some courses included assignments that were mandatory but did
ined differences in final grades and satisfaction among students who not affect the final grade. The main communication channel between
participated in different methods (F2F or online) of the same course. the instructor and the students was in class.
Some found higher achievements among students who participated The online courses, on the other hand, were delivered fully online
in the online courses (Russell et al., 2016) whereas others found on the Moodle learning management system and were asynchronous.
no differences between the two methods (Abdous & Yoshimura, The pedagogical model was based on a consistent instructional course
2010; Zhan & Mei, 2013). Thus, there is no clear understanding model was designed and implemented in all three online courses,
concerning the use of learning outcomes as an effective measure consisting of four main elements:
when comparing online courses to F2F courses.
• Learning units (12 units), which covered the course contents and
consisted of the video lecture of the instructor, presentation of the

3 | T HE P R E S E N T S TU D Y lecture, a textual summary of the lecture, materials related to the


unit (e.g., articles and YouTube links), and assignments in some units.
Following the mixed results presented in the literature review regard- • Additional materials, which were related to the overall course
ing the outcome‐related effectiveness of online learning compared subjects.
with F2F learning, there is a need for a more comprehensive • Communication channels, which included instructor‐to‐student dis-
approach that examines a wider scope of variables for effectiveness
cussion forums and student‐to‐student discussion forums. The stu-
in courses. This study examined differences between online and
dents could also contact the course instructors directly via email.
F2F courses by utilizing a holistic perspective that includes data from
• General information about the course subject, its instructor, and
several sources: a student attitude survey, learner grades in the
guidelines regarding learning in an online course.
courses, and rate of completion of the course. The purpose of the
study was to examine students' perceived effectiveness of the course The online format enabled the implementation of a pedagogical
and their learning achievements in online courses compared with F2F approach in which all of the course content was made accessible to
courses. the students from the beginning of the semester, in order to provide
4 SOFFER AND NACHMIAS

greater flexibility in the learning process (Mullaney & Reich, 2015). 3.3 | Measures
However, in order to create a learning framework, four scheduled
This study included five variables that represented instructional
assignments were set along the semester with fixed deadlines, every
aspects: course structure, learning content, lessons watched/presence,
3 weeks. The assignments constituted 15% of the final course grade.
assignment assessments, and communication. In addition, it included
The students were required to submit all assignments on time in order
two variables that represented learning engagement and satisfaction,
to receive the full grade. However, submitting the assignments was not
and two variables that represented the objective measures of learning
a condition to complete the course. Table 1 presents prominent simi-
outcomes: final grades and completion rate. In total, nine groups of
larities and differences between the F2F and the online courses
variables were examined in this study.
formats.
All responses to the following measures were given on a Likert
The students' perceptions were examined via a survey, which was
scale ranging from 1 (not at all) to 5 (in a very high degree), unless oth-
conducted at the end of the semester. All students, who participated in
erwise specified. Table 3 presents the instructional, engagement, and
the F2F or in the online courses, received an email containing a link to
satisfaction variable items as they were presented to students.
the questionnaire, with an invitation to fill it out. It was emphasized
that the questionnaire was anonymous and that answers would be
used for research purposes only. The questionnaire was composed
3.3.1 | Instructional aspects
based on the literature and recent studies in the field (Soffer, Kahan,
& Livne, 2017; Sebastianelli et al., 2015; Young & Norgard, 2006). Course structure design
The questionnaire included items assessing background and Students were asked to indicate the degree of their comprehension of
sociodemographic variables, instructional aspects (e.g., comprehension the course structure with two items. A score was computed by averag-
of the course design), communication aspects, and engagement and ing the items (r = .53, p < .001), with higher scores indicating greater
satisfaction aspects. Alpha Cronbach reliabilities were computed only comprehension of the course structure design.
for variables with more than two items (e.g., assignment assessments),
and correlations were performed for variables with two items. Learning content contribution
Students indicated the degree to which the reading materials and sup-
plementary materials contributed to their understanding of the learn-
3.2.1 | Participants ing materials.
All of the students (N = 968) were enrolled in the courses as part of their
BA degree. Of the 614 students enrolled in the online courses, 240 Lessons watched/presence
responded to the questionnaire (39% response rate), and of the 354 Students in the frontal courses were asked to indicate the frequency of
students enrolled in the frontal courses, 120 participated in the study their class attendance, whereas students in the online courses were
(34% response rate). Frequencies and percentages of participants in asked to indicate the number of video lectures they watched. Both
each format for each of the courses are presented in Table 2. Overall, items were rated on a Likert scale ranging from 1 (watched/attended
the sample consisted of 53% women. Participants' mean age was all of them) to 5 (did not watch any/attend any).
24.42 years (SD = 4.11). Thirty percent of the sample were in their first
year of studies, 34% were in their second year, and 36% were in their Assignment assessments
third year or more. The students came from various disciplines, with Students indicated their level of agreement with seven items assessing
the majority from engineering (22%), exact science (19%), life science their attitudes towards the assignments in the course. Cronbach's α
(17%), social science (11%), arts (8%), and humanities (8%). No differ- was found to be high (α = .88). Therefore, a score for students' assign-
ences were found between the F2F and the online courses in any of ment assessments was computed by averaging the items, with higher
the background variables (age, gender, and academic school year). scores indicating more positive assessments. This score was not

TABLE 1 Prominent similarities and differences between the F2F and the online courses
Category/course format F2F Online
Course duration 12 weeks 12 weeks
Instructor The same lecturer The same lecturer
Scope and objectives The same The same
Final exam The same format, on campus The same format, on campus
Lectures Weekly F2F lectures in class Online lectures—recorded while the course was
delivered in the F2F format
Course contents delivery Presentations of the lectures, additional materials Presentations and summaries of the lectures, materials
related to the unit and additional materials related
to the overall course
Assignments Course A = 1; Course B = 3; Course C = 0 Four assignments along the course in all three courses
Communication with the instructor F2F in class and email Discussion forums and email
Communication between the students F2F in class Discussion forums
SOFFER AND NACHMIAS 5

TABLE 2 Frequencies (percentages) of participants in each format for each of the courses, according to participation in the study (N = 360) and the
in the course (N = 968)
Course A Course B Course C Total

F2F Participation in the study 30 (8) 34 (9) 56 (16) 120 (33)


Participation in the course 68 (7) 138(14) 148 (15) 354 (36)
Online Participation in the study 86 (24) 73 (20) 81 (23) 240 (67)
Participation in the course 152 (16) 212 (22) 250 (26) 614 (64)
Total Participation in the study 116 (32) 107 (30) 137 (38) 360 (100)
Participation in the course 220 (23) 350 (36) 398 (41) 968 (100)

Note. F2F = face‐to‐face.

TABLE 3 Instructional, satisfaction, and engagement variable items

Course structure The course structure and its requirements were clear to me.
There was a logical and clear learning flow in the learning units.
Reading materials To what degree did the reading materials contribute to your understanding of the learning materials?
Supplementary materials To what degree did the supplementary materials contribute to your understanding of the learning materials?
Lessons watched/Presence To what degree did you watch the course video lectures?/To what degree did you attend the course lessons?
Assignment assessments Combining the assignments in the course contributed to the learning flow.
The course assignments were incorporated in accordance with the course material.
The number of assignments was reasonable.
The level of the tasks suited the material.
The assignments were clear to me.
The feedback I received on my assignments helped me understand the learning materials.
The assignments were good preparation for the final test.
Communication I received responses to my questions.
During the course, collaborative learning occurred between me and other students from the course.
Engagement Motivation, involvement, interest and curiosity, independence, responsibility
Satisfaction The course contributed a lot to my understanding of the subject.
I was satisfied with the course.

computed for Course C because assignments were not given to stu- was filled out anonymously, students' grades could not be matched
dents in the F2F version. to their questionnaire's answers and were therefore analysed sepa-
rately. It should be noted that the final course grades were calculated
Communication based on the assignments grades (15%) and the final exam grade (85%).
Students indicated their level of agreement with one item assessing
their attitudes towards communication with the course's staff and 3.4 | Data analyses
one item assessing communication with other students.
Frequencies, means, standard deviations, and bivariate correlations
were performed using SPSS Statistics 22. Two‐way analysis of vari-
3.3.2 | Satisfaction aspects
ance (ANOVA) were performed to examine the study's questions
Participants indicated their level of agreement with two statements
regarding differences between online and F2F courses and to assess
assessing their satisfaction with the course. A satisfaction score was
the unique and combined effect of course type with course subject.
computed by averaging the items (r = .77, p < .001), with higher scores
In these analyses, three effects were reported: (a) a main effect of
representing higher satisfaction with the course.
course format (the difference between F2F and online courses); (b) a
main effect of course subject (the difference between the three
3.3.3 | Engagement aspects courses); and (c) a course format—course subject interaction effect
Students were asked to rate five different types of engagement‐ (the combined effect of course format and course subject on the
related senses invoked during their learning in the course. Perceived dependent variable).
engagement was computed by averaging the items (α = .83), with
higher scores representing greater perceived engagement. An engage-
4 | RESULTS
ment score was not computed for Course C because these items were
not included in the online version.
4.1 | Differences between F2F and online courses in
3.3.4 | Achievements instructional aspects
Students' final course grades and completion rate were used as mea- Table 4 presents means and standard deviations for each instructional
surements of achievement (N = 1,039). Because the questionnaire aspect according to course format and F statistics for the main effect
6 SOFFER AND NACHMIAS

TABLE 4 Means, standard deviations, and F statistics for the instructional aspects, according to course format and subject
F2F Online Total sample
M SD M SD M SD F(df)1 F(df)2 F(df)3

Course structure design (N = 354) 3.75 1.10 4.24 0.80 4.08 0.94 20.83(1, 348)*** 5.24(2, 348)** 2.13(2, 348)
Reading materials contribution (N = 248) 3.14 1.48 2.77 1.42 2.87 1.45 6.46(1, 242)* 23.70(2, 242)*** 2.32(2, 242)
Supplementary materials contribution (N = 194) 2.75 1.49 2.20 1.34 2.35 1.39 6.19(1, 188)* 0.40(2, 188) 0.90(2, 188)
Lessons watched/presence (N = 334) 3.93 1.14 4.23 1.06 4.11 1.09 10.79(1, 328)** 15.76(2, 328)*** 2.77(2, 328)
Assignment assessments 3.46 0.97 3.56 0.81 3.54 0.85 0.89(1, 197) 13.07(1, 197)*** 2.89(1, 197)
Communication with course staff (N = 282) 3.78 1.18 4.07 1.03 3.97 1.09 4.05(1, 276)* 13.59(2, 276)*** 7.66(2, 276)**
Communication with other students (N = 314) 1.81 1.21 2.10 1.40 2.01 1.35 3.34(1, 308) 2.38(2, 308) 0.80(2, 308)

Note. F2F = face‐to‐face. 1= F of the main effect of course format; 2 = F of the main effect of course subject; 3 = F of the course format × course subject
interaction effect.
*p < .05. **p < .01. ***p < .001.

of course format, the main effect of course subject, and the interaction lectures or attended class more (M = 4.47, SD = 0.84) than students
effect between course format and course subject. in Course A (M = 4.05, SD = 1.06; p = .01) and students in Course B
(M = 3.80, SD = 1.29; p < .001).
4.1.1 | Course structure
On average, students reported high comprehension of the course 4.1.4 | Assignment assessments
structure design. ANOVA indicated significant main effect of course Predominantly, students had moderate to high positive attitudes
format, such that regardless of the course subject, course structure towards the assignments. ANOVA revealed a significant effect only
comprehension was higher in the online courses compared with the of course subject, such that students in Course A assessed the assign-
F2F courses. In addition, significant main effect of course subject ments more positively (M = 3.71, SD = 0.79) than students in Course B
was found, such that overall, course structure comprehension was (M = 3.34, SD = 0.88).
lower in Course B (M = 3.86, SD = 0.87) compared with Courses A
(M = 4.18, SD = 0.83) and C (M = 4.16, SD = 1.04; p = .02).
4.1.5 | Communication
Primarily, students reported that they received responses to their ques-
4.1.2 | Content learning contribution
tions from the course staff at a moderate to high degree. ANOVA revealed
Overall, students assessed the reading materials as contributing in a low a main effect of course format, a main effect of course subject, and an
to moderate level to their understanding and assessed the supplemen- interaction effect. Tukey post hoc comparisons indicated that students
tary materials at an even lower level. ANOVA for the reading materials in the online course reported better communication with the course staff
as dependent variable revealed a significant effect of course format, such than students in the F2F course, but only in Course C (p < .001), whereas
that students in the F2F courses rated them as contributing more to their no differences were found between online and F2F formats in Courses A
understanding than students in the online courses. In addition, a signifi- (p = .62) and B (p = .78). The interaction effect is depicted in Figure 1.
cant main effect of course subject was found. Tukey post hoc compari- In regard to communication with fellow classmates, students
sons indicated that overall (across course formats), students in Course reported relatively low levels of collaborative learning during the
A perceived the reading materials as contributing to their understanding course. ANOVA revealed no difference in the degree of collaborative
more (M = 3.61, SD = 1.21) than students in Course B (M = 2.20,
SD = 1.32) and students in Course C (M = 2.50, SD = 1.42; p < .001).
Furthermore, similar results were found for the supplementary
materials as dependent variable. Specifically, results revealed that
students in the F2F courses rated the supplementary materials as
contributing more to their understanding than students in the online
courses. No significant main effect of course subject or interaction
effect was found.

4.1.3 | Lessons watched/presence


In general, students reported watching the video lectures or attending
classes to a high degree. ANOVA revealed a significant effect of course
format, such that students in the online courses watched the video lec-
tures more than students attended the F2F courses. In addition, a sig- FIGURE 1 Communication with the course staff, according to course
nificant main effect of course subject was found. Tukey post hoc format and course subject. F2F = face‐to‐face [Colour figure can be
comparisons indicated that students in Course C watched the video viewed at wileyonlinelibrary.com]
SOFFER AND NACHMIAS 7

TABLE 5 Means, standard deviations, and F statistics of engagement and satisfaction according to course format and course subject
F2F Online Total sample
M SD M SD M SD F(df)1 F(df)2 F(df)3

Engagement (N = 190) 2.98 0.95 3.30 0.93 3.22 0.95 5.59(1, 186)* 15.22(1, 186)*** 0.09(1, 186)
Satisfaction (N = 314) 3.66 1.33 3.97 0.99 3.88 1.10 5.89(1, 303)* 7.56(2, 303)** 1.61(2, 303)

Note. F2F = face‐to‐face. Note: 1= F of the main effect of course format; 2 = F of the main effect of course subject; 3 = F of the course format × course
subject interaction effect.
*p < .05. **p < .01. ***p < .001.

learning between the online and F2F courses, between the course sub-
jects, or an interaction effect between course format and course
subject.

4.2 | Differences between F2F and online courses in


engagement and satisfaction aspects
Students rated their engagement and satisfaction from the courses as
relatively moderate‐high. Table 5 presents means, standard devia-
tions, and F statistics of engagement with and satisfaction from the
course, according to course format and course subject. As can be
seen, students in the online courses rated both their engagement FIGURE 2 Grades in the course, according to course type and course
and satisfaction higher than students in the F2F courses. In addition, subject. F2F = face‐to‐face [Colour figure can be viewed at
wileyonlinelibrary.com]
differences between the course subjects were found in both engage-
ment and satisfaction. Specifically, students in Course A reported
higher engagement (M = 3.48, SD = 0.87) than students in Course
TABLE 6 Frequencies (%) of students who finished and did not finish
B (M = 2.92, SD = 0.95). In addition, Tukey post hoc comparisons
their courses, according to course format (N = 968)
indicated that students in Course B reported significantly less satis-
Course type
faction (M = 3.54, SD = 1.16) than students in both Courses A
(M = 4.11, SD = 0.89) and C (M = 3.96, SD = 1.17; p = .001, F2F Online Total

p = .01, respectively). Finished Yes 309 (87) 538 (87) 847 (87)
No 45 (13) 76 (13) 121 (13)

Note. F2F = face‐to‐face.


4.3 | Differences between online and F2F courses in
students' achievements no dependency was found between finishing the course and the for-
4.3.1 | Grades mat of the course, χ2(1) = 0.02, p = .88.

Overall, students' grades in the courses were moderate‐high (grade


range across course formats and subjects was 68.98–90.08), and mean 5 | DISCUSSION AND CONCLUSIONS
grade score was 80.09 (SD = 14.80). ANOVA revealed a significant dif-
ference between the online and F2F courses in students' grades, F(1, This study aimed to examine differences between traditional F2F
841) = 22.56, p < .001; a significant difference between the course courses and online courses (designed and developed based on previ-
subjects, F(2, 841) = 48.77, p < .001; and an interaction effect between ous research recommendations), using students' achievements and
course format and subject, F(2, 841) = 19.72, p < .001. Post hoc com- perceived effectiveness.
parisons indicated that grades were higher in the online courses com- Nine groups of variables were used to examine the effectiveness
pared with the F2F courses, but this difference was mainly significant of online courses compared with F2F courses; of them, seven were
in Course C (p < .001) and marginally significant in Course A subjective measures (perceptions and attitudes) and two were objec-
(p = .05). In Course B, there were no significant differences between tive measures. The findings indicated that there were significant differ-
grades in the online and F2F courses (p = .32). Figure 2 presents the ences between online and F2F courses in seven variables relating to
interaction effect. instructional aspects: course structure, reading materials, complemen-
tary materials, lessons watch/presence, communication with course
4.3.2 | Completion rate staff, engagement and satisfaction, and learning outcomes (final
Frequencies and percentages of students who finished the courses, grades, completion rate). A lack of significant differences was found
according to the course format, are presented in Table 6. As can be in only three variables: assignment assessments, communication with
seen, no differences were found in completion rate regarding the other students, and completion rate. Specifically, students in the online
course format. In both the F2F and online formats, most of the stu- courses reported better understanding of the course structure, and
dents (87%) finished their courses, whereas only 13% did not. Indeed, watching the video lectures more than students in the F2F courses
8 SOFFER AND NACHMIAS

reported attending class. In addition, students in the online courses results suggest that although differences between online and F2F
reported better communication with the course staff compared with courses exist, sometimes they may be derived by the course subject
students in the F2F courses. Interestingly, students in the online and/or its unique characteristics. For example, the difference in com-
courses reported higher engagement and satisfaction compared with munication with the course staff in this case can be attributed to the
students in the F2F courses. These findings are consistent with previ- speed at which the lecturer and teaching assistant responded. Like-
ous studies indicating the beneficial potential of online courses com- wise, the difference in grades in Course C (and not in A and B) can
pared with F2F courses (Lao & Gonzales, 2005; Swan et al., 2000; be explained by the fact that the frontal format of Course C was the
Toven‐Lindsey et al., 2015). only one in which assignments were not given to students during the
Moreover, these benefits are also reflected in a more objective course. Assignments, as an interactive activity, may better motivate
measurement—students' grades, which were higher, in general, in the students and engage them with course content as well as better pre-
online courses compared to the F2F courses. These findings support pare them to get higher grades on the final exam (Chen et al., 2010;
those of previous studies, which found higher outcomes in online Lim et al., 2007). Therefore, the difference that was found may also
courses compared with F2F courses (Burkhardt et al., 2008; Connolly be attributed to course structure (with/without assignments) and not
et al., 2007; Lim et al., 2008). However, it should be noted that the only to format.
findings of this study are not in line with previous studies that indi- In summary, this study provides essential information regarding
cated no differences between F2F and online courses (Lim et al., the effectiveness of online academic courses compared with F2F
2007; Neuhauser, 2010; Russell, 1999), or even greater benefits for courses, based on a designed, instructional course model. According
the F2F courses regarding learning outcomes, engagement, and satis- to the findings of this study, in many of the examined effectiveness
faction (Summers, Waigandt, & Whittaker, 2005). aspects, online courses are as effective as or more effective than F2F
The null effect of course format in communication with other stu- courses. These findings support Nguyen's (2015) study, which claimed
dents may actually be another indication of the success of online that online learning is “at least as effective as” the F2F courses and
courses. Specifically, because good communication is harder to achieve suggested to move beyond the assumption that there are “no signifi-
in online courses than in F2F courses due to the absence of any phys- cant differences”. One possible explanation for these effects is that this
ical meetings, the lack of difference in communication with other stu- study tested online courses that were designed and developed based
dents (and better communication with the course staff) suggests that on research recommendations. Mainly, these courses were developed
online courses do not fail in comparison with F2F courses even when with the purpose of maximizing flexibility by enabling student access
considering their main limitation. Thus, this study supports previous to all the course content from the beginning of the semester (Mullaney
studies that recommend the provision of various means to facilitate & Reich, 2015). In addition, in order to create a consistent learning
communication, such as discussion forums and emails (Wallace, framework (Swan et al., 2000; Young & Norgard, 2006), units were
2004), as was applied in the study's examined online courses. This taught according to a defined structure along with scheduled assign-
way, students' perceived communication in online courses can be at ments. Moreover, the structure consisted of recommended communi-
least as good as it is in F2F courses. The same rational is applicable cation channels (Wallace, 2004) and provided general information and
for the null effect of course format in completion rate. Although com- guidelines about different aspects of the course (e.g., materials and
pletion rate is predicted to be lower in online courses than in F2F instructor). However, it is important to note that these findings reflect
courses (Dutton, Dutton, & Perry, 2001; Terry, 2001), this prediction certain types of courses in a special program for undergraduate stu-
does not hold in this study. In both course formats, the completion rate dents. Thus, in order to generalize the results for other types of
was relatively high (87%). Finally, the null effect of course format in courses, additional research of other contexts is needed.
assignment assessments suggests that although students in online
courses spend more time learning and performing tasks compared with
students in F2F courses (Maki & Maki, 2007; Robertson et al., 2005), 6 | L I M I TA T I O N S A N D F U T U R E R E S E A R CH
they do not necessarily perceive it to be more positive.
On the other hand, students in the F2F courses reported that the Several possible limitations should be noted in this study. First, despite
reading and supplementary materials contributed to their understand- the differences that were found between online and F2F courses, con-
ing more than students in the online courses. These findings may be clusions regarding causality should not be derived. That is, because
explained by the possibility that students in the online and F2F courses participants were not randomly assigned to course format, the differ-
rely on different learning tools. For example, it is possible that students ences that were found can be explained by other things such as the dif-
in the online courses mainly use the video lectures in their learning and ferences between students in various characteristics. For example,
therefore see the reading and supplementary materials as contributing students who chose online courses may have had better technological
less. Furthermore, students in the F2F courses attend classes less orientation and looked for flexible courses, and thus, online courses fit
often and therefore may rely more on reading and supplementary them better. In addition, differences in the instructors' characteristics
materials, which they rate as contributing more to their understanding. may have biased the results (e.g., personality, technological skills, and
It is important to indicate that these differences were independent teaching strategy). Nevertheless, it should be noted that the courses
of the course subject, except for the difference between online and that were selected for this study (F2F vs. online) were similar in regard
F2F courses in communication with course staff and grades, which to most of their characteristics (e.g., the same instructor, their final
were found only in Course C and not in Courses A and B. Thus, these exam content and place, and duration). Moreover, sociodemographic
SOFFER AND NACHMIAS 9

variables were not found to be related to the outcome variables; thus, Du, J., Havard, B., & Li, H. (2005). Dynamic online discussion: Task‐oriented
the possibility that they can explain the differences that were found interaction for deep learning. Educational Media International, 42(3),
207–218.
can be ruled out.
Duderstadt, J., Atkins, D., & van Houweling, D. (2002). Higher education in
Because the questionnaire was anonymous, we could not corre-
the digital age. Ann Arbor, MI: University of Michigan Press.
late between students' responses and their achievements (in the final
Dutton, J., Dutton, M., & Perry, J. (2001). Do online students perform as
exam and in the assignments), this could be explored in future well as lecture students? Journal of Engineering Education, 90(1), 131–
research. Notwithstanding, future research should examine a larger 136.
sample of courses from diverse disciplines and a heterogeneous stu- Farinella, J. A., Hobbs, B. K., & Weeks, H. S. (2000). Distance delivery: The
dent population. Moreover, further examination of the role of the faculty perspective. Financial Practice and Education, 10, 184–194.
instructional model in relation to students' engagement and satisfac- Garbett, C. (2011). Activity based costing models for delivering online
tion would enable a better understanding of the factors that influence courses. Formamente Rivista Internazionale di Ricerca Sul Futuro Digitale.
effectiveness of online academic courses and assist in their future Gaytan, J., & McEwen, B. C. (2007). Effective online instructional and
development. Students' satisfaction could further be explored by inte- assessment strategies. The American Journal of Distance Education,
21(3), 117–132.
grating qualitative study (e.g., interviews) to better understand the rea-
Hannafin, M., Hill, J. R., Oliver, K., Glazer, E., & Sharma, P. (2003). Cognitive
sons behind the positive or negative satisfaction of the online learners
and learning factors in web‐based distance learning environments.
as compared with the F2F learners. Handbook of Distance Education, 2, 245–260.
Jones, K. O., & Kelley, C. A. (2003). Teaching marketing via the Internet:
ORCID Lessons learned and challenges to be met. Marketing Education Review,
13(1), 81–89.
Tal Soffer http://orcid.org/0000-0003-3339-0394
Kim, K., & Bonk, C. J. (2006). The future of online teaching and learning in
higher education: The survey says. Educause Quarterly, 29(4), 22.
RE FE R ENC E S
Abdous, M. H., & Yoshimura, M. (2010). Learner outcomes and satisfaction: Kuo, Y. C., Walker, A. E., Belland, B. R., & Schroder, K. E. E. (2014). Interac-
A comparison of live video‐streamed instruction, satellite broadcast tion, Internet self‐efficacy, and self‐regulated learning as predictors of
instruction, and face‐to‐face instruction. Computers & Education, 55(2), student satisfaction in online education courses. The Internet and Higher
733–741. Education, 20, 35–50.
Lack, K. A. (2013). Current status of research on online learning in postsec-
Allen, I. E., & Seaman, J. (2014). Grade change. In Tracking online education in
ondary education. Ithaka S+ R, zuletzt geprüft am, 3. 2013
the United States. Babson Survey Research Group and Quahog
Research Group, LLC. Lam, P., Cheng, K. F., & McNaught, C., 2005. Asynchronous online discus-
sion: Empirical evidence on quantity and quality. In G. Richards & P.
Allen, I. E., & Seaman, J., 2015. Grade level: Tracking online education in the
Kommers (Eds.). ED‐MEDIA 2005 (pp. 3209–3215), Proceedings of
United States. Retrieved from: http://onlinelearningconsortium.org/
the 17th annual World Conference on Educational Multimedia, Hyper-
read/survey‐reports‐2014/
media & Telecommunications, Montreal, Canada, 27 June ‐ 2 July.
Baker, C. (2010). The impact of instructor immediacy and presence for Norfolk VA: Association for the Advancement of Computers in
online student affective learning, cognition, and motivation. Journal of Education.
Educators Online, 7(1), n1.
Lao, T., & Gonzales, C. (2005). Understanding online learning through a
Balkin, R. S., Buckner, D., Swartz, J., & Rao, S. (2005). Issues in classroom qualitative description of professors and students' experiences. Journal
management in an interactive distance education course. International of Technology and Teacher Education, 13(3), 459.
Journal of Instructional Media, 32(4), 363–372.
Lim, D. H., Morris, M. L., & Kupritz, V. W. (2007). Online vs. blended learn-
Bekele, T. A. (2010). Motivation and satisfaction in internet‐supported ing: Differences in instructional outcomes and learner satisfaction.
learning environments: A review. Educational Technology & Society, Journal of Asynchronous Learning Networks, 11(2), 27–42.
13(2), 116–127.
Lim, J., Kim, M., Chen, S. S., & Ryder, C. E. (2008). An empirical investigation
Bruckman, A. (2002). The future of e‐learning communities. Communica- of student achievement and satisfaction in different learning environ-
tions of the ACM, 45(4), 60–63. ments. [Electronic version]. Journal of Instructional Psychology, 35(2).
Burkhardt, J. M., Kinnie, J., & Cournoyer, C. M. (2008). Information literacy Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an
successes compared: Online vs. face to face. Journal of Library Adminis- “early warning system” for educators: A proof of concept. Computers
tration, 48(3–4), 379–389. and Education, 54(2), 588–599.
Chen, P. S. D., Lambert, A. D., & Guidry, K. R. (2010). Engaging online Maki, R. H., & Maki, W. S. (2007). Online courses. In Handbook of applied
learners: The impact of web‐based learning technology on college stu- cognition (Second ed.) (pp. 527–552).
dent engagement. Computers & Education, 54(4), 1222–1232.
Marks, R. B., Sibley, S. D., & Arbaugh, J. B. (2005). A structural equation
Connolly, T. M., MacArthur, E., Stansfiled, M., & McLesslan, E. (2007). A model of predictors for effective online learning. Journal of Management
quasi‐experimental study of three online learning courses in computing. Education, 29, 531–563.
Computers & Education, 49, 345–359.
Mullaney, T., & Reich, J., 2015. Staggered versus all‐at‐once content
Crawford‐Ferre, H. G., & Wiest, L. R. (2012). Effective online instruction in release in massive open online courses: Evaluating a natural experi-
higher education. Quarterly Review of Distance Education, 13(1), 11. ment. In Proceedings of the Second (2015) ACM Conference on Learning
@ Scale (pp. 185–194). ACM.
Dixson, M. D. (2012). Creating effective student engagement in online
courses: What do students find engaging? Journal of the Scholarship of Navarro, P., & Shoemaker, J. (2000). Performance and perceptions of dis-
Teaching and Learning, 10(2), 1–13. tance learners in cyberspace. American Journal of Distance Education,
14(2), 15–35.
Drago, W., Peltier, J., & Sorensen, D. (2002). Course content or the instruc-
tor: Which is more important in on‐line teaching? Management Research Neuhauser, C. (2010). Learning style and effectiveness of online and face‐
News, 25(6/7), 69–83. to‐face instruction. The American Journal of Distance Education..
10 SOFFER AND NACHMIAS

Nguyen, T. (2015). The effectiveness of online learning: Beyond no signifi- Sher, A. (2009). Assessing the relationship of student‐instructor and stu-
cant difference and future horizons. MERLOT Journal of Online Learning dent‐student interaction to student learning and satisfaction in web‐
and Teaching, 11(2), 309–319. based online learning environment. Journal of Interactive Online Learn-
Northrup, P. T. (2002). Online learners' preferences for interaction. Quar- ing, 8, 102–120.
terly Review of Distance Education, 3(2), 219–226. Soffer, T., Kahan, T., & Livne, E. (2017). E‐assessment of online academic
Osman, M. E. (2005). Students' reaction to WEBCT: Implications for courses via students' activities and perceptions. Studies in Educational
designing on‐line learning environments. International Journal of Instruc- Evaluation, 54, 83–93.
tional Media, 32(4), 353. Song, L., Singleton, E. S., Hill, J. R., & Koh, M. H. (2004). Improving online
Palmer, S., & Holt, D. (2010). Students' perceptions of the value of the ele- learning: Student perceptions of useful and challenging characteristics.
ments of an online learning environment: Looking back in moving The Internet and Higher Education, 7(1), 59–70.
forward. Interactive Learning Environments, 18, 135–151. Squires, D. (1999). Educational software for constructivist learning environ-
Pape, L., 2010. Blended Teaching & Learning. School Administrator, 67(4), ments: Subversive use and volatile design. Educational Technology,
16–21. 39(3), 48–54.

Parker, K., Lenhart, A., & Moore, K. (2011). The digital revolution and higher Summers, J. J., Waigandt, A., & Whittaker, T. A. (2005). A comparison
education: College presidents, public differ on value of online Learning. of student achievement and satisfaction in an online versus a tradi-
Pew Internet & American Life Project. tional face‐to‐face statistics class. Innovative Higher Education, 29(3),
233–250.
Peltier, J. W., Schibrowsky, J. A., & Drago, W. (2007). The interdependence of
the factors influencing the perceived quality of the online learning expe- Swan, K., Shea, P., Fredericksen, E., Pickett, A., Pelz, W., & Maher, G. (2000).
rience: A causal model. Journal of Marketing Education, 29, 140–153. Building knowledge building communities: Consistency, contact and
communication in the virtual classroom. Journal of Educational Comput-
Picciano, A. G. (2002). Beyond student perceptions: Issues of interaction,
ing Research, 23(4), 359–383.
presence, and performance in an online course. Journal of Asynchronous
Learning Networks, 6(1), 21–40. Tee, M. Y., & Karney, D. (2010). Sharing and cultivating tacit knowledge in
an online learning environment. International Journal of Computer‐Sup-
Pintrich, P. R., & Schunk, D. H. (2002). Motivation in education: Theory, research,
ported Collaborative Learning, 5(4), 385–413.
and applications (Second ed.). Columbus, OH: Merrill Prentice Hall.
Terry, N. (2001). Assessing enrollment and attrition rates for the online
Polloff, R. M., & Pratt, K. (2001). Lessons from the cyberspace classroom: The
MBA. THE Journal (Technological Horizons in Education), 28(7), 64.
realities of online teaching. San Francisco, CA: Jossey Bass.
Tichavsky, L. P., Hunt, A. N., Driscoll, A., & Jicha, K. (2015). “It's just nice
Robertson, J. S., Grant, M. M., & Jackson, L. (2005). Is online instruction
having a real teacher”: Student perceptions of online versus face‐to‐
perceived as effective as campus instruction by graduate students in
face instruction. International Journal for the Scholarship of Teaching
education? The Internet and Higher Education, 8(1), 73–86.
and Learning, 9(2), 2.
Roby, T., Ashe, S., Singh, N., & Clark, C. (2013). Shaping the online experi-
Toven‐Lindsey, B., Rhoads, R. A., & Lozano, J. B. (2015). Virtually unlimited
ence: How administrators can influence student and instructor
classrooms: Pedagogical practices in massive open online courses. The
perceptions through policy and practice. The Internet and Higher Educa-
Internet and Higher Education, 24, 1–12.
tion, 17, 29–37.
UDI Online Project (2010). Student perceptions of online/blended learning
Rodriguez, M. C., Ooms, A., & Montañez, M. (2008). Students' perceptions
(Technical Brief # 03). Storrs: University of Connecticut, Center on Post-
of online‐learning quality given comfort, motivation, satisfaction, and
secondary Education and Disability. Retrieved from http://www.udi.
experience. Journal of Interactive Online Learning, 7(2), 105–125.
uconn.edu
Russell, J.E., Van Horne, S., Ward, A.S., III, E.A.B., Sipola, M., Colombo, M.
Wallace, R. M. (2004). A framework for understanding teaching with the
and Rocheford, M.K., 2016. Large lecture transformation: Adopting evi-
Internet. American Education Research Journal, 41, 447–489.
dence‐based practices to increase student engagement and
performance in an introductory science course. Journal of Geoscience Wanstreet, C. E. (2006). Interaction in online learning environments: A
Education, 64(1), pp.37–51. review of the literature. The Quarterly Review of Distance Education,
7(4), 399–411.
Russell, T. L. (1999). The no significant difference phenomenon: A comparative
research annotated bibliography on technology for distance education: As Young, A., & Norgard, C. (2006). Assessing the quality of online courses
reported in 355 research reports, summaries and papers. Raleigh, NC: from the students' perspective. The Internet and Higher Education, 9(2),
North Carolina State University. 107–115.

Sargeant, J., Curran, V., Allen, M., Jarvis‐Selinger, S., & Ho, K. (2006). Facil- Zhan, Z., & Mei, H. (2013). Academic self‐concept and social presence in
itating interpersonal interaction and learning online: Linking theory and face‐to‐face and online learning: Perceptions and effects on students'
practice. Journal of Continuing Education in the Health Professions, 26(2), learning achievement and satisfaction across environments. Computers
128–136. & Education, 69, 131–138.

Sebastianelli, R., Swift, C., & Tamimi, N. (2015). Factors affecting per- Zhang, D., Zhou, L., Briggs, R. O., & Nunamaker, J. F. (2006). Instructional
ceived learning, satisfaction, and quality in the online MBA: A video in e‐learning: Assessing the impact of interactive video on learn-
structural equation modeling approach. Journal of Education for ing effectiveness. Information Management, 43(1), 15–27.
Business, 90(6), 296–305.
Shea, P., Fredericksen, E., Pickett, A., Pelz, W., & Swan, K. (2001). Measures
How to cite this article: Soffer T, Nachmias R. Effectiveness
of learning effectiveness in the SUNY Learning Network. Online Educa-
tion, 2, 31–54. of learning in online academic courses compared with face‐
Shea, P., Swan, K., Fredericksen, E., & Pickett, A. (2002). Student satisfac- to‐face courses in higher education. J Comput Assist Learn.
tion and reported learning in the SUNY learning network. Elements of 2018;1–10. https://doi.org/10.1111/jcal.12258
Quality Online Education, 3, 145–156.

You might also like