Does Technology Lower the Cost of Education without Reducing Quality? A Financial Modeling Approach

Does Technology Lower the Cost of Education without Reducing Quality?
A Financial Modeling Approach to Flipping the Classroom*
Allan Eberhart
McDonough School of Business
Georgetown University
Washington, DC 20057
(202) 687-4584
allan.eberhart@georgetown.edu
Website: https://bit.ly/33xqnEd
February 2021
*I wish to thank participants in workshops at Georgetown University and the University of Pennsylvania.
I am also grateful to Manuel S. González Canché, Tony Carnevale, Patrick Day, Carl Eberhart, Lisa
Fairchild, Joni Finney, Peter Garland, Rebecca Griffiths, Matt Hartley, Brooks Holtom, David Lefevre,
Mike Liberman, Teresa Mannix, Laura Perna, Lee Pinkowitz, Jason Priestly, Sharon Ravitch, and Bob
Zemsky for their comments. I am also indebted to Allen Ammerman, Valeria Bellagamba, Katherine
Gunter, Shelly Heinrich, Prashant Malaviya, Eddie Maloney, Dennis Quinn, Tom Stowell, Yianna
Vovides, Luke Weichbrod, and Brooke Wertan for their assistance with acquiring data. Finally, thanks to
Saumya Shrut for her assistance with data formatting.
Electronic copy available at: https://ssrn.com/abstract=3694205

Abstract
Colleges and universities are rapidly expanding their use of technology, often with the goal lowering costs
while improving educational quality (i.e., student learning outcomes). Yet, there is little reason to expect
technology will reduce costs if it is not used to transform the content covered during a traditional course’s
class time into asynchronous materials, such as recorded videos the students access online in a so-called
flipped course. Although there is evidence that flipped courses can match the quality of traditional
courses, the question of whether flipped courses reduce costs remains unanswered. I address this gap in
the literature by developing a model that compares the cost of a flipped course to its traditional version.
With this model and a database of 2,060 students enrolled in seven cohorts over a six-year period, I find
consistent evidence of a significant decrease (36% on average) in the cost of developing and delivering a
flipped course relative to its traditional version without a decrease in quality. I also show the
circumstances where a flipped course can be more expensive than its traditional version, and discuss how
the model can be a practical tool for deciding when it makes sense to invest in technology.

1
1. Introduction
Long before the COVID-19 pandemic caused educational institutions around the world to turn to
technology as a solution for instructional continuity, these same institutions were heavily investing in
technology, often with the goal of improving educational quality and lowering costs (McCandless,
2015).1 Yet, there is little reason to expect technology will reduce the cost of providing education if it is
not used to transform the content covered during a traditional course’s class time into asynchronous
materials, such as recorded videos the students access online in a so-called flipped course (Bowen and
McPherson, 2016). Unfortunately, there is no comprehensive evidence that flipping a course lowers costs,
and much of the empirical evidence suggests that, at best, flipped courses produce student learning
outcomes that are insignificantly different from traditional courses (Chingos et al., 2017, Deming et al.,
2015).
If technology does not improve quality or lower costs, then what is the point of using it?
Although technology may offer other benefits, such as improving access to education and providing more
flexibility with respect to where and when students learn, these benefits are ancillary compared to the
long-promised revolution in which technology empowers colleges and universities to provide students a
high-quality education at a more affordable cost (Christensen and Eyring, 2011).
Even if consistent evidence emerges demonstrating that technology increases educational quality,
using it to lower cost without sacrificing quality is arguably much more important. Just imagine how
much less ubiquitous automobiles would be if it cost, say, $500,000 to build an average car.2
Technological advancements have improved the quality of cars exponentially over the past century, but
they would be a niche product if technology had not also served to reduce their production costs. As
Bowen (2012) stated in a quote that remains relevant today:
1
I measure educational quality based on student learning outcomes.
2
My references to costs throughout this paper are to the cost of developing and delivering a
course (production costs). Technology may reduce these costs without reducing tuition because
universities think they must charge what the market will bear (e.g., to subsidize programs or services it
offers that lose money) or more generally because colleges and universities will inevitably increase their
non-technology-related costs to match their revenues, as Bowen’s (1980) Revenue Theory of Cost posits.
Nevertheless, as Bowen (2012) suggested, the market’s willingness to pay high tuition is ultimately
limited. When this day of reckoning occurs—and it already has for many colleges and universities, as
Zemsky and Shaman (2017) argued—schools that have not found ways to lower costs without
diminishing quality using technology will be more vulnerable to financial stress. An examination of the
relation between the (production) cost-reducing effects of technology and tuition changes, however, is
beyond the scope of this paper.

2
I continue to believe that the potential for online learning to help reduce costs without
adversely affecting educational outcomes is very real….Presidents and provosts should not
mince words in charging their deans and faculty with teaching courses of comparable quality
with fewer resources….We must recognize that if higher education does not begin to slow the
rate of increase in college costs, our nation’s higher education system will lose the public
support upon which it so heavily depends. (p. 34)
In short, my primary research question—Does technology lower the cost of education without
reducing quality?—should be of interest to researchers, administrators, and policy makers. The question
likely remains unanswered due to the challenge of comparing the uneven cost structure of a flipped
course, with its high upfront costs for the development of asynchronous materials followed by lower
ongoing costs, to the relatively constant cost per offering of the traditional version.
I fill this important gap in the literature with a model that provides a commensurable comparison
of the cost of a flipped course to its traditional version. I then apply the model and rigorous educational
quality metrics to a new database consisting of a total of 2,060 students enrolled in seven cohorts over a
six-year period. The evidence is consistent that the cost of developing and delivering a flipped course is
significantly lower than a traditional course (36% on average), without a reduction in quality.
However, I also show how easy it is for these technology-intensive courses to become more
expensive than their traditional versions. For example, as I discuss below, the development of a high-
quality flipped course can require a school to invest tens of thousands of dollars in asynchronous
materials’ production costs, and many more thousands of dollars to compensate faculty to prepare and
develop these materials. If these costs are not recovered through a reduced need for classroom space,
higher enrollments, and so on, then the cost of developing and delivering the flipped course can easily
exceed the cost of its traditional version, and the model provides a comprehensive and practical means of
computing these net effects.
If all the content covered during a traditional course’s class time is converted to asynchronous
materials in a so-called fully flipped course, then the cost per student decreases as enrollments increase
because there are no physical classroom space constraints and the school does not need to pay faculty for
live teaching, or may only pay them a nominal amount to hold office hours. Therefore, the extra cost of
adding another student to the class can be insignificant, but the studies I discuss in the next section report
that student learning outcomes are typically much lower in fully flipped courses compared to traditional
courses. Preserving some class time (synchronous sessions)—not just office hours, but classes where all
students are required to attend—with enrollment limits to ensure that students have sufficient

3
opportunities to participate appears to be critical to avoiding any decline in student learning in a flipped
course.3
So, the development and delivery of a high-quality flipped course may not only require an
extraordinary upfront investment in asynchronous materials development, but an ongoing commitment to
high quality live teaching time. Both facets require the active participation of faculty to succeed, and that
means the compensation must be enough to entice faculty to flip their courses (assuming it makes sense
from the school’s perspective to invest in flipping the course). I find that faculty can be paid more per
hour of synchronous session time they teach in a flipped course than its traditional version, and the school
can still save money because faculty are, for example, teaching fewer synchronous sessions in the flipped
course.
This paper is organized as follows. First, I continue with definitions of the various types of
courses. In Section 2, I briefly review the literature on the quality of flipped courses and their cost
savings. I explain the underlying approach to comparing the cost of a flipped course to its traditional
version in Section 3, and develop a model based on this approach. In Section 4, I discuss the data, and
present the empirical tests, test results, and limitations of the study. Finally, I summarize the study and
discuss the contributions to the literature in Section 5.
Taxonomy of Courses
An online course—also called an EdTech, e-learning, remote education, distance learning,
technology-intensive, and a virtual education course—is one in which the course materials are delivered
asynchronously or synchronously (e.g., through video conferencing) through the Internet. The course
deliverables (e.g., papers) and assessments (e.g., exams) are also submitted electronically.
A flipped course is an online course, but an online course is not flipped if there is no substitution
of asynchronous materials for required synchronous sessions.4 Moreover, a hybrid course—a flipped
course which preserves some required synchronous sessions—may hold synchronous sessions onsite or
online. Yet these distinctions are not always recognized in the literature. For example, a course may be
3
I use the phrase synchronous session interchangeably with the phrase required synchronous
session (i.e., they are a reference to class time, also called live teaching time, not to office hours).
4
Some definitions of a flipped course do not specify the substitution of asynchronous materials
for synchronous sessions (e.g., Swoboda and Feiler, 2016). As noted earlier, the substitution of
asynchronous materials for required synchronous sessions is the key to reducing costs. Therefore, I
specify this substitution in my definition of a flipped course (a definition also used in other studies that I
reference in the literature review).

4
referred to as online, but no information is provided to determine whether the course is also flipped, and
this lack of precision is important because the cost structures of these varying types of courses are
significantly different.
To clarify the different course definitions and how they are related to each other, I define five
categories of courses. This list is not meant to be exhaustive but to capture the relevant range of courses
discussed in the literature review.
Category 1: Traditional Face-to-Face Course
All classes in this non-flipped course are held in an onsite classroom. The course professor may
make use of technology through, for example, the display of PowerPoint slides and spreadsheets—that
may also be uploaded to a course site on a learning management system such as Canvas—but there is no
substitution of asynchronous materials for required synchronous sessions. This type of course is also
called a brick-and-mortar course, a traditional onsite course, and more succinctly a traditional course or
traditional version.
Category 2: Hybrid Onsite
In this course, asynchronous materials are used as a substitute for some portion of the required
synchronous sessions, and all required synchronous sessions are held onsite. The students are typically
directed to work through the materials to prepare for the synchronous sessions. The professor(s) who
developed the materials may be different from the professor who teaches the synchronous sessions. This
type of course is also referred to as a blended onsite course or a partly online course. Other more succinct
(and broader) synonyms include a flipped course, hybrid course, blended course, and an online course.
Category 3: Fully Flipped Course
As mentioned earlier, the asynchronous materials in this type of course are used as a substitute for
all required synchronous sessions. The professor, or the teaching assistants, may hold optional
synchronous sessions (e.g., office hours). This course is also called a fully online or a purely online
course. Other more succinct and broader synonyms include a flipped course and an online course.
The last two categories are variations of the first two categories:
Category 1a: Virtual Face-to-Face Course
Although this type of course has a traditional format, the synchronous sessions are delivered
online using a video conference software package such as Zoom instead of in an onsite classroom. Many
of the traditional courses that have been converted to an online course on an emergency basis during the
coronavirus crisis in 2020 are in this category because the instructors have not had time to develop the

5
asynchronous materials required to flip their courses (Gardner, 2020). These courses may also combine
onsite synchronous sessions with online synchronous sessions, and these are often referred to as hybrid
courses, but my subsequent references to hybrid courses will be for those that transform some, but not all,
synchronous session content into asynchronous materials. As with a fully flipped course, this course is
also called a fully online or a purely online course, but it is not flipped. Another more succinct (and
broader) synonym is an online course.
Category 2a: Hybrid Online
This type of course works just like a Hybrid Onsite course except that the required synchronous
sessions are held online. As with a fully flipped course and a virtual face-to-face course, this course is
also referred to as a fully online course, purely online course, or a blended online course. Other more
succinct (and broader) synonyms include a flipped course, hybrid course, blended course, and an online
course.
The flipped courses examined in Section 4’s empirical section are in Category 2a. Unless stated
otherwise, however, the courses discussed in the next section’s literature review are in categories 1, 2, or
3 (though the cost comparison model can be applied to any type of course).
To differentiate between the various types of courses more precisely, I define a variable called the
degree of flipping (δ); that is, the percentage of the traditional course version’s class time converted to
asynchronous materials that are expected to last N number of years before the materials need to be
redeveloped (e.g., because of new discoveries in the course’s subject area).
As an example of how δ works, consider a traditional three-credit semester course which has 24
(80-minute) classes taught over 12 weeks for a total of 32 hours of class time. If δ is equal to, say, 20%,
then 6.4 hours of class time are converted to asynchronous materials, and 25.6 hours are preserved for
synchronous sessions. Table 1 shows the division of class time converted to asynchronous materials and
time preserved for synchronous sessions for various levels of δ.
2. Brief Literature Review
Educational Quality of Fully Flipped Courses
Early evidence on the effectiveness of fully flipped courses was promising. For example, a meta-
analysis conducted by the Department of Education (Means et al., 2009) offered the following conclusion:
“Students who took all or part of their class online performed better, on average, than those taking the
same course through traditional face-to-face instruction” (p. xiv). However, this report was criticized by

6
Jaggars and Bailey (2010) for combining studies examining fully online courses—which could include
fully flipped courses—with studies analyzing hybrid onsite courses into a general online category.
In a follow-up report that acknowledged the Jaggars and Bailey (2010) criticisms, Means, et al.
(2010) offerred a modified conclusion: “Students in online conditions performed modestly better, on
average, than those learning the same materials through traditional face-to-face instruction” (p. xiv).
Notwithstanding this revision, the following summary from Jaggars and Bailey (2010) remains
applicable:
Perhaps one of the most interesting insights that emerged for us from reading the Department
of Education meta-analysis is the small number of reasonably rigorous studies in this area
despite the rapid growth of online education and the enthusiasm and high hopes associated
with it. (p. 9)
During the decade following the Means et al. (2010) meta-analysis and the Jaggars and Bailey
(2010) report, more rigorous evidence emerged showing that fully flipped courses provide educational
outcomes that are inferior to the traditional onsite course versions (e.g., Bettinger et al. (2017), Brown and
Liedholm (2002), Figlio, Rush, and Yin (2013), Xu and Jaggars (2014)). In their review of studies
analyzing online education, Protopsaltis and Baum (2019) made the following assessment:
[M]ore than a decade after Congress allowed online colleges full access to federal student
aid programs, and despite a subsequent explosion in their enrollment, a growing and
powerful body of evidence suggests that online learning is far from the hoped-for silver
bullet. (p. 1)
This assessment is primarily a reference to fully flipped courses. For example, Protopsaltis and
Baum (2019) stated “A review of the relevant evidence certainly confirms that interaction is essential for
ensuring quality and student success in online education” (p. 17). They further stated, “Instructor presence
is integral for achieving interpersonal interaction and activities that emulate those of a ‘real person.’
Personal interaction increases student satisfaction, and by extension, motivation to learn and succeed” (p.
18). As Banerjee and Duflo (2014) noted in their analysis of the well-publicized version of fully flipped
courses called Massive Open Online Courses (MOOCs), these courses place a student at a “disadvantage
with respect to a classroom participant: the need for self-discipline and focus without the benefit of a peer
group or a structured study time” (p. 515).5
5
Reich (2020) reviews the evidence showing that EdTech innovations, including those designed
to democratize education such as MOOCs, often exacerbate inequalities in educational outcomes.

7
Protopsaltis and Baum (2019) did acknowledge that fully flipped courses can work well for
excellent, disciplined students, taking courses in subjects that are more amenable to online education,
such as computer science. Indeed, Goel and Joyner (2016) provided favorable evidence on the learning
outcomes for students in the fully flipped version of Georgia Tech’s M.S. in Computer Science compared
to the students in the program’s traditional onsite format.
Although both versions of the program had high admission standards, the students in the fully
flipped format had somewhat weaker admission credentials than the traditional onsite students, but the
students in the fully flipped format generally performed better than their onsite counterparts on
assignments (test performance was similar) in the course that Goel and Joyner (2016) examined. The
course evaluation forms were also more favorable for the fully flipped course.
In short, Goel and Joyner’s work offers an intriguing example of how a fully flipped course can
provide learning outcomes that equal, and in many cases exceed, the outcomes achieved by students in the
traditional onsite version. Future research documenting a continuation of the program’s educational
quality, its cost structure, and more importantly the applicability of this format beyond the subject of
computer science—and beyond the select group of students with high aptitude and self-discipline—will
represent important contributions to the literature.
Educational Quality of Hybrid Flipped Courses
Many of the concerns raised regarding fully flipped courses—especially the lack of real-time
interactions between professor and student—are addressed by hybrid courses. Nevertheless, hybrid
courses by definition reduce the amount of time professors interact live with their students. The question
is whether the benefits of the flexibility provided by the asynchronous materials, and the ability of
students to review the materials multiple times (in contrast to the one-time opportunity they have with
traditional onsite lectures) are enough to offset the reduced live class time students spend with their
professors.
To answer this question, Bowen, Chingos, Lack, and Nygren (2014) analyzed a sample of 605
students randomly assigned to a hybrid or traditional version of an introductory statistics course offered at
six public universities.6 They analyzed the students’ performances on common course exams as well as
6
The synchronous sessions for the hybrid onsite version of the course do not appear to have been
required because Bowen et al. (2014) described them as supplemental in which “students could ask
questions or be given targeted assistance” (p. 97). So, this course may not have met my definition of a
hybrid course, but the regular synchronous sessions with the instructor, which may have extended beyond

8
the rate at which the students in each section completed and passed the course. The only significant
difference between the sections was the completion rate, which was five percentage points higher for the
students in the hybrid sections. Yet, based on an additional analysis of the students’ course evaluations,
the hybrid students were less satisfied on average than the traditional onsite students. The hybrid students
also thought they had learned less.
A subsequent study by Joyce, Crockett, Jaeger, Altindag, and O’Connell (2015) used an approach
similar to what Bowen et al. (2014) employed but arrived at a different conclusion. Based on the
performance of the students on the common midterm and final exam, Joyce et al. (2015) concluded that
“…reduced class time appears to diminish cognitive performance for most students in a large introductory
economics class at a public university in which the vast majority of students commute” (p. 65).
Alpert, Couch, and Oskar (2016) also used a randomized approach to investigate the learning
outcomes of a flipped course—both a fully flipped version and a hybrid version—compared to its
traditional version. They reported that the students in the fully flipped course learned significantly less
than the students in the traditional onsite course but the learning outcomes for the hybrid-course students
were insignificantly different from those of the traditional-course students.
More recent work by Chingos, Griffiths, Mulhern, and Spies (2017) avoided the limitation of
focusing on one course by studying 1,604 students enrolled in seven courses across five different subject
areas: biology, communications, computer science, precalculus, and statistics. Although the students were
not randomly assigned to the courses, the descriptive statistics in their study showed virtually no
significant differences in the student characteristics between the hybrid and traditional sections.
The primary results revealed no significant difference in the average pass rate or final grade of the
students enrolled in the hybrid sections compared to the traditional sections. Moreover, little evidence was
found that disadvantaged students (e.g., students with low prior GPAs, minority students) performed
worse in the hybrid sections. Nevertheless, as with the Bowen et al. (2014) study, the hybrid students’
overall course satisfaction was significantly lower on average than for the traditional course students. The
amount the students thought they learned was also significantly lower for the hybrid students than for the
traditional students.
In short, much of the extant empirical evidence suggests that hybrid courses can match the
educational outcomes achieved in traditional courses, but there is not consistent evidence that flipped
typical office hours, also did not fit neatly into my definition of a fully flipped course. With this caveat
noted, I continue to refer to this online course as a flipped or hybrid (onsite) course.

9
courses improve educational quality. Even if subsequent research shows strong evidence of technology
improving the quality of education the question of whether technology lowers cost without decreasing
quality is the more critical question as noted earlier.
The Cost of Flipped Courses
Early work by Bowen et al. (2014), Hodges (2010), and the NCAT (A Summary of NCAT
Program Outcomes, n.d.) showed that technology reduces teaching compensation costs compared to a
traditional course. However, none of these studies accounted for the large upfront production costs of
developing the asynchronous materials or the cost of compensating faculty for developing the materials.
Perna and Ruiz (2016) did not investigate the cost components of a flipped course, but they
recognized that there are many cost considerations beyond the reduction in faculty time spent teaching
synchronous sessions that results from flipping a course. The variety of costs associated with developing
and delivering online courses caused them to express doubts about the cost-reducing effects of
technology: “The implications of technology for higher education costs are ambiguous” (p. 449).
Indeed, Griffiths, Chingos, Mulhern, and Spies (2014) reported that faculty spent an average of
144 hours creating a hybrid course using asynchronous materials that had already been developed. One
professor who developed the asynchronous materials from scratch reported spending 470 hours building
the course (Goel and Joyner (2016) reported that they spent 1,000-plus hours developing their flipped
course). This considerable investment of time and money may be repaid with many repeated uses of the
course but Griffiths et al. (2014) did not provide information on the average life of the asynchronous
materials, nor did they compare all of the costs of a flipped course to its traditional version.
Bowen (2013) discussed the wide variety of challenges facing the use of technology to reduce
costs such as intellectual property (i.e., faculty compensation for use of the asynchronous materials they
developed), the reluctance of faculty to use asynchronous materials developed by other faculty, and the
likely notable differences among subject areas in the difficulty of converting class time content to an
asynchronous format. These concerns may be why a recent survey revealed that most faculty “doubt that
online education can lower the cost of education without decreasing quality” (Jaschik and Lederman,
2017, p. 23).
In another study, Hoxby (2018) examined the cost savings from technology using the Integrated
Postsecondary Education Data System (IPEDS) database. She reported that fully online schools had the
lowest annual teaching compensation costs followed by partly online schools (offering some combination
of fully online and traditional courses) and by schools that focus on traditional courses. Nevertheless,

10
fully online and partly online schools spent relatively more on administrative services and legal and fiscal
operations, for example, than traditional onsite schools.
In his review of Hoxby’s paper, Gordon (2018) noted the judgment calls Hoxby, or any other
researcher, has to make in using the IPEDS data to classify schools as fully online, partly online, or
primarily traditional courses. Straumsheim (2014) also reviewed questions about the highly aggregated
nature of the IPEDS data. Most importantly, the aggregated characteristic of the IPEDS data creates two
notable problems.
First, as noted in the course taxonomy discussed earlier, the fully online designation includes
fully flipped courses with no synchronous sessions and virtual face-to-face courses that consist entirely of
synchronous sessions. These two types of courses have significantly different cost structures, with the
fully flipped course scaling nearly infinitely whereas the virtual face-to-face course cannot scale more
than the traditional onsite course unless quality is not a concern (e.g., by removing section enrollment
caps notwithstanding pedagogical concerns (Urquiola and Verhoogen (2009)). Even the hybrid online
course, the third type of fully online course, cannot achieve economies of scale the way a fully flipped
course does.
Second, the question of whether technology can lower the cost of education without reducing
educational quality cannot be answered with the IPEDS data using the quality metrics I employ in this
study. Indeed, in a related study, Deming, Goldin, Katz, and Yuchtman (2015) used IPEDS data to
investigate the relation between the proportion of online students a college or university serves and its
tuition. They found that schools with a higher share of online students tended to charge lower tuition but
then quickly added, “[T]he impact of online technology on the quality of education remains uncertain.
Thus, one needs to be cautious before concluding that lower costs and prices in online programs will raise
the productivity of US higher education.” (Deming et al., 2015, p. 497).
3. Cost Comparison Model
To help explain the concept behind the cost comparison model, consider the following simple
numerical example. Suppose that it costs a school $30,000 a year to deliver a traditional course, where the
primary expense is probably the professor’s compensation.
Now consider an opportunity for the school to invest $45,000 upfront in asynchronous materials
to flip half the course, so that δ is 50%. With half as many synchronous sessions, suppose that it will only
cost the school $15,000 a year to deliver the flipped course at no lower quality than its traditional version
over the next five years before the materials must be updated. If the discount rate is, say, 10%, then it

11
makes sense to flip the course because the present value of the cost savings ($56,862) is greater than the
upfront investment ($45,000) in asynchronous materials.
However, many variables go into computing the cost of each course version and the cost
components can differ. For example, flipping half the course may not halve the annual cost of the flipped
course, compared to its traditional version, because the faculty compensation for each hour of
synchronous session time may be higher for the flipped course than for its traditional version, as the
empirical tests described in Section 4 reveal. Accounting for these complexities is more tractable with a
direct comparison of the cost of each course version, but the flipped-course costs are front-loaded because
of the asynchronous materials development. After the materials have been developed, the flipped-course
annual costs can decline significantly, possibly well below the relatively constant cost of the traditional
version.
The Equivalent Annual Cost Approach (EAC) to Comparing Costs
To overcome the problem of comparing the highly uneven cost of a flipped course to the
relatively constant cost per offering of a traditional version, I develop a model based on the Equivalent
Annual Cost (EAC) method (Brealey, Myers, and Allen, 2017). This technique can be used to express all
the costs of a flipped course on an annualized basis to facilitate a commensurable comparison with its
traditional version.7
The EAC approach is related to the cost-effectiveness and cost-benefit analysis approaches
discussed in Levin et al. (2017). These approaches fit in the general category of discounted cash flow
(DCF) analysis. The model I develop applies this foundational analysis to the specific question of whether
flipping a course reduces costs.
To make the course costs comparable, the EAC approach first requires future costs to be
expressed in today’s dollars. Given the number of years the asynchronous materials can be used before
needing updating (N), the present value (PV) of these flipped and traditional course costs (C0 for upfront
costs in year 0, C1 for costs in year 1, etc.) is
7
The EAC approach implicitly assumes that the revenue produced by the alternative forms of
production (the flipped course versus its traditional version in this study) is equivalent. If there is a
difference in revenue because, for example, of a difference in the tuition per student net of scholarships,
then this difference should be accounted for in the model as an additional “cost” associated with the lower
revenue per student program (i.e., this cost can be captured in the other cost category I discuss below in
more detail).

12
𝑁
𝐶𝑛
𝑃𝑉 = 𝐶0 + ∑ (1)
(1 + 𝑟)𝑛
𝑛=1
8
where r = discount rate.
If the traditional-course version’s annual costs are forecasted over the same number of years as
the flipped course, as in in the example above, then a comparison of the present value of the cost of these
courses is commensurable. However, an annualized cost provides a more intuitive comparison because
budgets are typically expressed on an annual basis. This annualization also standardizes the costs of
flipped courses that differ in the number of years their asynchronous materials last until the materials
must be updated. For example, the materials for a biology course may have to be updated every year or
two because of important new research findings, whereas the materials for an introductory statistics
course may last much longer.
A course’s EAC is computed by multiplying the PV computation derived in equation (1) by the
1
EAC conversion factor ( 1 1 1 𝑁), i.e., a factor that expresses a present value in terms of a
[( )−( )∗( ) ]
𝑟 𝑟 (1+𝑟)
constant stream of payments made at the end of each of the N periods, as shown in equation (2):
𝑃𝑉
𝐸𝐴𝐶 = 𝑁
(2)
1 1 1
[( ) − ( ) ∗ ( ) ]
[ 𝑟 𝑟 (1 + 𝑟) ]
With r = 10% and N = 5, then the EAC of the flipped course’s costs of $45,000 upfront followed
by $15,000 in subsequent annual costs is $26,871. Because the annual $30,000 cost of the traditional
course version is fixed in this example, the EAC equals $30,000 by definition. Therefore, the flipped
course is $3,129 cheaper per year.9
The Model
The simple numerical examples above illustrate the classic tradeoff a flipped course offers
relative to its traditional version: an upfront investment in asynchronous materials in exchange for lower
8
This framework can allow for the asynchronous materials to be updated partially over time (there
is no discrete, lumpy redevelopment of the materials in the future). In this case, N is the number of years
until all the original asynchronous materials have been fully updated.
9
If the annual cost of the traditional version is not constant—e.g., because the faculty
compensation costs may be expected to increase—then the same EAC method I show above for the
flipped course version applies (i.e., compute the present value of the annual costs and then the EAC).

13
future costs resulting from the reduction in the amount of required synchronous session time. To measure
whether this tradeoff reduces costs, I build a model centered on the EAC technique and the degree of
flipping variable (δ) as noted earlier.
Because educational quality is exogenous to the model, and educational quality is an important
component of educational output, the cost comparison model developed in this study is not derived from a
production function. Instead, the cost components of a flipped course and its traditional version are
exogenously specified. The primary contribution of the model is to consolidate and standardize these
components to ensure a commensurable comparison of the courses’ costs. The model also accounts for
the important effect of discontinuities in cost components caused by section enrollment caps, and for
differences in the cost components of a flipped course and its traditional version
To employ δ as the cornerstone of the model, I express the cost components on an hourly basis.
This hourly standardization allows for a seamless conversion of cost per hour to total cost for any given
level of δ. Although the model does not account for the educational quality effects of changing δ, it does
provide a consistent means of comparing the cost of courses with different degrees of flipping. I develop
the model in detail in Eberhart (2021) and focus on a succinct version of the model and the concept
behind it in the following sections.
The total number of class/lecture hours in a traditional course is referred to as HT, where each
hour is denoted by the variable ht. Each hour of class time in a traditional course converted to
asynchronous materials in a flipped course is designated by the variable ha, where the total number of
hours of traditional class time converted to asynchronous materials is denoted HA. The total number of
hours preserved for synchronous sessions in each flipped course section is a variable termed HS, where
each hour is denoted as hs.
The Cost of a Flipped Course
A flipped course costs include three primary components:
A. cost of developing and maintaining the asynchronous materials (i.e., production, faculty
compensation, and maintenance over the life of the asynchronous materials),
B. cost of delivering the synchronous sessions (i.e., faculty compensation and opportunity rental
cost of on-campus room for the synchronous sessions across all of the sections over the life of the
asynchronous materials),
C. other costs (i.e., catch-all category including costs of student support services, exam proctoring,
etc. across all of the sections over the life of the asynchronous materials).

14
Because all the cost components are expressed on an hourly basis (ht, ha, hs) in today’s dollars, I
multiply each cost component by the respective cumulative hours—HT, HT*δ (which equals HA), HS
(which equals (1-δ)*HA)—to arrive at a total cost in today’s dollars (PVF):
𝑃𝑉𝐹 = 𝐻𝑇 ∗ [𝛿 ∗ α𝐹 + (1 − 𝛿) ∗ ς𝐹 + ω𝐹 ] (3)
where
HT*δ*αF = asynchronous materials development and maintenance costs (where αF is the cost per
ha),
HT*(1-δ)*ςF = synchronous session delivery costs (where ςF is the cost per hs),
HT*ωF = other costs (where ωF is the cost per ht).
I then express the equivalent annual cost of PVF (EACF) using the EAC conversion factor
mentioned previously:
𝑁𝐹 −1
1 1 1
𝐸𝐴𝐶𝐹 = 𝑃𝑉𝐹 ∗ [( ) − ( ) ( ) ] (4)
𝑟𝐹 𝑟𝐹 (1 + 𝑟𝐹 )
These costs are then standardized on a per-student basis (EACSF) to account for the fact that spending, for
example, $50,000 on a course with 10 students is clearly more expensive than spending the same amount
on a course with 50 students.10 The per-student standardization also accounts for the scaling effect of
technology.
𝐸𝐴𝐶𝐹
𝐸𝐴𝐶𝑆𝐹 = (5)
𝑆𝐹
where
SF = total student enrollment in flipped course over the life of the asynchronous materials.
The Cost of the Traditional Course Version

The traditional course version costs consist of two primary components, expressed in today’s
dollars, as shown in the following equation:
𝑃𝑉𝑇 = 𝐻𝑇 ∗ [ς 𝑇 + ω 𝑇 ] (6)
where
HT*ςT = synchronous session delivery costs (where ςT is the cost per ht),
HT*ωT = other costs (where ωT is the cost per ht).
10
It is not necessary to standardize by student credit hours since the total credit hours for the
flipped course are set to be the same as for its traditional course version in my sample.

15
I then express the equivalent annual cost of PVT (EACT) using the EAC conversion factor:
𝑁𝑇 −1
1 1 1
𝐸𝐴𝐶𝑇 = 𝑃𝑉𝑇 ∗ [( ) − ( ) ( ) ] (7)
𝑟𝑇 𝑟𝑇 (1 + 𝑟𝑇 )
and finally standardize these costs on a per-student basis (EACST):
𝐸𝐴𝐶𝑇
𝐸𝐴𝐶𝑆𝑇 = (8)
𝑆𝑇
Cost Comparison
To examine the difference in costs, I first consider the case where the hourly cost components—
as well as N and r—are the same for a flipped course as they are for its traditional version. That is, all the
variables with a T subscript equal the variables with an F subscript (i.e., ςF=ςT, ωF=ωT, SF = ST, NF = NT).
So, all the subscripts can be removed, including αF, since there are no asynchronous material costs for the
traditional course. In this case, the standardization by the number of students and the EAC conversion
factor is the same for both courses, and EACSF and EACST can be simplified as follows:
𝐸𝐴𝐶𝑆𝐹 = 𝐻𝑇 ∗ [𝛿 ∗ α + (1 − 𝛿) ∗ ς + ω] ∗ 𝑍 (9)
𝐸𝐴𝐶𝑆𝑇 = 𝐻𝑇 ∗ [ς + ω] ∗ 𝑍 (10)
where
−1
1 1 1 𝑁
𝑍 = [𝑆 ∗ ((𝑟) − (𝑟) ((1+𝑟)) )] (11)
and the dollar difference is designated by the variable Δ$:

𝛥$ = 𝐸𝐴𝐶𝑆𝐹 − 𝐸𝐴𝐶𝑆𝑇 = [𝐻𝑇 ∗ 𝛿 ∗ [α − ς]] ∗ 𝑍 (12)
Equation (12) shows that technology reduces the cost of education (reduces Δ$) when the classic
tradeoff mentioned previously—the investment in asynchronous materials and maintenance in exchange
for lower future costs due to the reduction in synchronous session time—is beneficial. In other words,
𝐻𝑇 ∗ 𝛿 ∗ ς > 𝐻𝑇 ∗ 𝛿 ∗ α (13)
where
𝐻𝑇 ∗ 𝛿 ∗ ς = present value of the cost savings from the reduction in synchronous session time across
all of the sections over the life of the asynchronous materials (i.e., 𝐻𝑇 ∗ 𝛿 is the number
of hours of class time covered by the asynchronous materials in the flipped course and ς

16
is the present value of the hourly cost of “saved” synchronous session time; $56,862 in
the numerical example discussed above),
𝐻𝑇 ∗ 𝛿 ∗ α = upfront investment in asynchronous materials development and maintenance ($45,000

in the numerical example discussed above).
If the variables with a T subscript differ from the variables with an F subscript, then I add a prime
superscript to the Δ and the EACS variables. In this case the dollar difference (∆′ $) is more involved, as
shown in the following equation:
−1
1 1 1 𝑁𝐹
∆′ $ = 𝐻𝑇 ∗ [𝛿 ∗ α𝐹 + (1 − 𝛿) ∗ ς𝐹 + ω 𝑇 ] ∗ [𝑆𝐹 ∗ ((𝑟 ) − (𝑟 ) ((1+𝑟 )) )]
𝐹 𝐹 𝐹
−1
𝑁𝑇
1 1 1
[ς ]
−𝐻𝑇 ∗ 𝑇 + ω 𝑇 ∗ [𝑆𝑇 ∗ (( ) − ( ) ( ) )] (14)
𝑟𝑇 𝑟𝑇 (1 + 𝑟𝑇 )
where the first term on the right-hand side of the equation represents 𝐸𝐴𝐶𝑆𝐹′ and the second term
represents 𝐸𝐴𝐶𝑆𝑇′ .
As an example of how (14) captures nuances that (12) misses, suppose the number of students
enrolled in the flipped course is greater than its traditional version because it is more flexible or does not
require students to attend synchronous sessions in person, but all the other variables with an F subscript
equal their respective variables with a T subscript. In this case, technology can reduce the cost of
education (i.e., reduce ∆′ $) even if the present value of the future cost savings does not exceed the
investment in the asynchronous materials development and maintenance. With SF > ST, the larger number
of students in the flipped course exploits the infinitely scalable asynchronous materials and makes the
cost per student lower than it would be if SF = ST.
To illustrate, consider a revised version of the previous example with the flipped course requiring
a $45,000 upfront investment in asynchronous materials development. Instead of $15,000 in subsequent
annual costs for the flipped course’s synchronous sections over the next five years, suppose the annual
synchronous section costs are $25,000. With r =10%, the EAC is $36,871, notably more than the $30,000
annual cost of the traditional course version. If the number of students in the flipped course is the same as
for the traditional course version, then the flipped course is more expensive.
Now, suppose there are, say, 200 students enrolled in the flipped course over the life of the
asynchronous materials versus 100 students in the traditional course. In this case, the flipped course is

17
$116 less expensive per student each year than its traditional version. That is, the EACS = $184 (i.e.,
$36,871/200) for the flipped course compared to $300 for the traditional course (i.e., $30,000/100). This
example highlights the importance of accounting for the possibility of expanded enrollments in flipped
courses. With more students, the scaling effect of the flipped course’s scalable asynchronous materials are
exploited more fully and can cause a significant reduction in Δ′ $ (though this scaling cannot be done at
the expense of quality for the flipped course to be considered an improvement in education
productivity).11
Because the economic significance of the dollar difference in costs depends on the dollar level of
the costs—e.g., a $100-dollar difference is economically more significant if, say, EACS′F = $200 than if
EACS′F = $2,000—I compute the percentage difference in costs as defined by the variable Δʹ:
𝛥′ = ln(𝐸𝐴𝐶𝑆𝐹 ′) − ln(𝐸𝐴𝐶𝑆𝑇 ′) (15)
4. Data and Empirical Tests
Principal Sample
I gathered data from a flipped (hybrid online) graduate business program launched at the
beginning of 2014 by a private, nonprofit research university (School X). I examined two samples from
this program for the educational quality tests: the principal sample and the supplemental sample.
The principal sample contains the distribution data required to infer individual student responses
and performances for the educational quality metrics. The data are from the first course in the program
because there is a comparable traditional version of the course offered in another graduate business
program at the same university for which distribution data were also available. As the first course, it also
has the longest time series of available data.
The principal sample consists of 820 students, 567 of whom were enrolled in the flipped course
and 253 in its traditional version. Although the flipped course was a 3-credit course and its traditional
version a 1.5-credit course offered in a different program, the flipped course covered the same topics as
the traditional version and was taught—i.e., the asynchronous materials development and the delivery of
the synchronous sessions—by the same professor at the same school. The traditional-course cost
11
As I discuss in more detail below, the relation between Δ′ and the number of students in the
flipped course is complicated because, for example, of the section enrollment caps.

18
components were expressed in terms of a 3-credit course to enable a comparison to the flipped course.12,13
The flipped-course students were enrolled in seven cohorts between 2014 and 2019 (38 in the first cohort,
55 in the second, 78 in the third, 72 in the fourth, 84 in the fifth, 113 in the sixth, and 127 in the seventh),
while the students in the traditional-course version were enrolled in one cohort in 2011.14
The flipped course’s δ was 72% (i.e., 9 hours of synchronous session time compared to 32 hours
for the traditional version), as was the case for all the flipped courses in this program. The courses lasted
six weeks, with a mandatory 90-minute synchronous session each week that usually included an
application of the lessons learned from the asynchronous materials in a case-discussion format.15 Each
course concluded in the seventh week with a final exam or project. The traditional onsite course was also
six-weeks long, with a final exam in the seventh week.
The admissions profiles of the flipped-course students in all seven cohorts were similar to the
traditional course version, as shown in Table 2. The students had virtually identical average
undergraduate grade point averages (3.28 for the flipped course students vs. 3.31 for the traditional course
students) and age (29 vs. 28). Moreover, the average number of years of work experience (6.96 vs. 5.59),
percentage of students who were female (25% vs. 27%), and percentage of students who were minorities
(34% vs. 37%) were very similar.
The only notable differences were the students’ average GMAT (665 in the flipped course and
686 in the traditional course), the percentage who had majored in business as undergraduates (28% vs.
33%), and the percentage of international students (11% vs. 29%).
12
Because it involved twice as many credits as the traditional-course version, the flipped course
covered the topics in greater depth, and the deliverables and exams were calibrated to account for this
greater depth, so the course grades are comparable between the flipped course and the traditional-course
version.
13
The flipped course had two other professors who also developed some of the asynchronous
materials and taught some synchronous sessions, but the lead professor, who wrote and graded the exams
and assigned the course grades for the flipped course, was the one who taught the traditional-course
version. Moreover, the subjective educational-quality measures (i.e., the students’ evaluation of the
instructor in measures S8-S12) were for this lead professor.
14
Students in the flipped program were given the option to attend synchronous sessions in person
beginning with the third cohort, but I still classify the course as hybrid online because the students always
had the option to attend synchronous sessions online, and the vast majority of them did so.
15
The students in the flipped program also had to complete at least two 1.5-credit elective
(flipped) courses to qualify for graduation. These courses had the same format as the 3-credit courses but
had a duration of three weeks, with a fourth week set aside for the completion of a final exam or project.

19
In short, while the flipped course and its traditional version were not perfectly comparable and the
students were not randomly assigned to either course, both courses were taught in graduate business
programs at the same school, the student characteristics were similar, and the professor who taught the
traditional course version also developed the flipped-course’s asynchronous materials and taught its
synchronous sessions. Moreover, the higher GMAT and proportion of undergraduate business majors of
the students in the traditional program, two characteristics that differ between the programs, would
suggest that the flipped-course students would not perform as well as the traditional-course students.
Subjective Educational Quality Measures
Student course-evaluation surveys are a typical “subjective” measure of education quality
employed in past studies. Listed below are 12 evaluation questions in four categories which were
answered by the students. The questions are similar to those used by Bowen, Chingos, Lack, and Nygren
(2014) and Chingos, Griffiths, Mulhern, and Spies (2017).16 The questions were typically answered on a
five-point Likert scale, with five the highest rating and one the lowest.17 The students answered the
questions as part of an anonymous survey administered during the last week of the course, and the
professor did not see the responses until he or she had submitted the course grades.
Overall Course Evaluation
S1. Stated course objectives met?
S2. Effectiveness of asynchronous material?
S3. Amount learned in course?
Grading
S4. Exams, etc. represented course content?
S5. Exams, etc. were graded fairly?
Student Engagement
S6. Average weekly hours spent on course?
S7. Frequency of class (synchronous session) attendance?
16
There were slight differences in the wording of some questions over time. I show the exact
wording of each question for each cohort in Eberhart (2021).
17
The following questions were answered on a four-point scale with 4 representing the highest
score: Questions S1 and S2 for Flipped Cohort 3, questions S1-S5 and S8-S11 for Flipped Cohorts 3-5,
question S6 for Flipped Cohorts 1-2 and the traditional-course version. I conducted the empirical tests
below with these four-point scale questions converted to a five-point scale by adding one to each response
value. As a robustness check, I conducted the tests with these observations removed and found
qualitatively similar results.

20
(Synchronous) Instructor Evaluation

S8. Instructor well-prepared for class (synchronous session)?
S9. Contribution of classes (synchronous sessions) to learning?
S10. Helpfulness of instructor outside class (synchronous sessions)?
S11. Instructor challenged/encouraged students?
S12. Overall evaluation of instructor?
The dataset provided by School X shows the number of responses to each question, the average
response value, and the number of responses for each point on the Likert scale (the summary distribution).
For example, there may have been 30 responses to a question with an average response value of 3.73 with
ten students providing an answer of 5, ten others answering 4, five others answering 3, two answering 2,
and three answering 1. This summary distribution was used to recreate individual response data by
manually entering the data into an Excel spreadsheet. With this example, ten 5s, ten 4s, five 3s, two 2s,
and three 1s are typed into an Excel column containing 30 rows (one for each response). The accuracy of
the manual data entry was ensured by comparing the average of the manually entered data to the average
value reported for each question in the original database.
Converting the distribution data to individual student data allowed for the computation of the
standard deviation of the responses—and therefore the statistical significance of the differences in means
between the flipped course and its traditional version for each variable—and to compute the regression
tests discussed below in more detail.
Objective Educational Quality Measures
I applied three “objective” measures of education quality commonly used in the literature (Bowen
et al., 2014; Chingos et al., 2017):
O1. Completion Rate: Number of students who completed the course divided by the number
originally enrolled.
O2. Pass Rate: Number of students who earned a passing grade divided by the number who completed
the course.
O3. Course Grade: Average grade point average of students who completed the course.
This study’s database includes the number of students who completed the course, their average
grade—where A = 4.0, A- = 3.67, B+ = 3.33, B = 3.0, B- = 2.67, C = 2.0, and F = 0—the summary
distribution of letter grades (e.g., number of As, etc.), and the number of withdrawals in each course and
cohort. Following the procedure described for the subjective quality measures, individual student data

21
were recreated using the summary distribution with a dummy variable created for the completion rates
(i.e., the variable = 1 if the student completed the course, 0 otherwise) and the pass rates (i.e., the variable
= 1 if the student passed the course, 0 otherwise).
Educational Quality Test Results
I conducted a two-tailed t-test of the difference in the flipped course’s mean value for the 12
subjective quality measure questions and the three objective measures versus the mean value of its
traditional course version. I used the Satterthwaite (1946) approximation of the standard errors to account
for any difference in the samples’ standard deviations.
The test results are displayed in Table 3. In Panel A, the average response to each of the three
course evaluation questions is significantly higher for the flipped course than for its traditional version at
the one percent level. Moreover, the differences are economically significant, with the flipped course’s
average response ranging from 7% higher for the S1 measure (4.59 vs. 4.28) to 11% higher for the S3
variable (4.62 vs. 4.17).
The course grading questions S4 and S5 are shown in Panel B. The difference in the average
response for S4 is insignificant, while the average response to question S5 is significantly higher at the
one percent level for the flipped course, although the economic significance of the difference is less
notable (a 4% difference of 4.68 vs. 4.5).
In Panel C, the average responses to the student-engagement questions are presented. The
flipped-course students’ average response to question S6, regarding the students’ level of effort, is 23%
higher than for those in the traditional course (4.24 vs. 3.45). This difference is significant at the one
percent level. The average response to question S7 on the students’ class (synchronous session)
attendance is significantly higher for the traditional-course students (4.89 vs. 4.98). This difference is
statistically significant at the one percent level, but the economic significance is marginal (-2%). In short,
while the flipped-course students fell slightly short of the nearly perfect class attendance reported by the
traditional-course students, they reported exerting significantly more effort.
The average responses to questions S8 through S12 are shown in Panel D. Four of the five
differences are significant at the one percent level, with the most notable differences for S9 (4.84 vs.
4.35), S11 (4.84 vs. 4.52), and S12 (4.85 vs. 4.55). The flipped-course students clearly felt that the
synchronous sessions added more to their learning than the traditional-course students did about their
class sessions. The flipped-course students also thought that the instructor challenged/encouraged them

22
more than did the traditional course students about their instructor, even though the same instructor taught
both groups of students, and the flipped-course students’ overall evaluation of the instructor is higher.
Panel E displays the results of the three objective course measures, with the differences all
significant at the one-percent level. While the average grade is significantly higher for the flipped-course
students than for the traditional-course students (3.39 vs. 3.29), the completion rate (97.5% vs. 99.6%)
and pass rate (98.1% vs. 100%) are significantly lower, but the economic significance of these differences
is minimal. By comparison, the course completion rates for the course examined by Goel and Joyner
(2016) was approximately 87%, while Jaggars and Bailey (2010) cited online course completion rates as
low as 65%.
In summary, the results reveal that the flipped course was of significantly higher quality in 10 of
the 15 tests, with three of the results indicating that the traditional-course version possessed significantly
higher quality, and two showing insignificant differences. Overall, the quality of the flipped course was at
least no lower than its traditional version.
The difference in means tests pooled the flipped course’s seven cohorts offered between 2014 and
2019 into one sample. This pooling may mask differences in the average value of each quality measure
over time (i.e., across cohorts). For example, Zumeta, Breneman, Callan, and Finney (2012) reported that
flipping courses improved student learning outcomes. However, the improvements were not maintained in
subsequent course offerings, perhaps because the initial enthusiasm over the conversion had waned.
To examine the time-series properties of the quality measures, I estimated a simple linear trend
model shown in the following equation:
𝑄𝑢𝑎𝑙𝑖𝑡𝑦𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑗𝑖 = 𝜓0 + 𝜓1 𝐶𝑜ℎ𝑜𝑟𝑡𝑖 + 𝜖𝑖 (16)
where
𝑄𝑢𝑎𝑙𝑖𝑡𝑦𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑗𝑖 = subjective (S1 through S12) or objective (O1 through O3) educational
quality measure j for student i (i.e., the subjective measure student i
assigned to the course or the objective quality measure for student i),
𝐶𝑜ℎ𝑜𝑟𝑡𝑖 = cohort number (1 through 7) for student i.
Table 4 reveals little evidence of a notable trend in the measures over time (across cohorts),
though there is some evidence of a trend in quality measures S5, O2 and O3, and the insignificance of the
trend coefficient for the other variables may be attributable to the small sample size of the time series

23
(n=7). To account for the possibility of a change in the average value of a variable across time/cohorts, I
estimated the following fixed effects regression:
7
𝑄𝑢𝑎𝑙𝑖𝑡𝑦𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑗𝑖 = 𝛼 + ∑ 𝐶𝑡 + 𝛽𝑖 𝑌𝑖 + 𝜖𝑖 (17)
𝑡=2
where
𝐶𝑡 = dummy variable equal to 1 if student i’s cohort is cohort t (e.g., cohort 2), 0 otherwise,
Yi = dummy variable equal to 1 if student is in flipped course, 0 otherwise.
The fixed effects dummy variables capture any changes in the average value of a quality measure over
time.
The results for the subjective educational quality measures are reported in Panels A through C of
Table 5 and four of the quality measure test results (S3, S9, S11 and S12) have a positive estimate of β
significant at the 10% level or higher. In other words, the flipped course had a higher quality than its
traditional version for these quality measures. There is one significantly negative estimate of β (S6), and
the other seven regression results reveal an insignificant β estimate.
As for the objective educational quality measures, the results show a significantly negative
estimate for O2 (completion rates). In short, while the regression tests provide fewer results showing an
improvement in quality for the flipped course, there are also only two results of a higher quality for the
traditional course version. As with the difference in means test results, the overall evidence from the
regression tests suggests that the flipped course was no lower in educational quality than its traditional
version, and the flipped course may well have had higher overall quality.
Educational Quality Results for Supplemental Sample
The supplemental sample consists of 1,240 students enrolled in four other flipped (hybrid online)
courses and the traditional onsite versions of these courses (the flipped courses and their traditional
versions are from the same programs as the primary sample). I refer to this sample as supplemental
because summary distribution data for the subjective course-evaluation metrics or the objective course
metrics were not provided. Only average values were available for one subjective metric (the average
overall evaluation of the course instructor, measure S12, as discussed in the previous section) and one
objective metric (the students’ average grade, the O3 measure used in the previous section).
The educational-quality metrics for each course and its traditional version are reported in Table 6.
Panel A shows an average grade of 3.46 in the one section of flipped course #1 offered in 2017 with an
enrollment of 69 students. The three traditional onsite course versions have an average enrollment of 34,

24
fewer than half that of the flipped course, and yet receive a nearly identical average grade of 3.47. The
average overall evaluation of the instructor is higher for the flipped course (4.24) than its traditional
version (4.01). In other words, even though the professor—the professor who taught the flipped course
was the same professor who taught the traditional course version in three out of four cases (a total of five
professors, beyond the professor who taught the courses in the primary sample, taught these courses)—
had far fewer opportunities to interact with the students in person, the students still rated the professor
higher for the flipped course.
The quality metrics for flipped course #2 are displayed in Panel B, with an average grade of 3.74
and average overall evaluation of the instructor of 4.72, both much higher than the average values for the
traditional version (3.52 and 4.09 respectively). The enrollment of 74 students in the flipped course is also
higher than for any of the traditional sections.
Panel C shows yet another example of the average overall evaluation of the instructor being
higher for the flipped course (5.00) than for its traditional version (4.46). The average grade of 3.36 for
the flipped course is slightly lower than for the traditional version (3.49), although this can be interpreted
as demonstrating that the professor did not curry favor with the flipped students by assigning higher
grades. The enrollment in the flipped course (68) is also more than 150% greater than the average
enrollment of 27 in the traditional courses.
Panel D provides the one example in which the flipped course has a lower enrollment (38),
average grade (3.25), and average overall evaluation of the instructor (4.55) than its traditional version
(48, 3.30, and 4.73). Across all four flipped courses and their traditional versions, as shown in Panel E,
the average flipped course grade of 3.45 is nearly identical to the average grade for the traditional course
(3.44), while the average overall evaluation of the instructor is higher for the flipped course (4.63 versus
4.32 for the traditional-course version).
Without the distribution data required to infer individual student responses, I cannot conduct tests
of the statistical significance of the difference between the quality measures for these flipped courses and
their traditional versions. Nevertheless, these results augment the principal sample results in showing that
the flipped courses were on balance no lower in quality, and in many cases higher, than their traditional
versions. In the next section, I examine whether this quality can be provided at a lower cost.
Estimation of Δˈ
The limited data I have for traditional courses—e.g., only one year of educational quality data for
the traditional course version in the principal sample—were sufficient for the educational quality tests I

25
reported in the previous section. However, to compute Δˈ, I required a complete time series of cost data
for the traditional course version to compare to the cost data I had for the flipped course in the principal
sample (i.e., data covering seven cohorts over six-and-a-half years).
To address the problem of computing Δˈ, I constructed a benchmark traditional version of the
principal sample course based on School X’s compensation policy for teaching traditional courses, with
the assumption that the section enrollment caps (κ) and thresholds for the creation of a new section (τ)
each year were the same for the traditional course as for the flipped course.18 In other words, the
traditional course was constructed as an exact traditional version of the flipped course (e.g., the first
course in a startup traditional program launched in 2014).
To gauge the significance of Δˈ, I computed it for the 16 variations of the four variables where
there is uncertainty (student enrollments and cost of capital) or where there is an option (faculty
compensation and rental cost of flipped course’s synchronous sessions):
1. Student enrollments: SF = ST or SF > ST. The relative number of enrollments in the flipped
course—i.e., sTn each year and ST in total—could vary from the same enrollments as the flipped course or
25.4% of the enrollments in the flipped course, where the 25.4% ratio was based on relative number of
students enrolled in School X’s flipped program who were local residents and therefore could have
attended a traditional version of the program without moving (the flipped program was targeted primarily
to part-time students who were working full-time).
2. Cost of Capital: θ = 1 or θ = 0 (i.e., the cost of capital is estimated using the Capital Asset
Pricing Model (θ = 1) or the yield to maturity (YTM) on the university’s bonds (θ = 0)).
3. Faculty Compensation: η = 1 or η = 0. That is, the faculty may have been compensated on load
(η = 1) or off load (η = 0) for the flipped course and its traditional version.19 As I discuss below in more
detail, compensating faculty on load for teaching synchronous sessions, whether flipped or traditional,
was costlier for School X. Therefore, to avoid biasing the cost comparison, if the professor was
compensated on load or off load for the flipped course, then he or she was assumed to be compensated on
the same basis for the traditional course version.20
18
Additional details on all of the data used in the empirical tests are provided in Eberhart (2021).
19
On-load teaching counts toward the fulfillment of a full-time professor’s annual teaching
obligation. The compensation for off-load teaching is paid in cash to adjunct professors or to full-time
professors who wish to do additional teaching.
20
In the actual principal sample cost data, the professor who taught both the flipped course and its
traditional-course version was compensated on load for teaching the course in some years and off load in

26
4. Rental Cost for Flipped Course’s Synchronous Sessions: where χ = 1 when the flipped-course’s
synchronous sessions were held in a room on campus, zero when held in the professor’s home or campus
office (χ = 0). This variable was also assumed to be constant across the cohorts.
Panel A in Table 7 displays the various outcomes for the 16 combinations of input variables. In
every case, the flipped course is less expensive than the traditional version, but there are four cases where
Δ’ is the closest to zero:
Case 1 (SF = ST, θ = 1, η = 0, χ > 0): Δ’ = -6.0%
Case 2 (SF = ST, θ = 0, η = 0, χ > 0): Δ’ = -7.7%
Case 3 (SF = ST, θ = 1, η = 0, χ = 0): Δ’ = -7.4%
Case 4 (SF = ST, θ = 0, η = 0, χ = 0): Δ’ = -9.1%
These four cases reveal two consistent patterns where the flipped course offers the lowest
reduction in cost; that is, when the greater flexibility associated with flipping did not cause the enrollment
to be higher than for its traditional version (SF = ST) and when the professor was paid off load (η = 0). The
former case is important because the flipped course scaled better than its traditional version. So, the
opportunities to exploit these economies of scale were limited when there was no greater enrollment in
the flipped course than in its traditional version.
As for the professor’s compensation, the cost per hour of synchronous session teaching was $971
when the professor was teaching the flipped course off load.21 This cost was inferred from School X’s
compensation policy of 1.5 teaching credits per section for a flipped course times the school’s
compensation rate of $5,827 per credit when the professor was teaching a course, flipped or traditional,
off load, divided by the course’s nine hours of synchronous session time per section (the flipped course
was a 3-credit course for the students but faculty were paid 1.5 credits per section).
When a professor was teaching the traditional course off load, the faculty compensation cost per
hour of synchronous session time of $546 was inferred from School X’s traditional course compensation
policy of three teaching credits per section times the off-load rate of $5,827 per credit divided by the
others and there was no variation in the course quality measures over time, as noted earlier. The professor
was compensated on load for the traditional course version. Therefore, to make a commensurable
comparison of the cost consequences of on-load versus off-load compensation, the form of compensation
was assumed to be the same for all seven cohorts.
21
All dollar amounts referenced in this section are in 2013 dollars when the investment in the
flipped course’s asynchronous materials was made (as noted earlier, the course/program was launched in
2014).

27
course’s 32 hours of synchronous session time per section. So, the school was paying a 78% “premium”
($971/$546) per hour of synchronous session time for the professor teaching the flipped course, but the
flipped course still generated annual savings of $8,733 per section (32*$546–9*$971) in faculty
compensation for teaching synchronous sessions because the flipped-course professor was only teaching
28% of synchronous session time that the traditional-course professor was teaching.22
When the professor was compensated on load, the faculty compensation cost per teaching credit
of $11,500 was computed based on the professor’s salary; the percentage of his or her workload assigned
to teaching responsibilities, rather than research and service responsibilities; and his or her annual
teaching load. Therefore, the cost per hour of synchronous session teaching time for each section of a
flipped course was $1,917, or 1.5 teaching credits times $11,500 per credit divided by the nine
synchronous session hours. The cost per hour for the traditional course was three teaching credits times
$11,500 per credit divided by the 32 synchronous session hours, or $1,078. As with the off-load
compensation, the school was still paying a 78% premium ($1,917 /$1,078) per hour of synchronous
session time for the professor teaching the flipped course. However, the flipped course’s annual savings
in synchronous session teaching compensation per section was $17,243 (32*$1,078–9*$1,917), much
higher than the savings per section of $8,733 when the professor was compensated off load.
The flipped courses’ greater cost savings when the professor was compensated on load is
attributable to the combination of the higher dollar cost per hour of synchronous session time ($1,078) in
the traditional course, compared to the off load rate of $546, and the full 32 hours of synchronous session
time more than offsetting the combination of an even higher dollar cost per hour of synchronous session
time ($1,917) in the flipped course—compared to off load rate of $971—and the only nine hours of
synchronous session time.
I explore in more detail the importance of the relative number of enrolled students and on-load vs.
off-load teaching compensation in the next section. In the meantime, I used the 16 estimates of Δʹ shown
in Panel A to compute the variable’s average value and the statistical significance of this average. As
noted in Panel B, the average is -35.99%, which is significantly different from zero at the one-percent
level.
I place “premium” in quotes because the flipped course professor had to teach more students
22
across two sections to earn the same amount of teaching credits as the traditional course professor did for
one section.

28
Breakeven Analyses
The results in the previous sections are consistent in indicating that flipped courses were
significantly less expensive to produce with no reduction in quality. Indeed, the bulk of the evidence
suggests that quality was higher for flipped courses. To explore the conditions under which the flipped
course can be more expensive, I conducted a breakeven analysis for each of the following input
variables:23,24
αF1 = Production costs for development of asynchronous materials (e.g., videos),
𝛼𝐹2 = Faculty compensation for development of asynchronous materials (per hour of class time in
the traditional-course version converted to asynchronous materials in the flipped course (ha)),
ST/SF = ratio of students enrolled in the traditional onsite version relative to the flipped course,
𝛼𝐹𝑎 = annual asynchronous materials development and maintenance costs as a percentage of αF1 per
ha ,
I focused on αF1 and αF2 because they are decision variables administrators and faculty may be
able to calibrate to minimize the cost of the flipped course without affecting quality. The variable 𝛼𝐹𝑎
may be partially controllable and potentially important in determining Δ΄. Finally, I have already shown
the importance of ST/SF in the computation of Δ΄.
The breakeven point for each variable is the level that causes Δ΄ = 0, while holding the other
variables at their base levels. So, any “worse” level of the input, such as higher production costs for the
asynchronous materials, will cause the flipped course to be more expensive than its traditional version (Δ΄
> 0).
I set the base case parameter estimates as identical to the parameter estimates used in the previous
section, with the following important differences.
1. The ratio of ST/SF was set equal to the long-term average ratio of 25.4%, as estimated by the
percentage of students enrolled in the flipped program who were local (as discussed earlier).
2. The number of students in each cohort in the flipped program was set equal to a variety of
steady-state enrollments (i.e., constant enrollments each year over the seven cohorts) ranging from 89 to
170, with a focus on when a new section is created for the flipped course. These steady-state enrollments
23
Levin et al. (2017) emphasized the importance of evaluating the attractiveness of an educational
investment decision under a variety of input estimates.
24
The year subscript n is removed because the variables are constant over time as discussed below
in more detail.

29
are within the range of the last cohort examined in the previous section, with an enrollment of 127, and
they move beyond the startup nature of the course/program. In other words, a school may consider
launching a flipped course that is more expensive than its traditional onsite version during a startup phase
with the expectation that it will be less expensive long term in a steady state. I also examined a couple of
larger enrollment cases where the number of students in the flipped course varies from 409 to 410 to
reveal the cost consequences of multiple sections for the traditional-course version. That is, with the ratio
of ST/SF equal to 25.4%, there must be hundreds of students enrolled in the flipped course to cause the
traditional-course’s enrollment to be sufficiently high to trigger the creation of an additional section.
3. The section enrollment cap (κ) and threshold trigger for the creation of a new section (τ) were
set equal to the steady-state levels at the university (School X) that provided the data used in this study
(κT = 65, κF =40, τ F = τ T =10 for all n).25 These variables were determined by a combination of classroom
space constraints and, more importantly, pedagogical concerns raised by faculty about a significant
reduction in quality when enrollments in each section become “too large” (i.e., 75 or larger for the
traditional onsite class and 50 or larger for a synchronous session in the flipped course).
As a precursor to the breakeven analysis, I show the levels of Δ΄ across the various steady-state
enrollments in Table 8. The cost savings of the flipped course varied from 15.96% to 76.97%, which were
generated even though the flipped-course professors received a 78% premium per hour of synchronous
session time, as explained earlier, and the section enrollment cap for a flipped course was about two-
thirds that of the traditional course. Nevertheless, the fact that the flipped-course professor only taught
28% as much of the synchronous session time as the traditional course professor made the cost per
student lower. In other words, even if the first section was completely full for both course versions (74 for
the traditional course and 49 for the flipped course), the flipped-course’s synchronous session faculty
compensation cost remained only 75.5% as high as the traditional-course costs (28%*1.78*(74/49)).
Another way of viewing the lower faculty compensation cost for a flipped-course’s synchronous
sections is that, while the course was a 3-credit course from the student’s perspective, the professor in the
flipped course was paid half as many teaching credits per section (1.5) for teaching the synchronous
sessions while the professor teaching the traditional course received 3 teaching credits. So, while the
enrollment cap for the traditional course was 51% higher (74/49) for a completely full first section of each
25
In other words, the university may permit small section sizes for startup programs but the κ and
τ values used in this section are for courses in established traditional and flipped programs. The
enrollment caps used in the previous section are noted in Eberhart (2021).

30
course, the synchronous-session faculty compensation cost for the flipped course was only 75.5% as
much as for the traditional course ((1.5/3)*(74/49)).
I chose the levels of sFn—within the range discussed above—to highlight the discrete changes in
the relative cost savings of a flipped course when a new section is created. For example, when sFn = 89
and faculty are compensated on load (η=1), Δ΄ = -76.97%. When sFn = 90 and faculty are compensated on
load, there is a notable drop in cost savings with Δ΄ = -56.46%.
The reason for these discontinuities is the discrete change in the number of students per section.
At an enrollment of 89 students, the two sections of the flipped course have an average enrollment of 44.5
students. When the enrollment equals 90, the new section threshold is met (κF = 10), and the average
number of students in each section drops 33% to 30. Because the faculty compensation for teaching each
section is fixed, the opening of the new section increases the cost per student in the flipped course and
reduces its cost savings.
As student enrollments in the flipped course rise, the magnitude of the decrease in cost savings
from the creation of a new section falls for two reasons. First, the percentage decrease in the number of
students per section is lower. For example, when a fourth section of the flipped course is created, the
average enrollment per section drops only 24%, from 43 to 32.5 (compared to the 33% decrease when a
third section is created). Second, the upfront investment in asynchronous materials is spread across more
students.
The creation of new sections in the flipped course is also the explanation for the general decline
in the flipped course’s cost savings as sFN increases. For example, when sFN = 89, the average number of
students in the flipped course is 44.5 per section as noted earlier, whereas there are only 23 students
enrolled in the traditional course version. When sFN = 170, there are five sections of the flipped course
with an average enrollment of 34 students, while the traditional course has 43 students in its one section.
So, even though there are more students in the flipped course to spread the cost of the asynchronous
materials, the relatively higher number of students per section in the traditional course is sufficient to
reduce the flipped course’s cost saving (e.g., when η=0 and sFN = 89, Δ΄ = -50.58% while Δ΄ = -75.16%
when η=0 and sFN = 170).
When sFN = 409, sTN = 104, which means the traditional course has two sections. Nevertheless,
while there are still a relatively larger number of students per section in the traditional course (52 per
section compared to 40.9 for the flipped course), the savings from the reductions in classroom space
requirements and other costs are higher with more students in the flipped course. As shown in Table 8, the

31
result is that the magnitude of Δ΄ is much higher (-32.21% to -36.66%) than when sFN = 170 (-15.96% to -
25.75%). And when sFN = 410 and an eleventh section of the flipped course is created, the magnitude of
Δ΄ only decreases to -27.24% (η=0) and -30.20% (η=1).
The breakeven analyses follow a logic similar to the preceding analyses and are displayed in
Table 9 for the cases when faculty are compensated off load for teaching the synchronous sessions (η=0).
Panel A shows the breakeven level for each input, and Panel B shows the ratio of the breakeven level for
each input relative to its base case level. The production cost variable (𝛼𝐹1 ) shows a large breakeven level
of $346,460, or four times the base level ratio of $86,530, as shown in Panel B, when there are 89
students enrolled in the flipped course with two sections. In other words, it could cost up to four times
what is expected to produce the asynchronous materials, and the flipped course would be no costlier than
the traditional-course version, other things constant. So, as long as the production costs are less than
$346,460, the flipped course will be less expensive than its traditional version.
As with the discontinuities in Δ΄ displayed in Table 8, there are related discrete changes in the
breakeven levels in Table 9 when a new section is created. For example, the breakeven point drops from
$346,460 to $286,983 when student enrollment increases from 89 to 90 and the number of flipped course
sections rises from two to three. In other words, Δ΄ is more sensitive to changes in the production costs in
this case because they can only rise to $286,983 before the flipped course becomes more expensive than
the traditional onsite course (Δ΄ > 0).
The sensitivity of Δ΄ to the faculty compensation for the development of asynchronous materials
(𝛼𝐹2 ) input is the lowest of any of the inputs, varying from 9.59 times the base level when the flipped
course enrollment is 130 to 25.29 times when enrollment is 409. There is also low sensitivity of Δ΄ to
maintenance costs that differ from the base case.
At first glance, Δ΄ is the most sensitive to the ratio of the enrollments in the traditional course
version to the flipped course enrollments (ST/SF), varying from a level of 32.04% to 49.11% (compared to
the base level estimate of 25.4%). However, the iterative algorithm embedded in the Goal Seek function
in Excel used to compute the breakeven point finds a “local” solution that depends on the starting value of
the input (i.e., the value the iterative algorithm begins with to compute the breakeven point). For example,
when the base level input of 25.4% is the starting value for ST/SF, and the flipped-course enrollment is
169, the breakeven point for ST/SF is 37.20%. With a starting value of 100% for ST/SF, the breakeven level
is 111.71%. So, it is not imperative that the flipped course have higher enrollments for it to be less
expensive.

32
Table 10 (η = 1) shows the same breakeven analyses when faculty are compensated on load for
teaching synchronous sessions. The breakeven points are higher in every case, and in many cases much
higher, than for Table 9, where η = 0. In other words, when η = 1, the flipped course is less expensive
than its traditional version when there are “worse” input values such as higher production costs for the
development of asynchronous materials than when η = 0. These results underscore the importance of the
form of compensation in the cost of a flipped course relative to its traditional version. That is, on-load
teaching makes the flipped course less expensive than its traditional version, other things constant.
Relative Importance of Each Cost Component
To provide more insights into the relative importance of the cost components for Δ΄, I build upon
the steady-state inputs used in the breakeven analysis with a focus on the case of annual enrollments for
the flipped course equal to 160. Table 11 displays the results when the traditional-course version
enrollments equal those of the flipped course (SF = ST), the cost of capital is estimated based on the YTM
of the university’s bonds (θ = 0), and the synchronous sessions for the flipped course are held in a room
on campus (χ = 1).
When the faculty are compensated off load (η = 0), Δ΄ = -6.63% as shown in Panel A, and the
largest cost component for the flipped course is the present value of the faculty compensation for teaching
synchronous sessions, which comprises 39% of the course’s total cost. For the traditional course version,
this cost component is 55% of the total cost.
When the faculty are compensated on load (η = 1) as displayed in Panel B, the flipped course is
much less expensive (Δ΄ = -17.24%) because of the greater difference in the cost of compensating faculty
for teaching synchronous sessions in the flipped course compared to its traditional version. When faculty
are compensated off load, the present value of the faculty compensation for teaching synchronous
sessions is $228,474 for the flipped course, $114,238 less than the cost of this component for the
traditional version ($342,712 as shown in Panel A). When faculty are compensated on load, this cost
component is $225,459 less expensive for the flipped course ($450,918 for the flipped course compared
to $676,377 for its traditional version). Because the number of students in both course versions is the
same, the greater difference in this cost component results in the cost per student being much higher for
the traditional course than the flipped course when faculty are compensated on load, everything else
constant. Table 11 also reveals that flipped courses reduce costs by $58,100 ($67,471 - $9,371) in saved
classroom space and $60,612 in other costs, such as student services ($209,411- $148,799), relative to the
traditional course.

33
When enrollment in the traditional-course version is 25.4% of the flipped course’s enrollment, the
flipped course is notably less expensive as shown in Panel A of Table 12 when the faculty are
compensated off load (Δ΄ = -25.43%, compared to -6.63% when the enrollments are the same in both
courses, as shown in Table 11). When faculty are compensated on load, the cost saving for the flipped
course is even greater as displayed in Panel B (Δ΄ = -39.05% compared to -17.24% in Table 11). With
fewer students and sections in the traditional course, there is less demand for classroom space with the
traditional course, though the flipped course also still saves $13,119 ($22,490 - $9.371) in classroom
space. The flipped course has more other costs, but the other costs per student are lower than for the
traditional version.
These results show that the details matter in assessing the cost of a flipped course relative to its
traditional version. For example, economies of scale are regularly cited as a clear cost benefit of online
education, but these economies apply primarily, if not exclusively, to asynchronous materials cost
components. Fully flipped courses offer the greatest opportunity to exploit the economies of scale
possible from the use of asynchronous materials, but again the bulk of the empirical evidence discussed in
the literature review section shows that fully flipped courses have lower educational quality measures
than their traditional course versions.
Hybrid (flipped) courses offer the prospect of equaling or even exceeding the educational quality
of a traditional version, although they do not scale as much as fully flipped courses do, and that limits the
possible cost savings. Nevertheless, as this study shows, it is possible to develop and deliver hybrid
courses that meet the quality requirement while also providing significant cost savings. Moreover, these
cost savings are achieved with a simple compensation scheme for faculty that allows them to fulfill their
annual teaching obligation and pays them more per hour of synchronous-session teaching than for
teaching the traditional course (though again a flipped course professor has to teach more students to earn
the same amount of teaching credits as the traditional course professor does per section). This finding is
important because, as discussed in the literature review, faculty are typically skeptical of online teaching
and may also be concerned about the consequences of a reduced need for their synchronous-session
teaching. These results show that schools can pay faculty an online teaching “premium” and still save
money, primarily because faculty teach fewer synchronous sessions and online teaching can attract higher
enrollments.

34
Implications for Ex Ante Applications

The empirical test results discussed above provide the following insights into the how the model
can be used to make wise ex ante decisions about flipping a course. As suggested earlier, four variables
emerge—that must be forecasted or set by policy—as the most important inputs:
1. The ratio of SF to ST. While hybrid flipped courses do not scale the way that fully flipped
courses do, they still scale more than the traditional course versions because flipped courses contain
asynchronous materials and the marginal cost of adding another student to the course to work through
these materials is virtually zero. So, the larger the relative number of students expected to enroll in the
flipped course, the less costly it should be in general, with the possible notable exception of when a new
flipped-course section is created.
2. The payment to faculty for synchronous sessions. The more that faculty are paid to teach
synchronous sessions in the traditional course, the less costly the flipped course will be, everything else
constant. This can hold even if faculty are paid relatively more to teach each hour of synchronous session
time in a flipped course than in its traditional version.
A related implication is the assignment of faculty to different teaching tasks. If faculty with high
salaries are able to develop high-quality asynchronous materials, then nudging them to fulfill their annual
teaching obligations by developing these materials—for courses with high expected enrollments—is
likely to be the most efficient use of their teaching time because of the asynchronous materials’ virtually
zero marginal cost. Moreover, by increasing the pool of prospective applicants, flipped courses, especially
fully online versions, offer greater opportunities to exploit the economics of scale possible with
asynchronous materials. In short, while a school’s tenured-faculty compensation is a fixed cost,
reallocating professors to different teaching tasks could significantly reduce a school’s teaching
compensation cost per student, and provide students greater access to the school’s most highly-paid
professors.
3. The section cap (κ) and the threshold trigger for the creation of a new section (τ). As
mentioned in the breakeven analysis section, these variables are set for School X’s established programs
according to classroom constraints and feedback from the faculty regarding the maximum number of
students who can be accommodated in a section without diminishing educational quality. However,
unless the classroom constraints cannot be loosened or the faculty judgments on the maximum section
size are inviolable, then adjustments to κ and τ can have significant cost effects considering the
discontinuous increase in the cost of a flipped course when a new section is created. For example, if κ =

35
40 and τ=10, and there are 50 students who wish to enroll in the flipped course, then the cost increase of
opening a new section may be so great that the school should consider allowing for an overloaded
section—by one student in this case—or not allowing the student to enroll.
Finally, while educational quality is exogenous to the model, the empirical test results from this
study suggest one straightforward method of ensuring the quality of the flipped course is no lower than its
traditional version. That is, have the same professor(s) who taught the traditional course develop and
deliver the flipped version. For example, this professor(s) is best positioned to assess the appropriate
degree of flipping. The model then reveals the cost consequences of this choice.
Limitations
There are four primary limitations to this study. First, as with many prior studies (e.g., Bettinger
et al. (2017), Brown and Liedholm (2002), Chingos et al. (2017)), the students in my samples were not
randomly assigned to a flipped course or its traditional version. But out of the six professors who taught
courses in my samples, the same professor taught both versions of the course in all but one case and the
student profiles were highly similar. Most importantly, the primary contribution of this study in
quantifying the cost effect of flipping a course does not depend on a randomized sample.
Second, I do not investigate how hybrid online courses improve access to education—beyond
recognizing that the enrollment in these courses can be greater than in the traditional version because the
hybrid online course students are not required to live within commuting distance of the classroom—as
Goel and Joyner (2016) and Goodman et al. (2019) did.
Third, while the cost-comparison model is nonlinear with respect to variables such as the discount
rate, it is linear with respect to the degree of flipping variable (δ) for all values of δ > 0. However, the
model can be generalized to allow for the cost components to vary across different levels of δ (as they
vary when δ = 0 and the T subscript is used, compared to when δ > 0 and the F subscript is used).26
Fourth, this study does not address the broader question of the economic sustainability of courses
offered by many colleges and universities. In other words, even if a flipped course is expected to be much
less expensive than its traditional version, this cost reduction may not be significant enough to justify
offering the course if the cost is still higher than the tuition revenue it generates.
26
As noted earlier, School X’s synchronous-session teaching compensation policy is based on
teaching credits, a common measure. Because the model converts the faculty compensation measure from
teaching credits to a synchronous-session hourly basis, it is not necessary for a school to express faculty
compensation explicitly in terms of each hour of synchronous-session time.

36
5. Summary and Contributions to the Literature

Technology has exponentially improved the quality and lowered the cost of countless goods and
services, and it would seem to be straightforward to apply technological advances to teaching through, for
example, transforming the content delivered live in a traditional course to asynchronous materials (e.g.,
recorded videos) that students can access online in a so-called flipped or online course. Just as the
conversion of theatrical performances to film made the marginal cost of showing a movie to an audience
much lower than producing a live show, substituting asynchronous materials for live lectures would
appear to do the same for teaching.
However, students are not customers, at least not in the traditional sense. A customer pays the
provider for some good or service and then evaluates the value of what he or she has purchased. In
education, however, the “seller” of the service, the teacher, evaluates the “customers,” his or her
students.27 Of course, students also usually evaluate their teacher but each evaluation is but one among the
entire class, while the teacher is typically solely responsible for assigning a grade to each student.
In short, the value of the “educational service” that students’ purchase through their tuition
payments depends on how much they learn, and that outcome depends on how well the course is designed
and delivered and how much effort the students exert, which can be related to the quality of the course’s
design and delivery. Much of the empirical evidence I review in this study reveals that students learn less
in flipped courses than in their traditional versions—or the students are less satisfied with the flipped
course than its traditional version—especially fully flipped courses in which there are no required
synchronous interactions between the professor and the students. With hybrid courses that combine
asynchronous materials with required synchronous sessions, there has been more evidence of learning
outcomes equivalent to those obtained from traditional courses, but no consistent evidence of an
improvement.
Although hope remains that future advances in technology, along with greater educator
knowledge of how to use it, may lead to a breakthrough in improving the quality of education—and again
there already is some evidence of this improvement, as I discuss in this paper—the question of whether
technology reduces cost without diminishing quality is arguably more important. After all, the fact that
technology has made so many products and services cheaper without a sacrifice in quality is what has
made it so revolutionary.
27
Although there are customer ratings on ride-sharing apps such as Uber and Lyft, these ratings
are notably different from a formal grade that professors assign to their students.

37
In providing evidence that technology can reduce costs without reducing quality and exploring
the circumstances under which technology can increase costs, I make four primary contributions to the
literature. First, I develop a model that provides a commensurable comparison between the cost of a
flipped course and its traditional version. This model solves the problem of comparing the uneven cost
structure of a flipped course—with its large upfront investment followed by lower costs for each
subsequent course offering—to its traditional version, with its relatively constant cost per offering.
Second, I apply 15 measures of educational quality to a new pooled cross-section sample of
flipped courses and their traditional versions. Although these measures have been used in previous
studies, I am unaware of any study with a dataset that spans seven cohorts over approximately six years or
that conducts time series and cross-section tests of the statistical significance of these educational-quality
measures. The unprecedented length of the time series is important because there is evidence in the
literature of new innovations causing an initial increase in educational quality that recedes after the
enthusiasm stimulated by the innovation has waned. However, I find consistent evidence of a flipped
course equaling, and in many cases exceeding, the educational quality of its traditional version over the
entire six-year period.
Third, based on detailed cost-component estimates from the same flipped program and traditional
onsite version of the program that provided the data for the educational quality measures, I report
consistent evidence of a significant decrease (36% on average) in the cost of developing and delivering a
flipped course relative to its traditional version.
Fourth, I conduct breakeven analyses of the model to gauge the sensitivity of the model’s cost
estimate for a flipped course, relative to its traditional version, to input values that differ from their
expected values. While I show that the model’s estimates of cost savings associated with flipping a course
are robust to uncertainty in the input estimates, I also show how it is possible for a flipped course to be
more expensive than its traditional version. The details matter, and it is necessary to compute the net
effect of these details on the cost consequences of the flipped course investment.
In short, the research literature shows that hybrid courses are generally the only types of flipped
courses that provide a quality of education at least as high as a traditional course. Hybrid courses, by
definition, do not scale the way that fully flipped courses do, but I show consistent evidence that it is
possible to construct high-quality hybrid courses that provide significant cost savings using a model that
is broadly applicable.

38
Table 1
Numerical Example of Division of a Three-Credit Semester Course

Total Class Time (32 hours) into Asynchronous Materials and Synchronous Sessions
Total Class Total Class

Time (hours) Time (hours)
Degree of Converted to for
Flipping: Asynchronous Synchronous
δ Materials δ*32 Sessions Type of Course
0.00% 0 32 Traditional
10.00% 3.2 28.8 Hybrid
20.00% 6.4 25.6 Hybrid
30.00% 9.6 22.4 Hybrid
40.00% 12.8 19.2 Hybrid
50.00% 16 16 Hybrid
60.00% 19.2 12.8 Hybrid
70.00% 22.4 9.6 Hybrid
80.00% 25.6 6.4 Hybrid
90.00% 28.8 3.2 Hybrid
100.00% 32 0 Fully Flipped
This table shows the division of total hours of live class time in a three-credit traditional course into
asynchronous materials and synchronous sessions (i.e., live class time) sorted by the degree of flipping
variable δ.

39
Table 2
Descriptive Statistics of Admissions Profile for Principal Sample of Students in Flipped Course’s
Program and Students in Traditional Course Version’s Program
Flipped (Hybrid
Online) Course’s Traditional Course
Variable Program Version's Program
Average Undergraduate GPA 3.28 3.31
Average % Undergraduate Business Major 28% 33%
Average GMAT 665 686
Average Age 29 28
Average % Females 25% 27%
Average %U.S. Diversity* 34% 37%
Average % International 11% 29%
Average Work Experience (Years) 6.96 5.59
The admissions profile for the flipped-course students is based on the average statistics of students
enrolled in seven cohorts between 2014 and 2019, while the students in the traditional-course version
were enrolled in one cohort in 2011.
*U.S. Diversity signifies African American, Asian American, Hispanic American, and Native American.

40
Table 3
Difference in Means Tests
Subjective (S) and Objective (O) Course Quality Metrics for Flipped (Hybrid Online) Course vs.
Traditional-Course Version
Panel A: Course Evaluation Category (responses on 5-point scale with 5 the highest (best))
Ave. Response
Question Ave. Response Traditional Difference
Number Question Flipped Course Course (P-Value)
Stated course objectives 0.311
S1 4.59 4.28
met? (0.00)
Effectiveness of 0.421
S2 4.54 4.12
asynchronous material? (0.00)
0.451
S3 Amount learned in course? 4.62 4.17
(0.00)
Panel B: Course Grading Category (responses on 5-point scale with 5 the highest (best))
Ave. Response
Exams, etc. represented 0.09
S4 4.44 4.35
course content? (0.21)
Exams, etc. were graded 0.181

S5 4.68 4.50
fairly? (0.00)

41
Table 3 Continued
Panel C: Student Engagement Category (responses on 5-point scale with 5 the highest (best))
Ave. Response
Question Student Engagement Ave. Response Traditional Difference
How much effort exerted in 0.791
S6 4.24 3.45
course? (0.00)
Frequency of (synchronous) -0.091

S7 4.89 4.98
class attendance? (0.00)
Panel D: Synchronous (Syn.) Instructor Evaluation Category

(responses on 5-point scale with 5 the highest (best))
Ave. Response
Instructor well-prepared for 0.101
S8 4.91 4.81
(syn.) class? (0.00)
Contribution of (syn.) 0.491

S9 4.84 4.35
classes to learning? (0.00)
Helpfulness of instructor -0.02

S10 4.81 4.83
outside (syn.) classes? (0.75)
Instructor challenged 0.321

S11 4.84 4.52
/encouraged students? (0.00)
Overall evaluation of 0.301

S12 4.85 4.55
instructor? (0.00)

42
Table 3 Continued
Panel E: Objective Course Quality Metrics: Student Grades, Completion Rates, and Pass Rates for
Flipped (Hybrid Online) Course vs. Traditional Course Version
Ave. Metric
Metric Ave. Metric Traditional Difference
Number Metric Flipped Course Course (P-Value)
0.101
O1 GPA (on 4-point scale) 3.39 3.29
(0.01)
-2.11
O2 Completion Rate (%) 97.5 99.6
(0.00)
-1.91
O3 Pass Rate (%) 98.1 100
(0.00)
This table reports the results of difference in means tests for subjective (S) and objective (O) measures of
student learning outcomes for a flipped course relative to its traditional version. The p-values in
parentheses reflect the two-tailed t-test, where the Satterthwaite (1946) approximation of the standard
errors accounts for any difference in the samples’ standard deviations.
1
Significant at the 1 percent level.

43
Table 4
Time Series Tests of Educational Quality Measures
Panel A
Linear Trend Model
𝑺𝒋𝒊 = 𝝍𝟎 + 𝝍𝟏 𝑪𝒐𝒉𝒐𝒓𝒕𝒊 + 𝝐𝒊
Parameter 𝜓0 𝜓1 𝑅2
1
4.52 0.02
S1 0.002
(0.00) (0.40)
4.571 -0.01
S2 0.0003
(0.00) (0.78)
4.761 -0.03
S3 0.0071
(0.00) (0.15)
4.451 -0.003
S4 0.0001
(0.00) (0.88)
4.511 0.042
S5 0.0178
(0.00) (0.02)
4.391 -0.03
S6 0.0039
(0.00) (0.28)
4.911 -0.003
S7 0.0002
(0.00) (0.81)
1
4.86 0.01
S8 0.0011
(0.00) (0.25)
4.791 0.01
S9 0.0016
(0.00) (0.49)
4.811 0.001
S10 0.0000
(0.00) (0.93)
4.781 0.01
S11 0.0024
(0.00) (0.41)
4.831 0.005
S12 0.0004
(0.00) (0.72)
Panel B
Linear Trend Model
𝑶𝒋𝒊 = 𝝍𝟎 + 𝝍𝟏 𝑪𝒐𝒉𝒐𝒓𝒕𝒊 + 𝝐𝒊
Parameter 𝜓0 𝜓1 𝑅2
3.461 -0.02
O1 0.0022
(0.00) (0.28)
0.991 -0.001
O2 0.0005
(0.00) (0.03)2
0.941 0.012
O3 0.0076
(0.00) (0.03)
This table shows the parameter estimates (p-values in parentheses) of a linear trend model for subjective and
objective measures of student learning outcomes across cohorts (over time), where Sji = subjective quality measure j
(S1 through S12) assigned by student i, Oji = objective quality measure j (O1 through O3) for student i, and
Cohort i = cohort number (1 through 7) for student i. 1Significant at the 1 percent level. 2Significant at the 5 percent
level.

44
Table 5
Fixed Effects Regression Tests
Subjective (S) and Objective (O) Course Quality Metrics for Flipped (Hybrid Online) Course vs.
Traditional-Course Version
𝟕
𝑺𝒋𝒊 = 𝜶 + ∑ 𝑪𝒕 + 𝜷𝒊 𝒀𝒊 + 𝝐𝒊
𝒕=𝟐
Panel A: Parameter Estimates (P-values) for Questions S1-S6
Course Course Course Course Course Student

Evaluation Evaluation Evaluation Grading Grading Engage.
Question Question Question Question Question Question
S1 S2 S3 S4 S5 S6
4.281 4.121 4.171 4.351 4.501 3.451
α
(0.00) (0.00) (0.00) (0.00) (0.00) (0.00)
0.22 0.16 0.451 -0.26 -0.07 -0.492
βi
(0.17) (0.34) (0.01) (0.12) (0.59) (0.03)
R-square 0.06 0.08 0.11 0.02 0.04 0.20
Panel B: Parameter Estimates (P-values) for Questions S7-S12
Student Instructor Instructor Instructor Instructor Instructor

Engage. Evaluation Evaluation Evaluation Evaluation Evaluation
Question Question Question Question Question Question
S7 S8 S9 S10 S11 S12
4.981 4.811 4.351 4.831 4.521 4.551
α
(0.00) (0.00) (0.00) (0.00) (0.00) (0.00)
0.02 0.05 0.361 -0.02 0.233 0.262
βi
(0.80) (0.60) (0.01) (0.86) (0.09) (0.04)
R-square 0.03 0.02 0.13 0.01 0.07 0.07

45
Table 5 Continued
Fixed Effects Regression
𝟕
𝑶𝒋𝒊 = 𝜶 + ∑ 𝑪𝒕 + 𝜷𝒊 𝒀𝒊 + 𝝐𝒊
𝒕=𝟐
Panel C: Parameter Estimates (P-values) for O1-O3
Metric Metric Metric

Number Number Number
O1 O2 O3
3.291 0.9961 1.001
α
(0.00) (0.00) (0.00)
0.13 -0.052 0.00
βi
(0.20) (0.04) (1.00)
R-square 0.02 0.02 0.01
This table reports the parameter estimates (p-values in parentheses) of the fixed effects regressions of
subjective and objective measures of student learning outcomes on a dummy variable (Yi ) equal to 1 if
student is in the flipped course, 0 otherwise, where Sji = subjective quality measure j (S1 through S12)
assigned by student i, Oji = objective quality measure j (O1 through O3) for student i, and Ct = dummy
variable equal to 1 if student i’s cohort is cohort t (e.g., cohort 2), 0 otherwise.
1
Significant at the 1 percent level. 2Significant at the 5 percent level. 3Significant at the 10 percent level.

46
Table 6
Educational Quality Measures for Supplemental Sample
Panel A
Flipped Course #1
Average Overall Evaluation of

Year Section Enrollment Grade Instructor
2017 1 69 3.46 4.24
Traditional Version of Course #1

2017 1 32 3.46 4.53
2018 1 36 3.40 3.31
2018 2 33 3.56 4.18
Average 34 3.47 4.01

47
Table 6 Continued
Panel B
Flipped Course #2

2019 1 74 3.74 4.72

2017 1 67 3.53 3.64
2017 2 66 3.47 3.43
2017 3 65 3.45 3.45
2017 4 66 3.53 3.55
2018 1 67 3.49 4.54
2018 2 68 3.53 4.43
2018 3 53 3.47 4.44
2018 4 67 3.52 4.44
2018 5 46 3.69 4.89
Average 63 3.52 4.09

48
Table 6 Continued
Panel C
Flipped Course #3

2017 1 68 3.36 5.00

2017 1 40 3.44 4.39
2017 2 16 3.62 4.60
2018 1 27 3.51 4.45
2018 2 31 3.38 4.47
2018 3 20 3.50 4.40
Average 27 3.49 4.46

49
Table 6 Continued
Panel D
Flipped Course #4

2017 1 38 3.25 4.55

2017 1 43 3.16 4.80
2018 1 50 3.35 4.70
2018 2 48 3.32 4.61
2019 1 50 3.36 4.81
Average 48 3.30 4.73
Panel E
Average Values across all Courses
Flipped Courses Traditional Versions

Average Grade 3.45 Average Grade 3.44
Average Overall Evaluation Average Overall Evaluation of
4.63 4.32
of Instructor Instructor
This table displays the average grade and overall evaluation of an instructor for five flipped courses and
their traditional versions. The average grade is based on a four-point scale (where A = 4, A- = 3.67, B+ =
3.33, B = 3, C = 2, and F = 0) and the overall evaluation of the instructor is based on a five-point scale,
where five is the best.

50
Table 7
Cost Comparisons between the Flipped Course and its Traditional Version
Panel A: Range of Δ' Estimates
SF = ST, θ = 1 SF = ST, θ = 0 SF > ST, θ = 1 SF > ST, θ = 0

η=1 -26.0% -27.1% -65.4% -65.8%
χ>0
η=0 -6.0% -7.7% -42.0% -42.9%
χ>0
η =1 -27.1% -28.2% -66.5% -66.9%
χ=0
η=0 -7.4% -9.1% -43.4% -44.4%
χ=0
Panel B
Test of the Significance of the Average Δ'
Average Δ' -35.99%

(p-value) (0.00)1
This table shows the costs savings for a flipped course relative to its traditional version across various
parameter estimates, where Δ' = percentage difference in the annualized cost per student of the flipped
course compared to its traditional version (i.e., 𝛥′ = 𝑙𝑛(𝐸𝐴𝐶𝑆𝐹 ′) − 𝑙𝑛(𝐸𝐴𝐶𝑆𝑇 ′)), SF = number of students
enrolled in the flipped course over the life of the asynchronous materials, ST = number of students
enrolled in the traditional course version over the same period (note: ST = 0.254*SF when SF > ST), θ = 1
when the cost of capital is estimated using the capital asset pricing model (𝑟𝑅𝐹 + 𝛽𝐹 ∗ (𝑟𝑀 − 𝑟𝑅𝐹 )), θ = 0
when the cost of capital is estimated using the yield to maturity on the school’s bond(s) (∑𝐵𝑏=1 𝑤𝑏 𝑌𝑇𝑀𝑏 ),
η =1 if the professor is compensated on load—for flipped course and its traditional version—for teaching
synchronous sessions, 0 if off load in each year, χ = 1 if the flipped course’s synchronous sessions are
hosted in an on-campus room, χ = 0 when held from the professor’s home or campus office.
1
Significant at the 1 percent level.

51
Table 8
Cost Comparisons between the Flipped Course and its Traditional Version
Across Various Steady State Levels of Student Enrollment
sFn νF sTn νT Δ' (η = 0) Δ' (η = 1)

89 2 23 1 -50.58% -76.97%
90 3 23 1 -36.55% -56.46%
129 3 33 1 -36.38% -55.35%
130 4 33 1 -24.97% -39.24%
169 4 43 1 -25.56% -39.01%
170 5 43 1 -15.96% -25.75%
409 10 104 2 -32.21% -36.66%
410 11 104 2 -27.24% -30.20%
This table shows the costs savings for a flipped course relative to its traditional version across various
steady state enrollments (where the number of students per cohort (sFn or sTn, where sTn = 0.254*sFn) is
constant across the seven cohorts). The variable Δ' = percentage difference in the annualized cost per
student of the flipped course compared to its traditional version (i.e., 𝛥′ = ln(𝐸𝐴𝐶𝑆𝐹 ′) − ln(𝐸𝐴𝐶𝑆𝑇 ′)),
sFn = number of students enrolled in the flipped course for cohort n, sTn = number of students enrolled in
the traditional course version for cohort n (i.e., seven cohorts over six and a half years). The cost of
capital is estimated using the yield to maturity on the school’s bond(s) (∑𝐵𝑏=1 𝑤𝑏 𝑌𝑇𝑀𝑏 ), η =1 if the
professor is compensated on load—for flipped course and its traditional version—for teaching
synchronous sessions, 0 if off load in each year over the life of the asynchronous materials. The cost of
hosting synchronous sessions on campus (χ = 1) for the flipped course is accounted for in these
calculations. The number of course sections for the flipped (traditional) course is designated by the
variable νF (νT).

Table 9
Breakeven Analyses when η = 0
Panel A: Breakeven Points for each Input at Various Levels of sFn (νFn)
Base Case
Variable Inputs 89 (2) 90 (3) 129 (3) 130 (4) 169 (4) 170 (5) 409 (10) 410 (11)
𝛼𝐹1 $86,530 $346,460 $286,983 $301,583 $243,073 $257,798 $198,698 $529,404 $470,968
𝛼𝐹2 $792 $12,091 $9,526 $10,168 $7,595 $8,234 $5,668 $20,023 $17,493
ST/SF 25.40% 49.11% 40.51% 42.36% 35.80% 37.20% 32.04% NA NA
𝛼𝐹3𝑛 17.50% 68.93% 57.17% 60.06% 48.47% 51.39% 39.69% 105.14% 93.57%
Panel B: Breakeven Points—Relative to the Base Case Input—for each Input at Various Levels of sFn (νFn)
Base Case
Variable Inputs 89 (2) 90 (3) 129 (3) 130 (4) 169 (4) 170 (5) 409 (10) 410 (11)
𝛼𝐹1 $86,530 4.00 3.32 3.49 2.81 2.98 2.30 6.12 5.44
𝛼𝐹2 $792 15.27 12.03 12.84 9.59 10.40 7.16 25.29 22.10
ST/SF 25.40% 1.93 1.60 1.67 1.41 1.46 1.26 NA NA
𝛼𝐹3𝑛 17.50% 3.94 3.27 3.43 2.77 2.94 2.27 6.01 5.35
This table displays the breakeven points (i.e., the level of the variable that causes the cost of the flipped course to equal its traditional version (Δ' =
0)) for four key inputs into the cost comparison when the professor is compensated off load (η =0)—for the flipped course and its traditional
version—for teaching synchronous sessions in each year over the life of the asynchronous materials, where αF1 = production costs for development
of asynchronous materials (e.g., videos), 𝛼𝐹2 = faculty compensation for development of asynchronous materials per hour of lecture time in the
traditional course version converted to asynchronous materials in the flipped course (ha), ST/SF = ratio of students enrolled in the traditional onsite
version relative to the flipped course, and 𝛼𝐹𝑎 = annual asynchronous materials development and maintenance costs—as a percentage of αF1 per ha.
The number of students enrolled in the flipped course each cohort is designated by the variable sFn and the number of course sections (shown in
parentheses) is νFn, where the n subscript is not necessary because the number of students in cohort is the same. So, the ratio of the total number of
students enrolled in the flipped (SF) program and the traditional (ST) program across the cohorts is the same as the ratio of relative enrollments for
each cohort.

53
Table 10
Breakeven Analyses when η = 1
Panel A: Breakeven Points for each Input at Various Levels of sFn (νFn)
Base Case
Variable Inputs 89 (2) 90 (3) 129 (3) 130 (4) 169 (4) 170 (5) 409 (10) 410 (11)
$86,530 $672,795 $558,464 $573,232 $458,076 $472,705 $357,889 $849,146 $735,295
𝛼𝐹1
$792 $26,299 $21,309 $21,951 $16,923 $17,611 $12,622 $33,902 $28,980
𝛼𝐹2
25.40% 63.56% 49.16% 50.65% 41.03% 42.06% 35.20% NA NA
ST/SF
17.50% 133.50% 110.87% 113.79% 91.02% 93.92% 71.20% 168.41% 145.86%
𝛼𝐹3𝑛
Panel B: Breakeven Points—Relative to the Base Case Input—for each Input at Various Levels of sFn (νFn)
Base Case
Variable Inputs 89 (2) 90 (3) 129 (3) 130 (4) 169 (4) 170 (5) 409 (10) 410 (11)
$86,530 7.78 6.45 6.62 5.29 5.46 4.14 9.81 8.50
𝛼𝐹1
$792 33.22 26.91 27.73 21.38 22.24 15.94 42.82 36.60
𝛼𝐹2
25.40% 2.50 1.94 1.99 1.62 1.66 1.39 NA NA
ST/SF
17.50% 7.63 6.34 6.50 5.20 5.37 4.07 9.62 8.34
𝛼𝐹3𝑛
This table displays the breakeven points (i.e., the level of the variable that causes the cost of the flipped course to equal its traditional version (Δ' =
0)) for four key inputs into the cost comparison when the professor is compensated on load (η =1)—for the flipped course and its traditional
version—for teaching synchronous sessions in each year over the life of the asynchronous materials, where αF1 = production costs for development
of asynchronous materials (e.g., videos), 𝛼𝐹2 = faculty compensation for development of asynchronous materials per hour of lecture time in the
traditional course version converted to asynchronous materials in the flipped course (ha), ST/SF = ratio of students enrolled in the traditional onsite
version relative to the flipped course, and 𝛼𝐹𝑎 = annual asynchronous materials development and maintenance costs—as a percentage of αF1 per ha.
The number of students enrolled in the flipped course each cohort is designated by the variable sFn and the number of course sections (shown in
parentheses) is νFn, where the n subscript is not necessary because the number of students in cohort is the same. So, the ratio of the total number of
students enrolled in the flipped (SF) program and the traditional (ST) program across the cohorts is the same as the ratio of relative enrollments for
each cohort.

54
Table 11
Cost Components and EACS' in Steady State Scenarios where SF = ST
sFn = 160, sTn = 160, θ=0, χ=1
Panel A
A B C D E F G
Traditional FC’s Cost TC’s Cost FC’s Cost TC’s Cost
Course (TC) Component Component Component as % of Component as a %
Flipped Course (FC) Cost Numerical Numerical Course’s Total Cost of Course’s Total
Cost Component Cost Component Component Value Value (PV) Cost (PV)
Description Variable(s) Variable(s) η=0 η=0 η=0 η=0
Upfront production
costs for development 𝐻𝑇 ∗ δ ∗ α𝐹1
$86,530 15%
of asynchronous
materials
Upfront faculty
compensation for 𝐻𝑇 ∗ δ ∗ α𝐹2
$18,209 3%
development of
asynchronous materials
PV of course 𝐻𝑇 ∗ δ ∗ α𝐹3
$88,452 15%
maintenance, etc.
PV of faculty
compensation for 𝐻𝑇 ∗ (1 − δ) ∗ ς𝐹1 𝐻𝑇 ∗ ς 𝑇1
$228,474 $342,712 39% 55%
teaching synchronous
sessions
PV of room rental costs
𝐻𝑇 ∗ (1 − δ) ∗ ς𝐹2 𝐻𝑇 ∗ ς 𝑇2
for synchronous $9,371 $67,471 2% 11%
sessions
PV of other costs ω𝐹 ω𝑇 $148,799 $209,411 26% 34%
Total Cost (PV) $579,835 $619,594
Total number of
students enrolled in
SF ST 1,120 1,120
course across seven
cohorts
Δ' = -6.63%
This panel shows the components of the total cost of a flipped course and its traditional version when the professor is compensated off load (η =0)—for flipped
course and its traditional version—for teaching synchronous sessions in each year over the life of the asynchronous materials, the cost of capital is estimated
using the yield to maturity on the school’s bond(s) (θ = 0), and the flipped course’s synchronous sessions are hosted in an on-campus room (χ=1). The steady
state enrollment (where the number of students per cohort or is constant across the seven cohorts) is 160 for the flipped course (s Fn) and its traditional version
(sTn). The variable Δ' = percentage difference in the annualized cost per student of the flipped course compared to its traditional version (ln(𝐸𝐴𝐶𝑆𝐹 ′) −
ln(𝐸𝐴𝐶𝑆𝑇 ′)).

55
Table 11 Continued: Panel B
A B C D E F G
FC’s Cost
Traditional FC’s Cost TC’s Cost Component as TC’s Cost
Flipped Course Course (TC) Component Component a % of Component as
(FC) Cost Cost Numerical Numerical Course’s Total a % of Course’s
Cost Component Component Component Value Value Cost (PV) Total Cost (PV)
Upfront production
$86,530 11%
of asynchronous
materials
Upfront faculty
$18,209 2%
development of
$88,452 11%
maintenance, etc.
PV of faculty
$450,918 $676,377 56% 71%
sessions
𝐻𝑇 ∗ (1 − δ) ∗ ς𝐹2 𝐻𝑇 ∗ ς 𝑇2
for synchronous $9,371 $67,471 1% 7%
sessions
Total Cost (PV) $802,279 $953,259
Total number of
SF ST 1,120 1,120
course across seven
cohorts
Δ' = -17.24%
This panel shows the components of the total cost of a flipped course and its traditional version when the professor is compensated on load (η =1)—for flipped
state enrollment (where the number of students per cohort or is constant across the seven cohorts) is 160 for the flipped course (sFn) and its traditional version
(sTn). The variable Δ' = percentage difference in the annualized cost per student of the flipped course compared to its traditional version (ln(𝐸𝐴𝐶𝑆𝐹 ′) −

56
Table 12
Cost Components and EACS' in Steady State Scenarios where SF > ST
sFn = 160, sTn = 41, θ=0, χ=1
Panel A
A B C D E F G
Flipped Course Course (TC) Component Component Component as % Component as
(FC) Cost Cost Numerical Numerical of Course’s Total a % of Course’s
Cost Component Component Component Value Value Cost (PV) Total Cost (PV)
Upfront production
$86,530 15%
of asynchronous
materials
Upfront faculty
$18,209 3%
development of
$88,452 15%
maintenance, etc.
PV of faculty
$228,474 $114,237 39% 60%
sessions
for synchronous 𝐻𝑇 ∗ (1 − δ) ∗ ς𝐹2 𝐻𝑇 ∗ ς 𝑇2 $9,371 $22,490 2% 12%
sessions
Total Cost (PV) $579,835 $189,918
Total number of
SF ST 1,120 287
course across seven
cohorts
Δ' = -25.43%
This panel shows the components of the total cost of a flipped course and its traditional version when the professor is compensated off load (η =0)—for flipped
state enrollment (where the number of students per cohort or is constant across the seven cohorts) is 160 for the flipped course (sFn) and 41 for its traditional
version (sTn). The variable Δ' = percentage difference in the annualized cost per student of the flipped course compared to its traditional version (ln(𝐸𝐴𝐶𝑆𝐹 ′) −

57
Table 12 Continued: Panel B
A B C D E F G
Course (TC) Component Component Component as Component as
Flipped Course (FC) Cost Numerical Numerical a % of Course’s a % of Course’s
Cost Component Cost Component Component Value Value Total Cost (PV) Total Cost (PV)
Upfront production
$86,530 11%
of asynchronous
materials
Upfront faculty
$18,209 2%
development of
$88,452 11%
maintenance, etc.
PV of faculty
$450,918 $225,459 56% 75%
sessions
for synchronous 𝐻𝑇 ∗ (1 − δ) ∗ ς𝐹2 𝐻𝑇 ∗ ς 𝑇2 $9,371 $22,490 1% 7%
sessions
Total Cost (PV) $802,278 $301,140
Total number of
SF ST 1,120 287
course across seven
cohorts
Δ' = -39.05%
This panel shows the components of the total cost of a flipped course and its traditional version when the professor is compensated on load (η =1)—for flipped
state enrollment (where the number of students per cohort or is constant across the seven cohorts) is 160 for the flipped course (sFn) and 41 for its traditional
version (sTn). The variable Δ' = percentage difference in the annualized cost per student of the flipped course compared to its traditional version (ln(𝐸𝐴𝐶𝑆𝐹 ′) −

Bibliography
A summary of NCAT program outcomes. (n.d.). Retrieved from http://www.thencat.org/

Program_Outcomes_Summary.html
Alpert, W., Couch, K., and Oskar, H. (2016). A randomized assessment of online learning. American
Economic Review, 106(5), 378-382.
Average freshman tuition discount rate nears 50 percent. (2017). Retrieved from
https://www.nacubo.org/Press-Releases/2018/Average-Freshman-Tuition-Discount-Rate-Nears-
50-Percent
Banerjee, A., Duflos, E., (2014). (Dis)Organization and Success in an Economics MOOC, American
Economic Review, 104(5), 514-518.
Bettinger, E., Fox, L., Loeb, S., Taylor, E. (2017). Virtual Classrooms: How Online College Courses
Affect Student Success. American Economic Review. 107(9): 2855–2875
Bowen, H. (1980). The costs of higher education : How much do colleges and universities spend per
student and how much should they spend? San Francisco, CA. Jossey-Bass Publishers.
Bowen, W. G. (2012). The “Cost Disease” in higher education: Is technology the answer? (The Tanner
Lecture Series at Stanford University).
Bowen, W. G. (2013). Higher education in the digital age. Princeton, NJ: Princeton University Press.
Retrieved from https://proxy.library.upenn.edu:2457/lib/upenn-
ebooks/reader.action?docID=1122596&query=
Bowen, W. G., Chingos, M. M., Lack, K. A., and Nygren, T. I. (2014). Interactive learning online at
public universities: Evidence from a six-campus randomized trial. Journal of Policy Analysis and
Management, 33(1), 94-111.
Bowen, W. G., and McPherson, M. (2016). Lesson plan: An agenda for change in higher education.
Princeton, NJ: Princeton University Press.
Brealey, R. A., Myers, S. C., and Allen, F. (2017). Principles of corporate finance (12th ed.). New York:
McGraw-Hill Education.
Brown, B., Liedholm, C. (2002). Can Web Courses Replace the Classroom in Principles of
Microeconomics? American Economic Review, 92(2), 444-448.
Chingos, M. M., Griffiths, R. J., Mulhern, C., and Spies, R. R. (2017). Interactive online learning on
campus: Comparing students’ outcomes in hybrid and traditional courses in the university system
of Maryland. The Journal of Higher Education, 88(2), 210-233.
Christensen, C. M., and Eyring, H. J. (2011). The innovative university: Changing the DNA of higher
education from the inside out. San Francisco, CA. Jossey-Bass.
Deming, D., Goldin, C., Katz, L., and Yuchtman, N. (2015). Can online learning bend the higher
education cost curve? American Economic Review, 105(5), 496-501.

Eberhart, A. (2021). Does Technology Lower the Cost of Education without Diminishing Quality? A
Financial Modeling Approach to Flipping the Classroom: Appendices, working paper,
Georgetown University, Washington, DC. Available at
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3695111
Figlio, D., Rush, M., and Yin, L. (2013). Is it live or is it Internet? Experimental estimates of the effects
of online instruction on student learning. Journal of Labor Economics, 31(4), 763-784.
Gardner, L. (2020, March 20). Covid-19 has forced higher ed to pivot to online learning. Here are 7
takeaways so far. Chronicle of Higher Education. Retrieved from https://www-chronicle-
com.proxy.library.georgetown.edu/article/Covid-19-Has-Forced-Higher-Ed/248297
Goel, A., and Joyner, D. (2016). An experiment in teaching cognitive systems online. International
Journal for the Scholarship of Technology Enhanced Learning, 1(1), 3-23.
Goodman, J., Melkers, J., and Pallais, A. (2019). Can online delivery increase access to education?
Journal of Labor Economics, 37, 1-34.
Gordon, N. (2018). Comments on “Online postsecondary education and labor productivity.” In C. Hulten
and V. Ramey (Eds.), Education, skills, and technical change: Implications for future US GDP
growth. Chicago, IL: University of Chicago Press.
Griffiths, R. J., Chingos, M. M., Mulhern, C., Spies, R. R. (2014). Interactive online learning on campus:
Testing MOOCs and other platforms in hybrid formats in the university system of Maryland.
Retrieved from https://sr.ithaka.org/publications/interactive-online-learning-on-campus/
Hodges, K. (2010). Arizona Board of Regents: Learner-centered education course redesign initiative.
Retrieved from http://www.thencat.org/States/AZ/Abstracts/ASU Geology_Abstract.htm
Hoxby, C. (2018). Online postsecondary education and labor productivity. In C. Hulten and V. Ramey
(Eds.), Education, skills, and technical change: Implications for future US GDP growth (pp. 401-
463). Chicago, IL: University of Chicago Press.
Jaggars, S. S., and Bailey, T. (2010). Effectiveness of fully online courses for college students: response to
a Department of Education meta-analysis. Retrieved from
https://ccrc.tc.columbia.edu/publications/effectiveness-fully-online-courses.html
Jaschik, S., and Lederman, D. (2017). 2017 Survey of faculty attitude toward technology: A study by
Inside Higher Education and Gallup. Inside Higher Education. Retrieved from
https://www.insidehighered.com/system/files/media/2017 Survey of Faculty Atittudes on
Technology_0.pdf
Joyce, T., Crockett, S., Jaeger, D. A., Altindag, O., O’Connell, S. D. (2015). Does classroom time matter?
Economics of Education Review, 46, 64-77.
Levin, H., McEwan, P., Belfield, C., Bowden, A.B., Shand, R. (2017). Economic evaluation in education:
Cost-effectiveness and benefit-cost analysis (3rd ed.). Thousand Oaks, CA: Sage Publications,
Inc.

McCandless, J. (2015). U.S. education institutions spend $6.6 billion on IT in 2015. Retrieved from
https://www.govtech.com/education/higher-ed/us-education-institutions-spend-66-billion-on-it-
in-2015.html
Means, B., Toyama, Y., Murphy, R., Bakia, M., Jones, K. (2009). Evaluation of evidence-based practices
in online learning: A meta-analysis and review of online learning studies. Washington, DC. U.S.
Department of Education Office of Planning, Evaluation, and Policy Development Policy and
Program Studies Service.
Means, B., Toyama, Y., Murphy, R., Bakia, M., Jones, K. (2010). Evaluation of
evidence-based practices in online learning: A meta-analysis and review of online learning
studies (revised). Washington, DC. U.S. Department of Education Office of Planning, Evaluation,
and Policy Development Policy and Program Studies Service.
Perna, L. W., and Ruiz, R. (2016). Technology: The solution to higher education’s pressing problems? In
M. N. Bastedo, P. G. Altbach, and P. J. Gumport (Eds.), American higher education in the
twenty-first century: Social, political, and economic challenges (4th ed., pp. 432–461). Baltimore,
MD: Johns Hopkins University Press.
Protopsaltis, S., and Baum, S. (2019). Does online education live up to its promise? A look at the
evidence and implications for Federal policy. Working Paper. George Mason University and
Skidmore College.
Reich, J. (2020). Failure to Disrupt: Why Technology Alone Can’t Transform Education. Cambridge,
MA, and London, England: Harvard University Press.
Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics

Bulletin, 2(6), 110-114.
Straumsheim, C. (2014). IPEDS survey misses thousands of online students, but often counts too many.
Retrieved from https://www.insidehighered.com/news/2014/09/26/ ipeds-survey-misses-
thousands-online-students-often-counts-too-many
Swoboda, A., Feiler, L. (2016). Measuring the Effect of Blended Learning: Evidence from a Selective
Liberal Arts College, American Economic Review, 106(5), 368–372.
Urquiola, M., Verhoogen, E. (2009). Class-Size Caps, Sorting, and the Regression-Discontinuity Design,
American Economic Review, 99:1, 179–215.
Xu, D., and Jaggars, S. S. (2014). Performance gaps between online and face-to-face courses: Differences
across types of students and academic subject areas. The Journal of Higher Education, 85(5),
633-659.
Zemsky, R., and Shaman, S. (2017). The market imperative: Segmentation and change in higher
education. Baltimore, MD. Johns Hopkins Press. Retrieved from
https://jhupbooks.press.jhu.edu/content/market-imperative
Zumeta, W., Breneman, D. W., Callan, P. M., and Finney, J. E. (2012). Financing American higher
education in the era of globalization. Cambridge, MA: Harvard Education Press.

Does Technology Lower the Cost of Education without Reducing Quality? A Financial Modeling Approach

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Does Technology Lower the Cost of Education without Reducing Quality? A Financial Modeling Approach

Uploaded by

Copyright:

Available Formats

Does Technology Lower the Cost of Education without Reducing Quality?

A Financial Modeling Approach to Flipping the Classroom*

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

The Cost of the Traditional Course Version

Electronic copy available at: https://ssrn.com/abstract=3694205

and the dollar difference is designated by the variable Δ$:

Electronic copy available at: https://ssrn.com/abstract=3694205

𝐻𝑇 ∗ 𝛿 ∗ α = upfront investment in asynchronous materials development and maintenance ($45,000

Electronic copy available at: https://ssrn.com/abstract=3694205

4. Data and Empirical Tests

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

(Synchronous) Instructor Evaluation

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Implications for Ex Ante Applications

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

5. Summary and Contributions to the Literature

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Numerical Example of Division of a Three-Credit Semester Course

Total Class Total Class

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205

Exams, etc. were graded 0.181

Electronic copy available at: https://ssrn.com/abstract=3694205

Frequency of (synchronous) -0.091

Panel D: Synchronous (Syn.) Instructor Evaluation Category

Contribution of (syn.) 0.491

Helpfulness of instructor -0.02

Instructor challenged 0.321

Overall evaluation of 0.301

Electronic copy available at: https://ssrn.com/abstract=3694205

Electronic copy available at: https://ssrn.com/abstract=3694205