2020-Who Gains From Active Learning in Higher Education

EDUCATION ECONOMICS
2020, VOL. 28, NO. 3, 311–331

https://doi.org/10.1080/09645292.2020.1761298
Who gains from active learning in higher education?

Giulio Bosio and Federica Origo
Department of Management, Economics and Quantitative Methods, University of Bergamo Bergamo, Italy
ABSTRACT ARTICLE HISTORY

We study whether and how teaching style (i.e. traditional vs active mode) Received 4 September 2019
affects academic performance of young individuals in tertiary education. Accepted 23 April 2020
We focus on entrepreneurship education as an ideal subject for
KEYWORDS
experimenting alternative teaching methods. Identification relies on Entrepreneurship education;
Triple Difference estimates based on detailed administrative data for the teaching modes; academic
universe of students in a Master’s program in Management in Italy. Our performance; triple
preferred estimates show no significant effects of the teaching mode on difference; difference-in-
student’s achievement. However, further estimates reveal interesting difference
heterogeneities across students, being active teaching more effective in
the case of females and students from secondary schools with an JEL CODES
academic track. I20; J24; L26
1. Introduction
Research on educational production functions highlights the role of teacher quality in favouring stu-
dents’ learning and raising their academic achievement (Rivkin, Hanushek, and Kain 2005; Hanushek
and Rivkin 2010). However, there is no strong evidence of any effect of observable teacher character-
istics, such as gender, race, experience, credentials, and training, on students’ outcomes, although
such characteristics are highly correlated with teachers’ compensation (Hanushek 1992; Dee 2005,
2007; Boyd et al. 2006; Kane, Rockoff, and Staiger 2006; Harris and Sass 2011). In light of this, quite
recently there has been a shift in empirical research from what teachers are (in terms of observable
characteristics) to what teachers do, trying to identify the teaching practices that matter to students’
achievement (Dobbie and Fryer 2013; De Witte and Van Klaveren 2014; Comi et al. 2017; Falck, Mang,
and Woessmann 2018). Many studies distinguish between traditional teaching (mainly lectures, but
also videos and case studies) and active teaching. The latter consists of a wide range of activities (such
as simulations, experiments, role play, and other types of fieldwork) that place the students at the
centre of the learning process, asking them to play a more active role in the classroom rather than
being just passive learners (Walter and Dohse 2012). Most of these studies rely mainly on either
primary or secondary education and identify the effect of the teaching style by comparing students’
performance in different subjects (for example, comparing reading with maths or maths with
science), which may differ significantly in the extent to which they can actually be taught in an
active way. On the whole, they do not provide a clear-cut answer concerning the superiority of
active practices to traditional ones.
In this paper, we contribute to this strand of the literature by providing new evidence on the effect
of different teaching practices on students’ academic performance at the university (master’s) level.
Different from most previous literature, we estimate this effect by comparing students experiencing
different teaching styles in the same subject, which is entrepreneurship. We measure students’
CONTACT Federica Origo federica.origo@unibg.it Department of Management, Economics and Quantitative Methods,
University of Bergamo, Via dei Caniana 2, 24127 Bergamo, Italy
Supplemental data for this article can be accessed https://doi.org/10.1080/09645292.2020.1761298
© 2020 Informa UK Limited, trading as Taylor & Francis Group
312 G. BOSIO AND F. ORIGO
achievement with their grades both immediately after the treatment (with the average grade at the
end of the first year of the programme) and at the end of the master’s programme (the final grade).
Since we also have information on pre-treatment (i.e. bachelor’s) grades, we take into account stu-
dents’ selection into different courses using a difference-in-difference strategy and controlling for
student fixed effects.
We believe that entrepreneurship education is the ideal candidate for studying the effect of
different teaching modes on students’ achievement because most of its contents (such as how to
identify business opportunities, how to prepare a business plan, and how to start up a new business)
can be taught effectively in quite different ways (Rasmussen and Sorheim 2006; Van der Luis, Van
Praag, and Vijverberg 2008; Kerr, Nanda, and Rhodes-Kropf 2014). Additionally, entrepreneurship
education aims to influence entrepreneurial attitudes and non-cognitive skills (Fayolle 2013), there-
fore stimulating the adoption of innovative practices in the learning process.
While there is a large body of literature on the effects of entrepreneurship education on the cre-
ation and performance of new start-ups (Elert, Andersson, and Wennberg 2015), little is known about
the implications of such a type of education on the academic performance of young people in tertiary
education. From this perspective, we contribute to this second strand of the literature, focusing on
how different practices in teaching entrepreneurship influence students’ grades.
We base our empirical analysis on detailed administrative data from the universe of students in a
master’s programme in management offered by a medium-sized university in Italy, where entrepre-
neurship education is becoming increasingly crucial, particularly in the departments/degrees with a
management vocation. To identify a causal effect of the teaching mode in the entrepreneurship
course on students’ achievement, we rely on the features of the plan of studies of the master’s
degree to build a sort of ‘quasi-natural’ experiment. More specifically, we exploit the fact that,
within this master’s programme, a specific entrepreneurship track offers a compulsory entrepreneur-
ship course both in Italian and in English. The main difference between the two courses, other than
the official teaching language, is the way in which entrepreneurship is taught: while the entrepre-
neurship course taught in English provides a set of practical elements related to entrepreneurial
skills, the corresponding course taught in Italian mainly relies on a set of normative rules and pro-
cedures related to the entrepreneurial career. To take into account the possibility that students
with different characteristics will self-select into the two curricula based on the teaching language,
we exploit the fact that the same master’s programme also offers a track in finance, with both an
Italian and an English curriculum, the study plan of which does not include a compulsory course
in entrepreneurship. Hence, we exploit this setting in a triple-difference (DDD) framework, in
which the treatment is the different intensity of the active contents of the entrepreneurship
course in the two curricula of the entrepreneurship track, and we use students enrolled in the
finance track to control for potential self-selection of the teaching language. As a robustness
check, following an alternative strategy used in the literature (Leuven et al. 2007; Oosterbeek, Van
Praag, and Ijsselstein 2010), we also perform IV difference-in-difference (DD) estimates, using pre-
treatment information on both English proficiency and study-related mobility as instruments for
the choice of the teaching language.
Once we control for selection into the teaching language, our results indicate no significant effects
of the teaching style on students’ achievement both in the short and in the long run. However, further
estimates reveal large differences across college students, with girls and students from high schools
with an academic track gaining more from active teaching modes.
The remainder of the paper is structured as follows. In Section 2, we briefly discuss the relevant
literature on both the effectiveness of different teaching styles and the effects of entrepreneurship
education. In Section 3, we present the data used in the empirical analysis and the institutional
context characterizing the master’s programme under investigation. In Section 4, we discuss the
empirical strategy, while the empirical results and several robustness checks are reported in
Section 5. The last section concludes.
EDUCATION ECONOMICS 313
2. Literature review
There is a growing empirical literature on the effects of different teaching styles on students’ achieve-
ment. Most of these studies compare traditional practices (especially lectures) with more active teach-
ing styles.1 Such interest also comes from the fact that the two teaching styles should pursue different
objectives: while traditional modes aim at changing knowledge and conceptual skills, active modes
should have more influence on understanding, practical skills, and attitudes. Traditional modes are
usually based on contents that can easily be explained verbally and/or captured in writing and draw-
ings, thus creating explicit knowledge. On the contrary, active modes, by encouraging personal
experience and learning by doing, should influence tacit knowledge and intuition, including
sense-based knowledge and rules of thumb (Nonaka and von Krogh 2009). An ongoing debate
emphasizes how alternative teaching styles can promote different cognitive and non-cognitive
skills among students (Algan, Cahuc, and Shleifer 2013; Bietenbeck 2014). Therefore, understanding
how traditional and active teaching practices foster students’ learning represents an important issue
(Scott Cardell et al. 1996; Emerson and Taylor 2004). Despite the large consensus on the different fea-
tures and learning objectives of traditional versus active teaching practices, the empirical literature
does not provide a clear-cut answer concerning the relative effectiveness of the two groups of prac-
tices in enhancing students’ achievement.
Using data from Pakistan, Aslam and Kingdon (2011) find that a large number of process variables,
such as asking pupils questions during lessons or quizzing students on past lessons, raise pupils’
grades. Schwerdt and Wuppermann (2011) use data on a representative sample of eighth-grade
US students and show that a shift from problem solving to lecture style presentation results in an
increase in students’ achievement. Van Klaveren (2011) finds no relationship between the amount
of time that teachers spend on lecturing style teaching and the performance of Dutch students
who are in their second year of secondary school.2 Lavy (2011) concludes that both traditional teach-
ing (i.e. lectures that emphasize the instilment of knowledge and comprehension) and ‘modern’
teaching (such as use of techniques that endow pupils with analytical and critical skills) have a
strong positive effect on pupils’ achievement. In a more recent study, Korbel and Paulus (2018)
use TIMMS data on eighth-grade Czech students and show that ‘modern’ teaching practices, such
as working in small groups, positively influence both test scores and non-cognitive skills, especially
motivation and self-confidence. On the contrary, standard practices, such as lecturing or requiring
students to memorize concepts, have no impact on either these skills or the test scores.
Furthermore, a number of studies identify the effect of teaching style by exploiting students’ per-
ceptions, but this approach may also lead to questionable results as long as the perceptions related to
the effects of in-class work differ between teachers and students. In this direction, Hidalgo-Cabrillana
and Lopez-Mayan (2018) analyse the effect of teaching styles on students’ achievement in primary
education in Spain, considering at the same time the perspectives of the teachers and their students.
They find that modern practices are significantly associated with better achievement only from stu-
dents’ perspective. Interestingly, the authors also detect the presence of relevant heterogeneities
between boys and girls as well as depending on the socioeconomic background of the students.
Notice that most of these studies are focused on primary or secondary education or identify the
effect of the teaching style by comparing students’ performance in different subjects (such as com-
paring reading with maths or maths with science), sometimes taught by the same teacher. However,
even when it is possible to control for teacher fixed effects, the actual implementation of active teach-
ing practices can differ greatly across subjects. In this respect, entrepreneurship is one of the few sub-
jects that potentially can be taught both in a traditional and in a more active way (Walter and Dohse
2012).
Regardless of the teaching mode, the main goal of entrepreneurship education is to promote
general and specific entrepreneurial skills (Von Graevenitz, Harhoff, and Weber 2010), that is, those
personal attitudes, social skills, self-confidence, and creativity that may drive the individual decision
to pursue a business activity as well as the capacity to innovate (Johansen 2014). The existing
empirical evidence on the effect of entrepreneurship courses on entrepreneurial intentions and skills
is mixed. Indeed, some studies argue that entrepreneurship education raises the interest in entrepre-
neurship as a career option (Souitaris, Zerbinati, and Al-Laham 2007), while others yield opposite
results (Oosterbeek, Van Praag, and Ijsselstein 2010; Von Graevenitz, Harhoff, and Weber 2010).
However, the impact of entrepreneurship education may actually extend beyond entrepreneurial
intentions and business start-ups, since it should provide knowledge and skills that may be useful
to young people while they are studying or if they end up working as employees in existing compa-
nies or organizations.
An in-depth investigation of the effects of entrepreneurship education on students’ achievement
in higher education is relevant for a number of reasons. First, entrepreneurship is usually taught in
formal (university) courses and, as such, it should influence the knowledge and skills that may be
useful to students during their academic careers as well. Ideally, learning how to write a business
plan may help students to organize their study plan better and to use their time more efficiently.
Second, since academic performance is a key determinant of future labour market outcomes, the
impact of entrepreneurship education on academic achievement can be used as a leading indicator
of longer-run effects in terms of labour market performance. Finally, entrepreneurship is one of the
few courses that allow students to learn by doing through in-classroom simulations of entrepreneur-
ial activities, such as the simulation of a business start-up. From a pedagogical point of view, a more
active teaching mode may create a deeper learning experience (Zantow, Knowlton, and Sharp 2005),
with subsequent effects on academic performance.
More generally, to better understand the potential effects of different teaching styles in entrepre-
neurship education, it is also relevant to distinguish between general and specific skills and between
short- and long-term effects. Entrepreneurship education can provide both general (such as leader-
ship, time, and resource management, team building, etc.) and specific skills (such as how to write a
business plan or to carry out fundraising, knowledge on laws and procedures to start-up a new
venture, etc.). In a context of rapid technological change, specific skills may produce positive
effects in the short run, but these may fade away in the long run due to less adaptability to a changing
environment, especially in the labour market. Recent empirical evidence on the effects on general
versus vocational education actually shows that individuals with general education initially face
worse employment outcomes, but experience improves their employment probability later in their
working lives compared with individuals with vocational education (Hanushek et al. 2017). If an
active teaching mode of entrepreneurship influences general skills more deeply and quickly than tra-
ditional lectures (Salas, Wildman, and Piccolo 2009), an active teaching mode should enhance aca-
demic – and labour market – performance, especially in the long run.
3. Data and institutional context

The empirical analysis is based on longitudinal data on five cohorts of college students who enrolled
in a two-year master’s programme from 2011 to 2015 at a medium-sized public university in Italy.
Data allow to follow students’ career until they graduate – or until October 2017 if they have not
earned yet their master’s degree by that date.
Specifically, we consider students who enrolled in the Master’s Degree in Management and
Finance, which, since the academic year 2011/2012, has offered the possibility to choose between
two tracks: management and entrepreneurship (hereafter named the ‘entrepreneurship’ track) and
international business and finance (hereafter named the ‘finance’ track).
Each track involves taking a sequence of compulsory and elective courses spanning the entire dur-
ation of the two-year programme. Furthermore, starting from the academic year 2013–2014, both
tracks are available in two teaching languages: Italian and English (hereafter named ‘Italian curricu-
lum’ and ‘English curriculum’). Within each track, the English curriculum includes exactly the same
courses as those offered by the Italian one. Hence, when students enrol in this master’s programme,
they select both the track (i.e. the entrepreneurship or finance track) and the teaching language (i.e.
the Italian or English curriculum).
The entrepreneurship track refers to the management and entrepreneurship issues faced by
organizations and firms to compete in challenging international markets. Hence, the objective of
this track is to offer students a set of skills that, on the one hand, facilitates their participation in entre-
preneurial teams and the development of new businesses and, on the other hand, help students to
pursue a career path in several corporate functions requiring international openness, attitude to
change, and cross-functional knowledge. Contextually, the finance track addresses the role of multi-
national firms in a context of global competition, with a special focus on finance and international
business. The objective of this curriculum is therefore to provide the necessary skills to operate in
financial markets and to adapt corporate strategies to different needs and environments in foreign
markets.
The two tracks have similar plans of study in the first year, with some common compulsory exams,
such as a 12 ECTS course in international business.3 However, the entrepreneurship course is included
only in the entrepreneurship track as a compulsory exam of 12 ECTS, while the finance track offers a
corresponding specific course of 12 ECTS in financial markets and institutions. These are the longest
courses offered in the two tracks.4
Even though, within each track, the English curriculum includes exactly the same courses as the
Italian one, in the case of the entrepreneurship track, one main difference between the two curricula
concerns the teaching method for the entrepreneurship course, which is more active in the English
curriculum and more normative in the Italian one. These differences are clearly spelt out in the official
syllabus of this course in the Italian and the English curriculum (Panel A of Table 1). While the entre-
preneurship course in both curricula is aimed at teaching students how to start up a new business
from the definition and development of a business idea, the degree of students’ involvement in
and commitment to the class activities is clearly much higher in the English course (which also
refers to the concept of a ‘bootcamp’) than in the Italian one. Clear-cut differences also emerge in
the teaching activities. The Italian course is mainly based on traditional lectures, with active teaching
consisting of teamwork and discussions with professionals. On the contrary, the English course claims
to offer an innovative approach, in which students meet only once a month with the instructor, but
are continuously asked to participate actively in different projects and workshops, individually and in
groups, to learn ‘hands-on’. Overall, the comparison of the official syllabus of the entrepreneurship
course in the two curricula clearly highlights that the teaching language does not convey what stu-
dents should learn but how entrepreneurship should be taught (Duval-Couetil 2013).
Considering the other first-year courses offered within the entrepreneurship track, we do not find
any significant difference in teaching methods, especially for the 12 ECTS course in international
business. Indeed, as shown by the official syllabus of this course reported in Panel B of Table 1, in
both the Italian and the English curriculum, the teaching in this course is mainly based on lectures,
with additional activities consisting of specific projects, take-home assignments, and in-class
discussions.5
In light of these institutional characteristics, our identification strategy exploits the fact that the
entrepreneurship bootcamp in the English curriculum represents the only course in the academic
years 2011–2015 that implemented active teaching styles to promote students’ participation and
new learning experiences, while other courses in the English curriculum and all the courses in the
Italian one clearly relied on more traditional teaching methods, therefore providing an ideal
context to test whether and to what extent alternative teaching styles could benefit college students’
achievement.
Using university-level data from a single master’s programme provides a more homogeneous
setting and reduces the potential confounding effects due to unobserved heterogeneity (Oosterbeek,
Van Praag, and Ijsselstein 2010; Von Graevenitz, Harhoff, and Weber 2010). In a similar vein, entrepre-
neurship education reflects a peculiar subject and, in particular, supporting student entrepreneurship
is a significant part of the selected university’s self-declared mission, not only offering support
Table 1. Main features of the Entrepreneurship course and the International Business course from official syllabuses in different
curricula
Italian curriculum (official English translation) English curriculum
Panel A – Entrepreneurship
Goals: Goals:
. To introduce students to the main issues pertaining the . The Entrepreneurship bootcamp will push students, day
entrepreneurial process and the elements of a successful by day, through a pattern for the creation of a new
entrepreneurial project. venture out of a raw idea. To join this course students
. The students of this course will learn how to recognize don’t need an idea, a pre-existing team, or certainty to
and develop a new business idea and how to design the succeed.
related business model. . What we expect from them is their willingness to commit
time and energy to their entrepreneurial project, and an
open mind.
Teaching activities:
Lectures, group works, open discussions with professionals
The Entrepreneurship Bootcamp has an innovative design:
students meet with the instructors in intensive sessions
once a month. Students participate actively individually and
in groups to learn ‘hands-on’
Panel B- International Business
Goals: Goals:
. This course focuses on spreading theoretical and practical . Focus on the acquisition of knowledge, skills, and abilities
knowledge on the theory of creating value for the utilized in the workplace within the international context.
enterprise, on measuring the value and on the evaluation
of intangibles.
Teaching activities: The course is carried out with class lectures, assigned
The course is carried out with face-to-face lessons, during written exercises and discussion of case studies.
which theoretical literature and case studies will be
handled.
for those college graduates who intend to start up their own business, but also promoting and
encouraging entrepreneurial initiatives and a pro-active environment (Di Gregorio and Shane
2003). Precisely, this institutional attitude emphasizes the role of entrepreneurship courses as
natural candidates for experiments with various pedagogical techniques and more differentiated
learning contexts.
Our data cover the academic career of the universe of students during their master’s programme,
including term-by-term information concerning the credits earned, academic performance in each
single exam (GPA), and time taken to complete the master’s degree. We also have access to the
final graduation grade for all students who earned their master’s degree in the time period
covered by our data. Furthermore, for all students, we have information on a number of pre-treat-
ment characteristics that may influence both the selection into a specific curriculum and the students’
achievement, such as individual characteristics (gender, place of birth, and area of residence), the
type of high school and the corresponding final grade, and the type of bachelor’s degree and the
corresponding final grade.6 Further information on academic achievement in the bachelor’s
degree, especially the specific exam grades and the corresponding credits, is available only for stu-
dents who earned it at the same university. Since our empirical analysis exploits bachelor’s degree
grades as pre-treatment outcomes, we base the core of our empirical analysis on this sample of stu-
dents. The final sample consists of 555 master’s students (corresponding to around 70 per cent of the
initial population of students enrolled in the master’s programme under investigation), of which 241
followed the English programmes (101 in the entrepreneurship track and 140 in the finance track). 7
Tables A1 and A2 in the Appendix report the differences in observable characteristics between the
universe of master’s students and the selected sample by track and curriculum. On the whole,
within each track and curriculum, the mean observable characteristics of the selected sample are
not statistically different from those of the corresponding population, with the major exception of
the area of residence; as expected, the share of students living in the province where the university is
located is significantly larger in the selected sample than in the reference population.
Table 2 reports the summary statistics for college students in the selected sample, both for the
entire sample and by track and teaching language. Column (1) refers to the full sample. Columns
(2) and (3) refer to the sample of students enrolled in the entrepreneurship track, respectively in
the Italian and the English curriculum. Similarly, Columns (4) and (5) consider students enrolled in
the two curricula (i.e. Italian and English) of the finance track. The descriptive statistics show that
the gender composition is quite similar across curricula and teaching languages, but that students
in the entrepreneurship track are more likely to be Italian citizens and resident in the same province
as the university is located, in particular those enrolled in the English curriculum (respectively 98 and
88 per cent).
Information on students’ high-school background and grades as well as on their bachelor’s degree
provides potential indicators of individual ability. More specifically, the data in Table 2 indicate that,
on average, the finance track, regardless of the teaching language, attracts better students in terms of
high-school grades. In a similar vein, students enrolled in the English curriculum, regardless of the
track, are more likely to come from an academic-oriented high school (lyceum) and to hold a Bache-
lor’s Degree in Management.8
As outcome variables, we measure academic achievement through two indicators: the average
grade at the end of the first year of the master’s programme (i.e. immediately after the treatment)
and the final grade at the end of the programme. We interpret the former as a short-run college
outcome, while the latter reflects a long-run academic achievement. This choice is motivated by
Table 2. Sample mean characteristics by track and curriculum.

Entrepreneurship Entrepreneurship Finance Finance
All sample Italian English Italian English
Students’ outcome (Master’s)
Average grade - end of 1st year 26.235 25.668 26.458*** 26.314*** 26.712***
(2.00) (1.96) (1.94) (2.01) (1.92)
Master final grade 103.71 102.506 104.048*** 103.691** 105.017***
(6.01) (5.47) (5.78) (7.03) (5.78)
Pre-treatment students’ outcome
(Bachelor)
Average grade – end of 1st year 24.659 24.297 24.547 24.794** 25.086***
(1.86) (1.72) (1.84) (1.89) (1.96)
Bachelor final grade 94.023 92.481 93.782** 94.581*** 95.721***
(7.43) (6.96) (7.40) (7.49) (7.82)
Students’ pre-treatment
characteristics
Female 0.528 0.524 0.515 0.535 0.536
(0.50) (0.50) (0.50) (0.50) (0.50)
Italian citizen 0.969 0.978 0.980 0.969 0.950***
(0.17) (0.16) (0.17) (0.21) (0.26)
Resident in the same province 0.827 0.822 0.881* 0.791 0.829
as university (0.38) (0.39) (0.33) (0.42) (0.39)
High school with an academic 0.550 0.530 0.604 0.450** 0.629**
track (lyceum) (0.50) (0.50) (0.49) (0.50) (0.49)
Final high-school grade 78.182 77.029 77.350 79.421** 79.164**
(11.28) (11.30) (10.62) (11.47) (11.37)
‘Late student’ during Bachelor 0.030 0.032 0.020 0.047 0.014
(0.17) (0.18) (0.14) (0.21) (0.12)
Bachelor in management 0.622 0.586 0.784*** 0.458*** 0.699***
(0.49) (0.49) (0.41) (0.50) (0.46)
N. students 555 185 101 129 140
Note: standard deviation reported in parenthesis. Stars refer to the t-test for the difference in means, where the reference group are
students in the Entrepreneurship Italian curriculum. *** Significant at 1%, ** significant at 5%, * significant at 10%. Average
grades at the end of the first year are measured on a 30-point scale, final grades are measured on a 110-point scale for both
Bachelor and Master, final high-school grades are measured on a 100-point scale. Late students are students that took more
than the prescribed time (usually three years) to earn their Bachelor degree.
the relevance of the time dimension in determining the effectiveness of teaching styles based on stu-
dents’ active participation and innovative pedagogical techniques. The statistics in Table 2 highlight
that the highest average grades, both at the end of the first year and at the end of the programme,
are registered by students in the English curriculum within the finance track, while the lowest grades
are reported for students in the Italian curriculum within the entrepreneurship track.
4. Empirical strategy
The aim of the empirical analysis is to estimate the causal impact of alternative teaching styles in
entrepreneurship courses on college students’ achievement, comparing the outcomes of interest
for those who experienced active teaching methods characterized by student-centred learning
(the treated group) with the outcomes of those exposed to more traditional lectures in the same
course (the control group). Hence, both groups attended the entrepreneurship course, but the
treated group includes all the students who took the course taught in English, while the control
group consists of the students who attended the Italian one. As pointed out in discussing the insti-
tutional features of the master’s programme, the entrepreneurship course in the English curriculum
has more active contents than the corresponding course in the Italian one. Hence, the treatment is
the intensity of active modes in teaching entrepreneurship.
In this setting, we should take into account the potential self-selection of the student into the
teaching language. Hence, standard OLS estimates do not fully control for the potential selection
issues associated with the non-random assignment of students to a language track.
As a preliminary identification strategy, we rely on the institutional setting discussed above and on
the pre-treatment information about college students’ outcomes related to their bachelor’s degree,
applying a difference-in-difference (DD) framework. In the DD approach, we start by comparing stu-
dents from the English and Italian curricula within the entrepreneurship track and estimate the fol-
lowing equation:
yit = a + b1 Treatedi ∗Postt + b2 Postt + mi + 1it (1)
where yit identifies an indicator of academic performance for student i at time t, Treated is a dummy
for students taking an active entrepreneurship course in the English curriculum, and Post is the period
after the treatment. To control for selection driven by time-invariant individual characteristics, in our
preferred specification, we also include student fixed effects mi . Given the richness of our administra-
tive data, we further estimate an alternative specification controlling for pre-treatment individual
characteristics instead of student fixed effects. The pre-treatment controls include gender, citizen-
ship, area of residence, high-school type, high-school final grade, major of the bachelor’s degree,
and status of late students during the bachelor’s degree.9 In all the specifications, we compute
robust standard errors clustered at the individual level.
As mentioned in the previous section, we estimate this model with two main outcome variables:
the average grade at the end of the first year of the master’s programme and the final grade. In the
DD framework, the corresponding pre-treatment measures are the average grade at the end of the
first year of the bachelor’s programme and the final grade of the bachelor’s degree.
Notice that, given the features of the institutional setting, treatment implies both a difference in
the teaching style and a difference in the teaching language. From this perspective, students’ choice
of the English course might not be random, and it may influence the evolution of the outcome vari-
ables over time, regardless of the teaching style. This will also challenge the so-called ‘parallel trend
assumption’ needed to obtain unbiased DD estimates.
In particular, some individuals may have higher abilities or skills in terms of English proficiency and
could actually self-select into the English course. Furthermore, young students who are less averse to
risk and changes are more likely to self-select into active entrepreneurship courses (Oosterbeek, Van
Praag, and Ijsselstein 2010). These (unobserved) characteristics are also likely to influence students’
participation in international mobility programmes (such as the Erasmus one) in the bachelor’s
degree and then subsequent self-selection into English courses at the master’s level. Under these
assumptions, compared with students who took the entrepreneurship course taught in Italian, stu-
dents in the English course may also have experienced different trends in the outcome variables
without attending a more active entrepreneurship course.
To tackle this potential identification threat, we exploit the fact that the finance track also offers
two curricula, one taught in Italian and one taught in English. Assuming that selection into the teach-
ing language is the same for the two tracks, we estimate a triple-differences model (DDD) specified as
follows:
yit = a + b1DDD Englishi ∗EEi ∗Postt + b2 Englishi ∗Postt + b3 EEi ∗Postt + b4 Postt + mi + 1it (2)
where EE identifies the entrepreneurship track (i.e. the track with entrepreneurship education, EE),
English captures the two English curricula (either in the entrepreneurship or in the finance track),
and Post is the period after the treatment. In this specification, Englishi ∗EEi identifies students
taking an active entrepreneurship course, and it corresponds to Treated in equation (1). The main
coefficient of interest is now β1DDD, which should capture the causal effect of an active mode of teach-
ing entrepreneurship compared with a more traditional one, taking into account the possibility that
students enrolled in an English programme may also experience a different trend in the outcome
variables from students enrolled in an Italian one in the absence of the treatment. The student
fixed effects absorb both track and curriculum fixed effects.
As mentioned above, the identification of β1DDD relies on the assumption that selection into the
teaching language in the finance track is the same as in the entrepreneurship track. The descriptive
statistics reported in Table 2 seem to support this assumption at least on the basis of observable
characteristics: within each track, students in the English curriculum are more likely to come from
a high school with an academic track, to hold a Bachelor’s Degree in Management, and to live in
the province where the university is located.
Notice that, even if we consider as ‘treated’ all the students with the entrepreneurship course
taught in English in their study plans, some students may not have attended it yet by the end of
the first year of the programme. This is because, in the Italian university system, courses are formally
associated with different terms and years of the programme (for instance, the entrepreneurship
course is formally offered in the first term of the first year of the master’s programme) and students
are encouraged to attend their courses following the official schedule of the programme, but stu-
dents are free to take them whenever they want, unless some courses are prerequisites for other
advanced courses in the following years. Hence, our quasi-experimental design is based on the
initial assignment and not on the treatment actually received. In this light, estimates of the short-
run effect should be interpreted as an ‘intention-to-treat’ (ITT) effect, since there could be imperfect
compliance within the treatment group.10 From this perspective, ITT identification strictly relies on
the initial treatment assignment and ignores all sorts of non-compliance in the post-protocol
period. The ITT effect tends to be generally smaller in size than the standard average treatment
effect (i.e. it is likely to underestimate the true causal effect of interest) because of imperfect compli-
ance (Angrist and Pischke 2008). Hence, although it can be interpreted as a sort of lower-bound esti-
mate, the ITT is more relevant to policy implications than the average treatment effect (ATE)
parameter in the empirical analysis of ‘voluntary’ programmes (Bloom 2008).
Another potential concern is related to the non-random assignment of students to teachers and,
more generally, to differences in teacher quality between the two entrepreneurship courses. Indeed,
teachers may decide to introduce active teaching styles according to their observable (such as
gender or teaching experience) and unobservable (such as motivation or ability to use active teach-
ing methods) characteristics.
In our setting, teacher quality might represent a serious identification threat as long as better tea-
chers are more likely to adopt active teaching methods and teacher quality per se positively influ-
ences students’ performance. This problem is partly mitigated by the fact that the two courses are
coordinated by the same full professor, who personally teaches few lectures in both courses.
However, the teaching team is not the same in the two courses, although it is rather stable over time.
Unfortunately, we cannot control for teacher fixed effects in our estimates, and the positive effect
of innovative teaching styles in DD estimates could also capture the effect of teachers’ quality on stu-
dents’ achievement as long as better teachers are assigned to the entrepreneurship course in the
English curriculum than to the corresponding course in the Italian one. However, if the same assign-
ment process is used for all the courses and is similar across tracks (i.e. better teachers are assigned to
the English curriculum within each track and hence differences in teacher characteristics between the
English and the Italian curriculum are similar across the two tracks), then the potential bias related to
teacher quality would disappear in the DDD estimates. Although we cannot completely rule out the
potential effect of teacher quality, in Section 5 we shall provide some robustness checks and further
descriptive evidence on indicators of teachers’ quality in the two entrepreneurship courses.
Finally, another factor that can potentially bias our long-run estimates is dynamic selection: indi-
viduals who complete their master’s degree (and for whom we can observe the final grade in the time
window covered by our data) may be a non-random sample of the initial sample of students.11 The
composition over time of the treatment group could also change as a result of the treatment,
although it is difficult to predict the direction of this effect. On the one hand, active teaching may
provide students with better skills or higher motivation to complete the master’s degree earlier
than students who are exposed to more traditional teaching methods; on the other hand, active
teaching is more demanding and time consuming than traditional teaching, thus making it harder
to pass the entrepreneurship course and hence to complete the master’s degree earlier than students
taking the entrepreneurship course taught in a more traditional way. Unfortunately, we could not find
an instrument to deal with this type of selection in our data, but, in the robustness checks section, we
shall provide descriptive evidence on the distribution of students’ characteristics in the initial sample
and in the sample of those who have already earned their master’s degree in the period under
investigation.12
5. Main results
In this section, we present our main empirical results related to the impact of active teaching styles in
entrepreneurship courses on college students’ outcomes, looking respectively at the average grade
at the end of the first year of the master’s programme and at the final grade at the end of the master’s
degree. For each outcome, we discuss the results from the difference-in-difference (DD) approach and
the triple-difference (DDD) specification. Notice that, in the Italian university system, course grades are
measured on a 30-point scale, while the final grade is usually measured on a 110-point scale. To ease
the interpretation of the results, we standardize both short- and long-term outcomes in our
regressions. Then we investigate potential heterogeneous effects by gender and high-school back-
ground and we present a number of robustness checks.
5.1. Difference-in-difference results

Difference-in-difference estimates of the effect of the teaching style on both short- and long-run edu-
cational achievements are presented in Table 3. To account for the potential bias related to unob-
served factors influencing both course selection and academic performance, we present both
pooled OLS estimates with and without the set of controls capturing relevant students’ pre-treatment
characteristics (columns (1) and (2)) and student fixed effect estimates (column (3)). Precisely, in
column (2), we control for gender, citizenship, cohort fixed effects, area of residence,13 a dummy
for the specific bachelor’s degree obtained, status of late students during the bachelor’s degree, a
dummy for high-school type and the final high-school grade.
Each panel displays the results of separate regressions for the corresponding outcome, providing
the main parameter of interest, that is, the DD coefficient that captures the specific gain associated
Table 3. The effect of the teaching style on short- and long-run outcomes, DD estimates
(1) (2) (3) (4)
DD DD FE-DD FE-DD (total population)
Panel A: Short-run outcome
English*Post 0.257** 0.246** 0.248**
(0.10) (0.10) (0.11)
R^2 0.17 0.42 0.48
N 562 562 562
Panel B: Long-run outcome
English*Post 0.077 0.066 0.077 0.201**
(0.10) (0.10) (0.09) (0.09)
R^2 0.38 0.59 0.76 0.62
N 452 452 452 690
Personal controls No Yes No No
Cohort FE No Yes No No
Area of residence FE No Yes No No
Bachelor controls No Yes No No
High-school controls No Yes No No
Student FE No No Yes Yes
Note: Robust standard errors reported in parenthesis are clustered at the student level. Both short- and long-term outcome vari-
ables are standardized. ***Significant at 1%, **significant at 5%, *significant at 10%. Personal controls are gender and citizenship;
Bachelor controls is a dummy for the type of Bachelor degree (1 for management, 0 otherwise); high-school controls include a
dummy for the type of high school (1 for lyceum, 0 otherwise) and the final high school grade.
with the active teaching style. To allow for within-individual correlation of error terms related to
repeated observations, we cluster the standard errors at the individual level.
Overall, the results in Panel A seem to indicate a sizeable impact of a student-centred learning
style in the entrepreneurship course on the average grade at the end of the first year of the
master’s degree, although the effect is weakly statistically significant. All the coefficients are positive,
and the magnitude is quantitatively similar as we add pre-treatment controls or when we move to the
specification including students’ fixed effects in column (3). According to our estimates, attending an
active entrepreneurship course increases the average grade at the end of the first year of the pro-
gramme by around one-quarter of its standard deviation.
In Panel B, we concentrate on the long-run educational outcome, replicating the same set of
regressions. Again, the estimates suggest a positive effect of modern teaching methods, although
the coefficients are not statistically significant. To uncover potential differences related to sample
selection, in column (4), we replicate the FE-DD, also including students who earned their bachelor’s
degree at a different university.14 The estimates now suggest a positive and statistically significant
impact of teaching styles based on students’ active participation. The size of the coefficient is
larger with respect to the previous estimates based on the restricted sample of students who
obtained their bachelor’s degree at the same university, and it indicates that active teaching
methods also increase the final grade by 20 per cent of its standard deviation. Overall, our estimates
based on the restricted sample are thus more conservative than those based on the entire population
of students.
5.2. Triple-difference results

As argued above, it should be noted that the estimates of b1 reported in Table 3 may not provide a
causal relation if unobserved differences between students taking, respectively, the English and the
Italian entrepreneurship course also systematically influence the potential trends in their academic
performance. We control for selection into the teaching language by exploiting the sample of stu-
dents in the finance track (which is offered both in Italian and in English) as an additional control
group and estimating a triple-difference specification. Hence, we enlarge our sample by including
students in this additional curriculum and estimate a triple-difference model as specified in equation
[2] in Section 4.
Table 4 summarizes the results from the triple-difference strategy using the same set of regressions
as in Table 3 and reporting estimates for the pooled model with and without further pre-treatment con-
trols (columns (1) and (2)) as well as fixed-effects estimates (column (3)). In particular, the results con-
cerning short-run college achievement in Panel A show that the estimated triple difference (i.e.
estimates of b1DDD ) exhibits positive signs across all specifications, which is in line with the previous
DD results. However, none of them is statistically significant, corroborating the idea that teaching
styles based on students’ active participation do not significantly increase college performance. In
addition, in all the specifications, the size of the effect is smaller than the corresponding DD estimates.
When we focus on the long-run outcome in Panel B, we find positive coefficients across all the
model specifications, but they are never statistically significant. We find similar results when we
also include in our estimating sample those students who obtained their bachelor’s degree at a
different university (column (4)). Overall, these estimates indicate that, once we take endogenous
selection into a curriculum based on the teaching language into account, the academic performance
of college students seems to be unaffected by active teaching modes.15
To sum up, our DD estimates indicate that teaching styles relying on active learning may have a
positive impact on short-run college achievement, but these results may be biased by self-selection
into the teaching language. Actually, once we control for this source of potential bias with a DDD
approach, we conclude that students who are exposed to more active teaching practices in the entre-
preneurship course do not register a statistically significant improvement in their academic
performance.
However, the point estimates of average negligible effects may be a result of positive effects for
some groups of students and negative or null effects for others. In the remainder of this section, we
explore the existence of differences in the effectiveness of active teaching methods across groups of
students and conduct several sensitivity checks to test the robustness of our main results.
5.3. Heterogeneous effects

Overall, the previous estimates do not show a significant average effect of active teaching styles in
the entrepreneurship course on academic achievement, although the findings are not clear-cut
and small impact sizes in the short run cannot be completely excluded. Notably, these aggregate
results could hide potentially relevant differences by group of students. Indeed, even though the
Table 4. The effect of the teaching style on short- and long-run outcomes, DDD estimates
(1) (2) (3) (4)
DDD DDD FE-DDD FE-DD (total population)
English*EE*Post 0.195 0.176 0.162
(0.15) (0.15) (0.14)
R^2 0.17 0.41 0.46
N 1110 1110 1110
English*EE*Post 0.101 0.102 0.101 0.199
0.15) (0.15) (0.14) (0.14)
R^2 0.35 0.57 0.72 0.60
N 844 844 844 1182
Personal controls No Yes No
Cohort FE No Yes No
Area of residence FE No Yes No
Bachelor controls No Yes No
High-school controls No Yes No
Student FE No No Yes
ables are standardized. ***Significant at 1%, **significant at 5%, *significant at 10%. See Note of Table 3 for the detailed list of
controls.
analysis of differential effects by students’ characteristics is still an almost unexplored issue in the lit-
erature, it gains relevance especially from a policy perspective.16
In particular, given that a long-standing stream of literature addressing the observed gender
differences in the start-up rate (Minniti 2009; Koellinger, Minniti, and Schade 2013) emphasizes
that women traditionally have different entrepreneurship-related attitudes from men, investigating
potential effect heterogeneity in the relationship between innovative teaching styles in entrepre-
neurship courses and academic performance by gender could be of crucial interest. For this
reason, we estimate the DDD model separately for females and males. The main results by gender
are summarized in Table 5, in which we replicate the same FE-DDD regressions as in Table 3.
Overall, the estimates of the triple-difference coefficients in columns (1) and (2) clearly indicate
that the impact of active teaching methods in entrepreneurship education on short-run academic
achievements is almost entirely related to female students, while there is virtually no effect for
male students. More precisely, attending an active entrepreneurship course increases the average
grade at the end of the first year of the programme by around 36 per cent of its standard deviation.
This result suggests that innovative teaching modes in entrepreneurship can benefit mostly female
students, stimulating their participation in this educational training and potentially reducing future
gender gaps in the labour market (Furdas and Kohn 2010).
More interestingly, the results in column (3) show that, for females, the impact of teaching styles
based on students’ active participation also persists in the long run. In line with the previous findings,
the coefficient for males in column (4) is positive but statistically not different from zero.
Our results complement the existing evidence on the heterogeneous effects by gender of
different teaching styles on students’ achievement, with girls benefitting more than boys from
more active teaching practices (i.e. class discussion and work in small teams), in primary education
as well (Hidalgo-Cabrillana and Lopez-Mayan 2018).
This differential effect by gender may be explained by gender differences in the initial endowment
of general (soft) skills that are actually trained by active teaching practices in entrepreneurship.
Indeed, there is evidence showing that, even if the academic returns to soft skills are roughly the
same between boys and girls, the latter begin school with more advanced soft skills than the
former, and this comparative advantage grows over time (Di Prete and Janning 2012). Furthermore,
while girls refrain more than boys from public speaking in the classroom, they are more likely to feel
comfortable speaking in small groups (Tatum et al. 2013; Eddy, Brownell, and Wenderoth 2014) and
are more collaborative leaders in teamwork (De Paola, Gioia, and Scoppa 2018). The entrepreneurship
bootcamp actually requires students to work in small teams and to relate more directly to the pro-
fessor, thus creating a learning environment that can potentially favour the development of those
soft skills that are more typical for girls. Unfortunately, the administrative data do not contain detailed
information on students’ skills and we cannot test this mechanism directly.17
An alternative explanation may rely on differences in initial achievement by gender: if girls are on
average higher achievers than boys and active learning requires a greater effort or higher cognitive
skills, we may expect active teaching practices to be more effective in academic performance in the
case of high-achieving students. Our data actually show that girls have on average higher bachelor’s
Table 5. The effect of the teaching style on short- and long-run outcomes by gender, FE-DDD estimates.
(1) (2) (3) (4)
Short-run Long-run outcome
Females Males Females Males
English*EE*Post 0.364* −0.077 0.332* 0.177
(0.19) (0.22) (0.18) (0.23)
R^2 0.46 0.46 0.72 0.73
N 590 520 468 376
Student FE Yes Yes Yes Yes
Note: Robust standard errors reported in parenthesis are clustered at the student level. Both short- and long-
term outcome variables are standardized. ***Significant at 1%, **significant at 5%, *significant at 10%.
grades (96.3 on a 110 scale compared with 93.5 for boys; the difference is statistically significant) and
are more likely to finish their bachelor’s degree in the prescribed time (97.6 per cent of girls compared
with 96.4 per cent of boys, but the difference is not statistically significant).
Regardless of the mechanism behind the gender differences, our results suggest that, compared
with males, females seem to gain more from active learning; this can potentially enhance the gender
inequality in students’ performance, provided that female students have on average better grades
than their male peers.18
In a complementary way, following the literature that emphasizes the role of school type in
influencing students’ achievement (Cappellari 2004; Newhouse and Beegle 2006), we explore
whether modern teaching methods in entrepreneurship education affect students from different
high-school backgrounds differently. To this end, we split the sample into those students who
attended high schools preparing for college (lyceum) and those with a technical high-school back-
ground. The estimates in columns (1) and (2) of Table 6 show that student-centred learning
methods in entrepreneurship education seem to influence positively short-run academic outcomes
mainly for students from high schools with an academic track. In contrast, students from technical
high schools do not benefit from active teaching styles: the estimated short-run effect is negative
but not statistically significant. However, regardless of the high-school background and consistent
with the aggregate FE-DDD estimates, we do not find any robust evidence that active teaching
styles also matter from a long-run perspective.
As in the case of gender, differences in initial achievement could explain the differential effect of
active teaching by the type of high school: students from a high school with an academic track have
on average higher bachelor’s degree final grades (95.4 on a 110 scale, compared with 94.4 for stu-
dents from technical high schools; the difference is statistically significant) and earned their bache-
lor’s degree earlier than students from technical high schools.19 Additionally, while all high schools
specialize in a specific field and the corresponding degree allows students’ to enter college, the
scope of the field is narrow in the case of technical schools, which are also more likely to provide
highly specialized technical skills than high schools preparing for college. Hence, students from
the latter type of schools are less exposed to experiential/practical teaching during high school
and, like girls, are more likely to own the type of soft skills (such as critical thinking and problem
solving)20 that can be enhanced effectively by active learning in entrepreneurship.
5.4. Robustness checks

As an alternative strategy to take into account students’ self-selection into a certain entrepreneurship
course on the basis of the teaching language, we combine the difference-in-difference estimator with
an instrumental variable approach. In the literature, this approach is used by Oosterbeek, Van Praag,
and Ijsselstein (2010), who estimate an IV difference-in-difference (IV-DD) model to evaluate the effect
of entrepreneurship education on entrepreneurial skills and motivations by comparing students
enrolled at a Dutch university site offering such a course with students enrolled at another university
Table 6. The effect of the teaching style on short- and long-run outcomes by high-school background, FE-DDD estimates.
(1) (2) (3) (4)
Short-run outcome Long-run outcome
Academic high-school Technical high-school Academic high-school Technical high-school
English*EE*Post 0.414** −0.176 0.150 0.080
(0.19) (0.22) (0.19) (0.23)
R^2 0.48 0.44 0.76 0.68
N 606 504 466 378
Student FE Yes Yes Yes Yes
ables are standardized. ***Significant at 1%, **significant at 5%, *significant at 10%.
site that does not offer it. They use the distance of the university sites from the students’ home as an
instrument to take into account the potential endogeneity of the location choice.
In a similar vein, given our institutional context, we exploit information on students’ past bache-
lor’s degree career to create two ‘distance-related’ instruments for the teaching language choice. On
the one hand, we rely on the timing of the English test that students had to take in their first Bachelor
degree by measuring whether they passed the test in the same academic year in which it was for-
mally offered.21 As the second instrument, we compute a dummy variable based on any potential
experience abroad in a mobility programme offered by the university (e.g. the Erasmus programme
or other international mobility programmes). These exclusion restrictions could account for unobser-
vable characteristics related to English fluency that can make English courses at the master’s level
more attractive than Italian ones. The identifying assumption is that, conditional on a large set of
control variables related to potential ability indicators, the two instruments are unrelated to the
error term in the outcome equation.
Table A3 in the Appendix reports our first-stage regression regarding the choice of the entrepre-
neurship course taught in English, using a linear probability model with robust standard errors. As an
exclusion restriction, column (1) includes the dummy variable related to the timing of the English
exam during the first year of the bachelor’s degree, while column (2) controls for the mobility indi-
cator, capturing whether the student had a study experience abroad during the bachelor’s degree
years. Lastly, in column (3), we provide a specification including both instruments. One of the require-
ments for valid instruments is that they should significantly influence the choice of attending the
English course. In this light, the results confirm that the instruments significantly affect this choice,
both individually and jointly, although the F test is not very large, especially in the case of the instru-
ment capturing international mobility. Specifically, the point estimates show that passing the English
exam later during the bachelor’s degree reduces the probability of attending English courses at the
master’s level, while studying abroad for at least one term during the bachelor’s degree increases this
probability.
Following a similar structure, Table 7 reports the main second-stage estimates based on the IV-DD
approach for the same sample of students enrolled in the entrepreneurship track used to obtain the
DD estimates discussed above. In column (1), we use as an instrument the timing associated with the
English exam during the bachelor’s studies, while column (2) relies on the indicator capturing poten-
tial study experience abroad. Finally, in column (3), we include both exclusion restrictions. The results
in Panel A reveal that correcting for potential self-selection into the teaching language provides esti-
mates for short-run college achievements that are similar to the previous triple-difference findings.
Table 7. The effect of the teaching style on short- and long-run outcomes, IV-DD estimates.
(1) (2) (3)
IV-DD IV-DD IV-DD
English*Post 0.024 0.432 0.206
(0.769) (1.338) (0.650)
N 547 547 547
English*Post −0.637 −0.448 −0.571
(0.608) (1.074) (0.512)
N 438 438 438
Personal controls Yes Yes Yes
Cohort FE Yes Yes Yes
Area of residence FE Yes Yes Yes
Bachelor controls Yes Yes Yes
High-school controls Yes Yes Yes
Note: Robust standard errors are reported in parenthesis. Both short- and long-term outcome variables are standardized. ***Sig-
nificant at 1%, **significant at 5%, *significant at 10%. Column (1) relies on the Timing related to the English exam during the
Bachelor studies as exclusion restriction, while in Column (2) we introduce the mobility indicator that captures any potential
experience abroad in a mobility programme offered by University. In Column (3), we use both instruments as exclusion restric-
tions. See Note of Table 3 for the detailed list of controls.
Table 8. Teachers’ characteristics by entrepreneurship course.

Entrepreneurship course:
‘raditional’ teaching ‘Active’ teaching
Number of professors 4 7
Number of female professors 3 4
Number of foreign professors 0 4
% of the course taught by foreign professors 0 50%
Average years of teaching experience 17.0 16.9
Average H-index (Scopus) 2.8 5.1
Average H-index (Google Scholar) 4.3 11.0
Indeed, the coefficients for all the specifications are not statistically different from zero. Concerning
the long-run academic outcome in Panel B, we find a negative and not statistically significant effect
associated with active teaching practices regardless of the set of instruments used in the first stage.
Another relevant concern, as discussed in Section 4, is the potential confounding effect of sys-
tematic differences in teachers’ quality between the two entrepreneurship courses. While we have
already discussed the institutional features that should minimize this problem (see Sections 3 and
4), in this section we provide further estimates and descriptive evidence on teachers’ characteristics
in the two curricula.
As mentioned in Section 4, both entrepreneurship courses are taught by a team of teachers, but,
except for the full professor coordinating the two courses, the teachers differ between the two
courses. However, the descriptive statistics reported in Table 8 show that the average characteristics
of the two teams differ mainly in the average number of teachers and, as expected, the share of
foreign professors. Quite interestingly, the two courses do not differ significantly in terms of
average teaching experience, while a larger difference emerges in terms of research impact, with a
higher H-index registered for the teaching team in the ‘active’ course.22
As an additional robustness check, we replicate our DDD model by including the interaction
between cohort and curriculum fixed effects, which should also control for potential trends in
teacher quality across cohorts and curricula. The estimates (available on request) remain substantially
unchanged. Altogether, even though we cannot completely rule out the potential confounding effect
of teacher quality, this evidence suggests that this potential source of bias is rather limited and should
not fully explain our results.
A further issue that may threaten our long-run estimates is potential dynamic selection. As discussed
in Section 4, the sample of students for whom we can observe the final grade at the end of the master’s
programme may be a non-random sample of the students who initially enrolled in the programme. To
provide some indirect evidence on the potential differences between the two groups of students,
Tables A4 and A5 in the Appendix compare the mean observable characteristics of the two groups
of students within a track and a curriculum. Our estimates show that, on the whole, students who
have already earned their master’s degree are not statistically different from the sample of students
who initially enrolled in the same programme (i.e. the same track and curriculum). The only exception
is that graduates in the English finance track exhibit a higher Bachelor final grade than non-graduates in
the same track. However, the size of the difference is small (less than one point on a 110-point scale,
corresponding to less than 1 per cent of the average final grade of the Bachelor degree). This evidence
seems to suggest that dynamic selection, if present, acts in the expected direction (i.e. the students who
have already earned their master’s degree are on average better than the remaining ones along some
indicators of academic performance), but its influence should be rather limited due to the small differ-
ence in students’ characteristics between the two groups.
6. Conclusion
In this paper, we studied the effects of teaching styles on students’ academic achievement. Different
from previous studies, we considered a number of students’ outcomes after attending courses in the
same subject taught in different ways at the university level. More specifically, we focused on entre-
preneurship education, which is the ideal candidate for this type of analysis, since it can be taught
effectively both in a traditional and in an active way.
We exploited the institutional setting of a specific master’s programme offered at a university in
Italy, combined with detailed administrative data on the population of students attending this pro-
gramme from 2011 to 2017, to build a quasi-natural experiment and estimate both difference-in-
difference (DD) and triple-difference (DDD) models. We focused on the change in the students’
achievement both immediately after the entrepreneurship course and at the end of the programme.
As an alternative strategy to control for self-selection into the teaching language, we also performed
IV-DD estimates, using pre-treatment information on both English proficiency and study-related
mobility as instruments. Once we controlled for selection into the teaching language, the estimates
showed no significant effects of the teaching mode on students’ achievement both in the short and
in the long run. However, further estimates revealed interesting heterogeneities in the estimated
effects, an active teaching style being more effective in the case of females and students from
high schools with an academic track.
We proposed some mechanisms that can explain these heterogeneous effects, such as gender
differences in initial achievement or in the endowment of soft skills that may actually be enhanced
through active learning. In particular, active learning seems to benefit mainly those students who
start with a relatively low entrepreneurial propensity (e.g. females) or who were less exposed to
experiential/practical teaching during high school (e.g. students from high schools preparing for
college). We provide some evidence supporting these mechanisms, but more research is needed
to understand better why the effectiveness of the teaching style, at least in the case of entrepreneur-
ship education, may be highly heterogeneous by gender and student background.
From a policy perspective, these findings suggest that universities should invest more in innova-
tive teaching modes based on students’ active participation in entrepreneurship education,
especially in departments/courses that are traditionally less exposed to management/entrepreneurial
skills (such as humanities) and/or have a higher female ratio. Our results are in line with the European
Union agenda for entrepreneurship in higher education, which emphasizes the need to adapt entre-
preneurship training to different target groups (by level and field of study) and to promote more
interactive learning approaches, combining theoretical aspects with practical examples (European
Commission 2008). This is even more important since, in many entrepreneurship courses, female stu-
dents (and students from ‘soft’ sciences) are under-represented. Increasing female participation in
active learning can potentially increase the gender gaps in higher education, provided that female
students register higher average grades than male ones, but it may also contribute to reducing
future gender differences in the labour market by increasing female entrepreneurship. Hence, target-
ing students in non-business degrees, such as communication and education ones, might lead to a
greater influx of students who may potentially benefit more from active teaching styles (Brand,
Wakkee, and van der Veen 2007).
Notes
1. Notice that the distinction between traditional and active practices is not always clear-cut. In principle, active
teaching modes should include any activity in which the student has an active role, such as homework, but, in
practice, they require the activity to be carried out in the classroom, such as experiments or role play (Prince
2004). Furthermore, the categorization of some practices may change with the educational level or with the
nature of the subject being taught. For example, in-class discussions or quizzing students during lectures may
be considered as active teaching practices in primary and secondary schools and in the case of more standard
subjects (such as reading and maths), while they usually pertain to traditional teaching activities at higher
levels of education or in the case of subjects that inherently require an active role for the students (such as
science labs). Finally, there are some forms of cooperative learning, such as teamwork, that can pertain to
either active or traditional teaching modes. For example, teamwork requiring the students to read and present
papers is close to traditional teaching modes, while teamwork requiring them to simulate the start-up of a
new business can be considered to be an active teaching mode.
2. On this matter, De Witte and Van Klaveren (2014) investigate the extent to which the student score is affected by
the configuration of time that teachers adopt for alternative teaching modes.
3. ECTS is the acronym for ‘European Credit Transfer and Accumulation System’, which is a tool of the European
Higher Education Area used by many European countries to compare learning outcomes and their associated
workload across courses. It favours students’ mobility between countries, allowing them to have their academic
qualifications and study periods abroad recognized. Usually a master’s degree equates to 90 or 120 ECTS credits.
The latter is the case of the master’s programme under investigation in this study.
3. Within the entrepreneurship track, the entrepreneurship and international business courses account for 40% of
the total ECTS offered in the first year of the programme, while a similar share is covered by the financial markets
and institutions course and the international business course within the finance track.
5. This course is taught also in the finance track, in which similar teaching activities are adopted in both the English
and the Italian curriculum. Nonetheless, in both tracks, this course seems to differ partly between the two curri-
cula in the goals achieved: while the course in the Italian curriculum emphasizes learning related to firm value, the
course in the English one emphasizes learning of knowledge and skills that are necessary in an international
context. However, given that this course is offered in the two tracks and the two curricula, this difference in con-
tents between the two curricula should not be an issue in our estimation strategy: any difference in contents
between the English and the Italian curriculum is controlled for by taking the difference in students’ achievement
between curricula and tracks. For more details, see Section 4.
6. Information on both the type of high school and the final grade are missing for eight students who earned their
high-school diploma in a foreign country; hence, we excluded these students from our analysis.
7. To minimize dynamic selection, our final sample includes graduates, students who are still enrolled in the master’s
programme in the time spell considered, and students who dropped out at some point in the programme. Hence,
the selected sample does not change over time. However, some estimates using the final grade as the outcome
variable are based on the sub-sample of students who have completed their master’s programme. For a thorough
discussion of potential dynamic selection, see Section 4.
8. This may be due to the fact that the two English curricula were introduced for the first time in the academic year
2013–2014, while the two Italian curricula were offered also in previous years. Hence, these brand-new English
curricula where extensively promoted among students enrolled in the Bachelor degree in management at the
same university. Furthermore, students with a Bachelor degree in management seem more proficient in
English than the other students: compared to the latter, they passed the English exam in the bachelor degree
earlier (taking on average 1.5 years since the first date in which they could take this exam, compared to 3.2
years for the remaining students) and were more likely to study abroad (13% of students with a Bachelor
degree in management spent at least one term abroad with an official mobility programme, compared to
9.5% of the remaining students. However, the difference is not statistically significant).
9. Late students are those taking longer than the prescribed duration (usually three years) to earn their bachelor’s
degree.
10. This is not the case when we consider long-run outcomes, since students have to pass all the exams to graduate.
In this case, our estimates may be interpreted, more generally, as the effect of the entire entrepreneurship track
rather than of a single entrepreneurship course, even if the focus on the active teaching style is still working.
11. Dynamic selection is not an issue for our short-run outcomes, since we can observe at the end of the first year all
the students who initially enrolled in the programme. Actually, only two students have not passed any exam at
the end of the first year of the master’s degree, and hence they are dropped from the initial sample.
12. In Italy, students can virtually stay enrolled in a master’s programme for as long as they wish (provided that they
pay their annual tuition on a regular basis), and they can earn their final degree much later than the prescribed
time (which is two years). Hence, if a student in our sample has not graduated by the end of October 2017, this
does not necessarily mean that he/she will not complete the master’s programme in the following years.
13. We distinguish between those resident in the same province as the university, those resident in the same region,
and others (i.e. the reference category).
14. We cannot carry out a similar exercise with the short-term outcome because we do not have information con-
cerning the average grade at the end of the first year of the bachelor’s degree for the students who obtained
that degree at a different university, as already discussed in the data section.
15. It should be noticed that the somewhat low degree of precision of our estimated coefficients could be due to the
clustering of standard errors at the individual level. Furthermore, using the mean grade at the end of the master’s
degree as a long-run academic outcome without considering the extra points (up to 6) attributed to the discus-
sion of the final dissertation, the results (available on request) are qualitatively the same: we do not find any stat-
istically significant effect of active teaching style on students’ long-run performance.
16. As motivated by Huebener, Kuger, and Marcus (2017) in relation to the impact of additional instructional hours on
students’ performance, detecting heterogeneities in the effectiveness of school input factors is even more impor-
tant for policy makers in a context of scarce resources.
17. Indirect evidence can be drawn from the OECD PISA data, which show that, in Italy, as in all the OECD countries,
girls outperform boys in terms of collaborative problem solving. In Italy, in 2015, the mean score for this skill was
489 for girls compared with 466 for boys.
18. Notably, gender differences in the relationship between teaching practices and academic achievement are of par-
ticular interest for policy makers and should be taken into account when universities discuss whether to introduce
innovative pedagogies into their teaching supply. See Dahmann (2017) for a discussion on the relevance of
gender differences in evaluating changes in instructional time as well as in the timing of instruction.
19. The share of students who earned their bachelor’s degree within the prescribed legal time is 97.4 per cent among
students from high schools with an academic track and 96.9 per cent for the other students. However, as in the
case of gender, this difference is not statistically significant.
20. The OECD PISA data actually show that, while Italian students are below the OECD average in the mean score of
collaborative problem solving (in 2015, 478 and 500, respectively), Italian students from high schools preparing
for college are above the OECD average (511) and outperform students from technical (469) and vocational
schools (425).
21. At the end on the English course in the bachelor’s programme, students receive only a ‘pass/no-pass’ evaluation.
Hence, we cannot use the final grade in this course as a measure of pre-treatment English proficiency.
22. This difference is strongly driven by a highly productive foreign professor (whose H-index is equal to 16 in Scopus
and 41 in Google Scholar, compared with the average of the other professors on the same course of, respectively,
3.3 and 6).
Acknowledgement
We thank Giorgio Brunello, Annalisa Cristini, Simona Comi and Marcello Puca for their useful suggestions. Comments by
participants at the LEER Workshop in Education Economics (March 2017), the AIEL conference (September 2017), and at
the seminars at University of Milano Bicocca and University of Bergamo are much appreciated. We also acknowledge
support to our research from the Centre for Young and Family Enteprise (CYFE) of the University of Bergamo.
Disclosure statement
No potential conflict of interest was reported by the author(s).
References
Algan, Y., P. Cahuc, and A. Shleifer. 2013. “Teaching Practices and Social Capital.” American Economic Journal: Applied
Economics 5 (3): 189–210. doi:10.1257/app.5.3.189
Angrist, J., and J. Pischke. 2008. Mostly Harmless Econometrics. Princeton, NJ: Princeton University.
Aslam, M., and G. Kingdon. 2011. “What can Teachers do to Raise Pupil Achievement?” Economics of Education Review 30
(3): 559–574. doi:10.1016/j.econedurev.2011.01.001
Bietenbeck, J. 2014. “Teaching Practices and Cognitive Skills.” Labour Economics 30: 143–153. doi:10.1016/j.labeco.2014.
03.002
Bloom, H. S. 2008. “The Core Analytics of Randomized Experiments for Social Research.” In The SAGE Handbook of Social
Research Methods, edited by P. Alasuutari, L. Bickman, and J. Brannen, 115–134. London, UK: SAGE Publications Ltd.
Boyd, D., P. Grossman, H. Lankford, S. Loeb, and J. Wyckoff. 2006. “How Changes in Entry Requirements Alter the Teacher
Workforce and Affect Student Achievement.” Education Finance and Policy 1 (2): 176–216. doi:10.1162/edfp.2006.1.2.
176
Brand, M., I. Wakkee, and M. van der Veen. 2007. “Teaching Entrepreneurship to non-Business Students: Insights From two
Dutch Universities.” In Handbook of Research in Entrepreneurship Education: Contextual Perspectives. Volume 2, edited
by A. Fayolle, 52–83. Cheltenham, UK; Northampton, MA: Edward Elgar Publishing.
Cappellari, L. 2004. High School Types, Academic Performance and Early Labour Market Outcomes. IZA Discussion Paper No.
1048.
Comi, S., G. Argentin, M. Gui, F. Origo, and L. Pagani. 2017. “Is it the way They use it? Teachers, ICT and Student
Achievement.” Economics of Education Review 56: 24–39. doi:10.1016/j.econedurev.2016.11.007
Dahmann, S. C. 2017. “How Does Education Improve Cognitive Skills? Instructional Time Versus Timing of Instruction.”
Labour Economics 47: 35–47. doi:10.1016/j.labeco.2017.04.008
De Paola, M., F. Gioia, and V. Scoppa. 2018. Teamwork, Leadership and Gender. IZA Discussion Paper No. 11861.
Dee, T. S. 2005. “A Teacher Like me: Does Race, Ethnicity, or Gender Matter?” American Economic Review 95 (2): 158–165.
doi:10.1257/000282805774670446
Dee, T. S. 2007. “Teachers and the Gender Gaps in Student Achievement.” Journal of Human Resources XLII (3): 528–554.
doi:10.3368/jhr.XLII.3.528
De Witte, K., and C. Van Klaveren. 2014. “How are Teachers Teaching? A Nonparametric Approach.” Education Economics
22 (1): 3–23. doi:10.1080/09645292.2011.560448
Di Gregorio, D., and S. Shane. 2003. “Why do Some Universities Generate More Start-ups Than Others?” Research Policy 32
(2): 209–227. doi:10.1016/S0048-7333(02)00097-5
Di Prete, T., and J. Janning. 2012. “Social and Behavioral Skills and the Gender gap in Early Educational Achievement.”
Social Science Research 41 (1): 1–15. doi:10.1016/j.ssresearch.2011.09.001
Dobbie, W., and R. G. Fryer. 2013. “Getting Beneath the Veil of Effective Schools: Evidence From New York City.” American
Economic Journal: Applied Economics 5: 28–60. doi:10.1257/app.5.4.28
Duval-Couetil, N. 2013. “Assessing the Impact of Entrepreneurship Education Programs: Challenges and Approaches.”
Journal of Small Business Management 51 (3): 394–409. doi:10.1111/jsbm.12024
Eddy, S., S. Brownell, and M. Wenderoth. 2014. “Gender Gaps in Achievement and Participation in Multiple Introductory
Biology Classrooms.” CBE – Life Sciences Education 13 (3): 478–492. doi:10.1187/cbe.13-10-0204
Elert, N., F. W. Andersson, and K. Wennberg. 2015. “The Impact of Entrepreneurship Education in High-School on Long-
Term Entrepreneurial Performance.” Journal of Economic Behaviour & Organization 111: 209–223. doi:10.1016/j.jebo.
2014.12.020
Emerson, T. L. N., and B. A. Taylor. 2004. “Comparing Student Achievement Across Experimental and Lecture-Oriented
Sections of a “Principles of Microeconomics” Course.” Southern Economic Journal 70 (3): 672–693. doi:10.2307/4135338
European Commission. 2008. Entrepreneurship in Higher Education, Especially within Non-Business Studies. Final Report of
the Expert Group, March.
Falck, O., C. Mang, and L. Woessmann. 2018. “Virtually no Effect? Different Uses of Classroom Computers and Their Effect
on Student Achievement.” Oxford Bulletin of Economics and Statistics 80: 1–38. doi:10.1111/obes.12192
Fayolle, A. 2013. “Personal Views on the Future of Entrepreneurship Education.” Entrepreneurship and Regional
Development 25: 692–701. doi:10.1080/08985626.2013.821318
Furdas, M., and K. Kohn. 2010. What’s the Difference?! Gender, Personality and the Propensity To Start a Business. IZA
Discussion Paper No. 4778.
Hanushek, E. A. 1992. “The Trade-off Between Child Quantity and Quality.” Journal of Political Economy 100 (1): 84–117.
doi:10.1086/261808
Hanushek, E. A., and S. G. Rivkin. 2010. “Generalizations About Using Value Added Measures of Teacher Quality.” American
Economic Review 100 (2): 267–271. doi:10.1257/aer.100.2.267
Hanushek, E., G. Schwerdt, L. Woessmann, and L. Zhang. 2017. “General Education, Vocational Education, and Labor-
Market Outcomes Over the Lifecycle.” Journal of Human Resources 52 (1): 48–87. doi:10.3368/jhr.52.1.0415-7074R
Harris, D. N., and T. R. Sass. 2011. “Teacher Training, Teacher Quality and Student Achievement.” Journal of Public
Economics 95 (7–8): 798–812. doi:10.1016/j.jpubeco.2010.11.009
Hidalgo-Cabrillana, A., and C. Lopez-Mayan. 2018. “Teaching Styles and Achievement: Student and Teacher Perspectives.”
Economics of Education Review 67: 184–206. doi:10.1016/j.econedurev.2018.10.009
Huebener, M., S. Kuger, and J. Marcus. 2017. “Increased Instruction Hours and the Widening gap in Student Performance.”
Labour Economics 47: 15–34. doi:10.1016/j.labeco.2017.04.007
Johansen, V. 2014. “Entrepreneurship Education and Academic Performance.” Scandinavian Journal of Educational
Research 58 (3): 300–314. doi:10.1080/00313831.2012.726642
Kane, T. J., J. E. Rockoff, and D. O. Staiger. 2006. What Does Certification Tell Us About Teacher Effectiveness? Evidence from
New York City. NBER Working Paper No. 12155.
Kerr, W. R., R. Nanda, and M. Rhodes-Kropf. 2014. “Entrepreneurship as Experimentation.” Journal of Economic Perspective
28 (3): 25–48. doi:10.1257/jep.28.3.25
Koellinger, P., M. Minniti, and C. Schade. 2013. “Gender Differences in Entrepreneurial Propensity.” Oxford Bulletin of
Economics and Statistics 75 (2): 213–234. doi:10.1111/j.1468-0084.2011.00689.x
Korbel, V., and M. Paulus. 2018. “Do Teaching Practices Impact Socio-Emotional Skills?” Education Economics 26 (4): 337–
355. doi:10.1080/09645292.2018.1460320
Lavy, V. 2011. What Makes an Effective Teacher? Quasi-Experimental Evidence. NBER Working Paper No. 16885.
Leuven, E., M. Lindahl, H. Oosterbeek, and B. V. Wedding. 2007. “The Effect of Extra Funding for Disadvantaged Pupils on
Achievement.” Review of Economics and Statistics 89 (4): 721–736. doi:10.1162/rest.89.4.721
Minniti, M. 2009. “Gender Issue in Entrepreneurship.” Foundations and Trends in Entrepreneurship 5 (7–8): 497–621. doi:10.
1561/0300000021
Newhouse, D., and K. Beegle. 2006. “The Effect of School Type on Academic Achievement: Evidence From Indonesia.”
Journal of Human Resources XLI (3): 529–557. doi:10.3368/jhr.XLI.3.529
Nonaka, I., and G. von Krogh. 2009. “Tacit Knowledge and Knowledge Conversion: Controversy and Advancement in
Organizational Knowledge Creation Theory.” Organizational Science 20 (3): 635–652. doi:10.1287/orsc.1080.0412
Oosterbeek, H., M. Van Praag, and A. Ijsselstein. 2010. “The Impact of Entrepreneurship Education on Entrepreneurship
Skills and Motivation.” European Economic Review 54: 442–454. doi:10.1016/j.euroecorev.2009.08.002
Prince, M. 2004. “Does Active Learning Work? A Review of the Research.” Journal of Engineering Education 93 (3): 223–231.
doi:10.1002/j.2168-9830.2004.tb00809.x
Rasmussen, E. A., and R. Sorheim. 2006. “Action-based Entrepreneurship Education.” Technovation 26: 185–194. doi:10.
1016/j.technovation.2005.06.012
Rivkin, S. G., E. A. Hanushek, and J. F. Kain. 2005. “Teachers, Schools, and Academic Achievement.” Econometrica 73 (2):
417–458. doi:10.1111/j.1468-0262.2005.00584.x
Salas, E., J. L. Wildman, and R. F. Piccolo. 2009. “Using Simulation-Based Training to Enhance Management Education.”
Academy of Management Learning and Education 8 (4): 559–573.
Schwerdt, G., and A. C. Wuppermann. 2011. “Is Traditional Teaching Really all That bad? A Within-Student Between-
Subject Approach.” Economics of Education Review 30: 365–379. doi:10.1016/j.econedurev.2010.11.005
Scott Cardell, N., R. Fort, W. Joerding, F. Inaba, D. Lamoreaux, R. Rosenman, E. Stromsdorfer, and R. Bartlett. 1996.
“Laboratory-based Experimental and Demonstration Initiatives in Teaching Undergraduate Economics.” American
Economic Review 86 (2): 454–459.
Souitaris, V., S. Zerbinati, and A. Al-Laham. 2007. “Do Entrepreneurship Programs Raise Entrepreneurial Intention of
Science and Engineering Students? The Effect of Learning, Inspiration and Resources.” Journal of Business Venturing
22 (4): 566–591. doi:10.1016/j.jbusvent.2006.05.002
Tatum, H., B. Schwartz, P. Schimmoeller, and N. Perry. 2013. “Classroom Participation and Student–Faculty Interactions:
Does Gender Matter?” Journal of Higher Education 84 (6): 745–768. doi:10.1353/jhe.2013.0036
Van der Luis, J., M. Van Praag, and W. Vijverberg. 2008. “Education and Entrepreneurship Selection and Performance: A
Review of the Literature.” Journal of Economic Survey 22 (5): 795–841. doi:10.1111/j.1467-6419.2008.00550.x
Van Klaveren, C. 2011. “Lecturing Style Teaching and Student Performance.” Economics of Education Review 30: 729–739.
doi:10.1016/j.econedurev.2010.08.007
Von Graevenitz, G., D. Harhoff, and R. Weber. 2010. “The Effects of Entrepreneurship Education.” Journal of Economic
Behavior & Organization 76: 90–112. doi:10.1016/j.jebo.2010.02.015
Walter, S. G., and D. Dohse. 2012. “Why Mode and Regional Context Matter for Entrepreneurship Education.”
Entrepreneurship & Regional Development 24 (9–10): 807–835. doi:10.1080/08985626.2012.721009
Zantow, K., D. S. Knowlton, and D. C. Sharp. 2005. “More Than fun and Games: Reconsidering the Virtues of Strategic
Management Simulations.” Academy of Management Learning and Education 4 (4): 451–458. doi:10.5465/amle.2005.
19086786
Copyright of Education Economics is the property of Routledge and its content may not be
copied or emailed to multiple sites or posted to a listserv without the copyright holder's
express written permission. However, users may print, download, or email articles for
individual use.

2020-Who Gains From Active Learning in Higher Education

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2020-Who Gains From Active Learning in Higher Education

Uploaded by

Copyright:

Available Formats

EDUCATION ECONOMICS

2020, VOL. 28, NO. 3, 311–331

Who gains from active learning in higher education?

ABSTRACT ARTICLE HISTORY

3. Data and institutional context

Table 2. Sample mean characteristics by track and curriculum.

5.1. Diﬀerence-in-diﬀerence results

5.2. Triple-diﬀerence results

5.3. Heterogeneous eﬀects

5.4. Robustness checks

Table 8. Teachers’ characteristics by entrepreneurship course.

You might also like