You are on page 1of 31

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/360188655

Prolonged Emergency Remote Teaching: Sustainable E-Learning or Human


Capital Stuck in Online Limbo?

Article  in  Sustainability · April 2022


DOI: 10.3390/su14084584

CITATION READS

1 347

5 authors, including:

Petar Vrgović Jasmina Pekić


University of Novi Sad University of Novi Sad
30 PUBLICATIONS   143 CITATIONS    33 PUBLICATIONS   45 CITATIONS   

SEE PROFILE SEE PROFILE

Milan Mirkovic Andras Anderla


University of Novi Sad University of Novi Sad
27 PUBLICATIONS   164 CITATIONS    34 PUBLICATIONS   1,573 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Predictive churn modeling View project

Upgrading SMEs Innovation Success Through Open Innovation View project

All content following this page was uploaded by Jasmina Pekić on 26 April 2022.

The user has requested enhancement of the downloaded file.


sustainability

Article
Prolonged Emergency Remote Teaching: Sustainable
E-Learning or Human Capital Stuck in Online Limbo?
Petar Vrgović 1 , Jasmina Pekić 2 , Milan Mirković 1 , Andraš Anderla 1 and Bojan Leković 3, *

1 Department of Industrial Engineering and Management, Faculty of Technical Sciences,


University of Novi Sad, 21000 Novi Sad, Serbia; vrgovic@uns.ac.rs (P.V.); mmirkov@uns.ac.rs (M.M.);
andras@uns.ac.rs (A.A.)
2 Department of Psychology, Faculty of Philosophy, University of Novi Sad, 21000 Novi Sad, Serbia;
jpekic@ff.uns.ac.rs
3 Department of Management, Faculty of Economics in Subotica, University of Novi Sad, 24000 Subotica, Serbia
* Correspondence: bojan.lekovic@ef.uns.ac.rs; Tel.: +381-24-628-143

Abstract: We introduce the term “prolonged emergency remote teaching” to accentuate the temporal
aspect of the current educational crisis. In-depth lived experiences of students and teaching staff in
one engineering education institution were examined, in order to depict what being involved in an
exclusively online communication feels like from the personal perspective, thus examining e-learning
sustainability during the COVID-19 pandemic. Interviews with samples of students and teaching staff
were conducted, and then psychological phenomenology was employed to produce rich descriptions
of their personal experiences and impressions. Both the students and the faculty employees observed
e-learning during the Fall semester of 2020 as challenging and artificial. Although trying to function
as everything is in order, as the semester unfolded, motivation deteriorated, communication turned
more and more superficial, and limitations became more prominent. For both sides, although there

 were some conveniences, the educational process yielded sub-optimal results and was hard to sustain;
Citation: Vrgović, P.; Pekić, J.; intellectual and social capital of both sides appeared to be underutilized. We present conclusions and
Mirković, M.; Anderla, A.; Leković, B. recommendations, suggesting that modern technologies should help both sides to adapt to the new
Prolonged Emergency Remote reality, instead of feeling stuck in an online “limbo”.
Teaching: Sustainable E-Learning or
Human Capital Stuck in Online Keywords: sustainable e-learning; COVID-19; phenomenological study; lived experiences
Limbo? Sustainability 2022, 14, 4584.
https://doi.org/10.3390/su14084584

Academic Editors: Rollnik-Sadowska


Ewa and Marko Slavkovic 1. Introduction

Received: 7 March 2022


Impact of the COVID-19 pandemic on higher education is unprecedented and multi-
Accepted: 7 April 2022
fold [1]. In the spring of 2020, while classrooms were emptied and lockdowns were imposed,
Published: 12 April 2022
it was expected from universities, professors, and students to adapt to new circumstances
and to continue achieving educational goals [2]. Educational systems around the world
Publisher’s Note: MDPI stays neutral
swiftly shifted to “emergency remote teaching” (ERT) [3], meaning that the institution and
with regard to jurisdictional claims in
its users communicated at a distance during the crisis. Then, the autumn of 2020 witnessed
published maps and institutional affil-
a more organized approach as everybody had more time to prepare both technically and
iations.
mentally for another semester that was to be realized remotely. However, we observe that
in the autumn semester of 2020, the principles on which ERT usually resides on are not
applicable anymore, as the shift is no longer “temporary” and both the service providers
Copyright: © 2022 by the authors.
(teaching staff) and service consumers (students) require not only a temporary fix, but
Licensee MDPI, Basel, Switzerland. substantial support for distance learning for a longer period of time. Thus, in this study,
This article is an open access article we propose the phrase “prolonged emergency remote teaching” (PERT) to accentuate the
distributed under the terms and temporal spanning of this crisis, the fact that it is not apparent when it will end, and the
conditions of the Creative Commons need to assess the sustainability of engaging human capital in these circumstances.
Attribution (CC BY) license (https:// Questions related to the sustainability of e-learning that were already raised before
creativecommons.org/licenses/by/ the pandemic—what are the factors that influence sustainable e-learning [4] and what
4.0/). are the challenges to sustainability of e-learning initiatives [5]—are currently even more

Sustainability 2022, 14, 4584. https://doi.org/10.3390/su14084584 https://www.mdpi.com/journal/sustainability


Sustainability 2022, 14, 4584 2 of 30

exacerbated. There is no compelling evidence that the COVID-19 pandemic may completely
end soon, or that it will stop influencing education [6]; thus, it was our intention to
investigate distance learning as a “new normal” phenomenon, while emphasizing the
paradox between the words “new” and “normal”, which translates into the feeling of being
stuck in a state of limbo [7]. In this context, the state of limbo means that all sides are
expected to act normally, but the reality is that nothing is “normal”, as both the students
and the teaching staff do not essentially have any options to choose from—only a year ago,
hardly any of them intended to engage in distance learning for a prolonged period of time.
It is not uncommon to hear students complaining how they “tasted freedom” and now they
are “stuck” in their childhood bedrooms with their parents [8], and it is not uncommon to
hear professors admitting to how working from home was appealing when it was optional
but is quite challenging if it is the only option [9].
A significant body of knowledge is already present regarding higher education in
the COVID-19 era [10], and the usage and adaptation of ICT and mobile technologies to
enhance distance learning [11,12]. However, there is still not much information about
students’ and professors’ personal impressions and emotions in this context, especially
in the engineering field of studies, as most research focuses on either sides’ perceptions
and evaluations of educational content, process, or outcome. We witnessed that although
students and professors may find their learning/teaching environment, internet quality, and
physical surroundings adequate in this era, many of them still feel lost and hopeless [13],
and find this experience can be overwhelming [14].
Recent publications dominantly employ some sort of a quantitative approach; even the
ones that observe qualitative data tend to report their findings in the forms of classifications
or quantities of dominant emotions, answering to the question of “how much”, rather than
to the questions of “what” and “how”. This research intends to fill this gap by describing
in-depth “lived experiences” [15] (p. 57) of both the students and the teaching staff in
one engineering higher education institution during the PERT process. A qualitative
psychological phenomenology approach was used to process interview transcriptions in
order to understand and describe the phenomenon of “experiencing PERT” from both
sides. Here, we define this phenomenon as “participating in an educational process
where the subject’s initial intention was to attend/teach classes in person, but due to
external circumstances was at one point forced to shift to distance learning/teaching due
to an unforeseen emergency that lasts for unspecified and relatively long period of time”.
Separate research goals were defined for both cohorts, resulting in two studies being
conducted in parallel:
• Describe engineering students’ personal impressions and the feelings that they experi-
ence while attending PERT courses—study 1.
• Describe engineering teachers’ personal impressions and the feelings that they experi-
ence while conducting PERT courses—study 2.
This research aims to explore the aforementioned issues in light of the significant effect
that the COVID-19 pandemic has on human capital around the world [16,17], as recent
global projections on the impact of school closures linked to this pandemic suggest that
almost 0.6 years of schooling (adjusted for quality) will be lost due to education establish-
ment closures [18]. This research contributes to the current literature in a number of ways.
First, it introduces the term PERT in order to describe an ERT process that is indefinitely
prolonged. Second, it provides deep insights into personal experiences and impressions
of students and teaching staff in a completely new context that is globally relevant, but
still not fully described in the literature. Third, as individual and social requirements
are highly relevant for the sustainability of e-learning systems [19], this research helps in
understanding the process and limitations of engineering education during PERT from
personal viewpoints of the main actors by employing the phenomenological approach.
Additionally, we provide specific suggestions for improving prolonged emergency remote
teaching for engineering students.
Sustainability 2022, 14, 4584 3 of 30

The rest of the paper is organized as follows. Section 2 presents a brief literature review
related to distance learning in higher engineering education in general and in the context of
the COVID-19 pandemic. Section 3 describes the research setting for both studies. Section 4
provides details about the research methods. Section 5 provides detailed descriptions
of themes identified in both studies and the resulting textural–structural descriptions of
the phenomenon. Section 6 discusses the findings, concludes the research, and offers
suggestions for researchers and practitioners.

2. Literature Review
2.1. Sustainable E-Learning and E-Learning in Higher Engineering Education
Over the past two decades, e-learning has found its way to most higher education
institutions around the world, making information and communication technologies (ICTs)
an important constitutional element of educational infrastructure. The new generations
frequently prefer mobile applications and video content over the traditional forms of
learning, since they find the new technologies to be more convenient and interesting [20].
E-learning efficiency has been interpreted in light of a theoretical approach that emphasizes
the connection of ICTs with the modern constructivist paradigm in education—online in-
struction can be designed to foster students‘ active construction of knowledge instead of the
passive adoption of ready-made information, which characterizes the classic “instructivist”
pedagogy [21], mostly because of its versatility, interactivity and adaptivity to individual
student’s needs. Modern forms of education, such as E-learning, are much closer to young
peoples’ everyday habits related to use of technology, than a traditional classroom setting.
Furthermore, as ICTs have become an integral part of the human living space with the
Internet being a natural element of life for new generations, e-learning needs to evolve
into smart education, at least in the field of engineering education [22]. Hence, it may be
concluded that e-learning is already a part of the “new normal” in course delivery both
in terms of the technological modernization of the learning process, as well as in terms of
compliance with modern pedagogical paradigms [23].
However, e-learning adoption and continuation are not to be mistaken or taken for
granted. Gunn [5] states that in order for e-learning to be considered as sustainable, the
following conditions should be met:
• A learning design has been developed and implemented within the course of study,
and judged to be beneficial to teaching and learning (i.e., for both sides of the process).
• E-learning resources, design, and concept have the potential to be adapted and adopted
for use beyond the original environment in which they were developed.
• Further use and further development of the e-learning resources, design, and concept
should not remain dependent on the individuals who created them.
The same author concludes that not all e-learning initiatives are sustainable, as the
successful proof-of-concept stage does not guarantee that, in the longer time span, these
processes will lead to desired outcomes. Trentin [4] proposed the eight-dimensional model
for e-learning sustainability, comprising these dimensions: pedagogical, professional, socio-
cultural, informal, technological, economical, organizational and content; all dimensions
should be properly saturated if organization desires the educational process to sustain be-
yond initial enthusiasm. Alharthi et al. performed a systematic literature review observing
sustainability from similar perspectives as Trentin: human, social, technical, environmental,
and economic dimensions; they offer a comprehensive list of studies related to topic of
sustainable e-learning systems and conclude their analysis by identifying 18 high-level
sustainability meta-requirements for e-learning within software required for e-learning [19]:
personalization, learner-centered features, collaboration, leadership development, privacy
and security, analysis of the learning materials, reuse of the learning materials, integration
with social networks, standardization of the LORs, support for LORs, support of shared
services, software quality requirements, portability, modularity, cloud computing, green
and sustainable SE, reducing the cost, and ensuring the growth.
Sustainability 2022, 14, 4584 4 of 30

Despite theoretical and empirical evidence on the effectiveness of e-learning in the


context of higher education, there is a discrepancy between the growing online learning
practice in “narrative” disciplines (such as management or economy), and insufficient
incorporation of ICTs within learning strategies in the field of technical sciences [24]. Ex-
planations for such an unequal representation of online learning across different scientific
fields are multi-fold. First, technical fields, such as electrical engineering and mechanical
engineering, involve practical work in special laboratories in which students conduct physi-
cal experiments with real equipment; in such conditions full online learning is possible only
if there are “virtual laboratories” [24]. Second, the traditional engineering curriculum is
also designed to require competent mastery of mathematics and theoretical physics before
enrolling in engineering courses, which is difficult to perform in the total absence of the
face-to-face sessions [23]. Third, applying e-learning systems for engineering education
comes with a significant set of challenges and obstacles for all the parties included in the
process [25]. Finally, while many authors provide descriptions and suggestions that en-
gagement is crucial to learning, there is no widespread, standard definition for engagement
to guide engineering educators, as engineering students and faculty members do not have
the same perception of engagement. While students view engagement in terms of teaching
and faculty enthusiasm for the subject, faculty members believe that engagement rests with
the students [26].
Especially in the COVID-19 era, there is evidence that many higher education institu-
tions are not well prepared for an emergency crisis such as this one, lacking a permanent or
dedicated emergency management office, and lacking a general business continuity plan
for an emergency [27]. Both the management of education institutions and the education
policies should be more alert and responsive to such dramatic events [28], but they are
usually highly dependent on a number of complex factors such as management’s leadership
skills [29], which explains why institutions around the world have significantly different
reactions and results during the current pandemic.
Still, advances in technology over the years have permitted the representation of
complex structures and objects by computers. Thus, e-learning resources of diverse types
became available to engineering students to a greater extent. There are at least three kinds
of such e-learning resources: (1) digital or digitized content available for downloading
or studying online; (2) learning objects; (3) multi-user, dynamic and interactive learning
environments [23]. Some research studies aimed to determine the effect of such environ-
ments on different aspects of the learning process: the results obtained show that while
effective learning decreased for students in the online course format relative to the tradi-
tional course format, cognitive learning was comparable [30]. When it comes to special
laboratories, many institutions have created their own virtual and remote laboratories that
enable autonomous students’ learning activities in various disciplines [23,31], and even
virtual laboratory systems [32], that were also proven to be effective during the current
pandemic [33,34].
Relevant literature offers insights into research that compares virtual with physical
laboratories in the domain of students’ perception of key cognitive processes and specific
content afforded by different mediums of the laboratories [35]. The obtained results suggest
that many students were able to suspend disbelief in virtual laboratories and demonstrated
psychological presence, which creates a potential for a rich learning experience. When it
comes to potential difficulties in online learning, which could be caused by differences in
students’ and educators’ perception of engagement, it is good to know that online courses
designed to support feedback and formative assessment during the learning process have
a positive effect on learning outcomes and future attitudes toward the course content for
both engineering teachers and their students [36].

2.2. COVID-19 Pandemic and Higher Education


During the COVID-19 pandemic, higher education systems around the world were
faced with requirements of an ad hoc revision of learning and teaching methods in the
Sustainability 2022, 14, 4584 5 of 30

context of lockdowns, social distancing, and travel bans. Although a few countries expe-
rienced similar challenges during the SARS outbreak in 2003, where e-learning showed
potential for substituting some of the traditional course elements [37], the scale and length
of the new pandemic were unprecedented in the recent history of education. The only
possible solution for further continuation of the educational process was the complete
transition to remote teaching and online learning. It was quickly observed, however, that
this emergency response was not to be confused with classic online learning or remote
teaching programs [38]. ERT is a temporary shift of instructional delivery to an alternate
delivery mode due to accidental crisis circumstances, which is reflected in the realization of
all forms of educational activities within the online-only course format—within ERT, the
emphasis is not on recreating a robust educational ecosystem, but on providing temporary
access to instruction and instructional support in a manner that is quick to set up and is
reliably available during the extraordinary circumstances with the sign of emergency or
crisis [3]. This means that ERT has different dynamics, expectations and evaluation criteria
than standard online, blended and face-to-face instructional activities. Therefore, it can
be said that classical e-learning was recently forced to make another turn—from a “new
normal” paradigm toward the “online-only” learning situation that may represent another
“new normal” that will need to continue for some time.
Although less than two years have passed between the start of the COVID-19 pan-
demic and the moment in which this research was written, there are already hundreds of
journal and conference papers that deal with the topic of education during the current crisis.
Mostly, these papers cover initial reaction of educational institutions or governments, gen-
erating important, but often inconsistent conclusions. When it comes to assessing student
performance within the new learning format, it is still difficult to generate unambiguous
conclusions. For example, research studies of this type show that students prefer face-to-
face over online learning, while their motivation, self-efficacy, and cognitive engagement
decreased after the transition to online-only learning [39], as well as that deficient computer
skills and the perception of a higher workload prevent students from perceiving their own
improved performance in the new teaching environment [40]. Conversely, other findings
suggest that students improved their performance in the context of emergency remote
teaching, when compared with a cohort from the previous year [41].
Usage of ICT and distance/mobile learning is already addressed by many papers;
Saikat et al. [11] provide a systematic review of the current literature and conclude that:
the majority of academic institutions were hesitant to employ these methods for educa-
tion; due to a lack of time, they were compelled to use third-party platforms instead of
developing their own; both teachers and students significantly benefit from guidelines and
instructions on how to use these platforms, if provided. Althunibat et al. [42] propose and
confirm a robust model that includes a dozen of factors that affect mobile learning system
usage, spanning from technological issues, through organizational aspects, all the way to
user satisfaction.
When it comes to engineering education during the COVID-19 pandemic, relevant
literature shows that researchers in this field have been particularly interested in aspects of
adaptation to ERT. For example, researchers investigated factors that influence engineering
students’ willingness to adapt to the emergency online learning environment, emphasizing
the importance of several factors such as: initial preparedness and motivation for online
learning, self-efficacy beliefs about online learning, self-directed learning online, and insti-
tutional support [43]. Another study reveals that students are prone to using self-discipline
strategies to overcome adaptation challenges, while, conversely, support of the faculty
through a compassionate and flexible pedagogy is important in the process of transition
to the new learning format [44]. A recent study documented that this is sometimes not
enough—more than half of the observed engineering students from California indicated
lack of engagement in class, significant difficulty in maintaining their focus, and “Zoom fa-
tigue” after attending multiple online sessions [45]. In addition, research findings show that
learning during the pandemic has raised many concerns for future adaptations of online
Sustainability 2022, 14, 4584 6 of 30

learning among engineering students: students are generally disappointed by their experi-
ences of online learning effectiveness during the pandemic, due to the absence of adequate
interactions amongst students, caused by the reduction in informal student engagement
opportunities [46]; engineering students also experience systemic challenges and have a
limited scope of learning through a “virtual only” context [47]. Methods such as “flipped
classroom” may significantly help when combined with adaptive learning, thus reducing
load, burden, and stressors in the online environment of engineering education [48], while
recent research hints that in the near future, and in the current post-COVID-19 educational
scenario, virtual labs and real laboratories will coexist within the new hybrid models that
combine face-to-face and online teaching and learning [49].
A few studies observe the teaching staff’s opinions about transitioning to ERT during
the COVID-19 pandemic. For example, El Said [50] documented that university professors
in Egypt reacted positively to this shift, but that they perceived lecture recordings as
being the cause of poor class attendance and class participation. Hjelsvold et al. [51]
report that Norwegian university professors also have mostly positive experience as a first
reaction to transforming to online teaching during lockdown; however, they perceived
a significant lack of students’ interaction. By interviewing professors from Central and
Eastern Europe, Popa et al. [52] show that teaching staff lists platform-related problems,
digital skills, and interactivity as main challenges in the first semester of 2020. Durak and
Çankaya [53], as well as Mohmmed et al. [54], recorded that university professors in the
Middle East are satisfied with using Microsoft Teams platform for communicating with
students remotely. Finally, a few studies describe processes of rapid transition to distance
learning in engineering courses [55], and migration of active learning classroom online [56],
also providing insights and recommendations for other engineering education institutions.

3. Research Setting: Timeframe, Study Site, Pre-Pandemic Classroom Scenario, and


Remote Teaching during the COVID-19 Pandemic
Since this research aimed to explore experiences of both sides of the education process,
two studies have been conducted in parallel: study 1 was focused on a sample of engineer-
ing students, while study 2 observed the teaching staff in the field of engineering studies.
In both studies, the sampling techniques and sample sizes, the general instructions to the
participants, and the specific questions were chosen to achieve a full range of variation in
the set of possible cases, since the goal was to generate a fund of possible elements that can
be used in “determining the essential structure of the phenomena” [57] p. 48.
Data for both studies were collected in January 2021, a week after the end of the winter
semester. This semester was chosen as the most suitable one to observe the phenomenon
of being engaged in PERT—students and teachers were informed from the start that the
remote teaching, caused by the COVID-19 pandemic, will be prolonged indefinitely and
that there was little chance of going back to university classrooms any time soon.
For both studies, samples were drawn from the same institution, Faculty of Technical
Sciences, University of Novi Sad, Serbia (FTS). Different universities around the world have
utilized quite different methods and tools to facilitate remote teaching during the ongoing
pandemic [2], and in different settings regulated by the national governments and other
bodies; thus, it was decided to gather data only from one institution, in order to control as
much variables as possible. FTS was chosen for this study, as one of the largest publicly
owned technical faculties in the south-eastern part of Europe. It has more than 12,000 active
students, and 850 active teaching staff; it offers 28 undergraduate, 34 masters, and 17 PhD
study programs, most of which resulting in an engineering degree.
Prior to the COVID-19 pandemic, students and the faculty staff were almost com-
pletely engaged in face-to-face classes in this institution, attending or teaching classes
in situ, utilizing faculty’s numerous laboratories and engineering workshops daily, and
communicating in person with their peers and professors at the faculty premises. Groups
for practical work mostly comprised 12 to 16 students, and the faculty staff was frequently
present for individual consultations in their offices. Although FTS already had experience
Sustainability 2022, 14, 4584 7 of 30

with some learning management systems, these systems were not used to a large extent by
most of the staff; mostly, they were used for sharing supplementary educational files, or
for keeping the student records. Distance communication between students and the staff
were less frequent, and it was almost always realized via emails. Apart from a handful of
cases, there were no video links to attend classes from a distance, or any recorded lessons
for students to watch at their own pace.
At the start of the winter semester of 2020, the COVID-19 pandemic was showing no
evidence of slowing down. With more than 95% of classes needing to be held remotely,
FTS precisely defined communication channels and education tools for its staff to use:
all synchronous communication between students and teachers (classes and individual
consultations) was to be held via the Microsoft (MS) Teams platform: online classes were
scheduled weekly following the regular timetable. Besides a standard video conferencing
feature that was needed for the teaching and interaction with students, the MS Teams
platform offered usage of different apps such as digital whiteboards, online polls, quizzes,
and similar. In addition, FTS decided to also use the Canvas Learning Management System
(LMS), thus allowing for more effective asynchronous communication between students
and the teaching staff, especially for the purposes of distribution of different teaching
materials and for collecting and grading students’ assignments. Constant technical support
was provided for both MS Teams and Canvas LMS; students and the teaching staff were
able to seamlessly start utilizing these tools, as most of the initial settings and accounts
for all users were pre-set, with numerous usage tutorials available. Despite its size, FTS
did not have many solutions available that could be categorized as “virtual labs”. Only a
handful of courses were held in the faculty labs, workshops, or on terrain, yet the number
of these field classes and their dynamics were significantly reduced in comparison to the
regular plans from previous years (for example, two field classes being held during the
semester, instead of eight that were held the previous year).

4. Methods
Qualitative research methods are usually most suitable when one has an interest
in knowing more about other people’s practice and personal experiences [58]. Open-
ended questions and interviews are then frequently used to collect data, which are evident
in numerous recent studies related to the COVID-19 pandemic influence on education
processes [43,44,52,59,60]. However, the classic content analysis, most frequently used
to process qualitative data, usually ends up in quantification of the observed responses,
providing frequencies of certain topics, which limits the understanding of the underlying
phenomenon—this is because the main question being answered is usually “how much?”
instead of “what it is?”. When an “essence” of respondents’ experiences is sought after,
researchers turn to psychological phenomenological analysis as it aims to capture the
basic structure of a phenomenon, to identify what was similar to experiences of multiple
individuals [58]. Psychological phenomenology is a method that is often most suitable
when researchers want to ferret out the essence of a phenomenon—to understand what is
mutual in the perception of multiple individuals who have experienced the same complex
situation, focusing on the personal impressions and then describing what all participants
have in common as they experience the phenomenon [15].
In this method, impressions of individuals’ “lived experiences” are recorded, and then
they are systematically coded into meaningful units in order to find underlying thematic
structure and mutual topics [58]. This research method was already successfully used
in various studies related to higher education [61,62], in studies related to engineering
students [63], in studies related to distance learning [64], as well as in a few recent ones
related to higher education in the COVID-19 era [65–67], albeit thus far not in the field
of engineering education, and not in the context of prolonged ERT. Phenomenological
analysis removes any political, historical, or other external influences that may interfere
with individuals’ experiences [68]. Its goal is to develop “comprehensive descriptions of
the shared structures underlying a set of individuals” experiences within the context of a
Sustainability 2022, 14, 4584 8 of 30

particular situation, rather than explanations or interpretations [63], which makes it most
suitable to describe experiences of both engineering students and their teachers in the
context of prolonged forced remote education context.

4.1. Data Collection and Participants


A phenomenological interview is usually the most appropriate method of data collec-
tion during a phenomenological analysis, since interviewing allows for direct insights into
individual’s subjective perception of how it felt to experience something of interest [15,58].
In this kind of interview, usually only a few open-ended questions about the phenomenon
are asked, where the dialogue takes the form of semi-structured interview (SSI).
As a part of their course-related research project, nine master’s-level students were
grouped into three junior research teams in order to conduct the interviews and transcribe
the data for both studies, thus helping the three principal researchers to obtain data. They
were informed about the interview procedure in detail: how to conduct the interviews,
how to stimulate participants to share their experiences, how to record the sessions, and
how to transcribe them; they were especially advised on how to avoid suggesting their
own opinions or ideas to the participants. The three teams used both convenience and
snowball sampling to engage potential participants—this meant that they approached their
peers and friends who were enrolled at the same institution and asked them to provide
additional contacts and thus recruit potential new participants. Although these sampling
techniques may be problematic in quantitative studies in terms of representativeness
and generalization, in phenomenological research, they are acceptable since the aim of
a phenomenological study is to identify what is mutual and in common for all study
participants, without a need to obtain a sample with certain properties. Current COVID-19
pandemic-related safety regulations did not allow physical interviews to be held; thus, all
interviews were held online via MS Teams or Zoom video conferencing tools. The data
collection process strictly followed “ethical guidelines” proposed by the British Education
Research Association [69].

4.1.1. Study 1
The 43 participants of the first study were students who were enrolled in either
bachelor, master or PhD studies at FTS: 46.5% were female; 60.5% were at the bachelor
studies level. They were recruited from different study programs in order to take into
account distinct properties that different study programs might possess [46]: most students
were attending study programs such as “Information Systems Engineering”, “Computing
and Control Engineering”, and “Electronics, Energetics and Telecommunications”. The age
of the participants ranged from 18 to 32, with an average of 21.5 (more detailed analysis of
sample from study 1 can be found in Table 1).
After a short introduction and briefing, participants were asked about some general
information about themselves, and the context of their studies (as an ice-breaker activity).
Then, two general questions were presented: “What have you experienced in terms of
studying online during the last semester?” and “What contexts or situations have typically
influenced or affected your experiences of studying online during the last semester?”. After
the participants had responded, they were asked additional questions to explain their
experiences further or in more detail, as the interviewers deemed appropriate.
There were 43 successful interview sessions, spanning from 12 m 56 s to 36 m 21 s,
with a median of 25 m 1 s. Out of 43 interviewed students, 38 reported that all classes they
attended were held remotely via the MS Teams platform, while only 5 reported that they
had 10% of the classes (or labs) held in-person, at the faculty premises. Students stated that
they dominantly used desktop computers or laptops to attend the classes, but almost all of
them explained that in various situations they also used their mobile devices.
Sustainability 2022, 14, 4584 9 of 30

Table 1. Distributions of general information about participants from study 1—students.

Sample Property % of the Sample * n


Male 53.5% 23
Gender
Female 46.5% 20
Yes 32.6% 14
Employed
No 67.4% 29
1st 11.6% 5
2nd 11.6% 5
3rd 16.3% 7
Year of study
4th 20.9% 9
1st Master’s 37.2% 16
1st PhD 2.3% 1
Information Systems Engineering 44.2% 19
Computing and Control Engineering 18.6% 8
Electronics, Energetics and Telecommunications 14.0% 6
Applied Software Engineering 4.7% 2
Industrial Engineering 2.3% 1
Biomedical Engineering 2.3% 1
study Programs
Engineering Management 2.3% 1
Information Systems Engineering 2.3% 1
Geodesy and Geomatics 2.3% 1
Mechatronics 2.3% 1
Software Engineering and IT 2.3% 1
Architecture and Urbanism 2.3% 1
* Note: percentages are rounded to the first decimal, and the number of respondents is given in parentheses.

4.1.2. Study 2
Participants of the second study were professors and teaching assistants employed at
FTS. The first attempts to schedule and conduct SSIs with them were not successful and
were troublesome, as the semester was coming to an end and the exam period was in full
swing, which resulted in poor availability of the staff. Therefore, a decision was made to
use an online questionnaire with a few open-ended questions instead, in order to produce
written reflections [43,53]. Most professors stated that they preferred this method, since
they could write about their experiences at their own convenience. Similar to study 1, both
convenience and snowball sampling methods were used to engage potential participants—
this meant sending invitation emails to three random members from every department,
asking some colleagues via personal connection, and asking the willing participants to
nominate other contacts who might also join the study. Again, the sampling methods
were found to be acceptable since a phenomenological study does not ask for a sample
that would serve the need for representativeness and generalization of quantitative results.
Similar to the first study, our goal was to capture as diverse a sample as possible in terms
of academic roles and positions, gender and courses taught in distinct study programs (it
should be noted that one teacher or teaching assistant may teach courses in multiple study
programs), thus utilizing convenience sampling. Google Forms was used to collect the data
online and anonymously.
At the beginning of the questionnaire, general information about the purpose of the
research was stated, followed by information on how anonymity was being ensured and
how the answers would be processed. Special care was taken to ensure the respondents
understood that questions referred only to the last semester (Autumn 2020).
The participants were then presented with two broad open questions, similar to the
ones in study 1, but from the perspective of the faculty staff: “What have you experienced
in terms of teaching online during the last semester?” and “What contexts or situations
have typically influenced or affected your experiences of teaching online during the last
semester?”. Together with these, additional questions were added as a footnote, to act as an
inspiration for thinking: “Please explain how you felt while teaching a class online?”, “How
Sustainability 2022, 14, 4584 10 of 30

much stress did you experience while teaching online classes?”, “How would you describe
and assess your motivation to teach online in this semester?”, and “Try to describe how
you felt while conveying knowledge to your students and engaging them in the class?”.
Although this method of data collection appears to be a survey on first sight, it did not
significantly differ from the interviews used in study 1—the two presented questions were
quite broad and open, while the additional questions had the function of stimulation for
thinking and were not posted independently, nor were they expected to yield separate
answers. After two weeks, 25 useful reports were received, with word counts spanning
from 424 to 870, with a median of 614. The age of the participants in this study ranged from
23 to 68, with an average of 40.5 (additional properties of the sample analyzed within the
second study are given in Table 2).
Table 2. Distributions of general information about participants from study 2—teachers.

Sample Property % of the Sample * n


Male 56.0% 14
Gender Female 40.0% 10
Undisclosed 4.0% 1
Full Professor 16.0% 4
Associate Professor 36.0% 9
Position Assistant Professor 12.0% 3
Teaching Assistant 32.0% 8
Undisclosed 4.0% 1
Industrial Engineering 17.0% 9
Mechatronics 15.1% 8
Engineering Management 13.2% 7
Mechanical Engineering 11.3% 6
Information Systems Engineering 7.5% 4
Information Engineering 5.7% 3
Software Engineering and IT 5.7% 3
Study programs taught in
Computing and Control Engineering 5.7% 3
(any teacher may teach in
Traffic and Transport Engineering 3.8% 2
more than one study program)
Applied Software Engineering 3.8% 2
Biomedical Engineering 1.9% 1
Clean Energy Technologies 1.9% 1
Architecture 1.9% 1
Applied Computer Sciences 1.9% 1
Geodesy and Geomatics 1.9% 1
Undisclosed 1.9% 1
* Note: percentages are rounded to the first decimal, and the number of respondents is given in parentheses.

4.2. Data Analysis


4.2.1. Psychological Phenomenological Analysis
Next, Creswell’s simplified version [15,63] of Moustakas’s modification of the “Stevick–
Colaizzi–Keen” method of psychological phenomenological analysis [68] was utilized in
both studies, separately, with the following steps in the analysis (depicted in Figure 1):
1. All three principal researchers PR1, PR2, and PR3 (being university professors them-
selves) filtered out their own experiences of the observed phenomenon, in order
to reduce potential bias during the interpretation stage, a process called “bracket-
ing” [58,68]. Here, these principal researchers individually reflected on their own
experiences, and wrote short essays about their personal impressions on teaching
online during the COVID-19 pandemic, thus aiming to set aside their experiences,
as much as possible, and to take fresh perspective toward the phenomenon under
examination [15].
2. Researchers PR1 and PR2 independently read interview transcripts multiple times.
During the reading, relevant content units were identified, marked, and extracted by
Sustainability 2022, 14, 4584 11 of 30

both of them separately. All of these “significant statements” were observed as having
an equal value, without any judgement or value attribution, being perceived as parts
of the same “horizon” of the phenomenon, which is a process named “horizontal-
ization” [68], crucial for phenomenological analysis. Similar “significant statements”
have been identified as repeating ideas [70], and they were written in the third person
(for example, three similar statements: “at the start of the semester I felt enthusiastic,
but a few months later I felt drained”, “as the semester unfolded I had less and less
motivation to work”, “the initial optimism soon gave way to some form of apathy”
have been identified as the following idea: “Teachers’ levels of motivation decrease
during the semester”). This step resulted in lists of relevant, non-repetitive, and
non-overlapping ideas of all participants from the same study.
3. The identified ideas were then grouped into larger meaningful units called “themes”
by employing axial coding, where each axis gathers around itself ideas of similar
meaning [71] (for clarification, in other types of qualitative analysis these meaningful
units are also named “codes”, “categories” or “indices” [72]). This was achieved by
writing all of the identified ideas on “sticky notes”, and then by arranging them on
different axis based on their thematic similarities, thus forming “clusters of mean-
ing” [15] p. 61 (for example, three similar ideas: “Teachers’ levels of work engagement
decrease as the semester unfolds”, “Lack of feedback and students’ engagement is
detrimental for teachers’ work morale”, and “Teachers have a lot of enthusiasm at
the start of the semester” have been clustered on the theme “Motivation decreases
the semester unfolds”). Researchers PR1 and PR2 conducted this step independently
from each other.
4. Researcher PR3, the one that did not participate in the previous two steps, had then
compared, combined, and unified reports on themes from the first two researchers,
reconciling the differences and finding mutual patterns. Then, PR1 and PR2 together
commented on the PR30 s work, which resulted in the final list of themes that were
identified from the interviews, together with the associated ideas.
5. Using segments of identified themes that talk about participants’ thoughts, feel-
ings, impressions, and ideas that “portray what comprises an experience” [68] (p.
44), a description of “what” the participants in the study experienced with the phe-
nomenon was written within every theme, explaining what happened to the partici-
pants and how they felt about it—producing what is in phenomenology known as
“textural description”.
6. Using segments of identified themes that allow for insights into the dynamics, sur-
roundings, conditions, place, and various external factors, a description of “how” the
experience happened within the phenomenon was written related to every theme,
describing the setting and context in which the phenomenon was experienced, which
resulted in what is in phenomenology known as “structural description”.
7. A “composite description” of the phenomenon was written by incorporating both
the textural and structural descriptions, producing the description of the “essence” of
the experience.

4.2.2. Validation of Studies’ Results


Qualitative studies, albeit being superior in providing deeper insights into partici-
pants’ personal content, still have some challenges regarding their validity. The validity
of data and validity of data analysis in our two studies have been secured in multiple
ways: first, by employing adequate engagement in data collection (collecting data and
emerging findings until reaching the point of saturation, where similar phrases are be-
ing heard “over and over again, and no new information surfaces” [58] (p. 248); second,
by employing investigator triangulation [73], where two researchers have independently
processed the same datasets, and then the third one had compared their findings; third,
by using respondent validation [74], as feedback on preliminary findings was solicited
from three participants from each study. Additionally, the research procedures in all steps
and how they felt about it—producing what is in phenomenology known as “textural
description”.
6. Using segments of identified themes that allow for insights into the dynamics, sur-
roundings, conditions, place, and various external factors, a description of “how” the
experience happened within the phenomenon was written related to every theme,
Sustainability 2022, 14, 4584 describing the setting and context in which the phenomenon was experienced,12which of 30
resulted in what is in phenomenology known as “structural description”.
7. A “composite description” of the phenomenon was written by incorporating both the
textural against
were checked and structural descriptions,
Polkinghorne’s producing the
“validity-related description
doubts” of 57),
[57] (p. the “essence” of the
and Creswell’s
experience.
“standards for assessing the quality of a phenomenology” [15] (p. 215).

Figure1.1.Creswell’s
Figure Creswell’ssimplified
simplifiedversion
versionofofMoustakas’s
Moustakas’smodification
modificationof
ofthe
the“Stevick–Colaizzi–Keen”
“Stevick–Colaizzi–Keen”
method of psychological phenomenological analysis, as used in both studies.
method of psychological phenomenological analysis, as used in both studies.

5.4.2.2. Validation of Studies’ Results


Results
Following
Qualitativesteps 1–4 of
studies, the analysis
albeit presented
being superior above, 31 deeper
in providing ideas have beeninto
insights identified
partici-
inpants’
studypersonal
1, and seven themes
content, still related
have someto students’
challengesexperiences
regarding have
their been compiled
validity. from
The validity
these ideas;
of data and38validity
ideas have beenanalysis
of data identified
in in
ourstudy
two 2, and eight
studies havethemes related to
been secured inteachers’
multiple
experiences
ways: first,have been generated
by employing fromengagement
adequate them (detailed in lists
dataofcollection
ideas clustered around
(collecting datatheir
and
themes are available from authors upon request). For each study, these themes
emerging findings until reaching the point of saturation, where similar phrases are being are presented
through their textural
heard “over and overdescriptions—what
again, and no newdid the study participants
information surfaces” [58]experience regarding
(p. 248); second, by
that theme (verbatim
employing investigator examples are provided
triangulation in Appendices
[73], where A andhave
two researchers B), and structural
independently
descriptions—how
processed the same thedatasets,
experienceandrelated
then theto that
thirdtheme
one hadhappened.
compared This is followed,
their findings;again
third,
separately for each study, by describing the essence of the phenomenon, i.e., by presenting
the composite textural-structural description of the phenomenon. In the last part of this
section, the study results have been compared to previously established dimensions of
sustainable e-learning systems, revealing positive and negative aspects of PERT from the
perspectives of teachers and engineering students. Additionally, a comparative analysis of
frequent ideas found in both studies was conducted, with results provided in Appendix C
Table A1.
Sustainability 2022, 14, 4584 13 of 30

5.1. Study 1—Engineering Students


5.1.1. Identified Themes—What Did the Students Experience and How Did These
Experiences Happen
Theme 1: Self-discipline, motivation, and workflow
Textural description: Most students reported that at the start of the semester they
needed significant effort and time to organize everything and create a routine; for most,
that was challenging at first, as students hoped that the pandemic would end before the
semester had started. After the first few weeks of online classes, concerns about issues
with their devices and internet connectivity were mostly alleviated, but students reported
having challenges with planning their activities and establishing a routine. Some students
were successful at keeping up with the work related to their studies, while others had
problems in maintaining their discipline. However, as the semester moved on, students
started experiencing more and more stress from assignments and obligations that started
piling up. At the end of the semester, most of the students reported frustration and feeling
of being overwhelmed with tasks that were, in their perception, too demanding for them,
especially since they had to “juggle” between different subjects.
Structural description: Most students attended online classes from their own homes,
usually from their own rooms (less frequently from dormitory rooms); however, it was not
unusual for them to also join in while commuting, driving their car, or from an office if they
were employed. Most of the students needed to make some sort of arrangement with other
household members in order not to be interrupted, and to eliminate distractions as much
as possible. Some students even reported that other family members were sometimes also
listening to an online lecture, out of pure curiosity. They reported that they had significantly
more time during the day since they did not have to prepare for going to the facility or to
commute. Most of the time, the study routine was without problems, although students
sometimes experienced minor technical issues.
Theme 2: Learning experience and outcomes
Textural description: Most students reported that they had a feeling that they learned
significantly less than what would usually be possible, if they had subjects that required
tangible experience of the subject matter. Even more, if students were to attend auditory
practice classes, they felt that they were only passively receiving information. Students
expressed doubts that what they heard was practical enough, explaining that they mostly
do not hold the professors responsible for that perception; rather, they believed that
remote teaching was simply limited in its nature if the activities were limited to talks
and discussions. Conversely, if the course subject was related to any kind of software or
programming language, students mostly felt more convenient attending the class remotely.
Structural description: Lack of practical and lab experience was reported by one part of
the interviewed students, the part that expected this kind of experience during their studies.
Conversely, students who had practical work related to software design pointed out that
this way of learning was more effective for them, as they had better insights into what
TAs were doing via screen sharing, and were able to better follow teacher’s instructions.
Different tools for online teaching such as whiteboards, drawing boards, annotations, and
visualization methods were good for conveying knowledge. All materials for the classes
were available online, but students felt frustrated because they were sometimes scattered
across different parts of the MS Teams and Canvas platforms. Recorded teaching sessions
were seen as convenient, as students were able to review the recordings later. Many
students pointed out that in the courses where they had periodic knowledge checks during
the semester (quizzes, polls, and short tasks), they were able to learn more and better.
Theme 3: Affective reactions
Textural description: At the start of most interviews, most students stated that they
did not experience significant stress, and that they even felt more relaxed because they
were at their own homes and nobody was monitoring them. However, they admitted that
Sustainability 2022, 14, 4584 14 of 30

they were feeling uncomfortable to some extent, because they did not have the experience
of actually being part of an educational process—they stated that they felt weird, as if being
included in something artificial, virtual, or not immersive enough.
Structural description: Students experienced stress, but only sporadically and in
specific situations: mostly, if they had to turn on their cameras and speak in front of
the entire class, as most of them preferred not to turn on their cameras when answering
questions; at the start of the semester when they did not know what to expect or if they
would have technical difficulties; if there was a test approaching.
Theme 4: Peer communication and lack of social interaction
Textural description: To a large extent, students admitted that they missed being in
classrooms and university hallways together with other students. They longed for direct
contact with their peers, communicating during class breaks, or after classes. Not seeing
other students’ faces during the online classes was demotivating and disappointing for
some, as few of them kept their cameras on. For the freshmen, it was disappointing that
they did not have a chance to meet new peers and establish new friendships, which was
especially mentioned by the students who are not living in the city where the faculty is
situated, and who intended to move at the start of the semester. In most cases, students felt
good and inspired to communicate if they were part of a discussion during a class, but still
they found this way of communication to be limited and troublesome, since it was not easy
to jump into a discussion, and usual nonverbal cues were missing.
Structural description: Students communicated remotely to a large extent between
themselves during the semester: to strengthen individual contacts, to collaborate on course
projects, or to share course materials or information. They mostly communicated via
popular social networks or messaging apps, often in virtual groups that they created
themselves. Students mostly ignored and avoided informal peer communication on the
MS Teams platform that the faculty had set up for them, usually finding alternative places
to chat. Students from senior years, who knew their peers from the pre-pandemic era, used
every opportunity to meet in person—when they had a test or any other activity at the
faculty premises, they would frequently grab a drink together afterward.
Theme 5: Support by the teaching staff
Textural description: In most cases, students felt left out when it came to their indi-
vidual needs and questions—simply because there were no predetermined office hours or
the opportunity to approach the professor after the class had ended. Thus, in most cases
there was not much time for clarifications. In addition, students felt that their progress was
not individually tracked or looked after by majority of the professors, that there was less
interaction, and that the overall sense of connection between the two sides was weaker.
They also felt frustrated because they had to “figure a lot of things on their own”, includ-
ing understanding some topics or installing software and development environments,
as there was no possibility to easily ask for help. Students frequently stated that they
believed professors were “doing their best”, “going out of their way”, “very supportive
and forthcoming”, but added that these good intensions had a limited effect.
Structural description: Students stressed that they understood the circumstances and
limitations of distance learning. It was highly appreciated if a professor would leave extra
time when the class was officially over to address individual issues. Many students noted
that they had quite different experiences from one teacher to another—while they felt
supported from some teachers, they felt left behind by some others. Compared to regular
studies, students now reported less support from the teachers and their peers.
Theme 6: Attention and active engagement during a class
Textural description: Level of engagement during online classes was a frequent topic
during the interviews within the students. In some classes, students felt inspired to actively
participate in a class; in other cases, students felt disengaged, aloof, and passive, with their
minds wandering off. Students felt highly dependent on different external factors (the topic,
Sustainability 2022, 14, 4584 15 of 30

professor’s style of teaching, reactions of other students, and structured assignments) that
influenced their levels of engagement. In most cases, students did not experience significant
external distractions, as most of them had organized their living space for this purpose.
Some of them experienced periodical distractions due to different factors such as: other
people who share the same household, ambient or street noise, noise from other video-call
participants who did not mute their microphones, or notifications on their devices. As an
internal factor, students mentioned their own discipline as being crucial for staying focused,
but also added that their current mood was a significant factor. Students reported that it
was harder for them to concentrate on the class than in regular classroom settings, while
they happened to wander off more often. They were usually reluctant to speak out during
online classes, unless during an interesting discussion, and reported that it did not feel as
pleasant as it was in a classroom.
Structural description: One of the most frequent factors that contributed to students’
feeling and acting engaged during online classes was the specific topic that was being
presented—if a topic was personally interesting, students were more likely to contribute
to a discussion. The professor’s behavior also contributed significantly to students being
more active during the class—his or her enthusiasm, asking questions about the subject
being taught, but also about students’ opinions, sparking a discussion and contrasting
opinions, teaching with a camera turned on, and asking students to turn on their cameras.
Attention was higher if real-life examples, case studies, or dilemmas were presented, while
talking only about theory resulted in students losing focus. However, it was recognized
that the professor’s personality and approach were important factors in this, too—some
professors (although a small portion of them) were not seen as engaged in their own lecture,
they seemed too passive, “going through the slides”, or simply did not show that they care
for their students. Interactions between students and usage of online polls were making
them more active and engaged. In many cases, students were much more active in classes
in which their knowledge and activities were being monitored and tracked or assessed
via flash quizzes and short tests, when their progress was being measured during the
semester, and when their attendance was recorded. Having specific assignments, activities
and responsibilities during the class was also beneficial. Reactions from other students and
interaction with them during the class was also identified as a significant factor—students
were motivated to engage in a class when they witnessed others doing the same thing. Some
students reported that they are simply passive by nature, and that they preferred to stay
that way. Private chores and responsibilities were a negative factor when focusing attention
was in question, as were particular times of the day when classes were held: having classes
earlier in the morning or late in the evening resulted in less attention reported, as happened
with the situations where they had to attend online classes after work. Many students
outlined that having multiple classes in a row, with longer screen times, resulted in fatigue
and lack of focus. Frequently, students reported that they logged onto the online learning
platform but did not pay attention to the class at all, either because they were not interested
in the topic, or because they decided to do something else in parallel.
Theme 7: Hoping to return to classrooms soon
Textural description: Usually at the end of the interviews, students concluded their
thoughts by expressing hope that the things will soon return to how they were before
the current pandemic. Many students stated that they want to believe that the current
semester is the last one to be held remotely, but without any specific rationale or argument
for that wish.
Structural description: Students observed their current situation as a “status quo”,
perceiving themselves as being stuck in a process that they did not plan or wish to expe-
rience. Being physically distant from the faculty, teachers, and their peers, students had
frequent thoughts about the near future, wandering if they will soon start attending the
classes in person.
Sustainability 2022, 14, 4584 16 of 30

5.1.2. Composite Description—The Essence of Being Involved in PERT as an


Engineering Student
Attending PERT courses is a complex experience that forces students to take control
of their actions. They have to organize both their surroundings (their home premises and
other people living with them), and their own mind to prepare for online classes, which
asks for self-discipline and a sense of personal responsibility. Stress is situation specific: it
can be related to the start of the semester when students have doubts of whether they will
be able to attend the classes due to technical uncertainties and other responsibilities; during
the semester it is sometimes associated with connectivity issues and the way they organized
their day; at the end of the semester, stress is present when students feel overwhelmed with
assignments, as their motivation declines. Students look for the support, understanding
and openness of the teaching staff in order to feel secure. They are aware that the teaching
staff is doing their best, but understand that in given circumstances, there are limitations
of what can be achieved. They feel left on their own: when the online class ends, there is
usually no way to approach the professor informally and momentarily; also, they feel that
their individual progress is not being tracked to a high extent. While most of them start
with determination and discipline, their stamina wears off as the semester unfolds, and
they receive few incentives, still feeling stuck, and hoping (at the back of their minds) that
they will soon return to regular classes.
Attending PERT courses is an experience that brings with itself feelings of the whole
process being artificial, weird, and forced. Even when being highly disciplined, students
still feel that a lot is missing and that this experience is far from what is desired. Although
they compensate isolation with the increased usage of social networks and online commu-
nication tools, they still long for socializing with their colleagues in the flesh. Therefore,
they enjoy being part of a debate or being asked for their opinion, as this makes them
feel at least to some degree a part of a group. However, when the class starts, students
feel reluctant to show their faces, and prefer to stay silent unless an interesting discussion
develops. Even when they are motivated to join a discussion, they hesitate to do so because
it is not easy to get the attention of others due to multiple people talking at the same time.
Being called out by professors is seen as a good thing here, as students gain a perception
that time and attention are allocated especially for them.
Students’ focus and engagement during the classes vary significantly depending on the
topic of the class and the way the teaching is organized: they prefer topics that are appealing
to them personally, specific examples, and case studies. Although they experience some
stage-fright, they like to be asked for their personal opinion. They strongly resonate, with
both attention and activity, to the rest of their peer group. They feel stimulated if they
are exposed to individual activities and assignments and feel motivated to work if their
attendance and activities are being recorded. Their focus of attention is also affected by the
time of day or number of classes they have to attend, and they feel exhausted when they
spend too much time in front of the screen.

5.2. Study 2—The Teachers


5.2.1. Identified Themes—What Did the Teachers Experience and How Did These
Experiences Happen
Theme 1: Motivation decreases the semester unfolds
Textural description: The teaching staff reported that their levels of motivation and
their methods of teaching changed for the worse as the semester unfolded. For many,
deteriorating motivation was a common issue, especially as the workload increased in the
middle of semester, combined with screen-fatigue and partial lockdown. At the start of the
semester, many felt enthusiastic, but that dwindled as the semester was progressing. At the
end of the semester, they felt like they had given up, and became indifferent.
Structural description: The complex situation in which the PERT process was un-
folding was in most cases, teachers’ levels of motivation dropped because they lacked
significant feedback from students during the classes, and because students were mostly
Sustainability 2022, 14, 4584 17 of 30

passive participants of a class. Many teachers report that they had problems maintaining
their motivation without any feedback, as they had no clue on who was listening and if
their words had any effect on the listeners. Conversely, motivation would rise if some
students or groups of students were dynamic and discussed the topic. Another reason for
motivation levels to drop was the fact that professors and assistants spent many hours
sitting in front of their computers, instead of being in classrooms or in their offices.
Theme 2: Having to adapt and overcome, accepting the inevitable
Textural description: The teaching staff felt pressure to adapt their means of teaching
and their teaching material to the new circumstances. This frequently meant that significant
efforts had to be invested in before every class, which was stressful for the ones who got into
a routine in the previous years or even decades. However, if the topic was the professor’s
favorite, he or she then observed modifying slides and other materials as an interesting
challenge and a chance to think creatively. A few teachers reported that they felt the need
to motivate themselves and make their jobs more interesting in-home setting, due to the
lack of any other motivating factor. Nevertheless, the majority of the participants identified
the second half of the semester as the period in which they had to accept the fact that there
will be no return to classrooms any time soon, and that they had to find their peace with
the status quo.
Structural description: Many teachers reported that this need to adapt and modify was
constant during the semester, as every teaching unit and activity had to obtain a remote-
friendly alternative, or a proper substitution. In some cases, the teaching staff accepted the
challenges and tried to think of how to engage students in the class, by calling them out, by
asking questions pointed to a specific student, or giving them frequent pop quizzes.
Theme 3: Feeling uncomfortable due to the lack of any visual feedback while talking
Textural description: This was, by far, the most recurring and most emotional idea for
both professors and assistants. They felt challenged by the situation in which they lacked
students’ feedback on the topic that was being presented. Not being able to witness the
immediate impact of their teaching made the teachers feel stressed, helpless and frustrated,
and clueless about whether the students were really listening and understanding the lecture
(only a handful of participants stated that this was not an issue for them, explaining that
they concentrated on their own talk and preferred having a monologue). The teaching staff
also frequently reported that they felt absurd or weird, mostly because there was no good
interaction with the students, describing their impressions as “feeling stupid”, “like talking
into empty space/wall/screen”, “quite different and not natural”.
Structural description: Not having any visual feedback from students most of the time
during the remote classes was quite frustrating for many teachers. Although the online
platform allowed video streaming in two ways, students rarely showed their faces while a
teacher was talking. Some of the students had their cameras on at the start of the semester
or at the start of an individual class, but soon even they turned their cameras off. Most of the
students were even reluctant to turn their cameras on while talking, preferring to broadcast
only their voice to the rest of the class. Having been displaced from their usual offices and
classrooms made the teachers experience unpleasant feelings even stronger. Additionally, a
number of respondents indicated that they were feeling uncomfortable because they were
sharing their own private computer desktops or being seen in their private homes by the
students, which was observed by some as an intrusion into their privacy.
Theme 4: Students’ active participation as the most important motivation and work satis-
faction factor
Textural description: Most frequently, teachers stated that students’ active participa-
tion during the class—answering questions, asking questions, engaging in a discussion
with other students—was the strongest motivational factor for them during an online class.
Teachers often felt inspired and purposeful about what they do, and they even felt proud
when witnessing a debate during their lecture, as this proved that the students were paying
Sustainability 2022, 14, 4584 18 of 30

attention to what was being said and that they were influenced by teacher’s work. In
numerous cases, teachers stated that students’ engagement during a class was the most
important factor for them to feel as if they achieved something.
Structural description: In the absence of a visual feedback, teachers were relying
heavily on students’ verbal participation in order to find a purpose in what they were
doing during a class. Some teachers opted to make smaller breaks during their lectures to
allow students to engage in the topic, which the staff used not only for the students’ benefit,
but also to motivate themselves. Although the platform used for remote teaching allowed
seamless synchronous communication, teachers still observed students’ engagement as
something of extraordinary value and importance.
Theme 5: Work pressure was significantly experienced due to various challenges
Textural description: Professors and assistants experienced significant stress, and they
felt challenged due to perceived workload increase during the semester, related to different
aspects of remote teaching. They sometimes felt worried because of possible hardware
or software failure, or potential internet connectivity issues, which was exacerbated by
the fact that students’ feedback was limited or completely missing, as there was no one
to alarm the teachers that something was not working properly. Preparing for the class
in many cases took significantly more time than in regular settings, making teachers feel
challenged and overworked. They frequently felt “drained”, as a result of combination of
increased workload, lack of feedback, and overall pandemic context, feeling additionally
overwhelmed with numerous students’ emails that popped up after the class has ended. In
addition, many teachers felt pessimistic because they saw no significant positive aspects in
the remote teaching process, and no potential for anything to change soon.
Structural description: Challenges of conducting remote teaching during the whole
semester were numerous and various. Teachers were, in almost all cases, holding classes
from their own homes. Although they state that they mostly had support and understand-
ing from other family members, it was not always easy to use the same space for working
and private life; as the semester progressed, some teachers made different household
arrangements to have a space dedicated for holding online classes. Only a handful of
teachers held classes from their offices, usually because they had specific hardware to show,
or because they wanted to write on a physical whiteboard. Most students preferred to send
emails with questions after the class, rather than asking questions during the class, which
took significantly more time for teachers to address; this was especially problematic as the
semester was coming close to an end, with students increasingly asking for support and
clarification regarding their assignments. As the working day progressed, especially if the
teacher had more classes that day, things became more troublesome. It became evident that
the teachers who had topics with practical aspects, or complex processes to show during
their lectures had to invest more time in preparing for the classes, compared to teachers
who had more abstract or general topics in their subject’s curriculum.
Theme 6: Remote teaching has some benefits
Textural description: Most participants did not report any significant benefits from
having to teach remotely, stating that they find little value in that method regarding
educational outcomes. However, they frequently felt relieved because they did not have
to commute, and they had more freedom in organizing their other work and personal
activities. Communicating with students via MS Teams platform was convenient, as they
were able to respond to individual queries quicker, and felt that the communication was
more informal, compared to regular classes and office hours. They were even happy to have
any contact with the students in the chat section, in contrast to the online lectures where
they felt isolated from the students. Especially for the teaching staff that teaches IT related
courses such as software development or usage of software tools, this way proved beneficial
as they believed that students had better insight when screen sharing, as compared to using
video beams in a classroom. For some teachers, remote work allowed them to feel more
focused on what they were talking during the class.
Sustainability 2022, 14, 4584 19 of 30

Structural description: Absence of noise, interruptions, and distractions during the


class was most frequently reported as the good side of teaching remotely—some teachers
found that silence is good for their focus and performance. Especially when having their
own cameras turned off, teachers enjoyed comfort and privacy that was otherwise not
possible. Although they had to stick to a strict timetable regarding the classes, teachers
enjoyed the flexibility they had in spending the rest of their time.
Theme 7: Communication with the students during the class
Textural description: Teachers frequently commented that communication with stu-
dents felt artificial and sub-optimal. They observed communication with students mostly
as poor, stripped-down and one-directional. They did not enjoy having to explicitly ask
students for any comment or thoughts. Lack of proper two-way communication made
many teachers uncomfortable, as they lacked even tiny feedback that could have propelled
their further monologue. In addition, teachers frequently lacked any insights whether the
students understood the lecture, or what was their opinion on that subject. This was fol-
lowed by feeling powerless to change that situation or to influence students without having
to be strict—most professors do not like to enforce discipline and are rather motivated by
students’ enthusiasm and proactivity. Some teachers tried to spark a discussion, but in
many cases that had only modest effect, and then the teachers felt even more frustrated and
awkward, witnessing no response to their questions. Some teachers even felt disappointed
of students because of this, especially if they called out a student by name and he or she
did not respond, indicating that he or she is away from their logged device. They also
frequently felt limited and frustrated when having to present something without proper
equipment or demonstration tools available, such as specific hardware setup, machine
part, or a complex engineering process, thus feeling pessimistic about their chances to
teach students properly. Especially, if the teachers had to present a topic that develops in
numerous steps, they felt frustrated with limitations of virtual board if only keyboard and
a mouse were used.
Structural description: It was highly individual if the teachers wanted to use their
own cameras during the class or not, mostly depending on their own preferences and the
way they perceive the teaching process: some believe that it is part of their role as a teacher
to present themselves visually; others thought that it was their good manners to show their
faces at the end of every class; some did not use their cameras at all, believing that it makes
no difference. Professors rarely insisted on students getting their cameras on, because
majority of them realized that most of the students do not prefer to show their faces; even
the few ones who at the start of the semester asked that students have their cameras on
gave up after a few weeks, stating that they felt like they were enforcing something against
students’ will.
Theme 8: Nostalgia and hoping that things will soon return to normal
Textural description: Usually at the end of an interview, when asked to sum up their
thoughts, the teachers wanted to share that they felt nostalgia for the past times when
they taught students in the classroom. They frequently felt yearning for the past, with
romanticized descriptions of a teaching process that was once taken for granted, stating
that the physical presence of students during the class was the biggest thing they now miss.
However, most teachers did not express sadness or melancholy as much as they expressed
hope that things will return to normal soon, although they did not provide any specific
argument for such a statement.
Structural description: Being physically separated from their offices, classrooms,
students and other staff, teachers observe a lack of social stimuli compared to earlier times.
They experience an unwanted mix of personal and professional life and tend to dislike the
current ambiguous situation.
Sustainability 2022, 14, 4584 20 of 30

5.2.2. Composite Description—The Essence of Being Involved in PERT as a Teacher in an


Engineering Curriculum
Conducting courses during PERT is a completely different experience for teachers,
compared to the traditional means of having in-person classes. At the start of a semester,
there is an initial enthusiasm and a perception of an interesting challenge; however, as
the semester unfolds, motivation is undermined due to the monotonous routine and lack
of two-way communication. Teachers’ focusses of attention and morale are also variable,
depending on the class atmosphere and interaction (or lack of it) with students: if students
communicate, teachers feel motivated, purposeful, and accomplished; in the absence of
students’ reactions, teachers feel frustrated, uncomfortable, disappointed, demotivated,
and absurd.
The teaching staff feels challenged by numerous ways in which they are expected
to adapt their teaching practices and materials to online classes, which is less frequently
motivating and more frequently a continuous source of stress during the semester. They feel
overburdened by numerous emails, especially when the deadlines for student assignments
are approaching. They also feel frustrated if not being able to present something without
proper equipment, and worried if they will experience connection issues and not notice,
due to the usual silence from the other side. Only if they are teaching courses relevant
to software development or software application, they find this way of teaching more
appropriate than in the classroom, feeling more comfortable to share their screen online than
on a classroom wall. However, if they teach classes related to some hardware components,
or they need to show complex steps of a process, they find it hard to do in an online
class. Additionally, teachers feel disappointed if their efforts are not reflected in students’
reactions: they feel pessimistic about their situation, they feel clueless about whether the
students are even listening to them and understanding what is being said, and they tend to
feel powerless for not being able to influence students from a distance.
The teaching staff perceives only minor benefits in this teaching context. They are, to
some extent, enjoying the lack of classroom noise and less distractions during an online
class, and they also find it convenient that they do not have to commute. They find it easier
to respond to individual student queries, as long as they are sent in a timely manner.
Similar to students, teachers also start the semester with determination and discipline,
but it does not last long as the semester unfolds. They also feel stuck in an unwanted
situation, hoping (at the back of their minds) that they will soon return to regular classes.
They feel nostalgic for their usual routines and are looking forward to going back to a
classroom or lab setting, instead of feeling exhausted from long screen times.

5.3. Benchmarking PERT Sustainability


In the last part of the analysis, the interview transcripts and study results have been
compared to previously established dimensions of sustainable e-learning systems [19],
revealing some of the positive and negative aspects of PERT from the perspectives of
teachers and engineering students. The results of this analysis are presented in Table 3.
They show that different dimensions of sustainable e-learning systems may provide both
positive and negative aspects for both sides of the educational process. It may be, therefore,
interesting for the management of engineering education institutions to learn from these
insights when introducing similar changes in the near future, especially if it is expected to
have the changes last long and produce significant results.
Sustainability 2022, 14, 4584 21 of 30

Table 3. Engineering students and teachers’ PERT experiences compared to dimensions of sustainable
e-learning systems.

Dimensions of
Sustainable Perspective Positive Aspect Negative Aspect
E-Learning Systems *
Individuality is lost during
Student Online communication 1on1
classes
Human—Individual needs should be protected
Lack of clear distinction
and supported with dignity More privacy and working
Teacher between professional and
from home
private life
Social—Relationships of people within society Student No data for most Lack of peer communication
should be equitable, diverse, connected and Lack of feedback from the
democratic Teacher No data for most
other side, poor interaction
Learning platform available
on all devices, teaching Overwhelmed with
Student
Technical—Technology must cope with changes materials and class recordings technology
and evolution in a fair manner, respecting allow for individual pacing
natural resources High dependence on
All materials and data on one
Teacher technology to deliver teaching
platform
content
Environmental—Natural resources have to be Student Less commuting No data for most
protected from human needs and wastes Teacher Less commuting No data for most

Economic dimensions—A positive economic Less expenses due to distance Less worth in gained
Student
value and capital should be ensured and learning knowledge
preserved Unable to fully deliver
Teacher No data for most
knowledge that is appreciated
* According to [19] (p. 523).

6. Discussion and Conclusions


This paper aims to describe personal experiences of engineering students and teachers
in an emergency setting, using two samples of respectable size for a phenomenological
study; these descriptions are needed to understand, above others, individual and social
requirements that are highly relevant for the sustainability of e-learning systems [19]. How-
ever, unlike other studies that deal with initial perceptions [75] or transition challenges [76]
during the COVID-19 pandemic, this research focuses on the setting in which an educa-
tional process is running significantly altered for an unspecified (and undoubtedly long)
period of time. Thus, we propose a new term PERT to accentuate the temporal spanning of
the current global crisis, and we aim to depict how the human capital of both sides is being
utilized in a forced process.
This temporal spanning begs the question of sustainability of the PERT process, as the
prolonged e-learning puts stress on two of the three identified domains of literature related
to sustainable e-learning [77]. The first one is educational attainment, which observes
students’ human capital in the means of achievement, retention rates, skill acquisition,
and personal development. The second one is professional development and innovation,
which observes teachers’ human capital in the means of employees’ adaptation to change
by career development and skills acquisition.
Results of both studies depict a situation in which both sides, students and their
teachers, experience complex emotions and face multiple challenges in the situation of
PERT. These challenges are mostly unfavorable for the sustainability of this situation:
conditions required for the e-learning sustainability [5] are not met; nor are most of the
high-level sustainability meta-requirements for e-learning [19]. These facts explain why the
sentiments of key actors in the process are not optimistic, and why the starting enthusiasm
quickly fades as the semester unfolds. The fact that the emergency situation is prolonged
indefinitely brings along new obstacles for educational processes, but also for mental
Sustainability 2022, 14, 4584 22 of 30

processes of all the actors. Similar to other relevant studies [13,14], this research showed
that having an educational process adequately set up and running at the system level is not
enough for the participants to feel fulfilled and motivated—although they seem content,
they frequently report that they feel frustrated, lost or overwhelmed, isolated, clueless, and
even absurd. Nostalgic feelings for the “old normal” prevent subjects from adequately
engaging in the “new normal” setting, thus perceiving themselves as stuck in a limbo.
As these feelings are frequent companions of emergency remote learning/teaching,
implications for engineering education are multifold. For both engineering students and
their teachers, self-discipline and motivation are hard to maintain as the semester unfolds,
with a number of factors identified [46,47,50]. The educational process should be more
immersive and activating during the semester. Lack of interaction with others lowers indi-
vidual motivation, as lack of proper feedback makes students feel left out, and professors
ignored. During the classes, more emphasis should be placed on dialogue and two-way
exchange between the two sides, with rich co-creation processes that involve students to a
larger extent.
A number of implications for improving sustainability of this education process can
be drawn from these studies. As teachers experienced trouble finding alternative solutions
and new tools to keep the previous workflow, the question arises of whether is it even
the right aim to pursue. In the significantly altered state that lasts for months or even
years, the goal of the interaction (the “what”) has to change, and then the tools (the “how”)
will follow more easily. Software tools should thus be developed and implemented to
aid the learning process (from the students’ perspective), rather than to aid the teaching
process (from the professors’ perspective). It is pedagogy, and not technology, that should
drive change and improvement in engineering education, as quick fixes and ICT tools are
not effective if they are introduced to help the faculty staff, and are the only alteration
in the teaching equation [78]. The new generations possess a shorter attention span, and
have a constant need to receive new, different types of information, advocating for usage
of modern means of education that allow students to undertake learning at individual
pace [20]. Students, while feeling left out and not individually addressed, need more help
in understanding and witnessing relevant topics from a distance, which asks for technology
to help them not only in listening, but in experiencing remotely, where virtual labs are most
promising solution [33]. Such solutions could also help the teachers to reach more students
and engage them, which is also beneficial for their work morale and its results—closing
the circle of questioning who engages who in the process [26], as the teacher role should
transform from a content provider to a mentor and a coach [25].
This research, although comprehensive and thorough in its methodology, still has
some limitations that could also imply possible avenues for further research. Observing
only one engineering education institution had its rationale; however, generalizability
of the findings could be put in question, as other engineering schools may have other
solutions put to practice. Additionally, this research did not ask participants to express
ideas on how to improve their experience in the PERT context, which could be useful.
Finally, this research presented results from two cross-sectional studies, processing data
from one specific point in time. Further research should employ a longitudinal approach to
further depict the temporal dimension of ERT.

Author Contributions: Conceptualization, P.V. and J.P.; methodology, P.V.; software, M.M. and A.A.;
validation, A.A. and B.L.; formal analysis, J.P.; investigation, P.V., M.M. and A.A.; resources, M.M.
and A.A.; data curation, P.V. and M.M.; writing—original draft preparation, P.V. and J.P.; writing—
review and editing, P.V., J.P., M.M., A.A. and B.L.; visualization, M.M.; supervision, J.P.; project
administration, A.A. and B.L.; funding acquisition, M.M. All authors have read and agreed to the
published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Sustainability 2022, 14, 4584 23 of 30

Data Availability Statement: Not applicable.


Acknowledgments: This research has been supported by the Ministry of Education, Science and
Technological Development through project no. 451-03-68/2022-14/200156 “Innovative scientific and
artistic research from the FTS domain”.
Conflicts of Interest: The authors declare no conflict of interest.

Appendix A. Study 1. Engineering Students’ Verbatim Examples


Theme 1: Self-discipline, motivation, and workflow
“Most of the times I was in my room, but sometimes I was commuting or walking
from a grocery store. The biggest problem was that sometimes during the week my sister
had her own online classes to listen to at the same time, so we had to coordinate on who
will use the room, and who will go to the kitchen”.
“What could be better than having an hour more every day, as it usually took me
30mins to get there (to the University). I usually attended classes from home, but a few times
I went to a caffe and listened from there, just to add some dynamics to the daily routine”.
“I felt too relaxed, even spoiled”.
“At the start of the semester, it was piece-of-cake. However, as the time passed, I
felt I had less and less free time as we were operating within tight deadlines and had
to submit assignments regularly, no extensions. Many professors were now stricter than
before the pandemic”.
“Then, in December, assignments started piling up and I lost my nerves. All of a
sudden, I had so many things to do at once, and on my own”.
“At the start of the semester I was quite alert and enthusiastic, it all seemed new for
me (a freshmen), but it all gradually faded as the semester was progressing. At the end, I
felt like I almost completely lost discipline and focus, I got saturated”.
Theme 2: Learning experience and outcomes
“I believe I learned a lot, but I still have a feeling that something is missing, mostly
because I didn’t attend physically, since this was my first semester”.
“I believe we missed a lot since in this year we had a lot of practical work and field
work to do, and we went on terrain only two times and we had eight on the schedule. Our
practical knowledge suffered a lot”.
“We received technical knowledge pertinent to platforms we used, but other than that
we didn’t really gather much practical knowledge that we could leverage straight away. In
general, everyone did their best to organize classes given the circumstances, I don’t think
this could have been any better during this online arrangement”.
“A combination of online learning and in-person labs would be quite beneficial for
better knowledge retention, since we have some classes where we’re designing specific
mechanical products and it just can’t be all done virtually”.
“Computer labs were much easier to follow in this way compared to in-person classes
(since TA’s could share their screens and I wouldn’t miss as many details). Auditory labs
were a bit more difficult to follow since they usually rely on notes written on a blackboard;
TA’s would often use mouse to simulate it in online software which would make them hard
to read/understand”.
“I think there weren’t any problems with knowledge transfer, given that most of the
courses were dealing with IT-related topics which require us to use computers intensively
anyway; students in some other domains might not have been so lucky and might have
done better if the classes were held in-person”.
“Biggest problem was the lack of interaction between students. Maybe there could be
some polls, tests, quizzes during lectures, where the students could participate right away
and that would motivate them to be better prepared for next lectures”.
Sustainability 2022, 14, 4584 24 of 30

Theme 3: Affective reactions


“My overall experience was similar to watching tutorials on YouTube. It was quite
different compared to regular classes”.
“I didn’t feel very comfortable using online environment, I think it would be much
better if classes were held in-person. It’s difficult to have the same experience as you would
in real-world, as it’s not possible to answer all questions or convey complex messages, for
example by also using body language and mimic”.
“It was a bit weird in the beginning, but after a while everyone got used to the online
platform and everything went smoothly... The listening part was quite good, but the other
things were missing, as you can presume”.
Theme 4: Peer communication and lack of social interaction
“It was quite different compared to regular classes. I liked it in the beginning, but later
on I really started missing the company of other students and the entire social component”.
“The biggest difference was the abrupt cutoff of communication after the classes; in
in-person classes you would discuss different topics while walking out of the class or going
home, with online classes you just log out and that’s the end of it. Other than that, I used
social networks to communicate with fellow students and share material”.
“Communication during this semester wasn’t as good as it was when the lectures
were held in-person. I had only one colleague that I would see in person, but otherwise I
would use Facebook and Discord for online communication with other students”.
“I communicated intensively with a small number of colleagues, mostly related to
projects we were assigned to. We used social media and online tools, even Git for some of
the projects”.
Theme 5: Support by the teaching staff
“Professors were quite understanding and supportive in general and willing to help
us whenever we needed help. I mean, we had a good support as a group, but didn’t really
have any individual support which we might have received if classes were held in-person”.
“Professors were quite supportive in general, and I believe they faced significant
challenges during the preparation of online materials. They did a good job”.
“I really can’t complain, professors were always there for us and supported us. How-
ever, it’s much easier to clarify things when you can use professor’s office hours to come
and ask things in-person than when you need to communicate online”.
“Most of the professors were really helpful and answered all of our questions promptly,
although they didn’t check every student individually for progress and understanding of
the course material”.
“Some TA’s were paying real close attention to our progress and did regular check-ins,
while the others didn’t care as much. Personally, I received all the help I needed, but I’m the
kind of person who’s always going to ask for it when I feel I don’t understand something”.
Theme 6: Attention and active engagement during a class
“As I said, professors made an effort to make it easy for students to be active partici-
pants in the lectures, and I think that a very good thing was using cameras and microphones
to create a more pleasant and engaging atmosphere”.
“Depending on the subject, sometimes I was more active, other times less. If the topic
was interesting, I was more activated to join the discussions. I usually got more engaged
when professors started discussions, otherwise I was dominantly a passive listener: in that
case, me wondering off during the class was the main reason not to pay attention and to
later be confused on what is going on”.
“Courses were quite interesting in general in this semester, so the subject matter itself
made me want to be engaged and pay attention. Conversely, when professors wouldn’t
allow students to ask questions or simply read from the slides that made me want to leave
the class”.
Sustainability 2022, 14, 4584 25 of 30

“Professors motivated us by trying to include us actively in discussions, and some of


them gave us topics to read and prepare in advance so we would discuss them in classes. I
really liked that. Only when subject matter was “dry theory” would I be demotivated and
not pay much attention in classes”.
“TA’s could’ve insisted more on us being engaged. Students are shy in general and
don’t wish to turn on their microphones, so we need to be motivated to do so. If we have
multiple classes during the day that cover ‘dry theory’, attention usually drops later in the
day and students just stay logged in but don’t actually attend them”.
“I think the most important thing was the topic itself; if it was interesting, I would pay
attention. Other than that, when other students would engage in the discussion I would
wish to engage as well. Conversely, 90% of the time no one would answer the question
the professors would ask in the class and that’s what was demotivating to me as well, as I
wouldn’t wish to engage either”.
“Freedom made me distracted. It is not easy to stay concentrated when there’s nobody
to control whether I am listening or not”.
Theme 7: Hoping to return to classrooms soon
“This can’t go on like this forever, it doesn’t make any sense. I’m sure we will all laugh
together in a few months”.
“We all got used to this situation, and we are acting like this is normal, but we all
know it is not. Everybody wants to go back”.

Appendix B. Study 2. The Teachers’ Verbatim Examples


Theme 1: Motivation decreases the semester unfolds
“The semester started euphorically from my side, but after a while, it looked like
students drew a straw on who will answer me. I frequently had to ask if anybody was
present at all, and then had to wait a few seconds before anyone replied”.
“At the start of the semester I turned on my camera in every class, but later I gave up
on that and just went on with the flow. I saw no point in showing enthusiasm”.
“After a while, I realized that there are serious limitations to this way of teaching, and
that no matter how much I try, some things cannot be achieved”.
Theme 2: Having to adapt and overcome, accepting the inevitable
“Although at the beginning of the semester I felt like my work was pointless, after a
while I accepted the things as they were, and just moved on”.
“At the start of the semester things did feel a little bit awkward, but after a while I got
used to it”.
Theme 3: Feeling uncomfortable, due to the lack of any visual feedback while talking
“Most students didn’t even put their profile pictures on, so I felt like talking only to
their initials”.
“You have an impression that you are talking to a wall in front of you”.
“I missed an important part of the puzzle: visual contact with the students. I find their
facial expressions and body movements as extremely important cues into how they accept
my lectures and the content that I am presenting”.
“It was quite unpleasant, as I had no idea of who was actually listening and paying
attention to what I was talking about”.
Theme 4: Students’ active participation as the most important motivation and work satis-
faction factor
“I wish the students had more enthusiasm to turn on their cameras and talk. It meant
a lot to me”.
“As well as before, I continually asked questions, expecting students to respond and
comment. There weren’t a lot of students engaging in the discussion, but still they made
me feel comfortable and made me enjoy holding online classes. Ever more, although most
Sustainability 2022, 14, 4584 26 of 30

students were quiet and passive, a few of them were exceptional which is all I needed to
feel OK”.
“I believe I was able to activate students because I was calling them out randomly
during the problem-solving parts of my class, and they had no idea who will have to do
the next part, so they had to stay tuned. I also tried to frequently ask questions regarding
their own opinions to engage them, and I think it worked for both of us”.
“I am very dissatisfied with how much the students are active. Since I teach at the 4th
and 5th year of studies, I expect more responsibility from the students and more awareness
of their own obligations... If I had called them out and recorded their activity, maybe they
would have got more activated, but I am against forcing students to attend”.
“I miss the audience”.
Theme 5: Work pressure was significantly experienced due to various challenges
“I sometimes felt anxious while teaching, as I occasionally checked the bottom of my
screen to see the internet status. The PowerPoint slides prevented me from seeing if MS
Teams alerted of something being wrong, so I had to monitor the Wi-Fi symbol”.
“Before the class I felt dreaded because what was before 10 min during a regular class
with me writing and talking on a blackboard at the same time, became two hours of me
preparing slides just to show the same thing”.
“I felt challenged as I needed significantly more time to prepare some topics in order
for students to understand properly. I’ve been teaching for 12 years and thought I could
tell every lesson by heart, but now that just couldn’t apply”.
“Due to my significant effort to make up to the students and support them as much as
possible, I witnessed the best average score that I can remember. However, that took its toll
on my private life, my health and my sleep”.
“I felt furious every time when, immediately after the class has ended, I received
questions about issues that were explicitly covered just ten minutes ago; although this
surely indicated that students weren’t paying attention, I had to spend additional time to
answer them”.
Theme 6: Remote teaching does have some benefits
“I even felt motivated to update my classes with new case studies and examples,
which was interesting for me to do. It was out of necessity, but it was interesting to step out
of my usual examples, that were hard to share online”.
“Not having to commute and search for a parking lot saved me at least two hours
every day”.
“I found a number of silver linings in this situation—those were small things compared
to the overall COVID situation, yet I was happy for them... Being able to sip coffee during
the lecture, having the possibility to peek into the course book while I was talking, sitting
in comfortable clothes...”
Theme 7: Communication with the students during the class
“I was aware that some students were at their workplaces on in a car, so I knew that
they weren’t fully present. That made me feel like a host of a radio show, talking into empty
space and hoping that at least someone was actually listening”.
“I felt stupid having to talk about ball bearings without being able to give a few to my
students to feel and explore them. I usually bring a full box of them”.
“I always insist on two-way communication with students and in-person it really
works. But here nobody can guarantee if the students are really listening and everything
loses meaning. I ask a question once, twice, three times, and then after an awkward silence
I have to answer it myself. I feel like a fool”.
Theme 8: Nostalgia and hoping that things will soon return to normal
“I hope that next year things will get back to normal. We can’t go on like this, it makes
no sense”.
Sustainability 2022, 14, 4584 27 of 30

“I hope this semester is really the last one that will see me teach from home. Nobody
wants this to continue”.
“I see that students do not enjoy this any more than we do, poor things, I wish they
soon get back to the classrooms, but also to the campus and cafes”.

Appendix C
Table A1. Similar thematic units found in both studies, presented side by side.

Students Teachers
Technical aspects (internet, devices used) Technical aspects (internet, devices used)
Physical location Physical location
Positive aspects of learning online Positive aspects of teaching online
How the teachers explain How the teaching process develops
What makes them engaged Issues regarding students’ engagement
Self-discipline Motivation and organization of work
Absorbed knowledge Teaching outcomes
Stress Stress
Communication problems Communication problems
Feeling of nostalgia Feeling of nostalgia
Thematic units that did not match in both studies
Peer communication —
Teachers’ support —
— New challenges on how to teach
— Technical support from the institution

References
1. Marinoni, G.; van’t Land, H.; Jensen, T. The Impact of COVID-19 on Higher Education around the World-IAU Global Survey Report
2020; International Association of Universities: Paris, France, 2020.
2. Bozkurt, A.; Jung, I.; Xiao, J.; Vladimirschi, V.; Schuwer, R.; Egorov, G.; Lambert, S.; Al-Freih, M.; Pete, J.; Olcott, D.A., Jr.
Global Outlook to the Interruption of Education Due to COVID-19 Pandemic: Navigating in a Time of Uncertainty and Crisis.
Asian J. Distance Educ. 2020, 15, 1–126.
3. Hodges, C.; Moore, S.; Lockee, B.; Trust, T.; Bond, A. The Difference between Emergency Remote Teaching and Online Learning.
Available online: https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-
learning (accessed on 20 December 2020).
4. Trentin, G. A Multidimensional Approach to E-Learning Sustainability. Educ. Technol. 2007, 47, 36–40.
5. Gunn, C. Sustainability Factors for E-Learning Initiatives. ALT-J. Res. Learn. Technol. 2010, 18, 89–103. [CrossRef]
6. Prasetyanto, D.; Rizki, M.; Sunitiyoso, Y. Online Learning Participation Intention after COVID-19 Pandemic in Indonesia: Do
Students Still Make Trips for Online Class? Sustainablity 2022, 14, 1982. [CrossRef]
7. Marlay, B.; Attenborough, J.; Kutcher, V. “Living in Limbo” The Views and Experiences of Young People in Australia at the Start of the
COVID-19 Pandemic and National Response 2020; UNICEF: New York, NY, USA, 2020.
8. Kale, S. “I’m Stuck in Limbo”: Will the Covid Generation of Young People Face Long-Term Fallout? Available online:
https://www.theguardian.com/society/2020/may/30/im-stuck-in-limbo-will-the-covid-generation-of-young-people-face-
long-term-fallout (accessed on 14 December 2020).
9. Flaherty, C. Working from Home during COVID-19 Proves Challenging for Faculty Members. Available online: https://www.
insidehighered.com/news/2020/03/24/working-home-during-covid-19-proves-challenging-faculty-members (accessed on 5
December 2020).
10. Butler-Henderson, K.; Crawford, J.; Rudolph, J.; Lalani, K.; Sabu, K.M. COVID-19 in Higher Education Literature Database
(CHELD V1): An Open Access Systematic Literature Review Database with Coding Rules. J. Appl. Learn. Teach. 2020, 3, 1–6.
11. Saikat, S.; Dhillon, J.S.; Wan Ahmad, W.F.; Jamaluddin, R.A. A Systematic Review of the Benefits and Challenges of Mobile
Learning during the COVID-19 Pandemic. Educ. Sci. 2021, 11, 459. [CrossRef]
12. Kuleto, V.; Ilić, M.P.; Šević, N.P.; Ranković, M.; Stojaković, D.; Dobrilović, M. Factors Affecting the Efficiency of Teaching Process
in Higher Education in the Republic of Serbia during COVID-19. Sustainablity 2021, 13, 12935. [CrossRef]
13. Jeffery, K.A.; Bauer, C.F. Students’ Responses to Emergency Remote Online Teaching Reveal Critical Factors for All Teaching.
J. Chem. Educ. 2020, 97, 2472–2485. [CrossRef]
14. Hattingh, T.; van Niekerk, W.; Marais, H.; Geldenhuys, Y. Engineering Student Experiences of a Remotely Accessed, Online
Learning Environment. In Proceedings of the 2020 IFEES World Engineering Education Forum-Global Engineering Deans Council
(WEEF-GEDC), Cape Town, South Africa, 16–19 November 2020; pp. 1–6.
Sustainability 2022, 14, 4584 28 of 30

15. Creswell, J.W. Qualitative Inquiry and Research Design: Choosing among Five Approaches, 2nd ed.; Sage Publications, Inc.: London,
UK, 2007.
16. World Bank. The Human Capital Index 2020 Update: Human Capital in the Time of COVID-19; The World Bank: Washington, DC,
USA, 2021.
17. Yarovaya, L.; Mirza, N.; Abaidi, J.; Hasnaoui, A. Human Capital Efficiency and Equity Funds’ Performance during the COVID-19
Pandemic. Int. Rev. Econ. Financ. 2021, 71, 584–591. [CrossRef]
18. Corral, P.; Gatti, R. 21 Accumulation Interrupted: COVID-19 and Human Capital among the Young. In COVID-19 in Developing
Economies; Djankov, S., Panizza, U., Eds.; Centre for Economic Policy Research: London, UK, 2020; pp. 286–302.
19. Alharthi, A.D.; Spichkova, M.; Hamilton, M. Sustainability Requirements for ELearning Systems: A Systematic Literature Review
and Analysis. Requir. Eng. 2019, 24, 523–543. [CrossRef]
20. Szymkowiak, A.; Melović, B.; Dabić, M.; Jeganathan, K.; Kundi, G.S. Information Technology and Gen Z: The Role of Teachers,
the Internet, and Technology in the Education of Young People. Technol. Soc. 2021, 65, 101565. [CrossRef]
21. Dabbagh, N.H. The Challenges of Interfacing between Face-to-Face and Online Instruction. TechTrends 2000, 44, 37–42. [CrossRef]
22. Makarova, I.; Shubenkova, K.; Antov, D.; Pashkevich, A. Digitalization of Engineering Education: From E-Learning to Smart
Education BT-Smart Industry & Smart Education. In Lecture Notes in Networks and Systems, Proceedings of the 15th International
Conference on Remote Engineering and Virtual Instrumentation, REV 2018, Dusseldorf, Germany, 21–23 March 2018; Auer, M.E.,
Langmann, R., Eds.; Springer International Publishing: Cham, Switzerland, 2019; Volume 47, pp. 32–41.
23. Banday, M.T.; Ahmed, M.; Jan, T.R. Applications of E-Learning in Engineering Education: A Case Study. Procedia-Soc. Behav. Sci.
2014, 123, 406–413. [CrossRef]
24. Potkonjak, V.; Vukobratović, M.; Jovanović, K.; Medenica, M. Virtual Mechatronic/Robotic Laboratory–A Step Further in Distance
Learning. Comput. Educ. 2010, 55, 465–475. [CrossRef]
25. Abumandour, E.-S.T. Applying E-Learning System for Engineering Education—Challenges and Obstacles. J. Res. Innov. Teach. Learn.
2021. [CrossRef]
26. Heller, R.S.; Beil, C.; Dam, K.; Haerum, B. Student and Faculty Perceptions of Engagement in Engineering. J. Eng. Educ. 2010, 99,
253–261. [CrossRef]
27. Izumi, T.; Sukhwani, V.; Surjan, A.; Shaw, R. Managing and Responding to Pandemics in Higher Educational Institutions: Initial
Learning from COVID-19. Int. J. Disaster Resil. Built Environ. 2021, 12, 51–66. [CrossRef]
28. Orakci, Ş.; Gelişli, Y. Educational Policy Actions in the Times of Covid-19 and Suggestions for Future Applications in Turkey.
In Handbook of Research on Inequities in Online Education During Global Crises; Kyei-Blankson, L., Blankson, J., Ntuli, E., Eds.; IGI
Global: Hershey, PA, USA, 2021; pp. 475–493.
29. Melović, B. Educational Management and Leadership in Montenegro. In Educational Leadership in Policy: Challenges and Implemen-
tation Within Europe; Ingþórsson, Á.H., Alfirević, N., Pavičić, J., Vican, D., Eds.; Palgrave Macmillan: Cham, Switzerland, 2019;
pp. 245–264.
30. Mackey, K.R.M.; Freyberg, D.L. The Effect of Social Presence on Affective and Cognitive Learning in an International Engineering
Course Taught via Distance Learning. J. Eng. Educ. 2010, 99, 23–34. [CrossRef]
31. Roman-Sanchez, D.I.; Lopez-Arenas, T.; Sales-Cruz, M. An Interactive Virtual Laboratory for Process Engineering Education. In
Computer Aided Chemical Engineering; Elsevier: Amsterdam, The Netherlands, 2018; Volume 43, pp. 1707–1708.
32. Budai, T.; Kuczmann, M. Towards a Modern, Integrated Virtual Laboratory System. Acta Polytech. Hungarica 2018, 15, 191–204.
33. Kapilan, N.; Vidhya, P.; Gao, X.-Z. Virtual Laboratory: A Boon to the Mechanical Engineering Education during Covid-19
Pandemic. High. Educ. Futur. 2021, 8, 31–46. [CrossRef]
34. Mohammed, A.K.; El Zoghby, H.M.; Elmesalawy, M.M. Remote Controlled Laboratory Experiments for Engineering Education in
the Post-COVID-19 Era: Concept and Example. In Proceedings of the 2020 2nd Novel Intelligent and Leading Emerging Sciences
Conference (NILES), Giza, Egypt, 24–26 October 2020; pp. 629–634.
35. Koretsky, M.; Kelly, C.; Gummer, E. Student Perceptions of Learning in the Laboratory: Comparison of Industrially Situated
Virtual Laboratories to Capstone Physical Laboratories. J. Eng. Educ. 2011, 100, 540–573. [CrossRef]
36. Lawton, D.; Vye, N.; Bransford, J.; Sanders, E.; Richey, M.; French, D.; Stephens, R. Online Learning Based on Essential Concepts
and Formative Assessment. J. Eng. Educ. 2012, 101, 244–287. [CrossRef]
37. Wong, E.T.T. Impact of E-Learning during SARS Outbreak in Hong Kong. In The Social and Cognitive Impacts of E-Commerce on
Modern Organizations; IGI Global: Hershey, PA, USA, 2004; pp. 185–203.
38. Abdelhamid, H.; Ghali, H.; Attia, A.M.; Badran, W.; Fahim, S.; Fakhreldin, H.; Hassan, M.; Karam, O.H.; Khalil, R.M.R.; Khalil, S.
Pedagogy of the Pandemic: A Case Study of Emergency Remote Education in a Private Higher Education Institution in Egypt. In
Proceedings of the ICERI 2020 Conference, Online, 9–10 November 2020; Volume 9, pp. 618–627.
39. Patricia Aguilera-Hermida, A. College Students’ Use and Acceptance of Emergency Online Learning Due to COVID-19.
Int. J. Educ. Res. Open 2020, 1, 100011. [CrossRef] [PubMed]
40. Aristovnik, A.; Keržič, D.; Ravšelj, D.; Tomaževič, N.; Umek, L. Impacts of the COVID-19 Pandemic on Life of Higher Education
Students: A Global Perspective. Sustainability 2020, 12, 8438. [CrossRef]
41. Gonzalez, T.; De La Rubia, M.A.; Hincz, K.P.; Comas-Lopez, M.; Subirats, L.; Fort, S.; Sacha, G.M. Influence of COVID-19
Confinement on Students’ Performance in Higher Education. PLoS ONE 2020, 15, e0239490. [CrossRef] [PubMed]
Sustainability 2022, 14, 4584 29 of 30

42. Althunibat, A.; Almaiah, M.A.; Altarawneh, F. Examining the Factors Influencing the Mobile Learning Applications Usage in
Higher Education during the COVID-19 Pandemic. Electronics 2021, 10, 2676. [CrossRef]
43. Naji, K.K.; Du, X.; Tarlochan, F.; Ebead, U.; Hasan, M.A.; Al-Ali, A.K. Engineering Students’ Readiness to Transition to Emergency
Online Learning in Response to COVID-19: Case of Qatar. EURASIA J. Math. Sci. Technol. Educ. 2020, 16, em1886.
44. Gelles, L.A.; Lord, S.M.; Hoople, G.D.; Chen, D.A.; Mejia, J.A. Compassionate Flexibility and Self-Discipline: Student Adaptation
to Emergency Remote Teaching in an Integrated Engineering Energy Course during COVID-19. Educ. Sci. 2020, 10, 304. [CrossRef]
45. Asgari, S.; Trajkovic, J.; Rahmani, M.; Zhang, W.; Lo, R.C.; Sciortino, A. An Observational Study of Engineering Online Education
during the COVID-19 Pandemic. PLoS ONE 2021, 16, e0250041. [CrossRef]
46. Piyatamrong, T.; Derrick, J.; Nyamapfene, A. Technology-Mediated Higher Education Provision during the COVID-19 Pandemic:
A Qualitative Assessment of Engineering Student Experiences and Sentiments. J. Eng. Educ. Transform. 2021, 34, 290–297.
[CrossRef]
47. Kruger, K.; Wolff, K.; Cairncross, K. Real, Virtual, or Simulated: Approaches to Emergency Remote Learning in Engineering.
Comput. Appl. Eng. Educ. 2022, 30, 93–105. [CrossRef]
48. Clark, R.M.; Kaw, A.K.; Braga Gomes, R. Adaptive Learning: Helpful to the Flipped Classroom in the Online Environment of
COVID? Comput. Appl. Eng. Educ. 2021, 30, 517–531. [CrossRef]
49. Vergara, D.; Fernández-Arias, P.; Extremera, J.; Dávila, L.P.; Rubio, M.P. Educational Trends Post COVID-19 in Engineering:
Virtual Laboratories. Mater. Today Proc. 2022, 49, 155–160. [CrossRef] [PubMed]
50. El Said, G.R. How Did the COVID-19 Pandemic Affect Higher Education Learning Experience? An Empirical Investigation
of Learners’ Academic Performance at a University in a Developing Country. Adv. Hum.-Comput. Interact. 2021, 2021, 1–10.
[CrossRef]
51. Hjelsvold, R.; Nykvist, S.S.; Lorås, M.; Bahmani, A.; Krokan, A. Educators’ Experiences Online: How COVID-19 Encouraged
Pedagogical Change in CS Education. In Proceedings of the Norsk IKT-Konferanse for Forskning Og Utdanning; 2020. Available
online: https://ojs.bibsys.no/index.php/NIK/article/view/817 (accessed on 12 December 2021).
52. Popa, D.; Repanovici, A.; Lupu, D.; Norel, M.; Coman, C. Using Mixed Methods to Understand Teaching and Learning in COVID
19 Times. Sustainability 2020, 12, 8726. [CrossRef]
53. Durak, G.; Çankaya, S. Emergency Distance Education Process from the Perspectives of Academicians. Asian J. Distance Educ.
2020, 15, 2020.
54. Mohmmed, A.O.; Khidhir, B.A.; Nazeer, A.; Vijayan, V.J. Emergency Remote Teaching during Coronavirus Pandemic: The Current
Trend and Future Directive at Middle East College Oman. Innov. Infrastruct. Solut. 2020, 5, 72. [CrossRef]
55. Bojović, Ž.; Bojović, P.D.; Vujošević, D.; Šuh, J. Education in Times of Crisis: Rapid Transition to Distance Learning.
Comput. Appl. Eng. Educ. 2020, 28, 1467–1489. [CrossRef]
56. Luburić, N.; Slivka, J.; Sladić, G.; Milosavljević, G. The Challenges of Migrating an Active Learning Classroom Online in a Crisis.
Comput. Appl. Eng. Educ. 2021, 29, 1617–1641. [CrossRef]
57. Polkinghorne, D.E. Phenomenological Research Methods. In Existential-Phenomenological Perspectives in Psychology: Exploring the
Breadth of Human Experience; Valle, R.S., Halling, S., Eds.; Springer: Boston, MA, USA, 1989; pp. 41–60.
58. Merriam, S.B.; Tisdell, E.J. Qualitative Research: A Guide to Design and Implementation; John Wiley & Sons, Inc.: San Francisco, CA,
USA, 2016.
59. Browning, M.H.E.M.; Larson, L.R.; Sharaievska, I.; Rigolon, A.; McAnirlin, O.; Mullenbach, L.; Cloutier, S.; Vu, T.M.; Thomsen, J.;
Reigner, N. Psychological Impacts from COVID-19 among University Students: Risk Factors across Seven States in the United
States. PLoS ONE 2021, 16, e0245327. [CrossRef]
60. Hussein, E.; Daoud, S.; Alrabaiah, H.; Badawi, R. Exploring Undergraduate Students’ Attitudes towards Emergency Online
Learning during COVID-19: A Case from the UAE. Child. Youth Serv. Rev. 2020, 119, 105699. [CrossRef]
61. Flanigan, A.E.; Babchuk, W.A. Social Media as Academic Quicksand: A Phenomenological Study of Student Experiences in and
out of the Classroom. Learn. Individ. Differ. 2015, 44, 40–45. [CrossRef]
62. Tavakol, S.; Dennick, R.; Tavakol, M. Medical Students’ Understanding of Empathy: A Phenomenological Study. Med. Educ. 2012,
46, 306–316. [CrossRef] [PubMed]
63. Lee, C.S.; McNeill, N.J.; Douglas, E.P.; Koro-Ljungberg, M.E.; Therriault, D.J. Indispensable Resource? A Phenomenological Study
of Textbook Use in Engineering Problem Solving. J. Eng. Educ. 2013, 102, 269–288. [CrossRef]
64. Shafiei Sarvestani, M.; Mohammadi, M.; Afshin, J.; Raeisy, L. Students’ Experiences of e-Learning Challenges; a Phenomenological
Study. Interdiscip. J. Virtual Learn. Med. Sci. 2019, 10, 1–10.
65. Chattaraj, D.; Vijayaraghavan, A.P. The Mobility Paradigm in Higher Education: A Phenomenological Study on the Shift in
Learning Space. Smart Learn. Environ. 2021, 8, 15. [CrossRef]
66. Delgado, J.E.; Arellano, J. A Phenomenological Study of the Lived Experiences of Graduate Students Adapting Flexible Learning
Modality Due to COVID-19 Pandemic. Asian J. Educ. Soc. Stud. 2021, 15, 7–16. [CrossRef]
67. Shaw, S.C.K.; Hennessy, L.R.; Anderson, J.L. The Learning Experiences of Dyslexic Medical Students during the COVID-19
Pandemic: A Phenomenological Study. Adv. Health Sci. Educ. 2021. [CrossRef]
68. Moustakas, C. Phenomenological Research Methods; Sage Publications, Inc: Thousand Oaks, CA, USA, 1994.
69. British Educational Research Association Ethical Guidelines for Educational Research, 4th ed.; 2018. Available online: https://www.
bera.ac.uk/publication/ethical-guidelines-for-educational-research-2018-online (accessed on 20 January 2021).
Sustainability 2022, 14, 4584 30 of 30

70. Auerbach, C.F.; Silverstein, L.B. Qualitative Data: An Introduction to Coding and Analysis; New York University Press: New York,
NY, USA, 2003.
71. Lune, H.; Berg, B.L. Qualitative Research Methods for the Social Sciences; Pearson Education Limited: Harlow, UK, 2017.
72. Gibbs, G.R. Analyzing Qualitative Data; SAGE Publications Ltd.: London, UK, 2007.
73. Patton, M.Q. Qualitative Research & Evaluation Methods: Integrating Theory and Practice, 4th ed.; SAGE Publications Inc.: Thousand
Oaks, CA, USA, 2015.
74. Maxwell, J.A. Qualitative Research Design: An Interactive Approach, 3rd ed.; SAGE Publications Inc.: Thousand Oaks, CA, USA, 2013.
75. Cameron-Standerford, A.; Menard, K.; Edge, C.; Bergh, B.; Shayter, A.; Smith, K.; VandenAvond, L. The Phenomenon of Moving
to Online/Distance Delivery as a Result of Covid-19: Exploring Initial Perceptions of Higher Education Faculty at a Rural
Midwestern University. Front. Educ. 2020, 5, 203. [CrossRef]
76. Erlam, G.D.; Garrett, N.; Gasteiger, N.; Lau, K.; Hoare, K.; Agarwal, S.; Haxell, A. What Really Matters: Experiences of Emergency
Remote Teaching in University Teaching and Learning During the COVID-19 Pandemic. Front. Educ. 2021, 6, 397. [CrossRef]
77. Stepanyan, K.; Littlejohn, A.; Margaryan, A. Sustainable E-Learning: Toward a Coherent Body of Knowledge. J. Educ. Technol. Soc.
2013, 16, 91–102.
78. Coker, D. The Canary in the Mine: Remote Learning in the Time of COVID-19. J. Curric. Teach. 2020, 9, 76–87. [CrossRef]

View publication stats

You might also like