Which Strategies are Used in the Design of

Technical LA Infrastructure?: A Qualitative

Interview Study
1st George-Petru Ciordas-Hertel
Educational Technologies
DIPF |Leibniz Institute for Research and Information in Education
Frankfurt am Main, Germany
orcid.org/0000-0003-1589-7845
ciordas@dipf.de

2nd Jan Schneider

Educational Technologies
DIPF |Leibniz Institute for Research and Information in Education
Frankfurt am Main, Germany
orcid.org/0000-0002-1229-2579
schneider.jan@dipf.de

3rd Hendrik Drachsler

Educational Technologies
DIPF |Leibniz Institute for Research and Information in Education
Frankfurt am Main, Germany
orcid.org//0000-0001-8407-5314
drachsler@dipf.de

Abstract—In order to obtain a holistic perspective on learning,
technical infrastructure at an institutional level can be advanta-
geous for Learning Analytics (LA). If personal data is collected
and processed in such infrastructure, legal requirements are of
crucial importance. Recent studies have examined various aspects
of LA infrastructure, such as ethical trade-offs and stakeholder
needs. However, strategies for designing technical LA infrastruc-
ture at the institutional level and strategies for dealing with data
protection regulations have been lacking so far, although they
are crucial for the adoption of LA in Europe. The purpose
of this paper is to examine three research questions: (RQ1)
which strategies are currently used to design LA infrastructure;
(RQ2) how data protection and privacy affect the design of LA
infrastructure; and (RQ3) how could technical measures support
the adoption of LA. These research questions were investigated by
conducting eleven interviews with LA infrastructure developers
representing eight different higher education institutions and
ten different infrastructures. According to RQ1, the paper first
examines the domain specificity of LA infrastructure and the four
design strategies used. The paper also examines, according to
RQ2, the interviewees’ awareness of data protection, the conflict
with users’ consent, and the data subject rights. Finally, the
paper presents, in line with RQ3, the results of the strategies
in dealing with trust, stakeholder expectations, and engagement.
Researchers and infrastructure developers can use and adopt
these findings to improve their strategies for developing technical
LA infrastructure with regard to data protection, privacy, and
trust.
sign strategy, data protection, privacy, GDPR, qualitative inter-
view study
I. I NTRODUCTION
In recent decades, a growing number of educational institu-
tions have been using Learning Management Systems (LMSs)
such as Moodle1 to promote students with online learning
facilities. The users of these LMSs create digital traces that can
be utilized for Learning Analytics (LA). LA was established
with the purposes of understanding and optimizing learning
and the environments in which it occurs by collecting, measur-
ing, analyzing, and reporting data about learners and their con-
texts [1]. Small data is often a starter for LA infrastructure [2].
However, most LMSs do not have suitable tools for analyzing
the usage of data, nor can they be integrated with other LMSs
to gain a holistic perspective on learning. In order to gain
a more holistic perspective, more advanced infrastructure at
an institutional level is suitable. Therefore, we consider it
essential to acquire data from multiple heterogeneous data
sources such as LMSs, student registration, survey systems,
and library information systems, but also potentially cameras
and sensors [3] that equip the lecture halls. This type of
infrastructure design at the institutional level is subject to data

Index Terms—learning analytics, technical infrastructure, de- 1 https://moodle.com

978-1-7281-0930-5/20/$31.00 ©2020 IEEE 27–30 April, 2020, Porto, Portugal

2020 IEEE Global Engineering Education Conference (EDUCON)
Page 96

Authorized licensed use limited to: University of Vermont Libraries. Downloaded on July 26,2020 at 00:18:50 UTC from IEEE Xplore. Restrictions apply.
 characteristics of high volume, high velocity, and variety. Such
requirements qualify infrastructure as Big Data infrastructure
with all the associated challenges [4]. As there is a difference
between organizational and technical infrastructure, we want
to make clear that this paper is solely investigating technical
infrastructure for LA.
At present, European institutions must comply with the
GDPR [5]. When personal data is collected and processed,
legal privacy requirements are of crucial importance. Article
4 defines “personal data” as any information which is related
to an identified or identifiable natural person. These persons
are referred to in the GDPR as “data subjects”. Furthermore,
article 6 of the GDPR defines the lawful bases of processing
personal data. Among the lawful bases, the two bases “le-
gitimate interests” and “data subject consent” fit in particular
to the context of LA infrastructure in European institutions
of higher education. According to chapter 3 of the GDPR,
the GDPR grants data subjects certain rights they can execute
under particular conditions. Table I lists these “data subject
rights” in detail. As data processing is at the core of any
LA infrastructure, article 25 of the GDPR applies. The GDPR
introduces in article 25, the terms “data protection by design”
and “data protection by default”. Privacy is to be distinguished
from data protection. Data protection is essentially a technical
matter, while privacy is a legal matter. Data protection by
design declares that institutions have to take data protection
and privacy into account right from the beginning of software
design and through the entire development process of products
that involve processing personal data. In addition, data protec-
tion by default specifies that only personal data is processed
that is necessary for the respective purpose of processing. With
these two principles, data protection and privacy can no longer
be ex-post considerations but are an integral part of any LA
infrastructure design.
Recent studies on LA infrastructure examined several as-
pects, such as ethical trade-offs and stakeholder needs [6].
However, strategies used to design LA technically at an insti-
tutional level and strategies to deal with privacy constraints
were missing, although they are of crucial importance for
the adoption in Europe. Therefore, this paper examines the
strategies LA developers use to design LA infrastructure at
scale in higher education. Further, it investigates the influence
of data protection and privacy on the design strategies these
developers applied. Lastly, it explores how technical measures
might support the adoption of LA. For this purpose, we
conducted a qualitative interview study to analyze and gain
the knowledge of these developers. We were investigating the
following three research questions:
RQ1 Which strategies are currently used to design LA infras-
tructure?
RQ2 How do data protection and privacy affect the design of
LA infrastructure?
RQ3 How could technical measures support the adoption of
LA?
The following section will shortly summarize the related
978-1-7281-0930-5/20/$31.00 ©2020 IEEE
2020 IEEE Global Engineering Education Conference (EDUCON)
Page 97
Authorized licensed use limited to: University of Vermont Libraries. Downloaded on July 26,2020 at 00:18:50 UTC from IEEE Xplore. Restrictions apply.
literature and define the concepts used in this paper. Section III
then describes the methodology used to conduct the interviews.
After that, section IV will report on the main findings of the
interviews. Finally, the paper will conclude with a discussion
of the most important findings in section V.
II. R ELATED W ORKS
In a previously conducted structured literature review on LA
infrastructure [7] , we found many papers with drafts of LA
infrastructure. However, we found little information about the
design considerations and whether and how the authors applied
and maintained data protection and privacy in practice.
Over the past few years, there was plenty of literature
published on ethical frameworks, codes of ethical practice,
ethical consideration checklists, and ethical implications of
LA (e.g., [8]–[13]). Furthermore, there is literature concerned
with the challenge of defining universal principles for privacy
and data protection in a cross-cultural context [14], [15].
Several papers examine technical and social aspects that
need to be considered for the adoption of LA at higher
education institutions (e.g., [16]–[18]).
In connection with this work, there is a paper by Johanes
et al. [6]. They conducted interviews with “builders” of LA
infrastructure to report on their personal experiences. Their
findings primarily illuminate the themes: navigating designing
decisions, ubiquitous ethics, and human impact. Where their
study focuses on how builders perceive their role and activity,
we wanted our study to provide insights into the substantial
handling of data protection and privacy. Therefore, we have
explicitly focused our questions on practicable, technically
viable solutions for LA infrastructure.
Beyond that, we are not aware of any interview studies
with first-hand accounts that focus on experiences in designing
technical LA infrastructure with data protection by design.
With this interview study, we intend to close this gap.
III. M ETHODOLOGY
To learn more about strategies to design LA infrastructure,
including technical measures to address privacy concerns and
to support the adoption of such infrastructure, we conducted
eleven semi-structured interviews with LA infrastructure de-
velopers (see table II).
a) Interview Guide: The interview guide was designed
according to the three research questions mentioned in the
introduction and reflected in a semi-structured way. The ques-
tions were open questions, inviting the interview partners (IPs)
to introduce their areas of focus. The interview guide was
piloted with one technical proficient IP. This enabled us to
refine the interview guide.
b) Interview Partners: The IPs of this study were experts
in the development of LA infrastructure. They all are scholars
who were developing or have been developing technical in-
frastructure for institutions in higher education. When we use
the term develop, we refer to the design, implementation, or
operation of infrastructure. We identified potential respondents
by conducting snowball sampling using a focused sampling
27–30 April, 2020, Porto, Portugal
 TABLE I
A SHORT DESCRIPTION OF PARTICULAR ARTICLES AS SPECIFIED BY THE GDPR THAT FRAME THE DESIGN OF LA INFRASTRUCTURE . C ERTAIN
CONDITIONS NEED TO BE MET FOR QUALIFIED RIGHTS .

Article Data Subject Right Description

Art.15 Right of access Instructs the data controller to provide a copy of the personal data, free of charge, in an electronic format.
Art.16 Right to rectification Enforce the rectification of personal data.
Art.17 Right to erasure Entitles to have the data controller erase personal data (qualified right).
Art.18 Right to restriction Personal data may, except for storage, only be processed with consent (qualified right).
Art.20 Right to portability Demands a structured, commonly used and machine-readable format (qualified right).
Art.21 Right to object Stop at any time processing personal data on illegitimate grounds (qualified right).
Art.22 Automated decision making Not be subject to solely automated decision making, where legal or similarly significant effects are produced.
Art.25 Data protection by design Institutions have to take data protection and privacy into account right from the beginning of software design
and through the entire development process of products that involve processing personal data.
Art.25 Data protection by default Only personal data is processed that is necessary for the respective purpose of processing.

frame with multiple entry points [19]. As entry points, we satisfactorily categorized. Inductive and deductive approaches
chose authors from published literature and experts from con- allowed us to discover connections across coded categories.
ferences in the domain of LA corresponding to the themes of Another knowledgeable researcher from the field checked
the interview guide. Respondents across multiple institutions codes and categories for bias and misinterpretation. Wherever
and geographies with different infrastructures were selected this is relevant, we make the best effort to show alternative
to ensure thematic saturation. Finally, we conducted eleven or counter-balancing findings of multiple IPs. We state each
interviews with IPs located in Europe and Australia, represent- finding as verbatim as possible. Nevertheless, in order to
ing eight different institutions and ten different infrastructures. ease reading and increase comprehensibility, we removed filler
All but one IP were male. We show a summary of the IPs’ words, stutters, and repetitions. We emerged the categories
characteristics in table II. The column Role lists the role of within each theme by axial coding [21]. While the IPs empha-
the IPs as they stated it, or we were able to derive them. We, sized different aspects of their experiences, several IPs shared
first of all, differentiate between Leader and Researcher. All most of the experiences. Therefore, we believe that we can
of the IPs are actively involved in research, but we further give a sound overview. We demonstrate this by referring to
wanted to differentiate those who are empowered to set the the IPs for evidence.
direction, raise funding, and determine the overall vision. With d) Terminology: In the findings, we use the Big Data
the terms Learning and Technical, we additionally wanted to Value Chain (BDVC) [22] to map the statements of the Inter-
specify an IP’s contribution to the infrastructure as involved view Partners (IPs) for RQ1. The BDVC can be used to model
in the software development or technical design, respectively, the high-level activities that comprise any information system.
the educational or learning design. We, therefore, believe that the BDVC is a suitable model
for the design of LA infrastructure. The BDVC identifies
TABLE II the following key high-level activities: Data acquisition, data
T HIS TABLE SHOWS AN ANONYMIZED LIST OF THE INTERVIEW PARTNERS , analysis, data curation, data storage, and data usage. Data
THE LOCATION OF THEIR INSTITUTIONS , AND THEIR RESPECTIVE ROLES .
acquisition is the process of gathering, filtering, and cleaning
Interview Country Role data before it enters any storage solution where data analysis
IP01-NL Netherlands Learning & Technical Researcher can be performed. Data analysis is concerned with making
IP02-NL Netherlands Learning & Technical Leader the raw data obtained usable both for decision-making and
IP03-ES Spain Learning & Technical Researcher
for domain-specific use. Data analysis comprises the explo-
IP04-AU Australia Learning & Technical Researcher
IP05-EE Estonia Learning & Technical Researcher ration, transformation, and modeling of data with the aim of
IP06-NL Netherlands Technical Researcher highlighting and synthesizing relevant data and gaining useful
IP07-GB United Kingdom Learning Leader hidden information with high potential from a stakeholder
IP08-AU Australia Learning & Technical Leader
IP09-DE Germany Learning & Technical Leader
perspective. Data curation is the active management of data
IP10-AU Australia Learning & Technical Leader throughout its lifecycle to ensure that it meets the necessary
IP11-GB United Kingdom Learning & Technical Leader data quality requirements for its effective use. Data curation
processes can be divided into various activities such as content
c) Analysis: The interview sessions lasted from 40 to creation, selection, classification, transformation, validation,
150 minutes. After removing identifying information, a con- and long-term archiving. Data storage is the persistence and
tractor transcribed the recordings of the interviews. The data management of data in a scalable manner that meets the needs
synthesis process entailed refining the themes as well as of applications that require rapid access to the data. Data usage
identifying the codes by applying standard qualitative coding includes the data-driven activities that require access to data,
principles [20]. We grouped these codes then into broader its analysis, and the tools required to integrate the data analysis
categories. In an iterative process, we sorted, challenged, within the activity.
refined, and redefined the categories until all responses were As some of the IPs refer to data warehouses and data

978-1-7281-0930-5/20/$31.00 ©2020 IEEE 27–30 April, 2020, Porto, Portugal

2020 IEEE Global Engineering Education Conference (EDUCON)
Page 98

Authorized licensed use limited to: University of Vermont Libraries. Downloaded on July 26,2020 at 00:18:50 UTC from IEEE Xplore. Restrictions apply.
 lakes as important technical concepts for LA infrastructure,
we provide a definition here to get common ground on the
terminology. Amazon, one of the biggest providers of Big Data
infrastructure, for example, defines a data warehouse as:
A data warehouse is a central repository of informa-
tion that can be analyzed to make better-informed
decisions. Data flows into a data warehouse from
transactional systems, relational databases, and other
sources, typically on a regular cadence. Business
analysts, data scientists, and decision-makers access
the data through business intelligence (BI) tools,
SQL clients, and other analytics applications.2
Alternatively, a data infrastructure can be built as a data lake.
Amazon, for example, defines a data lake as:
A data lake allows you to store all your structured
and unstructured data, in one centralized repository,
and at any scale. With a data lake, you can store
your data as-is, without having to first structure the
data, based on potential questions you may have in
the future. Data lakes also allow you to run different
types of analytics on your data like SQL queries, Big
Data analytics, full-text search, real-time analytics,
and machine learning to guide better decisions.3
Furthermore, some IPs discussed the advantages and dis-
advantages of cloud computing. We, therefore, provide a
definition of the terms Infrastructure-as-a-Service (IaaS) and
Platform-as-a-Service (PaaS), which are mentioned in the
following sections. IBM, a major provider of cloud solutions,
defines IaaS as:
IaaS is a form of cloud computing that delivers fun-
damental compute, network, and storage resources
to consumers on-demand, over the internet, and on a
pay-as-you-go basis. IaaS enables end users to scale
and shrink resources on an as-needed basis, reducing
the need for high, up-front capital expenditures or
unnecessary “owned” infrastructure, especially in
the case of “spiky” workloads.4
In contrast, IBM, for example, defines PaaS as:
Platform-as-a-Service (PaaS) is a cloud computing
model that provides customers a complete plat-
form — hardware, software, and infrastructure —
for developing, running, and managing applications
without the cost, complexity, and inflexibility of
building and maintaining that platform on-premises.
The PaaS provider hosts everything — servers, net-
works, storage, operating system software, databases
— at their data center; the customer uses it all for a
monthly fee based on usage and can purchase more
resources on-demand as needed.5
These four terms are relevant and will be used extensively
in later sections of this paper.
2 https://aws.amazon.com/data-warehouse
3 https://aws.amazon.com/big-data/what-is-a-data-lake
4 https://www.ibm.com/cloud/learn/iaas
5 https://www.ibm.com/cloud/learn/paas
978-1-7281-0930-5/20/$31.00 ©2020 IEEE
2020 IEEE Global Engineering Education Conference (EDUCON)
Page 99
Authorized licensed use limited to: University of Vermont Libraries. Downloaded on July 26,2020 at 00:18:50 UTC from IEEE Xplore. Restrictions apply.
IV. F INDINGS
This qualitative interview study intends to understand the
practices of the experts in LA infrastructure development
beyond what is published in papers and documented online.
According to the structure of the interview schedule, the
findings are organized around the three research questions:
which strategies are currently used to design LA infrastructure
(RQ1); how data protection and privacy affect the design of
LA infrastructure (RQ2); and how could technical measures
support the adoption of LA (RQ3).
A. RQ1: Designing a Technical Infrastructure
From the answers of the IPs, we recognized that RQ1 has
two dimensions that are relevant for the design aspects of
technical LA infrastructure. Those are the dimensions domain
specificity and design strategies. In the following, we report
the main outcomes of both dimensions.
1) Domain Specificity: Since this paper deals with the
design and development of LA infrastructure, we asked the IPs
what it is that makes an infrastructure specific to LA. This is
important since there is plenty of established technology and
software architecture in the industry that can be used as a basis
for the design of LA infrastructure. In the following, we use
the Big Data Value Chain (BDVC) [22] to map the statements
of the IPs (see paragraph III-0d).
a) Data acquisition: Some IPs mentioned data acquisi-
tion as specific to education (IP05-EE, IP08-AU). This is in
their opinion because of the data models used by learning
applications, e.g., data model concerned with the relations
between people, and the heterogeneity of the data sources.
They state as well that the infrastructure needs to be more
flexible to be able to adapt it to different upcoming learning
settings, e.g., new data sources, and new learning scenarios.
This quote showcases the specificity by referring to Amazon6 :
With their data structure, how do they know whether
somebody learned something? It’s impossible.
b) Data analysis: Further IPs refer to the data analysis as
the differential high-level activity (IP05-EE, IP09-DE, IP11-
GB). These IPs refer to the data analysis as being more com-
plex because of the complex behavior of the data subjects in
learning processes. This reflects in their opinion in the data and
makes synthesizing and extracting useful hidden information
far more difficult. The following quote demonstrates this:
... what distinguishes Learning Analytics from other
forms of analytics is that learning is complex, and
the way how students learn is non-linear, which
is odd at times. ... I think learning is much more
complex than just consumer behavior.
c) Data curation: Most IPs explain that data curation is
most specific to their LA infrastructure (IP01-NL, IP03-ES,
IP04-AU, IP05-EE, IP06-NL, IP09-DE, IP10-AU). The IPs
mention that the technical infrastructure is not domain-specific,
but the classification and prediction is. The expertise of their
6 https://www.amazon.com/
27–30 April, 2020, Porto, Portugal
 work lies in their opinion in the selection, transformation, and
classification of the data. These activities result in specific
prediction models and indicators. As one IP explains:
We have the data, but how can we interpret the data?
We have a system. It is quite open in the sense that
it supports certain didactics, no problem, but it is
not aware of the didactics. And there are a lot of
problems we encounter. There is openness, which
is fine, I think, for the users and for the course
authors and course designers because they can make
all kinds of designs. How they like it, but it’s not fine
from the perspective of analyzing those data quite
easily. For example, we have courses where you
can do assignments and you can rate assignments of
students. But those assignments can mean a lot of
things. From our design in the learning management
system we can’t distinct between the one or the
other.
d) Data storage: Only one IP associated the activity of
data storage as specific to LA infrastructure. This IP mentioned
that due to the number of students, the activity data has become
so large that the institution is forced to delete most raw data
every week. The institution can preserve only the processed
data.
e) Data usage: Only a minor number of statements men-
tion data usage (IP05-EE, IP06-NL, IP09-DE). IPs distinguish
data usage for education as following a different intention
as opposed to, for example, marketing, where the aim is to
persuade the user.
I think that there is a major difference between what
we want in Learning Analytics and what they say
Amazon does. Amazon wants to know more about
you to persuade you to do something. Learning
Analytics does not want to persuade you. It only
wants to inform you about the decision making
process.
In general, we can state that the IPs consider the technology
and the design of technical infrastructure in LA to be domain
agnostic. The domain specifics are, in the opinion of the
IPs, defined in the capture of the pedagogy, the discovery
of learning, the presentation of results, and the modeling of
intervention.
2) Design Strategies: In our analysis of the interview
transcripts, we were able to identify various strategies the IPs
applied in their design of LA infrastructure.
Two IPs were talking about the convolution of their infras-
tructure processes (IP04-AU, IP05-EE). Over time their LA
infrastructure has grown in complexity, and technical dept7 .
This ended up in convoluted processes and inconsistent data
states. This is how the IP puts it:
There are many transformations happening at dif-
ferent times. So certain things will get exported
7 Technical debt is a concept in programming that reflects the additional
development effort that arises when easy-to-implement code is used in the
short term rather than applying the best overall solution.
978-1-7281-0930-5/20/$31.00 ©2020 IEEE
2020 IEEE Global Engineering Education Conference (EDUCON)
Page 100
Authorized licensed use limited to: University of Vermont Libraries. Downloaded on July 26,2020 at 00:18:50 UTC from IEEE Xplore. Restrictions apply.
weekly at 3:00 PM. Some other things happen daily
in the morning. And then you get this connection
like for three students there are no grades. How is
this possible? Well, because they enrolled within this
period of time when one database has them, and the
other does not. So we know in 3 more days there
will be data for them. So you have many of those
small ETL8 processes of transforming the data, and
it becomes very convoluted.
This convolution is the reason for their effort to promote a
collaborative design of technical infrastructure. The design
should, in their opinion, be the result of the collaboration of
all stakeholders, e.g., technical and learning experts, students,
teachers, and administration.
While some of the IPs focus on custom solutions to each
LA use case (IP01-NL, IP03-ES, IP05-EE, IP09-DE), e.g.,
standalone tools or course-level analysis, others invest in
broader infrastructure at the department (IP07-GB, IP10-AU)
or institutional level (IP02-NL, IP04-AU, IP06-NL, IP08-AU,
IP11-GB). The IPs with custom solutions claim that each LA
use case has its specific requirements and needs, therefore
particular measures to support its pedagogy. In contrast, most
of the IPs consider a commonly shared infrastructure for LA
on micro, meso, and macro-level, i.e., course, department, and
institution-level, as feasible and beneficial. They declare that
LA is level-specific because of somewhat similar requirements
resulting from the used tools and the context. Institutional
level generic infrastructure promises them advantages such as
shared software development and operations investment, but
some IPs were pointing out the dangers of trying to bring
about a one-size-fits-all solution. As one IP explains it:
To have a common infrastructure, you need to have
common goals.
What the IPs essentially argue is that technology can be shared,
but LA can not be standardized. They are worried that putting
an extended effort into the standardization of LA infrastructure
might end up with an LA process, essentially only providing
answers to shallow questions:
I am quite skeptical because I think what they
are eventually getting towards is what we call the
strategy information office model. Do you still have
a heartbeat? Are you still alive? These kinds of
things. But the more what I would call formative
analytics to real kind of analytics, it helps you to
provide useful, actionable feedback, to help you to
go to the next step as a learner.
More preferably, they argue that LA should provide useful,
actionable feedback to help the stakeholders to go to the next
step to improve the learning. Therefore, LA infrastructure
should stay flexible.
When the IPs talked about institutional level LA infrastruc-
ture, they had two different concepts in mind. Most had set up
or were heading towards a data warehouse concept. Only three
8 Extract, transform, load
27–30 April, 2020, Porto, Portugal
 IPs (IP04-AU, IP06-NL, IP08-AU) discussed the advantages of
a data lake concept in comparison to a data warehouse concept.
Especially one IP was discussing the struggles with identifying
learning from pre-aggregated views created by technology
experts without the learning domain expertise:
... a real pain is we’re not given access to the raw
tables. We are given access to many views. That is
really a problem, because if I am given access to the
raw tables, I can learn the structure of the system.
Whereas if I am given the views, I really cannot
learn that. That is the problem because views are
already pre-aggregated.
Confronted with similar issues, another IP was already in
the process of transforming the institution’s infrastructure into
a data lake. One reoccurring argument was preserving the
usefulness of the data by keeping it as raw as possible.
Another argument mentioned was keeping track of the history
of change of data. This statement states the IP’s wider vision:
The idea is to have a Big Data lake ... have a lot of
data streams that are for management, for Learning
Analytics, for researchers. They take out the data
they need and use the format or the systems which
are most suited for their application.
When it came to the operation of the respective infrastruc-
ture, some IPs had also reported on the various considerations
of operating the infrastructure on-premise or in the cloud
(IP01-NL, IP06-NL, IP08-AU). Specifically reported were
the advantages of using an Infrastructure-as-a-Service (IaaS)
or a Platform-as-a-Service (PaaS) approach. One IP praised
the simplicity and performance of using PaaS compared to
previously developed on-premise LA infrastructure. The IP
was able to develop a prototype in days rather than weeks.
Other IPs also considered moving part of the infrastructure to
the cloud but rejected this for privacy and security concerns.
Losing direct control of the data and stealing the data were
dominant phrases used. The outstanding infrastructure design
of one IP involves the cooperation of multiple educational
institutions to share a data pool. Where other institutions are
concerned with the rapid development of technology, this IP
has developed a concept for a shared data lake infrastructure
hosted in the cloud, designed and operated in cooperation
with industry partners. The cooperation with private software
development and hosting partners is supposed to reduce the
investment of time and resources in order to keep up with
innovations in technology. This reduction would instead be
invested in improving the learning specifics of the infrastruc-
ture. The IP told it this way:
The problem of this is that it hinders our ability to
conduct research in a collaborative manner as well.
While we are sort of working on developing our own
prototypes, it doesn’t really facilitate into university
collaboration because it’s hard to share data or bring
data in or export particular prototypes or for it to
be used in other institutional contexts to understand
sort of how should models work in that context.
978-1-7281-0930-5/20/$31.00 ©2020 IEEE
2020 IEEE Global Engineering Education Conference (EDUCON)
Page 101
Authorized licensed use limited to: University of Vermont Libraries. Downloaded on July 26,2020 at 00:18:50 UTC from IEEE Xplore. Restrictions apply.
TABLE III
T HE FOUR STRATEGIES THE IP S USED FOR THE DESIGN OF THEIR LA
INFRASTRUCTURE
Design Strategy Description
Experiment A design to fit an experimental setup,
e.g., the study of biometric data from learners
Custom Application A design supporting learning in the
restricted context of one application,
e.g., an LMS course learning progress indication
Data Warehouse A design to synthesis pre-processed data
organized into a schema that describes the layout
and type of data
Data Lake A design to synthesis raw data without
pre-defining structure or schema when acquired
Not only is it hard for us to compete and keep up
with technological innovations, current infrastructure
doesn’t really allow for research, collaboration and
sharing of data.
Further, the IP points out that a shared data lake of multiple
institutions could reduce the bias in the data of a single
institution as the data is informed by more than only one
pedagogy.
The four design strategies we could identify in the state-
ments of the IPs are summarized in table III.
B. RQ2: Aligning with Data Protection and Privacy
As we explained earlier, we are aware of the fact that
the GDPR is not legally binding to all of the IPs. However,
we regard it as a framework for data protection and privacy
that should apply to any infrastructure that handles sensitive
personal data. Data protection by design, as specified in the
GDPR, was, in most cases, not a priority of the IPs. However,
the GDPR seems to have made the IPs take privacy more into
consideration, where they did not before. In the following,
we will report about challenges, implications, and conflicts of
interest in implementing privacy in LA infrastructure.
Some IPs represent the opinion that the introduction of the
GDPR has changed the general perception of privacy (IP06-
NL, IP07-GB, IP08-AU, IP09-DE). In some cases, it even
seems that the GDPR made people recognize privacy for the
first time. Meanwhile, almost all IPs report that data subjects
have never requested any GDPR related data subject rights
(IP01-NL, IP02-NL, IP03-ES, IP05-EE, IP08-AU, IP09-DE,
IP10-AU) to their best knowledge. In other cases, which are
all located in Europe, data subjects have so far only requested
an export of their data (IP06-NL, IP07-GB, IP11-GB).
There were two different opinions between the IPs regarding
obtaining consent. Some IPs argue that they do not need the
consent of the students to do LA because it is obligatory to
the teaching-learning process. One IP states that those who
do not consent have the choice to pick another institution.
Even stronger is the argument that the university has a duty-
of-care that is balanced against the consent of a student.
Others seek to provide the user with individual automated
privacy settings aligned with the data subject rights as specified
by the GDPR. When we further investigated the motivation
27–30 April, 2020, Porto, Portugal
 behind not specifically asking for the consent of the students,
multiple concerns were raised. For one, the IPs are concerned
that even one student opting-out of LA might reduce or
undermine the expressiveness of predictions (IP02-NL, IP03-
ES, IP04-AU, IP05-EE, IP07-GB). Furthermore, the quality
of non-personalized LA services becomes reduced. Despite
these concerns, some of these IPs believe that most students
will not disable LA when clearly explained and shown the
benefits of LA at the task. In addition to these concerns, the
IPs stated several technical issues. These include that some
data is technically necessary to ensure the functionality of the
software, and therefore may not be subject to the requirements
of privacy. An example of such data is the storage of name,
postal address, and e-mail address. This data is necessary to
contact a student. Another example mentioned was that user
activity data is logged for security reasons. However, where the
institution might not always be able to restrict the acquisition
of data; it could limit the LA processing of data:
It’s not possible to turn some data collection mech-
anisms off. All the consent can be is: What are you
using it for? You can’t turn off the activity stream
in your VLE. But you can say: Don’t use this to
give me any help. Just use this to answer support
questions when the VLE breaks. I don’t want my
tutor to see this information.
The introduction of the GDPR has partially unsettled the
European IPs. They were uncertain how they should apply the
GDPR in practice. This uncertainty caused some institutions to
hesitate with the implementation of personalized LA or even
stop already established LA. The right to be forgotten was
mentioned in most cases by the IPs. For these IPs, this right
seems to be in contradiction with other legal regulations, such
as the requirement of traceability, e.g., the institution has a
legal obligation to keep student records for a certain number
of years, but students have a right to be forgotten. Likewise,
there is uncertainty concerning data ownership. The IPs asked
themselves how an institution can decide which data is owned
by whom, e.g., if one student mentions another by name in a
forum post. To other IPs, it is quite apparent that the students
consented by enrollment to the data collection. The argument
is that since the institution collected data by consent, it should
be the owner of the data. A few IPs intended to automate the
data subject rights and give the students direct agency. These
IPs concentrated on technical issues such as how to identify
data or keep track of consent in the distributed ecosystem of an
institution. They were also trying to figure out how to enforce
the GDPR in closed-source third-party software utilizing an
API9 . Concerning LA, an option mentioned was to isolate the
LA infrastructure from the institutional software ecosystem
with distinct data flows in and out. This way, every data subject
right could be implemented independently within the context
of the LA infrastructure.
9 Application-Programming-Interface
978-1-7281-0930-5/20/$31.00 ©2020 IEEE
2020 IEEE Global Engineering Education Conference (EDUCON)
Page 102
Authorized licensed use limited to: University of Vermont Libraries. Downloaded on July 26,2020 at 00:18:50 UTC from IEEE Xplore. Restrictions apply.
C. RQ3: Supporting the Adoption of LA
One aspect mentioned by the IPs that plays a role in the
adaptation of LA was the veracity of the data (IP02-NL, IP04-
AU, IP05-EE, IP11-GB). IPs claimed that data is a second
pillar an institution can rely on next to the experiences of
its staff. Further, they explained that data is a pillar the
institutional administration needs to first trust in order to make
use of it. One underlying problem mentioned was the historical
development of the software ecosystem. The data changed
with so many upgrades that the administration believed it
changed its meaning and got corrupted. Another problem
mentioned was that data sources were only accessible by
specific technical staff, which were, therefore, the only ones
able to handle the data. In such cases, the IPs reasoned, seeking
a technical solution to improve the adoption of LA would not
help, as it has to be taken into account that some stakeholders
do not even trust technology itself. The IPs explained that
there could be several reasons for this, such as the lack of
even essential technical expertise.
Similar to this situation, IPs explained that stakeholders
they encountered in some cases did not trust LA because
the analytical results provided by LA did not match their
expectations. Therefore, the IPs argued that LA needs to allow
inspection and comprehension of the prediction model, which
they explained might not be given, for example, in the case of
a neuronal network model. Approaches applied by the IPs to
tackle such lack of trust in the data and the analytical process
were co-creation and full transparency. An IP formulated it as
follows:
Universities will be able to provide legitimacy to the
tool-set they are developing, or we are developing
together. Co-creation and full transparency, I think
ideally should not only include the user and the
people who are offering the tools, but also the ones
who are developing the tools.
Engaging the instructors and the administration into decisions
and ideas increased their adoption and appreciation to LA
within the institution. Furthermore, the IPs noted that being
fully transparent to the data subjects as well as enabling user
integration and user interaction might raise their understanding
of LA applications. Thus, they might start to utilize these
applications and start to understand or see the benefits of it.
The IPs argued that implementing LA solely from the
perspective of a technical engineer, does not live up to its
social impact. One example given was that of high impact
consequences based on 80 percent accuracy out of a black-
box model. The IPs pointed out that LA is not ubiquitous
but restricted to its data sources. Thus this needs to be taken
into account for any claim LA makes. Therefore, many IPs
had spoken out in favor of putting more effort into improving
the comprehensibility of LA for the students (IP01-NL, IP02-
NL, IP03-ES, IP04-AU, IP09-DE, IP11-GB). One approach
mentioned was to be transparent about the process. This
included showing where the data was acquired from, how it is
processed, and which the metrics are that feed the indicators.
27–30 April, 2020, Porto, Portugal
 This certainly favors simple infrastructure. It was argued that
if done well, technical services around transparency enable
data subjects to investigate. The investigation might raise
their confidence and enables them to engage more with LA.
The same group of IPs, however, had also put forward some
arguments against more transparency. One IP feared that the
students would concentrate on targeting the metrics:
If you’re going to use any of these analytics for
grading. Then it’s immediately a big gamble. If
you’re going to say, we’re going to use automated
systems that will say, how engaging your messages
are in discussions? How do you count engagement?
We just count the number of characters. Oh, really?
Okay. I’m just gonna randomly type. ... As soon
as people know this is how you measure it, by the
number, then it becomes a bad measure because now
suddenly everybody will target to have that metric
satisfied. So then it becomes a bad metric.
This goes along with the argument of some IPs that students
should not see all their data because this might influence
their behavior around LA. An analogous example mentioned
was that doctors also do not reveal to their patients all the
diseases they suspect to be related to their symptoms. Some
IPs even advise against implementing an extended amount of
technical measures and notices about inspection possibilities
and transparency into the user interfaces. Instead of raising the
user’s trust in the application, they argue, that such emphasis
might raise awareness of possible negative consequences.
Therefore, such an effort might increase scrutiny and even
anxiety towards LA.
An appealing aspect mentioned is the role of a trusted
third party. Such a trustee could generate trust if stakeholders
do not fully grasp the processes of an institution for the
reasons given above, or the institution does not want to be
transparent to protect their intellectual property. An example
given for such a trustee was a certification authority that issues
digital certificates. A trustee could inspect certain aspects of
infrastructure and certify the correctness and conscientiousness
of those.
At last, one aspect that was very important to the IPs and
was mentioned by them several times was the aspect of trust
in the principles of the institution:
If you want to develop trust, it’s not only about
developing agency and interaction, but people also
need to understand sort of the underlying, if you like,
principles or values that are used to inform certain
recommendations or data collections, data storage,
use of data, all these kind of things.
V. D ISCUSSION
The first reason why we conducted this study was to gain
insights into the strategies LA developers use to design and
implement LA infrastructure at an institutional level in higher
education. The IPs stated that general Big Data infrastructure
is suitable to design institutional level LA infrastructure. As
978-1-7281-0930-5/20/$31.00 ©2020 IEEE
2020 IEEE Global Engineering Education Conference (EDUCON)
Page 103
Authorized licensed use limited to: University of Vermont Libraries. Downloaded on July 26,2020 at 00:18:50 UTC from IEEE Xplore. Restrictions apply.
we interpret the statements of the IPs, LA seems to have
no specific technical requirements that do not also apply to
regular Big Data infrastructure. Additionally, the IPs explained
where to put the focus on when designing infrastructure
for learning. The crucial parts are capturing the semantics,
identifying the need to support from analyzing the data with
the stakeholders, and presenting the results or an intervention
to the stakeholder at the right time. In our study, the IPs told us
about their diverse strategies used to design LA infrastructure.
The strategies are dependent on the context and the use cases
in which LA was done. We identified four design strategies:
experimental, custom application, data warehouse, and data
lake design. Experimental LA designs are done to serve a
specific use case or educational design. Therefore, the design
can include any technology, software architecture, or data
format that fits the expertise of the team. Similar conditions
apply to LA performed as or within a particular application. At
another scale is LA infrastructure based on a data warehouse
design. In this design strategy, data is acquired from various
data sources in one place in a highly structured way. The
benefit of a data warehouse is that the usage pattern feels
familiar to most IT staff. The issue with this design seems
to be the ETL processes feeding the data warehouse. Be-
cause of experiences with convoluted processes and corrupted
data, some IPs promoted the co-creation of LA for the data
warehouse design strategy. However, the data warehouse can
only provide particular views on the collected data what also
limits the potential insights of LA. The fact that the data
they are working with is already pre-processed hampers, in
their opinion, the validity of their research. Therefore, some
IPs have been considering the data lake design as the most
effective strategy. A data lake can provide full access to the
raw data without any restrictions on their LA design. This
design might advance LA by proving “a source of truth”,
as one IP stated. Nevertheless, the architecture of data lakes
might be challenging for IT staff, while the benefits are clear
to researchers. There were additional discussions about the
usage of cloud computing and their impact on privacy, security,
and trust. Pearson et al. [23], for example, are suggesting
transferable approaches include addressing Data Security Mit-
igation by the fully homomorphic encryption scheme [24] and
Cross-Layer Interaction and Enforcement by machine-readable
policies via policy languages like XACML10 .
The second reason why we conducted this study was to
understand the influence of data protection and privacy on the
design of LA infrastructure. There are two groups in the IPs.
One group did not want to invest in further technical privacy
measures. On the one hand, they believed to act on legitimate
interests. On the other hand, they were worried about the
consequences of further involving the data subjects. The other
group sought to force the interaction of students with technical
privacy measures. They wanted to engage the data subject in
order to raise their agency. Experiment and custom application
designs are a closed system with clear objectives. Therefore,
10 https://www.oasis-open.org/standards#xacmlv3.0
27–30 April, 2020, Porto, Portugal
 privacy regulations can be handled easier. In a data warehouse
design identifying and fulfilling privacy regulations might be
not so hard to handle since data structure and processes are
straight and transparent from the perspective of the IT staff.
This does not seem to apply equally to a data lake design.
The IPs mentioned several technical challenges, such as data
identification, enforceability, and data ownership. An issue not
discussed was that of the implications of fulfilling the rights
of the data subject. Singh and Cobbe [25], for example, argue
that data controllers must also take into account all security
risks in fulfilling the rights of the data subject, in particular
concerning the disclosure of the technical infrastructure or the
controller’s organizational processes to data thieves. Research
conducted by the DEFeND-Project11 , in particular, has been
working on the issue of designing a data protection by design
platform for GDPR compliance. Piras et al. [26] are proposing
the design of an architecture for a platform, which can reuse
and integrate peculiarities of heterogeneous tools, and support
organizations in achieving GDPR compliance.
The third and last reason why we conducted this study was
the investigation of how technical measures might support the
adoption of LA. Firstly, the introduction of LA cannot be
achieved by implementing technical measures alone. At the
social level, trust must be built between the various stakehold-
ers in LA. Secondly, in order to raise engagement with LA,
the IPs promoted co-creation and direct engagement with LA.
The direct feedback of the addressee about the correctness,
usefulness, or comprehensibility of the LA process and the
interventions could give LA a boost. For example, enabling
data subjects to label patterns of learning identified in the
acquired data could be useful to improve the training data
of prediction models. Thirdly, simplicity, and sophistication
in the design of infrastructure increases confidence in the
results and enables comprehensibility. It can be challenging
to disclose decision-making processes if they are opaque or
even based on unfounded assumptions. Any infrastructure
is vulnerable to accumulate technical debt over time, which
ultimately drives to convoluted processes [27]. Thirdly, where
full transparency is not possible or desirable, or where the
institution faces problems with its reputation, a trustee could
enable and consolidate trust through the vouching mechanism.
Finally, as so many IPs have stressed, when it is not clear
to data subjects why their personal data is requested, or how
and by whom it is processed, this lack of control and visibility
of the institutions LA could lead to suspicion and ultimately
distrust. As a critical aspect of trust is reputation, the basis of
every relationship in education is not only one of the technical
measures but one of the principles.
R EFERENCES
[1] D. Gasevic, G. Siemens, and C. P. Rose, “Guest Editorial: Special
Section on Learning Analytics,” IEEE Transactions on Learning
Technologies, vol. 10, no. 1, pp. 3–5, jan 2017. [Online]. Available:
http://ieeexplore.ieee.org/document/7880729/
11 https://www.defendproject.eu
978-1-7281-0930-5/20/$31.00 ©2020 IEEE
2020 IEEE Global Engineering Education Conference (EDUCON)
Page 104
Authorized licensed use limited to: University of Vermont Libraries. Downloaded on July 26,2020 at 00:18:50 UTC from IEEE Xplore. Restrictions apply.
[2] T. Broos, K. Verbert, G. Langie, C. Van Soom, and T. De Laet, “Small
data as a conversation starter for learning analytics,” Journal of Research
in Innovative Teaching & Learning, vol. 10, no. 2, pp. 94–106, 2017.
[3] D. Di Mitri, J. Schneider, M. Specht, and H. Drachsler, “From signals
to knowledge: A conceptual model for multimodal learning analytics,”
Journal of Computer Assisted Learning, vol. 34, no. 4, pp. 338–349,
aug 2018. [Online]. Available: http://doi.wiley.com/10.1111/jcal.12288
[4] B. Kei and D. Editor, Big Data and Learning Analytics in Higher
Education, B. Kei Daniel, Ed. Cham: Springer International Publishing,
2017. [Online]. Available: http://link.springer.com/10.1007/978-3-319-
06520-5
[5] C. o. t. E. Union and E. Parliament, “Regulation (EU) 2016/679
of the European Parliament and of the Council of 27 April 2016
on the protection of natural persons with regard to the processing
of personal data and on the free movement of such data, and
repealing Directive 95/46/EC (General Da,” Official Journal of the
European Union, vol. L119, pp. 1–88, 2016. [Online]. Available:
https://eur-lex.europa.eu/eli/reg/2016/679/oj
[6] P. Johanes and C. Thille, “The heart of educational data infrastructures
- Conscious humanity and scientific responsibility, not infinite data and
limitless experimentation,” British Journal of Educational Technology,
vol. 50, no. 6, pp. 2959–2973, nov 2019. [Online]. Available:
https://onlinelibrary.wiley.com/doi/abs/10.1111/bjet.12862
[7] G.-P. Ciordas-Hertel, J. Schneider, S. Ternier, and H. Drachsler,
“Adopting trust in learning analytics infrastructure: A structured
literature review,” Journal of Universal Computer Science,
vol. 25, no. 13, pp. 1668–1686, dec 2019. [Online]. Available:
http://www.jucs.org/jucs 25 13/adopting trust in learning
[8] R. Ferguson, T. Hoel, M. Scheffel, and H. Drachsler, “Guest editorial:
Ethics and privacy in learning analytics,” Journal of Learning
Analytics, vol. 3, no. 1, pp. 5–15, apr 2016. [Online]. Available:
http://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4912
[9] H. Drachsler and W. Greller, “Privacy and analytics,” in Proceedings
of the Sixth International Conference on Learning Analytics &
Knowledge - LAK ’16, vol. 25-29-Apri. New York, New
York, USA: ACM Press, 2016, pp. 89–98. [Online]. Available:
http://dl.acm.org/citation.cfm?doid=2883851.2883893
[10] N. Sclater, “Developing a code of practice for learning analytics,”
Journal of Learning Analytics, pp. 16–42, apr 2016. [Online]. Available:
http://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4512
[11] P. Prinsloo and S. Slade, “Ethics and Learning Analytics: Charting
the (Un)Charted,” in Handbook of Learning Analytics. Society for
Learning Analytics Research (SoLAR), may 2017, pp. 49–57. [Online].
Available: https://solaresearch.org/hla-17/hla17-chapter4
[12] K. Kitto and S. Knight, “Practical ethics for building
learning analytics,” British Journal of Educational Technology,
vol. 50, no. 6, pp. 2855–2870, nov 2019. [Online]. Available:
https://onlinelibrary.wiley.com/doi/abs/10.1111/bjet.12868
[13] S. Slade, P. Prinsloo, and M. Khalil, “Learning analytics at
the intersections of student trust, disclosure and benefit,” in
Proceedings of the 9th International Conference on Learning
Analytics & Knowledge - LAK19. New York, New York,
USA: ACM Press, 2019, pp. 235–244. [Online]. Available:
http://dl.acm.org/citation.cfm?doid=3303772.3303796
[14] T. Hoel, D. Griffiths, and W. Chen, “The Influence of Data Protection
and Privacy Frameworks on the Design of Learning Analytics
Systems,” in Proceedings of the Seventh International Learning
Analytics & Knowledge Conference on - LAK ’17. New York, New
York, USA: ACM Press, 2017, pp. 243–252. [Online]. Available:
http://dl.acm.org/citation.cfm?doid=3027385.3027414
[15] T. Hoel and W. Chen, “Privacy and data protection in learning
analytics should be motivated by an educational maxim —
Towards a proposal,” Research and Practice in Technology Enhanced
Learning, vol. 13, no. 1, p. 20, dec 2018. [Online]. Available:
https://telrp.springeropen.com/articles/10.1186/s41039-018-0086-8
[16] D. Gašević, S. Dawson, and J. Jovanović, “Ethics and privacy as
enablers of learning analytics.” Journal of Learning Analytics,
vol. 3, no. 1, pp. 1–4, apr 2016. [Online]. Available:
http://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4956
[17] Y.-S. Tsai and D. Gasevic, “Learning analytics in higher education -
Challenges and policies,” in Proceedings of the Seventh International
Learning Analytics & Knowledge Conference on - LAK ’17. New
York, New York, USA: ACM Press, 2017, pp. 233–242. [Online].
Available: http://dl.acm.org/citation.cfm?doid=3027385.3027400
27–30 April, 2020, Porto, Portugal
 [18] J. Lester, C. Klein, H. Rangwala, and A. Johri, “Learning
Analytics in Higher Education,” ASHE Higher Education Report,
vol. 43, no. 5, pp. 9–135, dec 2017. [Online]. Available:
http://doi.wiley.com/10.1002/aehe.20121
[19] S. J. Taylor, R. Bogdan, and M. DeVault, Introduction to qualitative
research methods: A guidebook and resource, 3rd ed. Hoboken, NJ,
US: John Wiley & Sons Inc, 2015.
[20] J. Corbin and A. Strauss, Basics of Qualitative Research: Techniques
and Procedures for Developing Grounded Theory, 4th ed. SAGE
Publications, 2015.
[21] J. Saldaña, The Coding Manual for Qualitative Researchers, 2nd ed.,
J. Seaman, Ed. London: SAGE Publications, 2013. [Online]. Available:
www.sagepublications.com
[22] E. Curry, “The Big Data Value Chain: Definitions, Concepts,
and Theoretical Approaches,” in New Horizons for a Data-
Driven Economy: A Roadmap for Usage and Exploitation of
Big Data in Europe, J. M. Cavanillas, E. Curry, and W. Wahlster,
Eds. Cham: Springer International Publishing, 2016, pp. 29–
37. [Online]. Available: https://doi.org/10.1007/978-3-319-21569-3 3
http://link.springer.com/10.1007/978-3-319-21569-3 3
[23] S. Pearson and A. Benameur, “Privacy, Security and Trust
Issues Arising from Cloud Computing,” in 2010 IEEE Second
International Conference on Cloud Computing Technology and
Science. IEEE, nov 2010, pp. 693–702. [Online]. Available:
http://ieeexplore.ieee.org/document/5708519/
[24] C. Gentry, “Fully homomorphic encryption using ideal lattices,” in
Proceedings of the 41st annual ACM symposium on Symposium
on theory of computing - STOC ’09. New York, New
York, USA: ACM Press, 2009, p. 169. [Online]. Available:
http://portal.acm.org/citation.cfm?doid=1536414.1536440
[25] J. Singh and J. Cobbe, “The Security Implications of Data Subject
Rights,” IEEE Security & Privacy, vol. 17, no. 6, pp. 21–30, nov 2019.
[Online]. Available: https://ieeexplore.ieee.org/document/8755390/
[26] L. Piras, M. G. Al-Obeidallah, A. Praitano, A. Tsohou, H. Mouratidis,
B. Gallego-Nicasio Crespo, J. B. Bernard, M. Fiorani, E. Magkos,
A. C. Sanz, M. Pavlidis, R. D’Addario, and G. G. Zorzino,
“DEFeND Architecture: A Privacy by Design Platform for GDPR
Compliance,” in Trust, Privacy and Security in Digital Business,
ser. Lecture Notes in Computer Science, S. Gritzalis, E. R. Weippl,
S. K. Katsikas, G. Anderst-Kotsis, A. M. Tjoa, and I. Khalil, Eds.
Cham: Springer International Publishing, 2019, vol. 11711, pp. 78–93.
[Online]. Available: http://link.springer.com/10.1007/978-3-030-27813-
7 http://link.springer.com/10.1007/978-3-030-27813-7 6
[27] Z. Li, P. Avgeriou, and P. Liang, “A systematic mapping study
on technical debt and its management,” Journal of Systems and
Software, vol. 101, pp. 193–220, mar 2015. [Online]. Available:
https://linkinghub.elsevier.com/retrieve/pii/S0164121214002854

978-1-7281-0930-5/20/$31.00 ©2020 IEEE 27–30 April, 2020, Porto, Portugal

2020 IEEE Global Engineering Education Conference (EDUCON)
Page 105

Authorized licensed use limited to: University of Vermont Libraries. Downloaded on July 26,2020 at 00:18:50 UTC from IEEE Xplore. Restrictions apply.