AI & SOCIETY

https://doi.org/10.1007/s00146-020-01051-6

OPEN FORUM

From computerised thing to digital being: mission (Im)possible?

Julija Kiršienė1 · Edita Gruodytė1 · Darius Amilevičius2

Received: 7 July 2019 / Accepted: 8 May 2020

© Springer-Verlag London Ltd., part of Springer Nature 2020

Abstract
Artificial intelligence (AI) is one of the main drivers of what has been described as the “Fourth Industrial Revolution”, as
well as the most innovative technology developed to date. It is a pervasive transformative innovation, which needs a new
approach. In 2017, the European Parliament introduced the notion of the “electronic person”, which sparked huge debates
in philosophical, legal, technological, and other academic settings. The issues related to AI should be examined from an
interdisciplinary perspective. In this paper, we examine this legal innovation—that has been proposed by the European
Parliament—from not only legal but also technological points of view. In the first section, we define AI and analyse its main
characteristics. We argue that, from a technical perspective, it appears premature and probably inappropriate to introduce AI
personhood now. In the second section, justifications for the European Parliament’s proposals are explored in contrast with
the opposing arguments that have been presented. As the existing mechanisms of liability could be insufficient in scenarios
where AI systems cause harm, especially when algorithms of AI learn and evolve on their own, there is a need to depart
from traditional liability theories.

Keywords  Artificial intelligence · Electronic personhood · Algorithms · Liability

1 Introduction world innovator in technology and computing. We should

Unlike more specialised innovations, artificial intelligence
(AI) is becoming a true general-purpose technology. AI is
evolving into a utility that is likely to infiltrate every industry
and sector of our economy, as well as nearly every aspect
of science, society, and culture. It permeates human spaces
silently and invisibly. As Arthur C. Clarke states in his Third
Law, though, any form of technology that is sufficiently
advanced is indistinguishable from magic (Clarke 1962).
On 25 October 2017, the announcement of the robot,
named Sophia, with Saudi Arabian citizenship, was a care-
ful piece of marketing to position Saudi Arabia as a major
* Darius Amilevičius
darius.amilevicius@vdu.lt
Julija Kiršienė
julija.kirsiene@vdu.lt
Edita Gruodytė
edita.gruodyte@vdu.lt
1 subject”. “While not biological intelligence systems may be granted
Faculty of Law, Vytautas Magnus University, K. Donelaičio
g. 58, 44248 Kaunas, Lithuania
2
Faculty of Informatics, Vytautas Magnus University,
K. Donelaičio g. 58, 44248 Kaunas, Lithuania
recognise that granting someone, or something, legal person-
hood is—and always has been—a highly sensitive political
issue. The recent case of Saudi Arabia enrolling Sophia as
a citizen is hence unsurprising.1 Indeed, this could turn into
the realm of discretionary political power that sometimes
devolves into mere sovereign arbitrariness (Atabekov and
Yastrebov 2018: 775–77). This reminds us of Suetonius’
“Lives of the Twelve Caesars” (121 AD), in which we find
Caligula planning to make his horse, Incitatus, a consul,
and “the horse would invite dignitaries to dine with him in a
house outfitted with servants there to entertain such events”
(Suetonius 1913, 55).
While Saudi Arabia is the first country to grant citizen-
ship to an AI-enabled android, it is not alone in pushing for
more rights for robots. In 2017, the European Parliament
proposed a set of regulations to govern the use and creation
1
  In this paper we do not discuss the issues of robot rights, but we
agree with the opinion of T. Jaynes that there is question to ask “how
not biological intelligence can gain citizenship in nations without a
monarchist system of government or being based within a human
citizenship, this citizenship does not provide many useful protections
for the not biological intelligence system—if any are granted at all,
which is a subject that has not even been adequately addressed by the
nation of Saudi Arabia” (Jaynes 2019, 8).

13
Vol.:(0123456789)

of robotic and artificial intelligence, including the granting
of “electronic personhood” to the most advanced machines
(European Parliament 2017). But not everyone agrees that
this is the best solution, and many experts fear that giving
robots the same kind personhood or citizenship as people
will infringe on human rights. In an open letter (https​://
www.roboti​ cs-openle​ tter.​ eu/), written to the European Com-
mission in 2018, 150 experts in medicine, robotics, AI, and
ethics criticised the plans as being “ideological, nonsensical
and non-pragmatic”. The letter outlined the belief that “from
an ethical and legal perspective, creating a legal personhood
for a robot is inappropriate”. The letter also demanded that
the EU ensure a legal framework weighted towards “the
protection of robots’ users and third parties”, rather than
the robots themselves.2 Despite the matter of Sophia’s citi-
zenship being intended as a publicity stunt, the questions
that were consequently raised have opened up a huge debate
(Pagallo 2018a; Singh 2017; Čerka et al. 2017; Aleksandre
2017) that is a long way from being resolved. This is espe-
cially notable in relation to granting legal personhood for
AI devices.
According to the EU Parliament’s proposition, electronic
personhood is meant only for the most sophisticated autono-
mous robots, in cases where robots make smart autonomous
decisions or otherwise interact independently. As such, a key
issue that warrants serious consideration in the context of the
legal status of intelligent AI-based machines is the definition
of the term “artificial intelligence”. In the “European Civil
Law Rules on Robotics”, the European Parliament appears to
consider AIs as being easily identifiable artefacts (European
Parliament 2017: sec.AF). In our opinion, however, such a
definition is a misleading one. Furthermore, it raises two
key questions. Firstly, what level of AI technology must we
achieve? Secondly, what qualifies as being “state-of-the-art”
technology at the present time? To answer these questions,
we must examine not only the working definition of artificial
intelligence but also the state-of-the-art of its main functions
like intellectual capacity and autonomy. In this study, we
interrogate the scope of the definition of “electronic person”
as it is presented in “Civil Law Rules on Robotics”, present-
ing and elaborating upon the main arguments of the docu-
ment. The main issue in this regard is that of ascertaining
whether the aim of the proposition is to grant an electronic
“person” rights and obligations or just to introduce one more
possible solution to the question of the legal accountability
of robots in torts and contracts.
2
 As was mentioned above, we do not discuss the question of the
robots rights, but we agree with the opinion of Dowel that actually
the non-biological intelligence does not clearly fit into any extent
category of persons and the path forward is uncertain, legislators are
beginning to explore potential issues (Dowell 2018). However, the
foundation is being put down now. E.g., see Gunkel (2018).
13
AI & SOCIETY
Analysis of the arguments in opposition to the idea of
electronic “person” shows asymmetry between purpose as
it described in “Civil Law Rules on Robotics” and criticism.
Which mainly concentrates on granting legal personhood to
robots as full subjects of the law, including rights like citi-
zenship, free speech, privacy, property ownership, inherit-
ance, and so on. Hence, we suggest that—at least regarding
the most sophisticated autonomous robots—the proposal
for electronic personhood has some grounds. The existing
liability instruments could be inadequate, ineffective, may
not fully reflect the digital economic needs, stifle innovation,
and might be unjust in terms of the apportionment of risk in
society. From legal and technical perspectives, research has
highlighted various issues that relate to the autonomy of AI,
such as the “black box problem”, foreseeability, traceabil-
ity, the establishment of liable or guilty persons, and others,
which cannot be addressed effectively by existing liability
instruments. We argue that, in the context of the Fourth
Industrial Revolution and rapid industrialisation, it would
be prudent to ensure that there is a robust legal instrument in
place for liability purposes when AI development reaches a
certain technological level. Thus, we suggest elaborating the
concept of “electronic persons” for the purposes of model-
ling a new liability instrument.
The definitions of AI—and the related issue of “electronic
personhood”—in scholarship are mostly considered from
one perspective: ethical, legal, technological, or another.
However, the issues related to AI should be examined from
an interdisciplinary perspective. In this paper, we examine
AI and the legal innovation of electronic personhood—as
proposed by the European Parliament—from not only legal
but also technological points of view.
2 How has the concept of the “electronic
person” been justified in the “Civil Law
Rules on Robotics” from a technical point
of view?
The term, “electronic person”, was first used in 1967, in
an article that introduced a robot named Shaky (Alexandre
2017: 22). In 2015, the Committee on Legal Affairs (JURI)
in the European Parliament established a working group for
legal questions relating to the development of robotics and
artificial intelligence in the European Union. The resulting
European Parliament resolution, “Civil Law Rules on Robot-
ics”, was released on 27 January 2017 (European Parliament
2017). It proposed a set of regulations for governing the use
and creation of artificial intelligence and suggested granting
“electronic personhood” to the most advanced machines to
ensure their accountability In this resolution, special legal
status for AI is stipulated as being one possible legal solution
to the problem of establishing the liability of robots, as cited:
AI & SOCIETY
creating a specific legal status for robots in the long
run, so that at least the most sophisticated autono-
mous robots could be established as having the status
of electronic persons responsible for making good any
damage they may cause, and possibly applying elec-
tronic personality to cases where robots make autono-
mous decisions or otherwise interact with third parties
independently (European Parliament 2017: 59 (f)).
Hence, in the EU Parliament’s proposition—electronic
personhood is meant only for the most sophisticated autono-
mous robots, in cases where robots make smart autonomous
decisions or otherwise interact independently. However, this
document does not provide any definition for the particular
criteria, features, or standards that are required for defining
the smartness, autonomy, or independence levels of such
robots.
In “Civil Rules on Robotics”, the European Parliament
appears to consider AI as being easily identifiable artefacts
(European Parliament 2017: sec.AF). We argue, though, that
to do so is misleading. In this section, we question what
qualifies as being the “most sophisticated autonomous”
machines, justifying “electronic person” in a technical con-
text. We will proceed in two steps. First, we will seek to
establish a working definition of AI. Secondly, we will look
present the common technical features of forms of AI, like
intelligence, autonomy, and embodiment.
After “Civil Rules on Robotics” were released, the
authors of EU documents regarding AI used erroneous or
convoluted terminology when providing definitions for
and the key characteristics of AI.3 Unfortunately, the defi-
nitions of AI (and “smart robot”) that are provided in the
nine-page document that was produced by the High-Level
Expert Group on AI are complicated and unclear (High-
Level Expert Group on Artificial Intelligence 2018). Conse-
quently, we agree with the findings of a study panel that was
organized as part of Stanford University’s “One Hundred
Year Study on Artificial Intelligence”: “The Study Panel’s
consensus is that attempts to regulate ‘AI’ in general would
be misguided, since there is no clear definition of AI, and
the risks and considerations are very different in different
domains” (Stone et al. 2016).
In 1955, John McCarthy coined the term “artificial intel-
ligence”, which he defined as “the science and engineering
of making intelligent machines” (McCarthy 2007). Most
definitions follow this lead by describing AI as “intelligence
exhibited by machines”. Common variants add that AI must
demonstrate “human” or “human-like” intelligence. Such
3
 (European Commission 2018a, b; European Commission’s High-
Level Expert Group on Artificial Intelligence 2018, 2019).
definitions assume that intelligence is clearly defined itself;
this too, though, is ambiguous.
Several features of AI make it exceptionally difficult to
define and to regulate when compared with other human-
made objects of the material world. Artificial intelligence
(or cognitive computing) is the theory and development of
computer systems that can perform tasks that were previ-
ously thought to require human intelligence. The term “AI”
is exceptionally wide in scope; it is a moving target. Techni-
cal AI definitions change as related technological advance-
ments develop further.4 AI is not a single technology. Rather,
it is many different technologies that are applied for differ-
ent functions through various applications. AI is helping
computers to achieve extraordinary feats that truly catapult
us into the future. It is only deemed to be “artificial” intelli-
gence before its functioning is really understood; after that, it
is merely “software”.5 The concept of AI is itself notoriously
elusive, with different groups strongly disagreeing over pre-
cisely what it entails (Legg and Hutter 2007). This constant
shift in the definition and scope of AI is known as the “AI
effect”, and it is best described by Tesler’s Theorem: “AI
is whatever hasn’t been done yet” (Hofstadter 1999: 601).6
We can further distinguish between “narrow AI” and
“general AI”. “Narrow AI” is already widely used. “Gen-
eral AI”, by contrast, requires intelligent behaviour that is (at
least) as broad, adaptive, and advanced as a human’s across a
full range of cognitive tasks (Goertzel and Pennachin 2007).
Notably, it is also debatable if—and when—general AI will
be achieved.
4
  For example, the European High-Level Expert Group has launched
a nine-page document containing only the definition of AI (European
Commission’s High-Level Expert Group on Artificial Intelligence
2018).
5
  It must be noted that the term “software” is not defined in EU law;
Directive 2009/24/EC provides a definition of “computer program” in
recitals. The two definitions are often used interchangeably, but there
is a technical difference: a “program” is a set of instructions that tell
a computer what to do while “software” can be made up of more than
one program.
6
 When our article was ready for publishing, EC Joint Research
Center in February 2020 has released technical Repot “AI Watch.
Defining Artificial Intelligence”, where JRC also tend to define the
object in consideration After have considered 55 documents that
address the AI domain from different perspectives, the authors of this
Report not surprisingly take the definition of AI proposed by HLEG
on AI as the starting point for further development, but also assert
that‚ considering that the HLEG definition is comprehensive, hence
highly technical and detailed, […] the definitions provided by the
EC JRC Flagship report on AI (2018) […] are suitable alternatives
(Samoili et  al. 2020: 9). But AI Watch also assert, that despite the
increased interest in AI by the academia, industry and public insti-
tutions, there is no standard definition of what AI actually involves
… human intelligence is also difficult to define and measure … as
a consequence, most definitions found in research, policy or market
reports are vague and propose an ideal target rather than a measurable
research concept” (Samoili et al. 2020: 7).
13

We agree with the partial definition of AI, as proposed by
Zevenbergen et al. that it consists of “human-created digital
information technologies and associated hardware that dis-
plays intelligent behaviour that comes not purely from the
programmer, but also through some other means” (Zeven-
bergen et al. 2018). Indeed, this definition assumes that we
are in relation to not only pure software or computerised
things but with some kind of intelligence.
In academic and political circles, there is currently no
consensus on the definition of the term “AI”. However, from
a technical point of view, the term “AI” is used to describe
a variety of technologies that have certain features in com-
mon. AI is not a discrete technology; rather, it is a series of
technologies, concepts, and approaches, all of which align
in our quest to create intelligent machines.
Indeed, in accordance with the EU Parliament’s proposed
use of the term “electronic personhood”, we can trace certain
technical features that are common to artefacts in considera-
tion: the candidates for a new kind of legal title. Those com-
mon features are intelligence, autonomy, and embodiment.
When attempting to ascertain which features ought to
be covered by the term “intelligence”, it should be noted
that “intelligence” in itself is hard to define; discussions
about doing so often cause controversy. It is debatable as to
whether consciousness and rationality are prerequisites for
intelligence or vice versa. The MIT Encyclopedia of Cog-
nitive Science states: “An intelligent agent is a device that
interacts with its environment in flexible, goal-directed ways,
recognizing important states of the environment and acting
to achieve desired results” (Rosenschein 1999). In practice,
intelligence is a relative attribute and is evaluated in con-
nection with human capabilities. For example, we would not
normally consider a rat to be an intelligent creature (imply-
ing a comparison to a human), but we would recognise it
to be more intelligent than a cockroach. We share Florian’s
view that we would consider an agent as being intelligent if
it can perform non-trivial, purposeful behaviours that adapt
to changes in the environment (Florian 2003). However, the
evaluation of behaviour is arbitrarily undertaken by humans;
intelligence is, thus, a property that is subjectively assigned.
We have no general mathematical theory of intelligence, as
all AI technologies are digital.7 Without a general theory
of “natural” intelligence to understand its scope—that of
7 which is closely related to it and is expressed in relatively unambigu-
 Max Tegmark proposed the hypothesis that our physical reality
is a mathematical structure (Tegmark 2014). Steven Pinker argued
that intelligence does not come from a special kind of spirit or mat-
ter or energy, but from a different commodity, namely information.
Information is a correlation between two things that are produced by
a lawful process. Correlation is a mathematical and logical concept
(Pinker 1998). However, Cathy O’Neil demonstrates the opposite,
namely that many algorithms are not inherently fair just because they
have a mathematical basis (O’Neil 2016).
13
AI & SOCIETY
general AI—it is a bit like trying to conduct particle physics
experiments without having a standard model. The abstrac-
tion of biological neurons and insights based on fragments
of biological network topology do not constitute a general
theory of intelligence. However, we agree with Borden’s
opinion that, in actual situations, Turing’s teasing discus-
sion of the imitation game8 was aimed at avoiding endless
discussions about definitions of intelligence in the context
of AI (Borden 2006).
Presently, forms of AI are certainly not conscious or
rational; it is often assumed that they possess these char-
acteristics when people are faced with advanced or over-
hyped technologies. The biggest problem that we face in
the context of present “machine learning” (ML) and “deep
learning” (DL) machines is that, although they are capa-
ble of learning things, they cannot understand them. The
most advanced AI systems are merely products that follow
processes that have been devised and defined by intelligent
people. AI techniques (natural language processing and ML/
DL) enable private and public decision-makers to analyse
big datasets to build profiles, which are then used to make
decisions in an automated way. Yet, for now, machines can-
not make decisions on their own (Atkinson 2016). Atkinson
states that there is a fundamental difference between infor-
mation processing and thinking (Atkinson 2016). Domingos
is in agreement, noting that “unfortunately, what we have so
far is only a very crude cartoon of how nature learns, good
enough for many applications but still a pale shadow of the
real thing” (Domingos 2018: 140).
The process of trying to define the interwire terms
“autonomy” and “agency” has been no less challenging.
The term “autonomous artificial intelligent agent” would be
closer to the notion of “personhood” instead of “electronic
agent”9 or “software agent”.10 We do not support the use
of the term “software agent” (closely related to “electronic
agent”) as it does not cover embodied agents, such as robots,
or hardware implementations, such as neural network chips.
8
  In 1950 A. Turing in his paper “Computing Machinery and Intel-
ligence” has introduced his famous test, called Turing test, but the
author by itself does not call his idea "Turing test", but rather the
"Imitation Game". Turing opens with the words: "I propose to con-
sider the question, ’Can machines think?’", because “thinking” is
difficult to define and he chooses to replace the question by another,
ous words (Turing 1950).
9
 E.g., for W. Barfield the term “electronic person” is sufficient to
describe virtual avatar or virtual agent, acting only in cyber space
(Barfield 2006), but it is not sufficient and does not explore all the
extend of applications of AI-based systems.
10
 An artificial agent may be instantiated by an optical, chemical,
quantum or, indeed, biological—rather than an electronic—comput-
ing process.
AI & SOCIETY
Autonomous intelligent agent research is a domain that
is situated at the forefront of AI. It has been argued by many
authors (e.g., Clark 2017; Menary 2007) that genuine intel-
ligence can emerge only in embodied, situated, cognitive
agents. Agency is closely connected to qualities like auton-
omy and embodiment. Most authors refrain from giving
precise definitions of “agent”. In their canonical book on
AI, Russell and Norvig assert the following: “The notion of
an agent is meant to be a tool for analysing systems, not an
absolute characterization that divides the world into agents
and non-agents” (Stuart and Norvig 2010). We generally
consider humans and most other animals to be agents; we
consider robots, autonomous systems, and software pro-
grams, meanwhile, to be artificial agents. But what really
distinguishes an agent from other artificial systems? We
agree with main opinion among scholars asserts that agents
can be constructed to mimic or even surpass the cognitive
functions of humans.
The most obvious feature of AI that separates it from ear-
lier technologies is the ability to act autonomously. Auton-
omy is a characteristic that enhances the viability of an agent
in a dynamic environment.11 Autonomy requires automatic-
ity and even goes beyond that, implying some degree of
adaptability. Automaticity means that the agent possesses
at least some mechanisms that allow the agent to sense the
environment in which it is situated and to act upon such
information accordingly. Automaticity does not require the
intervention of other agents to be executed. However, auton-
omy is a matter of degrees, not a clear-cut quality (Smithers
1995). Most animals and some robots can be autonomous
agents. Pattie Maes defines artificial autonomous agents
as being “computational systems that inhabit some com-
plex dynamic environment, sense and act autonomously in
this environment, and by doing so realize a set of goals or
tasks for which they are designed” (Maes 1995: 108–114).
In a study in which they seek to draw distinctions between
autonomous agents and other systems, Franklin and Graesser
provide the following definition: “An autonomous agent is a
system situated within a part of an environment that senses
that environment and acts on it, over time, in pursuit of its
own agenda and so as to effect what it senses in the future”
(Franklin and Graesser 1996).
Thus, for the reasons that we have described above, “arti-
ficial agent” is a more suitable term to use than “artificial
intelligence” or “intelligent agent”, as we wish to empha-
sise the embedded, social, real-world nature of the typical
11
  Autonomy is one of the main features of AI-based systems from a
technological point of view, but it presents serious ontological issues
(Chinen 2019). It must be noted that autonomy is presented as one of
main characteristics of AI, IoT and robotics technologies in the EC
Report on the safety and liability of AI, IoT and robotics (European
Commission 2020).
artificial agent (equal to “autonomous system” and “robots”
in the EU resolution) rather than merely disembodied intel-
ligence. We must also clarify whether only “autonomous
intelligent technologies” that are based in physical forms
in an unrestricted physical world are within the scope of
new kind of personhood or whether all algorithms should
be taken into consideration.
Embodiment is another important quality of many auton-
omous agents.12 It refers to the physical body that interacts
with the surrounding environment. This property is impor-
tant for the expression of cognitive capabilities. Embodiment
is closely connected to situatedness: a body is not sufficient
for embodiment if it is not also situated in an environment.
The body must be adapted to the environment to engage
in interactions. According to this interpretation, a robot or
a machine that does not perceive its environment, acting
according to a predefined plan or remote control, is not con-
sidered to be embodied or situated. Not all agents are the
same, and We propose that there are at least two types of
artificial agents: (1) inhabitants—these are agents that have
been paired with real-world hardware that uses geographic
location, and includes artificial agents in cars, drones, sen-
sors, cameras, mobile phones, or computers; (2) pure soft-
ware—these are artificial agents that exist in the digital
space only (geographical position has no meaning).
In summarising all of the above, we can confirm that the
current generation of AI is a long way away from the intel-
ligent machines of science fiction, despite demonstrating or
simulating some kind of intelligence. Modern machine intel-
ligence is a combination of intelligence imitation and self-
learning imitation. AI machines are completely lacking with
regard to common sense and critical thinking, which are
indispensable for evaluating data and information on one’s
own. The current wave of advances in artificial intelligence
does not actually bring us real intelligence but is, instead,
a critical component of intelligence, specifically, predic-
tion. Certainly, we cannot prevent political initiatives from
turning AI-based systems into persons or citizens. In this
regard, we agree with the views of the European Commis-
sion that, while the adoption of more technical approaches
to the definition of AI is possible, those approaches would be
less suitable in view of the fast pace of technological devel-
opment. The definition of AI must be sufficiently flexible to
12
 In this paper we do not question morality issues, but the oppo-
site opinion is expressed by E. Schwitzgebel and M. Garza. They
use notion psycho-social view of moral status and they suggest that
it shouldn’t matter to one’s moral status what kind of body one has,
except insofar as one’s body influences one’s psychological and social
properties. Similarly, it shouldn’t matter to one’s moral status what
kind of underlying architecture one has, except insofar as underly-
ing architecture influences one’s psychological and social proper-
ties. Only psychological and social properties are directly relevant to
moral status (Schwitzgebel and Garza 2015).
13

accommodate technical progress while providing the neces-
sary legal certainty (European Commission 2019). However,
while there may be future conditions to justify or even neces-
sitate AI personhood, doing so now appears to be techni-
cally premature and is likely to be inappropriate due to the
following: (1) currently there is no consensus regarding the
definition of the term “AI”; (2) common technical features of
AI like intelligence, autonomy, and embodiment cannot be
defined. There is no consensus concerning the definitions of
these main features and their status as far from sophisticated
autonomous agents; they cannot make intelligent autono-
mous decisions or (inter)act independently as stipulated in
the EU proposition. This means that, from a technical point
of view, it is unclear as to which specific entities should be
granted the proposed legal status of “electronic person.”
3 How is the concept of an “electronic
person” justified in the “Civil Law Rules
on Robotics” from a legal point of view?
The authors of the “Civil Law Rules on Robotics” (Euro-
pean Parliament 2017) invited the European Commission
to explore, analyse, and consider electronic personhood as
a possible legal solution for autonomous robots. This legal
solution, along with other alternatives, such as trainer’s
responsibility,13 mandatory insurance schemes, and sup-
plementary guaranty funds to compensate victims of acci-
dents (involving driverless cars), should be designed for
reducing the public risks that Al presents “without stifling
innovation”.14
Notably, this EU document is not binding; it presents
recommendations and, thus, the possibility of legal status
is expressed “cautious[ly] and noncommittal[ly]” (Bryson
et al. 2017: 276). On the other hand, this statement strongly
implies that the door is open for such legal innovation (Bry-
son et al. 2017: 276). Hence, its authors stress the need to
treat AI differently from previous technologies, including
“depart[ing] from traditional liability theories” (Kritikos
2019: 3). As in the case of autonomous robots, “they [rules]
would not make it possible to identify the party responsible
for providing compensation and for requiring that party to
13
  In accordance with the formula suggested in the resolution, liabil-
ity for damage caused by cognitive computing should be proportional
to the actual level of instructions given to the robot and its degree of
autonomy, so that the greater a robot’s learning capability or auton-
omy, and the longer a robot’s training, the greater responsibility of its
trainer should be.
14
  European AI strategy supports an ethical, secure, and cutting-edge
AI made in Europe, and based on three pillars: (1) increasing pub-
lic and private investments in AI; (2) preparing for socio-economic
changes; and (3) ensuring an appropriate ethical and legal framework
(European Commission 2018c: 1).
13
AI & SOCIETY
make good the damage it has caused” (European Parliament
2017: sec.AF). This problem demonstrates the necessity of
further exploration, including the legal regulation of elec-
tronic personhood.
Scholars argue that, from the wording of the European
Parliament resolution, it is unclear as to whether “the status
of electronic persons” (European Parliament 2017: sec.56)
refers to the legal personhood of robots as full subjects of
the law or only their accountability in torts and contracts
(Pagallo 2018b).15 However, as evidenced by the cited Euro-
pean Parliament resolution (European Parliament 2017), the
main reason behind this proposition is to ensure that per-
sons who are affected by AI can be fully compensated for
damages.
The debate over granting electronic personhood to the
most advanced artificially-intelligent machines is a long way
from resolution. Opponents of the European Parliament’s
resolution on “Civil Law Rules on Robotics” submitted the
so-called “Robotics Open Letter” (https​://www.robot​ics-
openl​etter​.eu/) to the European Commission in 2018. This
letter was written by 150 experts in medicine, robotics, AI,
and ethics, who criticised the plans for being “ideological,
nonsensical and non-pragmatic”. They argued that it was
unnecessary and misleading for the EC to claim that the
problem of proving liability for damages could be remedied
by introducing the concept of “electronic personhood”. More
generally, they feared that giving citizenship or personhood
to robots would infringe human rights. The letter outlined
the belief that “from an ethical and legal perspective, creat-
ing a legal personality for a robot is inappropriate”. The
letter also demanded that the EU ensured that they would
produce a legal framework that was weighted towards “the
protection of robots’ users and third parties” rather than the
robots themselves. Other scholars have echoed this posi-
tion. At this point, “smart autonomous robots with rights
and duties seems to be at least premature and would also
lead to a situation in which no certainty is given to damaged
parties as to who should and will, compensate the damage”
(Renda 2019: 87).
Another argument against the introduction of legal status
for AI is that “there would be inevitably situations where the
acts of robots would interfere with the rights of humans and
other legal persons” (Bryson et al. 2017: 285). The main
problem at this stage of development is that the right bal-
ance of rights and obligations cannot be reached because of
the difficulty of imposing obligations on AI (Bryson et al.
2017: 285). Consequently, “a legal system, if it will be cho-
sen to confer legal personhood on robots, would need to say
15
  This includes the possibility to bear fundamental legal rights and
duties, like citizenship, free speech, privacy, to sue and be sued, enter
contracts, own property, inherit, and so on.
AI & SOCIETY
specifically which legal rights and obligations went with this
designation” (Bryson et al. 2017: 285).
The prospective introduction of electronic personhood
has often been compared to the introduction of corporations
as legal persons, which led to an increase in economic power
in the hands of accumulated capital, growing image, and the
importance of corporations in society (Robson 2010: 113;
Gordon 2018).
The proponents of electronic personhood follow the so-
called pragmatic approach, where robots are not humans
and the term “electronic personality” is introduced for legal
convenience: “it is possible to create a legal status, which
would only be a ‘tangible symbol’ for the cooperation of
all the people creating and using that specific robot” (Beck
2016: 141). The underlying rationale is the legal system’s
recognition that such legal status is necessary “when doing
so suits its ends (Bryson et al. 2017: 277). For example,
autonomous vehicles have been given the status of legal enti-
ties in Nevada in the United States (Selwood 2017: 869).
Other scholars also argue that future technologies will be so
sophisticated and developed that AI will bear certain simi-
larities to human beings.
Analysis of arguments against electronic person shows
asymmetry between the purpose as it is described in Civil
Law Rules on Robotics to consider electronic person as a
possible liability instrument for autonomous artificial intel-
ligent agents, along with other alternatives, such as trainer’s
responsibility, mandatory insurance schemes, etc. However,
criticism of the concept of electronic personhood mainly
focuses on the issues of granting legal personhood to robots
as full subjects of the law.
3.1 Artificially intelligent agents and challenges
for product liability legal framework
As the main purpose of the proposition is to introduce one
more possible solution to the problem of establishing the
legal accountability of artificially intelligent agents in torts
and contracts, in this section, we examine the existing liabil-
ity framework and how it reflects digital economic needs.
The liability system in the EU functions in accordance
with basic principles, such as safety, security, transparency,
protection of privacy and data, non-discrimination, fairness,
and accountability (Independent High-Level Expert Group
on Artificial Intelligence set up by the European Union
2019). Following the principle of precaution, provided by
the PETL (Principles of European Tort Law), art. 2:102 (2),
the more serious the risk of threat created by the product
and the more likely the damage, the greater the level of pre-
caution that must be taken. The factors that are involved in
assessing the required standard of care under the PETL are
the value of endangered interest, the magnitude of risk, the
likelihood and gravity of injury, the cost of prevention, the
search for alternative means of reducing the danger and the
application of proportionality tests when weighing up all
interests at stake. In other words, a high level of expertise is
expected from those who specialize in dangerous areas. In
this regard, foreseeability and proportionality tests are often
used (Winiger et al. 2018: 400–402). Also, since its adoption
in 1985, the Product Liability Directive has been consid-
ered to be one of the cornerstones of European Private Law,
seeking to achieve maximum harmonization of consumer
protection in the field (Council Directive 1985; European
Parliament and Council Directive 1999). The Directive also
ascertained that product liability must be seen in the context
of new technologies and risk apportionment. The goal of the
Product Liability Directive is to protect individuals and soci-
ety from the risks that arise from defective products so that
everybody can live in a safe environment (BEUC 2017: 2).
Notably, the member states of the European Union are not
allowed to provide a higher level of protection to consumers
than that which has been set by the Directive.
The concept of strict liability (liability without fault)
of producers has traditionally been being a very important
principle that protects both the interests of consumers and
society. The rationale behind this concept is that producers
who make profits from dangerous activities should be held
accountable if this danger materializes (BEUC 2017: 2).
Several arguments speak in favour of the solution to attribute
liability to producers or whoever puts a product on the mar-
ket. The first argument is that of spreading costs. Instead of
putting the costs of the accident and the whole burden on the
shoulders of a single person (victim of the accident), there
is the option of spreading the costs of unavoidable accidents
among the whole community of persons who benefit from
the activity because risks and such expenses are intrinsic to
the production and the price of the product. Producers also
have the option of self-insuring against liability. Self-insur-
ance and cost-spreading thus alleviate the expenses that are
associated with accidents. The second argument relates to
injury prevention. By having to bear the accident costs with
which an activity is associated, the producer has the incen-
tive to take all the possible precautions against a recurrence
of the accident. So, producers are responsible for minimising
the potential risks to consumers that are associated with a
product that has been brought to the market. Legal regulation
should encourage manufacturers to become pro-active about
protecting customers from damage and improving the ex-
ante supervision of their own quality management systems
(Winiger et al. 2018: 400–402; Leeuwen and Verbruggen
2015: 912).
The concept of strict liability is closely related to the
standard of care issues. Normally, the professional stand-
ard of care is higher when a certain activity is dangerous.
Danger is often measured by the probability of occurrence
and the gravity of harm—which, in other words, means that
13

the greater the danger, the higher the necessary degree of
care. For example, one of the first technologies to put legal
regulation to the test was that of autonomous vehicles. In
many ways, autonomous vehicles are better at performing
precarious activities that are currently performed by humans
(such as driving) because machines never fall asleep, they
never drive drunk, they never get distracted by a text mes-
sage or conversation, they never drink coffee or eat, and they
do not get angry or drowsy. Bearing in mind these human
deficiencies, it would be no surprise if the mass introduction
of autonomous vehicles considerably lessened the rate of
accidents (Vladeck 2014: 126). In a world where driverless
cars or drones will have the capacity to “sense-think-act”
at their own wills and with their own plans (Vladeck 2014:
122), the legal system will have to address the standard of
care issues (Solum 2017: 241–42) of such autonomously-
thinking machines. Adhering to the current rationale, should
we—in terms of product liability rules—apply a lower
standard of care to autonomously-thinking machines just
because we expect them to be so technologically advanced
that they should fail much less than humans? (Vladeck 2014:
131–41).
Following the European consumer organization report:
safety standards and safety expectations must be tan-
dem criteria that encourage producers to place only
safe products on the market. Therefore, there must be a
smooth interplay between the Product Liability Direc-
tive and the Product Safety Directive, considering
both curative and preventive considerations (BEUC
2017:4).
An important role is played by public supervisory agen-
cies, who are responsible for the overall supervision of the
market and who can take products off the market if they do
not comply with the European legislation.16
However, the current rules on liability for defective prod-
ucts in the context of new technologies have turned out to
be ineffective in achieving their objectives. For example,
without any obvious reason, software is not covered by the
Directive as being a product, which is legally defined as
covering “movables” (Article 2). Another problem lies with
the definition of “damage” (Art. 9), which currently focuses
on the destruction of another item of property, excluding
damage to the digital environment. Also, consumers’ organi-
zations pose another problem: how do we identify the liable
16
 For example, pacemakers are active implants and as such come
within the scope of the Active Medical Devices Directive. To be able
to place such devices on the European market, manufacturers must go
through a conformity assessment procedure. This procedure is under-
taken by a certification body. After the products have been placed on
the market, notified bodies continue their supervising role and have to
carry out regular inspections (Leeuwen and Verbruggen 2015: 912–
913).
13
AI & SOCIETY
person when the same product is made by several producers
and contributors? There should be joint liability of profes-
sionals in the product supply chain: “Since the consumer
has the onus of [the] burden of proof, the victim will have
otherwise no possibility of recourse under the current Direc-
tive” (BEUC 2017: 4). Maybe existing liability instruments
are inadequate, ineffective, not reflecting the digital, partici-
patory economic needs, stifle innovation, and are unjust in
terms of risk apportionment in society?
Researchers argue that the current product liability legal
framework is likely to become inadequate as commercially
available AI machines become more sophisticated and
autonomous (Asaro 2012: 242). As aforementioned, where
human involvement in the decision-making processes of AI
is obvious, there is no need to re-examine legal regulations
(Vladeck 2014: 120). The companies that currently manu-
facture devices with AI are already subject to a well-devel-
oped doctrine of product liability (Asaro 2012: 170–76). So,
all harms that could potentially be caused by AI technolo-
gies are treated in the same way as any other technological
products (e.g., toys, cars, or weapons). As a matter of fact,
most accidents occur due to inevitable errors in the design,
programming, and production of such machines. Because
of this, possible failures are usually categorised into design
defects, manufacturing defects, information defects, and
failures to instruct (Vladeck 2014: 127–41; European Par-
liament 2017: 242).
Danger is usually the source of strict liability regula-
tory regimes in many legal systems—or, at least, in the
case of more-strict liability that is, for example, based
on the reversal of the burden of proof that is normally
set (Winiger et  al. 2018: 403–5). In this context, one
more issue that has been discussed by scholars is: why
should we apply a strict liability regime in AI cases if
developed AI devices are likely to be less dangerous than
the products that they replace? Strict liability rules are
an exemption to the fault-based liability regime. They are
based on risk theory, i.e., the notion that the activity or
device being used is dangerous. Some scholars argue that
applying this theory to AI devices might be problematic
because they are likely to be far less hazardous or risky
than the products that they replace (Vladeck 2014: 146).
However, others argue that there are strong policy reasons
for establishing an insurance-based strict liability regime
for cases that relate to intelligent devices.17 One reason is
the aforementioned traceability and opacity problem. The
injured person should not bear the loss when causal fail-
ure is hardly explicable (European Parliament 2017: 242).
Furthermore, as it has been shown that as the “complexity
17
 Some legal liabilities cannot be met by insurance (Solum 2017:
1245).
AI & SOCIETY
of such products rises geometrically, the cost of litigating
products liability cases would increase exponentially”,
strict liability regimes would then spare enormous trans-
action costs—that is if parties decided to litigate. Accord-
ing to insurance-based theory, the manufacturers are in
a better economic position to absorb the costs of loss
through pricing decisions, spreading the burden of loss.
These arguments conform to the notions of compensatory
justice and the apportionment of risk in society (Vladeck
2014: 146–48). Also, a predictable liability regime would
be likely to spur on innovation to a much greater extent
than an “uncertain fault-based liability system” (Vladeck
2014: 147). Eventually, in spite of how sophisticated and
autonomous in terms of decision making an AI machine
can be, even if it is capable of displaying an independent
initiative and formulating plans, some scholars suggest
that it is still an instrument of other entities and has no
attributed legal personhood (European Parliament 2017:
242; Vladeck 2014: 121).
For example, in one of the recent judgements of the
Court of Justice of the European Union in the case of Bos-
ton Scientific Medizintechnik, they extended the interpre-
tation of concepts of defect and damage in the Product
Liability Directive. In the case of implantable medical
devices, the court stated that a product is defective if it
does not provide the level of safety that a person is entitled
to expect, even if the risk was not caused by the use of the
product but was found in the “abnormal potential” of the
products to cause damage. They specifically referred to
the sixth recital of the Product Liability Directive, which
states that the assessment of whether a product is defec-
tive “must be carried out having regard to the reasonable
expectations of the public at large”. The Court then con-
cluded that products that have a potential defect can be
classified as being defective—there is no need to prove
that a specific product has a defect (Leeuwen and Ver-
bruggen 2015: 899–915). Still, it is necessary to prove that
the products had “a significantly higher than normal risk of
failure”. However, it is unclear to what extent the judgment
is relevant for other consumer goods, like “potentially”
defective AI technologies. Could it be that AI products
have an abnormal potential for damage to users that is
comparable to that of the pacemakers that were manufac-
tured by Boston Scientific Medizintechnik?
AI technology uses cognitive algorithms, which are not
merely programmed to perform specific tasks, but also to
learn and further develop themselves in interaction with their
environments. Given the complexity of the design, construc-
tion, and programming of AI devices, a central legal issue is
to track the reasons behind all past actions (and omissions)
(Vladeck 2014: 141–43; World Commission on the Ethics
of Scientific Knowledge and Technology 2017: 6–7). This
is ethically and legally crucial because “a robot’s decision
paths must be re-constructible for the purposes of litigation
and dispute resolution” (Riek and Howard 2014: 6).
Hence, there is an obvious tension between the “trace-
ability” requirement (considering the problem of epistemic
opacity) and the development of robots that have a high
level of autonomy in decision-making and advanced learn-
ing capabilities. For example, if “the software was developed
using open source code (with a vague number of developers
behind it), it would be quite difficult to identify the devel-
oper. The situation will become more complicated if the AI
is self-aware, capable of self-improvement, self-preserva-
tion, creativity, strives to obtain necessary resources, etc.”
(Radutniy 2017: 242). Who will be accountable, then? Is the
person who built the AI responsible, or does accountability
lie with the person who provided the data, the one who vali-
dated the data, or the company operating it, etc.?
So, liability issues will arise due to these traceability
and opacity problems. Court rulings to develop legal prec-
edents take place on a case-by-case basis, which is a long
and expensive process. At the same time, injured persons
should not bear losses when causal failure is hardly explica-
ble (European Parliament 2017). Furthermore, as “complex-
ity of such products rises geometrically, the cost of litigat-
ing products’ liability cases would increase exponentially”
(Vladeck 2014: 147).
Also, the lines between the responsibilities of manufac-
turers/developers and the responsibilities of users are blur-
ring (Asaro 2012: 174; European Parliament 2017: 242).
For example, if a manufacturer offered different versions of
AI algorithms, and a buyer knowingly chose one of them,
is the buyer to blame for the harmful consequences of the
algorithm’s decisions? In this regard, it would be difficult to
hold the manufacturer responsible for any creative—even if
dangerous—uses of their products. Cars and weapons are
also very dangerous consumer products, but users still tend
to be liable for most of the harms that they have caused
because the use of those potentially dangerous products
places an additional burden of responsibility on the user
(Asaro 2012: 170–74). Hence, the increasing autonomy of
smart devices poses an additional question: who exactly
should bear ethical and legal responsibility for the behav-
iour of autonomous (with learning abilities) machines? Cur-
rently, there typically seems to be a “shared” or “distributed”
responsibility between robot designers, engineers, program-
mers, manufacturers, investors, sellers, and users, because
none of these agents can be identified as the ultimate source
of action (World Commission on the Ethics of Scientific
Knowledge and Technology 2017: 42). Scholars argue that
this solution tends to dilute the notion of responsibility alto-
gether: if everybody has a part in the total responsibility, no
one is fully responsible (World Commission on the Ethics
of Scientific Knowledge and Technology 2017: 4). On the
other hand, most European jurisdictions already have rules
13

that determine that any damage that has been caused by the
concurrent actions of several tortfeasors generally results in
collective liability through the concept of “joint and several”
liability or solidary liability. This concept prevents actors
who have contributed to the damage from avoiding liability
and complex accidents from fractioning, in this sense, offer-
ing better protection for the victim (Winiger et al. 2018).
However, it is hardly the most efficient model in terms of
spurring innovation and the most righteous risk apportion-
ment in society.
So, granting subjectivity to robots would lead to “chang-
ing perceptions of appropriate allocation of risks…including
change of attitudes” (Robson 2010: 118–119). “Conferring
‘personhood’ on these machines would resolve the agency
question; the machines would become principals in their
own right, and along with new legal status would come
new legal burdens, including the burden of self-insurance”
(Vladeck 2014: 150).
Analysis from legal and technical points of view fore-
grounds the issues that are related to the autonomy of AI
like the “black box problem”, foreseeability, traceability, the
establishment of liable or guilty persons, and others—ones
which have not been addressed adequately by current prod-
uct liability legal frameworks.
3.2 How can the concept of the “electronic person”
work for modelling new liability instruments?
In recent EU documents (European Commission 2019,
2020) there has been a tendency to depart from the idea of
electronic personality. However, it should be noted that the
authors of these documents are not fully consistent. Despite
of long list of threats and gaps18 in the reports, their con-
clusion is that current safety and liability regulation seems
adequate (European Commission 2019: 36, 2020: 11). We
agree that for now, taking into consideration the state of art
of AI, which we analysed before, this is probably true. But
18
 For example, Union product safety legislation does not address
the human oversight in the context of self-learning AI, also the risks
to safety derived from faulty data (p. 8), the increasing risks derived
from the opacity of self-learning systems, fragmentation of liability
regimes and the significant differences between the tort laws of all
Member States, the outcome of cases will often be different depend-
ing on which jurisdiction applies; damage caused by self-learning
algorithms on financial markets, often remain uncompensated,
because some legal systems do not provide tort law protection of such
interests at all (European Commission 2020: 19); Emerging digital
technologies make it difficult to apply fault-based liability rules, due
to the lack of well-established models of proper functioning of these
technologies and the possibility of their developing as a result of
learning without direct human control (European Commission 2020:
23).
13
AI & SOCIETY
will the adequacy and completeness of liability regimes still
be unquestionable in 10 years or even less?
As the Report on the Safety and Liability Implications of
Artificial Intelligence, The Internet of Things and Robotics
points out, these emerging technologies share such charac-
teristics as complexity,19 opacity20 and openness.21 If the
goal of European regulation is trustworthy, human-centric
AI, liability issues of emerging digital technologies should
first be considered from the potential victim’s position, tak-
ing into consideration the changing technological, economic
and legal ecosystems. Due to the complexity of products,
different connected devices, software and services, long
supply chains, later updates and upgrades, huge amounts of
self-learning data for AI, contributory negligence of injured
persons, etc., it will be very difficult for victims to identify
the liable person.22 Furthermore, because the of so-called
black-box effect, understanding autonomous AI decisions
and the data behind them will require the cooperation of the
potentially liable party (which will not be easy to identify in
the long technological chain) and technical expertise that for
victim could be prohibitively costly.23 Reports also empha-
size that “allocation of the cost when damage occurs may
be unfair or inefficient under the current rules” (European
Commission 2020: 17). Taking into consideration the differ-
ences in legal liability frameworks between Member States,
although the market for technologies is increasingly global,
“the cost of proving all necessary conditions as required
19
  Complexity is reflected in plurality of economic operators involved
in the supply chain, multiplicity of components, software and services
as well as interconnectivity with other devices (European Commis-
sion 2020: 2).
20
  For ex-post mechanism of enforcement, it is decisive that humans
can be able to understand how the algorithmic decisions of the arti-
ficially intelligent agent have been reached. The self-learning artifi-
cially intelligent agents will be able to take decisions that may devi-
ate from what was initially intended by the producers. The producer’s
control may be limited if the product’s operation requires software or
data provided by third parties or collected from the environment, and
depends on self-learning processes and personalizing settings cho-
sen by the user. This dilutes the traditional role of a producer, when
a multitude of actors contribute to the design, functioning and use of
the AI product/system (European Commission 2020: 28).
21
  AI products are open to updates, upgrades, self-learning data after
placement on the market.
22
 The more complex the interplay of various factors that either
jointly or separately contributed to the damage, the more crucial links
in the chain of events are within the defendant’s control, the more
difficult it will be for the victim to succeed in establishing causation
(European Commission 2020: 20).
23
  The more complex the circumstances leading to the victim’s harm
are, the harder it is to identify relevant evidence. It can be difficult
and costly to identify a bug in a long and complicated software code.
Examining the process leading to a specific result (how the input data
led to the output data) may be difficult, very time-consuming and
expensive (European Commission 2019: 24).
AI & SOCIETY
under national laws may be unduly burdensome, economi-
cally prohibitive and discourage victims from claiming com-
pensation” (European Commission 2020: 14). Furthermore,
experts point out that “for most technological ecosystems,
however, no specific liability regimes exist.… The more
complex these ecosystems become with emerging digital
technologies, the more … difficult it becomes to apply liabil-
ity frameworks” (European Commission 2019: 17).
It is obvious that without clear and predictable legal
regulation, the goal of making Europe a world leader in AI
(European Commission 2020: 1) with “the promise of mak-
ing the world a safer, fairer, more productive, more conveni-
ent place” (European Commission 2019: 11) is just a nice
declaration. Moreover, national non-harmonised regimes24
can lead to further fragmentation of legal liability frame-
works, increasing the costs of introducing innovative AI
technologies and the insurance costs for producers. Thus,
the EU probably departed from the idea of electronic person-
hood as a possible liability instrument too early.
The 2019 European Commission report on liability for
artificial intelligence and other emerging digital technolo-
gies, admits that “giving robots or AI a legal personality
would not require including all rights natural persons, or
companies, have” (European Commission 2019: 38), so it is
clear that there is no problem in this regard, even if the main
criticism of electronic personhood concentrates on ethical
and human rights issues.
The main arguments from the liability point of view
against electronic personality are: (1) it “would not be prac-
tically useful, since civil liability is property liability, requir-
ing its bearer to have assets” (European Commission 2019:
38) and (2) “there is no need to give a legal personality to
emerging digital technologies”, as harm caused by them is
generally attributable to natural or legal persons (European
Commission 2019: 38). The concept of vicarious liability
is employed to argue that operators of machines, comput-
ers, robots or similar technologies should also be strictly
liable for their operations, based on an analogy to the basis
of vicarious liability (European Commission 2019: 25).
However, these arguments are not very convincing.
Indeed, the same “no assets” argument could then apply
to most limited liability companies, especially small and
medium size enterprises. As a matter of fact, the company
as a person is a legal fiction. Limited liability is a pivotal fea-
ture, which makes a company an attractive legal instrument
for doing business. Legal requirements for company assets
(capital) vary depending on the jurisdiction and the type of
24
  Only the strict liability of producers for defective products is har-
monized at EU level by the Product Liability Directive, while all
other regimes are regulated by the Member States themselves (Euro-
pean Commission 2019: 3).
company; however, even if the damage could be attributed
to the company as a legal person, there is no reimbursement
guarantee for victims. On the other hand, there is quite an
elaborate doctrine of lifting the corporate veil; however, the
courts rarely apply it.
Electronic personality as a liability instrument could
be coupled with mandatory insurance and strict liability
schemes. This would not only ensure compensation irrespec-
tive of fault in the long technological chain of products and
services, but also reduce the litigation costs for victims and
put a cap on liability. The synergy of three instruments (elec-
tronic personhood, strict liability and mandatory insurance)
would greatly contribute to legal certainty and, accordingly,
trust for customers and investors. Of course, an economic
analysis of the interplay of these three instruments would be
helpful, to identify the cheapest cost avoider and the cheap-
est taker of insurance, which most likely will be the producer
(putting product into circulation). Also, from the analysis of
autonomous artificial agents it is clear that it may be chal-
lenging to identify a benchmark for electronic personality.
One potential benchmark might be when the application of
artificial autonomous agents is safer and less likely to cause
damage to others than the activity of human actors. So, we
argue that idea of electronic personality should be further
elaborated as a possible instrument for liability purposes.
Relevant scholarship and legal documents in this field
emphasize that AI is a pervasive, transformative innova-
tion that needs a new approach. We agree that AI, as the
most innovative technology developed thus far, is one of
the main drivers of the Fourth Industrial Revolution and
should be treated differently to those who have preceded it.
In an open letter from the Future of Life Institute, signed by
more than 8000 scholars, leaders, and experts, it has been
argued that the impact of AI on society is likely to lead to
greater research investments in the field. The letter calls for
expanded research to reap the benefits of AI while avoiding
potential pitfalls (Future of Life Institute; Russell et al. 2015:
105–114). AI is pervasive because other innovations are
increasingly being based on AI technology as well, afforded
with human-like skills such as learning, speech recognition,
automated reasoning, sensing, interaction, problem-solving,
and creativity (World Commission on the Ethics of Scien-
tific Knowledge and Technology 2017: 4). From autonomous
vehicles to self-learning personal assistants, the application
scenarios are overwhelmingly vast, but so are the possible
ethical, societal, and legal implications.
One possible solution to the debate over whether AI
should be granted electronic personhood is the consideration
of existing levels of AI and justification of its rapid devel-
opment. The issue is primarily one of liability. In response,
we propose the following compromise: modified liability
mechanisms for harm that has been caused by autonomous
AI. The idea of sophisticated AI machines as agents would
13

perfectly match a strict, insurance-based liability regime
(Vladeck 2014). There should be a correlation between the
degree of AI autonomy and the applied liability regime. In
the case of deterministic AI devices that have basic or no
autonomy, the applied liability regime is strict, based on the
well-known product liability doctrine. In the case of cogni-
tive AI devices with high levels of autonomy, the applied
liability regime is also strict, but the novel regulation of lim-
ited liability schemes of autonomous artificially intelligent
agents, based on mandatory insurance, should be considered.
The proposed liability instrument would question, in part,
the traditional, well-developed doctrine of product liability
(Asaro 2012: 170–176), as well as negligence and the fault-
based liability system, which relies on a chain of causation.
However, this proposition would conform to notions of com-
pensatory justice and the apportionment of risk in society
(Vladeck 2014: 146–48). Also, a predictable liability regime
like this would spur innovation much more so than would
an “uncertain fault-based liability system” (Vladeck 2014:
147). Of course, this scheme should be examined against
other alternatives, such as product liability, trainer’s respon-
sibility, mandatory insurance, supplementary guaranty fund
to compensate victims of accidents, etc.
4 Conclusions
It is evident that the main reason behind the proposal of
electronic personhood in “Civil Law Rules on Robotics” is
to address scenarios where AI systems might cause harm,
especially in situations where AI algorithms learn and evolve
of their own accord. It is argued that there is a need to review
existing liability mechanisms due to the problems of foresee-
ability, traceability, and the establishment of liable or guilty
persons. However, there are also very strong arguments that
such decisions are premature, or simply acts of marketing—
in other words, political whims.
In “Civil Law Rules on Robotics”, the European Parlia-
ment appears to consider AIs as being easily identifiable
entities, but this is a misleading position. The definition of
AI is not obvious at all. The features of AI, such as intel-
ligence and autonomy, make these forms of technology
exceptionally difficult to regulate, as compared to other
human-made objects in the material world. The current
generation of AI completely lacks common sense, ration-
ality, consciousness, or critical thinking: all of which are
indispensable for evaluating data and information on one’s
own. The current wave of advances in AI does not actually
bring us intelligence that can act autonomously and safely.
So, from a technical point of view, it appears to be premature
and, likely, inappropriate to introduce electronic personhood
at the present time because (1) the scope of AI is unclear,
as a concept or as an artefact; (2) it is not clear what kind of
13
AI & SOCIETY
intelligence we are creating under the notion of “AI”; (3) it
is almost completely opaque technology.
In recent EU documents, there has been a tendency to
depart from the idea of electronic personality; however,
their arguments are not very convincing. We agree that for
now, taking into consideration the state of art of AI, the
current safety and liability regulation seems robust and reli-
able; however, very soon the adequacy and completeness of
liability regimes will become highly questionable.
The adoption of the concept of the “electronic person”
could provide an apt legal instrument for liability purposes
when AI development reaches a certain technological
level. AI, as the most innovative technology that has been
developed to date and one of the main drivers of the Fourth
Industrial Revolution, should be treated differently to previ-
ous technologies; there is a need to depart from traditional
liability theories. As one possible solution, we propose the
consideration of limited liability schemes of autonomous
artificially intelligent agents, based on mandatory insurance.
Mandatory insurance together with strict liability would not
only ensure compensation irrespective of the fault in the
long technological chain of products and services, but also
reduce the litigation costs for the victim and put a cap on
liability. Synergy of three instruments (electronic person-
hood, strict liability and mandatory insurance) would greatly
contribute to legal certainty and, accordingly, trust for cus-
tomers and investors.
Funding  This research is funded by the European Social Fund accord-
ing to the activity, “Improvement of Researchers”, qualification, by
implementing world-class R&D projects of Measure No. 09.3.3-LMT-
K-712. Special acknowledgment is dedicated to prof. J. Gordon (the
chief-researcher of this project) for his valuable ideas and contribution
to this research.
References
Aleksandre FM (2017) The legal status of artificially intelligent robots.
Personhood, taxation, and control. Dissertation, University of
Tilburg
Asaro PM (2012) A body to kick, but still no soul to damn: legal per-
spectives on robotics. In: Lin P, Abney K, Bekey GA (eds) Robot
ethics: the ethical and social implications of robotics, intelligent
robotics, and autonomous agents. MIT Press, Cambridge, pp
169–186
Atabekov A, Yastrebov O (2018) Legal status of artificial intelli-
gence across countries: legislation on the move. Eur Res Stud J
21(4):773–782
Atkinson R (2016) “It’s going to kill us!” and other myths about the
future of artificial intelligence. NCSSS J 21(1):8–11
Barfield W (2006) Intellectual property rights in virtual environments:
considering the rights of owners, programmers and virtual avatars.
Akron Law Rev 39(3):649–700
AI & SOCIETY
Beck S (2016) Intelligent agents and criminal law—negligence, dif-
fusion of liability, and electronic personhood. Robot Auton Syst
86:138–143
BEUC (2017) The consumer voice in Europe. Review of product liabil-
ity rules. BEUC Position Paper. https​://www.beuc.eu/publi​catio​
ns/beuc-x-2017-039_csc_revie​w_of_produ​ct_liabi​lity_rules​.pdf.
Accessed 7 Jan 2020
Borden MA (2006) Mind as machine: a history of cognitive science.
Oxford University Press, Oxford
Bryson JJ, Diamantis ME, Grant TD (2017) Of, for, and by people: the
legal lacuna of synthetic persons. Artif Intell Law 25:273–291
Čerka P, Grigienė J, Sirbikytė G (2017) Is it possible to grant legal
personality to artificial intelligence software system? Comput
Law Secur Rev Int J Technol Law Pract. https:​ //doi.org/10.1016/j.
clsr.2017.03.022
Chinen M (2019) Law and autonomous machines. Edward Elgar,
Cheltenham
Clarke A (1962) Hazards of prophecy: the failure of imagination. In:
Profiles of the future: an enquiry into the limits of the possible.
Gollancz, London
Clark E (2017) Embodied, situated, and distributed cognition. In: A
companion to cognitive science. Wiley
Council Directive (1985) On the approximation of the laws, regulations
and administrative provisions of the Member States concerning
liability for defective products, 85/374/EEC (OJ L 210, 7.8.1985)
Domingos P (2018) The master algorithm: how the quest for the ulti-
mate learning machine will remake our world. Basic Books, New
York
Dowell R (2018) Fundamental protections for non-biological intel-
ligences (or: how we learn to stop worrying and love our robot
Brethren). Minn J Law Sci Technol 19:305–335
European Commission (2018a) Staff Working Document on liability
for emerging digital technologies accompanying the document
Communication from the Commission to the European Parlia-
ment, the European Council, the Council, the European Economic
and Social Committee and the Committee of the Regions "Artifi-
cial Intelligence for Europe" SWD (2018) 137 final
European Commission (2018b) Communication from the Commission
to the European Parliament, the European Council, the Council,
the European Economic and Social Committee and the Committee
of the Regions on Artificial Intelligence for Europe COM (2018)
237 final
European Commission (2018c) Communication from the Commission
to the European Parliament, the European Council, the Council,
the European Economic, and Social Committee and the Commit-
tee of the Regions—Coordinated Plan on Artificial Intelligence.
COM (2018) 795 final
European Commission (2019) Liability for artificial intelligence and
other emerging digital technologies. Report from the Expert
Group on Liability and New Technologies—New Technologies
Formation, European Union. https​://ec.europ​a.eu/trans​paren​cy/
regex​pert/index​.cfm?do=group​Detai​l.group​Meeti​ngDoc​&docid​
=36608​Accessed 27 Dec 2019
European Commission (2020) Report on the safety and liability impli-
cations of artificial intelligence, the Internet of Things and robot-
ics. https:​ //ec.europa​ .eu/info/sites/​ info/files/​ report​ -safety​ -liabil​ ity-
artif​i cial​-intel​ligen​ce-feb20​20_en_1.pdf Accessed 30 Mar 2020
European Commission’s High-Level Expert Group on Artificial Intel-
ligence (2018) A definition of AI: main capabilities and scientific
disciplines. Brussel. https:​ //ec.europa​ .eu/digita​ l-single​ -market​ /en/
news/defini​ tion-​ artif​i cial-​ intell​ igenc​ e-main-capabi​ litie​ s-and-scien​
tific​-disci​pline​s. Accessed 17 June 2019
European Commission’s High-Level Expert Group on Artificial Intel-
ligence (2019) Ethics guidelines for trustworthy AI. Brussel. https​
://ec.europ​a.eu/digit​al-singl​e-marke​t/en/news/ethic​s-guide​lines​
-trust​worth​y-ai. Accessed 20 June 2019
European Parliament (2017) Civil rules on robotics. European Parlia-
ment resolution of 16 February 2017 with recommendations to the
Commission on Civil Law Rules on Robotics (2015/2103(INL)).
https​://www.europ​arl.europ​a.eu/doceo​/docum​ent/TA-8-2017-
0051_EN.pdf. Accessed 15 June 2019
European Parliament and Council Directive (1999) Amending Council
Directive 85/374/EEC on the approximation of the laws, regula-
tions and administrative provisions of the Member States con-
cerning liability for defective products, 1999/34/EC, (OJ L 141,
4.6.1999)
European Union (2019) Independent high-level expert group on arti-
ficial intelligence set up. Ethics guidelines for trustworthy AI.
https​://ec.europ​a.eu/digit​al-singl​e-marke​t/en/news/ethic​s-guide​
lines​-trust​worth​y-ai. Accessed 22 June 2019
Florian R (2003) Autonomous artificial intelligent agents. Center for
cognitive and neural studies. https​://coneu​ral.org/repor​ts/Coneu​
ral-03-01.pdf. Accessed 02 Feb 2020
Franklin S, Graesser A (1996) Is it an agent, or just a program? A
taxonomy for autonomous agents. In: Proceedings of the third
international workshop on agent theories, architectures, and lan-
guages. Springer
Goertzel B, Pennachin C (eds) (2007) Artificial general intelligence.
Springer, Berlin
Gordon JS (2018) What do we owe to intelligent robots?”. AI Soc. https​
://doi.org/10.1007/s0014​6-018-0844-6
Gunkel DJ (2018) Robot rights. The MIT Press, Cambridge
Hofstadter D (1999) Gödel, Escher, bach: an eternal golden braid.
Basic Books, New York
Jaynes TL (2019) Legal personhood for artificial intelligence: citizen-
ship as the exception to the rule. AI Soc https​://doi.org/10.1007/
s0014​6-019-00897​-9. https​://www.resea​rchga​te.net/publi​catio​
n/33400​3203_Legal​_perso​nhood​_for_artif​i cial​_intel​ligen​ce_citiz​
enshi​p_as_the_excep​tion_to_the_rule
Kritikos M (2019) European Parliament. Artificial intelligence ante
portas: Legal and ethical reflections. Briefing. European Par-
liamentary Research Service. https​://www.europ​arl.europ​a.eu/
at-your-servi​ce/files​/be-heard​/relig​ious-and-non-confe​ssion​al-
dialo​gue/event​s/en-20190​319-artif​i cial​-intel​ligen​ce-ante-porta​
s.pdf. Accessed 22 June 2019
Leeuwen VB, Verbruggen P (2015) Resuscitating EU product Liability
law? Eur Rev Private Law 23(5):899–915
Legg S, Hutter M (2007) A collection of definitions of intelligence, in
Proc. Conf. on Adv. in Artif. Gen. Intell, IOS, Amsterdam
Maes P (1995) Artificial life meets entertainment: life like autonomous
agents. Commun ACM 38:108–114
McCarthy J (2007) What is artificial intelligence? https​://www-forma​
l.stanf​ord.edu/jmc/whati​sai.pdf. Accessed 12 June 2019
Menary R (2007) Cognitive integration. Palgrave Macmillan, UK
O’Neil C (2016) Weapons of math destruction. Crown Publishers, New
York
Pagallo U (2018a) Vital, Sophia, and Co.—The quest for the legal
personhood of robots. Information 9:1–11
Pagallo U (2018b) Apples, oranges, robots: four misunderstandings in
today’s debate on the legal status of AI systems. Philos Trans R
Soc A. https​://doi.org/10.1098/rsta.2018.0168
Pinker S (1998) How the mind works. Penguin Press, London
Radutniy OE (2017) Criminal liability of the artificial intelligence.
Probl Legal 138:132–141
Renda A (2019) Artificial intelligence: ethics, governance, and policy
challenges. Report of a CEPS Task Force. Centre for European
Policy Studies (CEPS), Brussels
Riek LD, Howard D (2014) A code of ethics for the human–robot inter-
action profession, we robot. https​://robot​s.law.miami​.edu/2014/
wp-conte​nt/uploa​ds/2014/03/a-code-of-ethic​s-for-the-human​
-robot​-inter​actio​n-profe​ssion​-riek-howar​d.pdf. Accessed 14 June
2019
13

Robson RA (2010) Crime and punishment: rehabilitating retribution as
a justification for organizational criminal liability. Am Bus Law
J 47(1):109–144
Rosenschein SJ (1999) Intelligent agent architecture. In: Wilson RA,
Keil F (eds) The MIT encyclopedia of cognitive sciences. MIT
Press, Cambridge
Russell S, Dewey D, Tegmark M (2015) Research priorities for robust
and beneficial artificial intelligence. AI Mag 36(4):105–114
Samoili S, Lopez Cobo M, Gomez Gutierrez E, De Prato G, Martinez-
Plumed F, Delipetrev B (2020) AI WATCH. Defining Artificial
Intelligence, Publications Office of the European Union, Luxem-
bourg, 2020, (online). https​://doi.org/10.2760/38273​0 (online),
JRC118163. https:​ //ec.europa​ .eu/jrc/en/public​ ation​ /ai-watch-​ defin​
ing-artif​i cial​-intel​ligen​ce. Accessed 02 Apr 2020
Schwitzgebel E, Mara G (2015) A defense of the rights of artificial
intelligences. Midwest Stud Philos 30:98–119
Selwood M (2017) The road to autonomy. San Diego Law Rev
54:829–873
Singh S (2017) Attribution of legal personhood to artificially intelligent
beings. Bharati Law Rev 54:194–201
Smithers T (1995) Are autonomous agents information processing
systems? In: Steels L, Brooks R (eds) The artificial life route to
artificial intelligence: building embodied, situated agents. Law-
rence Erlbaum Associates, Hillsdale
Solum B (2017) Legal personhood for artificial intelligences. N C Law
Rev 70(4):1231–1287
Stone P, Brooks R, Brynjolfsson E, Calo R, Etzioni O, Hager G, Hirsch-
berg J, Kalyanakrishnan S, Kamar E, Kraus S, Leyton-Brown K,
Parkes D, Press W, Saxenian AL, Shah J, Tambe M, Teller A
(2016) Artificial intelligence and life in 2030. One hundred year
13
AI & SOCIETY
study on artificial intelligence: Report of the 2015–2016 Study
Panel, Stanford University, Stanford, CA, September 2016. https​
://ai100​.stanf​ord.edu/2016-repor​t. Accessed 17 July 2019
Stuart R, Norvig P (2010) Artificial intelligence: a modern approach,
3rd edn, Pearson Prentice Hall, Upper Saddle River
Suetonius De vita Caesarum (1913) Caligula, 55. https​://penel​ope.
uchic​ago.edu/Thaye​r/E/Roman​/Texts​/Sueto​nius/12Cae​sars/Calig​
ula*.html#55. Accessed 03 Mar 2020
Tegmark M (2014) Our mathematical universe. Random House LLC,
New York
Turing A (1950) Computing machinery and intelligence. Mind
59(236):433–460. https​://doi.org/10.1093/mind/LIX.236.433
Vladeck DC (2014) Machines without principals: liability rules and
artificial intelligence. Wash Law Rev 89(1):117–150
Winiger B, Karner E, Oliphant K (2018) Essential cases on miscon-
duct. Digest of European Tort Law. De Gruyter, Berlin
World Commission on the Ethics of Scientific Knowledge and Technol-
ogy (2017) Report of COMEST on robotics ethics. https​://unesd​
oc.unesc​o.org/ark:/48223​/pf000​02539​52. Accessed 20 July 2019.
Zevenbergen B, Finlayson M, Kortz M, Pagallo U, Borg JS, Zapušek T
(2018) Appropriateness and feasibility of legal personhood for AI
systems. In: International conference on robot ethics and standards
(ICRES 2018). https​://users​.cs.fiu.edu/~marka​f/doc/w16.zeven​
berge​n.2018.proci​cres.3.x_camer​a.pdf. Accessed 14 June 2019
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.