Cullings from AI Readings for XLRI

1. Artificial Intelligence Safety Engineering: Why Machine Ethics Is a Wrong Approach

Roman V. Yampolskiy* Roman V. Yampolskiy Department of Computer Engineering
and Computer Science, University of Louisville, V.C. Müller (Ed.): Philosophy and
Theory of Artificial Intelligence, SAPERE 5, pp. 389–396. springerlink.com © Springer-
Verlag Berlin Heidelberg 2012

The great majority of published papers are purely philosophical in nature and do little
more than reiterate the need for machine ethics and argue about which set of moral
convictions would be the right ones to implement in our artificial progeny
(Kantian [33], Utilitarian [20], Jewish [34], etc.). However, since ethical norms
are not universal, a “correct” ethical code could never be selected over others to
the satisfaction of humanity as a whole.
Consequently, we propose that purely philosophical discussions of ethics for
machines be supplemented by scientific work aimed at creating safe machines in
the context of a new field we will term “AI Safety Engineering.” Some concrete
work in this important area has already begun [17, 19, 18]. A common theme in
AI safety research is the possibility of keeping a superintelligent agent in a sealed
hardware so as to prevent it from doing any harm to humankind. Such ideas origi-
nate with scientific visionaries such as Eric Drexler who has suggested confining
transhuman machines so that their outputs could be studied and used safely [14].
Similarly, Nick Bostrom, a futurologist, has proposed [9] an idea for an Oracle AI
(OAI), which would be only capable of answering questions. Finally, in 2010
David Chalmers proposed the idea of a “leakproof” singularity [12]. He suggested
that for safety reasons, AI systems first be restricted to simulated virtual worlds
until their behavioral tendencies could be fully understood under the controlled
conditions.
Roman Yampolskiy has proposed a formalized notion of AI confinement pro-
tocol which represents “AI-Boxing” as a computer security challenge [46]. He de-
fines the Artificial Intelligence Confinement Problem (AICP) as the challenge of
restricting an artificially intelligent entity to a confined environment from which it
can’t exchange information with the outside environment via legitimate or covert
channels if such information exchange was not authorized by the confinement au-
thority. An AI system which succeeds in violating the CP protocol is said to have
escaped [46].
Similarly we argue that certain types of artificial intelligence research fall under
the category of dangerous technologies and should be restricted. Classical AI re-
search in which a computer is taught to automate human behavior in a particular
domain such as mail sorting or spellchecking documents is certainly ethical and
does not present an existential risk problem to humanity. On the other hand, we argue that
Artificial General Intelligence (AGI) research should be considered un-
ethical. This follows logically from a number of observations. First, true AGIs will
be capable of universal problem solving and recursive self-improvement. Conse-
quently they have potential of outcompeting humans in any domain essentially
making humankind unnecessary and so subject to extinction. Additionally, a truly
AGI system may possess a type of consciousness comparable to the human type
making robot suffering a real possibility and any experiments with AGI unethical
for that reason as well.
A similar argument was presented by Ted Kazynsky in his famous
manifesto [26]: “It might be argued that the human race would never be foolish
enough to hand over all the power to the machines. But we are suggesting neither
that the human race would voluntarily turn power over to the machines nor that
the machines would willfully seize power. What we do suggest is that the human
race might easily permit itself to drift into a position of such dependence on the
machines that it would have no practical choice but to accept all of the machines
decisions. As society and the problems that face it become more and more com-
plex and machines become more and more intelligent, people will let machines
make more of their decision for them, simply because machine-made decisions
will bring better result than man-made ones. Eventually a stage may be reached at
which the decisions necessary to keep the system running will be so complex that

human beings will be incapable of making them intelligently. At that stage the ma-
chines will be in effective control. People won't be able to just turn the machines
off, because they will be so dependent on them that turning them off would amount
to suicide. ” ( Kaczynski, T.: Industrial Society and Its Future. The New York Times
(September19, 1995)
Humanity should not put its future in the hands of the machines since it will not
be able to take the power back. In general a machine should never be in a position
to terminate human life or to make any other non-trivial ethical or moral judgment
concerning people.
2. Why and How Should Robots Behave Ethically?, Benjamin KUIPERS 1, Computer
Science & Engineering, University of Michigan, USA, 1University of Michigan, 2260
Hayward Street, Ann Arbor, Michigan 48109 USA, Email: kuipers@umich.edu
For an intelligent robot to function successfully in our society, to cooperate with
humans, it must not only be able to act morally and ethically, but it must also
be trustworthy. It must earn and keep the trust of humans who interact with it.
If every participant contributes their share, everyone
gets a good outcome. But each individual participant may do even better by
optimizing their own reward at the expense of the others. With self-centered utility
functions, each participant “rationally” maximizes their own expected utility,
often leading to bad outcomes for everyone.

 Should you use a sharp knife to cut into the body of a human being? Of
course not, unless you are a qualified surgeon performing a necessary op-
eration. (Deontology: a rule with an exception.)

 If you are that surgeon, is it permissible to sacrifice this patient in order to

save the lives of five others? Of course not! (Virtue ethics: a good surgeon
keeps faith with the patient.)

 Is it OK to throw the switch that saves five lives by directing a runaway trolley onto a
side track, where it will kill one person who would have been safe? Well, . . .
 (Deontology says it’s wrong to allow preventable deaths; Utilitarianism says fewer
deaths is better; Virtue ethics says the virtuous person can make hard choices.)

I argue that heuristics based on utilitarianism (decision theory), deontology (rule-

based and constraint-based systems), and virtue ethics (case-based reasoning) are
all important tools in the toolkit for creating artificial agents capable of partic-
ipating successfully in our society. Each tool is useful in certain contexts, and
perhaps less useful in others.

3. The Virtuous Machine -Old Ethics for New Technology? Nicolas Berberich* and Klaus
Diepold, Department of Electrical and Computer Engineering, Technical University of
Munich, Department of Informatics, Technical University of Munich, Munich Center
for Technology in Society,* E-mail: n.berberich@tum.de
Due to the inherent autonomy of these systems, the ethical considerations have to be
conducted by themselves. This means, that these autonomous cognitive machines are in
need of a theory, with the help of which they can, in a specific situation, choose the action
that adheres best to the moral standards.
This discrepancy between what people believe that technology can do, based on its
appearance, and what it actually can do, would not only elicit a strong uncanny valley effect,
but also pose a large safety risk. Taken together, we predict that this would lead to an
acceptance problem of the technology. If we want to avoid this by jumping over the
uncanny valley, we have to start today by thinking about how to endow autonomous
cognitive systems with more human-like behavior. The position that we argue for in this
paper is that the last discrepancy between the valley and its right shore lies in virtuous
moral behavior. In the near future
we will have autonomous cognitive machines whose actions will be akin to human actions,
but without consideration of moral implications they will never be quite alike, leading to
cognitive dissonance and rejection. We believe that taking virtue ethics as the guiding moral
theory for building moral machines is a promising approach to avoid the uncanny valley and
to induce acceptance.
Cybernetics can be seen as a historical and intellectual precursor of artificial intelligence
research. While it had strong differences with the cognitivistic GOFAI (good old-fashioned
AI), cybernetic ideas are highly influential in modern AI. The currently successful field of ar-
tificial neural networks (synonymous terms are connectionism and deep learning) originated
from the research of the cyberneticians McCulloch, Pitts and Rosenblatt. Goal-directed
planning is a central part of modern AI and especially of advanced robotics. In contrast
to other forms of machine learning like supervised or unsupervised learning, reinforcement
learning is concerned with the goal-driven (and therefore teleological) behavior of agents.

Applied to AI ethics this means that a machine cannot have practical wisdom (and thus can’t
act morally) before it has learned from realistic data. Machine learning is the im-provement
of a machine’s performance of a task through experience and Aristotle’s virtue ethics is the
improvement of one’s virtues through experience. Therefore, if one equates the task
performance with virtuous actions, developing a virtue ethics-based machine appears
possible.
A closer look at the structure of Aristotle’s ergon-argument allows to break with two
common misconceptions which seem to render a virtue ethical approach in machine ethics
impossible. The first misconception is ethical anthropocentrism, after which only humans
can act morally. This might have been correct in the past, but only because humans have
been the only species capable of higher-level cognition, which, according to Aristotle, is
a requirement for ethical virtues and thus moral action. If there was another species, for
example a machine, with the same capacity for reason and dispositions of character, then it
appears probable that its aret ̄e would also lie in excellent use and improvement of those.
The second misconception of Aristotle’s virtue ethics is that it takes happiness to be the goal
and measure of all actions. Since machines are not capable of genuine feelings of happiness,
it is argued, that virtue ethics can’t be applied to them. This argument is based on an
erroneous understanding of eudaimonia. Aristotle does not mean psychological states of
happiness nor maximized pleasure, as John Locke defines ’happiness’. The Greek term
eudaimonia has a much broader meaning and refers mainly to a successful conduct of life
(according to one’s ergon). A virtuous machine programed to pursue eudaimonia would
therefore not be prone to wireheading, which is the artificial stimulation of the brain’s
reward center to experience pleasure.
Out of the three subcategories of machine learning, supervised learning, unsupervised
learning and reinforcement learning (RL), the latter is the lifeworldly approach. In contrast
to the other two, RL is based on dynamic interaction with the environment, of which the
agent typically has only imperfect knowledge.
This partition originated in Aristotle’s soul theory in which he lists virtues of reason
(dianoetic virtues) next to virtues of character (ethical virtues) as properties of the
intelligent part of the soul. The virtues of reason comprise the virtues of pure reason and
the virtues of practical reason. Pure reason includes science (epist ̄em ē ), wisdom (sophia)
and intuitive thought (no ̄us). Practical reason on the other hand refers to the virtues of
craftsmanship (techn ē ), of making (poi ̄esis) and practical wisdom (phron ̄esis). According to
this subdivision in pure and practical reason, there exist two ways to lead a good life in the
eudaimonic sense: the theoretical life and the practical life. AI systems can lead a theoretical
life of contemplation, e.g. when they are applied to scientific data analysis, but to lead a
practical life they need the capacity for practical wisdom and morality. This distinction in
theoretical and practical life of an AI somewhat resembles the distinction into narrow and
general AIs, where narrow AI describes artificial intelligence systems that are focused on
performing one specific task (e.g. image classification) while general AI can operate in more
general and realistic situations.
In contrast to deontology and consequentialism, virtue ethics has a hard time giving
reasons for its actions (they certainly exist, but are hard to codify). While deontologists can
point towards the principles and duties which have guided their actions, a consequentialist
can explain why her actions have led to the best consequences. An AMA based on virtue
ethics on the other hand would have to show how its virtues, which gave rise to its actions,
have been formed through experience. This poses an even greater problem if its capability
to learn virtues has been implemented as an artificial neural network, due to it being almost
impossible to extract intuitively understandable reasons from the many network weights. In
this instance, the similarity between virtue ethics and machine learning is disadvantageous.
Without being able to give reasons to one’s actions, one cannot take over responsibility,
which is a concept underlying not only our insurance system but also our justice system. If
the actions of an AMA produce harm then someone has to take responsibility for it and the
victims have a right to explanation. The latter has recently (May 2018) been codified by the
EU General Data Protection Regulation (GDRP) with regards to all algorithmic decisions.
Condensed to the most important ideas, this work has shown that
1. Virtue ethics fits nicely with modern artificial intelligence research and is a promising
moral theory as basis for the field of AI ethics.
2. Taking the virtue ethics route to building moral machines allows for a much broader
approach than simple decision-theoretic judgment of possible actions. Instead it takes
other cognitive functions into account like attention, emotions, learning and actions.
Furthermore, by discussing several virtues in detail, we showed that virtue ethics is a
promising moral theory for solving the two major challenges of contemporary AI safety
research, the control problem and the value alignment problem. A machine endowed with
the virtue of temperance would not have any desire for excess of any kind, not even for
exponential self-improvement, which might lead to a superintelligence posing an existential
risk for humanity. Since virtues are an integral part of one’s character, the AI would not
have the desire of changing its virtue of temperance. Learning from virtuous exemplars
has been a process of aligning values for centuries (and possibly for all of human history),
thus building artificial systems with the same imitation learning capability appears to be a
reasonable approach.

4. Machines That Know Right And Cannot Do Wrong: The Theory and Practice of
Machine Ethics, Louise A. Dennis and Marija Slavkovik
“The fact that man knows right from wrong proves his intellectual superiority to the other
creatures; but the fact that he can do wrong proves his moral inferiority to any creatures
that cannot.”– Mark Twain
Wallach and Allen [35, Chapter 2] distinguish between operational morality, functional
morality, and full moral agency. An agent has operational morality when the moral
significance of her actions are entirely scoped by the agent’s designers. An agent has
functional morality when the agent is able to make moral judgements when choosing an
action, without direct human instructions.
5. What happens if robots take the jobs?The impact of emerging technologies on
employment and public policy By Darrell M. West
In this paper, I explore the impact of robots, artificial intelligence, and machine learning. In
particular, I study the impact of these emerging technologies on the workforce and the
provision of health benefits, pensions, and social insurance. If society needs fewer workers
due to automation and robotics, and many social benefits are delivered through jobs, how
are people outside the workforce for a lengthy period of time going to get health care and
pensions?
Robots are expanding in magnitude around the developed world. Figure 1 shows the
numbers of industrial robots in operation globally and there has been a substantial increase
in the past few years. In 2013, for example, there were an estimated 1.2 million robots in
use. This total rose to around 1.5 million in 2014 and is projected to increase to about 1.9
million in 2017.5 Japan has the largest number with 306,700, followed by North America
(237,400), China (182,300), South Korea (175,600), and Germany (175,200). Overall, robotics
is expected to rise from a $15 billion sector now to $67 billion by 2025.6
In the contemporary world, there are many robots that perform complex functions.
According to a presentation on robots, “the early 21st century saw the first wave of
companionable social robots. They were small cute pets like AIBO, Pleo, and Paro. As
robotics become more sophisticated, thanks largely to the smart phone, a new wave of
social robots has started, with humanoids Pepper and Jimmy and the mirror-like Jibo, as
well as Geppetto Avatars’ software robot, Sophie. A key factor in a robot’s ability to be social
is their ability to correctly understand and respond to people’s speech and the underlying
context or emotion.”
Amazon has organized a “picking challenge” designed to see if robots can “autonomously
grab items from a shelf and place them in a tub.” The firm has around 50,000 people
working in its warehouses and it wants to see if robots can perform the tasks of selecting
items and moving them around the warehouse. During the competition, a Berlin robot
successfully completed ten of the twelve tasks. To move goods around the facility, the
company already uses 15,000 robots and it expects to purchase additional ones in the
future.
In the restaurant industry, firms are using technology to remove humans from parts of food
delivery. Some places, for example, are using tablets that allow customers to order directly
from the kitchen with no requirement of talking to a waiter or waitress. Others enable
people to pay directly, obviating the need for cashiers. Still others tell chefs how much of an
ingredient to add to a dish, which cuts down on food expenses.
There are computerized algorithms that have taken the place of human transactions. We
see this in the stock exchanges, where high-frequency trading by machines has replaced
human decision-making. People submit, buy, and sell orders, and computers match them in
the blink of an eye without human intervention. Machines can spot trading inefficiencies or
market differentials at a very small scale and execute trades that make money for people.
Some individuals specialize in arbitrage trading, whereby the algorithms see the same stocks
having different market values. Humans are not very efficient at spotting price differentials
but computers can use complex mathematical formulas to determine where there are
trading opportunities. Fortunes have been made by mathematicians who excel in this type
of analysis.
Machine-to-machine communications and remote monitoring sensors that remove humans
from the equation and substitute automated processes have become popular in the health
care area. There are sensors that record vital signs and electronically transmit them to
medical doctors. For example, heart patients have monitors that compile blood pressure,
blood oxygen levels, and heart rates. Readings are sent to a doctor, who adjusts medications
as the readings come in. According to medical professionals, “we’ve been able to show
significant reduction” in hospital admissions through these and other kinds of wireless
devices.
There also are devices that measure “biological, chemical, or physical processes” and deliver
“a drug or intervention based on the sensor data obtained.”22 They help people maintain
an independent lifestyle as they age and keep them in close touch with medical personnel.
“Point-of-care” technologies keep people out of hospitals and emergency rooms, while still
providing access to the latest therapies.
Implantable monitors enable regular management of symptoms and treatment. For
example, “the use of pulmonary artery pressure measurement systems has been shown to
significantly reduce the risk of heart failure hospitalization.” Doctors place these devices
inside heart failure patients and rely upon machine-to-machine communications to alert
them to potential problems. They can track heart arrhythmia and take adaptive moves as
signals spot troublesome warning signs.
Unmanned vehicles and autonomous drones are creating new markets for machines and
performing functions that used to require human intervention. Driverless cars represent one
of the latest examples. Google has driven its cars almost 500,000 miles and found a
remarkable level of performance. Manufacturers such as Tesla, Audi, and General Motors
have found that autonomous cars experience fewer accidents and obtain better mileage
than vehicles driven by people.
6. CaseWestern Reserve Journal of International Law 47 (2015), Issue 1
The Debate Over Autonomous Weapons Systems Dr. Gregory P. Noone and Dr.
Diana C. Noone
The debate over Autonomous Weapon Systems (AWS) has begun
in earnest with advocates for the absolute and immediate banning of
AWS development, production, and use planting their flag first. They
argue that AWS should be banned because these systems lack human
qualities, such as the ability to relate to other humans and to apply
human judgment, that are necessary to comply with the law. In
addition, the weapons would not be constrained by the capacity for
compassion, which can provide a key check on the killing of civilians.2
The opposing viewpoint in this debate articulates numerous
arguments that generally include: it is far too premature and too
speculative to make such a proposal/demand; the Law of Armed
Conflict should not be underestimated in its ability to control AWS
development and future operations; AWS has the potential to
ultimately save human lives (both civilian and military) in armed
conflicts; AWS is as inevitable as any other technology that could
potentially make our lives better; and to pass on the opportunity to
develop AWS is irresponsible from a national security perspective.3
Some of the most respected and brilliant lawyers in this field are on
opposite sides of this argument.

1. Human-in-the-loop or semi-autonomous systems require a

human to direct the system to select a target and attack it, such
as Predator or Reaper UAVs.
2. Human-on-the loop or human-supervised autonomous systems
are weapon systems that select targets and attack them, albeit
 with human operator oversight; examples include Israel’s Iron
Dome and the U.S. Navy’s Phalanx Close In Weapons System
(or CIWS).
3. Human-out-of-the-loop or fully autonomous weapon systems
can attack without any human interaction; there are currently
no such weapons.

First and foremost, there is immediate common ground to be

found in this debate. Any weaponry development shall be done so in
accordance with the Law of Armed Conflict (LOAC, also referred to
as International Humanitarian Law, IHL, or the Law of War). With
respect to AWS, its development and deployment would be required
to adhere to LOAC’s core principles of distinction, proportionality,
humanity and military necessity. There is readily accepted treaty
law as well as customary international law that makes this area of
discussion easy. AWS is, as all weapons and weapon systems are, a
means of warfare (whereas a method of warfare involves deployment
and tactics). All AWS would have a legal review conducted prior to
formal development as a weapon (or prior to any modification of an
existing weapon) and another legal review prior to being deployed in
the field. Therefore, the concept of AWS is not per se unlawful. At
their core, autonomous weapon systems must be able to distinguish
combatants from non-combatants as well as friend from foe. LOAC is
designed to protect those who cannot protect themselves, and an
underlying driver is to protect civilians from death and combatants
from unnecessary suffering. Everyone is in agreement on this. No
academic or practitioner is stating anything to the contrary; therefore,
this part of any argument from either side must be ignored as a red
herring. Simply put, no one would agree to any weapon that ignores
LOAC obligations.
At the present time, there are many questions and as yet few
answers with respect to Autonomous Weapon Systems. Not the least
of which include the policy implications of such systems. For instance,“How does this
technology impact the likely successes of counter-
insurgency operations or humanitarian interventions? Does not such weaponry run
the risk of making war too easy to wage and tempt policy makers into killing when
other more difficult means should be undertaken?” Will countries be more willing to
use force because their populations would have less to lose (i.e. their loved ones)
and it would be politically more acceptable?

7. Robots, Rights and Religion, James F. McGrath, Butler University, 2011

To put this another way, we might decide that we could exclude from the category of
persons those artificial intelligences that were merely programmed to imitate personhood,
and whose interaction with humans resembled that of persons simply as a result of
elaborate programming created precisely to imitate human behavior. This must be
distinguished from the case of a machine that learns human behavior and imitates it of its
own volition. This distinction is not arbitrary. Children carry out patterns of behavior that
resemble those of their parents and others around them. This is part of the learning
process, and is evidence in favor of rather than against their true personhood. The evidence
that I am suggesting would count against genuine personhood is deliberate programming by
a human programmer that causes a machine to imitate personhood in a contrived manner.
The reason for this distinction is an important one. A machine that learns to imitate human
behavior would be exhibiting a trait we witness in human persons.
In concluding this section, it should be remarked that we give human rights to human beings
as soon as they are clearly categorized as such. A person does not have to be able to speak
to have rights. Indeed, small infants whose ability to reason, communicate and do many
other things that we tend to identify with intelligence is still in the process of formation
have their rights protected by law. The issue is thus not really rights for artificial
intelligences so much as rights for machine persons. It is the definition and identification of
the latter that is the crucial issue.
Nevertheless, the distinction would seem to be a valid one for as long as it remains a
meaningful one: machines that develop their own personhood in imitation of humans will
probably deserve to be recognized as persons, whereas mere simulacra designed as an
elaborate contrivance will not.
Our creations – whether through natural biological reproduction, in vitro fertilization,
cloning, genetic construction, or artificially intelligent androids made in our image – can be
viewed as in some sense like our children. And if the comparison to our children is a useful
analogy, then we can learn much from it. There is a “flip side” to the point that children are
their own people and sooner or later we need to let them go, to make their own mistakes.
The other side of the coin is that we are not living up to our responsibilities if we let them go
too soon. Yet Is it only wrong to tamper with humanity’s nature, or is it also wrong to create
a human being (with some differences)?
our artificial offspring will in an important sense not be human, even if they are made in our
image. Other species leave the nest far earlier than human children do. In “giving birth” not
to other humans but to artificial intelligence, we cannot assume that the process will even
closely mirror a typical human parent-child scenario.

8. Robot ethics: Mapping the issues for a mechanized world , Patrick Lin Keith Abney
George Bekey
Bill Gates recently observed that “the emergence of the robotics industry ... is developing in
much the same way that the computer business did 30 years ago” . As a key architect of the
computer industry, his prediction has special weight.
In a few decades—or sooner, given exponential progress forecasted by Moore’s Law—
robots in society will be as ubiquitous as computers are today, he believes; and we would be
hard-pressed to find an expert who disagrees.
In its most basic sense, we define “robot” as an engineered machine that senses, thinks, and
acts: “Thus a robot must have sensors, processing ability that emulates some aspects of
cognition, and actuators.
Surprisingly, relationships of a more intimate nature are not quite satisfied by robots yet,
considering the sex industry’s reputation as an early adopter of new technologies.
Introduced in 2010, Roxxxy is billed as “the world’s first sex robot” [17], but its lack of
autonomy or capacity to “think” for itself, as opposed to merely respond to sensors,
suggests that it is not in fact a robot, per the definition above.
In some countries, robots are quite literally replacements for humans, such as Japan, where
a growing elderly population and declining birthrates mean a shrinking workforce . Robots
are built to specifically fill that labor gap. And given the nation’s storied love of technology,
it is therefore unsurprising that approximately one out of 25 workers in Japan is a robot.
While the US currently dominates the market in military robotics, nations such as Japan and
South Korea lead in the market for social robotics, such as elderly-care robots. Other nations
with similar demographics, such as Italy, are expected to introduce more robotics into their
societies, as a way to shore up a decreasing workforce; and nations without such concerns
can drive productivity, efficiency, and effectiveness to new heights with robotics.
Like the social networking and email capabilities of the Internet Revolution, robotics may
profoundly impact human relationships. Already, robots are taking care of our elderly and
children, though there are not many studies on the effects of such care, especially in the
long term. Some soldiers have emotionally bonded with the bomb-disposing PackBots that
have saved their lives, sobbing when the robot meets its end. And robots are predicted to
soon become our lovers and companions. they will always listen and never cheat on us.
Given the lack of research studies in these areas, it is unclear whether psychological harm
might arise from replacing human relationships with robotic ones.
Harm also need not be directly to persons, e.g., it could also be to the environment. In the
computer industry, “e-waste” is a growing and urgent problem, given the disposal of heavy
metals and toxic materials in the devices at the end of their product lifecycle. Robots as
embodied computers will likely exacerbate the problem, as well as increase pressure on
rare-earth elements needed today to build computing devices and energy resources needed
to power them. Networked robots would also increase the amount of ambient
radiofrequency radiation, like that created by mobile phones—which have been blamed,
fairly or not, for a decline of honeybees necessary for pollination and agriculture, in addition
to human health problems.
 9. Networks of Social and Moral Norms in Human and Robot Agents, B. F. Malle ∗† M.
Scheutz ∗∗ J. L. Austerweil, Department of Cognitive, Linguistic, and Psychological
Sciences,Brown University, USA∗∗ Department of Computer Science, Tufts
University, USA
The design and construction of intelligent robots has seen steady growth in the past 20
years, and the integration of robots into society is, to many, imminent (Nourbakhsh, 2013;
Sabanovi ̆ c, 2010). Ethical questions about such integration have recently gained
prominence. For example, academic publications on the topic of robot ethics doubled
between 2005 and 2009 and doubled again since then, counting almost 200 as of the time
of this conference (Malle, 2015).
Economic scholars have puzzled for a long time why such free-riding is not more common—
why people cooperate much more often than they “defect,” as game theorists call it, when
defecting would provide the agent with larger utility.
The answer cannot be that humans are “innately” cooperative, because they are perfectly
capable of defecting. The answer involves to a significant extent the power of norms. A
working definition of a norm is the following: An instruction to (not) perform a specific or
general class of action, whereby a sufficient number of individuals in a community (a)
indeed follow this instruction and (b) expect others in the community to follow the
instruction.
10. Moral Machines and the Threat of Ethical Nihilism, Anthony F. Beavers
But, though my cell phone might be smart, I do not take that to mean that it is thoughtful,
insightful, or wise. So, what has become of these latter categories? They seem to be
bygones, left behind by scientific and computational conceptions of thinking and knowledge
that no longer have much use for them.
Respecting
Kantian ethics, the problem is apparent in the universal law formulation of the categorical
imperative, the one that would seem to hold the easiest prospects for rule-based
implementation in a computational system: “act as if the maxim of your action were to
become through your will a universal law of nature ”( Kant [1785] 1981 , 30).
One mainstream interpretation of this principle suggests that whatever rule (or maxim) I
should use to determine my own behavior must be one that I can consistently will to be
used to determine the behavior of everyone else. (Kant ’s most consistent example of this
imperative in application concerns lying promises. One cannot make a lying promise without
simultaneously willing a world in which lying is permissible, thereby also willing a world in
which no one would believe a promise, particularly the very one I am trying to make. Thus,
the lying promise fails the test and is morally impermissible.) Though at first the categorical
imperative looks implementable from an engineering point of view, it suffers from a
problem of scope, since any maxim that is defined narrowly enough (for instance, to include
a class of one, anyone like me in my situation) must consistently universalize. Death by
failure to implement looks imminent; so much the worse for Kant, and so much the better
for ethics.
Classical utilitarianism meets a similar fate, even though, unlike Kant, Mill casts internals,
such as intentions, to the wind and considers just the consequences of an act for evaluating
moral behavior. Here, “actions are right in proportion as they tend to promote happiness;
wrong as they tend to produce the reverse of happiness. By happiness is intended pleasure
and the absence of pain; by unhappiness, pain and the privation of pleasure” That internals
are incidental to utilitarian ethical assessment is evident in the fact that Mill does not
require that one act for the right reasons. He explicitly says that most good actions are not
done accordingly, Thus, acting good is indistinguishable from being good, or, at least, to be
good is precisely to act good; and sympathetically we might be tempted to agree, asking
what else could being good possibly mean. Things again are complicated by problems of
scope, though Mill, unlike Kant, is aware of them. He writes, “again, defenders of utility
often find themselves called upon to reply to such objections as this — that there is not
enough time, previous to action, for calculating and weighing the effects of any line of
conduct on the general happiness”( [1861] 1979 , 23). (In fact, the problem is
computationally intractable when we consider the ever-extending ripple effects that any act
can have on the happiness of others across both space and time.) Mill gets around the
problem with a sleight of hand, noting that “all rational creatures go out upon the sea of life
with their minds made up on the common questions of right and wrong”(24), suggesting
that calculations are, in fact, unnecessary, if one has the proper forethought and upbringing.
Again, the rule is of little help, and death by failure to implement looks imminent. So much
the worse for Mill; again, so much the better for ethics.

11. Moral Robots—Elizebeth Huh

His efforts to influence the public through reason also recall Plato’s theory of human
motivation, illustrated by his conception of the tripartite soul. In Book II of The Republic,
Socrates explains that the root of each human desire can find its origin in one part of a
three-part soul: the lowest part, epithumia, motivates the appetites and the desire for
bodily pleasure; the middle part of the soul, thumos, desires honor and a competitive form
of self-interest; and the highest part of the soul, logos, loves reason and knowledge.12
Socrates explains that the size of each of these three parts varies among individuals, but
that the largest piece of each person’s soul naturally guides her into one of three social
classes. The majority of the population, motivated by bodily appetites, become the
moneymakers and craftsmen of the city and fulfill its basic needs; those who desire honor
are best fit to serve as the guardians and warriors of the city; and philosophers, whose
actions are ruled above all by reason, ought to rule.
Though Singer does not go so far as to claim that
effective altruists ought to completely strip away their emotions,
he does insist that the emotions distract from his principle of
efficiency, and that this principle is essential for optimal moral
decision-making. So this is what I consider his ultimate mistake: it
is his conflation of morality with efficiency, and his belief that we do
not need the emotions and some acceptance of uncertainty on our
path to moral progress.
Why is this a mistake? Let’s look at the legalization of gay marriage
in the United States. Singer’s effective altruist doctrine would have
maintained that the suffering of homosexual couples was not as
great as the suffering of those starving to death, and that, therefore,
“maximally effective” altruists wishing to do “the most good”
possible should not have considered spending any time, money, or
resources fighting for some the right to some highly subjective form
of emotional fulfilment.

12. Moral Machines: Mindless Morality and its Legal Implications--Andrew Schmelzer
Not all of our devices need moral agency. As autonomy increases, morality becomes more
necessary in robots, but the reverse also holds. Machines with little autonomy need less
ethical sensitivity. A refrigerator need not decide if the amount someone eats is healthy,
and limit access accordingly. In fact, that fridge would infringe on human autonomy.
Ethical sensitivity does not require moral perfection. I do not expect morally perfect
decisions from machines. In fact, because humans are morally imperfect, we cannot base
moral perfection off of humanity by holding machines to human ideals. Our moral
development continues today, and I believe may never finish. Designing an artificial moral
agent bound by the morality of today dooms it to obsolescence: ethical decisions from a
hundred years ago look much more racist, sexist, etc., and less ‘good’ from today’s
perspective; today’s ethics might have the same bias when viewed from the future
(Creighton, 2016). Because the nature of our ethics changes, an agent will stumble
eventually. Instead, we strive for morally human (or even better than human) decisions
from machines. When a machine’s actions reflect those of a human, we will have met the
standards for artificial moral agency.
We can test for artificial moral agency with the Moral Turing Test (Allen, Varner, & Zinser,
2000). In the MTT, where a judge tries to differentiate between a machine and a person by
their moral actions. An agent passes the test when the judge cannot correctly identify the
machine more often than chance. Then, a machine qualifies as a moral agent. In the
comparative Moral Turing Test (cMTT), the judge compares the behaviors of the two
subjects, and determines which action is morally better than the other (Allen, Varner, &
Zinser, 2000). When a machine’s behavior consistently scores morally preferable to a
human’s behavior, then either the agent will have surpassed human standards, or the
human’s behavior markedly strays from those standards.
Frankena (1973) provides a list of terminal values — virtues that are valued for themselves,
rather than their consequences (Yudkowsky, 2011):
Life, consciousness, and activity; health and strength; pleasures and satisfactions of all
or certain kinds; happiness, beatitude, contentment, etc.; truth; knowledge and true
opinions of various kinds, understanding, wisdom; beauty, harmony, proportion in
objects contemplated; aesthetic experience; morally good dispositions or virtues; mutual
affection, love, friendship, cooperation; just distribution of goods and evils; harmony and
proportion in one’s own life; power and experiences of achievement; self-expression;
freedom; peace, security; adventure and novelty; and good reputation, honor, esteem,
etc.
Programming all of those values directly into a single utility function (the method of
determining positive or negative results) is ridiculous. Can engineers or ethicists quantify
each value and agree on a prioritization for each? Yudkowsky (2011) proposes a ‘one-wrong-
number’ problem: a phone number has 10 digits, but dialing one wrong number does not
mean you will connect with someone 90% like the person intended. The same may apply to
virtue-based machines.
Furthermore, some values we deem worthy of implementation in our machines may
contradict each other, such as compassion and honesty (e.g. a child’s professional baseball
potential). In this way virtue-based systems still require the caveats of a rule-based system
(Allen, Varner, & Zinser, 2000). But what about non-terminal virtues, that is, virtues we
value for their repercussions?
The three methods of bottom-up development I will discuss here are neural network
learning, genetic algorithms, and scenario analysis systems.
Neural networks function similarly to neurons: connections between inputs and outputs
make up a system that can learn to do various things, from playing computer games to
running bipedally in a simulation. By using that learning capability on ethical endeavors, a
moral machine begins to develop. From reinforcement of positive behaviors and penalty of
negative ones, the algorithm learns the pattern of our moral systems. Eventually, engineers
place the algorithm in charge of a physical machine, and away it goes. One downside to this
is the uncertainty regarding what the algorithm learned. When the army tried to get a
neural net to recognize tanks hidden in trees, what looked like a distinction between trees,
tanks, and partly concealed tanks turned out to be a distinction between a sunny and cloudy
day (Dreyfus & Dreyfus, 1992). Kuang (2017) writes about Darrel’s potential solution: having
two neural networks working side by side. The first learns the correlation between input and
output, challenging situation and ethically right decision, respectively. The second algorithm
focuses on learning language and connects tags or captions from an input and explains what
cues and ideas the second algorithm used to come up with a course of action.
The second weak point stems from allowing mistakes: no amount of learning can verify that
the machine will act morally in all situations in the real world, including those not tested and
learned from.
Genetic algorithms operate on a somewhat similar principle. Large numbers of simple digital
agents run through ethically challenging simulations. The ones that return the best scores
get “mated” with each other, blending code with a few randomizations, and then the test
runs again (Fox, 2009). After the best (or acceptably best) scores based on desired outcomes
are achieved, a new situation is added to the repertoire that each program must surpass. In
this way, machines can learn our moral patterns. Once thoroughly evolved, we implement
the program, and the machine operates independently in the real world. Alternatively to
direct implementation, we could evolve the program to learn patterns quickly and
efficiently, and then run it through the neural network training. This method suffers the
same downsides as neural networking: we cannot tell what it learned or whether it will
make mistakes in the future.
The final approach involves scenario analysis. Parkin (2017) describes a method of teaching
AI by having it read books and stories and learn the literature’s ideas and social norms.
While this may not apply to machines we do not intend to behave as humans, the idea still
applies to niche or domain-specific machines. Instead of using literature as learning input,
we provide a learning program with records of past wrongdoings and successful outcomes
of ethically-blind machines in its niche. Then the program could infer the proper behaviors
for real world events it may encounter in the future. After analyzing the settings and events
of each scenario, the program would save the connections it made for later human
inspection. If the program’s connections proved ‘good,’ it would then receive a new batch of
scenarios to test through, and repeat the cycle. One downside to this approach involves
painstaking human analysis. A new program would have to go through this cycle for every
machine niche that requires a moral agent, and a human evaluator would have to carefully
examine every connection and correlation the program develops. Darrel’s (2017) explaining
neural net could work in tandem with a scenario analysis system to alleviate the human
requirement for analysis. This approach does get closer to solving the issue of working in
new environments than the previous two approaches, but may nonetheless stumble once
implemented in reality.
Bottom-up approaches utilize continued development to reach an approximation of moral
standards. Neural networks develop connections and correlations to create a new output,
but we struggle to know why the system comes to a decision. Genetic algorithms refine
themselves by duplicating the most successful code into the next generation of programs,
with a little mutation for adaptation. A genetic algorithm’s learning process also remains
obscured without careful record of iterations, which may be beyond human comprehension.
Scenario analysis systems can learn the best conduct historically shown as ethically right,
but still retains potential for error. As of yet, we do not have a reliable method to develop
an artificial moral agent.
To build an artificial moral agent, DeBaets (2014) argues that a machine must have
embodiment, learning, teleology toward the good, and empathy.
DeBaets (2014) claims that moral functioning requires embodiment because if a machine
acts in and influences the physical world, it must have a physical manifestation.
“Embodiment [requires] that a particular decision-making entity be intricately linked to a
particular concrete action; morality cannot solely be virtual if it is to be real” (DeBaets,
2014). They can work from a distance, have multiple centers of action, and have distributed
decision-making centers, but each requires physical form. Embodiment constrains machines
to a physical form, so this definition of moral agency excludes algorithms and programs that
do not interact with the physical world.
Ethical machines need learning capacity so they can perform as taught by ethical and moral
rules and extrapolate what they have learned into new situations. This requirement
excludes any top-down approach that does not involve frequent patches and updates.
Hybrid systems combine rule sets and learning capacities, and so fulfil this requirement
since they can adjust to new inputs and refine their moral behavior.
Teleology toward the good and empathy both face a sizable complication: they both require
some form of consciousness. For a machine to empathize with and understand emotions of
others, it must have emotion itself. Coeckelbergh (2010) claims that true emotion requires
consciousness and mental states in both cognitivist theory and feeling theory. Thus, if
robots do not have consciousness or mental states, they cannot have emotions and
therefore cannot have moral agency. Additionally, if a machine innately desires to do good,
it must have some form of inner thoughts or feeling that it is indeed doing good, so
teleology also requires consciousness or mental states. Much of human responsibility and
moral agency relies on this theory of mind. In court, the insanity or state of mind defense
can counter criminal charges.
However, no empirical way to test for state of mind or consciousness in people exists today.
Why require those immeasurable characteristics in our robots?
Emotionless Machinery
We interpret other humans’ behaviors as coming from or influenced by emotion, but we
have no way to truly determine emotional state. Verbal and nonverbal cues give us insights
to emotions others feel. They may imitate or fake those cues, but we interact with them just
the same as long as they maintain their deception (Goffman, 1956). We measure other
people by their display or performance of emotion.
Since the appearance of emotion in people regulates social interaction and human
morality, we must judge robots by that same appearance. Even today, machines can read
breathing and heart rate (Gent, 2016), and computers do not need to see an entire face to
determine emotion displayed (Wegrzyn, Vogt, Kireclioglu, Schneider, & Kissler, 2017). Soon
enough, a machine could learn to display human emotion by imitating the cues they’re
designed to measure. In theory, a robot could imitate or fake emotional cues as well as
humans display them naturally. People already tend to anthropomorphize robots,
empathize with them, and interpret their behavior as emotional (Turkle, 2011). For
consistency in the way we treat human display of emotion and interpret it as real, we must
also treat robotic display of emotion as real.
If the requirement for empathy changes from true emotion to functional emotion — as is
consistent with how we treat people — then an imitating robot fulfills all the requirements
for empathy, effectively avoiding the issue regarding consciousness and mental state.
Compassion could be the reason an autonomous car veers into a tree rather than a line of
children, but the appearance of compassion could also serve the same effect.
Additionally, a robot can have an artificial teleology towards good, granted that all of the
taught responses programmed into the machine are ‘good.’ Beavers’ (2011) discussion of
classical utilitarianism, referencing Mill (1979), claims that acting good is the same as being
good. The same applies to humans, as far as we can tell from the outside. Wallach and Allen
(2009) note that “values that emerge through the bottom-up development of a system
reflect the specific causal determinates of a system’s behavior”. In other words, a ‘good’ and
‘moral’ robot is one that takes moral and good actions. Thus, while we may not get true
teleology, functional teleology can suffice.
13. Human Rights and Artificial Intelligence--An Urgently Needed Agenda. Mathias Risse
Algorithms can do anything that can be coded, as long as they have access to data they
need, at the required speed, and are put into a design frame that allows for execution of
the tasks thus determined. In all these domains, progress has been enormous. The
effectiveness of algorithms is increasingly enhanced through “Big Data:” availability of
an enormous amount of data on all human activity and other processes in the world
which allow a particular type of AI known as “machine learning” to draw inferences
about what happens next by detecting patterns. Algorithms do better than humans
wherever tested, even though human biases are perpetuated in them: any system
designed by humans reflects human bias, and algorithms rely on data capturing the
past, thus automating the status quo if we fail to prevent them. 2 But algorithms are
noise-free: unlike human subjects, they arrive at the same decision on the same problem
when presented with it twice.
Also, philosophers have long puzzled about the nature of the mind. One question is if
there is more to the mind than the brain. Whatever else it is, the brain is also a complex
algorithm. But is the brain fully described thereby, or does that omit what makes us
distinct, namely, consciousness? Consciousness is the qualitative experience of being
somebody or something, its “what-it-is-like-to-be-that”-ness, as one might say. If there
is nothing more to the mind than the brain, then algorithms in the era of Big Data will
outdo us soon at almost everything we do: they make ever more accurate predictions
about what book we enjoy or where to vacation next; drive cars more safely than we do;
make predictions about health before our brains sound alarms; offer solid advice on
what jobs to accept, where to live, what kind of pet to adopt, if it is sensible for us to be
parents and whether it is wise to stay with the person we are currently with – based on a
myriad of data from people relevantly like us. Internet advertisement catering towards
our preferences by assessing what we have ordered or clicked on before is a mere
shadow of what is to come.
Future machines might be composed and networked in ways that no longer permit easy
switch-off. More importantly, they might display
emotions and behavior to express attachment: they might even worry about being
turned off, and be anxious to do something about it. Or future machines might be
cyborgs, partly composed of organic parts, while humans are modified with non-organic
parts for enhancement. Distinctions between humans and non-humans might erode.
Ideas about personhood might alter once it becomes possible to upload and store a
digitalized brain on a computer, much as nowadays we can store human embryos.
Already in 2007, a US colonel called off a robotic land-mine-sweeping exercise
because he considered the operation inhumane after a robot kept crawling along losing
legs one at a time. 5 Science fiction shows like Westworld or The Good Place anticipate
what it would be like to be surrounded by machines we can only recognize as such by
cutting them open. A humanoid robot named Sophia with capabilities to participate in
interviews, developed by Hanson Robotics, became a Saudi citizen in October 2017.
Later Sophia was named UNDP’s first-ever Innovation Champion, the first non-human
with a UN title.6 The future might remember these as historic moments. The pet world
is not far behind. Jeff Bezos recently adopted a dog called SpotMini, a versatile robotic
pet capable of opening doors, picking himself up and even loading the dishwasher. And
SpotMini never needs to go outside if Bezos would rather shop on Amazon or enjoy
presidential tweets.
If there indeed is more to the mind than the brain, dealing with AI including humanoid
robots would be easier. Consciousness, or perhaps accompanying possession of a
conscience, might then set us apart. It is a genuinely open question how to make sense
of qualitative experience and thus of consciousness. But even though considerations
about consciousness might contradict the view that AI systems are moral agents, they
will not make it impossible for such systems to be legal actors and as such own property,
commit crimes and be accountable in legally enforceable ways. After all, we have a
history of treating corporations in such ways, which also do not have consciousness.
Perhaps T. M. Scanlon’s ideas about appropriate responses to values would help.10 The
superintelligence might be “moral” in the sense of reacting in appropriate ways towards
what it observes all around. Perhaps then we have some chance at getting protection, or
even some level of emancipation in a mixed society composed of humans and machines,
given that the abilities of the human brain are truly astounding and generate capacities
in human beings that arguably should be worthy of respect.11 But so are also the
capacities of animals, which has not normally led humans to react towards them, or
towards the environment, in an appropriately respectful way. Instead of displaying
something like an enlightened anthropocentrism, we have too often instrumentalized
nature. Hopefully a superintelligence would simply outperform us in such matters, and
that will mean the distinctively human life will receive some protection because it is
worthy of respect. We cannot know that for sure but we also need not be pessimistic.

There is an urgency to making sure these developments get off to a good start. The
pertinent challenge is the problem of value alignment, a challenge that arises way
before it will ever matter what the morality of pure intelligence is. No matter how
precisely AI systems are generated we must try to make sure their values are aligned
with ours to render as unlikely as possible any complications from the fact that a
superintelligence might have value commitments very different from ours. That the
problem of value alignment needs to be tackled now is also implied by the UN Guiding
Principles on Business and Human Rights, created to integrate human rights into
business decisions. These principles apply to AI. This means addressing questions such
as "What are the most severe potential impacts?", "Who are the most vulnerable
groups?" and "How can we ensure access to remedy?"
However, these laws have long been regarded as too unspecific. Various efforts have
been made to replace them, so far without any connection to the UN’s Principles on
Business and Human Rights or any other part of the human-rights movement. Among
other efforts, in 2017 the Future of Life Institute in Cambridge, MA founded around
MIT physicist Max Tegmark and Skype co-founder Jaan Tallinn, held a conference on
Beneficial AI at the Asilomar conference center in California to come up with principles
to guide further development of AI. Of the resulting 23 Asilomar Principles, 13 are listed
under the heading of Ethics and Values. Among other issues, these principles insist that
wherever AI causes harm, it should be ascertainable why it does, and where an AI
system is involved in judicial decision making its reasoning should be verifiable by
human auditors. Such principles respond to concerns that AI deploying machine
learning might reason at such speed and have access to such a range of data that its
decisions are increasingly opaque, making it impossible to spot if its analyses go astray.
The principles also insist on value alignment, urging that “highly autonomous AI
systems should be designed so that their goals and behaviors can be assured to align
with human values throughout their operation” (Principle 10). The ideas explicitly
appear in Principle 11 (Human Values) include “human dignity, rights, freedoms, and
cultural diversity.”
Russian manipulation in elections is a wake-up call; much worse is likely to come. Judicial
rights could be threatened if AI is used without sufficient transparency and possibility for
human scrutiny. An AI system has predicted the outcomes of hundreds of cases at the
European Court of Human Rights, forecasting verdicts with accuracy of 79%; and once
that accuracy gets yet higher it will be tempting to use AI also to reach decisions. Use of
AI in court proceedings might help generate access to legal advice to the poor (one of the
projects Amnesty pursues, especially in India); but it might also lead to Kafkaesque
situations if algorithms give inscrutable advice.
Any rights to security and privacy are potentially undermined not only through drones
or robot soldiers, but also through increasing legibility and traceability of individuals in
a world of electronically recorded human activities and presences. The amount of data
available about people will likely increase enormously, especially once biometric sensors
can monitor human health. (They might check up on us in the shower and submit their
data, and this might well be in our best interest because illness becomes diagnosable
way before it becomes a problem.) There will be challenges to civil and political rights
arising from the sheer existence of these data and from the fact that these data might
well be privately owned, but not by those whose data they are. Leading companies in
the AI sector are more powerful than oil companies ever were, and this is presumably
just the beginning of their ascension.
The Cambridge-Analytica scandal is a wake-up call here, and
Mark Zuckerberg’s testimony to US senators on April 10, 2018 revealed an astonishing
extent of ignorance among senior lawmakers about the workings of internet companies
whose business model depends on marketing data. Such ignorance paves the path to
power for companies. Or consider a related point: Governments need the private sector
to aid in cyber security. The relevant experts are smart, expensive, and many would
never work for government. We can only hope that it will be possible to co-opt them
given that government is overextended here. If such efforts fail, only companies will
provide the highest level of cyber security.
This takes me to my last topic: AI and inequality, and the connection between that topic
and human rights. To begin with, we should heed Thomas Piketty’s warning that
capitalism left to its own devices in times of peace generates ever increasing economic
inequality. Those who own the economy benefit from it more than those who just work
there. Over time life chances will ever more depend on social status at birth. We also
see more and more how those who either produce technology or know how to use
technology to magnify impact can command higher and higher wages. AI will only
reinforce these tendencies, making it ever easier for leaders across all segments to
magnify their impact. That in turn makes producers of AI ever more highly priced
providers of technology. More recently, we have learned from Walter Scheidel that,
historically, substantial decreases in inequality have only occurred in response to
calamities such as epidemics, social breakdowns, natural disasters or war. Otherwise it
is hard to muster effective political will for change.
Before this background we must worry AI will drive a widening technological wedge into
societies that leaves millions excluded, renders them redundant as market participants
and thus might well undermine the point of their membership in political community.
When wealth was determined by land ownership, the rich needed the rest because the
point of land ownership was to charge rent. When wealth was determined by ownership
of factories the owners needed the rest to work the machines and buy stuff. But those
on the losing side of the technological divide may no longer be needed at all. In his 1926
short story “The Rich Boy,” F. Scott Fitzgerald famously wrote, “Let me tell you about
the very rich. They are different from you and me.” AI might validate that statement in a
striking way.
14. From Machine Ethics To Machine Explainability and Back∗Kevin Baum, Holger
Hermanns and Timo Speith. Saarland University, Department of Philosophy
An important question arises: How should machines be constrained, such that they act
morally acceptable towards humans? This question concerns Machine Ethics – the search
for formal, unambiguous, algorithmizable and implementable behavioral constraints for
systems, so as to enable them to exhibit morally acceptable behavior.
We instead feel the need to supplement Machine Ethics with means to ascertain justified
trust in autonomous systems – and other desirable properties. After pointing out why this is
important, we will argue that there is one feasible supplement for Machine Ethics: Machine
Explainability – the ability of an autonomous system to explain its actions and to argue for
them in a way comprehensible for humans. So Machine Ethics needs Machine Explainability.
This also holds vice versa: Machine Explainability needs Machine Ethics, as it is in need of a
moral system as a basis for generating explanations.
15. The Ethics of Artificial Intelligence--Nick Bostrom, Future of Humanity Institute,
Eliezer Yudkowsky, Machine Intelligence Research Institute
AI algorithms play an increasingly large role in modern society, though usually not
labeled “AI.” The scenario described above might be transpiring even as we write. It
will become increasingly important to develop AI algorithms that are not just powerful
and scalable, but also transparent to inspection—to name one of many socially important
properties. Some challenges of machine ethics are much like many other challenges
involved in designing machines. Designing a robot arm to avoid crushing stray humans is no
more morally fraught than designing a flame-retardant sofa. It involves new programming
challenges, but no new ethical challenges. But when AI algorithms take on cognitive work
with social dimensions-cognitive tasks previously performed by humans—the AI algorithm
inherits the social requirements.
Transparency is not the only desirable feature of AI. It is also important that AI algorithms
taking over social functions be predictable to those they govern.
It will also become increasingly important that AI algorithms be robust against manipulation.
Robustness against manipulation is an ordinary criterion in information security; nearly the
criterion. But it is not a criterion that appears often in machine learning journals, which are
currently more interested in, e.g., how an algorithm scale up on larger parallel systems.
Responsibility, transparency, auditability, incorruptibility, predictability, and a tendency to
not make innocent victims scream with helpless frustration: all criteria that apply to humans
performing social functions; all criteria that must be considered in an algorithm intended to
replace human judgment of social functions; all criteria that may not appear in a journal of
machine learning considering how an algorithm scales up to more computers. This list of
criteria is by no means exhaustive, but it serves as a small sample of what an increasingly
computerized society should be thinking about.
Artificial General Intelligence (AGI)-- As the name implies, the emerging consensus is that
the missing characteristic is generality. Current AI algorithms with human-equivalent or
superior performance are characterized by a deliberately programmed competence only in
a single, restricted domain. Deep Blue became the world champion at chess, but it cannot
even play checkers, let alone drive a car or make a scientific discovery. Such modern AI
algorithms resemble all biological life with the sole exception of Homo sapiens.
To build an AI that acts safely while acting in many domains, with many consequences,
including problems the engineers never explicitly envisioned, one must specify good
behavior in such terms as “X such that the consequence of X is not harmful to humans.” This
is non-local; it involves extrapolating the distant consequences of actions.
A rock has no moral status: we may crush it, pulverize it, or subject it to any treatment we
like without any concern for the rock itself. A human person, on the other hand, must be
treated not only as a means but also as an end. Exactly what it means to treat a person as an
end is something about which different ethical theories disagree; but it certainly involves
taking her legitimate interests into account—giving weight to her well-being—and it may
also involve accepting strict moral side-constraints in our dealings with her, such as a
prohibition against murdering her, stealing from her, or doing a variety of other things to
her or her property without her consent. Moreover, it is because a human person counts in
her own right, and for her sake, that it is impermissible to do to her these things. This can be
expressed more concisely by saying that a human person has moral status.
It is widely agreed that current AI systems have no moral status. We may change, copy,
terminate, delete, or use computer programs as we please; at least as far as the programs
themselves are concerned. The moral constraints to which we are subject in our dealings
with contemporary AI systems are all grounded in our responsibilities to other beings, such
as our fellow humans, not in any duties to the systems themselves.
While it is fairly consensual that present-day AI systems lack moral status, it is unclear
exactly what attributes ground moral status. Two criteria are commonly proposed as being
importantly linked to moral status, either separately or in combination: sentience and
sapience (or personhood). These may be characterized roughly as follows:
Sentience: the capacity for phenomenal experience or qualia, such as the capacity to feel
pain and suffer
Sapience: a set of capacities associated with higher intelligence, such as self- awareness and
being a reason-responsive agent.
Others propose additional ways in which an object could qualify as a bearer of moral status,
such as by being a member of a kind that normally has sentience or sapience, or by standing
in a suitable relation to some being that independently has moral status.
Principle of Substrate Non-Discrimination: If two beings have the same functionality and
the same conscious experience and differ only in the substrate of their implementation,
then they have the same moral status.
Principle of Ontogeny Non-Discrimination: If two beings have the same functionality and
the same consciousness experience, and differ only in how they came into existence, then
they have the same moral status.
Parents have special duties to their child which they do not have to other children, and
which they would not have even if there were another child qualitatively identical to their
own. Similarly, the Principle of Ontogeny Non-Discrimination is consistent with the claim
that the creators or owners of an AI system with moral status may have special duties to
their artificial mind which they do not have to another artificial mind, even if the minds in
question are qualitatively similar and have the same moral status.
Even if we accept this stance, however, we must confront a number of novel ethical
questions which the aforementioned principles leave unanswered. Novel ethical questions
arise because artificial minds can have very different properties from ordinary human or
animal minds. We must consider how these novel properties would affect the moral status
of artificial minds and what it would mean to respect the moral status of such exotic minds.
a. Does a sapient but non-sentiet robot (zombie) have the same moral ststus as a full
AMA?
b. Another exotic property, one which is certainly metaphysically and physically
possible for an artificial intelligence, is for its subjective rate of time to deviate
drastically from the rate that is characteristic of a biological human brain. The
concept of subjective rate of time is best explained by first introducing the idea of
 whole brain emulation, or “uploading.” “Uploading” refers to a hypothetical future
technology that would enable a human or other animal intellect to be transferred
from its original implementation in an organic brain onto a digital computer.
Principle of Subjective Rate of Time: In cases where the duration of an experience is
of basic normative significance, it is the experience’s subjective duration that counts.
c. For example, human children are the product of recombination of the genetic
material from two parents; parents have limited ability to influence the character of
their offspring; a human embryo needs to be gestated in the womb for nine months;
it takes fifteen to twenty years for a human child to reach maturity; a human child
does not inherit the skills and knowledge acquired by its parents; human beings
possess a complex evolved set of emotional adaptations related to reproduction,
nurturing, and the child-parent relationship. None of these empirical conditions need
pertain in the context of a reproducing machine intelligence. It is therefore plausible
that many of the mid-level moral principles that we have come to accept as norms
governing human reproduction will need to be rethought in the context of AI
reproduction.
To illustrate why some of our moral norms need to be rethought in the context of AI
reproduction, it will suffice to consider just one exotic property of AIs: their capacity
for rapid reproduction. Given access to computer hardware, an AI could duplicate
itself very quickly, in no more time than it takes to make a copy of the AI’s software.
Moreover, since the AI copy would be identical to the original, it would be born
completely mature, and the copy could begin making its own copies immediately.
Absent hardware limitations, a population of AIs could therefore grow exponentially
at an extremely rapid rate, with a doubling time on the order of minutes or hours
rather than decades or centuries.
But if the population grows faster than the economy, resources will run
out; at which point uploads will either die or their ability to reproduce will be
curtailed.
This scenario illustrates how some mid-level ethical principles that are suitable in
contemporary societies might need to be modified if those societies were to include
persons with the exotic property of being able to reproduce very rapidly.
The general point here is that when thinking about applied ethics for contexts that
are very different from our familiar human condition, we must be careful not to
mistake mid-level ethical principles for foundational normative truths. Put
differently, we must recognize the extent to which our ordinary normative precepts
are implicitly conditioned on the obtaining of various empirical conditions, and the
need to adjust these precepts accordingly when applying them to hypothetical
futuristic cases in which their preconditions are assumed not to obtain. By this, we
are not making any controversial claim about moral relativism, but merely
highlighting the commonsensical point that context is relevant to the application of
ethics—and suggesting that this point is especially pertinent when one is considering
the ethics of minds with exotic properties.
Superintelligence
 Good (1965) set forth the classic hypothesis concerning superintelligence: that an AI
sufficiently intelligent to understand its own design could redesign itself or create a
successor system, more intelligent, which could then redesign itself yet again to
become even more intelligent, and so on in a positive feedback cycle. Good called
this the “intelligence explosion.”
Kurzweil (2005) holds that “intelligence is inherently impossible to control,” and that
despite any human attempts at taking precautions, by definition . . . intelligent
entities have the cleverness to easily overcome such barriers.” Let us suppose that
the AI is not only clever, but that, as part of the process of improving its own
intelligence, it has unhindered access to its own source code: it can rewrite itself to
anything it wants itself to be. Yet it does not follow that the AI must want to rewrite
itself to a hostile form.
Humans, the first general intelligences to exist on Earth, have used that intelligence
to substantially reshape the globe—carving mountains, taming rivers, building
skyscrapers, farming deserts, producing unintended planetary climate changes. A
more powerful intelligence could have correspondingly larger consequences.
This presents us with perhaps the ultimate challenge of machine ethics: How do
you build an AI which, when it executes, becomes more ethical than you? If we are
serious about developing advanced AI, this is a challenge that we must
meet. If machines are to be placed in a position of being stronger, faster, more
trusted, or smarter than humans, then the discipline of machine ethics must commit
itself to seeking human-superior (not just human-equivalent) niceness.

16. Towards the Ethical Robot by James Gips, Computer Science Department, Fulton
Hall 460 Boston College, Chestnut Hill, MA 02467
Asimov’s three laws are not suitable for our magnificent robots. These are laws for slaves.
We want our robots to behave more like equals, more like ethical people. (See Figure
1) How do we program a robot to behave ethically? Well, what does it mean for a
person to behave ethically?
2) On what type of ethical theory can automated ethical reasoning be based? At first
glance, consequentialist theories might seem the most "scientific", the most
amenable to implementation in a robot. Maybe so, but there is a tremendous
problem of measurement. How can one predict "pleasure", "happiness", or "well-
being" in individuals in a way that is additive, or even comparable?
3) Deontological theories seem to offer more hope. The categorical imperative might
be tough to implement in a reasoning system. But I think one could see using a moral
system like the one proposed by Gert as the basis for an automated ethical
reasoning system. A difficult problem is in the resolution of conflicting obligations.
Gert's impartial rational person advocating that violating the rule in these
circumstances be publicly allowed seems reasonable but tough to implement.
4) The virtue-based approach to ethics, especially that of Aristotle, seems to resonate
well with the modern connectionist approach to AI. Both seem to emphasize the
 immediate, the perceptual, the non-symbolic. Both emphasize development by
training rather than by the teaching of abstract theory.
5) Knuth [1973, p.709] put it well: It has often been said that a person doesn't really
understand something until he teaches it to someone else. Actually a person doesn't
really understand something until he can teach it to a computer, i.e., express it as an
algorithm. ... The attempt to formalize things as algorithms leads to a much deeper
understanding than if we simply try to understand things in the traditional way. That
as we build the artificial ethical reasoning systems we will learn how to behave more
ethically ourselves.
17. Can robots be responsible moral agents? And why should we care? Amanda
Sharkey, Department of Computer Science, University of Sheffield, Sheffield, UK
Patricia Churchland (2011) discusses the basis for morality in living beings, and argues that
the basis for caring about others lies in the neurochemistry of attachment and bonding in
mammals. She explains that it is grounded in the extension of self-maintenance and
avoidance of pain in mammals to their immediate kin. Neuropeptics, oxytocin and arginine
vasopressin underlie mammals’ extension of self-maintenance and avoidance of pain to
their immediate kin. Humans and other mammals feel anxious about their own well-being
and that of those to whom they are attached. As well as attachment and empathy for
others, humans and other mammals develop more complex social relationships, and are
able to understand and predict the actions of others. They also internalise social practices,
and experience ‘social pain’ triggered by separation, exclusion or disapproval. As a
consequence, humans have an intrinsic sense of justice.
By contrast, robots are not concerned about their own self-preservation or avoidance of
pain, let alone the pain of others. In part, this can be explained by means of arguing that
they are not truly embodied, in the way that a living creature is. Parts of a robot could be
removed from a robot’s body without it suffering any pain or anxiety, let alone it being
concerned about damage or pain to a family member or to a human. A living body is an
integrated autopoeietic entity (Maturana and Varela, 1980) in a way that a man-made
machine is not. Of course, it can be argued that the robot could be programmed to behave
as if it cared about its own preservation or that of others, but this is only possible through
human intervention.
18. Can machines be people? Reflections on the Turing Triage Test, Dr Rob Sparrow,
School of Philosophical, Historical & International Studies, Monash University.
Finally, imagine that you are again called to make a difficult decision. The battery system
powering the AI is failing and the AI is drawing on the diminished power available to the
rest of the hospital. In doing so, it is jeopardising the life of the remaining patient on life
support. You must decide whether to ̳switch off‘ the AI in order to preserve the life of
the patient on life support. Switching off the AI in these circumstances will have the
unfortunate consequence of fusing its circuit boards, rendering it permanently inoperable.
Alternatively, you could turn off the power to the patient‘s life support in order to allow
the AI to continue to exist. If you do not make this decision the patient will die and the
AI will also cease to exist. The AI is begging you to consider its interests, pleading to be
allowed to draw more power in order to be able to continue to exist.
My thesis, then, is that machines will have achieved the moral status of persons when
this second choice has the same character as the first one. That is, when it is a moral
dilemma of roughly the same difficulty. For the second decision to be a dilemma it must
be that there are good grounds for making it either way. It must be the case therefore that
it is sometimes legitimate to choose to preserve the existence of the machine over the life
of the human being. (He may choose the robot because it is more useful to the hospital than
the human patient. This test still doesn’t make the robot a moral person. Only sapience and
sentience will make a robot a moral person,)
19. Can a Robot Pursue the Good? Exploring Artificial Moral Agency, Amy Michelle
DeBaets, Kansas City University of Medicine and Biosciences, Journal of Evolution
and Technology - Vol. 24 Issue 3 – Sept 2014 – pgs 76-86
What then, might be necessary for a decision-making and acting entity to non-accidentally
pursue the good in a given situation? I argue that four basic components collectively make
up the basic requirements for moral agency: embodiment, learning, empathy, and
teleology.
First, I want to argue that artificial moral agents, like all moral agents, must have some form
of embodiment, as they must have a real impact in the physical world (and not solely a
virtual one) if they are to behave morally. Embodiment not only allows for a concrete
presence from which to act, it can adapt and respond to the consequences of real decisions
in the world. This physical embodiment, however, need not look particularly similar to
human embodiment and action. Robotic embodiment might be localized, having actions
take place in the same location as the decision center, in a discrete, mobile entity (as with
humans), but it might also be remote, where the decision center and locus of action are
distant in space. It could also be distributed, where the decision centers and/or loci of action
take place in. (Not convincing—a server is also an entity and is embodied).
This embodied decision-making and action must also exist in a context of learning. Learning,
in this sense, is not simply the incorporation of new information into a system or the
collection of data. It is adapting both the decision processes themselves and the agent’s
responses to inputs based on previous information. It is this adaptability that allows moral
agents to learn from mistakes as well as successes, to develop and hone moral reasoning,
and to incorporate new factual information about the circumstances of decisions to be
made. several places at once, as with distributed computing or multiple simultaneous
centers of action. The unifying theme of embodiment does require that a particular
decision-making entity be intricately linked to particular concrete action; morality cannot
solely be virtual if it is to be real.
Even if an embodied robot can learn from its own prior actions, it is not necessarily moral.
The complex quality of empathy is still needed for several reasons. First, empathy allows the
agent to recognize when it has encountered another agent, or an appropriate object of
moral reasoning. It allows the A.M.A. to understand the potential needs and desires of
another, as well as what might cause harm to the other. This requires at least a rudimentary
theory of mind, that is, a recognition that another entity exists with its own thoughts,
beliefs, values, and needs. This theory of mind need not take an extremely complex form,
but for an agent to behave morally, it cannot simply act as though it is the only entity that
matters. The moral agent must be able to develop a moral valuation of other entities,
whether human, animal, or artificial. It may have actuators and sensors that give it the
capacity to measure physical inputs from body language, stress signs, and tone of voice, to
indicate whether another entity is in need of assistance and behave morally in accordance
with the needs it measures. It may respond to cries for help, but it needs to be able to
distinguish between a magazine rack and a toddler in rushing in to provide aid. Empathy,
and not merely rationality, is critical for developing and evaluating moral choices; just as
emotion is inherent to human rationality, it is necessary for machine morality. (This is the
ethics of care logic—only emotions lead to action, to empathy)
What is sometimes forgotten in defining a moral agent as such, including in the case of
A.M.A.s, is that the entity must both be designed to be, and desire to be, moral. It must
have a teleology toward the good. Just as human beings have developed a sense of the
moral and often seek to act accordingly, machines could be designed to pursue the good,
even develop a form of virtue through trial and error. They will not, however, do so in the
absence of some design in that direction. A teleology of morality introduced into the basic
programming of a robot would not necessarily be limited to any one particular ethical
theory or set of practices and could be designed to incorporate complex interactions of
decisions and consequences, just as humans typically do when making decisions about what
is right. It could be programmed, in its more advances forms, to seek out the good, to
develop “virtual virtue,” learning from what it has been taught and practicing ever-greater
forms of the good in response to what it learns.
What is Not Required for Artificial Moral Agency?
Popular futurists Ray Kurzweil and Hans Moravec have argued that sheer increases in
computational processing power will eventually lead to superhuman intelligence, and thus,
to agency. But this is not the case. While a certain amount of “intelligence” or processing
power is necessary, it is only functionally useful insofar as it facilitates learning and
empathy, particularly. Having the most processing power does not make one the most
thoughtful agent, and having the most intelligence does not make one particularly moral on
its own.
Likewise, while a certain amount of rule-following is probably necessary for artificial moral
agency, rule-following alone does not make for a moral agent, but rather for a slave to
programming. Moral agency requires being able to make decisions and act when the basic
rules conflict with each other; it also requires being able to set aside “the rules” entirely
when the situation dictates. It has been said that one cannot truly be good unless one has
the freedom to choose not to be good. While I do not want to take on that claim here, I will
argue that agency requires at least some option of which goods to pursue and what
methods to pursue them by. A supposed A.M.A. that only follows the rules, and breaks
down when they come into conflict, is not a moral agent at all.
While a machine must move beyond simple rule-following to be a genuine moral agent
(even if many of its ends and goals are predetermined in its programming), complete
freedom is not necessary in order to have moral agency.
Some have thought that a fully humanoid consciousness is necessary for the development
of moral agency, but this too, may legitimately look quite different in machines than it does
in human beings.
Consciousness is itself elusory, without a clear definition or understanding of its processes.
What can be said for moral agency, though, is that the proof is in the pudding, that decisions
and actions matter at least as much as the background processing that went into them. In
deciding to consistently behave morally, and in learning from behavior in order to become
more moral, a machine can be a moral agent in a very real sense while avoiding the problem
of consciousness entirely.
Just as consciousness is used primarily as a requirement that cannot, by definition, be met
by any entity other than a human moral agent, so the idea of an immaterial soul need not
be present in order to have a moral agent. While the idea of a soul may or may not be useful
when applied in the context of human beings in relation to the Divine, it is unnecessary for
the more limited question of moral agency. A being also need not have a sense of God in
order to be a moral being. Not only is this true in the case of many humans, who may be
atheists, agnostics, or belong to spiritual traditions that do not depend on the idea of a
deity, but it is not necessary for moral action and the development of virtue. It may be
practically helpful in some cases for a robot to believe in a deity in order to encourage its
moral action, but it is by no means a requirement.
Yet, while the robots we build will not be subject to many of the same temptations as
human moral agents, they will still be subject to the limitations of their human designers
and developers. Robots will not be morally perfect, just as humans, even in the best of
circumstances, are never morally perfect.

20, Toward understanding the impact of artificial intelligence on

labor Morgan R. Franka, David Autorb, James E. Bessenc, Erik
Brynjolfssond,e, Manuel Cebriana, David J. Demingf,g, Maryann
Feldmanh, Matthew Groha, Jos´e Loboi, Esteban Moroa,j, Dashun
Wangk,l, Hyejin Younk,l, and Iyad Rahwana,m,n,1
Edited by Jose A. Scheinkman, Columbia University, New York, NY,
and approved February 28, 2019 (received for review January 18,
2019)
 Wassily Leontief, winner of the 1973 Nobel Prize in Economics,
noted in 1952, “Labor will become less and less important. . .
More workers will be replaced by machines. I do not see that
new industries can employ everybody who wants a job”.
 In general, depending on the nature of the job, a worker may
be augmented by technology or in competition with. For
example, technological advancements in robotics can diminish
wages and employment opportunities for manufacturing
workers. However, technological change does not necessarily
produce unemployment, and, in the case of AI, cognitive
technology may actually augment workers. For instance,
machine learning appears to bolster the productivity of
software developers while also creating new investment and
manufacturing opportunities (e.g., autonomous vehicles).
Complicating matters further, the skill requirements of
occupations do not remain static, but instead change with
changing technology.
 Even if technology depresses employment for some types of
labor, it can create new needs and new opportunities through
“creative destruction”. For instance, the replacement of
equestrian travel with automobiles spurred demand for new
roadside amenities, such as motels, gas stations, and fast food.
 While skill types inform the theory of labor and technological
change (1, 18, 21, 49), standard labor data focus on aggregate
statistics, such as wage and employment numbers, and can lack
resolution into the specifics that distinguish different job titles
and different types of work. For example, previous studies have
empirically observed a “hollowing” of the middle-skill jobs
described by increasing employment share for low-skill and
 high-skill occupations at the expense of middle-skill
occupations (16, 35) (reproduced in Fig. 1A). These studies use
skills to explain labor trends but are limited empirically to
measuring annual wages instead of skill content directly. While
wages may correlate with specific skills, wage alone fails to
capture the defining features of an occupation, and models
focused on only cognitive and physical labor fail to explain
responses to technological change (21).
 As an example, consider that civil engineers and medical
doctors are both high-wage, cognitive, non-routine occupations
requiring many years of higher education and additional
professional certification. However, these occupations require
distinct workplace skills that are largely non-transferable, and
these occupations are likely to interact with different
technologies. Wages and education— and even aggregations of
workplace skills—may be too coarse to distinguish occupations
and, thus, may obfuscate the differential impact of various
technologies and complicate predictions of changing skill
requirements. In turn, these shortcomings may help explain the
variability in current automation predictions that enable
contrasting perspectives.
 However, researchers and policy makers are underequipped to
forecast the labor trends resulting from specific cognitive
technologies, such as AI.