The Ethics of Human

Dignity
 What is Human Personhood?
• Immanence : Whereby we are embodied spirits, ends-in-themselves,
with a human and divine destiny.

• Individuality: Unique, non-repeatable, irreducible and irreplaceable

• Sociality: We do not live, move and have our being in isolation

• Transcendence: We have a soul that is immortal, thereby connecting

us to an immortal “Other”
St. Augustine – “….. that principle within us
by which we are like God, made in the image
of God.”
Plato – “……that when a person has
died, his soul exists.”
 What is Human Dignity?
• There is no single definition of human dignity as the term is “abstract
and highly ambiguous” (Kass 2008: 306; Fukuyama 2002:148). A
working definition, however, would be:

• “The dignity of a person is that whereby a person excels other

beings, especially other animals, and merits respect and
consideration from other persons,” (Lee & George 2008:410).

• HD is something special about human nature that confers on us a

“moral status” that makes us superior to other animals but “equal”
among all humans (Lee & George 2008: 415).
The Silent Scream
Child labour
The Atlantic slave trade
Forced to fish: Slavery on Thailand's trawlers
“ The postulate that personhood is a distinctly
human state within the natural order is
basically an assertion of human
exceptionalism. ”
“ Humanity as characterized by morality and
personhood requires no divine principle, nor
Imago Dei, but only the relentless force of
natural selection”
- Charles Darwin
 The Fourth Industrial revolution (4IR)
• The First Industrial Revolution-1750-1850- Steam Power to mechanize production.

• The Second Industrial Revolution-1850-1950-Electric Power and Assembly-line Production.

• The Third Industrial Revolution- 1950-1980-Electronics and the Internet to revolutionise

Communication and Flatten the World.

• The Fourth Industrial Revolution-1980 and ongoing- NBIC-leading to exponential change in

every field of human existence
 The Age of Transhumanism

The Ethics of Human Enhancement

 What is Human Enhancement
• Making something better than it was before by the technological,
genetic, or chemical improvements to the ‘species-specific’ normal of
healthy human beings. Enhancement is therefore distinct from
therapy, which would involve making some “abnormality” more
“normal”.

• The enhancement options being discussed include radical extension of human health-span,
eradication of disease, elimination of unnecessary suffering, and augmentation of human
intellectual, physical, and emotional capacities
 What Are The Human Enhancements So Far?
• The Bionic Man-Jesse Sullivan
• Braingate--allows a person to manipulate objects in the world using only the
mind
• Cochlear Implants and Night Vision and Silent Talk
• Affective BCIs: Electrocorticography (ECoG) and Electroencephalography (EEG)
• Exoskeletons and Flexible Battlesuits-MIT’s Soldier Nanotechnologies
• Reciprocyte-an artificial nano-red blood vessel
• Pharmacological Enhancements. Stimulant drugs-Ritalin and Adderall, used by
many college students to boost concentration and ward off sleep; Provigil, used
to improve working memory and brighten mood; Anabolic steroids ; Viagra;
Aricept-improves verbal and visual memory; Resvestrol– life extender.
What Are The Human Enhancements So Far?
• Hans Moravec, former director of robotics at Carnegie-Mellon University and
developer of advanced robots for both NASA and the military, popularized the
idea of living perpetually via a digital substrate.

• He envisioned a procedure in which the entirety of the information encoded

within the neurons of a human brain could be read, copied, and uploaded to a
computer
• Immortality through software existence.

• Embodied Cognition is the opposite of brain emulation. That the body is an

extension of the mind and helps the mind to think and recognize and decide.
What is Human Enhancement
 The Ethics of Human Enhancement
• Ethical Issues of Affective Brain–Computer Interfaces: a system that uses
neurophysiological signals to extract features that are related to affective states
(e.g. emotions and moods). Data protection and informed consent,
neurohacking, marketing and political manipulation, inauthentic\fake emotions
• Exacerbated Social Inequality
• Exacerbated Corporate Inequality at all Managerial Levels
• The Ethics of Autonomy , Choice and Social Life of the first Transhumans
• The Ethics of the Emaciated Family
• The Ethics of the Imbalanced Transhuman
• The Geo-Ethics of the Aryan Race
• Decease-free longevity for the privileged
• Superintelligence for the enhanced
Artificial Intelligence
Basic Understandings
 The Ethics of Artificial
Intelligence
• Issac Asimov’s Three Laws of Robotics:
• A robot may not injure a human being or, through inaction, allow a human
being to come to harm.
• A robot must obey the orders given it by human beings except where such
orders would conflict with the First Law.
• A robot must protect its own existence as long as such protection does not
conflict with the First or Second Laws.
 A Brief History of AI
• Algorithms and Machine Learning:
• Artificial intelligence is based on the assumption that the process of human thought
can be mechanized.
• In 1951, Marvin Minsky with Dean Edmonds, built the first neural net machine, the
SNARC, Stochastic Neural Analog Reinforcement Calculator, that started to mimic the
human brain.
• In 1955, Allen Newell and Herbert A. Simon created the "Logic Theorist“ that solved
the venerable mystery of mind\body existence. Was the mind an ethereal substance
that was not made of matter? These people proved that the mind was a replicable
neural network that worked on chemistry and electricity, whereas theirs worked on
mechanical parts, electricity and algorithms!
• The Turing Test: If a machine could carry on a conversation (over a teleprinter) that
was indistinguishable from a conversation with a human being, then it was
reasonable to say that the machine was "thinking".
 What is Big Data and Strong AI?
• Big data refers to a collection of data that cannot be captured,
managed, and processed by conventional software tools within a
certain time frame.
• Big data means that instead of random analysis (sample survey), all
data is used for analysis!
• General intelligence is the ability to solve any problem, rather than
finding a solution to a particular problem. Artificial general
intelligence (or "AGI") is a program which can apply intelligence to a
wide variety of problems, in much the same way as humans can. Also
referred to as "strong AI“.
• “Strong AI” is predicted to become reality in 2045!!
 What is this AI Revolution?
• It is the programmed agglomeration of algorithms that enable this
intelligence, embodied or disembodied, to analyse Big Data at super
speeds that the unenhanced human brain cannot and arrive at
correct and safe conclusions to make decisions.
• The fuel is Big Data and the technology is Machine learning
• AI can be with “man-in-the-loop”; “man-on-the loop” or “completely
independent”
• When AI becomes recursive and learns to create its own algorithms
and becomes independent and goes beyond human intelligence and
control, the point of “Singularity” would have arrived.
 The Value Alignment and Control Problems
• How do you ensure that the values of AMAs are aligned to that of Human Beings?
• Due to the inherent autonomy of these systems, the ethical considerations have
to be conducted by themselves. This means, that these autonomous cognitive
machines are in need of a theory, with the help of which they can, in a specific
situation, choose the action that adheres best to the moral standards.
• Which Ethical Tradition between Deontology, Utilitarianism and Virtue Ethics is
currently favoured and why?
• Deontology has a serious problem when it comes to ethical dilemmas. To lie to
save a life..not allowed in Deontology. How do you algorithimize this in an AMA?
• There is no room for learning in Deontology—the imperatives are categorical.
How do you assess what is “good” for you, let alone for others?
• Utilitarianism as an ethical theory for AMAs fail again in the hedonistic
calculations. The time available to do this calculation and the act is so limited.
• The calculation becomes even more complicated when fecundity and propinquity
have to be considered.
 The Value Alignment Problem
• Why are AI scientists veering towards Virtue Ethics?
• Because Machine learning is the improvement of a machine’s
performance of a task through experience and Aristotle’s virtue ethics is
the improvement of one’s virtues through experience.
Aristotle’s Soul Theory
VIRTUES OF REASON VIRTUES OF CHARACTER
Craftmanship= Techne
Science=episteme
Manufacturing = Poesis
Wisdom=Sophia
Practical Wisdom = Phronesis
Intuitive Thought= Nous

VIRTUES OF PURE REASON VIRTUES OF PRACTICAL REASON

Practical Life = Practical Wisdom and
Theoretical Life = Weak AI Morality = Strong AI
 The Value Alignment Problem
• Whatever be the Ethical Tradition employed, the AMA has to be able
to explain to humans, their logic for arriving at their decisions.
• There is a need for an “Explainability” algorithm, that is inputted
parallelly into the AMA. Without “explainability” there can be no legal
responsibility, like some one claiming insanity.
• While AMAs based on deontology can point towards the principles
and duties which have guided their actions, a consequentialist AMA
can explain why its actions have led to the best consequences.
• An AMA based on virtue ethics on the other hand would have to
show how its virtues, which gave rise to its actions, have been formed
through experience. A tough call for Machine Learning.
 The Value Alignment Problem
• AMAs based on Virtue Ethics solves the two major challenges of contemporary AI
safety research, the control problem and the value alignment problem.

• A machine endowed with the virtue of temperance would not have any desire for
excess of any kind, not even for exponential self-improvement, which might lead
to a superintelligence posing an existential risk for humanity. Since virtues are an
integral part of one’s character, the AI would not have the desire of changing its
virtue of temperance.
 Should We Allow AGIs?—The Control Problem
• True AGIs will be capable of universal problem solving and recursive self-improvement.

• Consequently, they have potential of outcompeting humans in any domain essentially making humankind
unnecessary and so subject to extinction.

• Kurzweil holds that “intelligence is inherently impossible to control,” and that despite any human attempts at
taking precautions, by definition . . . intelligent entities have the cleverness to easily overcome such barriers.”

• This presents us with perhaps the ultimate challenge of machine ethics: How do you build an AI which, when it
executes, becomes more ethical than you?

• “AI Safety Engineering” field emerging: A common theme in AI safety research is the possibility of keeping a
superintelligent agent in a sealed hardware so as to prevent it from doing any harm to humankind-- Eric
Drexler
Should We Allow AGIs?—The Control Problem
• Nick Bostrom, a futurologist, has proposed an idea for an Oracle AI (OAI), which would be only capable of

answering questions.

• Finally, in 2010 David Chalmers proposed the idea of a “leakproof” singularity. He suggested that for safety

reasons, AI systems first be restricted to simulated virtual worlds until their behavioral tendencies could be fully

understood under the controlled conditions.

• The Ted Kaczinsky Manifesto: ….What we do suggest is that the human race might easily permit itself to drift into

a position of such dependence on the machines that it would have no practical choice but to accept all of the

machines decisions…. we will be so dependent on them that turning them off would amount to suicide.”

• Technological slavery.
 The Ethics of AI
• The Ethics of Economic Inequality—Between nations and within nations. Need for
a Universal Minimum Wage Solution—Thomas Picketty
• The ethics of human substitution by industrial robots in employment-not a
problem in economies with declining birthrates and shrinking populations
• Since AGI can outsmart human cognitive and emotional intelligence, then they
are sapient and especially sentient and capable of robot suffering. Using such
robots as a means would then be unethical
• The issue of collateral damage and trigger-happiness in AI Warfare
• The ethics of increasing E-Waste due to robotisisation, including radio-frequency
radiation
• The Ethics of Face Recognition Technology-Loss of Privacy vs Reduction in Crime
• The Ethics of Singularity- Should we allow this?
A Dystopian View of AI
 2021-22 Cullings for Human Enhancement
1. Posthumanism, Transhumanism, Antihumanism, Metahumanism, and New
Materialisms--Differences and Relations, Francesca Ferrando, Columbia University,
Existenz, Volume 8, No 2, Fall 2013, z 8/2 (2013), 26-32
◼ Transhumanism offers a very rich debate on the impact of technological and
scientific developments in the evolution of the human species; and still, it holds a
humanistic and humancentric perspective which weakens its standpoint: it is a
"Humanity Plus" movement, whose aim is to "elevate the human condition." On the
contrary, speciesism has become an integral part of the posthumanist approach,
formulated on a post-anthropocentric and post-humanistic episteme based on
decentralized and non-hierarchical modes. Although posthumanism investigates the
realms of science and technology, it does not recognize them as its main axes of
reflection, nor does it limit itself to their technical endeavors, but it expands its
reflection to the technologies of existence.
2. Transhumanist Values, NICK BOSTROM, Oxford University, Faculty of Philosophy,
Ethical Issues for the 21st Century, ed. Frederick Adams (Philosophical
Documentation Center Press, 2003); reprinted in Review of Contemporary
Philosophy, Vol. 4, May (2005)]
◼ It promotes an interdisciplinary approach to understanding and evaluating the
opportunities for enhancing the human condition and the human organism opened
up by the advancement of technology.
◼ The enhancement options being discussed include radical extension of human
health-span, eradication of disease, elimination of unnecessary suffering, and
augmentation of human intellectual, physical, and emotional capacities. Other
transhumanist themes include space colonization and the possibility of creating
superintelligent machines, along with other potential developments that could
profoundly alter the human condition. The ambit is not limited to gadgets and
medicine, but encompasses also economic, social, institutional designs, cultural
development, and psychological skills and techniques.
◼ Transhumanism does not entail technological optimism. While future technological
capabilities carry immense potential for beneficial deployments, they also could be
misused to cause enormous harm, ranging all the way to the extreme possibility of
intelligent life becoming extinct. Other potential negative outcomes include
widening social inequalities or a gradual erosion of the hard-to-quantify assets that
we care deeply about but tend to neglect in our daily struggle for material gain, such
as meaningful human relationships and ecological diversity. Such risks must be taken
very seriously, as thoughtful transhumanists fully acknowledge.
◼ The limitations of the human mode of being are so pervasive and familiar that we
often fail to notice them, and to question them requires manifesting an almost
childlike naiveté. Let consider some of the more basic ones: Lifespan, Intellectual
capacity, Bodily functionality, Mood, energy, and self-control,
◼ In Christian theology, some souls will be allowed by God to go to heaven after their
time as corporal creatures is over. Before being admitted to heaven, the souls would
 undergo a purification process in which they would lose many of their previous
bodily attributes. Skeptics may doubt that the resulting minds would be sufficiently
similar to our current minds for it to be possible for them to be the same person. A
similar predicament arises within transhumanism: if the mode of being of a
posthuman being is radically different from that of a human being, then we may
doubt whether a posthuman being could be the same person as a human being,
even if the posthuman being originated from a human being.
◼ We can, however, envision many enhancements that would not make it impossible
for the post-transformation someone to be the same person as the pre-
transformation person. A person could obtain quite a bit of increased life
expectancy, intelligence, health, memory, and emotional sensitivity, without ceasing
to exist in the process. A person’s intellectual life can be transformed radically by
getting an education. A person’s life expectancy can be extended substantially by
being unexpectedly cured from a lethal disease. Yet these developments are not
viewed as spelling the end of the original person. In particular, it seems that
modifications that add to a person’s capacities can be more substantial than
modifications that subtract, such as brain damage. If most of someone currently is,
including her most important memories, activities, and feelings, is preserved, then
adding extra capacities on top of that would not easily cause the person to cease to
exist.

◼ Transhumanism promotes the quest to develop further so that we can explore

hitherto inaccessible realms of value. Technological enhancement of human
organisms is a means that we ought to pursue to this end. There are limits to how
much can be achieved by low-tech means such as education, philosophical
contemplation, moral self-scrutiny and other such methods proposed by classical
philosophers with perfectionist leanings, including Plato, Aristotle, and Nietzsche, or
by means of creating a fairer and better society, as envisioned by social reformists
such as Marx or Martin Luther King. This is not to denigrate what we can do with the
tools we have today. Yet ultimately, transhumanists hope to go further.
◼ What is needed for the realization of the transhumanist dream is that technological
means necessary for venturing into the post-human space are made available to
those who wish to use them, and that society be organized in such a manner that
such explorations can be undertaken without causing unacceptable damage to the
social fabric and without imposing unacceptable existential risks.
◼ Existential risk – one where an adverse outcome would either annihilate Earth-
originating intelligent life or permanently and drastically curtail its potential. Several
recent discussions have argued that the combined probability of the existential risks
is very substantial. The relevance of the condition of existential safety to the
transhumanist vision is obvious: if we go extinct or permanently destroy our
potential to develop further, then the transhumanist core value will not be realized.
Global security is the most fundamental and non-negotiable requirement of the
transhumanist project. Technological progress in this field be allowed to progress
unhindered and finally the benefits must have wide access and should not be
available only to an elite section of humankind.
 Cullings from AI Readings for XLRI
1. Artificial Intelligence Safety Engineering: Why Machine Ethics Is a Wrong Approach
Roman V. Yampolskiy* Roman V. Yampolskiy Department of Computer Engineering
and Computer Science, University of Louisville, V.C. Müller (Ed.): Philosophy and
Theory of Artificial Intelligence, SAPERE 5, pp. 389–396. springerlink.com © Springer-
Verlag Berlin Heidelberg 2012

The great majority of published papers are purely philosophical in nature and do little
more than reiterate the need for machine ethics and argue about which set of moral
convictions would be the right ones to implement in our artificial progeny
(Kantian [33], Utilitarian [20], Jewish [34], etc.). However, since ethical norms
are not universal, a “correct” ethical code could never be selected over others to
the satisfaction of humanity as a whole.
Consequently, we propose that purely philosophical discussions of ethics for
machines be supplemented by scientific work aimed at creating safe machines in
the context of a new field we will term “AI Safety Engineering.” Some concrete
work in this important area has already begun [17, 19, 18]. A common theme in
AI safety research is the possibility of keeping a superintelligent agent in a sealed
hardware so as to prevent it from doing any harm to humankind. Such ideas origi-
nate with scientific visionaries such as Eric Drexler who has suggested confining
transhuman machines so that their outputs could be studied and used safely [14].
Similarly, Nick Bostrom, a futurologist, has proposed [9] an idea for an Oracle AI
(OAI), which would be only capable of answering questions. Finally, in 2010
David Chalmers proposed the idea of a “leakproof” singularity [12]. He suggested
that for safety reasons, AI systems first be restricted to simulated virtual worlds
until their behavioral tendencies could be fully understood under the controlled
conditions.
Roman Yampolskiy has proposed a formalized notion of AI confinement pro-
tocol which represents “AI-Boxing” as a computer security challenge [46]. He de-
fines the Artificial Intelligence Confinement Problem (AICP) as the challenge of
restricting an artificially intelligent entity to a confined environment from which it
can’t exchange information with the outside environment via legitimate or covert
channels if such information exchange was not authorized by the confinement au-
thority. An AI system which succeeds in violating the CP protocol is said to have
escaped [46].
Similarly we argue that certain types of artificial intelligence research fall under
the category of dangerous technologies and should be restricted. Classical AI re-
search in which a computer is taught to automate human behavior in a particular
domain such as mail sorting or spellchecking documents is certainly ethical and
does not present an existential risk problem to humanity. On the other hand, we argue that
Artificial General Intelligence (AGI) research should be considered un-
ethical. This follows logically from a number of observations. First, true AGIs will
be capable of universal problem solving and recursive self-improvement. Conse-
quently they have potential of outcompeting humans in any domain essentially
making humankind unnecessary and so subject to extinction. Additionally, a truly
AGI system may possess a type of consciousness comparable to the human type
making robot suffering a real possibility and any experiments with AGI unethical
for that reason as well.
A similar argument was presented by Ted Kazynsky in his famous
manifesto [26]: “It might be argued that the human race would never be foolish
enough to hand over all the power to the machines. But we are suggesting neither
that the human race would voluntarily turn power over to the machines nor that
the machines would willfully seize power. What we do suggest is that the human
race might easily permit itself to drift into a position of such dependence on the
machines that it would have no practical choice but to accept all of the machines
decisions. As society and the problems that face it become more and more com-
plex and machines become more and more intelligent, people will let machines
make more of their decision for them, simply because machine-made decisions
will bring better result than man-made ones. Eventually a stage may be reached at
which the decisions necessary to keep the system running will be so complex that
human beings will be incapable of making them intelligently. At that stage the ma-
chines will be in effective control. People won't be able to just turn the machines
off, because they will be so dependent on them that turning them off would amount
to suicide. ” ( Kaczynski, T.: Industrial Society and Its Future. The New York Times
(September19, 1995)
Humanity should not put its future in the hands of the machines since it will not
be able to take the power back. In general a machine should never be in a position
to terminate human life or to make any other non-trivial ethical or moral judgment
concerning people.
2. Why and How Should Robots Behave Ethically?, Benjamin KUIPERS 1, Computer
Science & Engineering, University of Michigan, USA, 1University of Michigan, 2260
Hayward Street, Ann Arbor, Michigan 48109 USA, Email: kuipers@umich.edu
For an intelligent robot to function successfully in our society, to cooperate with
humans, it must not only be able to act morally and ethically, but it must also
be trustworthy. It must earn and keep the trust of humans who interact with it.
If every participant contributes their share, everyone
gets a good outcome. But each individual participant may do even better by
optimizing their own reward at the expense of the others. With self-centered utility
functions, each participant “rationally” maximizes their own expected utility,
often leading to bad outcomes for everyone.

• Should you use a sharp knife to cut into the body of a human being? Of
course not, unless you are a qualified surgeon performing a necessary op-
eration. (Deontology: a rule with an exception.)

• If you are that surgeon, is it permissible to sacrifice this patient in order to

save the lives of five others? Of course not! (Virtue ethics: a good surgeon
keeps faith with the patient.)

• Is it OK to throw the switch that saves five lives by directing a runaway trolley onto a
side track, where it will kill one person who would have been safe? Well, . . .
(Deontology says it’s wrong to allow preventable deaths; Utilitarianism says fewer
deaths is better; Virtue ethics says the virtuous person can make hard choices.)
I argue that heuristics based on utilitarianism (decision theory), deontology (rule-
based and constraint-based systems), and virtue ethics (case-based reasoning) are
all important tools in the toolkit for creating artificial agents capable of partic-
ipating successfully in our society. Each tool is useful in certain contexts, and
perhaps less useful in others.

1. The Virtuous Machine -Old Ethics for New Technology? Nicolas Berberich* and Klaus
Diepold, Department of Electrical and Computer Engineering, Technical University of
Munich, Department of Informatics, Technical University of Munich, Munich Center
for Technology in Society,* E-mail: n.berberich@tum.de
Due to the inherent autonomy of these systems, the ethical considerations have to be
conducted by themselves. This means, that these autonomous cognitive machines are in
need of a theory, with the help of which they can, in a specific situation, choose the action
that adheres best to the moral standards.
This discrepancy between what people believe that technology can do, based on its
appearance, and what it
actually can do, would not only elicit a strong uncanny valley effect, but also pose a large
safety risk. Taken together, we predict that this would lead to an acceptance problem of the
technology. If we want to avoid this by jumping over the uncanny valley, we have to start
today by thinking about how to endow autonomous cognitive systems with more human-
like behavior. The position that we argue for in this paper is that the last discrepancy
between the valley and its right shore lies in virtuous moral behavior. In the near future
we will have autonomous cognitive machines whose actions will be akin to human actions,
but without consideration of moral implications they will never be quite alike, leading to
cognitive dissonance and rejection. We believe that taking virtue ethics as the guiding moral
theory for building moral machines is a promising approach to avoid the uncanny valley and
to induce acceptance.
Cybernetics can be seen as a historical and intellectual precursor of artificial intelligence
research. While it had strong differences with the cognitivistic GOFAI (good old-fashioned
AI), cybernetic ideas are highly influential in modern AI. The currently successful field of ar-
tificial neural networks (synonymous terms are connectionism and deep learning) originated
from the research of the cyberneticians McCulloch, Pitts and Rosenblatt. Goal-directed
planning is a central part of modern AI and especially of advanced robotics. In contrast
to other forms of machine learning like supervised or unsupervised learning, reinforcement
learning is concerned with the goal-driven (and therefore teleological) behavior of agents.

Applied to AI ethics this means that a machine cannot have practical wisdom (and thus can’t
act morally) before it has learned from realistic data. Machine learning is the improvement
of a machine’s performance of a task through experience and Aristotle’s virtue ethics is the
improvement of one’s virtues through experience. Therefore, if one equates the task
performance with virtuous actions, developing a virtue ethics-based machine appears
possible.
A closer look at the structure of Aristotle’s ergon-argument allows to break with two
common misconceptions which seem to render a virtue ethical approach in machine ethics
impossible. The first misconception is ethical anthropocentrism, after which only humans
can act morally. This might have been correct in the past, but only because humans have
been the only species capable of higher-level cognition, which, according to Aristotle, is
a requirement for ethical virtues and thus moral action. If there was another species, for
example a machine, with the same capacity for reason and dispositions of character, then it
appears probable that its arete would also lie in excellent use and improvement of those.
The second misconception of Aristotle’s virtue ethics is that it takes happiness to be the goal
and measure of all actions. Since machines are not capable of genuine feelings of happiness,
it is argued, that virtue ethics can’t be applied to them. This argument is based on an
erroneous understanding of eudaimonia. Aristotle does not mean psychological states of
happiness nor maximized pleasure, as John Locke defines ’happiness’. The Greek term
eudaimonia has a much broader meaning and refers mainly to a successful conduct of life
(according to one’s ergon). A virtuous machine programed to pursue eudaimonia would
therefore not be prone to wireheading, which is the artificial stimulation of the brain’s
reward center to experience pleasure.
Out of the three subcategories of machine learning, supervised learning, unsupervised
learning and reinforcement learning (RL), the latter is the lifeworldly approach. In contrast
to the other two, RL is based on dynamic interaction with the environment, of which the
agent typically has only imperfect knowledge.
This partition originated in Aristotle’s soul theory in which he lists virtues of reason
(dianoetic virtues) next to virtues of character (ethical virtues) as properties of the
intelligent part of the soul. The virtues of reason comprise the virtues of pure reason and
the virtues of practical reason. Pure reason includes science (epist ē m ē ), wisdom (sophia)
and intuitive thought (no ̄us). Practical reason on the other hand refers to the virtues of
craftsmanship (techn ̄e), of making (poi ̄esis) and practical wisdom (phron ē sis). According to
this subdivision in pure and practical reason, there exist two ways to lead a good life in the
eudaimonic sense: the theoretical life and the practical life. AI systems can lead a theoretical
life of contemplation, e.g. when they are applied to scientific data analysis, but to lead a
practical life they need the capacity for practical wisdom and morality. This distinction in
theoretical and practical life of an AI somewhat resembles the distinction into narrow and
general AIs, where narrow AI describes artificial intelligence systems that are focused on
performing one specific task (e.g. image classification) while general AI can operate in more
general and realistic situations.
In contrast to deontology and consequentialism, virtue ethics has a hard time giving
reasons for its actions (they certainly exist, but are hard to codify). While deontologists can
point towards the principles and duties which have guided their actions, a consequentialist
can explain why her actions have led to the best consequences. An AMA based on virtue
ethics on the other hand would have to show how its virtues, which gave rise to its actions,
have been formed through experience. This poses an even greater problem if its capability
to learn virtues has been implemented as an artificial neural network, due to it being almost
impossible to extract intuitively understandable reasons from the many network weights. In
this instance, the similarity between virtue ethics and machine learning is disadvantageous.
Without being able to give reasons to one’s actions, one cannot take over responsibility,
which is a concept underlying not only our insurance system but also our justice system. If
the actions of an AMA produce harm then someone has to take responsibility for it and the
victims have a right to explanation. The latter has recently (May 2018) been codified by the
EU General Data Protection Regulation (GDRP) with regards to all algorithmic decisions.
Condensed to the most important ideas, this work has shown that
1. Virtue ethics fits nicely with modern artificial intelligence research and is a promising
moral theory as basis for the field of AI ethics.
2. Taking the virtue ethics route to building moral machines allows for a much broader
approach than simple decision-theoretic judgment of possible actions. Instead it takes
other cognitive functions into account like attention, emotions, learning and actions.
Furthermore, by discussing several virtues in detail, we showed that virtue ethics is a
promising moral theory for solving the two major challenges of contemporary AI safety
research, the control problem and the value alignment problem. A machine endowed with
the virtue of temperance would not have any desire for excess of any kind, not even for
exponential self-improvement, which might lead to a superintelligence posing an existential
risk for humanity. Since virtues are an integral part of one’s character, the AI would not
have the desire of changing its virtue of temperance. Learning from virtuous exemplars
has been a process of aligning values for centuries (and possibly for all of human history),
thus building artificial systems with the same imitation learning capability appears to be a
reasonable approach.

2. Machines That Know Right And Cannot Do Wrong: The Theory and Practice of
Machine Ethics, Louise A. Dennis and Marija Slavkovik
“The fact that man knows right from wrong proves his intellectual superiority to the other
creatures; but the fact that he can do wrong proves his moral inferiority to any creatures
that cannot.”– Mark Twain
Wallach and Allen [35, Chapter 2] distinguish between operational morality, functional
morality, and full moral agency. An agent has operational morality when the moral
significance of her actions are entirely scoped by the agent’s designers. An agent has
functional morality when the agent is able to make moral judgements when choosing an
action, without direct human instructions.
3. What happens if robots take the jobs? The impact of emerging technologies on
employment and public policy By Darrell M. West
In this paper, I explore the impact of robots, artificial intelligence, and machine learning. In
particular, I study the impact of these emerging technologies on the workforce and the
provision of health benefits, pensions, and social insurance. If society needs fewer workers
due to automation and robotics, and many social benefits are delivered through jobs, how
are people outside the workforce for a lengthy period of time going to get health care and
pensions?
Robots are expanding in magnitude around the developed world. Figure 1 shows the
numbers of industrial robots in operation globally and there has been a substantial increase
in the past few years. In 2013, for example, there were an estimated 1.2 million robots in
use. This total rose to around 1.5 million in 2014 and is projected to increase to about 1.9
million in 2017.5 Japan has the largest number with 306,700, followed by North America
(237,400), China (182,300), South Korea (175,600), and Germany (175,200). Overall, robotics
is expected to rise from a $15 billion sector now to $67 billion by 2025.6
In the contemporary world, there are many robots that perform complex functions.
According to a presentation on robots, “the early 21st century saw the first wave of
companionable social robots. They were small cute pets like AIBO, Pleo, and Paro. As
robotics become more sophisticated, thanks largely to the smart phone, a new wave of
social robots has started, with humanoids Pepper and Jimmy and the mirror-like Jibo, as
well as Geppetto Avatars’ software robot, Sophie. A key factor in a robot’s ability to be
social is their ability to correctly understand and respond to people’s speech and the
underlying context or emotion.”
Amazon has organized a “picking challenge” designed to see if robots can “autonomously
grab items from a shelf and place them in a tub.” The firm has around 50,000 people
working in its warehouses and it wants to see if robots can perform the tasks of selecting
items and moving them around the warehouse. During the competition, a Berlin robot
successfully completed ten of the twelve tasks. To move goods around the facility, the
company already uses 15,000 robots and it expects to purchase additional ones in the
future.
In the restaurant industry, firms are using technology to remove humans from parts of food
delivery. Some places, for example, are using tablets that allow customers to order directly
from the kitchen with no requirement of talking to a waiter or waitress. Others enable
people to pay directly, obviating the need for cashiers. Still others tell chefs how much of an
ingredient to add to a dish, which cuts down on food expenses.
There are computerized algorithms that have taken the place of human transactions. We
see this in the stock exchanges, where high-frequency trading by machines has replaced
human decision-making. People submit, buy, and sell orders, and computers match them in
the blink of an eye without human intervention. Machines can spot trading inefficiencies or
market differentials at a very small scale and execute trades that make money for people.15
Some individuals specialize in arbitrage trading, whereby the algorithms see the same stocks
having different market values. Humans are not very efficient at spotting price differentials
but computers can use complex mathematical formulas to determine where there are
trading opportunities. Fortunes have been made by mathematicians who excel in this type
of analysis.
Machine-to-machine communications and remote monitoring sensors that remove humans
from the equation and substitute automated processes have become popular in the health
care area. There are sensors that record vital signs and electronically transmit them to
medical doctors. For example, heart patients have monitors that compile blood pressure,
blood oxygen levels, and heart rates. Readings are sent to a doctor, who adjusts medications
as the readings come in. According to medical professionals, “we’ve been able to show
significant reduction” in hospital admissions through these and other kinds of wireless
devices.
There also are devices that measure “biological, chemical, or physical processes” and deliver
“a drug or intervention based on the sensor data obtained.” They help people maintain an
independent lifestyle as they age and keep them in close touch with medical personnel.
“Point-of-care” technologies keep people out of hospitals and emergency rooms, while still
providing access to the latest therapies.
Implantable monitors enable regular management of symptoms and treatment. For
example, “the use of pulmonary artery pressure measurement systems has been shown to
significantly reduce the risk of heart failure hospitalization.” Doctors place these devices
inside heart failure patients and rely upon machine-to-machine communications to alert
them to potential problems. They can track heart arrhythmia and take adaptive moves as
signals spot troublesome warning signs.
Unmanned vehicles and autonomous drones are creating new markets for machines and
performing functions that used to require human intervention. Driverless cars represent one
of the latest examples. Google has driven its cars almost 500,000 miles and found a
remarkable level of performance. Manufacturers such as Tesla, Audi, and General Motors
have found that autonomous cars experience fewer accidents and obtain better mileage
than vehicles driven by people.
4. CaseWestern Reserve Journal of International Law 47 (2015), Issue 1
The Debate Over Autonomous Weapons Systems Dr. Gregory P. Noone and Dr.
Diana C. Noone
The debate over Autonomous Weapon Systems (AWS) has begun
in earnest with advocates for the absolute and immediate banning of
AWS development, production, and use planting their flag first. They
argue that AWS should be banned because these systems lack human
qualities, such as the ability to relate to other humans and to apply
human judgment, that are necessary to comply with the law. In
addition, the weapons would not be constrained by the capacity for
compassion, which can provide a key check on the killing of civilians.
The opposing viewpoint in this debate articulates numerous
arguments that generally include: it is far too premature and too
speculative to make such a proposal/demand; the Law of Armed
Conflict should not be underestimated in its ability to control AWS
development and future operations; AWS has the potential to
ultimately save human lives (both civilian and military) in armed
conflicts; AWS is as inevitable as any other technology that could
potentially make our lives better; and to pass on the opportunity to
develop AWS is irresponsible from a national security perspective.
Some of the most respected and brilliant lawyers in this field are on
opposite sides of this argument.

1. Human-in-the-loop or semi-autonomous systems require a

human to direct the system to select a target and attack it, such
as Predator or Reaper UAVs.
2. Human-on-the loop or human-supervised autonomous systems
are weapon systems that select targets and attack them, albeit
with human operator oversight; examples include Israel’s Iron
Dome and the U.S. Navy’s Phalanx Close In Weapons System
(or CIWS).
 3. Human-out-of-the-loop or fully autonomous weapon systems
can attack without any human interaction; there are currently
no such weapons.

First and foremost, there is immediate common ground to be

found in this debate. Any weaponry development shall be done so in
accordance with the Law of Armed Conflict (LOAC, also referred to
as International Humanitarian Law, IHL, or the Law of War). With
respect to AWS, its development and deployment would be required
to adhere to LOAC’s core principles of distinction, proportionality,
humanity and military necessity. There is readily accepted treaty
law as well as customary international law that makes this area of
discussion easy. AWS is, as all weapons and weapon systems are, a
means of warfare (whereas a method of warfare involves deployment
and tactics). All AWS would have a legal review conducted prior to
formal development as a weapon (or prior to any modification of an
existing weapon) and another legal review prior to being deployed in
the field. Therefore, the concept of AWS is not per se unlawful. At
their core, autonomous weapon systems must be able to distinguish
combatants from non-combatants as well as friend from foe. LOAC is
designed to protect those who cannot protect themselves, and an
underlying driver is to protect civilians from death and combatants
from unnecessary suffering. Everyone is in agreement on this. No
academic or practitioner is stating anything to the contrary; therefore,
this part of any argument from either side must be ignored as a red
herring. Simply put, no one would agree to any weapon that ignores
LOAC obligations.
At the present time, there are many questions and as yet few
answers with respect to Autonomous Weapon Systems. Not the least
of which include the policy implications of such systems. For instance, “How does
this technology impact the likely successes of counter-
insurgency operations or humanitarian interventions? Does not such weaponry run
the risk of making war too easy to wage and tempt policy makers into killing when
other more difficult means should be undertaken?” Will countries be more willing to
use force because their populations would have less to lose (i.e. their loved ones)
and it would be politically more acceptable?

5. Robots, Rights and Religion, James F. McGrath, Butler University, 2011

To put this another way, we might decide that we could exclude from the category of
persons those artificial intelligences that were merely programmed to imitate personhood,
and whose interaction with humans resembled that of persons simply as a result of
elaborate programming created precisely to imitate human behavior. This must be
distinguished from the case of a machine that learns human behavior and imitates it of its
own volition. This distinction is not arbitrary. Children carry out patterns of behavior that
resemble those of their parents and others around them. This is part of the learning
process, and is evidence in favor of rather than against their true personhood. The evidence
that I am suggesting would count against genuine personhood is deliberate programming by
a human programmer that causes a machine to imitate personhood in a contrived manner.
The reason for this distinction is an important one. A machine that learns to imitate human
behavior would be exhibiting a trait we witness in human persons.
In concluding this section, it should be remarked that we give human rights to human beings
as soon as they are clearly categorized as such. A person does not have to be able to speak
to have rights. Indeed, small infants whose ability to reason, communicate and do many
other things that we tend to identify with intelligence is still in the process of formation
have their rights protected by law. The issue is thus not really rights for artificial
intelligences so much as rights for machine persons. It is the definition and identification of
the latter that is the crucial issue.
Nevertheless, the distinction would seem to be a valid one for as long as it remains a
meaningful one: machines that develop their own personhood in imitation of humans will
probably deserve to be recognized as persons, whereas mere simulacra designed as an
elaborate contrivance will not.
Our creations – whether through natural biological reproduction, in vitro fertilization,
cloning, genetic construction, or artificially intelligent androids made in our image – can be
viewed as in some sense like our children. And if the comparison to our children is a useful
analogy, then we can learn much from it. There is a “flip side” to the point that children are
their own people and sooner or later we need to let them go, to make their own mistakes.
The other side of the coin is that we are not living up to our responsibilities if we let them go
too soon. Yet Is it only wrong to tamper with humanity’s nature, or is it also wrong to create
a human being (with some differences)?
our artificial offspring will in an important sense not be human, even if they are made in our
image. Other species leave the nest far earlier than human children do. In “giving birth” not
to other humans but to artificial intelligence, we cannot assume that the process will even
closely mirror a typical human parent-child scenario.

6. Robot ethics: Mapping the issues for a mechanized world , Patrick Lin Keith Abney
George Bekey
Bill Gates recently observed that “the emergence of the robotics industry ... is developing in
much the same way that the computer business did 30 years ago” [18]. As a key architect of
the computer industry, his prediction has special weight.
In a few decades—or sooner, given exponential progress forecasted by Moore’s Law—
robots in society will be as ubiquitous as computers are today, he believes; and we would be
hard-pressed to find an expert who disagrees.
In its most basic sense, we define “robot” as an engineered machine that senses, thinks, and
acts: “Thus a robot must have sensors, processing ability that emulates some aspects of
cognition, and actuators.
Surprisingly,
relationships of a more intimate nature are not quite satisfied by robots yet, considering the
sex industry’s reputation as an early adopter of new technologies. Introduced in 2010,
Roxxxy is billed as “the world’s first sex robot” [17], but its lack of autonomy or capacity to
“think” for itself, as opposed to merely respond to sensors, suggests that it is not in fact a
robot, per the definition above.
In some countries, robots are quite literally replacements for humans, such as Japan, where
a growing elderly population and declining birthrates mean a shrinking workforce [35].
Robots are built to specifically fill that labor gap. And given the nation’s storied love of
technology, it is therefore unsurprising that approximately one out of 25 workers in Japan is
a robot. While the US currently dominates the market in military robotics, nations such as
Japan and South Korea lead in the market for social robotics, such as elderly-care robots.
Other nations with similar demographics, such as Italy, are expected to introduce more
robotics into their societies, as a way to shore up a decreasing workforce; and nations
without such concerns can drive productivity, efficiency, and effectiveness to new heights
with robotics.
Like the social networking and email capabilities of the Internet Revolution, robotics may
profoundly impact human relationships. Already, robots are taking care of our elderly and
children, though there are not many studies on the effects of such care, especially in the
long term. Some soldiers have emotionally bonded with the bomb-disposing PackBots that
have saved their lives, sobbing when the robot meets its end (e.g., [38,22]). And robots are
predicted to soon become our lovers and companions [25]: they will always listen and never
cheat on us. Given the lack of research studies in these areas, it is unclear whether
psychological harm might arise from replacing human relationships with robotic ones.
Harm also need not be directly to persons, e.g., it could also be to the environment. In the
computer industry, “e-waste” is a growing and urgent problem (e.g., [31]), given the
disposal of heavy metals and toxic materials in the devices at the end of their product
lifecycle. Robots as embodied computers will likely exacerbate the problem, as well as
increase pressure on rare-earth elements needed today to build computing devices and
energy resources needed to power them. Networked robots would also increase the
amount of ambient radiofrequency radiation, like that created by mobile phones—which
have been blamed, fairly or not, for a decline of honeybees necessary for pollination and
agriculture [37], in ADDITION to human health problems (e.g., [2]).

7. Networks of Social and Moral Norms in Human and Robot Agents, B. F. Malle ∗† M.
Scheutz ∗∗ J. L. Austerweil, Department of Cognitive, Linguistic, and Psychological
 Sciences,Brown University, USA∗∗ Department of Computer Science, Tufts
University, USA
The design and construction of intelligent robots has seen steady growth in the past 20
years, and the integration of robots into society is, to many, imminent (Nourbakhsh, 2013;
Sabanovi ̆ c, 2010). Ethical questions about such integration have recently gained
prominence. For example, academic publications on the topic of robot ethics doubled
between 2005 and 2009 and doubled again since then, counting almost 200 as of the time
of this conference (Malle, 2015).
Economic scholars have puzzled for a long time why such free-riding is not more common—
why people cooperate much more often than they “defect,” as game theorists call it, when
defecting would provide the agent with larger utility.
The answer cannot be that humans are “innately” cooperative, because they are perfectly
capable of defecting. The answer involves to a significant extent the power of norms. A
working definition of a norm is the following: An instruction to (not) perform a specific or
general class of action, whereby a sufficient number of individuals in a community (a)
indeed follow this instruction and (b) expect others in the community to follow the
instruction.
8. Moral Machines and the Threat of Ethical Nihilism, Anthony F. Beavers
But, though my cell phone might be smart, I do not take that to mean that it is thoughtful,
insightful, or wise. So, what has become of these latter categories? They seem to be
bygones, left behind by scientific and computational conceptions of thinking and knowledge
that no longer have much use for them.
Respecting
Kantian ethics, the problem is apparent in the universal law formulation of the categorical
imperative, the one that would seem to hold the easiest prospects for rule-based
implementation in a computational system: “act as if the maxim of your action were to
become through your will a universal law of nature ”( Kant [1785] 1981 , 30).
One mainstream interpretation of this principle suggests that whatever rule (or maxim) I
should use to determine my own behavior must be one that I can consistently will to be
used to determine the behavior of everyone else. (Kant ’s most consistent example of this
imperative in application concerns lying promises. One cannot make a lying promise without
simultaneously willing a world in which lying is permissible, thereby also willing a world in
which no one would believe a promise, particularly the very one I am trying to make. Thus,
the lying promise fails the test and is morally impermissible.) Though at first the categorical
imperative looks implementable from an engineering point of view, it suffers from a
problem of scope, since any maxim that is defined narrowly enough (for instance, to include
a class of one, anyone like me in my situation) must consistently universalize. Death by
failure to implement looks imminent; so much the worse for Kant, and so much the better
for ethics.
Classical utilitarianism meets a similar fate, even though, unlike Kant, Mill casts internals,
such as intentions, to the wind and considers just the consequences of an act for evaluating
moral behavior. Here, “actions are right in proportion as they tend to promote happiness;
wrong as they tend to produce the reverse of happiness. By happiness is intended pleasure
and the absence of pain; by unhappiness, pain and the privation of pleasure” That internals
are incidental to utilitarian ethical assessment is evident in the fact that Mill does not
require that one act for the right reasons. He explicitly says that most good actions are not
done accordingly, Thus, acting good is indistinguishable from being good, or, at least, to be
good is precisely to act good; and sympathetically we might be tempted to agree, asking
what else could being good possibly mean. Things again are complicated by problems of
scope, though Mill, unlike Kant, is aware of them. He writes, “again, defenders of utility
often find themselves called upon to reply to such objections as this — that there is not
enough time, previous to action, for calculating and weighing the effects of any line of
conduct on the general happiness” ( [1861] 1979 , 23). (In fact, the problem is
computationally intractable when we consider the ever-extending ripple effects that any act
can have on the happiness of others across both space and time.) Mill gets around the
problem with a sleight of hand, noting that “all rational creatures go out upon the sea of life
with their minds made up on the common questions of right and wrong” (24), suggesting
that calculations are, in fact, unnecessary, if one has the proper forethought and upbringing.
Again, the rule is of little help, and death by failure to implement looks imminent. So much
the worse for Mill; again, so much the better for ethics.

9. Moral Robots—Elizebeth Huh

His efforts to influence the public through reason also recall Plato’s theory of human
motivation, illustrated by his conception of the tripartite soul. In Book II of The Republic,
Socrates explains that the root of each human desire can find its origin in one part of a
three-part soul: the lowest part, epithumia, motivates the appetites and the desire for
bodily pleasure; the middle part of the soul, thumos, desires honor and a competitive form
of self-interest; and the highest part of the soul, logos, loves reason and knowledge.
Socrates explains that the size of each of these three parts varies among individuals, but
that the largest piece of each person’s soul naturally guides her into one of three social
classes. The majority of the population, motivated by bodily appetites, become the
moneymakers and craftsmen of the city and fulfill its basic needs; those who desire honor
are best fit to serve as the guardians and warriors of the city; and philosophers, whose
actions are ruled above all by reason, ought to rule.
Though Singer does not go so far as to claim that effective altruists ought to completely strip
away their emotions, he does insist that the emotions distract from his principle of
efficiency, and that this principle is essential for optimal moral decision-making. So this is
what I consider his ultimate mistake: it is his conflation of morality with efficiency, and his
belief that we do not need the emotions and some acceptance of uncertainty on our path to
moral progress.
Why is this a mistake? Let’s look at the legalization of gay marriage in the United States.
Singer’s effective altruist doctrine would have maintained that the suffering of homosexual
couples was not as great as the suffering of those starving to death, and that, therefore,
“maximally effective” altruists wishing to do “the most good” possible should not have
considered spending any time, money, or resources fighting for some the right to some
highly subjective form of emotional fulfilment.

10. Moral Machines: Mindless Morality and its Legal Implications--Andrew Schmelzer
Not all of our devices need moral agency. As autonomy increases, morality becomes more
necessary in robots, but the reverse also holds. Machines with little autonomy need less
ethical sensitivity. A refrigerator need not decide if the amount someone eats is healthy,
and limit access accordingly. In fact, that fridge would infringe on human autonomy.
Ethical sensitivity does not require moral perfection. I do not expect morally perfect
decisions from machines. In fact, because humans are morally imperfect, we cannot base
moral perfection off of humanity by holding machines to human ideals. Our moral
development continues today, and I believe may never finish. Designing an artificial moral
agent bound by the morality of today dooms it to obsolescence: ethical decisions from a
hundred years ago look much more racist, sexist, etc., and less ‘good’ from today’s
perspective; today’s ethics might have the same bias when viewed from the future
(Creighton, 2016). Because the nature of our ethics changes, an agent will stumble
eventually. Instead, we strive for morally human (or even better than human) decisions
from machines. When a machine’s actions reflect those of a human, we will have met the
standards for artificial moral agency.
We can test for artificial moral agency with the Moral Turing Test (Allen, Varner, & Zinser,
2000). In the MTT, where a judge tries to differentiate between a machine and a person by
their moral actions. An agent passes the test when the judge cannot correctly identify the
machine more often than chance. Then, a machine qualifies as a moral agent. In the
comparative Moral Turing Test (cMTT), the judge compares the behaviors of the two
subjects, and determines which action is morally better than the other (Allen, Varner, &
Zinser, 2000). When a machine’s behavior consistently scores morally preferable to a
human’s behavior, then either the agent will have surpassed human standards, or the
human’s behavior markedly strays from those standards.
Frankena (1973) provides a list of terminal values — virtues that are valued for themselves,
rather than their consequences (Yudkowsky, 2011):
Life, consciousness, and activity; health and strength; pleasures and satisfactions of all
or certain kinds; happiness, beatitude, contentment, etc.; truth; knowledge and true
opinions of various kinds, understanding, wisdom; beauty, harmony, proportion in
objects contemplated; aesthetic experience; morally good dispositions or virtues; mutual
affection, love, friendship, cooperation; just distribution of goods and evils; harmony and
proportion in one’s own life; power and experiences of achievement; self-expression;
freedom; peace, security; adventure and novelty; and good reputation, honor, esteem,
etc.
Programming all of those values directly into a single utility function (the method of
determining positive or negative results) is ridiculous. Can engineers or ethicists quantify
each value and agree on a prioritization for each? Yudkowsky (2011) proposes a ‘one-
wrong-number’ problem: a phone number has 10 digits, but dialing one wrong number does
not mean you will connect with someone 90% like the person intended. The same may
apply to virtue-based machines.
Furthermore, some values we deem worthy of implementation in our machines may
contradict each other, such as compassion and honesty (e.g. a child’s professional baseball
potential). In this way virtue-based systems still require the caveats of a rule-based system
(Allen, Varner, & Zinser, 2000). But what about non-terminal virtues, that is, virtues we
value for their repercussions?
The three methods of bottom-up development I will discuss here are neural network
learning, genetic algorithms, and scenario analysis systems.
Neural networks function similarly to neurons: connections between inputs and outputs
make up a system that can learn to do various things, from playing computer games to
running bipedally in a simulation. By using that learning capability on ethical endeavors, a
moral machine begins to develop. From reinforcement of positive behaviors and penalty of
negative ones, the algorithm learns the pattern of our moral systems. Eventually, engineers
place the algorithm in charge of a physical machine, and away it goes. One downside to this
is the uncertainty regarding what the algorithm learned. When the army tried to get a
neural net to recognize tanks hidden in trees, what looked like a distinction between trees,
tanks, and partly concealed tanks turned out to be a distinction between a sunny and cloudy
day (Dreyfus & Dreyfus, 1992). Kuang (2017) writes about Darrel’s potential solution: having
two neural networks working side by side. The first learns the correlation between input
and output, challenging situation and ethically right decision, respectively. The second
algorithm focuses on learning language and connects tags or captions from an input and
explains what cues and ideas the second algorithm used to come up with a course of action.
The second weak point stems from allowing mistakes: no amount of learning can verify that
the machine will act morally in all situations in the real world, including those not tested and
learned from.
Genetic algorithms operate on a somewhat similar principle. Large numbers of simple digital
agents run through ethically challenging simulations. The ones that return the best scores
get “mated” with each other, blending code with a few randomizations, and then the test
runs again (Fox, 2009). After the best (or acceptably best) scores based on desired outcomes
are achieved, a new situation is added to the repertoire that each program must surpass. In
this way, machines can learn our moral patterns. Once thoroughly evolved, we implement
the program, and the machine operates independently in the real world. Alternatively to
direct implementation, we could evolve the program to learn patterns quickly and
efficiently, and then run it through the neural network training. This method suffers the
same downsides as neural networking: we cannot tell what it learned or whether it will
make mistakes in the future.
The final approach involves scenario analysis. Parkin (2017) describes a method of teaching
AI by having it read books and stories and learn the literature’s ideas and social norms.
While this may not apply to machines we do not intend to behave as humans, the idea still
applies to niche or domain-specific machines. Instead of using literature as learning input,
we provide a learning program with records of past wrongdoings and successful outcomes
of ethically-blind machines in its niche. Then the program could infer the proper behaviors
for real world events it may encounter in the future. After analyzing the settings and events
of each scenario, the program would save the connections it made for later human
inspection. If the program’s connections proved ‘good,’ it would then receive a new batch of
scenarios to test through, and repeat the cycle. One downside to this approach involves
painstaking human analysis. A new program would have to go through this cycle for every
machine niche that requires a moral agent, and a human evaluator would have to carefully
examine every connection and correlation the program develops. Darrel’s (2017) explaining
neural net could work in tandem with a scenario analysis system to alleviate the human
requirement for analysis. This approach does get closer to solving the issue of working in
new environments than the previous two approaches, but may nonetheless stumble once
implemented in reality.
Bottom-up approaches utilize continued development to reach an approximation of moral
standards. Neural networks develop connections and correlations to create a new output,
but we struggle to know why the system comes to a decision. Genetic algorithms refine
themselves by duplicating the most successful code into the next generation of programs,
with a little mutation for adaptation. A genetic algorithm’s learning process also remains
obscured without careful record of iterations, which may be beyond human comprehension.
Scenario analysis systems can learn the best conduct historically shown as ethically right,
but still retains potential for error. As of yet, we do not have a reliable method to develop
an artificial moral agent.
To build an artificial moral agent, DeBaets (2014) argues that a machine must have
embodiment, learning, teleology toward the good, and empathy.
DeBaets (2014) claims that moral functioning requires embodiment because if a machine
acts in and influences the physical world, it must have a physical manifestation.
“Embodiment [requires] that a particular decision-making entity be intricately linked to a
particular concrete action; morality cannot solely be virtual if it is to be real” (DeBaets,
2014). They can work from a distance, have multiple centers of action, and have distributed
decision-making centers, but each requires physical form. Embodiment constrains machines
to a physical form, so this definition of moral agency excludes algorithms and programs that
do not interact with the physical world.
Ethical machines need learning capacity so they can perform as taught by ethical and moral
rules and extrapolate what they have learned into new situations. This requirement
excludes any top-down approach that does not involve frequent patches and updates.
Hybrid systems combine rule sets and learning capacities, and so fulfil this requirement
since they can adjust to new inputs and refine their moral behavior.
Teleology toward the good and empathy both face a sizable complication: they both require
some form of consciousness. For a machine to empathize with and understand emotions of
others, it must have emotion itself. Coeckelbergh (2010) claims that true emotion requires
consciousness and mental states in both cognitivist theory and feeling theory. Thus, if
robots do not have consciousness or mental states, they cannot have emotions and
therefore cannot have moral agency. Additionally, if a machine innately desires to do good,
it must have some form of inner thoughts or feeling that it is indeed doing good, so
teleology also requires consciousness or mental states. Much of human responsibility and
moral agency relies on this theory of mind. In court, the insanity or state of mind defence
can counter criminal charges.
However, no empirical way to test for state of mind or consciousness in people exists today.
Why require those immeasurable characteristics in our robots?
Emotionless Machinery
We interpret other humans’ behaviors as coming from or influenced by emotion, but we
have no way to truly determine emotional state. Verbal and nonverbal cues give us insights
to emotions others feel. They may imitate or fake those cues, but we interact with them just
the same as long as they maintain their deception (Goffman, 1956). We measure other
people by their display or performance of emotion.
Since the appearance of emotion in people regulates social interaction and human morality,
we must judge robots by that same appearance. Even today, machines can read breathing
and heart rate (Gent, 2016), and computers do not need to see an entire face to determine
emotion displayed (Wegrzyn, Vogt, Kireclioglu, Schneider, & Kissler, 2017). Soon enough, a
machine could learn to display human emotion by imitating the cues they’re designed to
measure. In theory, a robot could imitate or fake emotional cues as well as humans display
them naturally. People already tend to anthropomorphize robots, empathize with them,
and interpret their behavior as emotional (Turkle, 2011). For consistency in the way we treat
human display of emotion and interpret it as real, we must also treat robotic display of
emotion as real.
If the requirement for empathy changes from true emotion to functional emotion — as is
consistent with how we treat people — then an imitating robot fulfills all the requirements
for empathy, effectively avoiding the issue regarding consciousness and mental state.
Compassion could be the reason an autonomous car veers into a tree rather than a line of
children, but the appearance of compassion could also serve the same effect.
Additionally, a robot can have an artificial teleology towards good, granted that all of the
taught responses programmed into the machine are ‘good.’ Beavers’ (2011) discussion of
classical utilitarianism, referencing Mill (1979), claims that acting good is the same as being
good. The same applies to humans, as far as we can tell from the outside. Wallach and Allen
(2009) note that “values that emerge through the bottom-up development of a system
reflect the specific causal determinates of a system’s behavior”. In other words, a ‘good’ and
‘moral’ robot is one that takes moral and good actions. Thus, while we may not get true
teleology, functional teleology can suffice.
11. Human Rights and Artificial Intelligence--An Urgently Needed Agenda. Mathias Risse
Algorithms can do anything that can be coded, as long as they have access to data they
need, at the required speed, and are put into a design frame that allows for execution of
the tasks thus determined. In all these domains, progress has been enormous. The
effectiveness of algorithms is increasingly enhanced through “Big Data:” availability of
an enormous amount of data on all human activity and other processes in the world
which allow a particular type of AI known as “machine learning” to draw inferences
about what happens next by detecting patterns. Algorithms do better than humans
wherever tested, even though human biases are perpetuated in them: any system
designed by humans reflects human bias, and algorithms rely on data capturing the
past, thus automating the status quo if we fail to prevent them. 2 But algorithms are
noise-free: unlike human subjects, they arrive at the same decision on the same problem
when presented with it twice.
Also, philosophers have long puzzled about the nature of the mind. One question is if
there is more to the mind than the brain. Whatever else it is, the brain is also a complex
algorithm. But is the brain fully described thereby, or does that omit what makes us
distinct, namely, consciousness? Consciousness is the qualitative experience of being
somebody or something, its “what-it-is-like-to-be-that”-ness, as one might say. If there
is nothing more to the mind than the brain, then algorithms in the era of Big Data will
outdo us soon at almost everything we do: they make ever more accurate predictions
about what book we enjoy or where to vacation next; drive cars more safely than we do;
make predictions about health before our brains sound alarms; offer solid advice on
what jobs to accept, where to live, what kind of pet to adopt, if it is sensible for us to be
parents and whether it is wise to stay with the person we are currently with – based on a
myriad of data from people relevantly like us. Internet advertisement catering towards
our preferences by assessing what we have ordered or clicked on before is a mere
shadow of what is to come.
Future machines might be composed and networked in ways that no longer permit easy
switch-off. More importantly, they might display
emotions and behavior to express attachment: they might even worry about being
turned off, and be anxious to do something about it. Or future machines might be
cyborgs, partly composed of organic parts, while humans are modified with non-organic
parts for enhancement. Distinctions between humans and non-humans might erode.
Ideas about personhood might alter once it becomes possible to upload and store a
digitalized brain on a computer, much as nowadays we can store human embryos.
Already in 2007, a US colonel called off a robotic land-mine-sweeping exercise
because he considered the operation inhumane after a robot kept crawling along losing
legs one at a time. 5 Science fiction shows like Westworld or The Good Place anticipate
what it would be like to be surrounded by machines we can only recognize as such by
cutting them open. A humanoid robot named Sophia with capabilities to participate in
interviews, developed by Hanson Robotics, became a Saudi citizen in October 2017.
Later Sophia was named UNDP’s first-ever Innovation Champion, the first non-human
with a UN title.6 The future might remember these as historic moments. The pet world
is not far behind. Jeff Bezos recently adopted a dog called SpotMini, a versatile robotic
pet capable of opening doors, picking himself up and even loading the dishwasher. And
SpotMini never needs to go outside if Bezos would rather shop on Amazon or enjoy
presidential tweets.
If there indeed is more to the mind than the brain, dealing with AI including humanoid
robots would be easier. Consciousness, or perhaps accompanying possession of a
conscience, might then set us apart. It is a genuinely open question how to make sense
of qualitative experience and thus of consciousness. But even though considerations
about consciousness might contradict the view that AI systems are moral agents, they
will not make it impossible for such systems to be legal actors and as such own property,
commit crimes and be accountable in legally enforceable ways. After all, we have a
history of treating corporations in such ways, which also do not have consciousness.
Perhaps T. M. Scanlon’s ideas about appropriate responses to values would help.10 The
superintelligence might be “moral” in the sense of reacting in appropriate ways towards
what it observes all around. Perhaps then we have some chance at getting protection, or
even some level of emancipation in a mixed society composed of humans and machines,
given that the abilities of the human brain are truly astounding and generate capacities
in human beings that arguably should be worthy of respect.11 But so are also the
capacities of animals, which has not normally led humans to react towards them, or
towards the environment, in an appropriately respectfully way. Instead of displaying
something like an enlightened anthropocentrism, we have too often instrumentalized
nature. Hopefully a superintelligence would simply outperform us in such matters, and
that will mean the distinctively human life will receive some protection because it is
worthy of respect. We cannot know that for sure but we also need not be pessimistic.

There is an urgency to making sure these developments get off to a good start. The
pertinent challenge is the problem of value alignment, a challenge that arises way
before it will ever matter what the morality of pure intelligence is. No matter how
precisely AI systems are generated we must try to make sure their values are aligned
with ours to render as unlikely as possible any complications from the fact that a
superintelligence might have value commitments very different from ours. That the
problem of value alignment needs to be tackled now is also implied by the UN Guiding
Principles on Business and Human Rights, created to integrate human rights into
business decisions. These principles apply to AI. This means addressing questions such
as "What are the most severe potential impacts?", "Who are the most vulnerable
groups?" and "How can we ensure access to remedy?"
However, these laws have long been regarded as too unspecific. Various efforts have
been made to replace them, so far without any connection to the UN’s Principles on
Business and Human Rights or any other part of the human-rights movement. Among
other efforts, in 2017 the Future of Life Institute in Cambridge, MA founded around
MIT physicist Max Tegmark and Skype co-founder Jaan Tallinn, held a conference on
Beneficial AI at the Asilomar conference center in California to come up with principles
to guide further development of AI. Of the resulting 23 Asilomar Principles, 13 are listed
under the heading of Ethics and Values. Among other issues, these principles insist that
wherever AI causes harm, it should be ascertainable why it does, and where an AI
system is involved in judicial decision making its reasoning should be verifiable by
human auditors. Such principles respond to concerns that AI deploying machine
learning might reason at such speed and have access to such a range of data that its
decisions are increasingly opaque, making it impossible to spot if its analyses go astray.
The principles also insist on value alignment, urging that “highly autonomous AI
systems should be designed so that their goals and behaviors can be assured to align
with human values throughout their operation” (Principle 10). The ideas explicitly
appear in Principle 11 (Human Values) include “human dignity, rights, freedoms, and
cultural diversity.”
Russian manipulation in elections is a wake-up call; much worse is likely to come. Judicial
rights could be threatened if AI is used without sufficient transparency and possibility for
human scrutiny. An AI system has predicted the outcomes of hundreds of cases at the
European Court of Human Rights, forecasting verdicts with accuracy of 79%; and once
that accuracy gets yet higher it will be tempting to use AI also to reach decisions. Use of
AI in court proceedings might help generate access to legal advice to the poor (one of the
projects Amnesty pursues, especially in India); but it might also lead to Kafkaesque
situations if algorithms give inscrutable advice.
Any rights to security and privacy are potentially undermined not only through drones
or robot soldiers, but also through increasing legibility and traceability of individuals in
a world of electronically recorded human activities and presences. The amount of data
available about people will likely increase enormously, especially once biometric sensors
can monitor human health. (They might check up on us in the shower and submit their
data, and this might well be in our best interest because illness becomes diagnosable
way before it becomes a problem.) There will be challenges to civil and political rights
arising from the sheer existence of these data and from the fact that these data might
well be privately owned, but not by those whose data they are. Leading companies in
the AI sector are more powerful than oil companies ever were, and this is presumably
just the beginning of their ascension.
The Cambridge-Analytica scandal is a wake-up call here, and
Mark Zuckerberg’s testimony to US senators on April 10, 2018 revealed an astonishing
extent of ignorance among senior lawmakers about the workings of internet companies
whose business model depends on marketing data. Such ignorance paves the path to
power for companies. Or consider a related point: Governments need the private sector
to aid in cyber security. The relevant experts are smart, expensive, and many would
never work for government. We can only hope that it will be possible to co-opt them
given that government is overextended here. If such efforts fail, only companies will
provide the highest level of cyber security.
This takes me to my last topic: AI and inequality, and the connection between that topic
and human rights. To begin with, we should heed Thomas Piketty’s warning that
capitalism left to its own devices in times of peace generates ever increasing economic
inequality. Those who own the economy benefit from it more than those who just work
there. Over time life chances will ever more depend on social status at birth. We also
see more and more how those who either produce technology or know how to use
technology to magnify impact can command higher and higher wages. AI will only
reinforce these tendencies, making it ever easier for leaders across all segments to
magnify their impact. That in turn makes producers of AI ever more highly priced
providers of technology. More recently, we have learned from Walter Scheidel that,
historically, substantial decreases in inequality have only occurred in response to
calamities such as epidemics, social breakdowns, natural disasters or war. Otherwise it
is hard to muster effective political will for change.
Before this background we must worry AI will drive a widening technological wedge into
societies that leaves millions excluded, renders them redundant as market participants
and thus might well undermine the point of their membership in political community.
When wealth was determined by land ownership, the rich needed the rest because the
point of land ownership was to charge rent. When wealth was determined by ownership
of factories the owners needed the rest to work the machines and buy stuff. But those
on the losing side of the technological divide may no longer be needed at all. In his 1926
short story “The Rich Boy,” F. Scott Fitzgerald famously wrote, “Let me tell you about
the very rich. They are different from you and me.” AI might validate that statement in a
striking way.
12. From Machine Ethics To Machine Explainability and Back∗Kevin Baum, Holger
Hermanns and Timo Speith. Saarland University, Department of Philosophy
An important question arises: How should machines be constrained, such that they act
morally acceptable towards humans? This question concerns Machine Ethics – the search
for formal, unambiguous, algorithmizable and implementable behavioral constraints for
systems, so as to enable them to exhibit morally acceptable behavior.
We instead feel the need to supplement Machine Ethics with means to ascertain justified
trust in autonomous systems – and other desirable properties. After pointing out why this is
important, we will argue that there is one feasible supplement for Machine Ethics: Machine
Explainability – the ability of an autonomous system to explain its actions and to argue for
them in a way comprehensible for humans. So Machine Ethics needs Machine Explainability.
This also holds vice versa: Machine Explainability needs Machine Ethics, as it is in need of a
moral system as a basis for generating explanations.
13. The Ethics of Artificial Intelligence--Nick Bostrom, Future of Humanity Institute,
Eliezer Yudkowsky, Machine Intelligence Research Institute
AI algorithms play an increasingly large role in modern society, though usually not
labeled “AI.” The scenario described above might be transpiring even as we write. It
will become increasingly important to develop AI algorithms that are not just powerful
and scalable, but also transparent to inspection—to name one of many socially important
properties. Some challenges of machine ethics are much like many other challenges
involved in designing machines. Designing a robot arm to avoid crushing stray humans is no
more morally fraught than designing a flame-retardant sofa. It involves new programming
challenges, but no new ethical challenges. But when AI algorithms take on cognitive work
with social dimensions-cognitive tasks previously performed by humans—the AI algorithm
inherits the social requirements.
Transparency is not the only desirable feature of AI. It is also important that AI algorithms
taking over social functions be predictable to those they govern.
It will also become increasingly important that AI algorithms be robust against manipulation.
Robustness against manipulation is an ordinary criterion in information security; nearly the
criterion. But it is not a criterion that appears often in machine learning journals, which are
currently more interested in, e.g., how an algorithm scale up on larger parallel systems.
Responsibility, transparency, auditability, incorruptibility, predictability, and a tendency to
not make innocent victims scream with helpless frustration: all criteria that apply to humans
performing social functions; all criteria that must be considered in an algorithm intended to
replace human judgment of social functions; all criteria that may not appear in a journal of
machine learning considering how an algorithm scales up to more computers. This list of
criteria is by no means exhaustive, but it serves as a small sample of what an increasingly
computerized society should be thinking about.
Artificial General Intelligence (AGI)-- As the name implies, the emerging consensus is that
the missing characteristic is generality. Current AI algorithms with human-equivalent or
superior performance are characterized by a deliberately programmed competence only in
a single, restricted domain. Deep Blue became the world champion at chess, but it cannot
even play checkers, let alone drive a car or make a scientific discovery. Such modern AI
algorithms resemble all biological life with the sole exception of Homo sapiens.
To build an AI that acts safely while acting in many domains, with many consequences,
including problems the engineers never explicitly envisioned, one must specify good
behavior in such terms as “X such that the consequence of X is not harmful to humans.” This
is non-local; it involves extrapolating the distant consequences of actions.
A rock has no moral status: we may crush it, pulverize it, or subject it to any treatment we
like without any concern for the rock itself. A human person, on the other hand, must be
treated not only as a means but also as an end. Exactly what it means to treat a person as an
end is something about which different ethical theories disagree; but it certainly involves
taking her legitimate interests into account—giving weight to her well-being—and it may
also involve accepting strict moral side-constraints in our dealings with her, such as a
prohibition against murdering her, stealing from her, or doing a variety of other things to
her or her property without her consent. Moreover, it is because a human person counts in
her own right, and for her sake, that it is impermissible to do to her these things. This can be
expressed more concisely by saying that a human person has moral status.
It is widely agreed that current AI systems have no moral status. We may change, copy,
terminate, delete, or use computer programs as we please; at least as far as the programs
themselves are concerned. The moral constraints to which we are subject in our dealings
with contemporary AI systems are all grounded in our responsibilities to other beings, such
as our fellow humans, not in any duties to the systems themselves.
While it is fairly consensual that present-day AI systems lack moral status, it is unclear
exactly what attributes ground moral status. Two criteria are commonly proposed as being
importantly linked to moral status, either separately or in combination: sentience and
sapience (or personhood). These may be characterized roughly as follows:
Sentience: the capacity for phenomenal experience or qualia, such as the capacity to feel
pain and suffer
Sapience: a set of capacities associated with higher intelligence, such as self- awareness and
being a reason-responsive agent.
Others propose additional ways in which an object could qualify as a bearer of moral status,
such as by being a member of a kind that normally has sentience or sapience, or by standing
in a suitable relation to some being that independently has moral status.
Principle of Substrate Non-Discrimination: If two beings have the same functionality and
the same conscious experience and differ only in the substrate of their implementation,
then they have the same moral status.
Principle of Ontogeny Non-Discrimination: If two beings have the same functionality and
the same consciousness experience, and differ only in how they came into existence, then
they have the same moral status.
Parents have special duties to their child which they do not have to other children, and
which they would not have even if there were another child qualitatively identical to their
own. Similarly, the Principle of Ontogeny Non-Discrimination is consistent with the claim
that the creators or owners of an AI system with moral status may have special duties to
their artificial mind which they do not have to another artificial mind, even if the minds in
question are qualitatively similar and have the same moral status.
Even if we accept this stance, however, we must confront a number of novel ethical
questions which the aforementioned principles leave unanswered. Novel ethical questions
arise because artificial minds can have very different properties from ordinary human or
animal minds. We must consider how these novel properties would affect the moral status
of artificial minds and what it would mean to respect the moral status of such exotic minds.
a. Does a sapient but non-sentiet robot (zombie) have the same moral ststus as a full
AMA?
b. Another exotic property, one which is certainly metaphysically and physically
possible for an artificial intelligence, is for its subjective rate of time to deviate
drastically from the rate that is characteristic of a biological human brain. The
concept of subjective rate of time is best explained by first introducing the idea of
whole brain emulation, or “uploading.” “Uploading” refers to a hypothetical future
technology that would enable a human or other animal intellect to be transferred
from its original implementation in an organic brain onto a digital computer.
Principle of Subjective Rate of Time: In cases where the duration of an experience is
of basic normative significance, it is the experience’s subjective duration that counts.
c. For example, human children are the product of recombination of the genetic
material from two parents; parents have limited ability to influence the character of
their offspring; a human embryo needs to be gestated in the womb for nine months;
it takes fifteen to twenty years for a human child to reach maturity; a human child
does not inherit the skills and knowledge acquired by its parents; human beings
possess a complex evolved set of emotional adaptations related to reproduction,
nurturing, and the child-parent relationship. None of these empirical conditions need
pertain in the context of a reproducing machine intelligence. It is therefore plausible
that many of the mid-level moral principles that we have come to accept as norms
governing human reproduction will need to be rethought in the context of AI
reproduction.
To illustrate why some of our moral norms need to be rethought in the context of AI
reproduction, it will suffice to consider just one exotic property of AIs: their capacity
for rapid reproduction. Given access to computer hardware, an AI could duplicate
itself very quickly, in no more time than it takes to make a copy of the AI’s software.
Moreover, since the AI copy would be identical to the original, it would be born
completely mature, and the copy could begin making its own copies immediately.
Absent hardware limitations, a population of AIs could therefore grow exponentially
at an extremely rapid rate, with a doubling time on the order of minutes or hours
rather than decades or centuries.
But if the population grows faster than the economy, resources will run
out; at which point uploads will either die or their ability to reproduce will be
curtailed.
This scenario illustrates how some mid-level ethical principles that are suitable in
contemporary societies might need to be modified if those societies were to include
persons with the exotic property of being able to reproduce very rapidly.
The general point here is that when thinking about applied ethics for contexts that
are very different from our familiar human condition, we must be careful not to
mistake mid-level ethical principles for foundational normative truths. Put
differently, we must recognize the extent to which our ordinary normative precepts
are implicitly conditioned on the obtaining of various empirical conditions, and the
need to adjust these precepts accordingly when applying them to hypothetical
futuristic cases in which their preconditions are assumed not to obtain. By this, we
are not making any controversial claim about moral relativism, but merely
highlighting the commonsensical point that context is relevant to the application of
ethics—and suggesting that this point is especially pertinent when one is considering
the ethics of minds with exotic properties.
Superintelligence
Good (1965) set forth the classic hypothesis concerning superintelligence: that an AI
sufficiently intelligent to understand its own design could redesign itself or create a
successor system, more intelligent, which could then redesign itself yet again to
become even more intelligent, and so on in a positive feedback cycle. Good called
this the “intelligence explosion.”
Kurzweil (2005) holds that “intelligence is inherently impossible to control,” and that
despite any human attempts at taking precautions, by definition . . . intelligent
entities have the cleverness to easily overcome such barriers.” Let us suppose that
the AI is not only clever, but that, as part of the process of improving its own
intelligence, it has unhindered access to its own source code: it can rewrite itself to
anything it wants itself to be. Yet it does not follow that the AI must want to rewrite
itself to a hostile form.
Humans, the first general intelligences to exist on Earth, have used that intelligence
to substantially reshape the globe—carving mountains, taming rivers, building
 skyscrapers, farming deserts, producing unintended planetary climate changes. A
more powerful intelligence could have correspondingly larger consequences.
This presents us with perhaps the ultimate challenge of machine ethics: How do
you build an AI which, when it executes, becomes more ethical than you? If we are
serious about developing advanced AI, this is a challenge that we must
meet. If machines are to be placed in a position of being stronger, faster, more
trusted, or smarter than humans, then the discipline of machine ethics must commit
itself to seeking human-superior (not just human-equivalent) niceness.

14. Towards the Ethical Robot by James Gips, Computer Science Department, Fulton
Hall 460 Boston College, Chestnut Hill, MA 02467
Asimov’s three laws are not suitable for our magnificent robots. These are laws for slaves.
We want our robots to behave more like equals, more like ethical people. (See Figure
1) How do we program a robot to behave ethically? Well, what does it mean for a
person to behave ethically?
2) On what type of ethical theory can automated ethical reasoning be based? At first
glance, consequentialist theories might seem the most "scientific", the most
amenable to implementation in a robot. Maybe so, but there is a tremendous
problem of measurement. How can one predict "pleasure", "happiness", or "well-
being" in individuals in a way that is additive, or even comparable?
3) Deontological theories seem to offer more hope. The categorical imperative might
be tough to implement in a reasoning system. But I think one could see using a moral
system like the one proposed by Gert as the basis for an automated ethical
reasoning system. A difficult problem is in the resolution of conflicting obligations.
Gert's impartial rational person advocating that violating the rule in these
circumstances be publicly allowed seems reasonable but tough to implement.
4) The virtue-based approach to ethics, especially that of Aristotle, seems to resonate
well with the modern connectionist approach to AI. Both seem to emphasize the
immediate, the perceptual, the non-symbolic. Both emphasize development by
training rather than by the teaching of abstract theory.
5) Knuth [1973, p.709] put it well: It has often been said that a person doesn't really
understand something until he teaches it to someone else. Actually a person doesn't
really understand something until he can teach it to a computer, i.e., express it as an
algorithm. ... The attempt to formalize things as algorithms leads to a much deeper
understanding than if we simply try to understand things in the traditional way. That
as we build the artificial ethical reasoning systems we will learn how to behave more
ethically ourselves.
15. Can robots be responsible moral agents? And why should we care? Amanda
Sharkey, Department of Computer Science, University of Sheffield, Sheffield, UK
Patricia Churchland (2011) discusses the basis for morality in living beings, and argues that
the basis for caring about others lies in the neurochemistry of attachment and bonding in
mammals. She explains that it is grounded in the extension of self-maintenance and
avoidance of pain in mammals to their immediate kin. Neuropeptics, oxytocin and arginine
vasopressin underlie mammals’ extension of self-maintenance and avoidance of pain to
their immediate kin. Humans and other mammals feel anxious about their own well-being
and that of those to whom they are attached. As well as attachment and empathy for
others, humans and other mammals develop more complex social relationships, and are
able to understand and predict the actions of others. They also internalise social practices,
and experience ‘social pain’ triggered by separation, exclusion or disapproval. As a
consequence, humans have an intrinsic sense of justice.
By contrast, robots are not concerned about their own self-preservation or avoidance of
pain, let alone the pain of others. In part, this can be explained by means of arguing that
they are not truly embodied, in the way that a living creature is. Parts of a robot could be
removed from a robot’s body without it suffering any pain or anxiety, let alone it being
concerned about damage or pain to a family member or to a human. A living body is an
integrated autopoeietic entity (Maturana and Varela, 1980) in a way that a man-made
machine is not. Of course, it can be argued that the robot could be programmed to behave
as if it cared about its own preservation or that of others, but this is only possible through
human intervention.
16. Can machines be people? Reflections on the Turing Triage Test, Dr Rob Sparrow,
School of Philosophical, Historical & International Studies, Monash University.
Finally, imagine that you are again called to make a difficult decision. The battery system
powering the AI is failing and the AI is drawing on the diminished power available to the
rest of the hospital. In doing so, it is jeopardising the life of the remaining patient on life
support. You must decide whether to ̳switch off‘ the AI in order to preserve the life of
the patient on life support. Switching off the AI in these circumstances will have the
unfortunate consequence of fusing its circuit boards, rendering it permanently inoperable.
Alternatively, you could turn off the power to the patient‘s life support in order to allow
the AI to continue to exist. If you do not make this decision the patient will die and the
AI will also cease to exist. The AI is begging you to consider its interests, pleading to be
allowed to draw more power in order to be able to continue to exist.
My thesis, then, is that machines will have achieved the moral status of persons when
this second choice has the same character as the first one. That is, when it is a moral
dilemma of roughly the same difficulty. For the second decision to be a dilemma it must
be that there are good grounds for making it either way. It must be the case therefore that
it is sometimes legitimate to choose to preserve the existence of the machine over the life
of the human being. (He may choose the robot because it is more useful to the hospital than
the human patient. This test still doesn’t make the robot a moral person. Only sapience and
sentience will make a robot a moral person,)
17. Can a Robot Pursue the Good? Exploring Artificial Moral Agency, Amy Michelle
DeBaets, Kansas City University of Medicine and Biosciences, Journal of Evolution
and Technology - Vol. 24 Issue 3 – Sept 2014 – pgs 76-86
What then, might be necessary for a decision-making and acting entity to non-accidentally
pursue the good in a given situation? I argue that four basic components collectively make
up the basic requirements for moral agency: embodiment, learning, empathy, and
teleology.
First, I want to argue that artificial moral agents, like all moral agents, must have some form
of embodiment, as they must have a real impact in the physical world (and not solely a
virtual one) if they are to behave morally. Embodiment not only allows for a concrete
presence from which to act, it can adapt and respond to the consequences of real decisions
in the world. This physical embodiment, however, need not look particularly similar to
human embodiment and action. Robotic embodiment might be localized, having actions
take place in the same location as the decision center, in a discrete, mobile entity (as with
humans), but it might also be remote, where the decision center and locus of action are
distant in space. It could also be distributed, where the decision centers and/or loci of action
take place in. (Not convincing—a server is also an entity and is embodied).
This embodied decision-making and action must also exist in a context of learning. Learning,
in this sense, is not simply the incorporation of new information into a system or the
collection of data. It is adapting both the decision processes themselves and the agent’s
responses to inputs based on previous information. It is this adaptability that allows moral
agents to learn from mistakes as well as successes, to develop and hone moral reasoning,
and to incorporate new factual information about the circumstances of decisions to be
made. several places at once, as with distributed computing or multiple simultaneous
centers of action. The unifying theme of embodiment does require that a particular
decision-making entity be intricately linked to particular concrete action; morality cannot
solely be virtual if it is to be real.
Even if an embodied robot can learn from its own prior actions, it is not necessarily moral.
The complex quality of empathy is still needed for several reasons. First, empathy allows the
agent to recognize when it has encountered another agent, or an appropriate object of
moral reasoning. It allows the A.M.A. to understand the potential needs and desires of
another, as well as what might cause harm to the other. This requires at least a rudimentary
theory of mind, that is, a recognition that another entity exists with its own thoughts,
beliefs, values, and needs. This theory of mind need not take an extremely complex form,
but for an agent to behave morally, it cannot simply act as though it is the only entity that
matters. The moral agent must be able to develop a moral valuation of other entities,
whether human, animal, or artificial. It may have actuators and sensors that give it the
capacity to measure physical inputs from body language, stress signs, and tone of voice, to
indicate whether another entity is in need of assistance and behave morally in accordance
with the needs it measures. It may respond to cries for help, but it needs to be able to
distinguish between a magazine rack and a toddler in rushing in to provide aid. Empathy,
and not merely rationality, is critical for developing and evaluating moral choices; just as
emotion is inherent to human rationality, it is necessary for machine morality. (This is the
ethics of care logic—only emotions lead to action, to empathy)
What is sometimes forgotten in defining a moral agent as such, including in the case of
A.M.A.s, is that the entity must both be designed to be, and desire to be, moral. It must
have a teleology toward the good. Just as human beings have developed a sense of the
moral and often seek to act accordingly, machines could be designed to pursue the good,
even develop a form of virtue through trial and error. They will not, however, do so in the
absence of some design in that direction. A teleology of morality introduced into the basic
programming of a robot would not necessarily be limited to any one particular ethical
theory or set of practices and could be designed to incorporate complex interactions of
decisions and consequences, just as humans typically do when making decisions about what
is right. It could be programmed, in its more advances forms, to seek out the good, to
develop “virtual virtue,” learning from what it has been taught and practicing ever-greater
forms of the good in response to what it learns.
What is Not Required for Artificial Moral Agency?
Popular futurists Ray Kurzweil and Hans Moravec have argued that sheer increases in
computational processing power will eventually lead to superhuman intelligence, and thus,
to agency. But this is not the case. While a certain amount of “intelligence” or processing
power is necessary, it is only functionally useful insofar as it facilitates learning and
empathy, particularly. Having the most processing power does not make one the most
thoughtful agent, and having the most intelligence does not make one particularly moral on
its own.
Likewise, while a certain amount of rule-following is probably necessary for artificial moral
agency, rule-following alone does not make for a moral agent, but rather for a slave to
programming. Moral agency requires being able to make decisions and act when the basic
rules conflict with each other; it also requires being able to set aside “the rules” entirely
when the situation dictates. It has been said that one cannot truly be good unless one has
the freedom to choose not to be good. While I do not want to take on that claim here, I will
argue that agency requires at least some option of which goods to pursue and what
methods to pursue them by. A supposed A.M.A. that only follows the rules, and breaks
down when they come into conflict, is not a moral agent at all.
While a machine must move beyond simple rule-following to be a genuine moral agent
(even if many of its ends and goals are predetermined in its programming), complete
freedom is not necessary in order to have moral agency.
Some have thought that a fully humanoid consciousness is necessary for the development
of moral agency, but this too, may legitimately look quite different in machines than it does
in human beings.
Consciousness is itself elusory, without a clear definition or understanding of its processes.
What can be said for moral agency, though, is that the proof is in the pudding, that decisions
and actions matter at least as much as the background processing that went into them. In
deciding to consistently behave morally, and in learning from behavior in order to become
more moral, a machine can be a moral agent in a very real sense while avoiding the problem
of consciousness entirely.
Just as consciousness is used primarily as a requirement that cannot, by definition, be met
by any entity other than a human moral agent, so the idea of an immaterial soul need not
be present in order to have a moral agent. While the idea of a soul may or may not be useful
when applied in the context of human beings in relation to the Divine, it is unnecessary for
the more limited question of moral agency. A being also need not have a sense of God in
order to be a moral being. Not only is this true in the case of many humans, who may be
atheists, agnostics, or belong to spiritual traditions that do not depend on the idea of a
deity, but it is not necessary for moral action and the development of virtue. It may be
practically helpful in some cases for a robot to believe in a deity in order to encourage its
moral action, but it is by no means a requirement.
Yet, while the robots we build will not be subject to many of the same temptations as
human moral agents, they will still be subject to the limitations of their human designers
and developers. Robots will not be morally perfect, just as humans, even in the best of
circumstances, are never morally perfect.