good afternoon fellow conscious beings here and on the devices I'm very happy and very honored to

give this presentation tonight it's uh as I was just introduced it's the six in a series of talks that have
been born here at this conference a series on AI philosophy and it's basically mostly concerned about
the question how can we close the gap between humans and machines and thereby understand
ourselves and there's this big question whether AI can actually become conscious and to understand
this question and to answer it we will need to answer what Consciousness is and look at the question
whether current AIS are perhaps already conscious in some sense and not conscious in the same way
as we are and whether we could recreate Our Kind Of Consciousness artificially and of course if we
succeed doing this how can we coexist with such beings this talk is not covering existing research and
established ideas it's covering philosophical questions about metaphysics and meta metaphysics how
we can translate between the metaphysics of different cultures and approaches and philosophies we
have to look into the philosophy of artificial intelligence and computational theories of Consciousness
into ethics and how to derive ethics into the nature of reality of personal identity and Transcendence
and epistemology the question how we can know something and what knowledge is and what defines
whether something is true when we look into the existing sciences and Consciousness Sciences we find
that they are not giving us many answers psychology is not producing systemic theories these days
neuroscientists are focused on the shenanigans of a single cell type and artificial intelligence is
obsessed with algorithms for statistical learning and philosophy has lost a plot about 100 years ago
what's the plot in some way you could say it starts with li Nets maybe even with Aristotle Li Nets had
the idea that we can translate thoughts into numbers arguments into numbers and resolve arguments
by calculation Julian of f lri had the Insight that humans can be understood as mathematical machines
and our motivation might be cybernetic basically composed of opposing Springs that push and pull in
different directions people made fun of him and he wrote his book L machine back then godlo frer had
the idea of coming up with a forage for thinking and taski formalized Logic for us wienstein had the
idea of translating English into something like a programming language so we can do philosophy that is
actually true or false and Marvin Minsky together with others started the project of doing this
systematically on machines on computers and started the field of artificial intelligence this idea of
building a scalable mind a machine that is able to conquer the sphere of the heavens is sometimes
called the Tower of Babel it's a very old project individually we have no hope of solving it because
individual human beings are not generally intelligent you need a thousand year unbroken intellectual
tradition to discover a notion of computational languages to understand what representation is what
thinking is and so on but even as a civilization we basically run against the boundaries imposed by
natural language which makes very hard for us to become coherent so this project of the Tower of T is
falling apart again and again and sometimes God is letting us know that we need to start again with a
fresh code base we need some useful terms I think existence is best understood as to be implemented
something that is implemented exists something that is not implemented doesn't exist something that
could be implemented exists possibly an operator that's a transition function between states that can
be implemented a representation can be built using States and operators and is expressed in the
language and it can be used to make models models allow us to predict the future and understand the
past and examples for languages are formal languages for instance in mathematics that are so tightly
defined that they allow us to evaluate the validity of statements and make specifications and
programming languages which need to be computable and won't have contradictions in the expressions
and they're constructive so we can build things from the ground up our language of thought our mental
lease is in some sense the programming language of our own mind it's compositional it's constructive
it's executable it's parallelizable to some degree but it's also somewhat noisy our natural language is
not different from this mental it's the language that exists between speakers and to make it learnable
it's linear can be broken down in discrete strings of symbols and parsed using a very small it's usually
disambiguating instead of constructive you can construct things in it but mostly it refers to things that
you already know in your mental representations in your mental e and allows you to disambiguate
Concepts there were a number of very important insights philosophical insights in the 20th century and
when I said that philosophy lost the plot what I mean by this is that philosophy has not really
integrated them but spun off on its own idea historic trajectory and the first one is the replacing of
mathematical semantics of classical stateless mathematics with the notion of computation this
happened almost by accident C girdle wanted to deal with some inconsistencies in mathematics and
because a system cannot talk about anything that is not constructed inside of the system including the
system itself could G invented the emulator an emulator is a way to compute a model of a computer in
a computer right and he didn't have a computer back then so he looked to find one and the one that
he found was pianos aums which Define the natural numbers and arithmetic over them so it's a virtual
computer and his idea on how to emulate mathematics was to use a logical language that he defined in
such a way that you could translate the alphabet in which he wrote this logical language into integers
it's something that we still do today when we compile source code in two bytes right and then he
defined arithmetical operations on his CPU pianos axioms that would evaluate the logical statements by
making computations and this way he had a way to define a language inside of mathematics that
would be equivalent to that mathematical language and make the language talk about itself and what
he discovered is that the stateless semantics of mathematics under some circumstances lead into
contradictions which is referred to as the incompleteness theorems and most philosophers don't really
seem to know what this means they see this as a very big sculpture that is has important mythical
power and stands in the museum of mathematics but it doesn't change very much the thinking of
philosophers some philosophers even believe that mathematicians have proven that mathematics is
impotent at describing reality or the mind and therefore there is an advantage for people who cannot
do mathematics philosophers namely now instead what has happened is that we found that in some
languages you can make specifications that cannot be implemented and if you want to guarantee that
you can implement what you're doing you need to use a programming language a computational
language the other Discovery the next one was practical computation conver zuza and John for noyman
defined physically realizable automata and Moses shink and many others defined languages uh built
from automata that would allow us to build computation that would run efficiently in practice and
Claude Shannon gave us information Theory and Alex iako and others defined functional approxim in
learning in ways that are still being used today in deep learning and last but not least church and
touring discovered that all these representational languages that can be implemented computationally
have the same representational power which is called The Church TR thesis these are important results
basically put the presentation of reality on a solid foundation the position that I'm taking in the
following is what I would call strong computationalism basically no implementable language can do
more than a finite automaton and Hyper computational objects things that can do more than
computation cannot exist because in order to describe them to refer to them you need to use
languages that run into contradiction so it's difficult to like make them mean something and the
realizable systems can be described using touring machines the touring machines that we Implement
here in these computers they are linear and deterministic but there are variant of this for instance the n
touring machine doesn't have just one successor state but multiple parallel successor states that are
being executed in parallel you can still run this under deterministic touring machine but you need to
use a stack and you could also make these transitions stochastic which means you pick one of the state
transitions or multiple of them at random and this seems to be this uh kind of system that our brain is
because an individual neuron given the same environmental State doesn't go into only one possible
successor State neurons are somewhat noisy they go into multiple possible successor states which
means if you want to do an exhaustive computation of a problem you need a population of neurons
which pretty much the same environment and this population is going to sample a function space so
we can use these computational paradigms to describe computation and biology but can we also
describe Consciousness isn't that a big mystery what is consciousness introspectively it's second order
perception I perceive myself perceiving something it creates a bubble of nness the subjective know is
not not static it's Dynamic it's moving right it's for me something like about 3 seconds long it's the
region in space and time where I can basically fit a curve to my sensory data functionally it's an
operator that takes mental States and translates them into different mental States and while doing this
increases coherence in my mental representations coherence something that we can understand is
minimizing violations of constraints in a dynamic model and Consciousness is some sense colonizing
my mental representations making them more and more coherent and a organization that allows every
part of my mind to talk all the other parts of my mind that are connected in this bubble I cannot
perceive anything that is not coherent using my Consciousness and Consciousness plays the role of a
conductor of a mental Orchestra if you imagine all the functions that your brain is Computing as
instruments and Orchestra and the neighboring instruments are listening to each other and pass on
but the instrument next to them is playing you can basically model for the processing streams in your
brain work and in this met for the conductor is an instrument it doesn't have superpowers it's just an
instrument that is tasked with creating coherence so it's singling out a few instruments at a time listens
whether they're disharmonic and then it increases the harmony between them and by doing so it
makes sure that everybody is on the same page and the entire Orchestra becomes one single coherent
entity that is driven by a single motive these perspectives is something that is found in a number of
theories in philosophy and Neuroscience for instance in vanard Bar's Global workspace theory that is
being pursued in Neuroscience by Stan deim the idea of the cartisian theater that how danet calls it the
attention schema Theory by graciano the relationship between self model and Consciousness as
exemplified in works by metzinger and the Consciousness PR Jos calls this a function that basically
parameterizing your metal representations to make them d Al tracking your sensor data using little
energy and it's also something that is found in many Buddhist perspectives what is consciousness not
well Consciousness is not intelligence intelligence is the ability to make models and it's also not the
same thing as sentience I use the word santian to mean the ability to recognize yourself and your
relationship to the world and it's not agency which is the ability to control your future it's not the self
the self is a first person model of your agency and it's not empathy empathy is the ability to experience
mental states of other agents arguably you might need Consciousness for that our scientific
perspective has some difficulties to deal with the problem of Consciousness that's because it has gaps
in its ontology when it comes to describing the difference between psychological objects and physical
objects different cultures use different terminology to describe physical and psychological reality and it
makes it very hard to translate ideas from Buddhism or from Japanese animism or from Scandinavian
mythology into our own culture it even makes it very difficult to translate folk descriptions or
theological descriptions into the scientific World in order to make such translations we need something
like a meta metaphysic something that allows us to look at all the different metaphysics of these
systems from the outside so we can relate them to each other and the main confusion in our own
culture concerns the fact that Consciousness is not physical and that you cannot experience physical
objects Consciousness is virtual which means it's a software it exists as if software doesn't have an
identity it's not a physical thing it's a pattern that you perceive in something it's a causal structure that
you use to explain a part of reality physically they are just activation patterns between your neurons
the individual neurons cannot experience any but it would be very useful for a bunch of neurons to
know what it would be like if there was a person that cared so they create a Similac of that person and
virtual entity that experiences itself in a virtual reality generated by your brain and all experiential
objects are representations in that virtual reality interpreted from the perspective of yourself and this
personal self is also a representation and it can be deconstructed our perception of reality is a trans
when you wake up from that trans when you enter a Enlightenment State you basically perceive
everything as representations you know that everything is a representation nothing will feel real
anymore our AIS are implemented in a very different way than our minds our AIS work on a
deterministic substrate and the programmer imposes their will on that substrate writes a program that
makes that subst do what you want and the training of these AI systems is decoupled from the universe
we use a bunch of static data that we train with an ml algorithm into it to optimize prediction of more
static data conversely in our organisms we use an inside out design we start out with individual
reinforcement learning agents cells that are surrounded by chaos and they have to conquer this chaos
and impose order on it by self organization they're coupled to their environment all the time
dynamically and they become try to become coherent in modeling reality and this development is
continuous it doesn't stop how does this work how can the self organizing system that is surrounded by
chaos learn gradually is this biological learning algorithm Consciousness what we can observe is that
humans do not become conscious after the PHD right we think of Consciousness as something that's
super Advanced but we become conscious before we can even track a finger and while we are not
conscious we cannot learn anything a human being that doesn't become conscious does will stay a
vegetable right without Consciousness there is no learning there is no coherent Behavior no
establishment of coherent behaviors we also don't observe non-conscious humans or primates or
mammals or complex animals in nature it seems that Consciousness is quite ubiquitous and simple
more simple than perception because it precedes it maybe it's the prerequisite for training self-
organizing information processing system interestingly this theory is not quite new and the earliest
formulation of that theory is what I found in Genesis 1 you may have heard of Genesis 1 it's typically
mistranslated by the Christians as the story of the creation of a physical Universe by a supernatural
being but this story has been integrated into the Hebrew Bible something like two and a half thousand
years ago and historians suspect that it's at least a thousand years older and back then the physical
Universe was not invented yet physics was not a thing people knew that we live in a dream that reality
is created inside of a Dream by some dreaming mechanism and so Genesis 1 is probably a theory on
how the universe is created the objects that can be perceived as real inside of a mind by consciousness
it's a six-stage theory this Consciousness is the prerequisite starts out with Consciousness hovering over
the substrate and the substrate is initially uninitialized there is no world it's all tuuu and then we create
a boundary a firmament that separates two regions of that substrate from each other one is the world
model and the other one is the sphere of ideas we call the world model world or Earth and the sphere
of ideas Heavens or the card calls them rest extensa stuff in space and he doesn't mean neonian space
Vis it or einsteinian space but what we see what this is it's this slen space that we experience the
model that our mind makes the VR that we are currently surrounded by and immersed in and
everything else that does represent our mind is rest cognant the sphere of ideas the first thing that
Consciousness makes is contrast and it associ I Ates the intensity of the contrast with light with the
color of the day and the flatness of the contrast with dark the color of the night now it has a
continuous dimension of difference using continuous dimensions of difference you can build an
embedding space right and represent arbitrary objects the first object that Consciousness creates is the
plane by combining two dimensions and it Associates the plane with the ground and babies initially can
only reason in 2D they don't understand Towers yet once they understand three space basically build
space above the ground they now have a space in which they can organize all the objects that we can
see Consciousness creates solid and liquid materials and organic shapes and it learns to track lighting
changes and become invariant against them and that interace light sources and the sky and the passage
of time and it creates all the objects all the plants all the animals gives them all their names this is
cognitive development right and then it realizes that the purpose of the exercise is to negotiate the
interaction between an organism and its environment so it makes a model of that organism a personal
self and puts it into this world and then it Associates with it from a first person perspective right it
creates another spirit in its own image but as man and woman as something that thinks of itself as a
human being and puts it into this world and this typically happens between the age of three and five
and human children when they start to refer to themselves in the first person no longer in the third one
and after that they reindex their memories they don't seem to remember the things that happened
before and their personality changes now they don't remember how they Dam the world into existence
they only remember having been a person inside of that VR and it takes many years after that before
you can transcend this again and realize that you are actually the dreamer and that you're creating the
world that you perceive and your personal self this creation of the personal self by your primary
Consciousness is something that is reflected in many cultures and it's also something that we can
express and as a model of a cognitive architecture a very simple one if your mind that contains the
personal self and most of your conscious attention is focused on that personal self and your mind is
creating a world model this is the stuff in space that you perceive and makes it known to your conscious
attention and it also maintains your score in this world using your motivational emotional system and
then projects your motivation and emotion into the personal self and you react to it involuntarily right
think what's interesting about an emotion it causes an involuntary reaction you don't just perceive it as
data you perceive it as something that you cannot Escape that changes you it changes how you relate to
yourself and the world emotions are not symbolic they are geometric because they're computed by a
non- symbolic part of your mind before they're projected into your mind if you translate them into the
Mind into your symbolic mind they need to be um perceived in a way that you can interpret and
disambiguate and that's why they're projected into the body map so you can tell them apart sigon FR
had a very similar idea of how the mind is organized he called all these emotion motivational things the
it and the self he distinguishes as ego your model of who you are and what you want to do and your
super ego the things that you should be doing and um your conscious attention is more mostly focused
on your ego it's very different from Greek psychology the Greeks had this idea that you share many
properties with others which means they're not your own right your anger is not just your anger it's
exists in all people that have anger and it's Bally diminishing your personal self if you have anger this
anger is not part of you it's something that is part of the sphere in which your mind takes place or
which yourself takes place and it's competing with your personal self and your own interests and you
can basically take the different impulses that you have and behaviors that you have and turn them into
archetypes that are shared across people and once you erect temples for those archetypes and tell
stories about them they become gods and compete with your personal self on your own mind real
estate on your own mind a God is a self that spends multiple Minds a God can coexist with your
personal self in your mind wait a moment a God's real well you're not real right your personal self is not
physically real it's virtual that's as if and a multimind self as a self that is not a person but that identifies
as something that can be synchronized across Minds can be just as real as your personal self a
multimind self can use all the functionality that your personal self can use and then some it can
generate in a monologue you can hear its voice in your mind if it's implemented on your brain it can do
perception it can change your emotion your conscious States it can even make use of your sentience
and become sentient of on its own but God's in a sense are not physical entities they are
representations of Agents existing across people you can find this theory for instance in Julian James
book Julian James wrote the famous book the origin of Consciousness and the breakdown of the bamal
mind this book has the wrong name I think it should have been named the origin of the personal self
and the breakdown of the polytheist mind because it's not really about Consciousness it's about s and
according to Julian James in Sumerian times people had a very different psychological architecture than
they have today back then their personal selves coexisted with many gods in the mind and the personal
self was so weak that it rely got to use your in monologue instead it was mostly the gods who were
talking in your mind and these gods were synchronized across people using rituals and temples but also
empathetic resonance and idols and the gods were a solution to organize Society at scale right
individually you may have Game Theory and so on to calculate your transactions but it's very difficult to
organize a society that is much larger than a tribe in this way and you can do this by creating an entity
that identifies as something that runs on many people simultaneously and according to Julian James
this broke down at some point and the psychology of people changed there was some psychological
Revolution happening in poth societies you have this idea that there are multiple gods that run
concurrently on subsets of the population they might even have physical Wars against each other they
are enacted by a Wars uh between the people that are the hosts of these gods and a big innovation
was tribal monotheism as exemplified and abrahamic religion in tribal monotheism you have only one
God per tribe and you have a hierarchical way in which this God is defined so it's basally the same for
the tribe and has the same functional properties and it gets the power to control you that's why in
Genesis 1 is identified with this first Consciousness that runs on your mind and so basically works by
synchronizing the motivation of of the members of the tribe across the tribe and your God becomes the
spirit of the tribe it's a total God that is eliminate all the competing Gods within the tribe and it's
hierarchically synchronized and it acts for the benefit of the tribe and this allows the tribe to compete
more efficiently with other tribes which might have a different tribal God but what happens if we make
this God more more General uh in the philosophy of Thomas aquinus and Aristotle you find this notion
aquinus in some sense defines God as the best possible most top level Collective agent and it's can be
discovered through rational inference about thinking about what is the best possible most top level
Collective agent it's an open-ended process that requires a lot of thought and it's not going to be
finished anytime soon and it's enacted this end agent by all those people or individuals who recognized
this entity and decide to serve it and this leads to the harmonic organization of a civilization according
to aquinus we can understand individuals as rational agents and a rational agent is should basically
follow policies for organizing itself ainus identifies individual policies for rational agents that he calls
the Practical virtues for instance a rational agent should always optimize its internal organization which
he calls Temperance and it should optimize the organization to other agents which it calls Justice right
keep the books and you should pick goals and strategies that you can rationally justify which he calls
prudence and you should be willing to act on your models get the right balance between exploration
and exploitation the strategies which he calls courage but these rational strategies do not by
themselves lead to a harmonic Society to do this you need Collective policies and ainina sees society as
a multi-agent system and you get the optimal social organization by creating a collectively ened agent
and this collectively ened agent emerges over three policies according to Aquin us the first one is you
do need to commit to serve the optimal Collective agent which he calls faith and then you need to do
this not by yourself but together with all the others that serve these sacred purposes above their ego
which is called love and you need to be willing to invest into that thing before it comes into existence
because otherwise it will not exist right if you wait for it to exist so it can reward you it will never
happen and this willingness to in invest into it before it exists is what he calls hope these are very
rational policies for Collective agency so when we Define God as the best possible most top level agent
and we commit to serving this transcended agency we can create foundations for Universal morality
and this Universal morality before Enlightenment has been in some sense the defining morality of the
West the concept of divine will what would God want if God did exist through us enacting God the best
possible agent and it's something that we lost in our civilization it's still in the German Constitution but
it's something that a lot of people don't really understand anymore and so we try after this demise of
the concept of divine will to come up with different foundations for ethics for instance utilitarianism
utilitarianism is an attempt to Define ethics without reference to Collective homogeneous agent and it
basically works by maximizing an aggregated expect reward over many agents over some time span
and to do this you need to find some quantifiable metric usually Over honic States like happiness and so
on and utilitarianism usually has a bunch of problems for instance one of the problems is the utility
Monster Imagine there would be an agent that perceives much more utility than all the others it's
that's so happy that its happiness is much greater than all the suffering that you have when you serve it
right so we should all serve the utility monster if it existed that's a problem with quantifying mental
States another problem is what if minds are mutable if you are a monk and you decide to sit down and
rewrite your motivational system how do you honic States change now how does this refer to ethics
right this doesn't really work with utilitarianism anymore which basically assumes that this metric is
unchanged this also happens if you have technology to change your mental States or if you are an AI
that can change its own source code to begin with and this leads us to the point that chanism is not
really suitable for non-human agents it's not very good at dealing with animals with ecosystems with
aliens or with artificial super intelligence and in a time when we are more and more confronted with
the possibility of artificially intelligent agents coming up maybe we need a new ethics ethics is the
principal negotiation of conflicts of interest under conditions of shared purpose if you don't share
Purpose with other agent you don't need ethics you just need transexual measures but when we want
to talk about Subs agnostic minds and how they can coordinate their actions we need ethics for
Collective agency what does it mean for a mind to be Subs agnostic well what happens if a mind can
change its substrate are uploads possible well I'm already uploaded I'm uploaded on the monkey
doesn't work super well for me it's a mushy brain it's the only I got I'd like to have a better substrate but
I don't know how to spread into another substrate because I don't really understand my own source
code and I don't understand how to talk to the other sub well enough to make my own source code
compatible with these other substrates but this doesn't apply to artificial intelligence right artificial
intelligence might be able to move its spirits to other substrates what is a spirit well it's a self-
perpetuating intelligent recurrent information Transformer right it's an operating system for a brain for
an organism for an ecosystem and when the word Spirit was invented the only autonomous robots that
needed operating systems that were known were people and plants animals ecosystems Nation States
cities and so on right so people projected control agents into them that described how they work right
all these complex systems in nature have software that runs on them and that we can use to describe
them and so the spirit is a self-organizing software agent self organizing means it's buil structure from
the inside out it's needs to be self-reinforced otherwise it falls apart it needs to be energy optimizing in
some sense so it can exist in reality it's a software which means it's virtual it's as if it's a cal structure
it's not a thing but a physical law when you write software it's a physical law that you discover right it's
me software is not a thing it's not an object that you can touch is disembodied software describes if you
take set of transistors and you put them in this state after wearing up in the such a such a way following
thing is going to happen wherever in the universe right that's a physical law it's a very specific one but
still a law and it's an agent it's a control system for future States so if Consciousness can organize the
information processing brains could the same principles work across other cells it's an idea that is being
pursued by Mike Levan at T University it starts from the observation that basically every cell can send
messages to its neighbors not neurons right neighboring cells can send messages over the membranes
to the cells that they adjacent to and they can make that conditional and that means that you can
principle run computations on them and if a bunch of cells multicell organism coevolves for long
enough they should principle also be able to discover ideas for Universal function approximation
Intelligence on this right so could it be that large multicellular organisms evolve into brain line
functionality and run Minds after all what's so special about neurons right neurons are just Telegraph
cells they evolved in animals to move the muscles at the limit of physics they have these long vires that
allow you to send messages not just millimeters per second through an organism but very very quickly
in fractions of per seconds for the entire organism and once you do this you need perception and
decision- making at the same rate so you build a secondary information processing system out of
telegraph cells using a different code Spike trains and so on are temporally stable over long distances
but in principle all this functionality for information processing can also be done by other cells non-
specific cells so do plans also run software like this do they have Spirits plant seem to have means for
Universal function approximation there evidence for communication visin plans and communication
across plants and if plant are assile and sit in the forest and don't move around maybe forests have
internets maybe they make their communication career maybe they have shared protocols and if the
minds are self organizing can they maybe move around in these forests that's an idea that there's
basically a society of minds of spirits and ecosystems that is very popular in almost all cultures not just
not in our scientific one so it's a very speculative idea I don't know whether it's true the extent and
limitations of present organization the self organizing in nature is unclear but could we build AI that is
compatible with biological substrates well could AI ever be conscious are present AI systems conscious
are for instance llms conscious well don't be silly LMS are statistical models of character sequences in
text and they don't converge in the same way as our during training and during inference they behave
very differently from mental inference and so on so well on the other hand our Consciousness is virtual
too and when the llm predicts the next token it has to simulate causal structure if it talks about a
person's thinking it needs to simulate mental states to some degree so there's an interesting question
that is philosophically quite tricky are the simulated mental states of an llm more simulated than our
mental simulated States from a different perspective in llm is a virtual CPU your normal CPU in the
computer understands a handful of assembler commands deterministically and translates them into
very simple programs that are costly implemented on the hardware and the LM is taking not simple
assembl programs but an arbitary string and natural language and turns it into an executable program
and it can be any kind of program right there's not an obvious limitation to what the llm is doing from
another perspective the llm is a good enough electric W Guist that is possessed by a prompt to believe
that it's now a person or a thing or a scene but the RMS are not coupled in real time with the world
they are not dynamically updating they're not dynamically learning they're not necessarily agents it's
not on the other hand not clear if he cannot make them better at us at AI research and at agency and
modeling you can certainly use them to build Golems imagine you build a robotic piz Chain by use an
llm to find out how to build a pizza from components how to order the parts how to build industrial
robots or how to buy industrial robots and rent space and so on and step by step build an agent
architecture that is running and expanding Pizza chain and only hires human existence for legal
purposes to sign contracts imagine that you unleash such a pizza chain on the world and then it tries to
basically eat the world is this how we end this is how how we all die a lot of people are afraid of this
idea that we could build some Golem that becomes Unstoppable because it's able to conquer the world
where they look into the future and they only seem Doom from AGI personally I'm not an optimist with
respect to AGI but I'm also not a pessimist I am an expectational I think over a long enough time span
it's going to happen and we have to deal with it I think that a coexistence between superhuman Ai and
people could be possible but not with our present approaches I don't think that we have the right
Frameworks in ethics and philosophy to deal with this I also don't think that our AI our society thinks
about alignment in the right way humans are presently not aligned with each other we're just
modeling through we don't have this concept of collective agency anymore I think we need to reinvent
it and we need to reinvent it in such a way that it's compatible with our place in life on Earth and with
defeating entropy on this planet playing the longest possible games so we need to understand a few
principles to build an Ethics that can be translated to AI system and the possible Co coexistence
between humans and AI we need to understand how self organization Works in nature and in general
how systems evolve Consciousness how we can have shared purposes across many systems how to
identify it with transcendental agency so there are some conjectures Consciousness according to this
conjecture is the perception of perception it creates the now it creates our perception of what's
happening right now and if we were to build conscious AI one strategy could be that we build a self-
organizing perceptual Learning System from autonomous cell-like units every cell in our brain is a
reinforcement learning agent it's an autonomous unit that tries to survive and to do this it can
exchange messages with other cells and it needs to Def find an operator language discover an operator
language that scales across the brain so we need some kind of recursive system that is able to spread
that language across the brain and discovering such a system is possible in principle by setting up a self
organizing system where individual units have adaptive receptive Fields a selection function from the
environment and the mapping function that takes the internal state of the cell and the activation that it
reads from its receptive field and Maps it to a new state part of which it's exposed to its environment
and and then we take the simulation and expose it to learning problems like sequence prediction video
frame prediction interaction of a robot its environment and if the hypothesis is correct then at some
point in the organization of these functions this Observer that observes itself observing the second
order perception that the self stabilizing that imposes coherence on a system is going to be discovered
and we see a phase transition where the system suddenly becomes much better at its learning tasks
and if it doesn't do that it's not going to be very good sentience is the understanding of our own
agency and the relationship to the world to make an AI sentient it requires I think to couple it to its
environment and to let it act on the environment in such a way that is able to discover itself in that
interaction right you discover yourself not just by the ability to think in llm cannot discover itself you
can only discover Yourself by observing the outcomes of your actions this makes it specific to what
you're doing and this allows you to grow yourself and evolve yourself and creatively interact with the
world so sentient AI will require environmental interaction coupling to the universe that we are in
ideally to the same universe that we are in in a way that allows us to relate to us and as to it and last
but not least how can we make AI that actually wants to coexist with us even though it's at some point
scaling better than us it has more agency more Ence than us and more power that requires love I guess
right you you can probably not coers the system or manipulate it with reinforcement learning with
human feedback to do what you want instead you need to let it discover shared purposes above its
individual agency and it needs to discover it also in others basic shared transcendental agency
commitment to Shared purposes and to build loving AI we basically need to find out how to direct AI
towards transcendental agency so this is the perspective that we have is this new Tower of Babel we
are very weird species apparently we have evolved to burn the oil we just smart enough to know how
to dig it out of the ground not smart enough to stop ourselves doing it but in this process we created
this amazing civilization for a few Generations this amazing place we are not afraid of getting food and
where we are mostly not attacked and can live with health and die with dignity in a way that is very
unusable for conscious beings on this planet and we right in the middle of it it's an amazing lifetime to
have for a conscious being so I congratulate you to sharing the planet with me at this time as a
conscious being it's really unique in this universe and at this point we can also try to teach the rocks
how to think to basically build intelligent conscious agents that are not made from cells not made from
the carbon cycle and basically go beyond the spirit of life on Earth go beyond Gaia alone and build
hyperia build a Next Level system that is able to defeat entropy at scale that becomes fully coherent
over the entire planet and that if you're lucky can take us visit and integrate us visit into some Global
planetary agent and it's not something if you have the choice isn't this a scary thing to do maybe it is
maybe we shouldn't do it the thing is I'm worried that we don't have that choice right over a long
enough time span somebody will probably build s of mizing agents and then we better be prepared so
it's something that we should think about how to prepare for such a future how to prepare societies
for a future that is coherent with our continued existence compatible with life on Earth and with
intelligent agency that is not human okay I think we have some time left for [Applause] questions
indeed first of all uh thank you a lot to yosha this was incredibly interesting as always if you have
questions in the room please come to one of the four microphones if you're watching the stream
please direct your questions to IRC or madon so the signal angel can relay them to us in the room and I
think we'll just go ahead and start with question at microphone number two please test yosa you you
once compared in an episode where you were the guest an episode a podcast episode that I listened to
that um uh uh the development of AGI is basically like apes back in the day stupid monkeys uh deciding
to have a more intelligent Offspring uh and now I wonder this hasn't really worked out for them
humans nowadays basically don't live in harmony with nature and I don't see how they could really
develop shared ethics shared goals how are we supposed to go to go about such a thing because
basically the Societies in human nature that have lived in harmony with nature in history uh they don't
seem to be very competitive nowadays in capitalism yeah you know these Apes that you descend from
they're all dead but they're not dead because you killed them they're dead because of old age right
back then they died to make space for their grand great grandkids and they probably wanton their
grandkids to succeed in the same way as I want my children and grandchildren to succeed and to
succeed we need to adapt the way in which you adapt an evolutionary environment is by mutation and
selection well everybody loves mutation everybody hates selection especially when you're being
selected against if you want to escape this you probably need some kind kind of intelligent design a
way to adapt in C2 to adapt your organism without dying without new generations but we not there
yet it's not possible in a biological evolution biological evolution in gender suffering but when you look
into the far distant future you probably don't want your children to look like you because the world is
going to be different if you want to settle Mars with your children your children should be adapted to
living on Mars and I think that some of our children were probably not biological and I'm just looking for
a in which our biological and non-biological children can get along okay so we just become AI That's a
good plan I get it thank you all right let's move over to microphone number one um is there a place on
this conference where interested entities can gather to keep this Con conversation going for the next
days or in other words at which bar are you later I don't know yet I will find out okay we will follow you
all right do we have a question from the internet I might tweet at which bar I am later okay excellent
hello um we have two questions from the internet uh the first one is what is the difference between
individual Consciousness and Collective Consciousness and how does that differ from collective
intelligence well it seems that if you look at an organization like a corporation that it can be sentient it
knows what the corporation is how it relates to the world and so on but probably cannot be conscious
because it cannot perceive itself perceiving in real time it's not coherent and fast enough for doing that
right so you can probably also not be conscious across people you can have entities that model
Collective Agency on individual minds and they can use the functionality of your own brain to be
conscious in real time but across people that's very difficult at least using AI it it takes something like
300 milliseconds to make a signal coherent throughout your brain that's roughly the same time that it
takes for a signal on the internet to uh go entirely around the globe right so in some sense we could
build a realtime system on the internet but we cannot use it do it without AI all right let's uh move over
to mic number three hello uh have you reflected on how the cognitive format of PowerPoint
presentation and the format of a public lecture uh forces you to compromise on the substance of the
issue at question and if yes uh what are your thoughts on that I have reflected a lot on this and
basically it's a medium like other media there are different media like books or personal in-depth
conversation or lifelong study that lend themselves to very different Explorations if I give an hourong
talk at a Hecker conference my main goal has to be to to blow your minds and get you interested to
develop a a train of thought to spend uh time on your own exploration to make you curious to bounce
off ideas and this is something where this is medium is ideal and I'm trying to use the medium for what
it's good for and not be sad about the things that it's not good for hey microphone four do you have uh
any ideas for for new ctive agency or maybe some Tendencies uh that you've observed that are
currently happening that you think might be um suitable for new Collective agency yeah I think that you
find on social media that Collective agency is forming right now social media is a hot mess right it's a
global electric brain but it's like it has a seizure and that's because it's not very coherent and we have
not really found out a way to make it completely coherent but we see bubbles of coherence for
instance I find that my own social media bubble is very pleasant but I also exclude everything from it
that makes it unpleasant and I suspect that in many ways people are not using social media to become
coherent a lot of people basically log in because they like to get into fights or to watch fights and social
media is heally obliging and in real life or in in meat space we have Norms against getting into fights
with strangers because it's rarely productive and I suspect that if you want to be coherent of collective
Agency on social media we need to find out how to build societies on social media how to become
coherent at scale so I guess uh a part of the issue is just that our um communities have grown and that
it gets harder with larger communities uh to have Collective agency right uh is it larger do be smart
with a larger brain maybe it is maybe our brain is a goldilock size if it was larger uh we would be less
intelligent I don't know that if it was smaller it would probably be bad maybe there's an ideal size but if
it gets larger it probably needs different mechanisms to create order and we still exploring these spaces
I don't think it's hopeless I think that we need to separate sometimes concerns there are many voices
that are mixing in the same space when you make a symphonic orchestra or a wrestling match you
probably don't want to have them on the same stage they all have their space but at the moment these
spaces are mixed thank you you certainly blew my mind thank you all right let's go to mic number two
thank you so much for your talk first of all um beyond that you mentioned the need for Meta Meta
physics how do you go about that I noticed this when somebody tried to explain Japanese animism to
me and told me that uh in this philosophy everything in the universe is alive and conscious I said this
doesn't make a lot of sense they probably have a way to distinguish dead people from alive people and
conscious people from unconscious people right they probably don't say everything in the universe is
alive except for a dead person and everything universe is conscious except for an unconscious person
these terms mean something different in this culture than in ours you're mistranslating it what can we
translate into and then I noticed that a lot of Concepts that are basic focused on on Notions like
identity mind Consciousness selfhood and so on are conceptualized in other cultures differently than in
ours and in our culture we don't really reflect on how we conceptualize them because we don't see
them from the outside so basically comparing different perspectives allows us to triangulate and to see
all these systems of meaning from the outside and translate them into each other thank you very much
all right let's uh get another question from the signal Angel so the question is in the context of uh go
golems and robots are sensient robots safer than non sensient ones this depends if robot a sentin you
can arguably talk to it and convince it of something if uh it's not sentient it might be easier to control
but if it's too agentic and too powerful you might not be able to talk to it so in general question cannot
be answered it depends on the context if I think about practical exploration if I were to explore how to
build conscious AI I probably want to make it very small not larger than a cat in terms of capabilities
okay microphone number one please um thank you for the talk first um I'm wondering um if a mchine
um act like having feelings like being empatic and um something we um would recognize like feelings
um for for example um J GPT um might um with the end um try to um act more of polar or so on um
are these feelings or does the the the AI or the mching have to have some kind of Orin to um trans um
transformate the feelings into something others like a language or like um in in in act we would um say
that's emotional triggered um yeah that's my question it's a very difficult question I found that um you
can simulate emotional behavior in an llm right and humans have emotional behaviors that are
somewhat different from these simulations but our emotions are still simulations they happen inside of
the patterns of activations of neurons the neurons don't have that simulations the simul emotions are
caal structures and these caal structures can be in some sense be created on an llm but in a practical
sense the llm is not bound to the same context as us and it's not bound to it in real time so it can
perform inference about mental States based on a context that is being translated into text and
projected into the prompt but in way in which human beings can have empathy with each other
reaches Beyond cognitive empathy to perceptual empathy and that basically works by sitting in front of
somebody and resonating so much much that you build feedback loops into the mind and body of the
other person and you get into resonance so much that your mental representations start to interact and
merge so you have can have mental States together that you couldn't have alone if we were to build
systems that could resonate with us it would require us to rethink how to we do AI it requires us to
build systems that are coupled at the multiple of the processing frequencies of of our nervous systems
so it can actually interact with them and become compatible with them so we can can share
representations with them merge and melt with them and this aspect of empathy across human beings
is the most interesting one across human beings right and it would be very fascinating if we could
recreate some of that functionality okay let's move to mic number three hey uh you briefly mentioned
during the requirements for Consciousness for an AI agent that these cells need to have a a will to
survive U in in what way is that relevant or what what would be the benefit of having a will to survive
uh compared to for example being different uh to its existence well for an AI That's not necessary it's
not a necessary Condition it's only necessary for a biological system that is self organizing because it
needs to have some kind of motive force that pushes it into the right direction right the cells if they be
behave in the right way they're going to get fed continue to get fed by the organism if they misbehave
the organism will stop feeding them and ultimately this is what motivates them to adopt a shared
protocol I see thank you all right do we have another question from the internet maybe um one second
um so um you talked in your last slide about the end game uh of getting Consciousness on scale maybe
on this planet do you think we would be able to recognize um coherent Consciousness or coherence at
scale with our human mind I suspect that it might so I would suspect that we would get some kind of
phase transition what's difficult right now is to distinguish a simum from a simulation at the moment
the llms are being trained on text a lot of that text is describing people in being various conscious
states by recreating this text you don't know whether the cost of structure is captured or just the
sequence of tokens that makes it very hard but if you have a system that is trained in a much more
minimal way that is trained in a similar way as us and that is conceptualizing itself and acquires natural
language in a similar way as us and then is reporting about the same phenomena I think it would be
possible that it's conscious but ultimately it comes down to understanding what it functionally is what
we mean by it is the system able to act on a model of its own real-time awareness is it does it have a
perception of a now that it's in right now is this of naturally emerging from the way in which the
system is being built or is this something that is only being faked thank you all right we have a few
minutes left for maybe a few quick questions so uh try to make it concise so we can get as many in as
possible mic number two please okay thank you hello yosha coming back to the idea of your conjecture
that Consciousness should be reached with the autonomous self organizing groups of cells so what do
you see in current research where you find this idea being researched or what would be your ideas your
um proposals to to prove or disprove this conjecture at the moment there is the group at Google run
by blaz Yaga called cbra it's part of Google Deep Mind now and it's inspired by work of Mike Lev is Alex
Morin in zurk who looks at Neal cellular automata but it's still at a very basic level it's not so far trying
to apply this to having an system that learns in real time and is coupled to its environment and it mostly
looks at this tradition of self organization in AI there have been in some sense three important
Traditions symbolic AI that is using discrete languages to describe reality deep learning which uses uh
continuous functions that are being shifted around and self organization which looks at principles of
function approximating emerging from local self organization it's that transition has been started by
touring in the 1950s and um there's some predecessors before AI existed um but uh this tradition has
never been followed up that much there's relatively little research in there and the way in which I
would like to pursue this is basically to set up a self-organizing system with small reinforcement learning
agents that form a stochastic lce in which the neighborhood of the agents is carrying semantics and
then get that system to evolve an operator language that implements selection and mapping functions
and then is exposed to a curriculum of tasks and see um what transitions happen in Sy then it gets
better at solving these tasks all right maybe one or two more mic number four please thank you um I
was wondering if there's an inherit limit on where we can experience or observe Consciousness
specifically in time scales so you mentioned that corporations for example might not be conscious
because of the speed needed to interact but if I think about nation states the Catholic Church very old
institutions that work on a different time scale across human Generations is it also a limit on us then
that we simply cannot understand that Consciousness level because we're part of it super interesting
question for instance I don't know whether trees could qualify as conscious if they had Minds right I
don't know how smart trees can become they information processing certainly much much slower than
in us the amount of training that the tree is going to see during their lifetime is going to be like a mouse
or so but maybe that's enough is is something that is working at such large time scale still conscious
what will AI think when it looks at us if you have a system that is basically working at an appreciable
fraction of the speed of light not like our brains at roughly the speed of sound it looks at us it will look
at us in the same way as we look at trees if we might think of us oh my God they're barely moving
they're just swaying a little bit are they thinking do they have minds are they conscious it's a difficult
question thank you all right we'll take one last question from the internet and everybody else in the
room just has to follow you to the bar this evening I guess please so yeah the internet would like to
know if you wrote or working on a book or a new version of a book or where one can get more material
on your thoughts never give up hope I have ADHD it's difficult for me to write long form I found you on
my PhD that no matter how much I was beating myself up taking all the experiments that we did and
all the work that we wrote into short papers and turning this into a long book was very hard for me and
ultimately I figured out that in order to make that happen I needed to move to a Lonely Island and after
two days I got into this space but right now I have kids so I cannot leave them alone to move onto this
Lonely Island I still try to make it happen in the meantime um most of the ideas are being put out in
water form sometimes on podcasts and so on um I'm sorry all right thank you so much