Second-Order Cybernetics

Foresight Rather than Hindsight?

Future State Maximization As a Computational
Interpretation of Heinz von Foerster’s
Ethical Imperative Hannes Hornischer et al.
ARTIFICIAL INTELLIGENCE CONCEPTS IN Second-Order Cybernetics

Hannes Hornischer • University of Graz, Austria • hannes/at/hornischer.net

Simon Plakolb • University of Graz, Austria • simon.plakolb/at/uni-graz.at
Georg Jäger • University of Graz, Austria • georg.jaeger/at/uni-graz.at
Manfred Füllsack • University of Graz, Austria • manfred.fuellsack/at/uni-graz.at

> Context • Many AI and machine-learning techniques are primarily focused on past-to-future extrapolations of sta-
tistical regularities in large amounts of data. We introduce a method that builds on an in-action sampling of probes
from possible futures with preference for those that prove promising for maximizing the perceivable space of possibil-
ities. This foresight-oriented (rather than hindsight-oriented) method is particularly promising for handling non-linear
or abruptly emerging developments. > Problem • What von Foerster called the Ethical Imperative seems less strictly
derived from physical principles than other well-known concepts in his work. Regarding investigations in recent AI re-
search, however, it appears that the Ethical Imperative corresponds almost literally to the so-called principle of Future
State Maximization, a principle that lately has been applied successfully to a range of coordination and learning tasks.
> Method • We discuss the principle of Future State Maximization, as previewed by von Foerster, against a background
of a general need for tackling uncertain futures by way of modeling, and introduce three computational investigations
on different coordination tasks based on Future State Maximization. > Results • We show that the principle of Future
State Maximization corresponds to von Foerster’s Ethical Imperative and to constructivist principles, and that it lends
itself to opening up interesting new horizons for AI research. > Implications • The article suggests an interpretation
of how von Foerster’s Ethical Imperative can be understood as a foresight- rather than hindsight-oriented method
against a background of computer-based modeling and AI research. Furthermore, it shows that computer-based
methods conform well with the epistemology of constructivism. > Keywords • Agent-based modeling, coordination
and learning tasks, Ethical Imperative, Future State Maximization, Heinz von Foerster.

36

Introduction
« 1 »  When asked how to deal with the
future, to scientists it may not seem far-
fetched to suggest the use of a simulation
model. Using a model can help to navigate
uncertain futures. However, as we shall see,
there seems to be an even more fundamen-
tal and generic principle under which mod-
el use can be subsumed and which currently
is subject to a range of insightful scientific
investigations in artificial intelligence (AI).
To introduce this principle, we will take a
short detour and argue that model use is not
only a possible answer to our initial ques-
tion, but already a precondition for ask-
ing it. The reason for this is, as we propose
here, that space and time are constructions
emerging from the use of models. No model
means no space and no time, and hence also
no future.
« 2 »  Let us illustrate this with bacteria
in a Petri dish, containing patches with nu-
trients. These bacteria prosper where there
is food and they die where there is none. To
live and to die, in this case, does not neces-
sitate the use of a model.
« 3 »  Now, assume that some bacteria
have a basic facility for nutrient procure-
ment, such as the ability to follow a gradient
to higher concentrations. Obviously, these
bacteria have an advantage. However, this
simple ability already implies some funda-
mental concepts, which, as such, enlarge the
space of possible actions of the bacteria. As
a minimum, it implies two distinctions: “me
here”/“food there” and “now hungry”/“then
satiated.” In other words, it implies a spatial-
ly and temporally structured environment,
which cannot be had other than within a
virtual reconstruction of what is at stake in
the given situation. And this means that it
implies a model: clearly, a very basic model,
at first, a model, however, that includes self-
positioning – a self-model (Metzinger 2007)
– and that holds a conception of space and
time. The future, one of the aspects of time,
thus emerges from the use of models that
provide evolutionary advantage through
opening up additional, though maybe just
virtual, options.

Handling Editor • Alexander
CONSTRUCTIVIST FOUNDATIONRiegler • Vrije
s vol. Universiteit Brussel, Belgium
16, N°1
 Second-Order Cybernetics
« 4 »  This advantage becomes obvi-
ous as soon as the options include an
“acting”/“not acting” distinction, e.g., in
situations in which the option of acting in-
volves risks. A system with such a model
can pre-test and evaluate decisions about
whether to act, and it can anticipate con-
sequences of actions. Such an anticipatory
system (Rosen et al. 2012) would be able
to design strategies, for, e.g., food procure-
ment, in the safe virtual scenario of its
model. Maintaining a model may be costly,
though. However, when these costs get war-
ranted in evolutionary selection, anticipato-
ry systems can become complex, along with
the models they use. The number of options
increases as the model makes it possible to
consider more and more details, up to the
point where their number triggers deci-
sion problems and thus creates a higher-
order demand for guiding principles such
as norms, morals and imperatives, which
again are inconceivable other than within a
(next-order) model.
« 5 »  What we aim to point out with all
this is that the future, on a most general lev-
el, can be handled by generating additional,
albeit just virtual, options, as is exemplar-
ily done when deploying a model. This may
seem counterintuitive, at first, as it implies
mitigating uncertainty through the increase
of uncertainty, i.e., fighting fire with fire.
Nevertheless, what we claim is that an an-
swer to the question of how to handle the
future could, on a most general level, read:
Choose your options so as to increase the
number of possible further options.
« 6 »  Such a claim may sound famil-
iar to constructivists, as it coincides with
a maxim that second-order cybernetician
Heinz von Foerster called the Ethical Im-
perative: “Act always so as to increase the
number of choices” (Foerster 2003: 227).1
And, as mentioned, it also conforms to a
method that has recently gained much in-
terest in various areas of AI research, where
it is applied in coordination and learn-
1 | As pointed out on http://www.cybsoc.
org/heinz.htm (retrieved on 17 August 2020), von
Foerster later changed this wording to: “I always
try to act so as to increase the number of choices,”
which seems more in line with his idea of sepa-
rating ethics (which is only about the “I”) from
morality/morals (about the “Thou/You”).
Foresight Rather than Hindsight? Hannes Hornischer et al.
ing tasks, from cosmology (Bousso et al.
2007) to geosciences and biology (Martyu-
shev & Seleznev 2006), to computer science
(Charlesworth & Turner 2019). The method
has a physical generalization under the
name Causal Entropic Forces (CEF) (Wiss-
ner-Gross & Freer 2013; Hornischer 2015)
and a computer-based variant under the
term Fractal AI (Cerezo & Ballester 2018;
Cerezo, Ballester & Baxevanakis 2018). And
it has a somewhat older predecessor with
the name Empowerment (Klyubin, Polani
& Nehaniv 2005). In the context of this ar-
ticle, we will follow the suggestion of Henry
Charlesworth and Matthew Turner (2019)
and discuss it as Future State Maximization
(FSX).
« 7 »  A core aspect of the method is
what could be called a “pronounced orien-
tation toward the future.” Foresight, rather
than hindsight, is the unifying orientation
of its manifold variants, with which they
meet yet another core aspect in the interests
of von Foerster (Foerster, Mora & Amiot
1960; Foerster 1972). In contrast to other
AI and machine-learning techniques, FSX
is not primarily focused on past-to-future
extrapolations of statistical regularities in
large amounts of data, but builds on an
in-action sampling of probes from pos-
sible futures with preference for those that
prove promising for maximizing the per-
ceivable space of possibilities. With this,
the principle seems particularly promising
for handling non-linear or abruptly emerg-
ing developments, prone to critical phase
transitions or even singularities. In view of
forecasts on current global developments
like climate change, digital transformation
or the social and economic consequences
of Covid-19, FSX could prove useful as a
methodology for dealing with highly dy-
namic conditions, as we will show in more
detail at the end of this article.
« 8 »  Of particular interest is that FSX
seems to conform to fundamental construc-
tivist assumptions. Note that the above for-
mulation addresses the maximization of the
perceivable space of possibilities, which al-
ludes to the space that is perceivable within
the framework of the model used. As we
will discuss in more detail in the context
of the third experiment we introduce, FSX
builds on the use of a model that emerges
as a sort of eigenvalue – a temporarily stable
configuration in the sense of von Foerster
– from an iterated interplay of irritations
(for more details, see also Füllsack 2018). In
other words, FSX can be based on models
that emerge and are adapted in action. The
principle to maximize future options thus
resonates with the concept of operational
closure, illustrated by Maturana and Vare-
la’s (1987: 137) well-known metaphor about
the submarine pilot who, on being con-
gratulated for avoiding underwater reefs,
answers that all she did was to read certain
dials and maintain correlations between in-
dicators within the limits of the equipment
of the submarine. FSX, like this pilot, oper-
ates on the basis of its own on-board means.
« 9 »  In the rest of the article we will
proceed as follows: First, we outline the
derivation and justification of the Ethical
Imperative in von Foerster’s constructivist
framework and show how it may conform
to FSX. Afterwards, we introduce the prin-
ciple in more detail, briefly compare its vari-
ants, and illustrate its functioning using the
example of a simple grid-world-navigation
task. In the subsequent three sections we
present simulation experiments regarding
ƒ the group dynamics in game-theoretic
payoff optimization;
ƒ the coordination of a water-based robot
swarm in data gathering tasks; and
ƒ a revised version of the simple grid-
world example, which we updated to
conform with basic constructivist as-
sumptions.
« 10 »  These simulation experiments
are supposed to demonstrate the power of 37
the principle to yield interesting insights in
very different problem fields. In the context
of the last one, we review the implications
the principle may have for navigating un-
certain futures. In the conclusion, finally,
we venture into the risk of an interpretation
of how von Foerster could have seen his
Ethical Imperative on the foreground of the
FSX principle. Last but not least, this article
should be considered as yet another illustra-
tion of how computer-based methods con-
form to the epistemology of constructivism
(Füllsack 2013).
https://constructivist.info/16/1/036.hornischer
 Heinz von Foerster’s
Ethical Imperative
« 11 »  A central aspect of von Foerster’s
constructivist conception was the claim that
interacting dynamics generate stable and
lasting forms – “objects” – irrespective of
their initial conditions (Foerster 1981). In-
teractions run up to stabilities, even if there
ARTIFICIAL INTELLIGENCE CONCEPTS IN Second-Order Cybernetics
is no stable ground whatsoever for starting
them in the first place. They suffice to cre-
ate structure, patterns and forms. There
is no need to assume the existence of an a
priori structured mind-independent reality
(Riegler 2007). Order may simply emerge
from noise. This can be read as the core
proposition of constructivism.
« 12 »  Von Foerster took this insight
from mathematical as well as from psy-
chological findings, such as the insight that
simple recursive operations – i.e., operations
that feed their output back into themselves
as input for generating the next iteration’s
output – may have eigenvalues or eigen-
forms to which they converge, regardless of
their starting conditions. For example, the
operation op(x) = x / 2 + 1 converges to the
eigenvalue of 2, irrespective of the integer it
is started from. Analogously, in psychology,
Jean Piaget (1960) referred to the “object in-
variance” emerging in the course of an iter-
ated interplay of observation and movement
of a person trying to come to terms with her
environment. A small child, for instance,
may appear irritated by a sensory input, an
observation, and react to it with coordina-
38 tive movement in order to come to terms
with the irritation. The movement, how-
ever, changes the observational viewpoint of
the child, albeit slightly, and thereby again
triggers an observation that might induce
movement and so on. Step by step from
these sensorimotor recursions, the child,
just by reacting to irritations, may construct
stable concepts of the irritation – a ball
with which she can play, the food in front
of her, etc. – which eventually become part
of the child’s “external reality.” Hence, in the
course of her recursive computations (sensu
von Foerster), the child creates virtual sta-
bilities, to which higher-level recursions can
connect and subsequently mutually condi-
tion the emergence of sufficiently robust
conceptions. In “cognitive homeostasis”
(Chin 2007), these conceptions may eventu-
CONSTRUCTIVIST FOUNDATIONs vol. 16, N°1
ally be perceived as a “reality.” “The nervous
system,” von Foerster (2003: 225) writes, “is
organized (or it organizes itself) so that it
computes a stable reality.”
« 13 »  As known from the literature,
such a view on cognition may provoke a
strong objection that it is about solipsisti-
cally enclosed entities whose world is just
a world to themselves and who therefore
claim to be the center of the world. If what-
ever is outside is just a projection from the
inside, then this inside may be seen as all
that matters. Von Foerster was well aware
of the ethical implications of this objection.
Therefore, he stressed that solipsists may
project other solipsists with similar views,
and argued that when two such solipsists (be
they only mutual projections to each other)
insist on each being the world’s center, a
contradiction arises. Consequently, they will
argue with each other and thus interact. And
this bears the solution. By interacting, by re-
acting to mutual irritations, they may gener-
ate eigenforms, which eventually evolve into
stable “selves.” This led von Foerster (2003)
to formulate the equation, “reality = com-
munity.” Since no solipsist can exist without
others, each solipsist would do well to grant
the same rights to them.
« 14 »  In several of his writings, von
Foerster lets this reasoning directly be fol-
lowed by the Ethical Imperative. The logi-
cal link, however, between the arguments
against solipsism and the imperative itself
seems a bit vague, one may argue, in par-
ticular, if compared with other propositions
of his oeuvre, which are diligently derived
from physical principles. Von Foerster
himself repeatedly claimed that his consid-
erations on ethics were reflections inspired
by epistemology rather than the result of a
linear causal dependency deriving from the
premises of epistemology. “For if it were a
consequence it would be a necessity. I assert
however: it is not a necessity. It is an attitude
that we can select from amongst all possible
other attitudes” (Foerster & Bröcker 2002:
64). However, even as just an attitude, the
logical foundation of the conclusion may
appear somewhat weak, or at least not as
steadfast as one would wish. From the per-
spective of the FSX principle, however, it
seems that the vagueness could be mended
to some extent and a sort of logical reason-
ing could be interpreted into it via the ref-
erence to models, as argued above. We sug-
gest that conceiving constructions such as
the “self ” or the “other” as components of
a mental model, i.e., as a virtual conception
that creates new options, makes it possible
to derive the Ethical Imperative as a societal
extrapolation of the very basic principle that
is expressed in it, i.e., the principle of choos-
ing one’s options so as to increase the num-
ber of future options.
« 15 »  To see this more clearly, let us have
a look at some details of the FSX principle.
Future State Maximization
« 16 »  FSX is a quantifiable – and hence
computer-implementable – way of using
the information (sensu Shannon) a model
provides for testing and optimizing future
options. Its core aspect is a suggestion for
quantifying the degrees of freedom (or op-
tions) an agent (e.g., an organism) has in
controlling its environment based on a
model, with the model itself possibly emerg-
ing in the course of this quantification.
More precisely, the concept distinguishes a
specific value for an agent’s current state ac-
cording to its model of its environment and
the respective values of all possible states as
perceivable with this model, weighted by
their probability of opening further options.
That is, the current control over the agent’s
environment contrasts with the possible
control that it could reach, given an action
within its control. The agent uses these dif-
ferences for directing its actions towards
states that, compared to the current state,
maximize its options. To put it very simply,
the agent (or the organism), according to the
FSX principle, simply screens all current op-
tions that are perceivable with its on-board
means, compares their values, and then goes
for the option that promises most control,
with control here meaning possibilities for
further actions. By striving to maximize its
control, the agent simply selects actions that
increase its space of possibilities, thereby fol-
lowing a very simple, but universal principle
that is solely based on local information.
« 17 »  Technically, the principle can be
implemented in several ways. In the case of
empowerment, for instance (Klyubin, Pola-
ni & Nehaniv 2005), the maximum amount
of options is defined as the “maximal poten-
 Second-Order Cybernetics
Foresight Rather than Hindsight? Hannes Hornischer et al.

tial causal flow” (Ay & Polani 2008) from

an agent’s actuators (the means it has for
performing some action) to its sensors (the
means it has for sensing the state of its en-
vironment). This is formalized as channel
capacity in the sense of Claude Shannon
(1948), i.e., as the maximum mutual infor-
mation, measured in bits, that a probability
distribution of received signals on average
contains about the probability distribution
of the signals that were originally sent.2
« 18 »  Another implementation of the
FSX principle utilizes the connection be-
tween options and an information-based
entropy where the so-called causal entropy
is defined on a finite sequence of actions
(Wissner-Gross & Freer 2013). This entropy
reflects the amount of possible future evo-
lutions associated with a certain state of the
system, i.e., the amount of options available
to an agent starting from a certain state of
the system. From the causal entropy one
can derive the causal entropic force, which
points along the gradient of the causal en-
tropy. Letting an agent move along the caus-
al entropic force, it effectively maximizes
the amount of accessible future states.3 The
two implementations of the FSX principle,
empowerment and causal entropic force, are
generally closely related.
« 19 »  For an illustration and an exem-
plary implementation of the FSX principle,
consider the following scenario of an agent
being positioned on a two-dimensional fi-
nite grid world, in which five kinds of ac-
tions are possible: going North, going East,
going South, going West, or staying put. An
agent with a self-model that has a horizon of
one time step will be able to check these five
possibilities with respect to its neighboring
patches. Placed somewhere in the center
of the grid world, it will perceive that any
of these five actions is possible. However,
if placed at one of the edges, it will perceive
that it has one option fewer. Positioned in a
corner, it even has two options fewer. If this
agent is made to follow the empowerment
principle, it will thus move away from the
corners and edges, because avoiding them
means higher empowerment. Figure  1a
2 |  For a formal definition of empowerment,
see Klyubin, Polani & Nehaniv (2005).
3 |  For a formal discussion of causal entropy,
see Wissner-Gross & Freer (2013).
2 1.58
2.32 2.32
2.32
(a)
Figure 1 • (a) Empowerment agent moving within one-step horizon, empowerment values
indicated. (b) Empowerment agent within five-step horizon.
shows such an agent having moved two
steps according to the empowerment prin-
ciple, with the one-step-horizon empower-
ment values indicated.4
« 20 »  An agent in the same situation
with a larger horizon, however, i.e., with the
ability to look further into the future, will
move towards the center of the grid world,
since there, empowerment will be maxi-
mum (Figure 1b).
« 21 »  However, as easily imaginable,
beyond the limited complexity of a grid
world, the number of options an agent has to
compare can become vast. For this reason,
related concepts such as CEF and Fractal
AI build on computing not all, but a suffi-
ciently large sample of the options an agent
perceives with its model. The idea here is
to equip agents with probes, which we call
“walkers” and which, similarly to test pilots,
virtually rate options in the agent’s model.
Walkers can pre-screen the possibility space
of the agent by simulating model-based ac-
tions to determine further steps through re-
orientating based on the distribution of the
actions’ consequences.
4 
| Empowerment here is calculated as
the base-2 logarithm of visible actions, that is
E = log2(2n2 + 2n + 1) for patches without con-
straints, with n indicating the number of steps the
agent can look into the future.
4.91 4.7 4.39
2 5.36 5.21 5 4.7
2 5.58 5.43 5.21
5.81 5.73 5.58 5.36
5.91 5.88 5.81 5.67
5.91 5.93 5.91 5.83
5.91 5.88
(b)
« 22 »  To accomplish this task, the
density of walkers in different parts of the
(modeled) future horizon is altered by re-
distributing them in correspondence to the
consequences of options at a time t in the
future. The causal slice in Figure 2 (blue el-
lipsis) represents the set of consequences at
time t resulting from initial actions. If cer-
tain regions in this causal slice appear more
promising (green circles compared to red
circles) in regard to the maximization of op-
tions, the walkers are redistributed propor-
tionally to the assessed prospectivity, i.e., a
fraction of them is cloned to the new region
and deleted at the point where they are. The 39
size of the fraction is determined based on
how much exploration over exploitation
is to be considered. In this way, it can be
guaranteed that walkers prevalently screen
promising futures while not completely ig-
noring the less promising ones, which may
prove advantageous at another moment in
time (see Code 1).
« 23 »  In the simple grid-world example
of Figure 1, as well as in the examples in the
next two sections, the on-board means of the
agent, i.e., the walkers and their determin-
ing parameters, are specified at setup. Con-
ceptionally, however, the on-board means
could be seen as emerging in the course
of an agent’s interactions with its environ-
ment, i.e., as “constructed” stabilities in the
sense of von Foerster (1981). The construc-

https://constructivist.info/16/1/036.hornischer
 tion, i.e., the development of parameters
Consequences of actions at time t such as the number of walkers, the number
ce at time t of steps they can look into the future, and
a usal sli
C their adaptability (i.e., the variability of the
size of the fraction of them being cloned in
function of exploiting or exploring new op-
tions) can be conceived in concordance with
the FSX principle. In other words, the model
and its components evolve if this evolution
ARTIFICIAL INTELLIGENCE CONCEPTS IN Second-Order Cybernetics

e
increases the agent’s fitness, which in the

on
lc
present context amounts to the increases in

a
Walkers

us
its number of choices, as will be illustrated

Ca
in Example 3, where we will revisit the grid-
world example and show that an evolution
Initial actions A Initial actions B along the FSX principle can be initiated
from very basic starting conditions.
x (0) « 24 »  Surprisingly, the relatively simple
FSX principle can be applied to a wide range
Figure 2 • Causal cone as viewed by the agent x at time step 0, with walkers sampling future of tasks, which all focus on the one principle
paths indicated in green (prospective) and red (detrimental). of maximizing the number of possible fu-
tures. Alexander Wissner-Gross and Camer-
on Freer (2013), for instance, showed that it
easily solves the classical problem of balanc-
ing a rod upright on a moving cart. Sergio
// INITIALIZATION: Create N walkers with copies of the system’s state: Cerezo and Guillem Ballester (2018) pre-
FOR i := 1 TO N DO BEGIN sented insightful examples of steering agents
// Walkers start at the system’s initial state: through mazes or coordinating simulated
Walker(i).State := System.State spaceships with a hook on a rubber band,
// Take walker’s initial decision: forming a chaotic oscillator that is highly
Walker(i).Initial_decision := random values sensitive to small changes in initial condi-
END tions, see https://youtu.be/HLbThk624jI. In
// SCANNING PHASE: Evolve walkers from time = t to t + Tau in M ticks: another setting Cerezo, Ballester and Spiros
FOR t := 1 TO M DO BEGIN Baxevanakis (2018) applied the principle to
// PERTURBATION: 55 Atari 2600 games from OpenAI Gym,
FOR i := 1 TO N DO BEGIN https://gym.openai.com, a feat that lately
// At first tick use the stored initial decision has become a widely used benchmark for AI
40 IF (t=1) THEN methods. They show that an algorithm with
Walker(i).Degrees_of_freedom := Walker(i).Initial_decision this principle can learn to play these games
ELSE faster and more efficiently than human play-
Walker(i).Degrees_of_freedom := random values ers and some of the most advanced deep
// Use the simulation to fill the other state’s component: learning methods. Christian Guckelsberger,
Walker(i).State := Simulation(Walker(i).State, dt := Tau/M) Christoph Salge, and Julian Togelius (2018)
END used the principle to generate non-person
END characters with lifelike behavior in computer
// DECIDING PHASE: games, and Charlesworth and Turner (2019)
Best := ArgMax(Reward(Walker(i).State)) deployed it successfully for the simulation
Decision := Walker(Best).Initial_decision of swarms, showing that it may provide an
explanation that is even more fundamental
Code 1 • Pseudo code as suggested by Cerezo & Ballester (2018: 25) as a “simple starting point” than the well-known swarm principles of
for the implementation of the FSM procedure, in which walkers are cloned in regard to the separation, alignment and cohesion.
prospectivity of their positions in the “causal slice” (i.e., the reward they receive at time t). Note, « 25 »  In order to illuminate the prin-
however, that, as mentioned, FSM is applied in various versions in different disciplines. Imple- ciple further, in the next two sections, we in-
mentations can differ significantly. troduce two examples of our own research,
before we present a revised version of the
above grid-world example with a more ba-

CONSTRUCTIVIST FOUNDATIONs vol. 16, N°1

 Second-Order Cybernetics
sic setting, suggesting that the FSX principle
conforms to the epistemology of construc-
tivism. Note that not all problems can be
easily captured in terms of immediate util-
ity, which makes defining a metric with re-
spect to which the FSX principle is made to
maximize options one of the more challeng-
ing tasks in applying it. For this reason, we
chose the examples in regard to the diversity
of their problem contexts.
Example 1: Group dynamics
and the emergence
of leadership
« 26 »  In our first example, the FSX
principle is applied to a set of social-psycho-
logical experiments on collective behavior,
which were originally designed to investi-
gate the emergence of leadership in groups
of humans (Boos et al. 2014). We imple-
mented these experiments in an FSX-based
computational simulation model to see if
the results could be reproduced.5
« 27 »  In the human-based version, each
of the ten participants (“players”) were asked
to move computer-generated avatars on a
virtual playing field presented to them on
a screen hidden from the view of the other
players. All verbal, visual or aural interaction
among the players was prohibited. The play-
ing field is depicted in Figure 3. Avatars are
represented by black circles, with a player’s
own avatar being displayed larger than the
rest. Players started out to move their avatars
at the center hexagon and were free to move
from their current position to any adjacent
hexagon. The number of moves they can
make is limited. The small tails in the repre-
sentation in Figure 3 indicate the directions
from which an avatar was moved towards its
current position. Six hexagons are marked
with a Euro symbol. These are considered
reward fields and have to be reached within
the allowed number of moves. If an avatar
reaches a reward field, its player is rewarded
0.5 points, which directly translates to the
amount of money a player receives at the end
of the experiment. After reaching a reward
5 |  The results of this experiment are current-
ly submitted for separate publishing and hence are
subject to publication restrictions. Therefore, de-
tailed data cannot be shown here.
Foresight Rather than Hindsight? Hannes Hornischer et al.
Figure 3 • Virtual play field in the experimental setup (Boos et al. 2014).
field, the avatar remains on that field until
the end of the game. The reward for each
player is multiplied by the number of avatars
on a reward field, which provides an incen-
tive for players to coordinate their action and
reach the same reward field.
« 28 »  In order to introduce a mecha-
nism allowing for the emergence of leader-
ship, two of the ten players are told that one
of the reward fields, marked with two Euro
symbols on their screens, yields the double
reward. Neither these two “informed play-
ers” nor the others are aware of the informa-
tional imbalance. For all, however, it is ad-
vantageous to reach reward fields to which
more than just one avatar is moving. The
aim of this setting was to see if the informed
players, who were not allowed to communi-
cate with others, would be able to “lead” the
uninformed majority towards the special
reward field just by moving their avatars.
As the experiments by Boos et al. (2014)
showed, this proved possible.
« 29 »  To test the applicability of the
FSX principle to this experiment, the set-
ting was implemented with software-based
agents making decisions based on Future
State Maximization. As in the human ver-
sion, ten agents were made to start out at
the center hexagon with a finite number
of moves available to navigate through the
playing field. Two of them could see one of
the six reward fields as yielding a double
reward, and the rewards were multiplied
by the number of agents reaching a reward
field.
« 30 »  In contrast to the experiments
with human players, the software agents
were made to choose moves in correspon-
dence to the FSX principle, with the num-
ber of choices being defined as the number
of states an agent can be in, i.e., the number
of hexagonal fields an agent can visit at least
once using its remaining available moves. 41
To account for these choices, agents could
sample their environment with large num-
bers of exploratory random walkers, which
rated the adjacent fields in function of the
amount of future options. The number of
unique fields visited by a walker are used as
an approximation for the amount of future
options associated with the respective adja-
cent field.
« 31 »  Since, obviously, software agents
have no intrinsic understanding of money
or Euro signs, a different metric for the in-
centive to move towards reward fields had to
be used. In our experiments, successful sam-
pling activities, i.e., those that indicate or
reach a reward field, were rewarded with an
additional move the agent in question was
allowed to make. Consequently, agents con-
https://constructivist.info/16/1/036.hornischer
 tain extent, appear to be reproducible with « 37 »  The swarm has the goal of col-
0.3 the FSX principle. lecting measurements of environmental pa-
« 33 »  This proposition was further rameters of the swarm’s surroundings, such
Relative frequency f

substantiated by the finding that, in most as temperature, turbidity or oxygen concen-

0.2 games, human and computer-based, either
no or nearly all uninformed agents reached
the double-reward field (see Figure 4 and 5
0.1 for results with humans). Obviously, hu-
mans as well as FSX agents in this setting
ARTIFICIAL INTELLIGENCE CONCEPTS IN Second-Order Cybernetics
tration of the water. The swarm as a whole
has the ability to move as a collective based
on the general goal of increasing available
choices, with choices defined as the swarm-
wide variance in collected measurements of

tend to move in groups, so that once some environmental parameters.

0 uninformed agents find the double-reward,
None €€
Type of reached reward field
Figure 4 • The relative frequency of “unin-
formed” (not-leading) players positioned
at the end of the game, with (€) indicating
at regular reward fields, (€€) at the higher
reward field, and “None” anywhere else. The
result for regular reward fields is averaged
over all five regular reward fields (Boos et al.
2014).
0.6
Relative frequency f
€ the bulk of them will do so, as well. The FSX
principle reproduces this particular behav-
ior, too.
« 34 »  Nevertheless, in highlighting the
close match of human and FSX-agent be-
havior, we do not want to imply that human
behavior can be expressed by equations or a
small set of behavioral rules, in general. Our
experiments are meant to show that the FSX
principle, and thus implicitly von Foerster’s
Ethical Imperative, may provide a general
method for mimicking and reproducing as-
pects of human behavior.
Example 2: Coordinating
« 38 »  If this swarm is located in a uni-
form environment without any significant
deviations in environmental parameters,
every robot in the swarm collects the same
measurements and the swarm as a whole
has little variety in information to choose
from. In a diverse environment, however,
the swarm would collect a variety of dif-
ferent measurements. The behavioral rule
that dictates the movement direction of the
swarm is based on maximizing the variance
in measurements of environmental param-
eters the swarm investigates. The resulting
emergent collective behavior lets the swarm
move towards the most diverse areas, i.e.,
towards areas with the largest variance in

measurements. Needless to say, the swarm

0.4
an aquatic robot swarm
« 35 »  In the second example, we ap-
0.2 plied the principle of maximizing future
choices to an aquatic robotic swarm (first in
software and later in physical robots) with
0 the task of long-term monitoring and ex-
0 1 2 3 4 5 6 7 8 ploration of a dynamic environment (Horn-
Number of uninformed agents on €€ field ischer et al. 2020).
42 « 36 »  Marine environments are a chal-
Figure 5 • The relative frequency of finding lenging target location in the field of robotics
a number of uninformed players reaching since reliable communication underwater is
the higher reward field by the end of a game non-trivial and can be energy consuming.
(Boos et al. 2014). Therefore, long-term operation dictates an
energy-efficient design. With this in view, a
sidering reward fields in terms of additional swarm of aquatic robots was programmed
choices faced more options in terms of FSM. to coordinate in a self-organizational man-
« 32 »  This procedure proved success- ner, i.e., without the presence of a central
ful in making agents find the reward fields. coordinating entity, with in-swarm com-
What is more, in both investigations, in the munication limited to local communica-
computer-based as well as in the human- tion only. The swarm consisted of 50 to 100
based, the relative frequency of finding an identical robots (designed after Donati et al.
uninformed agent at the special (double- 2017), each of which was endowed with the
yielding) reward field was significantly high- ability to take measurements at its position
er than that of finding it on regular reward and to communicate with its direct neigh-
fields. Obviously, deploying “informed” bors. Furthermore, each robot has a sense of
agents to steer collective behavior can have direction of incoming messages from other
significant effects, and these effects, to a cer- robots.
has no intrinsic understanding of the rel-
evance of its different measurements. In the
following, we describe the communication
mechanism, the agents’ behavioral rule, and
the resulting emergent collective behavior of
the swarm.
« 39 »  Figure  6 shows schematic il-
lustrations of the swarm. Squares repre-
sent robots, which randomly, in time, send
messages to their nearest neighbors, who
in their turn send the messages on to their
neighbors (Figure  6a–d), ultimately result-
ing in radially outward propagating mes-
sages. Figure 6e shows several trajectories of
a message. Each message contains measure-
ments of environmental parameters, which
a robot has locally sampled. When receiving
a message, each robot appends its own mea-
surements to it and sends it to its neighbors.
Over time, in this way, messages grow and
all robots receiving the message have access
to the measurement values of the sending
robots. This communication mechanism is
described as pseudo code under the INFOR-
MATION GATHERING PHASE in Code 2.
« 40 »  Figure  6f highlights one of the
robots in the center of the swarm together
with three trajectories of messages initiated

CONSTRUCTIVIST FOUNDATIONs vol. 16, N°1

 Second-Order Cybernetics
Foresight Rather than Hindsight? Hannes Hornischer et al.

in the periphery of the swarm, which sub-

sequently propagated inwards, plus a cone
of possible future states to choose from. The
robot thus receives messages from all direc-
tions, which here correspond to the infor-
mation mediated by the exploratory random
walkers as deployed in the examples above.
The robot uses those incoming messages for
sampling the environment and evaluating
R
it with respect to the variance of measure-
ments6 in order to determine the direction (a) (b) (c)
associated with the largest variance in acces-
sible information (see EVALUATION PHASE in
Code 2). This principle is applied by all ro-
bots in the same way. Each robot determines
the direction of increasing variance in infor-
mation as it occurs from its perspective, i.e.,
its own preferred direction, and communi-
cates it repeatedly to its nearest neighbors,
thereby aligning its own preferred direction
slightly towards the preferred direction of its
neighbors. Over time, the preferred direc- (d) (e) (f)
tions of all agents converge to a common
preferred direction close to the average of all Figure 6 • (a–d) The communication mechanism of the swarm. When agents (black squares)
preferred directions (see COLLECTIVE DECI- send a message, it gets relayed by neighboring agents, which ultimately produces a wave-like
SION PHASE in Code 2). propagation of messages through the swarm. (e) Three exemplary trajectories of messages,
« 41 »  This mechanism enables the which are relayed from one robot to the next, respectively. (f) The agent underneath the cone
swarm to collectively decide which direc- receives three exemplary messages, with the cone indicating the possible future states of the
tion provides a maximum of accessible agent to choose from.
information. Figure  7 shows the swarm in
an environment containing two domains
of different environmental parameters
(including a small noise), represented by
different background colors. Each square
represents a robot and the arrow indicates
its individual preferred direction. Most ro-
bots in the yellow domain have a preferred 43
direction towards the red domain, whereas
most robots in the red domain have a pre-
ferred direction towards the yellow domain.
As a consequence, the overall preferred
direction of the swarm is towards the bor-
der of the two domains, since it provides
the largest variance in measurements. Fig-
ure 7b shows the positions of the center of
the swarm over time, i.e., the average posi-

6 | The variance an agent calculates during

INFORMATION GATHERING is defined as
1 n (a) (b)
V dir
k = n / (m kj - m k) 2
j=1
where agent k received n measurements m from Figure 7 • (a) A swarm in an environment containing two different domains of environmental
direction dir, with m j d R and m k the arithme- parameter values, shown in yellow and red. Each square represents a robot and the arrow its
tic mean of all measurements associated with the respective individual preferred direction. (b) The “averaged” positions of the swarm moving
respective dir. towards the red domain and eventually along the border of the two domains.

https://constructivist.info/16/1/036.hornischer
 // INFORMATION GATHERING PHASE: Communicate measurements within the swarm while the swarm maintains its distribution
FOR t := 1 TO information_gathering_time DO BEGIN around the locations associated with the
// Agents randomly in time initiate sending a message: largest amount of accessible information.
IF (random_value < probability_to_initiate_sending_message) THEN « 43 »  This algorithm was also imple-
message := sensor_readings_of_agent mented on physical robots and successfully
send message to neighboring agents applied under laboratory conditions in a
// Briefly ignore incoming messages to avoid repeatedly receiving setup conceptually equivalent to the setup
// the same messages described here. Note that this article only
addresses the task of collective decision
ARTIFICIAL INTELLIGENCE CONCEPTS IN Second-Order Cybernetics

ignore incoming messages for a brief period

ELSE IF (received a message) THEN making, while other common challenges
append sensor_readings_of_agent to messages in robotic swarms such as actuation, colli-
send message to neighboring agents sion avoidance, and prevention of loss of
// Evaluate received message with respect to variance and direction members are not in the scope of this work.
variance := variance( measurements_in_received_message) Further details on functionality and limits
direction := direction_from_which_message_was_received of the algorithm as well as implementation
list_of_measurements(direction) ← append received measurements and results can be found in Hornischer et al.
// Briefly ignore incoming messages to avoid repeatedly receiving (2020).
// the same messages
ignore incoming messages for a brief period
ELSE Example 3: Learning
wait to behave
END
// EVALUATION PHASE: Determine preferred direction of agent « 44 »  The previous two examples in-
FOR direction IN list_of_variances DO BEGIN troduced applications of the FSX principle
average_variances(direction) := average(list_of_measurements(direction)) to the scientific research work mentioned
END in the first section. In these cases, FSX op-
// Choose direction associated with maximum variance as preferred direction erates on the base of a model for screening
own_preferred_direction ← direction of max( average_variances ) maximization options, which is pre-deter-
// COLLECTIVE DECISION PHASE: Let the swarm reach a common preferred direction mined to a certain extent, be it in the form
FOR t := 1 TO collective_decision_time DO BEGIN of a minimalistic notion of remaining pos-
// Agents randomly in time initiate sending a message: sible moves, as in the coordination task in
IF (random_value < probability_to_initiate_sending_message) THEN Example 1, or be it as a comprehension of
message := own_preferred_direction maximal variance, as in the case of the robot
send message to neighbors swarm in Example 2. This pre-determina-
ELSE IF (received a message) THEN tion implies a sort of input from an assumed
adjust own_preferred_direction by factor X external world to the agent, which limits the
44 towards preferred direction contained by the received message relevance of the examples for a constructiv-
ELSE ist epistemology. From a constructivist per-
wait spective, it might be more interesting what
END the minimal requirements are for an FSX
agent to start maximizing its options. Our
Code 2 • Pseudo code of the underlying algorithm. For ease of understanding, the algorithm was objective, therefore, was to find out what the
divided into three phases: INFORMATION GATHERING, EVALUATION, and COLLECTIVE DECISION. preconditions are that an agent can use for
screening its environment in action while
trying to get along in it.
« 45 »  In order to answers this question,
we revised the simple grid-world scenario
tion of all robots, with the initial position responds to the largest amount of accessible introduced in §§19ff. In this scenario, an
being the one of the swarm in Figure 7a and states of the environment and thus adheres agent, when placed randomly somewhere
eventually moving along the border of the to the FSX principle. Figure 8 illustrates this on the grid, simply compares all the op-
two domains. by showing the average variance in informa- tions it can see within its vision and settles
« 42 »  In this way, the swarm is able to tion accessible to the swarm during its tem- for the one that yields the highest “freedom,”
follow a gradient towards maximum in- poral evolution. The variance increases until i.e., the largest space of further options. As
formation variance, i.e., towards the most the swarm reaches the border of the two illustrated in Figure 1b, this makes it move
diverse areas in the environment. This cor- domains and then fluctuates around V = 4, towards the center of the grid.

CONSTRUCTIVIST FOUNDATIONs vol. 16, N°1

 Second-Order Cybernetics
Foresight Rather than Hindsight? Hannes Hornischer et al.

« 46 »  In the revised version, however,

5
the agent’s setup is different. It does not
compare anything in the first place, but
starts out by randomly choosing one of its 4
possible options: going North, South, East,

West, or staying put. From these random
choices, it can derive probabilities about
the states that result from its actions. Ini-
tially this will not yield more than overly
uncertain information conveying a very
rudimentary and inaccurate picture about
what might follow from a particular action.
Nevertheless, obviously, this information
suffices for the agent to send its walkers out
to screen possible options and to follow the
ones among them that reach the most di-
verse states. Many of these tentative actions
will be misguided, however. The FSX prin-
ciple will not be recognizable as such in this
phase. However, step by step, experiences
will accumulate and start providing an, at
first, rudimentary and rather inaccurate, but
eventually increasingly more useful model
for choosing options that enlarge the agent’s
abstract space of future options.
« 47 »  In other words, in this experi-
mental setting, the agent’s model emerges
Variance V
3
2
1
0 5 10 15 20 25 30
Steps s
Figure 8 • The average variance in information the swarm as a whole has access to, defined as
the arithmetic mean of the variances calculated by each agent in the swarm, respectively. The
variance in measurements, i.e., the amount of accessible information, increases up to the point
where the swarm reaches the border between the two domains. From then on it fluctuates
around V=4.
as information about the probability of all of its options and incorporating those
being in a certain state after taking a par- into the model that seem to open up fur-
ticular action in an initial state. The model ther possibilities – just as in Piaget’s exam-
gets iteratively constructed from the (ini- ple (§11), the small child constructed the
tially random) actions of the agent trying ball’s “object invariance” from sensorimo-

(a) (b) (c) (d) (e) (f) (g) (h)

50 samples 100 samples 200 samples 400 samples 1250 samples 1600 samples 2500 samples complete knowledge
of training data of training data of training data of training data of training data of training data of training data of the orignal model

Looking
2 steps
ahead

45

Looking
3 steps
ahead

Looking
4 steps
ahead

Looking
5 steps
ahead

Figure 9 • Effect of the amount and nature of training data with respect to the size of the screened horizon. The quality of the agent’s choices is
influenced by two parameters: the number of samples and the number of steps it looks ahead.

https://constructivist.info/16/1/036.hornischer
 // INITIALIZATION: Create N walkers with copies of the system’s state: « 48 »  For the learning process in this
Options := {UP, DOWN, LEFT, RIGHT, STAY} experiment, an artificial neural network has
FOR i := 1 TO N DO BEGIN been applied, but, in principle, any machine-
// Walkers start at the agent’s initial state: learning technique will do. The learning
Walker(i).State := Agent.State process is illustrated in Figure 9, with higher
// Take walker’s initial decision: “FSX-empowerment” being depicted in
Walker(i).Initial_decision := random_choice(Options) brighter colors, allowing tracking of the in-
// Initialize set of reached states: appropriateness of early model stages and its
gradual improvement. Columns 2 to 6 show
ARTIFICIAL INTELLIGENCE CONCEPTS IN Second-Order Cybernetics

Walker(i).Reached_States := Set()
END that the agent coincidentally gained more
// SCANNING PHASE: Evolve walkers from time = t to t + Tau in M ticks: information at first about the upper, the left
FOR t := 1 TO M DO BEGIN and the right edges of its environment. For
// PERTURBATION: quite some time it considers the lower edge
FOR i := 1 TO N DO BEGIN patches as providing nearly as much “FSX-
// At first tick use the stored initial decision empowerment” as the central patches. Only
IF (t=1) THEN after 2500 samples (7th column) does it ap-
Walker(i).Decision := Walker(i).Initial_decision pear to get it right and adjust its model ac-
ELSE cordingly.
Walker(i).Decision := random_choice(Options) « 49 »  Another interesting aspect in
// Use the learned model to fill the other state’s component: our results is that, in addition to aggregat-
Walker(i).State := Agent.Model.predict(Walker(i).State, ing promising feedback (or in other words:
Walker(i).Decision, to collecting experiences), the agent can
dt := Tau/M) optimize the scope of its future horizon. In
Walker(i).Reached_States.add(Walker(i).State) the early part of the model-building pro-
END cess (the left-most column of the images
END in Figure 9), a smaller scope (here, looking
// DECIDING PHASE: three steps ahead) seems to yield a more
Max := Size(Unique(Walker(i).Reached_States)) appropriate preliminary model than the
Decisions := Set() 5-step scope, where the agent maneuvers
FOR i := 1 TO N DO BEGIN itself into a corner trap. In contrast, in the
IF (Size(Unique(Walker(i).Reached_States)) == Best) THEN later stages of the model building, a larger
Good_Choices.add(i) scope yields the appropriate model. Obvi-
END ously, given enough sample time, the agent
Agent.Decision := Walker(random_choice(Good_Choices)).Initial_decision is able to adapt its future horizon optimally
// EXECUTION: to the complexity of its environment. It con-
Init_State := Agent.State structs its model while constantly sampling
46 Agent.State := Environment(Agent.State, Agent.Decision, dt := Tau/M) the result of choices and applying the FSX
// LEARNING PHASE: principle. Or, in other words, it constructs
Agent.Model.learn(Init_State, Agent.Decision, Agent.State, dt := Tau/M) its world, its “reality” as perceived through
(and only through) its model, by choosing
Code 3 • Pseudo code as used for a learning agent on a 10x10 grid. Note that no reward function with increasing certainty those of its actions
is applied as all states have equal reward values. However, the boundaries of the grid are impen- that maximize its future possibility space.
etrable, rendering the agent with fewer degrees of freedom close to the edges of its world. « 50 »  The FSX principle thus builds
on the use of a model that emerges as an
“eigenvalue” in the sense of von Foerster. It
gets constructed from the iterated attempts
of an agent to come to terms with its op-
tor recursions of its reactions to irritations higher the certainty about the actions’ re- tions. Initially, this agent’s possibility space
(compare this also to the concept of the sults. Once certainty is sufficiently high, the is too vast. It doesn’t give reasons for any
self-modeling robot by Bongard, Zykov & FSX principle is applied, in practice, nearly particular action. It has to be constrained at
Lipson 2006). As described in Code 3, the as efficiently as in the original simple grid first, and this is done by ongoing attempts
model emerges as an “eigenvalue,” so to say, example, yielding a clear preference of the at deriving probabilities for effects of pos-
from the feedbacks the agent receives from agent for the central patches of the grid, as sible actions, with these effects then being
its actions. The greater the number of states can be seen in the bottom-left-most images weighted in regard to the action space they
sampled (by initially random choices), the in Figure 9. are opening up, that is, in regard to what we

CONSTRUCTIVIST FOUNDATIONs vol. 16, N°1

 Second-Order Cybernetics
here call Future State Maximization. This
process is re-iterated up to the point where
a stable concept emerges, an “eigenvalue” of
what to do and what better not to do in a
situation where a maximum of further op-
tions is at stake.
« 51 »  The results of the above-de-
scribed experiment may illuminate yet an-
other aspect of the FSX principle. If, for a
moment, we assume that it is not the agent
that actively changes its place in its grid
world, but that the grid itself shifts, say by a
patch at a time in a random direction, with
the agent staying put in its place, it seems
clear that these environmental changes will
have different consequences depending on
the original position of the agent. If, for
instance, the agent is positioned at the up-
per edge of the grid and the grid is moved
down by one patch, the agent will find itself
beyond the grid’s limits and hence stripped
of all of its options. Or if the agent is posi-
tioned one step away from an edge or a cor-
ner when the grid moves, it will be shifted
towards the edge and lose a large share of its
options. Only in the case where the agent
is placed somewhere near the center of the
grid, may an environmental change by one
or even by several patches have no fatal
consequences. The agent may still be able to
move to one of its four neighboring fields
and thus stand the blow to its environment
without too many problems.
« 52 »  In other words, the center fields
on this grid are the ones that offer the agent
highest resilience in case of environmental
changes. Whatever may happen, the pos-
sibilities to react remain largest in this part
of the world. At the same time, these center
fields are the ones that are sought out ac-
tively according to the FSX-principle. The
states that an agent prefers by following the
FSX-principle hence seem to coincide with
what maximizes its sustainability in the case
of a blow to its environment.
« 53 »  This may not seem to be of huge
relevance in the case of small and incre-
mental changes. Adaptation to an environ-
ment that changes slowly or with moderate
speed can be achieved by conventional re-
actions. The agent must just keep moving
while its environment shifts one patch at a
time. However, in a dynamical environment
with more severe changes, it can make sense
to anticipate the most sustainable states, i.e.,
Foresight Rather than Hindsight? Hannes Hornischer et al.
the ones that provide highest resilience.
And this can obviously be achieved by fol-
lowing the FSX principle. Under conditions
of non-linearity and sudden, unpredictable
transitions, as diagnosed in the debates on
climate change, on digital transformation
or on the social and economic consequenc-
es of Covid-19, the FSX principle seems to
gain its significance. That is where it makes
sense to prefer options that open up a maxi-
mum of further options.
Conclusion
« 54 »  Heinz von Foerster might have
had something similar in mind when he
reasoned about his Ethical Imperative that
“  in times of socio-cultural change the future
will not be like the past. With a future not clearly
perceived, we do not know how to act. With only
one certainty left, if we don’t act ourselves, we
shall be acted upon. Thus, if we wish to be sub-
jects rather than objects, what we see now, i.e.,
our perception, must be foresight rather than
”
hindsight. (Foerster 1972: 31)
« 55 »  As agreeable as these indeed
rather ethical formulations of von Foerster
are, they seem to stand somehow apart from
other concepts in his oeuvre, such as the
nontrivial machine, the second-order ob-
server, and eigenbehavior, which all seem to
build on formal argumentation. We still can
only surmise what von Foerster intended to
aim at when he formulated his Ethical Im-
perative as “act always so as to increase the
number of choices.” Maybe he had ideas for
experiments similar to the ones presented
here but did not have the (computational)
means to implement them. Or maybe he re-
mained in the vagueness of ethics because
he did not see a way to substantiate his in-
kling about how to answer possible objec-
tions against constructivism. As mentioned
in §§13f, we think that his rejection of a
solipsistic interpretation of constructivism,
for instance, would find support from the
FSX principle if one conceives of modeling
– as in the context we created here – as an
opening up of new (virtual) options. Fol-
lowing this line, self-modeling is then just a
particularly complex form of it, and the con-
struction of a (virtual) concept of an “other”
is – due to the need for differentiation – its
consequence. Both the “self ” and the “oth-
er” (as model components) are, in terms of
FSX, options, which when confronted with
each other must constrain each other’s pos-
sibility spaces, which is the case when, for
example, one solipsistically disallows the
other’s existence. As mentioned above, von
Foerster reasoned in this context that this
leads to a contradiction, if the other’s dis-
allowance of existence is reciprocal. From
here, it seems reasonable to opt for interac-
tion instead, for acting so as to not exclude
options, and through this to gain a chance
for the emergence of stable eigenforms, of
“selves,” and from their interaction eventu-
ally of a “reality” (= “community,” Foerster
2003) that again entails further options.
« 56 »  There may be reason in this, but,
as von Foerster admits, there is no necessity,
and there is no way to prove it, in particu-
lar without the computational means to test
and analyze the emergence of eigenforms
from interacting particles, as it is now com-
monly done, e.g., in agent-based modeling.
Maybe all that von Foerster could do back
in his days was to restrict himself to the
realm of ethics in proposing to “act always
so as to increase the number of choices.”
« 57 »  However, our prime intention
here was not to enter into a debate on eth-
ics or philosophy, but to point out that von
Foerster’s formulation of the Ethical Imper-
ative nearly literally conforms to the prin-
ciple that we here call Future State Maximi-
zation. Furthermore:
ƒ As shown in Experiment 3, FSX can be 47
considered in terms of the emergence of
eigenvalues.
ƒ FSX allows for an interpretation of as-
pects in von Foerster’s oeuvre and thus
seems to blend in well with the frame-
work of a constructivist epistemology.
ƒ The principle could prove to be an in-
teresting and fruitful method for fur-
ther research, particularly in times of
dynamic changes and high demand for
possibilities to locate resilient and sus-
tainable solutions.
https://constructivist.info/16/1/036.hornischer
 { HANNES HORNISCHER
is a PhD candidate at the Systems Science group of the University of Graz, with a
background in physics. In his research, he focusses on self-organization in complex
systems, using agent-based modeling. He is mainly involved in interdisciplinary projects
touching the fields of systems sciences, physics, biology and robotics, as well as social
psychology. While major parts of his scientific work consist of fundamental research
on artificial intelligence and swarm intelligence, they find application in the control of
autonomous robotic swarms as well as the investigation of dynamics in human groups.
ARTIFICIAL INTELLIGENCE CONCEPTS IN Second-Order Cybernetics

{ SIMON PLAKOLB
is a PhD candidate at the Systems Science group of the University of Graz. His primary
research interests are the simulation of complex, social systems, using parallel
computing, agent-based modeling, and artificial intelligence. As a boundary object
for his investigations, he mostly uses traffic-related topics ranging from meso- to
microscopically detailed models. His research touches the boundary in between computer
science and social simulation, where implementation details on parallel hardware are
as much of an issue as providing policy-relevant results. https://www.behaviour.space

{ GEORG JÄGER
obtained his PhD in theoretical and computational physics, where he simulated ultra-
cold quantum systems with applications in quantum computing and matter-wave
interferometry. During his Postdoc he was using similar methods, but his research focus
shifted towards complexity and sustainability research. Now, he is Assistant Professor
for Computational Systems Sciences at the University of Graz. In his research he is
interested in the simulation and analysis of complex systems, using tools like agent-based
modeling, machine learning and network science. Investigated systems range from purely
abstract ones to subjects closely related to sustainability. https://www.jaeger-ge.org

{
48
MANFRED FÜLLSACK
is Professor of Systems Sciences at the University of Graz. His research focuses
on complex systems and the possibility of simulating and analyzing them with the
help of computer models. He is particularly interested in the emergence and loss of
stable equilibria in these systems and the possibility of anticipating state changes
using statistical methods. Furthermore, he is interested in current developments
in the field of machine learning and artificial intelligence, and especially in their
consequences for all social, economic and ecological aspects of human work.

CONSTRUCTIVIST FOUNDATIONs vol. 16, N°1

 Second-Order Cybernetics
References
Ay N. & Polani D. (2008) Information flows
in causal networks. Advances in Complex
Systems 11(1): 17–41.
Bongard J., Zykov V. & Lipson H. (2006)
Resilient machines through continuous self-
modeling. Science 314(5802): 1118–1121.
Boos M., Pritz J., Lange S. & Belz M. (2014)
Leadership in moving human groups. PLOS
Computational Biology 10.4: e1003541.
Bousso R., Harnik R., Kribs G. D. & Perez G.
(2007) Predicting the cosmological constant
from the causal entropic principle. Physical
Review D 76(4): 043513.
Cerezo S. H. & Ballester G. D. (2018) Frac-
tal AI: A fragile theory of intelligence.
ArXiv:1803.05049 [Cs], March. http://arxiv.
org/abs/1803.05049.
Cerezo S. H., Ballester G. D. & Baxevanakis S.
(2018) Solving Atari Games using fractals
and entropy. ArXiv:1807.01081 [Cs], July.
http://arxiv.org/abs/1807.01081
Charlesworth H. J. & Turner M. S. (2019)
Intrinsically motivated collective motion.
Proceedings of the National Academy of Sci-
ences 116(31): 15362–15367. https://www.
pnas.org/content/pnas/116/31/15362.full.pdf
Chin G. J. (2007) Cognitive homeostasis. Science
318(5851): 717.
Donati E., van Vuuren G. J., Tanaka K., Romano
D., Schmickl T. & Stefanini C. (2017)
aMussels: Diving and anchoring in a new
bio-inspired under-actuated robot class for
long-term environmental exploration and
monitoring. In: Gao Y., Fallah S., Jin Y. &
Lekakou C. (eds.) Towards autonomous
robotic systems. Springer, Cham: 300–314.
Foerster H. von (1972) Perception of
the future and the future of percep-
tion. Instructional Science 1(1): 31–43.
▶︎ https://cepa.info/1647
Foerster H. von (1981) Objects: Tokens for
(eigen-)behaviors. In: Observing systems.
Intersystems Publications, Seaside CA:
274–285. Originally published in 1976.
▶︎ https://cepa.info/1270
This target article is part of a bigger picture that encompasses several open peer commentaries and the response
to these commentaries. You can read these accompanying texts on the following pages (or, if you have only the target
article, follow the embedded link that takes you to the journal’s web page from which you can download these texts).
Foresight Rather than Hindsight? Hannes Hornischer et al.
Foerster H. von (2003) On constructing a reality.
In: Understanding understanding. Springer,
New York: 211–227. Originally published in
1973. ▶︎ https://cepa.info/1278
Foerster H. von & Bröcker M. (2002) Teil der
Welt: Fraktale einer Ethik – Ein Drama
in drei Akten. Carl-Auer-Systeme Verlag,
Heidelberg.
Foerster H. von, Mora P. M. & Amiot L. W.
(1960) Doomsday: Friday, November 13,
AD 2026. Science 132(29): 1291–1295.
▶︎ https://cepa.info/1596
Füllsack M. (2013) Constructivism and com-
putation: Can computer-based model-
ing add to the case for constructivism?
Constructivist Foundations 9(1): 7–16.
▶︎ https://constructivist.info/9/1/007
Füllsack M. (2018) Plasticity, granularity
and multiple contingency: Essentials for
conceiving an artificial constructivist agent.
Constructivist Foundations 13(2): 282–291.
▶︎ https://constructivist.info/12/2/282
Guckelsberger C., Salge C. & Togelius J. (2018)
New and surprising ways to be mean: Ad-
versarial NPCs with coupled empowerment
minimisation. arXiv:1806.01387 [cs.AI].
http://arxiv.org/abs/1806.01387.
Hornischer H. (2015) Causal entropic forces:
Intelligent behaviour, dynamics and pattern
formation. Master’s Thesis, Georg-August-
Universität Göttingen, Germany.
Hornischer H., Varughese J. C., Thenius R.,
Wotawa F., Füllsack M. & Schmickl T.
(2020) CIMAX: Collective information
maximization in robotic swarms using
local communication. Adaptive Behavior,
1059712320912021.
Klyubin A., Polani D. & Nehaniv C. L. (2005)
Empowerment: A universal agent-centric
measure of control. In: 2005 IEEE Congress
on Evolutionary Computation. Volume 1.
IEEE Press, Piscataway NJ: 128–135.
Martyushev L. M. & Seleznev V. D. (2006)
Maximum entropy production principle
in physics, chemistry and biology. Physics
Reports 426(1): 1–45.
Maturana H. R. & Varela F. J. (1987) The tree of
knowledge: The biological roots of human
understanding. Shambhala, Boston.
Metzinger T. (2007) Empirical perspectives from
the self-model theory of subjectivity: A brief
summary with examples. In: Banerjee R. &
Chakrabarti B. K. (eds.) Models of brain and
mind: Physical, computational and psycho-
logical approaches. Elsevier, Amsterdam:
215–278.
Piaget J. (1960) Child’s conception of geometry.
Basic Books, New York.
Riegler A. (2007) Is Glasersfeld’s constructivism
a dangerous intellectual tendency? In: Glan-
ville R. & Riegler A. (eds.) The importance
of being Ernst. Echoraum, Vienna: 263–275.
▶︎ https://cepa.info/1776
Rosen R., Rosen J., Kineman J. J. & Nadin M.
(2012) Anticipatory systems: Philosophical,
mathematical, and methodological founda-
tions. Second edition. Springer, New York.
Shannon C. E. (1948) A mathematical
theory of communication. The Bell System
Technical Journal 27(3): 379–423.
Wissner-Gross A. D. & Freer C. E. (2013) Causal
entropic forces. Physical Review Letters
110(16): 168702.
Received: 14 April 2020
Accepted: 9 September 2020
49
https://constructivist.info/16/1/036.hornischer