Special Issue: Spotlight on 4E Cognition Research in Mexico

Adaptive Behavior
1–14
Ó The Author(s) 2018
Embodied Cognitive Robotics and the Reprints and permissions:
sagepub.co.uk/journalsPermissions.nav
learning of sensorimotor schemes DOI: 10.1177/1059712318780679
journals.sagepub.com/home/adb

Bruno Lara1 , Dadai Astorga1, Emmanuel Mendoza-Bock2,

Manuel Pardo1, Esaú Escobar1 and Alejandra Ciria1,3

Abstract
Embodied Cognitive Robotics focuses its attention on the design of artificial agents capable of performing cognitive tasks
autonomously. A central issue in this consists in studying process by which agents learn through interaction with their
environment. Embodied Cognitive Robotics aims to implement models of cognitive processes coming from Cognitive
Sciences. The guidelines in this research area are a direct response to the shortcomings of Classical Artificial
Intelligence, where high-level tasks and behaviors were studied. This article describes the work carried out in the
Cognitive Robotics Laboratory at the Universidad Autónoma del Estado de Morelos (UAEM). Our work is based on the
concept of low-level sensorimotor schemes coded by Internal Models, thus falling as a matter of course within the tenets
of Embodied Cognition, particularly with the idea that cognition must be understood as occurring in agents that have a
body with which they interact in a specific environment. It is through this interaction that learning emerges laying the
ground for cognitive processes. Our research includes theoretical work laying the foundations of Embodied Cognitive
Robotics, as well as work with artificial and with natural agents.

Keywords
Embodied Cognitive Robotics, Internal Models, sensorimotor schemes, Embodied Cognition

Handling Editor: Ximena González-Grandon, National Autonomous University of Mexico, Mexico

1. Introduction of success is one of the main reasons why critics of

Artificial Intelligence (AI), now known as Classical or
Traditional AI emerged as a discipline aiming to repro-
duce certain aspects of human intelligence in machines.
Its hypothesis was that if all aspects of learning or any
other intelligence capability could be described with
precision, then it should be possible to be reproduced in
machines (McCarthy, Minsky, Rochester, & Shannon,
2006). Classical AI was strongly influenced by the
school of cognitivism, a paradigm affirming that human
cognition is based on the manipulation of symbolic
representations through a set of rules (Newell & Simon,
1976). However, the idea that intelligent behavior can
be captured using only symbolic representations has
been widely criticized.
After decades of research, even though AI has man-
aged significant success in specific domains such as the
generation of algorithms capable of solving abstract
problems, it has not been able to produce artificial sys-
tems imitating basic human abilities such as sensorimo-
tor processes or perception (Mingers, 2001). This lack
Classical AI proposed the need to design artificial
agents situated in the world, emphasizing the funda-
mental role the body plays during the interaction with
the environment in producing and learning intelligent
behavior (Dreyfus, 1967). The most natural platforms
to study this interaction are artificial agents (Asada
et al., 2009; Moravec, 1984).
Perception and sensorimotor abilities have come to
be the really hard problems to solve. However, accord-
ing to this approach, once an agent has basic sensing
1
Cognitive Robotics Laboratory, Center for Science Research,
Universidad Autónoma del Estado de Morelos (UAEM), Cuernavaca,
México
2
Instituto de Investigaciones Filosóficas, UNAM, Mexico City, Mexico
3
Facultad de Psicologı́a, UNAM, Mexico City, Mexico
Corresponding author:
Bruno Lara, Cognitive Robotics Laboratory, Center for Science Research,
Universidad Autónoma del Estado de Morelos (UAEM), Av. Universidad
No. 1001, Cuernavaca CP 62100, México.
Email: bruno.lara@uaem.mx
2 Adaptive Behavior
and moving capabilities, higher order abilities should
become easier to implement. High-order abilities
include problem solving and language expert knowl-
edge, among others (Brooks, 1991). Following these
lines, a fraction of researchers in AI turned to the use
of robots as situated agents in what became known as
Embodied Cognitive Robotics (Clark & Grush, 1999;
Pfeifer, Lungarella, & Iida, 2007; Pfeifer & Scheier,
2001). Embodied Cognitive Robotics is in accordance
with the tenets of the embodied and embedded frame-
works, which highlight the importance of the interac-
tion of agents with their environment for the
development of cognitive processes (Pfeifer & Scheier,
2001).
Development is about change throughout the life-
span of agents and this, without a doubt, requires learn-
ing and adaptive capabilities. Movement is perhaps the
most pervasive and fundamental of all psychological
activities providing a highly significant source of per-
ceptual experiences (Adolph & Berger, 2006). Humans
are endowed genetically with the capability and propen-
sity to learn sensorimotor behavioral structures also
referred to as sensorimotor schemes (Harvey, 2018).
Human fetal motility starts at 7.5 weeks of gestational
age with isolated stereotyped movements of arms, legs,
and head, leading to more varied and complex patterns
of movement as early as at 9 weeks of gestational age
(Lacquaniti, Ivanenko, & Zago, 2012; Lüchinger,
Hadders-Algra, Van Kan, & de Vries, 2008). Later on,
the newborn child is prepared to interact with the envi-
ronment and adapt to it, displaying functional goal-
directed, prospective and flexible movements (von
Hofsten, 2003, 2009).
Movement cannot rely on sensorimotor feedback
because of the delays in efferent and afferent pathways
and the corresponding processing delays. These delays
are known to vary across modalities in the order of
approximately 200 ms (Desmurget & Grafton, 2000;
Miall & King, 2008; Miall, Weir, Wolpert, & Stein,
1993; von Hofsten, 2009). In order to overcome this
control problem, agents progressively and successfully
predict the consequences of behavior by learning the
associations between actions and their consequences
during the body–environment interaction. This interac-
tion leads to the continuous learning of rules and regu-
larities of events, further building more complex
sensorimotor schemes (von Hofsten, 2007, 2009).
Moreover, a constant recalibration of learned sensori-
motor schemes, in addition to the acquisition of new
ones, and the development of the physical body enable
behavioral patterns to become more complex, increas-
ing the possibilities for interaction with the world ‘‘for
further development of intelligence to proceed in an
incremental fashion’’ (Harvey, 2018, p. 26). Thus,
‘‘from a functional perspective, cognitive development
has to do with expanding the prospective control of
actions’’ (von Hofsten, 2007, p. 58).
Internal Models have been proposed to code the sen-
sorimotor schemes that emerge from the interaction of
agents with their environment. Originating in control
theory (Miall & Wolpert, 1996), these models have been
used to explain a wide range of cognitive phenomena
(for a review of the topic, see Schillaci, Hafner, and
Lara, 2016). The core of the research reported here is
Internal Models and the multimodal associations that
are necessary for them to emerge.
This article is organized as follows: the next section
attempts to lay out the links between AI and Embodied
Cognition, positioning the work reported here in the
wide 4E cognition framework (cognition as embodied,
embedded, extended, and enactive) and clarifying the
reasons to frame it mainly within Embodied Cognition.
The section contains a subsection on Internal Models,
which are at the core of the research carried out in our
lab. The next section ‘‘Embodied sensorimotor
schemes,’’ presents an overview of the work carried out
in our laboratory. First, it outlines some of the theore-
tical discussions on artificial agents and intelligence,
then it presents work carried out on mobile agents,
humanoids, and finally natural agents. The experiments
are ordered according to the complexity of the architec-
tures used as well as the different cognitive processes
they model. In the final section, we draw conclusions
and discuss further research.
2. AI and Embodied Cognition
AI aims to create tools or agents that are capable of
exhibiting intelligent behaviors. Among other things,
the AI community managed to create a machine that
successfully plays chess against the best human players.
However, to date there is still not a machine capable of
physically moving the pieces on a chess board as skill-
fully as a child. As a consequence, the idea of having a
body in order to act in the real world and give meaning
to the internal processing produced a major shift in AI
research. Having a body is necessary for cognition and
thus a prerequisit for intelligent behavior (e.g. Pfeifer
and Bongard, 2006). The implementation of these ideas
in AI became known as Embodied Cognitive Robotics.
Importantly, the Embodied Cognitive Robotics agenda
and its theoretical foundations acquired more solid
ground as Cognitive Sciences suffered a major shift of
paradigm, namely, the conception of the Embodied
Cognition framework.
The 4E cognition has been characterized, in general
terms, as the claim that cognition is embodied,
embedded, extended, and enactive. The core of this
view is that motor skills (the body), the material and
sociocultural world (the environment), and their combi-
nation in specific contexts (the body–environment
interaction) are indispensable components of cognition
(Anderson, 2003; Chiel & Beer, 1997; Clark, 1999;
Lara et al.
Lakoff & Johnson, 1999; Noë, 2004; Pezzulo et al.,
2011; Pfeifer et al., 2007; Robbins & Aydede, 2009;
Shapiro, 2010; Varela, Thompson, & Rosch, 2017;
Wilson, 2002). These four claims of cognition represent
a family of individual explanatory frameworks, which,
in a strict sense, cannot be reduced to four labels. At
the same time, there are also frameworks that include
the affective approach, sometimes presented as 4EA
cognition (see Kiverstein and Clark, 2009; Ward and
Stapleton, 2012). Cognitive Sciences literature has
emphasized the fact that there is no homogeneity
between the different 4E cognition frameworks, mainly
because there are significant differences in the models
and explanations for the subject of study. It is thought
that an attempt to characterize and generalize their
interests might lead to neglect peculiar aspects that are
particular to the explanatory frameworks for each of
the claims (Anderson, 2003; Rowlands, 2010; Shapiro,
2010). Also, placing the labels under the same general
scheme risks losing both the individual nuances and the
real disagreements between the different proposals (Di
Paolo, 2009; Kiverstein and Clark, 2009; Menary,
2010).
The work conducted in our laboratory follows
mainly the ideas of Embodied Cognition, based on the
notion that cognition is determined not only by brain
processes but also by a combination of these, together
with the body morphology and capabilities particular
to the agents situated in an environment (Rowlands,
2010). Specifically, our work takes, from the notion of
‘‘embodiment,’’ the principles of processing of sensori-
motor schemes, simulation models, and multimodal
associations. In doing so, we distance ourselves from
the controversies arising when attempting to bring all
the E’s together under one umbrella.
The bodily skills to respond to the environment are
as constitutive to cognition as internal processing. This
statement implies that in order to understand cognition
it is necessary to understand the complex and dynamic
interaction between the brain, body, and environment
(Clark & Grush, 1999; Van Gelder & Port, 1995). It
has been suggested that some behaviors that can be
classified as cognitive arise from the dynamic interac-
tion between the basic physical abilities of the agents
and their environment. Thus, the physiology, the evolu-
tionary history, the practical activity of the agent, and
the situation it is immerse in play the role of modeling,
limiting, and grounding the projects of Embodied
Cognition (Anderson, 2003).
The physical structure of the body sets limits, gives
precise indications, and offers opportunities for neural
control and cognitive processes (Chiel & Beer, 1997;
Rowlands, 2010; Shapiro, 2007). Pfeifer et al. (2007)
argue that the continuous and dynamic interaction of
the agent with its environment (physical processes and
incoming sensory information) induces statistical
3
regularities in sensory input enhancing internal infor-
mation processing, and from these regularities cognitive
behavior can emerge. It is suggested that the associa-
tion of sensory and motor information, known as sen-
sorimotor schemes, could be the ideal unit of analysis
for the study of cognition. Again, we believe that beha-
vior is directly affected by two elements, the ecological
niche in which the agent is immersed and the morphol-
ogy of the agent (location of its sensors and effectors).
Furthermore, Anderson (2003) suggests that beha-
vior represents emergent phenomena that can be attrib-
uted to the physical foundations that have evolved over
time. Therefore, the foundations of cognitive processes
lie in the different bodily modalities of the sensorimotor
system. Cognition must be understood in terms of how
it functions under the pressures of real-time interaction
with the environment (Wilson, 2002, p. 626). When cog-
nition takes place in a context of inputs and outputs rel-
evant to a specific task, it means cognition is situated.
That is to say, during cognitive processing, perceptual
information continues to affect processing, and at the
same time motor activity affects the environment (on-
line cognition).
Given that not all cognitive agents are capable of
carrying out certain tasks, it is essential to bear in mind
that it is body morphology which allows agents to
diversify their interaction with a constantly changing
environment. At the same time, it is the environment
which allows an agent to evolve particular cognitive
capabilities (Pfeifer et al., 2007; Varela et al., 2017;
Wheeler & Clark, 2008).
During development, agents learn sensorimotor
schemes; these provide predictions of sensory informa-
tion to efficiently interact with the environment
(Harvey, 2018). Sensorimotor schemes associate differ-
ent sensory modalities (e.g. exteroceptive and interocep-
tive) which reflect the knowledge of an agent about the
regularities of the environment as consequences of its
actions. Moreover, the knowledge about sensory conse-
quences of self-generated actions provides the feeling of
agency and, as a result, a subjective point of view of the
world. Importantly, the sensorimotor schemes provide
a link to Gibsonian affordances, which are particularly
appealing for Embodied Cognitive Robotics (Nasuto &
Hayashi, 2016).
Traditionally, cognitive processing implies the inte-
gration of what has been known as high- and low-level
processes (Koeinig, Ktihnberger, & Kietzmann, 2013).
However, rather than insisting on this distinction, we
assume that sensorimotor schemes underlie all types of
cognitive processes.
A telling example is off-line cognition. Humans can
have processes that are not necessarily situated. If
situatedness is a condition for cognition and it is not
fulfilled, then it seems that these processes cannot be
considered as cognitive. This intuitive idea is
4 Adaptive Behavior
contradictory since humans perform cognitive pro-
cesses that can be carried out in the absence of inputs
and outputs relevant to the task at hand. Humans are
capable of cognitive processing that is not in direct
interaction with the environment (off-line cognition;
Wilson, 2002). For Embodied Cognition, non-situated
processes are based on mechanisms that evolved for
agents to interact with their envorinment while
immersed in it. Thus, off-line cognition is based on sen-
sory and motor control mechanisms (Wilson, 2002, p.
626). It is possible that human and non-human agents
may invoke previously situated modalities. In non-
situated processes, cognition can resort to a simulation
of bodily processes or ‘‘simulated situations’’ that oper-
ate under the same physical and situational limitations.
In the simulated situation, the central nervous system
performs a reactivation of sensory information from
different sources (e.g. visual, proprioceptive) and pre-
dicted consequences of states (Barsalou, 2008). The
multimodal associations, used during simulated off-line
situations, rely on previously learned schemes and on-
line experiences. In other words, simulations are a reac-
tivation of the representations of a relevant learned epi-
sode and the bodily resources associated with
sensorimotor strategies of the past (Pezzulo et al.,
2011). Barsalou (2008) argues that such a system allows
the brain to admit different kinds of simulation in dif-
ferent cognitive processes, such as high-level percep-
tion, working memory, implicit memory, long-term
memory, and conceptual knowledge.
Finally, we come to an important question, namely,
how can cognitive abilities be defined or, more pre-
cisely, what does a cognitive process involve. In
Cognitive Sciences, there exists a discussion on the
notion of cognition. The issue in these disputes lies in
defining cognition in different terms and therefore in a
disagreement about future experiments that would be
essential to arbitrating their differences (Buckner &
Fridland, 2017).
We agree with the suggestion of Newen (2017) that
to define cognition it is necessary to first characterize a
cognitive process using an example-based approach.
Newen (2017) proposed the comparator mechanism as
a typical cognitive process. This mechanism is based on
Internal Models and has been widely studied in
Embodied Cognitive Robotics (Schillaci et al., 2016).
Rodrı́guez (2012) presents a general discussion of the
relationship between Embodied Cognition and
Embodied Cognitive Robotics, presenting the concept
of Internal Models as a suitable methodological tool to
test the ideas of Embodied Cognition.
Namely, that having a body is necessary for intelli-
gent behavior and the interaction of body and environ-
ment is the fundament for the development of cognitive
abilities. Embodied Cognitive Robotics conceives intel-
ligent behavior as the actions that occur during the situ-
ated body–environment interaction.
2.1. Internal Models
Cognitive processing depends on the sensorimotor cap-
abilities an agent possesses, based on its body, situated
in a specific environment. Our main research interest is
centered on the understanding of how agents acquire,
modify, and improve their sensorimotor schemes dur-
ing interaction with the environment to accurately pre-
dict the sensory consequences of self-generated actions.
Internal Models have been shown to be a good strat-
egy to acquire the regularities of the environment as
they are constructed by learned multimodal associa-
tions. Stretching the capabilities of Internal Models, the
authors followed the ideas of Dewey and Heidegger
and presented arguments linking a version of ‘‘inten-
tionality’’ with the notion of Internal Models as the
core structure for coding basic sensorimotor schemes
(Lara Guzmán, Hermosillo Valadez, & Baeza Mariscal,
2017). The proposed notion of ‘‘intentionality’’ is
understood in a pragmatic sense, that is, as a basic-level
agent–environment interaction.
The notion of Internal Models implies that any
model for motor adaptation knows something about its
motor apparatus and the environment (Kawato, 1999).
In the study of Internal Models, forward and inverse
models have been proposed (Jordan & Rumelhart,
1992), and studied in many other disciplines, such as
neuroscience (Kawato, 1999; Wolpert, Ghahramani, &
Flanagan, 2001), biology (Webb, 2004), and philosophy
(Karniel, 2002; Kiverstein, 2007). Each of these disci-
plines presents different ontological commitments to
the notion of Internal Models. It is even possible that
the different disciplines understand the terms related to
the notion in a different and perhaps incompatible
manner, which means that this can present epistemolo-
gical repercussions (Mendoza Bock, 2013). However, in
Embodied Cognitive Robotics, care has been taken to
use only their core capabilities, namely, their ability to
fuse multimodal information and provide predictions.
An inverse model, or controller, generates a motor
command to the motor plant and a forward model pre-
dicts the output of the system, that is, the sensory con-
sequences based on the motor command (Karniel,
2002). A forward model incorporates knowledge about
the sensory changes that will be produced by self-
generated actions of an agent. In other words, the for-
ward model predicts the sensory situation St+ 1 given a
motor command Mt applied to an initial sensory situa-
tion St (see Figure 1). Thus, forward models play a cen-
tral role in cognitive agents as they provide predictions
of the sensory consequences of motor commands
(Maye & Engel, 2012; Schillaci et al., 2016; Zambelli &
Demiris, 2015).
The essence of Internal Models or the process of the
comparator mechanism, as the name suggests, is the
comparison between expected sensory output from the
model and the actual sensory input generated by the
Lara et al.
Figure 1. Given a current sensory situation St and a desired
sensory situation, St + 1 , the inverse model suggests a motor
command, Mt . The forward model uses the current sensory
situation and the efferent copy of the motor command and
predicts the next sensory situation, St + 1 .
execution of a motor command. The comparison may
result in either a match or a specific error, which in the
literature has become known as prediction error. This
mechanism allows an online goal-directed adjustment
of the motor command required to achieve the desired
sensory consequences during a specific task situated
in a context. Extensive experimental and theoretical
arguments support the hypothesis that the brain uses
Internal Models, for motor planning and execution
(Wolpert, Ghahramani, & Jordan, 1995; Wolpert,
Goodbody, & Husain, 1998a), distinguishing
between self and others (Casile & Giese, 2006;
Knoblich & Flach, 2001), and feeling of agency due
to a sensory attenuation sensation of self-generated
actions (Blakemore, Frith, & Wolpert, 1999; Blakemore,
Smith, Steel, Johnstone, & Frith, 2000a, Blakemore,
Wolpert, & Frith, 2000b; Frith, Blakemore, & Wolpert,
2000; Wolpert et al., 1998a), among other tasks.
Conceiving the comparator mechanism as a typical
cognitive process, we argue that cognition relates to
sensorimotor interactions leading to constant learning
and adjustment of sensorimotor schemes which can be
potentially reactivated during simulated interactions.
Knowledge arises through this interaction and forms
the basis of the cognitive abilities that shape the mind
(Lara, Ciria, Escobar, Gaona, & Hermosillo, 2018).
The morphology of an agent (e.g. the distance
between its ears or eyes) affects and modifies the
perception–action cycle. Cognition depends on the type
of body and sensory and motor abilities that have epi-
genetically and ontogenetically developed for specific
environments (Shapiro, 2004). Thus, our body is the
only one with the abilities to perceive as we do (Nagel,
1974; Noë, 2004; Shapiro, 2004). It is for all these rea-
sons that betting on sensorimotor schemes, multimodal
associations, and prediction seems like a plausible
model to understand some of the mechanisms that
underlie cognition.
3. Embodied sensorimotor schemes
An open problem in Robotics and AI is how autono-
mous agents can generate internal representations that
allow them to manipulate symbols. Furthermore, how
5
these representations are linked to the external world
remains elusive. In response, Embodied Cognitive
Robotics has proposed that these representations arise
from the motor and sensorial capabilities of the agents,
where the characteristics of the perceived data are
grouped into general categories. Such categorical repre-
sentations act as concepts that allow agents to select
referents of the manipulated symbols (Rodrı́guez,
Hermosillo, & Lara, 2012).
The physical grounding framework holds that the
unit of analysis to understand cognition is a sensorimo-
tor cycle (Pfeifer et al., 2007). Here, the development of
cognitive skills demands a body that is situated and
acts, situated actions, under specific contexts, in the
real world. These tenets fall within the framework of
Embodied Cognition, as mentioned earlier, particularly
in what relates to the processing of low-level sensori-
motor schemes, internal simulations, and multimodal
associations. Still, it has been argued that, in Embodied
Cognitive Robotics, there might be the need to take
into account behaviors related to interoceptive infor-
mation. Particularly, Hernandez Ochoa (2017) argued
that agents must respond in the lowest levels to emo-
tions by satisfying basic body needs and drives that
allow them survival. Artificial emotions represent an
incipient line of research, highly neglected so far in the
area.
3.1. Implementations in artificial agents
To put the concept of Internal Models to the test, dif-
ferent implementations have been carried out. These
implementations are presented here in the order of the
complexity of the models and the type of behaviors they
try to address.
3.1.1. Navigation. One of the early implementations is
presented by Escobar, Hermosillo, and Lara (2012),
where an internal model provided an agent with basic
capabilities for navigation by means of self body-map-
ping. Using a stereo camera mounted on a Pioneer 3-
DX robot, the disparity value coming from the two
images was used as St . The value of Mt was kept con-
stant, and the output from the model was the disparity
values for t + 1 as well as the predicted bumper
states. The agent was let to perform straight trajec-
tories colliding with obstacles. The forward model
learned the associations between the observed world
and the tactile consequences of moving forward. A
forward model was coded using standard Multi-layer
Perceptron Networks (Rumelhart, Hinton, &
Williams, 1985) trained with resilient Error Back-pro-
pagation (Riedmiller & Braun, 1993). Once the model
was trained, the tactile prediction was active in the
parts of the field of view where collisions would occur
allowing a second system to drive the robot to areas
6 Adaptive Behavior

where there were no obstacles. It is worth noting that

in this work, the calculation of the disparity values is
a geometrical solution to the problem of 3D percep-
tion. To avoid using such solution, a system was
designed to learn the distance to obstacles in relation
to its movement in the world (Gaona, 2014; Gaona,
Hermosillo, & Lara, 2012). Here, the authors used a
system that takes as input the two images coming
from the stereo camera and one of 3 different motor
commands. The output of the forward model is the
visual prediction as well as the tactile situation. The Figure 2. The use of St + 1 as the input to the forward model.
tactile information is an array of the same dimensions Given a current sensory situation, St , and a desired sensory
as the images entirely filled with 1’s when there is a situation, St + 1 , the inverse model suggests a motor command,
Mt . The forward model uses the current sensory situation and
collision and 0’s otherwise. The system is composed
the efferent copy of the motor command and predicts the next
of an array of Multi-layer Perceptron Networks that
sensory situation, St + 1 .
function as local predictors. Each of these networks
takes as input a small window of each of the input
images and the motor command. The output of each
(Moons, 1998; Tsai, 1987). This is also true in robotics,
network is a smaller window of the predicted images
where distances are necessary to, among other things,
and a small window of the tactile situation. Together,
navigate safely or grasp objects in reach. In the systems
the networks are capable of predicting the two full
presented here, the agent experiences the affordance of
images and the tactile situation for the same area as
‘‘distance to objects’’ through direct interaction with its
the field of view of the three-dimensional (3D) scene.
surroundings. The affordance learned is based on the
An essential feature of the forward model concept is
motor and sensor capabilities particular to the agent.
that it can be used to perform long-term predictions
The objects are located x number of motor commands,
(LTPs). That is, the predicted output can be fed back to
the agent does not acquire an obstacle notion, and the
the system, and a new prediction is made, which can be
only knowledge it acquires is: ‘‘if I were to execute x
done as long as the error permits, as this accumulates
number of motor commands, I will experience a change
over time (see Figure 2). In Gaona (2014), the author
in sensory situation.’’ It is worth noting that the change
reported reliable predictions of up to 12 steps, allowing
in sensory situation becoming relevant for action is the
the agent to plan collision-free trajectories before mov-
tactile situation, which is learned through interaction.
ing. Taking the initial two images, a tree of visual and
In similar work, Möller and Schenck (2008) let a
tactile predictions is constructed, and the agent can then
robot learn an inverse and forward model to predict
take the path where the tactile predictions are lower.
the visual consequences of executed motor commands.
As in all previous works, the decision of which
They use a set of Internal Models, one for every possi-
motor command to execute, or which prediction branch
ble motor command, which creates conceptual prob-
to follow, is based on the predictions of the tactile sen-
lems on the need of a new pair of models for every new
sory situation. This prediction is not part of the input
motor command, deeming the system very inflexible.
to the system but it is learned from the sensorimotor
Hoffmann (2007) presents a forward model to predict
associations during the interaction of the agent with its
changes in visual information due to motor commands.
environment. However, the motor command to execute
The main difference with our work is that we use multi-
comes from a separate decision-making mechanism not
modal associations, which deliver predictions not only
related to learning or the environment. One could argue
in the visual space but also in the tactile space. These
that it is a non-grounded decision. To address this
predictions contain no credit or value per se, but can be
issue, Gaona, Escobar, Hermosillo, and Lara (2014)
further used by a separate system to aid the navigation
used LTPs to ground the concept of collisions. The sys-
process.
tem was developed in two stages. In the first, a forward
model was trained with disparity values and motor
commands as inputs and disparity values and bumper 3.1.2. Self and others. Bearing in mind that the context
predictions as outputs. Once trained, the forward with which an agent interacts is not limited to the con-
model, using LTP (off-line), associates negatively high ditions and physical characteristics of their environ-
bumper values to the motor commands producing ment, but also to interact with other agents, Arceo,
them. Finally, when in use, the system uses as input Escobar, Hermosillo, and Lara (2013) used sensorimo-
only disparity values and executes motor commands tor learning as the basis for the recognition and imita-
associated with low bumper values. tion of motor actions of another agent. It has been
In traditional artificial vision research, methods to proposed that Internal Models can be used to explain
estimate distance are based on geometric models the functioning of the mirror neuron system (Miall,
Lara et al.
2003). Through interaction with their environment,
agents acquire a repertoire of sensorimotor schemes.
The observed sensory consequences of an action allow
a conversion by the inverse model into potential motor
commands so that the observed movement can be pre-
dicted by the forward model enabling imitation.
The authors adhere to the possibility that mirror
neurons in premotor areas could function as predictors
of the observed behaviors from an associative learning
(Heyes, 2010). These associations are based on the sen-
sorimotor experiences of the agent and coded by
Internal Models. Similar work was performed by
Oztop, Kawato, and Arbib (2006) in a simulated robot
arm and by Demiris and Hayes (2002) on a
demonstrator–imitator simulated environment.
Using mobile agents, this idea was put to the test
(Arceo et al., 2013). Here, the authors let an agent asso-
ciate the visual consequences of small movements to the
right and left. The visual information is the coordinates
of a fiduciary marker in its field of view. The forward
model takes as input visual information and motor
commands and delivers as output the changes in visual
information. The model is coded with a Multi-Layer
Perceptron network. The inverse model is a function
that finds a vector in a pool containing all collected tri-
plets of visual information for time t, denoted by Vt ,
motor command, Mt , and the corresponding change in
visual information at time t + 1, denoted by vt + 1 .
Once trained, the agent observes movements of the
marker mounted on a second agent. Using the observed
vt and vt + 1 , it uses the inverse model containing its
motor repertoire, to find the motor commands that best
bring about the visual changes. These motor commands
are passed to the forward model, together with the ini-
tial visual situation to perform a set of predicted visual
situations. The system then compares the suggested
visual situations with the observed ones in order to exe-
cute that motor command in the triplet with the smal-
lest error in visual information.
A final work using mobile agents adheres to the
claim that ownership of actions and its consequences in
the world are fundamental components of the cognitive
process of agency (Tsakiris, Schütz-Bosbach, &
Gallagher, 2007; Weiss, Tsakiris, Haggard, & Schütz-
Bosbach, 2014). The association between actions and
sensory consequences in the world is at the core of the
notion of forward models and has been put to the test
by Pico, Schillaci, Hafner, and Lara (2017). The
authors attempt to tackle the ego-noise problem in an
autonomous agent. The main issue here is for the agent
to suppress the sounds produced itself while moving
around to increase its performance on the processing of
external sound sources, for tasks such as speech recog-
nition or localization of other sound sources.
Traditional methods for audio processing, such as
static noise reduction techniques or the use of different
microphone arrays’ geometries, fail to address this
7
problem (Ince et al., 2009; Schmidt et al., 2016), because
of the non-stationary nature of the ego-noise generated
by the autonomous agents (Ince et al., 2010). In an
early bio-inspired approximation, Ince et al. (2009)
showed that it is possible to suppress this noise by let-
ting the agent perform predictions about the sound pro-
duced when executing a motor command and then
subtracting that prediction from the total sound input.
Following these ideas, Pico et al. (2017) propose that
it is possible to not only solve this problem using pre-
dictions and forward models, but more importantly the
ego-noise encodes useful information about the agent
and its environment. To test this, Pico et al. (2017)
developed a two-wheeled differential agent, capable of
synchronizing motor commands and sensory data using
two controllers. A low-level controller is in charge of
the sensors, and a high-level controller is in charge of
making predictions and pre-processing the audio.
Using this agent and forward models, implemented
as a Multi-layer Perceptron Network, Pico et al. (2017)
run two different experiments: In the first experiment
named ‘‘ego-noise classification,’’ the authors try to
classify the ego-noise of the agent as a consequence of
its own motor commands using predictions. The motor
commands correspond to the velocity profiles of the
two motors in the agent. In total, 11 forward models
were trained, each corresponding to a velocity profile
and predicting the ego-noise produced. Once the mod-
els are trained, the agent executes motor profiles and
the ego-noise produced is compared to the predictions
of all the models. Finding the smallest Euclidean dis-
tance between the two signals, the motor profile caus-
ing the noise is found. The results of this experiment
demonstrate that forward models can, in fact, code the
relation between a motor command and the sound that
this produces.
In a second experiment ‘‘Motor babbling and ego-
noise prediction,’’ the focus was on the detection of
changes in the environment of the agent. A forward
model was trained in a flat arena to predict the sounds
produced by the robot using random velocities for the
motors. Once the model is trained, the robot is tested
in the original arena and in a second one, containing a
ramp. When the robot moves on the ramp, the error
between the ego-noise and the prediction increases sig-
nificantly, suggesting that this type of implementation
can not only give information about the agent, but also
detect changes in its environment.
3.1.3. Humanoids. While all of the previous works have
been carried out using mobile agents, research has also
been conducted using forward models in humanoid
platforms, namely, the NAO robot from Aldebaran.
The agent has been let to learn its own body capabilities
by performing motor babbling (Demiris & Dearden,
2005). The NAO robot learned the visual consequences
8 Adaptive Behavior
of moving its arm around and how these changed when
the use of a tool modified its motor capabilities
(Schillaci, Hafner, & Lara, 2012a).
Forward models were also used to let the agent learn
different actions by associating the motor commands to
execute them with the visual consequences while execut-
ing those actions (Schillaci, Lara, & Hafner, 2012b).
Once trained, the system was capable of correctly recog-
nizing observed actions. The same principle is extended
to let the agent know whether an observed hand trajec-
tory is performed itself or by another agent based on
the amount of prediction error processed (Schillaci,
Hafner, Lara, & Grosjean, 2013).
In all the previously described implementations, the
forward models were coded using Multi-layer
Perceptron Networks, a straigthforward strategy that
allows the designer to concentrate on the modeled cog-
nitive processes as well as the cognitive architecture.
However, the nature of these tools presents some short-
comings such as the impossibility of adding knowledge
after training and more importantly does not work cor-
rectly when there is information missing.
The case of missing information is of particular
interest given that, as an example, when an agent learns
the reaching space of its arm, it does so by performing a
multimodal association. This association can include
motor, tactile, and visual information. However, when
an agent observes an arm moving, it is capable of recog-
nizing the movement and reproduce the action even
when the agent does not have access to first-person
motor or visual information. Among other reasons, this
led to a change of machine learning paradigm.
To tackle some of these issues, Escobar Juárez,
Schillaci, Hermosillo Valadez, and Lara Guzmán
(2016) presented a Self-Organizing Internal Models
Architecture (SOIMA) based on Hebbian learning and
self-organizing maps (SOMs; Kohonen, 1998). This
architecture is inspired by the work of Morse, Greef,
Belpaeme, and Cangelosi (2010) where the iCub huma-
noid robot associates different modalities in a similar
fashion but without the use of predictions based on
Internal Models.
In SOIMA, the regularities of a sensory modality are
coded in a SOM so that there can be as many SOMs as
sensory modalities. The modalities are then connected
using Hebbian learning, specifically between a sensory
situation, a motor change, and the sensory conse-
quences of this. The connections are organized in what
we call a Multimodal Representation Map (sensorimo-
tor schemes). The result is a modular architecture that
codes the forward and the inverse model under the
same structure. Furthermore, SOMs and in particular
dynamic self-organizing maps (DSOMs; Rougier &
Boniface, 2011) can keep learning, and new schemes
are added to the knowledge of the agent as this inter-
acts with its environment. When the architecture is in
use, information of one or more modalities can be
absent without affecting its performance. Different cog-
nitive processes have been modeled using SOIMA,
including coding the reaching space of the NAO’s arm,
saccade movements of the eyes, foveation of a point in
space and then reaching it with the arm (see Figure 3),
and the characterization of shorthand trajectories
(Escobar Juárez, 2016). As can be seen from the exam-
ple, the SOIMA can propagate information in any
direction depending of the needs of the task, depending
on the availble sensory or motor information.
We believe that this type of knowledge acquisition,
based on a Piagetian motor babbling strategy (Harvey,
2018), is not only in accordance with the principles of
Embodied Cognition but also serves as a proof of con-
cept. Namely, an agent can acquire the most primitive
tools to interact with its environment by first forming a
basic body image based on its sensorimotor capabilities
and then use this to further explore the world incremen-
tally acquiring more complex and adaptive behaviors.
3.2. Natural agents
To further test the concept of Internal Models (Wolpert
et al., 1995; Wolpert, Miall, & Kawato, 1998b) as expla-
natory mechanisms, several experiments have been car-
ried out with natural agents. As stated before, Internal
Models are thought to be involved in learning and
motor control; at the same time, they are supposed to
be at the center of action and perception. We have car-
ried out several experiments with humans related to
tasks that involve the sense of agency, recognition of
actions, and the prediction of external events.
A core element in the proposal of Internal Models is
their ability to explain the sense of agency understood
as the sense of control over our actions and their effects
(Haggard & Tsakiris, 2009). The sense of agency in the
framework of Internal Models arises as the result of the
ability of the sensorimotor system to predict the conse-
quences of our actions. The predicted consequences
and the incoming sensory information are then com-
pared in low-level sensorimotor processes (Haggard &
Chambon, 2012). However, alternative proposals sug-
gest that the agency experience is the result of high-level
cognitive processes, specifically causal inferences about
the relationships between thoughts and actions
(Wegner & Wheatley, 1999). Therefore, from this per-
spective, our experience of agency, like any other causal
inference, lays its foundations around contingencies
that allow us to infer, in this particular case, an appar-
ent authorship.
Taking this panorama as a reference, Astorga (2016)
investigated the effect of perceptual differences between
judging the onset of a voluntary action and the onset of
its sensory consequence (sense of agency) and the judg-
ments related to the onset of two contingent events
(causal inference). This work was carried out using the
paradigm of temporal binding proposed by Haggard,
Lara et al. 9

Figure 3. The connections in this SOIMA can be used by the NAO humanoid robot for foveation of a stimulus in its field of view
and then to bring its arm to point at it. The bottom maps are modal maps organizing the regularities of sensory information. The top
maps join events together as multimodal representations. So, for instance, an initial and a final (goal) position in the vision map are
connected to a unit in the Saccadic MMR, and this unit is connected to a motor command for the head which will bring the visual
stimulus to the center of the image. The right section of the SOIMA will bring the head and the arm to absolute positions in space,
with the arm pointing to the stimulus.

Clark, and Kalogeras (2002). Astorga (2016) found a
temporal binding between events that involves agency
significantly greater than when related events are judged
contingently. The result is consistent with Haggard and
Chambon (2012), in that the sense of agency is the result
of comparisons made by sensorimotor low-level cogni-
tive processes. The experimental results suggested that
when there is a sensory consequence associated with a
voluntary action (motor command), and because this
information is highly relevant for the organism, priority
is given to this association, strengthening it through a
temporal link between both events.
Under the hypothesis of Internal Models and their
relationship with the sense of agency, Rojas (2014)
worked on the issue of action recognition. This
research was based on the assumption that individu-
als process perceptual information together with
motor information. The result of this processing is
the acquisition of sensorimotor schemes, particular to
the individual. When recognizing an action, these
schemes are used in simulation to solve the task suc-
cessfully. A fundamental element in action recogni-
tion comes from the discrimination between an action
recognized as our own, or as executed by someone
else (Knoblich, Seigerschmidt, Flach, & Prinz, 2002).
Rojas (2014) found that individuals recognized faster
between actions (take a rubber ball, hit it with the
fist, or make it roll) when these were previously exe-
cuted by them. These findings can be seen as a result
of perceptual anticipation, a phenomenon that exem-
plifies the processing and preparation of actions.
Recognition of actions is carried out through simula-
tions using as a reference the sensorimotor informa-
tion coded in the Internal Models of the individual
(Wilson & Knoblich, 2005; Wolpert et al., 1998b).
4. Conclusion and further research
The research carried out in our laboratory follows the
principles of the 4E cognition framework, specifically the
ideas of Embodied Cognition. Interaction of agents with
their environment and sensorimotor schemes that emerge
through this interaction have been the backbone of our
research. It is our conviction that Internal Models are
the natural candidates for coding the aforementioned
schemes, as the capabilities to perform sensorimotor pre-
dictions is an essential feature of cognitive agents.
As previously described, instead of imitating high-
level cognitive tasks in order to understand cognition, a
fraction of the AI community turned to study what
become known as low-level cognitive processes. We
believe that the study of sensorimotor schemes and the
processes and capabilities that emerge from these repre-
sent the fundaments for further research in Embodied
Cognitive Robotics. The quest is not over yet; however,
important steps have been taken.
The experiments summarized here present results
that support the main tenets of Embodied Cognition
and the 4E cognition approaches in general, namely,
the fact that cognition must be understood as occurring
in agents that have a body, and with this, they interact
in a specific environment. The use of Internal Models
in our work serves as a proof of concept of different
strategies for the acquisition of basic cognitive tools for
agents, laying the ground for further development of
more complex and adaptive behaviors. The implemen-
tations of Internal Models can be read as agents acquir-
ing the capability of experiencing affordances of the
environment they interact with. These affordances are
coded using sensorimotor schemes particular to the
agents, their bodily properties, and their environment.
10
Another significant contribution comes from the
experiments centered on prediction, and the study of
the capabilities agents acquire for successful interaction
with their environment when they become predicting
agents. It has been shown that using Internal Models
to code sensorimotor schemes in order to predict the
sensory consequences of actions allows different types
of behaviors to emerge. These behaviors are in accor-
dance with the principles of the Embodied Cognition
framework; particularly, we have addressed acquisition
of affordances, agency, recognition and imitation of
actions, and simulations.
Our experiments in natural agents have allowed us
to investigate important cognitive processes, such as
action recognition and agency, in the light of Internal
Models. At the same time, these experiments provide
valuable insights for the implementation of Internal
Models in artificial agents.
Following the latest paradigm in the Cognitive
Sciences, current research in our laboratory also focuses
on modeling of perception and action in the framework
of predictive processing (Clark, 2013, 2015a). Predictive
processing represents one of the most radical views
about the flow of sensory information processing dur-
ing perception, asserting that it is primarily a top-down
process, rather than a bottom-up process. However, the
suggestion of an inverted structure of the perceptual
system requires agents to first learn forward (genera-
tive) models which capture the statistical structure of
the incoming sensory information by tracking the
causes of that specific structure (Clark, 2013). Cognitive
agents are thought to be actively predicting the stream
of sensory stimulation before it occurs, and only unex-
pected sensed deviations from the predicted state (pre-
diction error) are further processed bottom-up (Clark,
2015a, 2015b). The perceptual and motor systems work
together to predict incoming sensory information in all
modalities: exteroceptive, interoceptive, and, in the case
of the motor system, proprioceptive (Adams, Shipp, &
Friston, 2013). In addition, in this framework, percep-
tion and action constitute ‘‘active inference’’ (Friston,
Daunizeau, Kilner, & Kiebel, 2010) or ‘‘action-oriented
predictive processing’’ (Clark, 2013). Through actions,
cognitive agents minimize prediction error about
incoming sensory information, evidencing the intimate
relationship between perception and action (Clark,
2013; Friston, Kilner, & Harrison, 2006).
In this line of research, we are focusing on the fol-
lowing open questions and future implementations.
First, active inference suggests that there is no need for
an inverse model or an optimal control mechanism, as
motor commands are conceived as proprioceptive pre-
dictions that drive action (Clark, 2015a; Pickering &
Clark, 2014). In this framework, an integral forward
model (IFM) is a forward (generative) model and the
core machinery of perception and action (Pickering &
Adaptive Behavior
Clark, 2014). In the IFM, motor commands are
replaced by top-down sensory predictions issued by the
forward model, removing any distinction between
motor and sensory processing and also the need of an
inverse model. Thus, IFMs posit a more complex (dis-
tributed) forward model, mapping prior beliefs about
the desired trajectories to sensory consequences
(Pickering & Clark, 2014, p. 456). However, in classic
control theory, the inverse model has played a signifi-
cant role despite learning difficulties inherent to the
model. Rejecting the idea and need of an inverse model
brings interesting issues but should be considered care-
fully. As stated earlier, SOIMA intrinsically codes
inverse and forward models, so the relevance of the
inverse model could be further studied. Second, predic-
tive processing suggests the concept of precision
weights as an attentional mechanism which modulates
the incoming sensory information relevant to a context.
This attentional mechanism has generated many con-
troversies and demands further research and under-
standing. We argue that given the modular architecture
of the SOIMA, SOMs could be added to potentially
code context to control the flow and processing of rele-
vant sensory information. Such a structure could be
considered as a precision weighting mechanism.
Furthermore, the effects of prediction error on the flow
of upcoming information has not been widely modeled
and explored. In the SOIMA, the flow of information
is not restricted to one direction. The bidirectional flow
of information allows studying the effects of prediction
error on Hebbian learning processes and the way infor-
mation propagates.
Considering the core results of our work so far, we
are convinced that using Internal Models to learn and
recalibrate sensorimotor schemes leads to more com-
plex behavioral patterns for further development of
intelligence. In this framework, we hold that intelligence
can be concieved as a behavior which leads an agent to
the succesful execution of a task situated in a certain
context. Consequently, we claim that Internal Models
are valuable tools for research which can significantly
contribute to the a better understanding of cognition.
Acknowledgements
The authors would like to thank the current and former
members and close collaborators of the Cognitive Robotics
Laboratory, particularly Guido Schillaci, Jorge Hermosillo,
Verena Hafner, Esau Escobar, Wilmer Gaona, Karla Baeza,
Diana Valenzo, Jorge Hernandez, Abraham Martinez, Javier
Vargas, David Castillo, Dairon Rodriguez, Eduardo Rojas
and, Antonio Pico.
Declaration of conflicting interests
The authors declared no potential conflicts of interest con-
cerning the research, authorship, and publication of this
article.
Lara et al.
Funding
The author(s) received no financial support for the research,
authorship, and/or publication of this article.
ORCID iD
Bruno Lara https://orcid.org/0000-0002-9844-6435
References
Adams, R. A., Shipp, S., & Friston, K. J. (2013). Predictions
not commands: Active inference in the motor system.
Brain Structure & Function, 218, 611–643.
Adolph, K., & Berger, S. (2006). Motor development. In
W. Damon, R. Lerner, D. Kuhn, & R. S. Siegler (Eds.),
Handbook of child psychology: Cognition, perception, and
language (vol. 2, pp. 161–213, 6th ed.). New York, NY:
John Wiley & Sons.
Anderson, M. L. (2003). Embodied cognition: A field guide.
Artificial Intelligence, 149, 91–130.
Arceo, D. C., Escobar, E., Hermosillo, J., & Lara, B. (2013).
Modelado de un sistema de neuronas espejo en un agente
autónomo artificial model of a mirror neuron system in an
artificial autonomous agent. Nova Scientia, 5, 51–72.
Asada, M., Hosoda, K., Kuniyoshi, Y., Ishiguro, H., Inui, T.,
Yoshikawa, Y., & Yoshida, C. (2009). Cognitive develop-
mental robotics: A survey. IEEE Transactions on Autono-
mous Mental Development, 1, 12–34.
Astorga, D. (2016) Efecto de Vinculación Temporal: un efecto
dependiente de la agencia o de la contigüidad y predictibili-
dad de los eventos? (Master’s thesis). Centro de Investiga-
ción en Ciencias Cognitivas, Cuernavaca, Mexico.
Barsalou, L. W. (2008). Grounded cognition. Annual Review
of Psychology, 59, 617–645.
Blakemore, S. J., Frith, C. D., & Wolpert, D. M. (1999). Spa-
tio-temporal prediction modulates the perception of self-
produced stimuli. Journal of Cognitive Neuroscience, 11,
551–559.
Blakemore, S. J., Smith, J., Steel, R., Johnstone, E., & Frith,
C. D. (2000a). The perception of self-produced sensory sti-
muli in patients with auditory hallucinations and passivity
experiences: Evidence for a breakdown in self-monitoring.
Psychological Medicine, 30, 1131–1139.
Blakemore, S. J., Wolpert, D., & Frith, C. (2000b). Why can’t
you tickle yourself?Neuroreport, 11, R11–R16.
Brooks, R. A. (1991). Intelligence without representation.
Artificial Intelligence, 47, 139–159.
Buckner, C., & Fridland, E. (2017). What is cognition? Angsty
monism, permissive pluralism(s), and the future of cogni-
tive science. Synthese, 194, 4191–4195.
Casile, A., & Giese, M. A. (2006). Nonvisual motor training
influences biological motion perception. Current Biology,
16, 69–74.
Chiel, H. J., & Beer, R. D. (1997). The brain has a body:
Adaptive behavior emerges from interactions of nervous
system, body and environment. Trends in Neurosciences,
20, 553–557.
Clark, A. (1999). An embodied cognitive science?Trends in
Cognitive Sciences, 3, 345–351.
Clark, A. (2013). Whatever next? Predictive brains, situated
agents, and the future of cognitive science. Behavioral and
Brain Sciences, 36, 181–204.
11
Clark, A. (2015a). Embodied prediction. In: T. Metzinger, &
J. M. Windt (Eds.). Open mind (pp 1–21). Frankfurt Am
Main, Germany: Mind Group.
Clark, A. (2015b). Surfing uncertainty: Prediction, action, and
the embodied mind. Oxford, UK: Oxford University Press.
Clark, A., & Grush, R. (1999). Towards a cognitive robotics.
Adaptive Behavior, 7, 5–16.
Demiris, J., & Hayes, G. M. (2002). Imitation in animals and
artifacts. In C. L. Nehaniv, K. Dautenhahn (Eds.), Imita-
tion as a dual-route process featuring predictive and learning
components: A biologically plausible computational model
(pp. 327–361). Cambridge, MA: The MIT Press.
Demiris, Y., & Dearden, A. (2005). From motor babbling to
hierarchical learning by imitation: A robot developmental
pathway. Retrieved from http://cogprints.org/4961/1/
demiris.pdf
Desmurget, M., & Grafton, S. (2000). Forward modeling
allows feedback control for fast reaching movements.
Trends in Cognitive Sciences, 4, 423–431.
Di Paolo, E. (2009). Extended life. Topoi, 28(1), 9–21.
Dreyfus, H. L. (1967). Why computers must have bodies in order
to be intelligent. The Review of Metaphysics, 21(1), 13–32.
Escobar, E., Hermosillo, J., & Lara, B. (2012). Self body
mapping in mobile robots using vision and forward mod-
els. In Proceedings of the 9th IEEE Electronics, Robotics
and Automotive Mechanics Conference (CERMA) (pp.
72–77). New York, NY: IEEE Computer Society.
Escobar Juárez, E. (2016) Modelado de una arquitectura de
modelos internos para la generación de esquemas sensorimo-
trices (PhD thesis). Instituto de Investigación en Ciencias
Básicas y Aplicadas, Cuernavaca, Mexico.
Escobar Juárez, E., Schillaci, G., Hermosillo Valadez, J., &
Lara Guzmán, B. (2016). A self-organized internal models
architecture for coding sensory–motor schemes. Frontiers
in Robotics and AI, 3, Article 22.
Friston, K., Kilner, J., & Harrison, L. (2006). A free energy
principle for the brain. Journal of Physiology-Paris, 100,
70–87.
Friston, K. J., Daunizeau, J., Kilner, J., & Kiebel, S. J. (2010).
Action and behavior: A free-energy formulation. Biological
Cybernetics, 102, 227–260.
Frith, C. D., Blakemore, S. J., & Wolpert, D. M. (2000).
Explaining the symptoms of schizophrenia: Abnormalities
in the awareness of action. Brain Research Reviews, 31,
357–363.
Gaona, W. (2014) Percepción de Distancia en un Agente Artifi-
cial a trave´s de la Simulación de Esquemas Sensorimotrices
(PhD thesis). Facultad de Ciencias, Montevideo, Uruguay.
Gaona, W., Escobar, E., Hermosillo, J., & Lara, B. (2015).
Anticipation by multi-modal association through an artifi-
cial mental imagery process. Connection Science, 27, 68–88.
Gaona, W., Hermosillo, J., & Lara, B. (2012). Distance per-
ception in mobile robots as an emergent consequence of
visuo-motor cycles using forward models. In Proceedings
of the IEEE 9th Electronics, Robotics and Automotive
Mechanics Conference (pp. 42–47). New York, NY: IEEE
Computer Society.
Haggard, P., & Chambon, V. (2012). Sense of agency. Current
Biology, 22, R390–R392.
Haggard, P., Clark, S., & Kalogeras, J. (2002). Voluntary
action and conscious awareness. Nature Neuroscience, 5,
382–385.
12
Haggard, P., & Tsakiris, M. (2009). The experience of agency:
Feelings, judgments, and responsibility. Current Directions
in Psychological Science, 18, 242–246.
Harvey, K. (2018). An open-ended approach to Piagetian
development of adaptive behavior. Open Access Library
Journal, 5, e4434.
Hernandez Ochoa, J. L. (2017) Robotica Cognitiva y Emo-
ciones (Master’s thesis). Centro de Investigación en Cien-
cias Cognitivas, Cuernavaca, Mexico.
Heyes, C. (2010). Where do mirror neurons come from?Neur-
oscience and Biobehavioral Reviews, 34, 575–583.
Hoffmann, H. (2007). Perception through visuomotor antici-
pation in a mobile robot. Neural Networks, 20, 22–33.
Ince, G., Nakadai, K., Rodemann, T., Hasegawa, Y., Tsujino,
H., & Imura, J. I. (2009). Ego noise suppression of a robot
using template subtraction. In Proceedings of the IEEE/
RSJ International Conference on Intelligent Robots and
Systems (IROS) (pp. 199–204). New York, NY: IEEE.
Ince, G., Nakadai, K., Rodemann, T., Hasegawa, Y., Tsujino,
H., & Imura, J. I. (2010). A hybrid framework for ego
noise cancellation of a robot. In N. Xi, & W. R. Hamel
(Eds.), Proceedings of the IEEE International Conference
on Robotics and Automation (pp. 3623–3628). New York,
NY: IEEE.
Jordan, M. I., & Rumelhart, D. E. (1992). Forward models:
Supervised learning with a distal teacher. Cognitive Sci-
ence, 16, 307–354.
Karniel, A. (2002). Three creatures named ‘‘forward model.’’
Neural Networks, 15, 305–307.
Kawato, M. (1999). Internal models for motor control and
trajectory planning. Current Opinion in Neurobiology, 9,
718–727.
Kiverstein, J. (2007). Could a robot have a subjective point of
view?Journal of Consciousness Studies, 14, 127–139.
Kiverstein, J., & Clark, A. (2009). Introduction: Mind embo-
died, embedded, enacted: One church or many?Topoi, 28,
1–7.
Knoblich, G., & Flach, R. (2001). Predicting the effects of
actions: Interactions of perception and action. Psychologi-
cal Science, 12, 467–472.
Knoblich, G., Seigerschmidt, E., Flach, R., & Prinz, W.
(2002). Authorship effects in the prediction of handwriting
strokes: Evidence for action simulation during action per-
ception. The Quarterly Journal of Experimental Psychol-
ogy: Section A, 55, 1027–1046.
Koeinig, P., Ktihnberger, K. U., & Kietzmann, T. C. (2013).
A unifying approach to high-and low-level cognition. In
U. Gähde (Ed.), Models, simulations, and the reduction of
complexity (vol. 4, pp. 117–139). Berlin, Germany: De
Gruyter.
Kohonen, T. (1998). The self-organizing map. Neurocomput-
ing, 21, 1–6.
Lacquaniti, F., Ivanenko, Y. P., & Zago, M. (2012). Develop-
ment of human locomotion. Current Opinion in Neurobiol-
ogy, 22, 822–828.
Lakoff, G., & Johnson, M. (1999). Philosophy in the Flesh
(vol. 4). New York, NY: Basic books.
Lara, B., Ciria, A., Escobar, E., Gaona, W., & Hermosillo, J.
(2018). Cognitive robotics: The new challenges in artificial
intelligence. In O. O. S. Villegas, M. Nandayapa, & I. Soto
(Eds.), Advanced topics on computer vision, control and
Adaptive Behavior
robotics in Mechatronics (pp. 321–348). Berlin, Germany:
Springer International Publishing AG.
Lara Guzmán, B., Hermosillo Valadez, J., & Baeza Mariscal,
K. (2017). Intentionality and internal models in artificial
agents. Pragmatics & Cognition, 23, 209–237.
Lüchinger, A. B., Hadders-Algra, M., Van Kan, C. M., & de
Vries, J. I. (2008). Fetal onset of general movements.
Pediatric Research, 63, 191–195.
McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C.
E. (2006). A proposal for the Dartmouth summer research
project on artificial intelligence. AI Magazine, 27, 12.
Maye, A., & Engel, A. K. (2012). Using sensorimotor contin-
gencies for prediction and action planning. In T. Ziemke,
C. Balkenius, & J. Hallam (Eds.), Proceedings of the Inter-
national Conference on Simulation of Adaptive Behavior
(pp. 106–116). Berlin, Germany: Springer.
Menary, R. (2010). Introduction to the special issue on 4E
cognition. Phenomenology and the Cognitive Sciences, 9,
459–463.
Mendoza Bock, E. (2013) Los términos nómadas de las Cien-
cias Cognitivas: el caso de los Modelos Directos (Master’s
thesis). Facultad de Humanidades, Montevideo, Uruguay.
Miall, R. C. (2003). Connecting mirror neurons and forward
models. Neuroreport, 14, 2135–2137.
Miall, R. C., & King, D. (2008). State estimation in the cere-
bellum. The Cerebellum, 7, 572–576.
Miall, R. C., & Wolpert, D. M. (1996). Forward models for
physiological motor control. Neural Networks, 9,
1265–1279.
Miall, R., Weir, D. J., Wolpert, D. M., & Stein, J. (1993). Is
the cerebellum a smith predictor? Journal of Motor Beha-
vior, 25, 203–216.
Mingers, J. (2001). Embodying information systems: The con-
tribution of phenomenology. Information and Organiza-
tion, 11, 103–128.
Möller, R., & Schenck, W. (2008). Bootstrapping cognition
from behavior a computerized thought experiment. Cogni-
tive Science, 32, 504–542.
Moons, T. (1998). Lecture notes in computer science. In
R. Koch, & L. V. Gool (Eds.), 3D structure from multiple
images of large-scale environments (Chapter: A guided tour
through multi view relations) (vol. 1506, pp. 304–346). Ber-
lin, Germany: Springer.
Moravec, H. (1984) Locomotion, vision and intelligence. In
M. Brady, & R. Paul (Eds.), Robotics Research (pp. 215–
224). Cambridge, MA: The MIT Press.
Morse, A. F., Greef, J. D., Belpaeme, T., & Cangelosi, A.
(2010). Epigenetic robotics architecture (era). IEEE Trans-
actions on Autonomous Mental Development, 2, 325–339.
Nagel, T. (1974). What is it like to be a bat? The Philosophical
Review, 83, 435–450.
Nasuto, S. J., & Hayashi, Y. (2016). Anticipation: Beyond
synthetic biology and cognitive robotics. Biosystems, 148,
22–31.
Newell, A., & Simon, H. A. (1976). Computer science as
empirical inquiry: Symbols and search. Communications of
the ACM, 19, 113–126.
Newen, A. (2017). What are cognitive processes? An example-
based approach. Synthese, 194, 4251–4268.
Noë, A. (2004). Action in perception. Cambridge, MA: The
MIT Press.
Lara et al.
Oztop, E., Kawato, M., & Arbib, M. (2006). Mirror neurons
and imitation: A computationally guided review. Neural
Networks, 19, 254–271. doi:10.1016/j.neunet.2006.02.002.
Pezzulo, G., Barsalou, L. W., Cangelosi, A., Fischer, M. H.,
McRae, K., & Spivey, M. J. (2011). The mechanics of
embodiment: A dialog on embodiment and computational
modeling. Frontiers in Psychology, 2, 5.
Pfeifer, R., & Bongard, J. (2006). How the body shapes the
way we think: A new view of intelligence. Cambridge, MA:
The MIT Press.
Pfeifer, R., Lungarella, M., & Iida, F. (2007). Self-organiza-
tion, embodiment, and biologically inspired robotics. Sci-
ence, 318, 1088–1093.
Pfeifer, R., & Scheier, C. (2001). Understanding intelligence.
Cambridge, MA: The MIT Press.
Pickering, M. J., & Clark, A. (2014). Getting ahead: Forward
models and their place in cognitive architecture. Trends in
Cognitive Sciences, 18, 451–456.
Pico, A., Schillaci, G., Hafner, V. V., & Lara, B. (2017). How
do I sound like? Forward models for robot ego-noise pre-
diction. In J. Santos - Victor, & G. Sandini, (Eds.), Pro-
ceedings of the Joint IEEE International Conference on
Development and Learning and Epigenetic Robotics (pp.
246–251). New York, NY: IEEE.
Riedmiller, M., & Braun, H. (1993). A direct adaptive method
for faster backpropagation learning: The RPROP algo-
rithm. In Proceedings of the IEEE International Conference
on Neural Networks (pp. 586–591). New York, NY: IEEE.
Robbins, P., & Aydede, M. (2009). The Cambridge handbook
of situated cognition. Cambridge, UK: Cambridge Univer-
sity Press.
Rodrı́guez, D. (2012) Análisis de la Cognición Corporizada
desde la perspectiva de la fenomenologı´a (Master’s thesis).
Facultad de Humanidades, Montevideo, Uruguay.
Rodrı́guez, D., Hermosillo, J., & Lara, B. (2012). Meaning in
artificial agents: The symbol grounding problem revisited.
Minds and Machines, 22, 25–34.
Rojas, E. (2014) Estudio sobre la caracterización de acciones
basado en modelos internos (Master’s thesis). Facultad de
Humanidades, Montevideo, Uruguay.
Rougier, N., & Boniface, Y. (2011). Dynamic self-organising
map. Neurocomputing, 74, 1840–1847.
Rowlands, M. (2010). The new science of the mind: From
extended mind to embodied phenomenology. Cambridge,
MA: The MIT Press.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985).
Learning internal representations by error propagation
(Technical Report). California Univ San Diego La Jolla
Inst for Cognitive Science. Retrieved from http://
www.dtic.mil/dtic/tr/fulltext/u2/a164453.pdf
Schillaci, G., Hafner, V. V., & Lara, B. (2012a). Coupled
inverse-forward models for action execution leading to
tool-use in a humanoid robot. In J. Han, A. Tapus, & J.
Crandall. (Eds.), Proceedings of the 7th ACM/IEEE Inter-
national Conference on Human-Robot Interaction (pp. 231–
232). New York, NY: IEEE.
Schillaci, G., Hafner, V. V., & Lara, B. (2016). Exploration
behaviors, body representations, and simulation processes
for the development of cognition in artificial agents. Fron-
tiers in Robotics and AI, 3, 39.
Schillaci, G., Hafner, V. V., Lara, B., & Grosjean, M. (2013).
Is that me? Sensorimotor learning and self-other
13
distinction in robotics. In J. Crandall, & S. Sabanovic,
(Eds.), Proceedings of the 8th ACM/IEEE International
Conference on Human-Robot Interaction (pp. 223–224).
New York, NY: IEEE.
Schillaci, G., Lara, B., & Hafner, V. (2012b). Internal simula-
tions for behaviour selection and recognition. In A. Salah,
J. Ruiz-del Solar, C. Mericli, & P. Y. Oudeyer (Eds.), Human
behavior understanding (Lecture notes in computer science)
(vol. 7559, pp. 148–160). Berlin, Germany: Springer.
Schmidt, A., Deleforge, A., Kellermann, W., Schmidt, A.,
Deleforge, A., Kellermann, W., & Kellermann, W. (2016).
Ego-noise reduction using a motor data-guided multichan-
nel dictionary. In W. Burgard (Ed.) Proceedings of the
IEEE/RSJ International Conference on Intelligent Robots
and Systems (IROS) (pp. 1281–1286). New York, NY:
IEEE.
Shapiro, L. (2007). The embodied cognition research pro-
gramme. Philosophy Compass, 2, 338–346.
Shapiro, L. (2010). Embodied cognition. London, England:
Routledge.
Shapiro, L. A. (2004). The mind incarnate. Cambridge, MA:
The MIT Press.
Tsai, R. Y. (1987). A versatile camera calibration technique
for high-accuracy 3D machine vision metrology using off-
the-shelf TV cameras and lenses. Journal of Robotics and
Automation, 3, 323–344.
Tsakiris, M., Schütz-Bosbach, S., & Gallagher, S. (2007). On
agency and body-ownership: Phenomenological and neu-
rocognitive reflections. Consciousness and Cognition, 16,
645–660.
Van Gelder, T., & Port, R. F. (1995). It’s about time: An over-
view of the dynamical approach to cognition. In R. Port,
& T. van Gelder (Eds.), Mind as motion: Explorations in
the dynamics of cognition (pp. 1–43). Cambridge, MA: The
MIT Press.
Varela, F. J., Thompson, E., & Rosch, E. (2017). The embo-
died mind: Cognitive science and human experience. Cam-
bridge, MA: The MIT press.
von Hofsten, C. (2003). On the development of perception and
action. London, England: SAGE.
von Hofsten, C. (2007). Action in development. Developmen-
tal Science, 10, 54–60.
von Hofsten, C. (2009). Action, the foundation for cognitive
development. Scandinavian Journal of Psychology, 50,
617–623.
Ward, D., & Stapleton, M. (2012). Es are good. In F. Paglieri
(Ed.), Consciousness in interaction: The role of the natural
and social context in shaping consciousness (pp. 86–89).
Amsterdam, The Netherlands: John Benjamins Publishing.
Webb, B. (2004). Neural mechanisms for prediction: Do insects
have forward models? Trends in Neurosciences, 27, 278–282.
Wegner, D. M., & Wheatley, T. (1999). Apparent mental cau-
sation: Sources of the experience of will. American Psychol-
ogist, 54, 480–492.
Weiss, C., Tsakiris, M., Haggard, P., & Schütz-Bosbach, S.
(2014). Agency in the sensorimotor system and its relation
to explicit action awareness. Neuropsychologia, 52, 82–92.
Wheeler, M., & Clark, A. (2008). Culture, embodiment and
genes: Unravelling the triple helix. Philosophical Transactions
of the Royal Society B: Biological Sciences, 363, 3563–3575.
Wilson, M. (2002). Six views of embodied cognition. Psycho-
nomic Bulletin & Review, 9, 625–636.
14 Adaptive Behavior

Wilson, M., & Knoblich, G. (2005). The case for motor invol-
vement in perceiving conspecifics. Psychological Bulletin,
131, 460–473.
Wolpert, D. M., Ghahramani, Z., & Flanagan, J. R. (2001).
Perspectives and problems in motor learning. Trends in
Cognitive Sciences, 5, 487–494.
Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995). An
internal model for sensorimotor integration. Science, 269,
1880–1882.
Wolpert, D. M., Goodbody, S. J., & Husain, M. (1998a).
Maintaining internal representations: The role of the
human superior parietal lobe. Nature Neuroscience, 1,
529–533.
Wolpert, D. M., Miall, R. C., & Kawato, M. (1998b). Inter-
nal models in the cerebellum. Trends in Cognitive Sciences,
2, 338–347.
Zambelli, M., & Demiris, Y. (2015). Online ensemble learning
of sensorimotor contingencies. In R. Téllez, G. Alenyà, C.
Angulo, & J. Kevin O’Regan, (Eds.) Proceedings of the
Sensorimotor Contingencies for Robotics Workshop at
IROS. Berlin, Germany: Springer.

About the Authors

Bruno Lara is a Professor at the Universidad Autónoma del Estado de Morelos (UAEM) and
the Head of the Cognitive Robotics Lab at the Science Research Center at the UAEM since
2005. He holds a PhD in Mechatronics from King’s College London. He did a postdoc in the
TheoriLabor in the University of Jena, working on evolutionary robotics, and then a postdoc in
the Max Planck Institute for Psychology Research in Munich, focusing on research on Cognitive
Robotics. In 2011, he spent a sabbatical stay in the Cognitive Robotics Lab at the Humboldt-
Universität zu Berlin. He is an Alexander von Humboldt Fellow. His research interests include
Internal Models, sensorimotor representations, and evolutionary robotics and learning.

Dadai Astorga obtained a Degree in Psychology from the UAEM and a Master’s degree in
Cognitive Sciences by the same university. She is an Active Member of the Cognitive Robotics
Laboratory. Her research is focused on Cognitive Sciences, specifically predictive coding, cogni-
tive robotics, agency, and autism.

Emmanuel Mendoza-Bock obtained a Master’s degree in Cognitive Sciences from the UAEM.
He is currently doing a PhD in Philosophy of Sciences at the National Autonomous University
of Mexico (UNAM), in the area of Philosophy of Cognitive Sciences. He is an Associate Student
of the Institute of Philosophical Investigations of the UNAM (EA-IIFs UNAM). His research is
focused on the philosophical analysis of the epistemic, theoretical, and methodological assump-
tions that underlie the postulates of Embodied Cognitive Sciences.
Manuel Pardo obtained a Degree in Philosophy and a Master’s degree in Cognitive Sciences
from the UAEM. He is an Active Member of the Cognitive Robotics Lab. His research interests
are focused on evolutionary robotics, Cognitive Sciences, and the philosophy of science.

Esaú Escobar received his BSc degree in the area of Computer Sciences from the Sciences
Faculty at UAEM in 2011 and PhD in Computational Modeling and Scientific Computing from
the Science Research Center at UAEM in 2017. He is currently a part-time Professor of
Computer Sciences in the Science Research Center at UAEM. He is an Active Member of the
Cognitive Robotics Lab. His research is focused on the study, development, and application of
Artificial Intelligence and machine learning in the field of cognitive robotics.

Alejandra Ciria obtained a Degree in Psychology from the Universidad de las Américas Puebla
(UDLAP). She obtained her Master’s degree in Cognitive Sciences from the UAEM. Currently,
she is about to obtain a PhD in Psychology, in the area of Experimental Psychology of
Behavior, from the Faculty of Psychology at the UNAM. She is an Active Member of the
Cognitive Robotics Lab. Her current research focuses on the study of Cognitive Sciences, specifi-
cally time perception, visual cognition, predictive processing, cognitive robotics, Internal
Models, and the philosophy of Artificial Intelligence.