DE GRUYTER Cognitive Semiotics.

2019; 20192006

Piero Polidoro1

Image schemas in visual semiotics: Looking for

an origin of plastic language
1 LUMSA University, Rome, Italy, E-mail: p.polidoro@lumsa.it. 0000-0002-2330-5910.

Abstract:
The aim of this article is to present a hypothesis explaining the origin of plastic meaning. In visual semiotics,
plastic meaning is that produced by visual configurations per se, i.e. independently from what they represent.
This meaning can be assimilated to the kind of effects studied by (Arnheim, R. 1954/1974. Art and visual percep-
tion: A psychology of the creative eye, 2nd edn. Berkeley, Los Angeles, and London: University of California Press).
In his book The Body In the Mind, (Johnson, M. 1987. The body in the mind. Chicago: The University of Chicago
Press) is the first to propose that image schemas and their metaphorical projections could be used to explain
some of these visual effects. Nevertheless, I think that his approach presents some shortcomings. Above all,
Johnson’s examples always concern cases in which visual stimuli match an image schema, while Arnheim’s
observations are mostly about effects of tension and dynamism generated by a conflict with our expectations.
I will propose that, to complete Johnson’s proposal, we need an inferential theory of aesthetic experience, de-
rived from Meyer’s and Eco’s works. This theory would explain how expectations and their verifications can
produce different kinds of tension and arousal, the basic mechanisms of plastic meaning.
Keywords: visual semiotics, plastic semiotics, image schema, embodiment, inferences, aesthetics
DOI: 10.1515/cogsem-2019-2006

1 Plastic level in visual semiotics

Visual semiotics distinguishes between two levels of the image: the figurative level and the plastic one. The
figuratve level has been called so by Greimas (1984) and his school, but it has also been named “iconic” by
Groupe μ (1992) or “pictorial” by Sonesson (1989). It concerns the way images depict objects of the world. The
plastic level, on the other hand, concerns the way visual elements (colours, shapes, spatial relations, et cetera)
and their arrangements convey meaning per se, i.e. regardless of whether or how they represent objects of the
world.
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

In Greimas’s tradition (Greimas, 1984; Floch, 1985; Thürlemann, 1982) the plastic level expresses, by means
of these visual elements, oppositions and relations existing on the content plane. This approach has some limits,
such as the fact that the plastic level becomes parasitic on the figurative (or iconic/pictorial) one; as Sonesson
(1989) has demonstrated, the semi-symbolic system, i.e. the main theoretical tool used by Greimas’s school to
analyse plastic level, always needs a certain meaning to have been identified at the figurative level in order to
associate it to plastic elements: “the content of plastic language […] can be derived from a series of iconical
contents, and is thus redundant in relation to these iconical contents” (ibid.: 154).
Another way to conceive plastic language is closer to the kind of effects studied by Arnheim (1954/1974).
In this case, the plastic level “uses shapes whose properties are abstract, to convey to us meanings often being
at even higher levels of abstraction” (Sonesson 2013: 534, see also, Sonesson 2015).
In this article, I will propose a theory to explain the origin of the meaning effects of the plastic level (for a
more comprehensive discussion of this hypothesis, cf. Polidoro 2004, Polidoro 2005). This theory is based on
the concept of image schema (even if I would prefer the expression embodied schema) as it has been presented by
Johnson (1987). According to Johnson (1987: 2), an image schema is “a dynamic pattern that functions some-
what like the abstract structure of an image, and thereby connects up a vast range of different experiences that
manifest this same recurring structure.” Image schemas originate from our bodily experience and they extend
their domain through metaphorical mappings (following the theory of conceptual metaphors introduced by
Lakoff and Johnson 1980).
The concept of image schema is anything but uniform: it has been used in different ways by many authors
(Hampe 2005). Zlatev (2005) and Sonesson (2007) have analyzed and criticized it thoroughly with interesting
and solid arguments. Nevertheless, the aim of this article is not to discuss image schemas, but to use some ideas
presented by Johnson (1987) to explain how some plastic effects are possible. My main purpose is to show how
Piero Polidoro is the corresponding author.
© 2019 Walter de Gruyter GmbH, Berlin/Boston.

Brought to you by | Stockholm University Library

Authenticated 1
Download Date | 5/27/19 11:34 AM
 Polidoro DE GRUYTER

these effects originate from our experience of the world in a way that, in my opinion, is close to how Sonesson
conceives plastic meaning:

Just as, according to Piaget, conceptual schemas are abstracted from actions through the many stages
of intellectual development, physiognomic properties could also be conceived to take their origin in the
actions of one’s own body, yet remain bound up with the body in all their further applications as the
deeper source of their sense.(2013: 544)

My discussion will be limited to one image schema, that of balance, but I am persuaded it can be extended to
others.

2 Plastic meaning according to Rudolf Arnheim

In order to understand the theoretical claim that I am going to develop here, it is useful to partially summarize
Arnheim’s (1954/1974) argument on the meaning effects in visual configurations. Arnheim’s implicit starting
point is the difference between the physical features in visual configurations (how they actually are) and their
perceptual outcome (how we see them). Commenting on the square in Figure 1(a), Arnheim (ibid.: 11) writes
that “if the disk is seen as striving toward the center of the square, it is being attracted by something not phys-
ically present in the picture.” A few pages after this, he adds: “As far as retinal input is concerned, the black
lines and the disk are all there is to our figure” (ibid.: 15).

Figure 1: Some of Arnheim’s (1954/1974: 10–18) examples of dynamism in a square. Author’s drawings, following Arn-
heim (1954/1974).

Looking at Figure 1(a), we immediately see that the position of the disk is not centred. There is no graphic sign
(e.g. a dot) indicating the centre of the disk, however we “see” it immediately, without any measurement. But,
most importantly,
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

there is something restless about it [the disk]. It looks as though it had been at the centre and wished
to return, or as though it wants to move away even farther. And the disk’s relations to the edges of the
square are a similar play of attraction and repulsion. (ibid.: 11)

There are different points of attraction in a square: the centre, the horizontal and vertical axes, the diagonals,
and so on. In general, each figure holds its own structural skeleton (cf. Figure 2), which organizes its main visual
characteristics (Arnheim refers to structural skeleton as “the configuration of visual forces that determines the
character of the visual object” [ibid.: 93]).

Figure 2: Structural skeletons of five triangles (ibid.: 94). Author’s drawings, following Arnheim (1954/1974).

But we do not only deal with single figures that interact with a background. We also need to analyse more
complex cases of interactions between figures. Let us take Figure 1(b). The two disks seem to be “balanced,”
even though, if each is considered by itself, they would seem “unbalanced.” On the other hand, Figure 1(c) has
no balance. Why? Because of the Gestalt laws of figural unification (in particular for those of similarity and

Brought to you by | Stockholm University Library

2 Authenticated
Download Date | 5/27/19 11:34 AM
 DE GRUYTER Polidoro

proximity), we tend to see the two disks as a unity. In the first case we would have the feeling of a balanced
structure because the centre of the identified unity (which is placed midway between the two disks) coincides
with the centre of the square. In the second case, we will have an “imbalance:” in Figure 1(c), the centre of the
unity composed by the two circles is displaced in relation to the centre of the square.
Arnheim focuses his attention on the dynamism of the visual configuration.

Visual experience is dynamic. This theme will recur throughout the present book. What a person or animal
perceives is not only an arrangement of objects, of colors and shapes, of movements and sizes. It is,
perhaps first of all, an interplay of directed tensions. (Ibid.: 11)

Hence, movements that are not present but rather “induced” (ibid.: 11–12), tensions which are always at play in
each Gestalt, are also part of the phenomenal outcome of a visual configuration. As argued by Arnheim (ibid.:
416), this idea is an elaboration of the concept of tension proposed by Kandinsky (1926).
But where is the origin of these directed tensions? Arnheim reviews some theories, which he criticises or
deems of little relevance. These theories share the assumption that our knowledge and experience of the en-
vironment plays a central role in making sense of visual configurations. What is Arnheim’s hypothesis then?
Mostly, he considers the tension as a consequence of the deviation from the good gestalt.

By now it will be evident that all tension derives from deformation. Whether we are dealing with a bent
steel blade, a sheet of rubber, a funhouse mirror, an expanding bubble, or the rising emotion of a heated
argument, there is always a forceful deviation from a state of lower tension in the direction of tension
increase. (Arnheim 1954/1974: 428)

Consistently with Gestaltpsychologie, Arnheim’s hypothesis relies on a nativist conception, from which my
approach here radically departs.
Once we have argued that the tension comes from the deviation from the good gestalt, we need to question
the origin of such good gestalts. Arnheim points out that he does not believe that the good gestalt comes from
our experience of the world.

Directed tension is as genuine a property of visual objects as size, shape, and color. The nervous system
of the observer generates it at the same time that it produces the experience of size, shape, and color
from the stimulus input. There is nothing arbitrary or willful in these dynamic components of percepts,
although they can be ambiguous. They are strictly determined by the nature of the visual pattern, even
in the range of their ambiguities. (Ibid.: 423)

The quotation is to be read as follows: the directed tensions are properties of the percept because, just as size
and other visual characteristics, they result from the action of the stimulus pattern on our nervous system. The
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

latter reacts automatically, without inferential processes but only based on the action of the constitutive genetic
structures. Tensions exist in the percept, namely: given a nervous system and a percept, the former cannot react
to the latter but in that way.
Arnheim goes even further and elaborates a hypothesis (which is inspired by the gestalt isomorphism) about
how tensions originate in the nervous system. A specific pattern of stimuli hits the retina. If it is different from
a good gestalt, the nervous system tries to contrast it.

A struggle must result as the invading forces try to maintain themselves against the physiological field
forces, which endeavour to eliminate the intruder or at least to reduce it to the simplest possible pattern.
The relative strength of the antagonistic forces determines the resulting percept […] I suggest that it
is these forces which we perceive as “directed tension” or “movement” in immobile patterns. In other
words, we are dealing with the psychological counterpart of the physiological processes that result in
the organization of perceptual stimuli. (Ibid.: 438)

However, Arnheim’s idea raises many doubts. In fact, it requires us to accept that an underspecified physio-
logical tension – originated by the competition between two patterns – is transformed into a visual tension at
some higher level. How this physiological process would emerge into our consciousness and why in this way
(i.e. as a perceived tension) is not very clear.

3 Mark Johnson’s reinterpretation of Arnheim

By drawing on Arnheim’s observations about the structural skeleton of the square, Johnson (1987) argues for
interpreting the visual effects described by Arnheim in terms of image schemas. Arnheim talks about balance

Brought to you by | Stockholm University Library

Authenticated 3
Download Date | 5/27/19 11:34 AM
 Polidoro DE GRUYTER

or imbalance (and then of tensions) according to the disposition of the disks within the square. Johnson argues
that we formulate such a judgment because we describe the visual experience metaphorically in terms of a
schema, whose origin is the body. The schema in question is that of the balance, which in its most general
form can be described as “a point or axis around which forces and weights must be distributed, so that they
‘counteract’ or ‘balance off’ one another” (ibid.: 80).
In contrast to Arnheim, Johnson makes two important clarifications. First, he stresses that balance and lack
of balance are not features inherent in the object but result from our perceptive experience: “the balance does
not exist objectively in the figure drawn on the page, as if the balance were just there to be passively perceived
by anyone. The disk on the white square is only balanced in our acts of perception” (ibid.: 79). Actually, even if
Arnheim’s words seem to suggest that tensions are indeed inherent in the object, we have already seen that those
words have to be interpreted differently: the fact that tensions exist in the percept means that a specific nervous
system cannot react but in that way to a specific percept. From this perspective, Arnheim’s point of view is not
very far from what Johnson writes: “However, since all humans have the same perceptual hardware, more or
less, it will usually make perfectly good sense to speak of the balance being in the perceived object” (ibid.: 79).
Actually, in my opinion there is a slight but important difference between them. In Arnheim’s case, we deal
with two elements: the morphological properties of the shapes and the genetically determined characteristics
of the nervous system; dynamic tensions are generated by the way the latter respond to the former. In Johnson,
we have to introduce a third element: image schemas, which – in some way we still ignore – are implemented in
the nervous system, but ontogenetically, i.e. by means of our interaction with the environment. These schemas
may be the same for all humans, but the reason is that we share the same kind of environment and bodily
experience and not only – more or less – the same genetic material. In other words, Johnson’s explanation of
visual dynamism is partly based on acquired schemas, while Arnheim’s one relies on innate structures.
Johnson’s second clarification concerns the nature of the “forces” and “weights” which Arnheim talks about:
they are not physical forces or tensions, but psychological or perceptive ones.

[…] what Arnheim calls “forces,” “tensions,” and “hidden structures” active in my perceptual grasp
of balance or lack of balance in the figures are clearly psychological or perceptual forces. They are not
gravitational or physical forces of the sort that operate on me when I am balancing my body. We might
call them visual forces or weights to emphasize this important difference. In these cases, “forces” and
“weight” are obviously interpreted metaphorically. (Ibid.: 79–80)

Soon after Johnson specifies:

“Weight” is used metaphorically here in a standard sense of the term “metaphor”: we structure and
understand a domain of one kind (here, the psychological/perceptual) in terms of structure projected
from a domain of a different kind (here, the physical/gravitational). (Ibid.: 80)
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

In the next two paragraphs, we will see how these acquired image schemas are structured and how they can
explain visual dynamism and tensions.

4 The balance schema

All Johnson’s observations on visual language concern the balance schema. In fact, balance (or lack thereof) is
one of the most important and pervasive plastic effects. Hence, it is important to analyse the balance schema
more in depth.
Like all image schemas, the balance schema originates from the bodily interaction with the world.

The baby stands, wobbles, and drops to the floor. It tries again, and again, and again, until a new world
opens up – the world of the balanced erect posture. There are those few days when the synapse con-
nections are being established and then, fairly suddenly, the baby becomes a little homo erectus. (Ibid.:
74)

There are several senses of the term balance, but Johnson does not believe that these are cases of homonymy;
indeed, he points out that all these senses share the term balance because they actually stem from the general
concept of balance.
This general concept is represented by the prototypical schema of balance (cf. Figure 3), which describes
a situation in which there is “a symmetrical (or proportional) arrangement of forces around a point or axis”
(ibid.: 85).

Brought to you by | Stockholm University Library

4 Authenticated
Download Date | 5/27/19 11:34 AM
 DE GRUYTER Polidoro

Figure 3: Prototypical balance schema or axis balance (ibid.: 86). Author’s drawings, following Johnson (1987).

There are also some additional and more specific schemas, which are derived from the prototypical one. One
of theme is what Johnson calls twin-pan balance (cf. Figure 4): the axis becomes a simple dot, a fulcrum against
which we apply some forces.

Figure 4: Twin-pan balance schema (ibid.: 86). Author’s drawings, following Johnson (1987).
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

Another schema is called equilibrium (cf. Figure 5):

Figure 5: Equilibrium schema (ibid.: 87). Author’s drawings, following Johnson (1987).

Brought to you by | Stockholm University Library

Authenticated 5
Download Date | 5/27/19 11:34 AM
 Polidoro DE GRUYTER

5 Johnson’s visual analysis

By drawing on the just described image schema and its metaphorical projection, Johnson analyses and explains
some cases of visual balance. As a first example, Johnson takes a Kifwebe ritual mask, which he considers par-
ticularly balanced. First of all, Johnson points out that we should discard the idea that what is at stake in these
cases is an actual physical balance. In other terms, we should not believe that the mask is balanced for real and
that this would be the reason, for example, that it does not fall forward.

We experience our visual space as having parts or elements with different weights and forces of different
magnitudes. These visual forces exist only in our act of perception. As we view the Kifwebe mask, these
forces give rise to tensions, movement, rest, and balance that we experience in our active grasp of rela-
tions among elements in the face. There are two sides of the face with parts (eyes, ears, scars, etc.) having
different visual weights that are, in the total visual gestalt, in balance. The key point here is that there
are not physical weights or forces in balance; rather, the shapes, lines, forms, and spatial relationships
involve visual weights and forces. (Ibid.: 81)

The case of the Kifwebe mask could be misleading in that it may make us believe that the balance coincides
with symmetry. Actually, symmetry represents just a specific case of balance, which can also be the result of
a proportional arrangement of forces (ibid.: 85). Another analysis, now of an Udo bronze, shows how a visual
balance can emerge even without a symmetrical structure (in that case, for example, the right hand holds a
sword that the left hand does not have).
However, we may still believe that the visual balance is to be directly attributed to the physical balance. In
other words, the mask and the bronze appear as balanced to us because they represent objects of the real world
that would be more or less balanced in those positions. If we accept this argument, we would not need to talk
about “metaphorical projection,” because our appraisal would just come from the actual physical balance of an
object.
But, by applying such an analysis to abstract paintings, it is indeed possible to show how a visual balance –
certainly metaphorical – actually exists.
Johnson takes as his example a Kandinsky painting: Accompanied Contrast. In his analysis, Johnson considers
that painting as particularly balanced. Accompanied Contrast and, more generally, abstract and non-figurative art,
do not represent objects of the real world; thus, it is not possible to make them the object of any judgment con-
cerning physical balance. The judgment of greater or lesser balance is purely metaphorical, because it is based
on the attribution of certain characteristics (weight, force, etc.) to colour spots, which clearly do not actually
have any of them: “the metaphor consists in the projection of structure from one domain (that of gravitational
and other physical forces) onto another domain of a different kind (spatial organization in visual perception)”
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

(ibid.: 82).

6 Some reflections on Johnson’s theory

Johnson’s theory provides us with a possible basis for explaining plastic effects. In fact, forms and colours do
not have weight and do not exert a force by themselves; a dynamic or little balanced painting does not move or
tilt to one side by itself: we are the ones who attribute to plastic elements some characteristics that they cannot
have. A hypothesis is that we do this by projecting on them schemas that result from, and respond to, our bodily
interactions with the world.
We have to credit Johnson as the first to see the possible application of the embodied schemas to what in
semiotics is defined as the plastic level. As mentioned above, this idea has always been present, more or less
explicitly, in the aesthetic domain and its analysis: in particular, from Point and Line to Plane and on, Kandinsky
(1926) seems already to envision such a hypothesis. However, Johnson’s specific contribution was to move the
problem of the metaphorical mechanism from the background to the centre stage of the discussion.
Nevertheless, I think that his approach presents some shortcomings. In order to be an integral part of a
theory of the plastic language, Johnson’s approach has to be further developed in some of its aspects. We need
to understand how schemas work and how they are projected on our experiences and perceptions. Furthermore,
Johnson always takes as his examples (two- or three-dimensional) images that are recognized as “balanced.”
In other words, in his cases the projection of a specific schema (the balance schema) is always successful. But,
as is evident in the examples that Johnson himself mentions, Arnheim’s observations are mostly about effects
of tension and dynamism generated by a deviation from our expectations and, thus, from the schemas we have
activated. Hence, it is important to understand also what happens when the projection fails.

Brought to you by | Stockholm University Library

6 Authenticated
Download Date | 5/27/19 11:34 AM
 DE GRUYTER Polidoro

7 Recognition of schemas and meaning-making

We can see schemas as systems that respond to patterns of stimuli. The schema activates – i.e. it responds posi-
tively – when it receives a pattern of stimuli that possesses certain characteristics. Once stabilized, schemas are
capable of managing different set of experiences; they can elaborate information not only coming from differ-
ent sensory channels (proprioception, visual, and so on), but also from domains that can be elaborated after a
metaphorical extension (for example when we say that “a discourse is balanced”). At a certain moment in their
path, these patterns have to be transformed into something that is abstract enough to make their form homoge-
neous, regardless of their sources. Thus, the schema of balance should allow the evaluation both of a situation
in which our body is in balance and the tension in a painting. This means that, before the schema processes
them, the patterns of stimuli coming from proprioception or from visual perception have to be converted into
a pattern of the same format. This can happen through processes of abstraction that are generated outside the
schema or that are integrated in the schema itself.
Let us take the specific case of the schema of balance. From birth we learn to move in the world and acquire
gradually the ability to stay in balance. We learn that if we feel an imbalance of force we have to compensate
it with one or more equivalent and opposing forces. Through this kind of experience, a schema takes shape,
consolidates, and becomes more and more solid. Once this schema is stable, its domain can be metaphorically
extended, up to include aspects of our experience that are different from those it originally applied.
I want now to propose a theoretical hypothesis that develops Johnson’s idea in more detail. What does
happen when a pattern of stimuli activates a given schema? It is not very clear if the activation of the schema is
the consequence of an immediate matching between the pattern and the schema or a more articulated (although
unconscious) processing of perceptual data. The latter option seems to be supported by the fact that Johnson
considers image schemas as propositional, where proposition has to be intended as “a continuous, analogue
pattern of experience or understanding, with sufficient internal structure to permit inferences” (Johnson 1987:
3–4). Actually, these two hypotheses are not necessarily mutually exclusive.
Let us now consider the case of balance schema. If the information conveyed by the pattern matches the
structures – or the conditions – of the schema (because the pattern is a set of symmetrical and proportionate
elements and forces), this will be activated, and we will have the recognition of a situation of balance. We can
also suppose that this recognition does not necessarily emerge to consciousness. At least for an adult, walking
is an almost automatic action and demands a minimal awareness; the different stimuli coming from the pro-
prioceptive system, the visual perception and the vestibular system may activate the schema, but without an
evident impact on our experience. In this case, image schemas operate in the domain of what Gallagher (1986:
548) calls body schema:
The body schema is neither the perception, nor the cognitive understanding, nor the emotional appre-
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

hension of the body. The body schema, as distinct from the body image, is a non-conscious performance
of the body – i.e., a performance that is not an intentional object present to my consciousness. In this
performance the body acquires a certain organization or style in its relations with its environment.
However, according to Gallagher there are “limit-situations” in which “the body manifests itself in conscious-
ness” (ibid.: 545); this is the domain of body image and certainly applies to the balance image schema too.
We can postulate that image schemas are part of the network of semiotic units that constitute what Eco (1984)
calls the encyclopaedia, i.e. the overall semantic system that let us make sense of the world. The encyclopaedia is
not made up only of conscious knowledge, but also of perceptual schemas (1997) and, more generally, of every
element which may function as a Peircean interpretant in the emergence of meaning; if something such as image
schemas exists, from Eco’s point of view it must be part of the encyclopaedia and thus it is interconnected with
the rest of this semantic system.
In Gallagher’s limit-situations, the activity of image schemas becomes more relevant. In these cases, we can
suppose that the activation of a schema such as balance schema quickly leads to the activation of other semiotic
units (or interpretants), which are linked to the schema of balance. Thus, a semiotic chain, in the proper sense,
will emerge in which the recognition of a given situation is linked to a series of feelings, emotions, memories,
actions, and so on … For instance, in the case of balance, we can think that the schema is linked to a set of areas
that deal with certain motor sensations or emotions (from the more general ones, dealing with the opposition
euphoria/dysphoria, to the more complex ones). For what concerns the specific case of emotions, we should
remember how Mandler mentions (and endorses) a series of theories arguing for the existence of “‘pure’ feeling
entities in the underlying mental representation” (1984: 276). In other words, the cognitive system includes a
set of nodes that are responsible for the affective experiences. They are activated by the analysis of external
situations, but they are also linked to memories and schemas of evaluation. On one hand, this implies that a
given emotion or mood can bring back to memory some past experiences; on the other hand, the activation of
certain memories and schemas of evaluation can trigger a specific emotion.

Brought to you by | Stockholm University Library

Authenticated 7
Download Date | 5/27/19 11:34 AM
 Polidoro DE GRUYTER

If we accept Johnson’s thesis that image schemas, by means of a metaphorical process, can be applied to
domains different from those which originated them, we have also to presume that the activation of a schema
– whether it deals with the original domains or with their metaphorical extensions – produces under certain
conditions the semiotic process I have just described.

8 Projections and transformations

Up to now, we have seen how we can use the theory of image schemas in order to explain the mechanisms that
are present at the plastic level and how the recognition of one such mechanism within a visual configuration
produces meaning. Yet, as we have seen, Arnheim mostly focused on the effects produced in cases of a lack of
balance (and thus cases of dynamism and tension). In order to completely reconnect the image schemas and
the plastic effects, we need to add another element, namely an inferential theory of aesthetic experience.
Such a theory runs through most parts of Eco’s aesthetic thought, from his first pre-semiotic work (1962) to,
at least, the end of the 1970s (1979). Eco’s theory is based both on the pivotal role of the inferential mechanisms in
Peirce’s thought and on the musicological theory by Leonard B. Meyer (1956); Meyer’s and Eco’s ideas are then
the point of departure for the work of several scholars (Marconi, 2001; Barbieri, 2004; Polidoro, 2004, Polidoro
2015, Polidoro in press).
According to Eco, in order to elaborate hypotheses on how a text will develop, the user continuously makes
inferences: for instance, he/she tries to guess what is about to happen in a crime novel or how a melody line
in a composition will continue. Drawing on Meyer’s studies, Eco considers of great importance not just the
inferential activity, but also the cases in which the forecasts of the user turn out to be wrong, namely what
happens when there is a deviation between the text and the norm assumed by the user. Indeed, Eco (1962,
1979) seems to identify in this deviation the very engine driving many aesthetic effects. Eco’s reflections have
been further developed by Marconi (2001) and Barbieri (2004); both agree in recognizing at least two funda-
mental processes in aesthetic effects. The first is the inferential activity in the proper sense, the formulation of
a hypothesis on the text, which we can call tendency or protention (a concept that seems to be very similar to
its Husserlean homonym). The second is the actual tension, namely what follows the confutation of our textual
hypothesis. Both protention and tension bring about a state of arousal, the physiological and cognitive basis of
the aesthetic experience, which can be further oriented and defined according to the elements and meanings
present at other levels of the text.
In sum, the just described inferential theory of aesthetical experience explains how during our interaction
with a text some elements lead us to develop hypotheses about its continuation. The user expects that these
hypotheses will be confirmed, but this is not always the case. These mechanisms can generate three different
types of effects, which can obviously overlap and that are all connected with different forms of arousal:
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

– Tendency or protention: the user expects a certain continuation of the text;

– Indecision or ambiguity: the user faces an ambiguous situation and is not able to decide how the text will
further develop;

– Tension: the user’s hypothesis is not confirmed; this causes a surprise and leads the user to look for an expli-
cation for the failure of the hypothesis.

Marconi (2001) and Barbieri (2004) have showed these mechanisms at work in different kinds of texts, such as
pieces of music, poetries, comics, videogames, and so on. In a previous work (Polidoro 2004), I have tried to
show how this set of concepts is compatible with other researches in the neurosciences. Fiorillo et al. (2003) have
studied the role of monkey ventral mid-brain dopamine neurons in the prediction of reward or in uncertain
situations. Dopamine is strongly involved in primate motivational salience, attention, reward, and reinforce-
ment. Fiorillo and his team used for their experiments monkeys conditioned by means of a Pavlovian procedure
and able to recognize signs of different reward probabilities. Once conditioned, these monkeys were studied
in situations in which the expected reward was actually given or, on the contrary, not given, or in which an
unexpected reward was given. The authors discovered three kinds of reaction from these monkeys’ dopamine
neurons: 1) between the sign and the expected reward (a situation we have called protention), neuron response
is proportional to the probability of the reward; 2) after the period of time during which the reward is expected
to arrive, neuron response is proportional to what we can call the surprise, i.e. the difference between the event
and its probability (the same mechanism we have presumed in tension); 3) uncertain situations (in which proba-
bility is 50%, equivalent to our condition of ambiguity) produce a peak in the so-called sustained response, which
is low when probability is 0% or 100%.

Brought to you by | Stockholm University Library

8 Authenticated
Download Date | 5/27/19 11:34 AM
 DE GRUYTER Polidoro

How can we apply this theoretical approach to our discussion on the plastic level and image schemas? Let
us take again our example of the schema of balance in the observation of a painting. In such a situation, we can
assume that the user expects to find balanced structures in the painting (for example, for cultural reasons). This
entails the formulation of a hypothesis and then the triggering of a first meaning process, i.e. the protention,
involving the schema of balance. This also could imply that the schema of balance will not operate in the deep
of the body schema, as in everyday life, because aesthetic experience – as proposed by Barbieri (2004) – is a
specific condition in which our attention is already focused on some aspects of our environment (in this case,
for instance, the visual balance of the painting we are admiring).
The schema will give a positive (balance) or negative (lack of balance) response. In the case of a positive
response (i.e. confirmation of our hypothesis), the schema will activate all the related meaning effects, as we
have seen in the previous section. For example, the balance could transmit a sense of stillness, safeness, and
so on (these associations depend on the ontogenetic development of the subject as well as on his/her cultural
environment). Conversely, if the projection fails and the hypothesis of the presence of balance is then rejected,
a tension will be immediately produced, triggering an arousal and a magnification of the following effects.
Our experience tells us that normally all the systems are in a state of balance and if they move away from it,
they will finally tend to return to a situation of balance; we will then suppose that the initial state was disrupted
by some transformation that brought about the imbalance. The system will next start to compare the normal
(the balanced one) with the actual configuration, singling out the differences and trying to understand their
cause. If we take the square in Figure 1(a), analysed by Arnheim, the imbalance will probably be interpreted
as the consequence of a “displacement” of the black disk from one of its stable positions, the centre. But at an
early age, we already experience displacements and we have developed other schemas able to explain them – for
example, the schema of “force” (Johnson 1987: 2). Like the schema of balance, also the schema of force is linked
to other specific meanings. In this way, the plastic meaning further develops. Dynamism and meaning effects
that we experience looking at the square analysed by Arnheim can be thus arise from a series of mechanisms:
– the tension coming from the failure in the projection of the schema;
– the possible dysphoric feeling, or “unnatural” disposition, coming from the situation of imbalance;
– the perception of a force that can explicate the displacement of the disk from its position of stability (this
perception is due to the detection of a transformation and its explication through another schema);
– the perception of a force of reaction, which tends to restore the initial state, bringing the disk back to the
centre;
– the activation of meanings linked to the experience of force.
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

9 Conclusions: Image schemas and deviation

In this last section I want, first of all, to explain why this discussion about the metaphorical origin of plastic
language should be relevant for a semiotic theory of metaphor. Johnson’s image schemas derive from the con-
ceptual metaphors he introduced in his work with Lakoff (Lakoff and Johnson 1980). Actually, neither Lakoff
and Johnson (1980) nor Johnson (1987) provide a theory of how metaphors really work, i.e. why it is possible
to substitute a term for another one even if they belong to different domains and which are the steps and the
internal mechanisms of this process. They take for granted that metaphors exist and that they work on the ba-
sis of some kind of analogy, but their main focus is on the consequences for our conceptual system and our
reasoning of metaphorical extensions. Several semiotic traditions (Groupe μ 1970, Groupe μ 1992; Eco 1984;
Sonesson 1989) have instead provided detailed semantic models describing conditions and rules for the cre-
ation of metaphors. Eco (1984), for instance, elaborates an explanation of metaphor based on his componential
semantics; this model is compatible with the proposals of both Lakoff and Johnson and could provide to these
theories the semiotic basis (a semiotic description of the creation of metaphors) they lack. Nevertheless, here I
do not want to discuss this compatibility; I would rather focus on other intersections between semiotic theories
of metaphor and theories of conceptual metaphors and image schemas.
I hope to have shown that Johnson’s theory of image schemas can be useful for explaining the origin of plas-
tic semiotics, a central issue in visual semiotics. It is very hard to explain how we can describe (and experience)
a painting in terms of balance and weight if we do not rely on a theory of metaphorical extensions through
different domains such as the one provided by Johnson with image schemas. Metaphorically extended image
schemas are thus essential to explain a truly semiotic and aesthetic question such as the functioning of plastic
meaning. On the other side, Johnson’s theory, once applied to this kind of phenomena, needs to be comple-
mented with semiotic contributions. For instance, we need an inferential theory of the aesthetic experience in

Brought to you by | Stockholm University Library

Authenticated 9
Download Date | 5/27/19 11:34 AM
 Polidoro DE GRUYTER

order to explain why, given a visual configuration, the plastic meaning does not take place only when a schema
is recognized, but also when the schema is not recognized.
However, there is an important difference between all these theories and Lakoff and Johnson’s approach.
Semiotics usually devotes its efforts to analysing metaphors in aesthetic texts (from poetry to advertisement),
which are aimed to attract attention and lead to reflection through the deviation from a norm. Semiotics is
mostly interested in new metaphors that need to be interpreted every time (this is the reason they are aesthet-
ically effective). Conversely, Lakoff and Johnson (1980) deal with metaphors that are rarely perceived as such,
because already strongly present in language. To take a well-known example (“ARGUMENT IS WAR”), the fact
that we can “attack” the arguments of another person is not perceived as a metaphor, but as a literal expression
that expresses the way we see discussion.
The hypothesis that I have proposed in this article can be considered as a bridge between these two ap-
proaches and in particular between the classical and semiotic tradition and the idea of image schemas intro-
duced by Johnson. On one hand, in fact, image schemas explain the origin of some meaning effects that would
be otherwise very difficult to understand (the extension of typically bodily evaluations – the balance – to a
visual experience); on the other hand, their integration into an inferential theory of the aesthetic experience
allows us to overcome the shortcomings in Lakoff and Johnson’s approach (which is limited to the situation in
which a schema is recognized) and to take into account the dynamics of norm-deviation on which the classical
and semiotic studies on metaphor are based.
In conclusion, the aim of this article can be considered threefold. The first aim has been to outline a theo-
retical proposal about the origin of plastic meaning based on Johnson’s image schemas. The second one has
been to observe that semiotic theories of metaphors are not incompatible with conceptual metaphors and im-
age schemas: the former can provide semantic models of metaphor structure to the latter. Finally, I wanted to
show that the only way for image schemas to explain the kind of plastic effects described by Arnheim is to
be supported by an inferential theory of aesthetic experience, but this theory is based on the same deviation-
norm mechanism shared by traditional and semiotic theories of metaphors. Hence, the discussion about plastic
meaning gives the opportunity to include in the same theoretical frame both image schemas and the rhetorical
tradition.

References
Arnheim, R. 1954/1974. Art and visual perception: A psychology of the creative eye, 2nd edn. Berkeley, Los Angeles, and London: University of Cali-
fornia Press.
Barbieri, D. 2004. Nel corso del testo. Milano: Bompiani.
Eco, U. 1962. Opera aperta. Milano: Bompiani.
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

Eco, U. 1979. The role of the reader. Bloomington: Indiana University Press.
Eco, U. 1984. Semiotica e filosofia del linguaggio. Torino: Einaudi.
Eco, U. 1997. Kant e l’ornitorinco. Milano: Bompiani.
Fiorillo, C. D., P. N. Tobler & W. Schultz. 2003. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299(3).
1898–1902.
Floch, J.M. 1985. Petites mythologie de l’oeil et de l’esprit: Pour une sémiotique plastique. Paris and Amsterdam: Hadès-Benjamin.
Gallagher, S. 1986. Body image and body schema: A conceptual clarification. The Journal of Mind and Behavior 7(4). 541–554.
Greimas, A. J. 1984. Sémiotique figurative et sémiotique plastique. Actes sémiotiques. documents 60. 5–24.
Groupe μ. 1970. Rhétorique générale. Paris: Larousse.
Groupe μ. 1992. Traité du signe visuel. Paris: Seuil.
Hampe, B. (ed). 2005. From perception to meaning: Image schemas in cognitive linguistics. Berlin and New York: Mouton de Gruyter.
Johnson, M. 1987. The body in the mind. Chicago: The University of Chicago Press.
Kandinsky, W. 1926. Punkt und Linie zu Fläche. München: Langen.
Lakoff, G. & M. Johnson. 1980. Metaphors we live by. Chicago: The University of Chicago Press.
Mandler, G. 1984. Mind and body: Psychology of emotions and stress. New York and London: Norton.
Marconi, L. 2001. Musica espressione emozione. Bologna: Clueb.
Meyer, L. B. 1956. Emotion and meaning in music. Chicago: The University of Chicago Press.
Polidoro, P. 2004. Inferenze, tensioni e metafore: I meccanismi del linguaggio plastico. Versus 98/99. 39–66.
Polidoro, P. 2005. La semiotica visiva da un punto di vista interpretative: Iconismo e teoria del linguaggio plastico. Ph.D. Thesis. Bologna: Università di
Bologna.
Polidoro, P. 2015. L’attività inferenziale e le aspettative nel pensiero estetico di Umberto Eco. Zagadnienia Rodzajów Literackich 58(116). 63–74.
Polidoro, P. In press. Tensioni e passioni nella prima sequenza di Saving Private Ryan. Carte semiotiche.
Sonesson, G. 1989. Pictorial concepts. Lund: Lund University Press.
Sonesson, G. 2007. From the meaning of embodiement to the embodiement of the meaning: A study in phenomenological semiotics. In T.
Ziemke, J. Zlatev & R. M. Frank (eds.), Body, language, and mind, 85–128. Berlin and New York: Mouton de Gruyter.
Sonesson, G. 2013. Aspects of “physiognomic depiction” in pictures: From macchia to microgenesis. Culture & Psychology 19(4). 533–547.

Brought to you by | Stockholm University Library

10 Authenticated
Download Date | 5/27/19 11:34 AM
 DE GRUYTER Polidoro

Sonesson, G. 2015. Signs and Gestalten: From visual thinking to pictorial concepts. Gestalt Theory 37(3). 257–272.
Thürlemann, F. 1982. Paul Klee: Analyse sémiotique de trois pentures. Lausanne: L’age de l’homme.
Zlatev, J. 2005. What’s in a schema? Bodily mimesis and the grounding of language. In B. Hampe (ed.), From perception to meaning: Image
schemas in cognitive linguistics, 313–342. Berlin and New York: Mouton de Gruyter.

Bionotes
Piero Polidoro is an associate professor in Semiotics at LUMSA University in Rome. He has a Ph.D. in
Semiotics (University of Bologna, 2005). His thesis, supervised by Umberto Eco and Patrizia Violi, dealt
with two important aspects of visual semiotics: recognition and the origins and mechanisms of plastic
language. From 2006 to 2008, he had a post-doctoral fellowship at Istituto Italiano di Scienze Umane. His
research project, supervised by Omar Calabrese, was about the interpretative cooperation in visual texts. His
main research interests are in general semiotics, visual semiotics (visual perception, visual identity, visual
narration), communication strategy, and qualitative website analysis. His approach is based on interpretative
and structural semiotics, but he is also open to cognitive sciences and visual studies.
Automatically generated rough PDF by ProofCheck from River Valley Technologies Ltd

Brought to you by | Stockholm University Library

Authenticated 11
Download Date | 5/27/19 11:34 AM