I What Is A Diagram and How Does It Function?

All rights reserved.
May not be reproduced in any form without permission from the publisher, except fair uses permitted under U.S. or applicable copyright law.
I What is a diagram and how does it function?

Copyright 2016. De Gruyter Mouton.
EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 4/6/2023 12:15 PM via UNIVERSITT FR WEITERBILDUNG KREMS
AN: 1289667 ; Sybille Krmer, Christina Ljungberg.; Thinking with Diagrams : The Semiotic Basis of Human Cognition
Account: ns153208.main.eds
EBSCOhost - printed on 4/6/2023 12:15 PM via UNIVERSITT FR WEITERBILDUNG KREMS. All use subject to https://www.ebsco.com/terms-of-use
Michael Marrinan
1 On the “thing-ness” of diagrams
1 Introduction: Peirce looks at paintings
Icons are so completely substituted for their objects as hardly to be distinguished from
them. A diagram, indeed, so far as it has a general signification, is not a pure icon; but
in the middle part of our reasonings we forget that abstractness in great measure, and
the diagram is for us the very thing. So in contemplating a painting, there is a moment
when we lose the consciousness that it is not the thing, the distinction of the real and
the copy disappears, and it is for the moment a pure dream – not any particular
existence, and yet not general. At that moment we are contemplating an icon.
– Charles Sanders Peirce (W 5: 163)
Any art historian interested in looking at pictures must attend to this passage in
which a founding figure of the theory of signs – most commonly recognized
in the triad Icon/Index/Symbol – reflects upon the act of looking at a painting.
An art historian also interested in diagrams cannot afford to ignore the way
Peirce couples this act of contemplation to his account of diagrams (Bender
and Marrinan 2010), especially his claim that for a moment “the diagram is for
us the very thing.” In this moment, when the distinction between “the real and
the copy disappears,” we confront the icon. What does it mean for a diagram
to become “the very thing”? How does Peirce explain this transformation? Does
his account square with the way art historians look at pictures? Are there other
ways to think of diagrams as “things” in their own right? These are some of the
questions I will address in this essay.
2 Some Vagaries of Peirce’s thinking about

Diagrams
Frederik Stjernfelt’s meticulous close-reading of Peirce’s writings about diagrams
makes it possible for non-specialists (like myself) to understand more fully
the internal mechanics of Peirce’s thinking (Stjernfelt 2007, chapters 3 and 4;
Stjernfelt 2000; see also Stjernfelt and Østergaard in this volume). There is no
doubt that diagrams are necessary to rational thought, as when Peirce writes in
1885 “deduction consists in constructing an icon or diagram the relation of
Michael Marrinan, Stanford
24 Michael Marrinan
whose parts shall present a complete analogy with those of the parts of the
objects of reasoning, of experimenting upon this image in the imagination, and
of observing the results so as to discover unnoticed and hidden relations among
the parts” (W 5: 164). At about the same time, he articulates four steps of deduc-
tive reasoning in which diagram construction plays the essential role, noting
that “even though the diagram exists only in the imagination . . . after it has
once been created, though the reasoner has power to change it, he has no power
to make the creation already past and done different from what it is. It is, there-
fore, just as real an object as if drawn on paper” (Peirce 1976a, 4: 275–276). A few
years later, he insists that “it is by icons only that we really reason, and abstract
statements are valueless in reasoning except so far as they aid us to construct
diagrams” (CP 4: 127). Peirce begins “Prolegomena for an Apology to Pragma-
tism” of 1906, his most detailed discussion of diagrams, by stating: “All neces-
sary reasoning is diagrammatic; and the assurance furnished by all other
reasoning must be based on necessary reasoning. In this sense, all reasoning
depends directly or indirectly upon diagrams” (Peirce 1976b, 4: 314). Stjernfelt’s
study of this essential text helps us to isolate the principal ideas underpinning
Peirce’s account of diagrammatic reasoning.
Most important, Peirce insists the diagram “is an Icon of a set of rationally
related objects” so that it is amenable to rational experiments and compatible
with reasoning (Peirce 1976b, 4: 316). Stjernfelt points out that “as soon as an
icon is contemplated as a whole consisting of interrelated parts whose relations
are subject to experimental change, we are operating on a diagram” (Stjernfelt
2007: 92). Peirce writes in an alternate version of his text that “the Diagram
represents a definite Form of Relation. This Relation is usually one that actually
exists,” although that condition is not “essential to the Diagram as such” (Peirce
1976b, 4: 316). He suggests that “the pure Diagram is designed to represent and
to render intelligible, the Form of Relation merely” (Peirce 1976b, 4: 316). Peirce
wrestles with the problem of how to align mental apprehension with the fact
that “Diagrams remain in the field of perception and imagination”; he settles
on a two-pronged formula: the drawn diagram and its universal signification
“taken together constitute what we shall not too much wrench Kant’s term in
calling a Schema, which is on one side an object capable of being observed
while on the other side it is General” (Peirce 1976b, 4: 318; see also Stjernfelt
2007: 82–83 and 94–95). On Peirce’s account, pure diagrams – comprised only
of rational relations – depend upon what Stjernfelt calls “applied” or “token
diagrams” to communicate their ideal entity (Stjernfelt 2007: 96).1 They do so
1 In the alternate text of “Prolegomena for an Apology to Pragmatism” Peirce writes: “A

Diagram, in my sense, is in the first place a Token, or singular Object used as a Sign; for it is
essential that is should be capable of being perceived and observed” (Peirce 1976b, 4: 315).
On the “thing-ness” of diagrams 25
because “reading rules” guide our “prescission” of “accidental characters that

have no significance”: in the alternate version of his text Peirce describes these
as “Conventions embodied in Habits”; in the final version he writes that “certain
modes of transformation of Diagrams of the system of diagrammization used
have become recognized as permissible” (Peirce 1976b, 4: compare 317 and
318). None of these modes is spelled out with great precision; Peirce brushes
aside the difficulty by saying only that “sometimes in one way, sometimes in
another, we need not pause to enumerate the ways” they are recognized. I want
to underscore exactly this imprecision.
It is also something of an inconvenience that Peirce neglects to say much
about a taxonomy of diagrams. In a circa 1895 text that sketches four steps of
deductive reasoning, Peirce does invoke “a diagram, or visual image, whether
composed of lines, like a geometric figure, or an array of signs, like an algebraic
formula, or of a mixed nature, like a graph” that is constructed “so as to embody
in iconic form, the state of things asserted in the premise,” but he assumes it
“exists only in the imagination” (Peirce 1976a, 4: 275). Stjernfelt rightly remarks
that pure diagrams must be coextensive with mathematics, which means
that considering them also entails thinking about the nature of mathematics
(Stjernfelt 2007: 111–112). On the other hand, applied or token diagrams come in
many stripes and – as demonstrated by the often-cited studies of Alan Blackwell
and Yuri Engelhardt – there is little agreement amongst thinkers across disciplines
concerning their salient qualities (Blackwell and Englehardt 1998).2 Stjernfelt
remarks somewhat laconically that “the construction of a rational taxonomy of
diagrams will be a major future challenge for (not only) Peircean semiotics”
(Stjernfelt 2007: 111). I will eventually suggest that one way to avoid interminable
debates about the nature of diagrams is to adopt a cognitive systemic view that
obviates many of the problems and vagaries of taxonomy.
Definitional dawdling is not the only soft spot in Peirce’s analysis. Crucial
to his account of diagrammatic reasoning in “Prolegomena for an Apology to
Pragmatism” – both the principal version and its variant – is a moment that
Stjernfelt properly calls “imaginary” (Stjernfelt 2007: 83–87 and Stjernfelt 2000:
376–378). Already in Peirce’s 1895 account of the four steps of deduction (cited
above) he writes about the second step: “Upon scrutiny of this diagram, the
mind is led to suspect that the sort of information sought may be discovered,
by modifying the diagram in a certain way. This experiment is tried” (Peirce
1976a, 4: 275). Peirce allows that something like suspicion or hunch suggests
possible avenues of experiment to the interpretant. Peirce reformulates this
2 Updated and revised in Blackwell and Englehardt 2002. Also see Schmidt-Burkhardt 2012: 41–
68.
26 Michael Marrinan
comprehension in his 1906 description of how diagrammatic reasoning works:

“The Diagram sufficiently partakes of the percussivity of a Percept to determine,
as its Dynamic, or Middle, Interpretant, a state [of] activity in the Interpreter,
mingled with curiosity. As usual, this mixture leads to Experimentation” (Peirce
1976b, 4: 318).3 Stjernfelt signals “a certain tension” in the “seductive welding
together of object and representation in this phase which constitutes the major
source of error in diagrammatic reasoning” (Stjernfelt 2007: 112). Moreover, he
admits Peirce’s formulation introduces “a strange psychological tone alien to
him” with its recourse to an excitement (see variant) of activity and curiosity
(Stjernfelt 2007: 102).
Peirce works around this danger by embedding the incitement for experiment
within a process of trial-and-error that asks: has the experiment I just performed
yielded an iconic result that expands upon the initial symbol under considera-
tion? Whatever the answer, the motivating moment of forgetfulness, lost con-
sciousness, and dream remains just that – an instant of seduction held in check
by immediate constraints of the on-going experiment and the ultimate object
of our reasoning. Stjernfelt reminds us that “the whole formalist endeavor in
the philosophy of mathematics and the emphasis upon symbolic calculi and
mistrust of geometry since the late nineteenth century is based on attempts
at getting rid of the dangers of seduction by intuition in this very moment”
(Stjernfelt 2007: 112). He attempts to rescue Peirce from this danger by suggesting
“the decisive thing is that this moment is made possible by structural iconicity
between diagram and object – not by the psychology of he or she who con-
templates that iconicity.” I am not convinced Stjernfelt’s formulation neutralizes
Peirce’s remark that “the action of the Diagram . . . has the same percussive
action on the Interpreter that any other Experience has” (Peirce 1976b, 4: 317,
variant). Nonetheless, I agree with Stjernfelt’s conclusion that “the pragmatist
trial-and-error feedback between initial and final symbols in the diagrammatic
reasoning process must be the Peircean means of avoiding being caught up
in the ‘imaginary moment’” (Stjernfelt 2007: 114). We can ask if Peirce is truly
successful in avoiding this pitfall, but I want to take away two points from my
admittedly partisan reading of Peirce’s account of diagrammatic reasoning: the
crucial place in the process of an interpretant’s intutive response to the diagram;
the coterminous fiction that the diagram has become a thing.
3 The variant of this passage reads: “But the action of the Diagram does not stop there. It
has the same percussive effect on the Interpreter that any other Experience has. It does not
stimulate any immediate counter-action, nor does it, in its function as a Diagram, contribute
particularly to any expectation. As Diagram, it excites curiosity as to the effect of a transforma-
tion of it” (Peirce 1976b, 4: 317).
3 Peirce’s Prescission: Diagrams of no Substance

Peirce put little stock in the materiality of diagrams, which makes sense because,
on his account, pure diagrams of the most general application are virtual con-
structs of the mind. “One contemplates the Diagram,” he writes in the variant
version of 1906:
. . . and one at once prescinds from the accidental characters that have no significance.
They disappear altogether from one’s understanding of the Diagram . . . one can contem-
plate the Diagram and perceive that it has certain features which would always belong
to it however its insignificant features might be changed. What is true of the geometrical
diagram drawn on paper would be equally true of the same Diagram put on the black-
board (Peirce 1976b, 4: 317, note).
He remarks a bit further in the main body of the same essay:
Logic requires great subtlety of thought, throughout; and especially in distinguishing those
characters which belong to the diagram with which one works, but which are not signifi-
cant features of it considered as the Diagram it is taken for, from those that testify to the
Form represented. For not only may a Diagram have features that are not significant at all,
such as being drawn upon “laid” or upon “wove” paper; not only may it have features that
are significant but not diagrammatically so; but one and the same construction may be,
when regarded in two different ways, two altogether different diagrams (Peirce 1976b, 4:
324).
Nonetheless, a diagram’s field of support is significant even if its materiality is

not because, on the most abstract level, the support stands for at least three
distinct types of discourse. First, geometrical figures drawn upon a flat plane
where “the parts of the diagram are seen in the visual image to have the
relations supposed.” Second, algebraic formulas on a support where “the parts
have shapes to which conventions or ‘rules’ are attached, by means of which the
supposed relations are attributed, or imputed, to the parts of the diagrams”
(Peirce 2010: 46–47). Finally, underpinning what Peirce calls Existential Graphs,
the support is a “Sign of a logical Universe” upon which graphs are scribed with
a different kind of attachment, “since the Entire Graph of the Area is after a
fashion predicated of that Universe” (Peirce 1976b, 4: 322).4 Stjernfelt correctly
summarizes by noting that the first two types of support are “continuous, requir-
ing the imaginary translation of geometric objects on the surface; the latter
4 Scribing has a particular meaning for Peirce (1933b, 4: 397): “The graphist may place replicas
of graphs upon the sheet of assertion; but this act, called scribing a graph on the sheet of
assertion, shall be understood to constitute the assertion of the truth of the graph scribed.”
Also see Peirce 1976b, 4: 322 and Peirce 1933c 4: 418–423.
28 Michael Marrinan
Figure 9: Pietro Perugino, Christ giving the Keys to Saint Peter, ca 1481–82. Fresco. Rome
(Vatican), Sistine Chapel (330 x 550 cm). Photo credit: Public Domain via Wiki Commons.
discontinuous, involving the adding or erasing of whole discrete structures on

the sheet. Thus, diagrams comprise both continuous and discontinous systems”
(Stjernfelt 2007: 101). Among the latter group, Peirce singles out two instances
that “are remarkable for being truly continuous both in their matter and in their
corresponding Signification . . . the Graph of Identity represented by the Line of
Identity, and the Graph of coëxistence, represented by the Blank” (Peirce 1976b,
4: 324). Obviously, in the last instance, the “sheet of assertion” is itself a graph
(Peirce 1933b, 4: 396). I do not intend to dive into the murky waters of Existential
Graphs, but I underscore Peirce’s attention to the field of diagrammatic inscrip-
tion because it raises a question that interests me: what does Peirce see when
looking at pictures? What part of that experience provokes in him a “moment
when we lose the consciousness that it is not the thing”?
To simplify matters, let us imagine Peirce before Perugino’s famous fresco,
Christ giving the Keys to Saint Peter (Figure 9). While clearly not a pure icon,
elements of the image powerfully elicit apprehension of a geometric skeleton:
his imagination draws – supposedly on the surface of the image – lines of the
piazza that converge in the doorway of the octagonal temple at the center of
the middleground. Peirce’s accumulated habits of looking at pictures lead him
to recognize the formal pattern of a perspective rendering of space that explains
why some figures are smaller and appear to stand on the heads of the larger
figures – they are further away. But actually, of course, they are not; all the
figures lie in the same wall plane at an equal distance from where Peirce is
standing. The apprehension of “depth” belongs to the imaginary moment that
requires the support to disappear. His attention returns to the diagram inscribed
mentally on the surface: Peirce realizes that an imaginary line drawn between
himself and the door to the temple coincides with a large key seen in profile,
which is connected to a second key held by a figure just to the left, and grasped
by the figure to the right. In a second imaginary moment, Peirce eliminates the
support to recognize that the left figure is handing the set of keys to the right
figure – an action that the diagram has helped him to discover. And so on:
a give-and-take process of experiment and discovery, motivated and directed
by the diagram his imagination inscribes invisibly on the surface, but whose
impact can only be assessed when that surface is thought away. In this textbook
case, I draw attention to the polyvalence of the image support that is integral to
the process but whose permutations are largely ignored.
I will complicate the issue a bit by placing our fictive Peirce before a charac-
teristic landscape of Jacob van Ruisdael (Figure 10). A picture like this does not
readily present an armature of perspective like Perugino’s fresco. Ruisdael’s
engagement with an unorthodox representation of space that does not begin by
thinking of the sky as a void has long been recognized. Eugène Fromentin, in a
famous text of nineteenth-century criticism, explains that Ruisdael:
. . . felt things differently and established once and for all a much more daring and true
principle. He thought of the immense vault that rises spherically over the countryside
or sea as if it were the real, dense, solid ceiling of his pictures. He bends it, unfurls it,
measures it, calculates its tonal brightness relative to the accidents of light scattered on
the distant ground; he nuances its large areas, models them in relief, makes them – in a
word – concrete, as if an item of primary interest (Fromentin 1876: 251–252).
For Fromentin, Ruisdael does not privilege the geometry of sight organized around
a fixed point of view essential to the diagram of linear perspective; rather, he
celebrates an embodied, mobile, and time-dependent process of comprehension.
Fromentin suggests that Ruisdael’s concept of a picture is less a window through
which we look onto a parallel world towards infinity – the model of painting
practiced by Perugino – than a continuous, palpable mass manipulated – almost
sculpted – in the process of representation.
Traces of this process are numerous in our example. Note the shared rela-
tionships of shape and mass that connect visually and perceptually the stand
of trees to the formations of dense clouds. Note the three widely-distributed
areas of bright light – hillock and building at center-right, farmhouse at far left,
blasted tree rising from the running water in the lower right – that encourage
30 Michael Marrinan
Figure 10: Jacob van Ruisdael, Landscape near Muiderberg, ca 1652. Oil on canvas. Oxford,
Ashmolean Museum, inv. A875 (66 x 75 cm). Photo credit: Ashmolean Museum, Oxford.
our eye to traverse the image laterally in a circular motion. Note the darkened
and clotted center of the image that refuses to provide us with a commanding
view of deep space. We can imagine that our fictive Peirce performs similar per-
ceptual experiments on the picture while trying to determine its diagrammatic
relations.
We might also imagine that he is thrown off at times by the intruding pres-
ence of Ruisdael’s pictorial matter: colored pigment applied in various densities
that present homologies between, for example, the visibly brushed clouds and
the thickly painted stand of trees that meet one another materially on the
surface of the canvas. Ruisdael’s picture will probably disturb the confidence
with which Peirce usually relates form to matter, as when he writes:
. . . there is a rational explanation of the precedence of Form over Matter in natural classi-
fications. For such classifications are intended to render the composition of the entire
classified collection rationally intelligible, – no matter what else they may be intended to
show; and Form is something that the mind can “take in,” assimilate, and comprehend;
while Matter is always foreign to it, and though recognizable, is incomprehensible (Peirce
1976b, 4: 322).
Not incidentally, this passage follows an aside in which Peirce remarks:
The same precedence of Form over Matter is seen in the classification of psychical products.
Some of Rafael’s greatest pictures, – the Christ bearing the cross, for example, – are suffused
with a brick red tinge, intended, I doubt not, to correct for the violet blueness of the deep
shade of the chapels in which they were meant to be hung. But who would classify Rafael’s
paintings according to their predominant tinges instead of according to the nature of the
composition, or the stages of Rafael’s development? There is no need of insisting upon a
matter so obvious (Peirce 1976b, 4: 321–322).
Nevermind that art history does, in fact, classify paintings “according to their
predominant tinges,” as when discussing Picasso’s “Blue” or “Rose” periods or
the “sour period” of Renoir. The point to retain from my little detour is that the
thrust of Peirce’s thinking will disincline him to attend to the “recognizable” but
“incomprehensible” manipulations of material that are essential to any paint-
ing, and especially in works like the Ruisdael.
To seek and to isolate “form” in such pictures, to “prescind” much of their
material presence, is how Peirce keeps in line and under control the curiosity
and experiment that constitute the dangers of the “imaginary moment.” The
paradox is that when Peircean prescission takes over, some elements integral
to a configuration like Ruisdael’s are forced to relinquish their meaning, to
become only “background” or “support” for the rest. When writing about
diagrams in general, Peirce both admits and dismisses the difficulty I have in
mind:
It is, however, a very essential feature of the Diagram per se that while it is as a whole an
Icon, it yet contains parts which are capable of being recognized and distinguished by the
affixion to each of a distinct Semantic Index (or Indicatory Seme, if you prefer the phrase).
Letters of the alphabet commonly fulfill this office. How characteristic these Indices are of
the Diagram is shown by the fact that in one form or another they are indispensable to
using the diagram, yet they are seldom wanted for the general enunciation of the proposi-
tion which the Diagram is used for demonstrating (Peirce 1976b, 4: 317).
This is a vivid statement suggesting a problematic conception of the diagram-

matic field: we recognize letters guiding our use of the diagram, but they do
not disturb its iconic structure nor interfere with its iconic meaning. Where,
32 Michael Marrinan
then, do these letters exist if they are not part of the image defined by the plane
of material support? Pictures like the Ruisdael accentuate this difficulty because
its indices are not easily isolated letters or numbers, but integral components of
the image itself.
Frederik Stjernfelt recognizes this problem in Peirce’s thought. He proposes
to amend slightly Peirce’s requirement that diagrams be intentional (for example,
the diagram of perspective deployed by Perugino), by suggesting that “as soon
as an icon is contemplated as a whole consisting of interrelated parts whose
relations are subject to experimental change, we are operating on a diagram . . .
as soon as you imagine yourself wandering along the path into the landscape,
you are operating on the icon – but doing so in this way is possible only by
treating it as a diagram” (Stjernfelt 2007: 92 and note; 278–279). Stjernfelt’s
amendment, with which Peirce might well disagree, invests the entire configura-
tion with significance, as if the fact of being a painting pre-empts the prescission
that Peirce takes to be essential. Stjernfelt’s scheme also raises the problem that
looking at pictures is unlike looking at anything else (Stjernfelt 2007: 285). He
apparently embraces this consequence when stating “a (sufficiently complicated)
picture, however, only exists to the extent that it is already a diagram” (Stjernfelt
2007: 285).
Finally, Stjernfelt moves quite far from Peircean foundations when sketching
the process of viewing an abstract picture; he uses the example of Suprematist
Composition: White on White (Figure 11), painted by Kazimir Malevich in 1918, to
remark: “in viewing such a picture, you are forced, as an observer, to an abductive
trial-and-error process, attempting the use of diagrams and consecutive manipu-
lations with no other guaranty than that of perception for one diagram being
better than another” (Stjernfelt 2007: 288). It is not surprising that a long foot-
note to this section of text introduces the problem of phenomenology and the
body. Stjernfelt takes up those issues with a discussion of Husserl in the follow-
ing chapter, where he ultimately posits the viewing of pictures as a form of
“wandering access” that is “highly dependent upon the bouquet of possible
peripathetical [sic] action possibilities determining which kind of access to the
pictorial space is possible“ (Stjernfelt 2007: 310). I will eventually agree with
him in principle, but not because I believe pictures materialize the Peircean
model of diagrams.
I want to introduce a third type of example, one that bears an elliptical rela-
tionship of subject to the Ruisdael and recurs to Peirce’s comments about letters
aiding our use of a diagram. Consider the first plate of Diderot and d’Alembert’s
Encyclopédie, an image devoted to “Agriculture” (Figure 12). The large, upper
image appears to depict space rather like the Ruisdael: a series of planes maps
the distance from the foreground tree at far left to the ruined fortress on the
Figure 11: Kazimir Malevich, Suprematist Composition: White on White, 1918. Oil on canvas.
New York, The Museum of Modern Art, inv. 817.1935 (79.4 x 79.4 cm). Photo credit: Digital
Image © The Museum of Modern Art / Licensed by SCALA / Art Resource, New York.
horizon, which is nonetheless framed by the nearby arching branches. A village

is nestled into the center of the image; its spire breaks the horizon line to echo
both the ruins at left and the active windmill to the right. Our first impression is
a progression of space – perhaps even historical time – through which our eyes
move. But this illusion is disrupted in at least two ways: first, the figures and
horses of the middle-ground are too large for their implied distance; second, we
cannot avoid stumbling upon captions (Figure 12, Figure 13, etc) scattered across
the ground plane. Words and numbers of this sort – and on this scale – usually
do not appear spontaneously in the natural world. Surely we are mistaken if
34 Michael Marrinan
Figure 12: “Agriculture, Labourage,” pl. I. Engraving by Robert Bénard for Diderot and
d’Alembert, Recueil des Planches, vol. 1. Courtesy Department of Special Collections, Stanford
University Libraries. Photo credit: Michael Marrinan.
Figure 13: “Agriculture, Labourage,” pl. II. Engraving by Robert Bénard after a design by Louis-
Jacques Goussier for Diderot and d’Alembert, Recueil des Planches, vol. 1. Courtesy Department
of Special Collections, Stanford University Libraries. Photo credit: Michael Marrinan.
36 Michael Marrinan
we think they are inscribed in the fields; however, they are inscribed on the
sheet of paper supporting the entire image. These elements point deictically to
captions and verbal explanations that supplement the visual forms and – most
important – direct our attention to other near-at-hand textual sources via many
cross-references. Here the paper support of the image performs two simultaneous
roles: on one hand, it presents markings that conjure up a fictive, three-
dimensional world; on the other, it supports letters and numbers that link the
entire icon to the space of a text. The deictic markers and accompanying lan-
guage structure join forces to diagram the image as surely as the geometry of
perspective diagrams the fresco of Perugino, but they are no longer imaginary
additions; rather, they coexist paradoxically with icons of horses and trees,
buildings and people to elicit a virtual explosion of attention.
That coexistence is drawn even more sharply beneath the line, inside the
smaller box presenting two different types of plows. The machines appear to be
situated on a ground plane marked by tufts of grass and cast shadows, but they
are also entirely surrounded by letters and numbers marking the various parts of
each device. If the ground plane is taken seriously the numbers and letters float
mysteriously in space. This is unlikely. However, when seen alongside its com-
panion plate (Figure 13), we immediately grasp that we are invited to recon-
ceptualize the blank sheet from spatial box to measurable flat surface (note the
scale of measurement), even if the move is ambiguous when we attend to the
lingering shadows and three-quarter views.
The ensemble of related imagery (I illustrate only two of the four plates) is
noteworthy for striving to keep both types of support simultaneously viable in a
dialectic of experiment. Before this set of images, I doubt our fictive Pierce
would exit cleanly his “imaginary moment” of curiosity and experiment, for the
unfolding information tends to branch continuously in many directions. Indeed,
that was the purpose of the Encyclopédie’s configuration, as explained in this
famous passage from d’Alembert’s Preliminary Discourse of 1751:
The reader opens a volume of the plates; he sees a machine that whets his curiosity; it
is, for example, a powder mill or a paper mill, a sugar mill, or a silk mill, etc. Opposite it
he will read: figure 50, 51, or 60, etc., powder mill, sugar mill, paper mill, silk mill, etc.
Following that he will find a succinct explanation of these machines with references to
the articles “Powder”, “Paper”, “Silk”, etc (d’Alembert 1963 [1751]: 126–127).
D’Alembert makes perfectly clear that users of the Encyclopédie will move across
the discursive breach between images and texts when working with the assembled
materials. This is not because the publication mobilizes new technologies of image-
making or textual presentation, but because it brings together and interconnects
existing forms in new ways. We will return shortly to consider this attention to
a systemic coupling of parts.
4 Thinking with Materials

Sceptics might reply to my remarks that I am not dealing with diagrams in the
sense that Peirce uses the word. It is true, as Stjernfelt emphasizes, that Peirce
maintains a distinction between diagrams “constructed with the explicit inten-
tion of experimentation and endowed with an explicit or precise syntax of
transformation” and the “more comprehensive class of diagrammatic unfolding
of information from more ‘innocent’ icons” (Stjernfelt 2007: 101–102). The middle
term joining pure diagrams to diagrammatic uses of ordinary icons are the
“empirical diagrams” mentally or actually constructed in the process of diagram
experimentation (Stjernfelt 2007: 101 and note 108). Since empirical diagrams
must be “capable of being perceived and observed” (Peirce 1976b, 4: 315), it is
likely that the practice of diagrammatic experimentation will spill over to icons
that were never intended for that purpose: Stjernfelt uses the example of a
person spontaneously “measuring” the proportions of a tree on a photograph
by comparing mentally the height of its trunk relative to the crown of its
branches. He concludes that the “defining feature of the diagram – its possibility
of being rule bound transformed in order to reveal new information – is what
makes it the base of Gedankenexperimente, ranging from routine everyday
what-if to scientific invention” (Stjernfelt 2007: 102). I fully agree with his assess-
ment; I will explore it by looking at how thought-experiments are encouraged
within the pages of the Encyclopédie where, I believe, the material constraints
of print culture can shed light on its recurring structures of diagrammatic think-
ing.
By specifying print culture of the eighteenth century I appeal deliberately to
an historical moment of imaging technology in which the underlying support –
a sheet of paper – was a given. My interest is not strictly antiquarian. Bruno
Latour reminds us that much of science today is still directed by cascades of
inscriptions – we might say recorded, annotated, and diagrammed clusters of
data – that build consensus beliefs among researchers working upon related
research problems (Latour 1986). Latour is primarily interested in the exercise
of rhetorical and polemical power within scientific practice: how does one
theory or explanation come to dominate a field (Latour 1986: 20–29)? This is
not exactly my concern, but I am interested in Latour’s remark about the role
of the printing press in constructing compelling scientific explanations: “The
printing press does not add anything to the mind, to the scientific method, to
the brain. It simply conserves and spreads everything no matter how wrong,
strange or wild. It makes everything mobile but this mobility is not offset by
adulteration” (Latour 1986: 12). The result is a concentration of information –
sometimes conflicting and contradictory – at nodal points where comparisons
38 Michael Marrinan
are made, contradictions become visible, and positive feedback begins to influ-
ence new thinking “as soon as one is able to muster a large number of mobile,
readable, visible resources at one spot to support a point” (Latour 1986: 13).
The Encyclopédie of Diderot and d’Alembert was exactly a project designed
to assemble a large corpus of such resources in one spot; it was also “mobile”
insofar as it was published in bound volumes that traveled around the globe.
Latour lists nine advantages of what he calls the inscriptions of “paper
work,” of which four are of special interest to me: inscriptions can easily be re-
shuffled, superimposed, integrated into a written text, and merged with geometry
(Latour 1986: 21–22). Such operations are possible because inscriptions exhibit an
“optical consistency” established by the sheet of paper (Latour 1986: 7–9 and
22–23), only one of which is the “space-making” system of Renaissance perspec-
tive as practiced by Perugino (Figure 9). Latour summarizes the effect:
On paper hybrids can be created that mix drawings from many sources. Perspective is
not interesting because it provides realistic pictures; on the other hand, it is interesting
because it creates complete hybrids: nature seen as fiction, and fiction seen as nature,
with all the elements made so homogeneous in space that it is now possible to reshuffle
them like a pack of cards (Latour 1986: 9).
On his account, optical consistency makes it possible to work on paper with

rulers and numbers and to manipulate three-dimensional objects “out there” in
the fictive space of the representation; to convert everything into diagrams,
numbers, or tables; to measure the diameter of the sun with a ruler on a photo-
graph that can be rescaled to cosmic dimensions.
The “hands on” manipulation of inscriptions described by Latour is far from
Peirce’s mental experiments that produce and analyze empirical diagrams. How-
ever, it is essential to the discursive mechanics of the Encyclopédie (Bender and
Marrinan 2010: 7–8). Diderot and d’Alembert were perfectly aware that theirs
was not the first such publication; they describe their work as both a great
“interlinking of the sciences” similar to their predecessors, and a catalogue of
the bewildering diversity and range of activities that comprise human existence
(Diderot 1994, 1: 211, note). To navigate this labyrinth, the editors adopted the
alphabetic, serial order of a dictionnary, with full knowledge that such an open-
ended structure threatens the unity of their project:
If one raises the objection that the alphabetical order will ruin the coherence of our system
of human knowledge, we will reply: since that coherence depends less on the arrangement
of topics than on their interconnections, nothing can destroy it; we will be careful to
clarify it by the ordering of subjects within each article and by the accuracy and frequency
of cross-references (Diderot 1994, 1: 235).
Here Diderot describes – especially with his attention to “interconnections” and

“frequency of the cross-references” – a process of learning and discovery that
cuts across the dictionary order in complex and unpredictable ways (Becq 1987;
Ludwig 1987). His aim is “to point out the indirect and direct links amongst
natural creatures that have interested mankind; to demonstrate that the inter-
twining of both roots and branches makes it impossible to know well a few parts
of the whole without going up or down many others” (Diderot 1994, 1: 212).
Knowledge for Diderot does not entail absolute inclusiveness; rather, it emerges
from a process of enchaînement [linking] guided by cross-references that expects
a user to exercise an active relational judgment. Today, a composite data array of
this sort is called a mashup.5 How does such a device work in print culture of
the eighteenth century?
Consider the plate of the Encyclopédie devoted to the métier of pastry-
making (Figure 14).6 The upper section offers the view of a workroom in which
many tools and ingredients – including game birds and a wild hare – are
arranged on tables, displayed on shelves, and hung on the walls of a fictive
space that is the product of a perspective diagram very like Perugino (Figure 9).
The sheer number of visible things produces an orderly clutter, but there are
also several pockets of emptiness: the right foreground; openings at left and
right implied by figures who come and go; the measurable distances separating
workstations where figures perform specialized tasks. The many components of
the pastry shop are spatially and functionally isolated from one another. They
offer themselves to inspection by curious eyes, while the emptiness that sepa-
rates them invites the imaginary movement of Stjernfelt’s “wandering access”
through the space. We could paraphrase Peirce by saying this part of the plate
excites “a state of activity in the Interpreter, mingled with curiosity” that leads
to experiments with the image. Part of what we discover are numbers – are they
floating in space or inscribed on the surface? – that key nearby items or tasks
to an accompanying text in which they are described. In short, our attention
is strongly encouraged to alternate between seeing and reading: this is an
important point to which I shall return.
The lower section is something else. Here we re-encounter up close, and
from multiple viewing positions, some of the implements visible in the overall
view of the upper section. The kneading station, for example, is nearly hidden
at the back wall of the workroom; it appears below in a three-quarters view
that clarifies its size and features of use. Conventions of perspective rendering
appear to elicit the object’s three-dimensionality but without displacing any
5 en.wikipedia.org/wiki/Mashup_(web_application_hybrid).
6 Bender and Marrinan 2010: 19–33 for further discussion of these examples.
40 Michael Marrinan
Figure 14: “Patissier, Tour à Pâte, Bassines, Mortier, &c,” pl. I. Engraving by Robert Bénard
for Diderot and d’Alembert, Recueil des Planches, vol. 8. Courtesy Department of Special
Collections, Stanford University Libraries. Photo credit: Michael Marrinan.
measureable volume: there is no ground plane upon which we might imagine

ourselves standing before the kneading station; it casts no shadows onto the
surrounding white surface. Numbers lying on the same white surface close to
objects point to explanations in the accompanying textual legend. Yet things
are also rendered in the round with shading: they perforate the whiteness with
a materiality of physical volume that encourages us to locate them within the
fictive space of the workroom above. We are invited to construct a diagram. The
paradoxical combination of visual clues in the lower section signals to an atten-
tive viewer that the white of the page is neither a spatial void nor a flat surface,
but simply a material whiteness.7 This whiteness permeates the plates of the En-
cyclopédie and bleeds over to its pages of explanatory text. It serves as the
medium for a construction of space within perspective views (Figure 12, 14), a
neutral field of presentation for the visual arrays in which objects are dissected
or displayed (Figure 13), and a continuous material support for a wide range of
linguistic markers. Diderot framed his ambitions for the Encyclopédie to achieve
coherence in terms of verbal cross-references; the pervasive white of the printed
sheet establishes a visual ground of “optical consistency” (Latour) that enables our
acts of mental correlation to project unexpected hybrids of knowledge.
If we think of the plates of the Encyclopédie as components of a total system
rather than merely illustrations of a pre-existing world, their strangeness comes
into sharper focus. Thus, we might not register at first glance the nearly invisible
ruptures within a plate from the section on “Dessein” that purports to show
legs and feet (Figure 15). Two separate renderings of legs in space merge imper-
ceptibly across the field of white, as if forming a fragment of an unseen larger
picture containing two seated figures. Note, however, the front and profile views
of a foot presented just above: they seem to be placed on a ground plane
in space, but also include scalar measurements and annotations, very like the
plows of “Agriculture” (Figure 13). Also note the figure numbers near these
feet, because they establish a sequence continued in the “picture” part of the
image: 1, 2, 3, 4. The numerical scales imply an area analogous to the visual
array of more overtly divided plates, but here the implicit conceptual boundaries
dissolve in the whiteness of the sheet to merge two very different kinds of think-
ing into a single experience.
We discover a similar hybrid in an adjacent plate (Figure 16), where four
renderings of disembodied hand gestures appear in what seem to be localized
spaces: in each vignette the hands cast shadows and are, in turn, modeled with
7 Elkins (1999: 85–89) invokes the idea of “negative white space” in his response to the
nominalism of Nelson Goodman when considering the conceptual intersection among writing,
pictorial elements, and notations.
42 Michael Marrinan
Figure 15: “Dessein, Jambes et Pieds,” pl. XIII. Engraving by Robert Bénard after a design by
Edmé Bouchardon for Diderot and d’Alembert, Recueil des Planches, vol. 3. Courtesy Depart-
ment of Special Collections, Stanford University Libraries. Photo credit: Michael Marrinan.
Figure 16: Dessein, Mains,” pl. XII. Engraving by Robert Bénard for Diderot and d’Alembert,
Recueil des Planches, vol. 3. Courtesy Department of Special Collections, Stanford University
Libraries. Photo credit: Michael Marrinan.
44 Michael Marrinan
Figure 17: “Anatomie,” pl. Seconde IV. Engraving by Robert Bénard for Diderot and and
d’Alembert, Recueil des Planches, vol. 1. Courtesy Department of Special Collections, Stanford
University Libraries. Photo credit: Michael Marrinan.
shadows to develop something like believable volume and mass. Figure numbers
referring to an external text again lurk at the edges of each fragment, much as
the objects arranged in the visual array of our pastry plate (Figure 14). But there
is also a fifth vignette of a hand, marked by secondary reference points of small
letters very like the feet just discussed; it casts no shadow. Here too, the optical
consistency of the paper’s whiteness provokes an ongoing dialectic between two
very different modes of seeing and thinking. It also provokes even more pro-
nounced hybrids: for example, juxtaposition with a plate two volumes away
in the section on “Anatomie” where hands and feet are subject to a completely
different kind of analysis (Figure 16). The dialogue between an anatomist’s view
of the body’s interior and an artist’s view of its exterior, enabled by visual cross-
references and textual prompts, elicits an understanding of one’s physical self
that is located in neither of the images separately. Rather, it is generated by an
active process of correlating two disparate disciplines, a process made possible
by the optical consistency that links the plates and accompanying texts in a
unified system of data display.
5 Systemic Thinking
My account of the Encyclopédie depends absolutely upon the idea of an active
user who turns pages, looks up references, reads text, and continues the process
for as long as the material continues to generate curiosity. There is no “end”
towards which one works, especially because the Encyclopédie was never thought
to be a teaching manual. Could we really expect to learn how to make pastries
by looking at Diderot’s plates and reading his long and quite detailed explana-
tions? Probably not: we would be better served buying flour, eggs, sugar and
butter to get our hands into real dough. Yet there is a dimension of learning – a
cognitive component – to the mental and physical interactions established
within what I will call the “space” of the Encyclopédie. Like the carrefour of a
city created when several individual streets intersect, but without a structure of
its own, the Encyclopédie creates an environment for cognitive activity. I prefer
to talk about “users” of the Encyclopédie rather than viewers or readers because
to make it work we must physically and psychically enter the space of its
components – plates, captions, and texts – held together by the material white-
ness of its pages.
The interactive turn crucial to the Encyclopédie seems to mark an historical
moment. Consider the parallels of its experience and chronology with the carte
chronographique [chronological map] invented about 1753 by Jacques Barbeu-
Dubourg, who was also the French translator of Benjamin Franklin’s works and
46 Michael Marrinan
one of Diderot’s many collaborators on the Encyclopédie (cf. Aldridge 1951).

Comprising a chronological chart more than sixteen meters long, and housed in
a special viewing case fitted with cranks and scrolls, the carte chronographique
allowed users to move freely through history – both forward and backward –
and to draw correlations among simultaneous events and world leaders from
creation to the present day.8 Astrit Schmidt-Burkardt correctly ends her impor-
tant discussion of this device by noting: “Das Bild des selbständig Studierenden,
das Barbeu-Dubourg vor Augen schwebt, fügte sich in den intellektuellen Prozess
der Aufklärung. Den Weg aus der selbstverschuldeten Unwissenheit zeichnete de
Carte chronographique didaktisch-diagrammatisch vor. Die Idee des interaktiven
Users war geboren” (2012: 80).9 The Encyclopédie depends upon the traditional
components of illustrated books, yet its serial organization, incitement to make
cross-references amongst different media, and cognitive yield of branching rather
than closure are – for all practical purposes – identical to Barbeu-Dubourg’s
device: not surprisingly, Diderot devoted an entire article to the carte chronogra-
phique based on information supplied by its inventor (Diderot 1753).10 But more
important is the emergence in both instances of an interactive user coupled by
deictic gestures, physical touch, and mental browsing before a non-perspective,
visual array of data.
In literature, the earliest examples of free indirect discourse (FID) – La
Fontaine in France and Jane Austen in England – bracket the years when Diderot
was writing and producing the first volumes of the Encyclopédie. The linguistic
form of FID is a sentence or syntax of narration in which first-person thoughts of
a character are rendered in the third-person, as if objectively present in the text
independent of a narrator’s intervention.11 When used in novels, an imperfect
(past) tense is combined with present-tense deictic markers such as this, here,
or now to register the sense of “this was now here.” In these statements, im-
personal past tense narration presents as if real the interior mental activity of
others that is not accessible in ordinary life. Ann Banfield remarks that the
8 On the device see Rosenberg and Grafton 2010: 112–113. See Ferguson 1991 for a detailed
study of the only extant working example now in the Princeton University Library.
9 Schmidt-Burkardt 2012: 69–80 for full discussion of the diagrammatic importance of Barbeu-
Dubourg’s device.
10 The “Avertissement” at the head of volume 3 (page xv) reports that “M. Barbeu du Bourg,
Docteur en Medecine de la Faculté de Paris, nous a communiqué sa machine chronologique &
l’explication de cette machine.”
11 Free indirect discourse (called style indirect libre in French theory) consists of the use of
third-person, past-tense grammar to represent present first-person thoughts and mental states
in narration. The literature on FID is very large but see the following for good overviews:
Banfield 1982: esp. 225–254; Banfield 1987; Brown 2005: chapters 3 and 4.
reader encounters “in the very language of the novel a real and empirically
determinable possibility, the disturbing presence of something impersonal,
inhuman, past and, in that instant, distant” (Banfield 1987: 278–279). Her account
of the impersonal narration of thought in FID, evoking the appearance of data
without any speaker, parallels the material whiteness of support in the Encyclo-
pédie that mobilizes a user’s deictic gestures of seeing, reading, and attention
to produce forms of knowledge not explicitly stated. I am naturally interested
in the irruption of reality from this matrix, for it recalls the “thing-ness” of
diagrams that deeply concerns Peirce.
My brief trip through free indirect discourse is seconded by a development
in the visual arts of the eighteenth century that Michael Fried calls “absorption”
(Fried 1980, esp. Ch.1). Writing about certain pictures of Jean-Baptiste-Siméon
Chardin (Figure 18)12 and Jean-Baptiste Greuze (Figure 19)13 whom Diderot also
admired, Fried suggests that depictions of self-absorbed figures tend to screen
out the audience, “to deny its existence, or at least refuse to allow the fact
of its existence to impinge upon the absorbed consciousness of his figures.
Precisely that refusal, however, seems to have given Greuze’s contemporaries a
deep thrill of pleasure and in fact to have transfixed them before the canvas”
(Fried 1980: 68). The paradox is that “because his presence was neutralized in
that way, the beholder was held and moved by Greuze’s paintings as by the
work of no other artist of his time” (Fried 1980: 69). The rapture invoked by
Fried depends upon extricating representation from the usual parameters of
placing some thing or action before a spectator in favor of a situation that simul-
taneously alerts the viewer to his non-existence even as the subject displays
itself. “What is called for,” remarks Fried, “is at one and the same time the
creation of a new sort of object – the fully realized tableau – and the constitution
of a new sort of beholder – a new “subject” – whose innermost nature would
consist precisely in the conviction of his absence from the scene of represen-
tation” (Fried 1980: 104). Following the critical lead of Grimm, Diderot and
Shaftesbury, Fried argues that such a tableau presents its material in a pictorially
unified, instantaneous, self-sufficient instant (Fried 1980: 82–92). To which I will
add: a tableau under absorption, like FID in literature, makes evident that it
addresses no one in particular – least of all its spectator – and for that very
12 Jean-Baptiste-Siméon Chardin, The House of Cards, ca 1737. Oil on canvas. Washington, DC,
The National Gallery of Art, Andrew W. Mellon Collection, inv. 1937.1.90 (82.2 × 66 cm). Photo
credit: The National Gallery of Art, Washington DC.
13 Jean-Baptiste Greuze, A Boy with a Lesson Book, exhibited 1757. Oil on canvas. Edinburgh,
Scottish National Gallery, inv. NG 436 (62.5 × 49 cm). Photo credit: Erich Lessing / Art Resource,
New York.
48 Michael Marrinan
Figure 18: Jean-Baptise-Siméon Chardin: The House of Cards, ca 1737. Oil on canvas.
Washington DC, The National Gallery of Art, Andrew W. Mellon Collection, inv. 1937.1.90
(82.2 x 66)
Figure 19: Jean-Baptiste Greuze: A Boy with a Lesson Book, exhibited 1757. Oil on canvas.
Edinburgh, Scottish National Gallery, inv. NG 436 (62.5 x 49 cm). Photo credit: Erich Lessing /
Art Resource, New York.
50 Michael Marrinan
reason holds the beholder spellbound by a fiction of real presence that we might
call its “thingness.” Thus, Fried’s paradox returns us by a route through visual
culture to that “moment of a pure dream – not any particular existence, and
yet not general” when the “thingness” of a diagram leads Peirce to contemplate
the icon.
The visual arrays of the Encyclopédie’s plates – frontality of presentation,
discontinuity with adjacent spaces rendered in perspective, pervasive material
whiteness joining visual arrays to a network of numbers and letters keyed to
texts – fail to converge in a single vantage point or entity that might be called
a viewer in the sense that Perugino takes for granted. Users of the Encyclopédie
and Barbeu-Dubourg’s carte chronographique resemble the impersonal readers
of FID described by Banfield and the alienated beholders of pictures described
by Fried. Quite unlike embodied viewers, they are impersonal but functional
components, inseparable from the systems of representation in which they are
embedded. They are empowered to initiate a process of correlation, even as
they realize that their subjective presence is almost non-existent; very like the
non-person to whom free indirect discourse is addressed or the unacknowledged
viewer of absorptive paintings. Taken together, these phenomena define histori-
cally the decades when diagrammatic knowledge moved to center stage.
6 Cognitive Turn: Thinking outside the Head

Obviously my reading of the Encyclopédie’s operation is strongly influenced by
research in cognitive psychology. I use “visual array” to name the field of images
in the lower section of the “Patissier” plate, for example, to suggest that the
individual objects are not situated in open space but interrelated to one another
and to the workshop above via the material whiteness of the supporting sheet.
The term is loosely related to the idea of “optical arrays” first developed some
years ago in James Gibson’s influential book, The Ecological Approach to Visual
Perception (Gibson 1986, first published 1979). Gibson’s ambition in this early
text is to detach perception from mental processes of the mind, to make it
“an achievement of the individual, not an appearance in the theater of his
consciousness . . . Perceiving is a psychosomatic act, not of the mind or of the
body but of a living observer” (Gibson 1986: 239 – 240). Gibson wants to replace
the “act” of perception with what he calls “information pickup,” by which
he means “a continuous act, an activity that is ceaseless and unbroken . . .
perceiving is a stream” (Gibson 1986: 240). On his account, information is not
knowledge communicated to a receiver because “the information in the sea of
energy around each of us, luminous or mechanical or chemical energy, is not

conveyed. It is simply there” (Gibson 1986: 242). Faced with a flux of data, he
believes it is less useful to think of humans with five discrete senses attuned
to particular kinds of stimuli than a perceptual system, endowed with organs
interconnected via input/output loops, able to seek data actively, reflect upon
the stimulation received, and learn about new phenomena never before experi-
enced (Gibson 1986: 244–246). This last point is important, as when he writes:
“The process of pickup is postulated to be very susceptible to development and
learning. The opportunities for educating attention, for exploring and adjusting,
for extracting and abstracting are ulimited” (Gibson 1986: 250–251). Perhaps the
most radical dimension of Gibson’s argument concerns the relationship between
perceiving and knowing: where the traditional view holds that perceptual inputs
are processed in the brain to become knowledge, Gibson argues that “knowing
is an extension of perceiving” (Gibson 1986: 258). He believes a theory of infor-
mation pickup closes the body/mind gap that still haunts western thought since
Descartes.14 He also names “three obvious ways to facilitate comprehension:
the use of instruments, the use of verbal descriptions, and the use of pictures”
(Gibson 1986: 259). Although Diderot’s Encyclopédie does not make use of actual
instruments, the results of using them inform some of the most visually gripping
plates (Figure 20). Verbal descriptions and pictures, on the other hand, are
integral to its systemic arrangement. If we forgo thinking about the Encyclopédie
as an illustrated text, it becomes easier to understand it as an environment
designed to stimulate information pickup.
Gibson’s writings exercised a wide influence in cognitive psychology, espe-
cially for his sustained attack on the two-stage process by which sensory percep-
tion of the world leads to mental processing of an image of that world. Some
researchers working in Gibson’s wake have sidestepped the problem completely.
Francisco Varela and his colleagues, writing in 1992, describe their orientation:
“Our intention is to bypass entirely this logical geography of inner versus outer
by studying cognition not as recovery or projection but as embodied action”
(Varela, Thompson and Roach 1992: 172). Embodied, as they use the word, means
simply that cognition depends upon experience “that comes from having a body
with various sensorimotor capacities . . . embedded in a more encompassing
biological, psychological, and cultural context.” Similarly, action specifies that
sensory and motor processes, perception and action are “fundamentally insepa-
rable in lived cognition” (Varela, Thompson and Roach 1992: 173). One key to
their thinking is the principle of “perceptually guided action,” by which they
14 See Varela, Thompson and Rosch (1992: 133–145) for a discussion of the legacy of “The
Cartesian Anxiety.”
52 Michael Marrinan
Figure 20: “Histoire Naturelle, Le Pou vu au Microscope,” pl. LXXXIV. Engraving by Robert
Bénard after a design by François-Nicolas Martinet for Diderot and d’Alembert, Recueil des
Planches, vol. 6. Courtesy Department of Special Collections, Stanford University Libraries.
Photo credit: Michael Marrinan.
mean “how a perceiver can guide his actions in his local situation” when that
situation is not fixed but changes constantly in response to his activity of
perceiving.
On their account, the sensorimotor activity of perception effectively guides
and determines what is perceived. This part of their thinking is compatible with
Gibson’s idea of information pickup. By contrast, they part company with
Gibson concerning the nature of the information likely to be picked up: he
believes there are “invariances in the stimulus flux” that relate to constancies
in the environment (Gibson 1986: 139 and 249 – 250). They reject the idea
of “invariances” because it assumes that the environment is independent
and that perception is “direct detection” whereas their view holds that percep-
tion is “sensorimotor enactment” and can only be studied by attending to the
“structural coupling” between a perceiver and the surrounding environment.
“Cognition is no longer seen as problem solving on the basis of representa-
tions,” they write; “instead, cognition in its most encompassing sense consists
in the enactment or bringing forth of a world by a viable history of structural
coupling” (Varela, Thompson and Roach 1992: 204 – 205). My point in taking
this detour through the arguments of Varela and colleagues is to introduce the
idea of a feed-back loop that couples perceiver to environment, along with their
insight that cognition is a process that brings forth a world under the aegis of
this coupling. I will eventually incorporate these points into my analysis of
how users manipulate the plates and texts of the Encyclopédie, but must first
explore some of their consequences.
Once cognitive scientists shrug off the anxiety of Descartes to dissolve the
barrier of skin and skull separating body and environment, it is inevitable that
more radical formulations will follow. Andy Clark, one of the leaders of this
tendency, published in 1998 a polemical article with David Chalmers that has
become a classic of cognitive literature (Clark and Chalmers 1998).15 In it they
argue for “an active externalism, based on the active role of the environment in
driving cognition processes” including “the general paraphenalia of language,
books, diagrams, and culture” in which “the individual brain performs some
operations, while others are delegated to manipulations of external media”
(Clark and Chalmers 1998: 7–8). Essential to their view is the potential that “the
human organism is linked with an external entity in a two-way interaction,
creating a coupled system that can be seen as a cognitive system in its own
right . . . our thesis is that this sort of coupled process counts equally well as a
cognitive process, whether or not it is wholly in the head” (Clark and Chalmers
1998: 8–9). One example they cite is a Scrabble player who constantly arranges
and re-arranges letters on his tray: in this case, they argue, “the re-arrangement
of tiles on the tray is not part of the action; it is part of thought” (Clark and
Chalmers 1998: 10). Equally essential to their account is the requirement that a
coupling system be reliable so that the external resources will be at hand when
they are needed: this does not mean that the resources must be always present
(as in the brain), but present when required (Clark and Chalmers 1998: 11–12).
They advance a notorious example that been much discussed in the cognitive
literature: a fictive Otto suffering from Alzheimer’s disease who relies heavily
upon information he carries in a notebook when making his way in the world.
“We are advocating a point of view,” they write, “on which Otto’s internal pro-
cesses and his notebook constitute a single cognitive system” (Clark and Chalmers
1998: 16).
Clark and Chalmers assign an essential role to language, which they describe
as “a central means by which cognitive processes are extended into the world”
(Clark and Chalmers 1998: 11). Later in the essay, after citing several examples
from everyday life, they conclude:
15 The thesis is much expanded and developed in Clark 2011, where the entire 1998 article is
reprinted as an appendix.
54 Michael Marrinan
In each of these cases, the major burden of the coupling between agents is carried by
language. Without language, we might be much more akin to discrete Cartesian “inner”
minds, in which high-level cognition relies largely on internal resources. But the advent
of language has allowed us to spread this burden into the world. Language, thus
constructed, is not a mirror of our inner states but a complement to them. It serves as a
tool whose role is to extend cognition in ways that on-board devices cannot (Clark and
Chalmers 1998: 18).
I invoke the literature of cognitive science to set the stage for re-thinking what
occurs when users of the Encyclopédie manipulate the plates and pages of corre-
sponding text in order to follow a chain of ideas sketched by Diderot’s cross-
references. Such users are not simply looking, not simply reading, but are
actually thinking through the materials about ideas, problems, or technical matters
that exist nowhere but in the interaction of their bodies with the oversized folios
of the publication. They are coupled to a cognitive system that does not merely
catalogue component parts but generates knowledge by eliciting study of those
parts in new and unpredictable formations guided to a large extent by the indi-
vidual user’s personal orientation, life experience, and expertise. Curiosity, the
hunger to know, is the motor force that brings to life the inert materials of ink
and paper.
7 Conclusion: Re-thinking Peirce’s Thing

The Encyclopédie invites the invention of correlations by living, embodied sub-
jects – perceptually guided through visual arrays by a pervasive material whiteness
and linguistically coupled to a labyrinth of textual arrays – who unfold a learning
experience that is constantly renewed in the here-and-now of physical encounter.
The images are objects of use embedded in a system of discovery from which the
user cannot extract himself without destroying the cognitive chain: nothing
happens without a user; the visual arrays of discontinuous components do
not tell a story of their own. In this sense, we can claim the imagery of the
Encyclopédie escapes “representation” to achieve a “thingness” comparable to
other objects in the user’s environment, and does so without becoming a strictly
abstract form.
It is unlikely that contemporary discussion of cognitive embodiment would
find much sympathy from Peirce, who upholds a clear separation between think-
ing and thought:
‘Thinking’ is a fabled ‘operation of the mind’ by which an imaginary object is brought

before one’s gaze. If that object is a Sign upon which an argument may turn, we call it
a Thought. All that we know of the ‘Thinking’ is that we afterwards remember that
our attention was actively on the stretch, and that we seemed to be creating Objects or
transformations of Objects while noting their analogy to something supposed to be real
(Peirce 1976b: 314).
Peirce goes on to describe the “operation of the mind” as an ens rationis – an

abstract logical entity having no positive existence outside the mind. He calls it
“fabled” or imaginary. The passage just cited appears in the context of Peirce
defending himself from those who complain that his “account of logic as a
science of signs” is not sufficiently anthropomorphic; that it does not make
“explicit allusion to the human mind.” He replies to the charge that “nothing
but confusion” can result from a more anthropomorphic view of logic (Peirce
1976b, 4: 313). All of which suggests that Peirce would have little regard for the
extended mind thesis of Clark and Chalmers.
We thus understand why it is important for Peirce to specify precise and
narrow limits on the “imaginary moment” when a diagram becomes “the very
thing” upon which we experiment, for it raises the spectre of anthropomorphism
amidst the workings of logic. We also understand why Stjernfelt is obliged to
loosen Peirce’s requirement that diagrams be intentional as a way to reserve a
space for the unmotivated but analytic viewing of paintings that he claims is a
legitimate practice of diagrammatic experimentation. He justifies this corrective
by appealing to Peirce’s in actu requirement for sign use, arguing that a
“diagram only becomes a diagram in actu when it becomes part of the inference
process” (Stjernfelt 2007: 97 and 278–279). For Stjernfelt, when a viewer infers
distance or space in a picture, compares the lighting or coloration of one part
of it to another, imaginatively moves objects around or walks into the depiction
he or she is performing diagrammatic operations. It seems to me that Stjernfelt
is describing, on one hand, a type of engagement that Peirce dismisses as
merely thinking and, on the other, a systemic coupling of viewer and image fully
consonant with what we encounter in the pages of the Encyclopédie and con-
temporary accounts of the extended mind. Peirce himself, in the citation that
opens this paper, slips surreptitiously from abstract reasoning with a diagram
that “is for us the very thing” to contemplating a picture and the fleeting
“moment when we lose the consciousness that it is not the thing . . . it is for the
moment a pure dream” (W 5: 163). He is careful to frame and limit that moment
and so ensure that thought retains the upper hand over thinking. Stjernfelt’s
corrective attempts to provide a Peircean motive for looking at pictures, but it
opens the experience to externalist considerations of duration and the body
that are anathema to Peirce’s internalist analysis.
56 Michael Marrinan
In cognitive science, the two antithetical camps – internalists and exter-

nalists – will probably never convince the other of their errors. Most current
work focuses on systems, autonomy, and sense-making that dispenses with the
dichotomy by assuming that the body – including the brain – is fully embedded
in an arena of cognition (Thompson and Stapleton 2009; Di Paolo 2009; Hutto
2011). This is neither the time nor the place to engage in that fascinating debate,
except to note in passing that the production of knowledge schematized in the
Encyclopédie along the lines I have sketched here appear even more timely than
originally thought. Systemic analysis of the materials assembled by Diderot and
his colleagues escapes antiquarianism by virtue of demonstrating, in a print
medium that remains somewhat manageable, the mechanics of knowledge
formation when images, texts, and visual arrays are brought together in a
mashup that I will call diagrammatic: meaning that they present materials in a
non-narrative, non-mimetic, discontinuous manner. On my account, diagrams
construct circumscribed environments conducive to thought-experiments by
coupling users to a specific bandwidth of information without stipulating the
rules of engagement. My reading stands diametrically opposed to that of Peirce,
who requires in the first instance that a diagram “is an Icon of a set of rationally
related objects” (Peirce 1976b, 4: 316). The distance between us closes some-
what when Peirce invokes that “state of activity in the Interpreter, mingled with
curiosity” that leads to experimentation (Peirce 1976b, 4: 318). My argument
suggests that Peirceans long to have it both ways, but that looking at pictures
presents them with impenetrable troubles. Perhaps the way forward is to follow
the lead of our colleagues in cognitive science, to resist arguing about the
logical status of diagrams, and to celebrate those moments when contemplating
pictures leads us not to reason but to dream.
Acknowledgments
Much of my thinking about diagrams depends upon the many hours of discus-
sion, and sometimes argument, that I spent with my colleague and co-author
John Bender during the time that we wrote The Culture of Diagram. I am grateful
to him for those months of invigorating collaboration. I also thank Christina
Ljungberg and Sybille Krämer for the invitation to participate in this volume. I
presented some of this material to the conference “Thinking with Diagrams”
organized in July 2011 at the Freie Universität Berlin. The good discussions, and
probing questions about why Charles Sanders Peirce figures so little in The
Culture of Diagram, inspired the present essay.
Sun-Joo Shin
2 The role of diagrams in abductive
reasoning
1 Explicative versus ampliative reasoning

A standard classification of arguments goes like this: There are two kinds of
reasoning, explicative and ampliative, depending on whether the conclusion
is fully or partially supported by the premises. Deductive arguments belong to
the first and inductive to the second category. While I am not contesting this
well-accepted division of arguments, I attribute the trend of work on arguments
to this framework and would like to show what has been missing in our discus-
sions of the topic.
Explicative reasoning guarantees the certainty of the conclusion as long as
the truth of the premises is assumed or guaranteed and deductive processes are
correctly carried out. On the other hand, ampliative reasoning does not guarantee
the truth of the conclusion even if the premises are true. Sacrificing certainty, we
obtain expansion of knowledge as the name ‘ampliative’ indicates. By the same
token, we can then see a shortcoming (?) of deductive reasoning: It gives us
assurance of the conclusion, but only within what is asserted by the premises.
That is, the strengths and weaknesses of explicative and ampliative reasoning
are opposite from each other, and as we see below, their strengths have been
the focus of the research on each side.
Mathematics and logic, where deductive reasoning is carried out, examine
deductive processes so that a formal system may be established. If we could
mechanize the processes, the certainty of derivation and, hence, the truth of
the conclusion, would be guaranteed even at a mechanical level. Not surpris-
ingly, formalization has been a main topic where deduction plays a crucial role.
Our expectation for science is a different story. If we cared only about certainty
and formalization, we would not have science as we know it now. We expect a
scientific theory to go beyond what we observe or experience so that we may
explain what is going on and we may predict what will come. All of us involved
in scientific activities not only acknowledge a gap between data and a theory,
but we also welcome it. Without a leap, science would not provide us with
anything new. Hence, much work of ampliative reasoning has been focused on
Sun-Joo Shin, Yale
58 Sun-Joo Shin
the leap from data to a theory, mainly how to construct it, how to justify it, and
how to evaluate it. A theory is waiting to see how well reality will bear it out.
While practitioners of deductive reasoning are interested in mechanization of
the process, scientists who exercise ampliative reasoning aim to conquer an
indefinitely expanded territory of our knowledge. Both sides trade on their
merits, and nothing is wrong with that.
How about philosophers? One of the main tasks of philosophy is to investi-
gate a subject at a meta-level. What kinds of meta-issues arise from deductive
and inductive processes? First of all, there is an issue of justification. Sure
enough, Hume’s skepticism has generated a tremendous amount of literature
about the justification of induction. The justification of deduction has also been
discussed rather extensively in the literature (Carroll 1895, Prior 1960 and Belnap
1962). As Hume himself admitted, skepticism toward justification stays at a
theoretical level and life goes on. We keep making inductive judgments all the
time, from ordinary daily events to scientific activities. This is even more so
in the case of deductive reasoning. Mathematicians do not stop to dwell on the
justification of modus ponens. The majority of them would not even feel the
need of its justification.
Relating to the justification issue, the philosophy of science has raised more
specific questions as to the practice of ampliative reasoning involved in scien-
tific theories – how to come up with a theory, how to select one out of many
possible theories, and how to test out a theory for indefinitely many future
cases, etc. As a contrast, discussions in the philosophy of logic and the philosophy
of mathematics have focused on more foundational questions, for example, logical
truth, mathematical truth, ontology of numbers, etc., instead of the deduction
process itself. As a result, there is less overlapping concern between philoso-
phers and practitioners in explicative reasoning than in ampliative reasoning.
One of the reasons for the discrepancy is that many believe the deduction pro-
cess, being certain, is less controversial than the induction process, being only
probable. Logicians and mathematicians focus on formal systems themselves,
without much ontological commitment. Axioms and inference rules should fill
in a gap between premises and conclusion, and we know how to check whether
each step is valid within a system.
On the other hand, scientists have to find a way to get to a hypothesis in
order to establish their own theories for explaining phenomena, and there is no
specific guideline to bridge between observation and a hypothesis. What’s more
intriguing, no hypothesis could fill in a gap between data and a theory. There
is no 100% proof, so to speak. We expect constant challenges both from other
theories and from future events. It is not surprising why many philosophers find
these questions more interesting at a theoretical level than finding algorithms
The role of diagrams in abductive reasoning 59
and establishing formal systems for explicative reasoning. Hence, much of the
literature in philosophy of science directly addresses the inductive process while
the literature in the philosophy of logic and mathematics is more occupied with
fundamental questions instead of the deductive process itself. I claim that what
exactly is going on with a deductive step has been massively overlooked and
argue that deductive reasoning is full of mysteries, in spite of its certainty.
Suppose a system is given to us. Suppose a deductive step is taken by citing
an axiom (or a previous line) and an inference rule. Then, we may check its
correctness easily since a finite number of axioms and inference rules are
involved. A deductive proof consists of this process only finitely many times.
We do not seem to have any interesting theoretical questions inside a system.
Only a few questions about a system seem to remain, such as “Is the system
valid and/or complete?” “How powerful is the system?” “Are there logically
equivalent systems?” If that were the case, deductive steps themselves would
be mystery-free.
Let me raise simple-minded questions. Why did it take more than three cen-
turies to prove Fermat’s last theorem successfully? Does the fact that Goldbach’s
conjecture is not yet proven show the lack of rules or axioms of a system we are
working on? Or, are we waiting for more effort, more time, and more insight?
More mundane questions: Why do we struggle in a geometry class to prove a
theorem? We know axioms, definitions, and rules. We are told that the proposi-
tion we are supposed to prove is a theorem, which means that the proposition
follows as a consequence from axioms, definitions, or previous theorems. All
we are doing is to extract the information of the proposition from the informa-
tion of the previous theorems and axioms. After all, the conclusion is contained
in the premises, we say. The number of axioms, previous theorems, and rules
are all finite as well. Then, why are we having a hard time to prove the logical
consequence from the premises to the conclusion? Let me call this question the
mystery of deduction, and this is the mystery that has been overlooked in the
literature which I would like to take up in the paper.
Interestingly enough, the deduction process itself, unlike the inductive process,
has not drawn much philosophical attention. Deductive steps have been con-
sidered as a crystal clear procedure while guessing and agony have been granted
for inductive steps. I believe the division between explicative versus ampliative
reasoning has provided us with assurance of deductive reasoning while a gap
between premises and conclusion in inductive reasoning has constantly fascinated
philosophers.
Let us pause here to examine what has been assured in explicative reason-
ing, and how. Deduction, which is used in explicative reasoning, if correctly
carried out, assures us of the truth of the conclusion if the premises are true.
60 Sun-Joo Shin
Hence, two conditions for the assurance are: (i) true premises and (ii) valid infer-
ence steps. The first condition is assumed and is not expected to be shown
within an argument. The second condition requires a meta-theory where the
rules of our system are valid, that is, the soundness of the system. Then, given
(i) and (ii), any step taken from premises adopting an inference rule is assured
to be correct, and so is the proposition reached in the step.
This is a question: Do (i) and (ii) assure us what a next deductive step will
be? I could make a valid step, by citing a previously proven theorem in the
system, say ‘(p v ~p).’ The truth of the statement is assured, but not its useful-
ness. Most probably, this is not the step I need to take to get to the conclusion in
an argument. That is, the assurance is given to the result, not to the process
itself. Even when the number of premises and the number of inference rules
are finite, it seems as if there is an infinite number of valid steps we could take
in a deductive argument. This is where the mystery of deduction lies, I claim. In
spite of the ‘containment’ relation in information between premises and con-
clusion, many deductive arguments are not easy to make or even to follow.
Some of them are so hard that many decades or centuries pass until a genius
guides us through a mysterious deduction forest.
Why are some deductive steps are trivial while some are not? Are there dif-
ferent kinds of deduction? For non-trivial cases, what is the source of difficulty?
In the case of ampliative reasoning where the conclusion contains more than
what the premises say, difficulty is inherent when we move to a next step. For
this reason, how to search a hypothesis and how to set up a theory have been
major topics in the philosophy of science. In the next section I would like to
justify a division within deductive reasoning − trivial versus non-trivial. After
examining the source of the difficulty of deductive reasoning in the third section,
I borrow Charles Peirce’s discussions of abduction to argue in the fourth section
that non-trivial deduction requires abductive reasoning. In the fifith section, I
make a case for the claim that a form of representation is closely related to the
success of abductive reasoning as I characterize it in the paper.
2 Classification of explicative reasoning

Kant’s transcendental philosophy started with his puzzle over mathematical
truth. Is it analytical, meaning that the conclusion reached in mathematics is
contained, that is, is an explication of, given premises? Kant’s answer was a
decisive “No.” In a mathematical theorem, say “The sum of the interior angles
of a triangle is 180 degrees,” the analysis of the concept ‘triangle’ does not tell
us anything about the sum of its interior angles. There is a difference between
mathematical truth and an analytic statement such as “All bachelors are un-
married.” Hence, Kant classifies mathematical truth as synthetic.
On the other hand, mathematical truth seems to be different in two aspects
from other synthetic truth, say, “The earth has one moon” or “It is sunny today.”
First, mathematical statements are universally accepted unlike with other syn-
thetic contingent truth. Second, even though mathematical judgments are not
made by the analysis of concepts, they do not come from our experience, either.
Hence, Kant classifies mathematical truth as a priori. Here is the birth of the
category ‘synthetic a priori.’ Then what is the source of our mathematical judg-
ments? This is where Kant’s long journey starts.
For our discussion, we do not need to follow his journey, but will focus on
Kant’s division between analytic a priori and synthetic a priori. We are aware
that Kant’s logic is different from ours, and modern logic covers a larger territory
for conceptual analysis than Kant’s logic. Hence, many would like to say Kant’s
synthetic a priori category is outdated. Also, the analytic/synthetic distinction
itself has been controversial, and some do not want to buy into the distinction.
Nonetheless, I believe Kant’s distinction is illuminating: he saw something dif-
ferent between simple and complicated deduction processes. In Kant’s mind,
mathematical truth is something beyond a process of deduction. His logic is
limited to Aristotelian monadic logic. When the logic of relations gets in the
picture, his terminology ‘synthetic’ might be questionable, but his original divi-
sion still stands as a demarcation between monadic and relational logic. This
demarcation is directly related to our investigation into the deduction mystery,
I claim.
There is a deduction sequence to show that the sum of the interior angles of
a triangle is 180 degrees. Whether the proposition is reached by analyzing the
concept of triangle is not an issue, but it is clearly an explicative process. Then,
is the triangle proposition of the same kind as Kant’s analytic example of “All
bachelors are unmarried”? Some might think so, believing that the difference
lies only in the degree of difficulty. Similarly, those who believe Kant’s distinc-
tion between (Aristotelian) logical and mathematical truth comes from Kant’s
ignorance of modern logic might say the distinction should be extinct. I do not
deny that monadic is simpler than relational logic. When we move to relational
logic, carrying out deduction is more complicated, and hence more difficult. I
claim the difference is not a matter of degree, but a matter of kind, and a different
kind of difficulty involved in relational logic is the source of the deduction
mystery itself. In that sense, Kant’s separation of mathematical truth from
monadic logical truth is remarkable.
62 Sun-Joo Shin
Let me introduce another philosopher’s insight about different kinds of

deduction – Charles Peirce’s theorematic versus corollarial reasoning:1
It [deduction] is either Corollarial or Theorematic. A Corollarial Deduction is one which

represents the conditions of the conclusion in a diagram and finds from the observation
of this diagram, as it is, the truth of the conclusion. A Theorematic Deduction is one which,
having represented the conditions of the conclusion in a diagram, performs an ingenious
experiment upon the diagram, and by the observation of the diagram, so modified, ascer-
tains the truth of the conclusion. (CP 2.267)
Kant’s analytic/synthetic distinction is drawn, more or less, in terms of the infor-

mation between premises and conclusion. In the case of analytic reasoning, the
information of the conclusion is ‘contained’ in the premises, and synthetic a
priori statements are something more than what the premises say, and hence
are ampliative, Kant believed. On the other hand, Peirce acknowledges both
(Kant’s) analytic and (Kant’s) synthetic a priori reasoning are deductive, i.e.
explicative, but made a distinction between them in terms of the process – one
by observation and the other by manipulation. Kant’s ‘containment’ (between
premises and conclusion), I think, roughly corresponds to Peirce’s phrase ‘obser-
vation.’ Kant’s beyond-conceptual-analysis, but a priori, syntheticity is captured
by Peirce’s ‘performing an experiment.’ Hence, Peirce’s corollarial/theorematic
deduction more or less lines up with Kant’s analytic/a priori-synthetic distinction.
Kant’s ‘beyond’ idea invites us to his transcendental philosophy. Instead, for
our purpose I would like to suggest that we investigate what Peirce means by
‘experiment.’ What is involved in performing an experiment? Since it is part
of the deductive process, the experiment should be carried out within given
premises and by given rules of inference. Then, why is an ingenious experi-
ment needed? I speculate that these issues are directly related to our deduction
mystery.
3 Searching for a deductive step

Peirce’s characterization of theorematic reasoning is expressed as follows:
The peculiarity of theorematic reasoning is that it considers something not implied at all
in the conceptions so far gained, which neither the definition of the object of research
nor anything yet known about could of themselves suggest, although they give room for
it. Euclid, for example, will add lines to his diagram which are not at all required or
suggested by any previous proposition. (Peirce 1976: 49)
1 I made a case for an (approximate) correspondence between Kant’s and Peirce’s distinction in
Shin 1997.
An immediate question is raised: Peirce says theorematic reasoning brings in a

step not implied in the previous lines. If so, could it be deductive? Following
Peirce’s citation of Euclid and considering that his terminology – theorematic
and corollary − comes from geometry, let us cite the first proposition from
Euclid’s Elements Book 1.2
Proposition 1: On a given straight line to construct an equilateral triangle.

(Heath 1956)
Proof: Let AB be the given straight line.
(1) Draw a circle ABD, with A as a center and radius AB.

(2) Draw a circle BAE, with B as a center and radius BA.
(3) Let C be the point where the two circles, circle ABD and circle BAE, inter-
sect. Then, draw straight lines CA and CB.
(4) AC = AB.
(5) BC = BA.
(6) Hence, triangle CAB is an equilateral triangle. QED
Steps (1)−(3) require construction and steps (4)−(6) are taken by observation.
Anyone who tried to solve the proposition would agree that the construction
process is more difficult than the observation steps. Among the construction
steps, drawing the first circle in step (1) is the most difficult one. Even though
connecting points in (3) to form a triangle is a construction process, it is rela-
tively easier than step (1). Borrowing Peirce’s characterization of theorematic
reasoning, we may say that step (1) requires ‘an ingenious experiment’ on line
AB. Why is the construction ingenious? The first answer would be: It brings in
a new geometric object in the proof (Hintikka 1980). This is not incorrect, but
invites the next question: Why does the introduction of a new object cause
much trouble?
2 The point I am making here could be even stronger in the case of later propositions.
64 Sun-Joo Shin
This is a crucial question to answer and this is where the mystery resides in
the deduction process! In the above proof, given line AB, we are almost certain
that something should be constructed with the line. That is, we need to intro-
duce a permissible geometrical object into a proof. There is no clear hint or
implication about what should be the right object we need to bring in at this
stage. This is, I claim, what Peirce meant by “not implied at all in the concep-
tions so far gained.”
Let me suggest three components of a theorematic step and name them as
follows:
e [Multiple Choices] There are other legitimate geometrical objects (instead of
a circle ABD) we could introduce at that stage.
e [No Algorithm] There is no specific guideline which object should be drawn
for a proof.
e [Ex-post Evaluation] Which is the right one among multiple permissible
choices will be known only after a proof is complete.
Let me continue the above proof example to illustrate the three components.
Suppose we introduced the following objects, instead of a circle ABD:
Figure 21: A theorhematic deductive step
All of them are justifiable: not only a circle ABD but also these objects follow
from postulates or definitions.3 All of them being deductively permissible
objects, each construction would be a valid deductive step. This is the Multiple
Choices component. Then, how do we come to choose the circle ABD over the
other choices? The No-Algorithm clause answers “We are on our own.” If so,
what prevents us from introducing other objects? What if we choose to draw a
parallel line, instead of circle ABD? As the Multiple Choices clause says, it is a
deductively valid step and we are in the middle of a deductive proof. However,
we do not want to take any of these steps other than drawing circle ABD.
How do we know circle ABD is the right object we should be looking for, not
any other objects? It seems only after we may prove that the circle ABD proves
(!) the proposition! There is no ex-ante rationalization, but only an ex-post
evaluation.
3 Let’s recall this is the first proposition, and we can easily imagine more possibilities for later
propositions, citing previously proven propositions.
The three components together tell us that deduction, which delivers certainty,
does not provide us with certitude as to how to deliver certainty. Yes, that is an
ironical mystery. Embracing it, we should stop wondering why we often, more
often than we would like, get stuck in deductive reasoning even when we are
assured that a given conclusion follows from given premises.
How to make the right choice is at the heart of the deduction mystery. Ex-
tending the mystery to the context of deduction in general beyond geometrical
proofs, we may generalize the issue in the following way: Without an explicit
guideline for the next step we need to take, how do we navigate through avail-
able premises and rules to make the right choice? Which axiom, which theorem,
which definition, which inference rule do we want to recall to take the next step?
This is indeed the most immediate question for those who actually carry out
deductive reasoning to prove a proposition. My hunch is: We make the right
choice in a theorematic step, not deductively, but abductively. In the next
section I would like to present arguments to back up this bold suggestion.
4 Abductive elements in deduction

I suggest that we look into how a scientific theory is presented. And I argue for a
strong similarity between the search for a mysterious deductive step and the
search for a scientific hypothesis. The reader would immediately point out that
one is deductive while the other is inductive, hence explicative versus amplia-
tive. A deductive step, how mysterious it may be, logically follows from previous
steps, and a hypothesis of a scientific theory is not a logical consequence of
given data. I do not deny a major difference here, but would like to point out
that an emphasis on this contrast has caused us to overlook a similarity between
the two different kinds of reasoning. Moreover, I claim that the similarity is
significant to practitioners. When we recognize similar aspects of the search-
procedures in a theorematic step and in a hypothesis proposal, each type of
reasoners may get benefit from the other side’s discussions. Hence, in order to
explore our topic, the deduction mystery, I borrow an idea of Charles Peirce
which has been advanced and greatly discussed in the context of the philosophy
of science. First of all, I need to convince the reader of the analogy between a
theorematic step and a hypothesis-launching process.
The role of a hypothesis − how to come up with a hypothesis, how to justify
it, how to test it, how to amend it, when to abandon it − these are only some of
the main topics related to hypotheses in the philosophy of science literature.
Rich discussions of the issues correctly assume that a hypothesis, in essence,
says more than given data. Hence, it is a prime example of ampliative reasoning.
66 Sun-Joo Shin
Many philosophers have been fascinated with the ampliative aspect of a scien-
tific theory. My question is: Is the ampliative nature the only source of these
hotly debated discussions? Are these hypothesis-related issues controversial
and interesting just because a hypothesis covers more than what data show?
Without any definite answer in mind, I am going to apply the three com-
ponents of a theorematic (hence, deductive) step we presented in the previous
section to a hypothesis proposal in a scientific theory. (i) Are there multiple
candidate hypotheses for given data? Definitely, “yes.” More than one hypothesis
is compatible with the given data a theory aims to explain. (ii) Is an algorithm
available for our choice of a hypothesis? Definitely, “no.” Lack of mechanical pro-
cedure has been taken for granted in the search for a hypothesis and has been
attributed solely to its inductive and ampliative nature. (iii) How do we evaluate
hypotheses? How do we know which hypothesis is better than or superior to
others? Not by its own truth-value alone: but its merit absolutely depends on
the scope of its coverage or its ability to predict future data.
Multiple choices, lack of an algorithm, and evaluation − all of the three
features are common between a theorematic step and a hypothesis proposal.
Now it is time to tap into the related rich literature to hypothesis in the philosophy
of science in order to understand the deduction mystery better. Charles Peirce’s
abduction is a good place to start. Two preliminary remarks are in order.
First, please note that theorematic reasoning − where I placed the deduction
mystery − is also Peirce’s concept. However, interestingly enough, Peirce himself
never connected theorematic reasoning and abductive reasoning. On the con-
trary, he placed these two categories of reasoning at the opposite sides of a
spectrum, one at an explicative and the other at an ampliative end. Therefore,
my view that abductive reasoning could provide us with insight about the
mystery of theorematic reasoning would be almost against Peirce’s categoriza-
tion of reasoning.
Second, Peirce’s classification of reasoning went through various stages,
and so did his abduction.4 Hence, Peirce’s abduction itself, i.e. what Peirce
meant by ‘abduction,’ has formed an interesting and independent research topic
both among Peircean scholars and among philosophers of science, and I do not
intend to get into those debates in this paper. Instead I rely on well-accepted
and uncontroversial parts of Peirce’s own writing about abduction and advance
my claim how deduction requires abductive elements.
Not surprisingly, Peirce’s tireless inquiry into scientific discoveries and scien-
tific explanations centered around reasoning related to hypotheses. Considering
4 For the development of abduction in Peirce’s own philosophy, please refer to Fann 1970. For
general introduction of the topic, refer to Burks 1946; Douven 2011a; Douven 2011b.
a crucial role of a hypothesis in a scientific theory, how to form a hypothesis

becomes one of the main topics in scientific inquiry. This is one of the reasons
why Peirce zoomed in the process and labeled it with his own new word ‘abduc-
tion’: “Abduction is the process of forming an explanatory hypothesis” (CP,
5.171). Since a hypothesis usually tells us more than observable data, abduction
has been considered a prime example of ampliative reasoning. However, I will
argue some deductive steps, which are explicative, also share essential features
with abductive reasoning, and moreover those properties are at the heart of the
deduction mystery discussed in the previous sections.
In addition to the fact that Peirce carved up the hypothesis production pro-
cess and called it ‘abduction,’ many commentators agree that Peirce’s mature
view placed abduction at a meta-level, that is, a method of methods, or reason-
ing of reasoning. As Fann put it, “Peirce wished to show that reasoning towards
a hypothesis is of a different kind than reasoning from a hypothesis” (Fann 1970:
4). Precisely it is the meta nature of abduction that I would like to relate to
theorematic steps we are interested in.
We ended the last section with the following hunch: searching for a correct
step in a tricky deduction is done abductively. Let me substantiate the claim.
Just as Peirce makes a distinction between constructing a hypothesis and infer-
ring from a hypothesis, it is important to distinguish the following two kinds of
reasoning in deduction as well: a newly introduced step should follow deduc-
tively from previous steps, but producing the deductive step itself is not guided
by deduction. When we ask why finding a deductive step is difficult, we are mak-
ing an inquiry into the reasoning towards a deductive step, not the reasoning
from previous steps to the new deductive step. That is, the mysterious element
of explicative reasoning resides at a meta-level, not at the first level of reasoning
from previous steps or axioms to a next step. Please note that a hierarchy of
reasoning exists in explicative as well as in ampliative arguments. It is the
meta-level of a deductive step where abduction plays a role. I suspect that since
Peirce’s abduction is introduced in the context of ampliative reasoning where a
hypothesis is adopted, many of us, including Peirce himself, group abduction
together with ampliative reasoning and important features of abduction are
not associated with deduction at all. Once we raise a meta-question about a
deductive step, that is, how this step is suggested (as opposed to whether this
step follows from previous steps), Peirce’s abduction gets into the picture.
Peirce states that abduction is reasoning, but at the same time he empha-
sizes that the process is inherently different from other reasoning: “. . . and yet
it [adopting a hypothesis on probation] is reasoning’’ (CP 8.388). “No reason
whatsoever can be given for it, as far as I can discover; and it needs no reason,
since it merely offers suggestions” (CP 5.171).
68 Sun-Joo Shin
From given data to a hypothesis, there is no clear algorithm and it seems to

be a mere suggestion, which implies that there is more than one candidate for a
hypothesis. When we suggest something, the suggestion is tentative, not ruling
out that there might be other suggestions. Why is it so? Peirce highlights the
lack of an algorithm for our reasoning toward a hypothesis:
This step of adopting a hypothesis as being suggested by the facts is what I call abduction.
. . . There would be no logic in imposing rules, and saying that they ought to be followed,
until it is made out that the purpose of hypothesis requires them. (CP 7.202)
Multiple choices and lack of an algorithm go hand in hand as important features

of abduction. As we discussed in the previous section, searching for a theorematic
step cannot be deductively conducted, since there is more than one choice and
there is no guideline for the right choice. Both processes − constructing a
hypothesis and constructing a theorematic step − require a creative mind to
suggest an ingenious and insightful guess.
Another striking similarity between a hypothesis suggested by abduction
and a deductive step proposed by abduction (I claim) is the evaluation process:
“How could one justify one’s choice?” A hypothesis cannot be evaluated by its
own truth-value. We have an infinite number of true statements which have
nothing to do with our data in terms of explanations. Being a true statement is
a necessary condition. Peirce, not surprisingly, discussed the issue extensively
in many places. Let me cite some of them:
It became axiomatical that a hypothesis adopted by abduction could only be adopted on

probation, and must be tested. (CP 7.202)
Its [abduction’s] only justification is that from its suggestion deduction can draw a predic-
tion which can be tested by induction, . . . (CP 5.171)
What is good abduction? . . . [I]t must explain the facts. . . . Its end is . . . to the avoidance of
all surprise and to the establishment of a habit of positive expectation . . . (CP 5.197)
A hypothesis will be evaluated in terms of its explanatory power and its ability
to predict the future. That is, its legitimacy is proven in terms of its utility. Many
of us could easily be reminded of Peirce’s pragmatism, and realize why abduc-
tion plays such an important role in Peirce’s philosophy.
I argue a deductive step chosen in theorematic reasoning is accepted or
rejected in a similar way. Being true is necessary, but far from being sufficient.5
Is there something similar to the explanatory power of a hypothesis in the case
of a deductive step? A scientific theory aims to explain certain observable data
with our current knowledge. We may consider a deductive proof as an explana-
5 One could repeat tautologies which might be technically valid, but of no use, period.
tion of how a conclusion follows from given premises. Deductive explanations

are clearly defined in terms of logical consequences. Each step is reducing a
gap between what is proven to be true and what we would like to prove to be
true, and the relation between steps should be a logical consequence. Some
gaps are quite trivial and easy to fill in, while some are difficult and require a
clever mind at work. We have called the latter theorematic steps, and when a
correct choice is made, the rest of the proof becomes less surprising and less
gappy. Suppose a presented step is true and follows from previous steps. That
would not be enough to be a good explanatory step to the next step or to the
conclusion. Just like a good or useful hypothesis, a theorematic step we are look-
ing for should lead us to the conclusion more smoothly. That is, the legitimacy
of a deductive step is also evaluated in terms of its utility.
There are two major differences between the utility of a hypothesis and the
utility of a deductive step. First, what they aim to explain is different from each
other. A hypothesis is presented to explain facts or observed data. A theorematic
step is proposed to demonstrate the logical consequence from premises to the
conclusion by bridging a gap between what we know and what we want to
know. Many have accepted the explanatory role of a hypothesis, but have not
related it to a deductive step. Since a deductive proof has been considered as
demonstration, I suspect its explanatory role has not been recognized. There
are various ways to explain things, and demonstration is one of them. If so,
taking a proof to be an explanation helps us think of the utility of a deductive
step.
The other difference in utility between a hypothesis and a deductive step is
predictive power. When two hypotheses, say H1 and H2 , equally explain given
data, their predictive power matters: Which hypothesis predicts future events
better? I do not think we can talk about the predictive power of a deductive
step. There would be no future event a proof needs to demonstrate or explain.
However, there is an analogy to be made here: A proof is a sequence of steps,
and a correct theorematic step should lead us to the conclusion. Suppose there
are two theorematic steps which would lead us to the conclusion, say S1 and S2.
Here, how S1 and S2 lead us to the conclusion could matter. Which one helps us
to proceed to the next steps, and eventually, to the conclusion, more easily or
more smoothly?
5 Abduction and representation

It is easy to see why the proposal of a hypothesis is a difficult task. A hypothesis
covers more than what given data seem to tell us. If we focus on the ampliative
nature of a hypothesis only, the difficulty of some deductive steps poses a
70 Sun-Joo Shin
puzzle. They are explicative steps, meaning that a deductive step logically
follows from previous ones. Nonetheless, our practice shows how difficult it
could be to find a correct step in a deductive proof. In the third section, we iden-
tified the deduction mystery as a process of making a right choice among many
alternatives without any guideline. Even though each alternative is a logical
consequence of the previous steps along with axioms or definitions, a choice
itself cannot be made deductively, but (I claim) only abductively. That is, meta-
reasoning involved in interestingly difficult deductive steps requires abductive
reasoning.
In the previous section, we looked into Peirce’s abduction which has been
applied and discussed only in the context of constructing hypotheses and recog-
nized that there are striking similarities between the process of searching for
deductive steps and Peirce’s abductive reasoning toward hypotheses. Both take
us to the meta-reasoning level. Given more than one legitimate choice, both
processes need to appeal to our experience, insight, and intuition since there is
no algorithm we could rely on. After a hypothesis is proposed or a deductive
step is constructed, its evaluation could be made only after we have a chance
to check its utility − how a proposed hypothesis or a presented deductive step
leads us to the final goal more smoothly or more comprehensively.6
If my argument for abductive elements in deductive reasoning is convincing,
I would like to go back to the deduction mystery and raise the following prac-
tical question: What helps us to perform abduction more successfully? We
realized that there is no set menu for the search process and creative and in-
genious work is needed. In this section I claim that different forms of represen-
tation could make a difference in steering our mind to find a right step. I do not
intend to generalize about the relationship between representation and finding
a deductive step in the paper, but would like to open discussions about how
representation affects our theorematic reasoning.
Let’s go back to the example of section 3, to construct an equilateral triangle
for a given straight line. In a proof, the crucial theorematic step is to draw a
circle whose radius is the given straight line. How could one come up with an
idea to draw a circle? First, it is important to realize that we need to bring in a
new object to the proof. Second, we need to choose or find a correct object,
while there are many other possibilities involving with the line. We could have
drawn a parallel line to the line, we could try to bisect the line, etc. I claim this
is where forms of representation could make a difference.
Suppose we do not draw a line and rely on symbolization only. We may
arithmetize the entire proof without using a diagram at all. We do not lose any
6 Of course, the final goal of a hypothesis and the final goal of a deductive step are different
from each other.
rigor or legitimacy of the proposition.7 Hence, one might conclude that the form
of representation is only a vehicle which expresses the same proposition and
the same process from premises to the conclusion. If we focus only on what is
represented in each step of a proof, we are ignoring a meta-process − how to get
to each step. It would be just like zooming in on the process from a hypothesis
to observed data or to a prediction of future events. What would be missing is to
reason how to reason toward a hypothesis, which is meta-reasoning. However,
when we draw our attention to the meta-level, that is, how to form a hypothesis
or how to bring in a new object into a proof, the mode of representing informa-
tion becomes highly relevant to the process. Therefore, abduction, which is a
process of search, is sensitive to a form of representation.
There being no clear algorithm, insight and guessing are at play in abduc-
tive reasoning, as we discussed above. How we may become more creative or
more insightful so that we may propose the right guess is a fascinating question
whose answers are far from being clear. One safe response is to list items, e.g.
being more knowledgeable, being more experienced or trained, etc. I propose
that how we represent information, e.g., symbolic representation or diagram-
matic representation, is one of the items which facilitate abductive reasoning
more efficiently. Why is it the case?
I claim diagrammatic representation stimulates our mind so that we may
introduce a new object more easily than with corresponding symbolic represen-
tation. In the case of our example in section 3 of drawing a circle using the given
line as a radius, it would be even more difficult to come up with an idea of
drawing a circle if we took our premise represented only symbolically, say line
AB, instead of looking at a line drawn on the sheet of paper. When we encounter
a geometric line visually, our imagination is more active to search for various
geometric objects to be added to the line. Also, after adding a new object, it
would be easier for the mind to manipulate newly added information or to read
off information from newly constructed one. In our example, when two circles
are drawn, radii appear in a new diagram and all we need is to connect lines to
show all of the three lines are of the same length since AB and AC are radii of
one circle and BA and BC are radii of the other circle.
Let’s consider another often-cited example before we explore further why
diagrammatic representation is helpful for abductive reasoning:
7 On the contrary, some might think symbolization is more rigorous and more legitimate than
drawing diagrams.
72 Sun-Joo Shin
Proposition 10: To bisect a given a straight line. (Heath 1956)
Proof: Let AB be the given straight line. [We show how to bisect it.]
(1) Draw the equilateral triangle, ABC. (Proposition 1)
(2) Bisect the angle ACB and draw line CD. (Proposition 9)
(3) Then, triangle ACD and triangle BCD are congruent to each other, since AC =
BC (by the definition of an equilateral triangle), CD is common, and angle
ACD = BCD (by (2) above).
(4) Therefore, AD = BD. QED
Both (1) and (2) are theorematic steps which require abductive reasoning: (i)
New geometric objects are introduced. (ii) An equilateral triangle and a bisecting
line are not the only objects we can introduce at those moments. (iii) There is no
guideline to tell us which objects need to be brought in. (iv) Whether these are
correct objects is known only after they deliver the conclusion (smoothly).
In spite of legitimate warnings against the misuse of diagrams in a rigorous
proof, we cannot deny that figures in the proof prompt our mind to bring in
another figure more easily than does symbolic representation. It is true that
what we are seeing on the paper are tokens of geometric figures with specific
sizes and shapes, and some of these particular properties could mislead us.
For example, if we concluded D is bisecting the line AB by observation of the

diagram of (2), we would be committing a fallacy. Hence, particularity could be
an obstacle to proving a general proposition.
On the other hand, when we need to introduce a new object into a proof, an
object presented with its own size and shape seems to stimulate our mind better
so that we might come up with a new specific entity which could help us to get
to the conclusion. Another important benefit of diagrammatic representation is
the way we can take advantage of newly introduced objects. For example, in
Proposition 10 above, the line CD introduced by abduction helps us to see the
two new triangles that emerged, triangle ACD and triangle BCD. That is exactly
the insight we need to proceed to the conclusion! Diagrammatic representation
shows there are two new triangles created by the auxiliary line BD.
We could represent the same information symbolically, but the issue here is
not the representation of information in a static sense, but how to proceed to the
next stage. Our inquiry is about meta-reasoning or reasoning in a dynamic
sense. At that meta-level where we do not have a guideline (except one neces-
sary condition that the next move should logically follow from previous ones),
how information is represented could matter in more easily and more success-
fully performing abduction for the next right step, as our examples show.
Some might point out that our examples, so far, have been limited to geo-
metric proofs and that geometry is about geometric objects which are inherently
diagrammatic. Fair enough. We started our discussions with geometric proofs
mainly because we wanted to show a distinction between theorematic and
collorarial reasoning in the second section, and the terminology was borrowed
by Peirce from geometry. Outside geometry, my claim that how information
is represented makes a big difference in our problem-solving process can be
shown even more clearly.
First of all, many of us have experienced that we use diagrams for brain-
storming, either in geometric or non-geometric problems. Polya’s classic book,
How to solve it, is full of examples where geometric representation helps with
non-geometric problems.8 Outlining a given problem and, more importantly,
finding a way to get to the conclusion by insight are the main goals of brain-
storming, and most of us have relied on diagrammatic representation at that
stage. That is, diagrams seem to be more helpful (than symbols) for endowing
our mind with a clever idea to choose the right one among permissible, but not
necessarily helpful, paths. Hence, diagrams are accepted as a heuristic tool.
8 I have pointed out an asymmetry between geometric and non-geometric problems: We adopt
diagrams in both kinds of problems, but we hardly adopt symbolic representations for geometric
problems at the brainstorming stage. Refer to Shin 2012.
74 Sun-Joo Shin
However, this credit has not been theoretically justified. Why are diagrams
more helpful than symbols for brainstorming? What’s worse, the emphasis on
the heuristic side of diagrams has wrongly implicated the weakness of diagrams
in a non-heuristic proof context. The arguments presented in the paper provide
us with the rationale why diagrams could be a better form of representation at
a heuristic level, without ruling them out from a formal proof.9 Abduction is
crucial in navigating us in a directionless search for a proof step, and I argue
that diagrams are in many cases more useful for performing successful abduc-
tion. In the case of a geometric proof, finding an auxiliary geometric object is
more easily prompted by diagrammatic than symbolic representation. After all,
we need to bring in a figure into an existing figure. Also, after adding a new
object, it is easier for us to perceive new geometric figures and in many cases
these new objects are the key to the next step.
How about non-geometric problems? A diagonal proof, I believe, makes my
case. Let’s try to prove that truth functions for the set of sentence symbols
(which is countably infinite) are not countable. A proof is given by reductio ad
absurdum. I present the same information in two slightly different ways, one
being more diagrammatic than the other.
Let A1, A2, A3, . . . be sentence symbols.
[1] Suppose (by reductio) that we enumerate the truth functions for the set
{A1, A2, A3, . . . }, say v1, v2, v3, . . . . Then, vn = {<A1, X1>, <A2, X2 >, <A3, X3 >, . . . }
[2] Suppose (by reductio) we assume that v1, v2, v3, . . . are enumerated in the
following way:
Figure 22: A truth table
9 There is no good reason why we accept one form of representation as a heuristic tool, but not
as a formal proof, as long as we can formalize representation. Refer to Shin 1994.
The next step is to construct a truth function, say vk which is not on our list.
How?
The diagrammatic representation above hints for us to diagonalize out our
list. That is,
Figure 23: Constructing a truth function which is not on our list by diagonalization
Let vk = {<A1, X*11>, <A2 , X*22 >, <A3, X*33 >, . . . }, where X*nn ≠ Xnn.
Suppose vk were on our list. Let’s check what is the truth-value of Ak by vk. Two
cases:
(i) It is when vk (Ak) = T. That is, Xkk = T in the above diagram. Since X*nn ≠
Xnn, X*kk = F.
(i) It is when vk (Ak) = F. That is, Xkk = F in the above diagram. Since X*nn ≠
Xnn, X*kk = T.
But, X*kk = vk (Ak) by our definition of vk. By (i) and (ii), vk (Ak) = T if and only if
vk (Ak) = F, which is a contradiction! Therefore, vk cannot be on the list. QED
Of course, we can construct vk in the other representation [1] as well without the
diagram of [2]. This is a subtle but important point I have been making: What
we are interested in is not whether we can represent information in one system
and not in another, but how we get inspiration for a crucial step. That is, we are
making an inquiry as to which representation helps us with the abduction
process. Even though our truth-function example is not intrinsically diagram-
matic (unlike geometry), representing truth functions diagrammatically in [2]
stimulates our mind so that if we diagonalize out the list, we can easily con-
struct a truth function which is not on our list.
76 Sun-Joo Shin
6 Conclusion
Deduction, being explicative, does not produce any new knowledge in some
sense. On the other hand, in a scientific theory there is a leap from given data
to a hypothesis. Logical consequence would not provide us with a hypothesis
we are looking for. This is one of the main reasons why philosophers have been
focusing on the hypothesis searching process. In the paper I argued that there
are two different levels on which we can make our inquiry, and a confusion
between the two levels is one of the main reasons why we have been puzzled
with the difficulty encountered in deductive proofs. While each step follows
from previous ones, why is it difficult to make progress in a proof? In the case
of finding a hypothesis, we need new ideas which say more than what we have.
Strictly speaking, a valid deductive step should not introduce a new idea, but
extract a piece of information from given information. Then, why is finding a
logical consequence so difficult? Indeed, this is the mystery of deduction.
When philosophers are interested in exploring hypothesis-related theoretical
issues, the focus could be either on a logical leap from data to a hypothesis or on
how to come up with a hypothesis from data. Let me call each step, the first-
level and the meta-level of reasoning, respectively. As far as the first level goes,
there is a clear difference between a deductive step and the construction of
a hypothesis, in terms of logical consequence. However, at the meta-level,
we find striking similarities between difficult deductive steps and hypothesis-
finding processes: (i) There are more than one deductive steps which follow from
previous ones and more than one hypotheses which are consistent with given
data. (ii) There is no set menu to follow, and insight and clever guessing are at
play. (iii) Which is the right step or the right hypothesis is known in the context
of a bigger picture, that is, in an overall proof or in the prediction of future events.
Based on these important features shared between deductive steps and
hypothesis proposals, I suggested we identify the meta-reasoning process for
deduction as abduction, which Peirce introduced to explore the hypothesis
proposal procedure. How to perform abduction more successfully and more
economically, I believe, could be one of the exciting topics in various areas,
both theoretically and practically. In the last section of the paper I presented
one way to approach the topic, by exploring the relation between forms of
representation and abduction. Stimulating the mind to find a right path to the
conclusion can be done in different ways, and how information is represented
must be one of the factors to steer our mind in a creative way. How to gather
and organize information, not surprisingly, should make a huge difference in
how we navigate through information so that we may get to where we aim to
reach.
Valeria Giardino
3 Behind the diagrams: cognitive issues
and open problems
1 Why diagrams matter

In a passage from De Architectura, Vitruvius tells an anecdote about the Socratic
philosopher Aristippus, who got shipwrecked on the coast of Rhodes. Once on
the beach, he perceived some diagrams drawn on the sand and exclaimed to
his companions that there was no reason to worry: the place was safe, since
there were “marks of civilization” (Vitruvius, Book VI ). This story reminds us
very vividly of the importance of diagrams for human culture. Indeed, diagrams
are the products of many different cultures and rather sophisticated artifacts.
Their relation with other cultural phenomena is very rich. First, diagrams can
be compared to language: as well as language, they are introduced and con-
ceived as tools for communicating some content. Second, they give a particular
structure to their content by displaying it spatially, and for this reason they
happen to affect memory and inference. Third, their nature of images makes
them potential targets for issues concerning the relationship between percep-
tion, cognition, truth and knowledge. Fourth, diagrams can be also related to
art: their appearance is subject to aesthetic choices and occasionally they can
be considered from within a specifically artistic framework. Therefore, any theory
aiming at elucidating diagrams’ features and functioning should be able to
account for such a complexity.
Consider for the moment the role of diagrams in human reasoning. A com-
mon suggestion is that there is a sharp distinction between visual reasoning on
the one hand and sentential reasoning on the other. Nonetheless, throughout
the article I will avoid this distinction as ill-posed: as I will show, visual reason-
ing cannot be considered in isolation from sentential reasoning and vice versa
(even if the acknowledgment of this second direction is more controversial).
Human reasoning is heterogeneous, and different representational formats play
different roles in enhancing human inference by interacting repeatedly. My
suggestion is to consider instead the much more useful operative distinction
between informational equivalence and computational difference in comparing
two representations, which was introduced in a famous article by Larkin and
Simon (1987). To illustrate the distinction, a physical pulley system is given as
Valeria Giardino, Institut Jean Nicod, Département d’Etudes Cognitives, Ecole Normale Supérieure
– PSL Research University
78 Valeria Giardino
en example; its features are presented by relying on two different formats: a

linguistic description and a diagram.
The linguistic description runs as follows:
We have three pulleys, two weights and some ropes arranged as follows:
1. The first weight is suspended from the left end of a rope over Pulley A. The
right hand of this rope is attached to, and partially supports, the second
weight.
2. Pulley A is suspended from the left end of a rope that runs over Pulley B, and
under Pulley C. Pulley B is suspended form the ceiling. The right end of the
rope that runs under Pulley C is attached to the ceiling.
3. Pulley C is attached to the second weight, supporting it jointly with the right
end of the first rope.
The same information is then displayed by relying on the diagram in Figure 24.
Figure 24: The diagram of the pulley system in Larkin and Simon (1987)
According to the authors, the two representations of the pulley system − its
linguistic description and its diagram − are to be considered as informationally
equivalent, since they happen to have the same content: the statements in the
Behind the diagrams: cognitive issues and open problems 79
description express the same information as the one displayed by the arrange-
ment of the elements of the diagram. However, consider someone who has to
solve a problem about the pulley system; for example, she is asked what the
ratio of the second weight to the first would be when the system is in equilib-
rium. Of course, some extra information is needed: from the context, one knows
that the pulleys and ropes are to be considered as weightless and the pulleys
as frictionless; furthermore, the rope segments are all vertical, except when
they run over or under the pulleys. In any case, in order to solve the problem,
whoever reads the description without having been shown the diagram would
very spontaneously reach for paper and pencil and draw a diagram based on
the description, thus obtaining something similar to Figure 24. In fact, the
linguistic description and the diagram do not seem to be equivalent from a
computational point of view: it is much easier to extract information about the
pulley system by looking at its diagram than it is by reading its linguistic
description. Despite being informationally equivalent, the two representations
are therefore computationally different.
The question to ask then is which of the diagram’s features would make this
‘computational difference’. Larkin and Simon’s suggestion is that, differently
from the linguistic description, the diagram allows for the information at one
location or close to it to be simultaneously accessed and presented. For this
reason, the search for the relevant information takes shorter time, thus facilitat-
ing computation. Note that in some sense both the linguistic description and the
diagram externalize the information about the pulley system in space, i.e., they
put information ‘out there’ for the user to be available for reasoning; however,
the diagram clearly exploits space in a way that is for some reason much more
convenient to enhance inference. Furthermore, both representations must be
interpreted in order to be used: one has to understand to which element of the
pulley system the words in the sentences as well as the elements of the diagram
refer to; nonetheless, the diagram seems to facilitate not only the search for
the relevant objects but also the very recognition of the objects involved in the
reasoning.
Let me clarify at this point what I will intend by ‘diagram’ in this context.
For the moment, my assumption is that a diagram is a two-dimensional arrange-
ment of content. To some extent, also the linguistic description above has
diagrammatic elements, because it is given as a numbered list; however, the
fact that the sentences are given in an ordered sequence is in this case not
enough to make it computationally more advantageous than the diagram. Con-
sider now a simpler problem that still illustrates the distinction between infor-
mational equivalence and computational difference. Imagine someone working
in a Parisian RER station who wants to display on a signpost the hours of the
80 Valeria Giardino
trains going from Paris to Versailles from 9 to 11 a.m. during the week, so as to
prevent too many tourists to keep asking him. He could choose to write the
hours in the three following ways:
(1) 10:13 08:01 08:32 09:14 09:32 10:48 09:52 10:02 09:01 10:33 08:12
(2) 08:01 08:12 08:32 09:01 09:14 09:32 09:52 10:02 10:13 10:33 10:48
(3) 08:01 08:12 08:32

09:01 09:14 09:32 09:52
10:02 10:13 10:33 10:48
As anyone can see, (1) to (3) have the same content, and therefore they are infor-
mationally equivalent. However, from a computational point of view, (2) is better
than (1) for the reason that the hours in (2) are chronologically ordered while in
(1) they are randomly displayed; moreover, in (3) the content is not only chrono-
logically ordered but also arranged along three different horizontal lines, repre-
senting the time intervals 8–9 a.m, 9–10 a.m, and 10–11 a.m. respectively. It is
then evident that (3) would be the more appropriate information display to
avoid long queues of tourists not sure about which train they are supposed to
take. Why is it so? (3) is computationally more efficacious because of its spatial
configuration, which allows the user to know almost immediately where to look
for to find the relevant information: now it is 9.15 a.m., it is a Monday morning,
and I see that there are 2 more trains going from Paris to Versailles before 10 a.m.
This example shows even more strikingly than the previous one that an
appropriate spatial configuration can facilitate inference, even when the infor-
mation is given in the same format (in this case, by displaying numbers repre-
senting the train hours). If the pulley system case still seems to allude to a
distinction between visual and sentential reasoning by comparing a linguistic
description and a diagram having the same content, the timetable example
centers the discussion on one crucial feature characterizing diagram-like repre-
sentations: diagram-like representations externalize information in space in a
relevant fashion thus supporting reasoning and inference. Note that such facili-
tation may also result from the interaction between different formats. Consider
again the pulley system diagram and the crucial role played in it by labels:
letters and numbers (for example A, W1, . . .) serve there as indexes, because
they are placed in the appropriate spatial location so as to ‘name’ the various
relevant elements of the diagram. As Peirce has suggested, “The index asserts
nothing; it only says “There!” It takes hold of our eyes, as it were, and forcibly
directs them to a particular object, and there it stops” (Peirce 1885: 181). Being
symbolic, letters and numbers are not ‘diagrammatic’ in nature, at least not in
any intuitive meaning of the term; nonetheless, they assume here an indexical
function, that is, they guide the attention of the observer towards the relevant
objects in the diagram, and they can do that precisely because, as Larkin and
Simon claim, they are in the same location or ‘close to’ these very objects.
I hope to have already shown the limits of a sharp distinction between
sentences on the one hand and diagrams on the other. Other arguments can be
given as well to reject such a distinction. Take for example specific diagram-
matic practices, such as the use of diagrams in geometry, where something
analogous to what we have just seen in the pulley system example happens:
displaying the problem spatially helps recognizing the relevant objects and find
connections with previous background knowledge. Nonetheless, the case of
geometrical proofs and Euclidean proofs in particular is of course more complex.
As it has been thoroughly discussed, diagrams and text in Euclid are interdepen-
dent, in such a way that the latter gives instructions on how to interpret the
former (see for reference Netz 1999 and Manders 2008). As Netz explains,
The perceived diagram does not exhaust the geometrical object. This object is partly
defined by the text, e.g. metric properties are textually defined. But the properties of the
perceived diagram form a true subset of the real properties of the mathematical object.
This is why diagrams are good things to think with. (Netz 1999: 35, emphasis added)
For reasons of space, I won’t go much further into this issue here, but I have
done it elsewhere (Giardino forthcoming 2016).
Based on these examples, we can come to some tentative conclusions that
will be the starting point of this article. First, diagrams are widespread in many
different cultures because they amount to cognitive advantages, that is, they
happen to be very good inferential shortcuts in problem solving. By externalizing
the information in space, they reduce the amount of search required to find the
relevant information and solve the task. Second, diagrams cannot be considered
in isolation from a context they refer to, and therefore they are not only spatial
tools: the space of the diagrams is an interpreted space, in line with textual and
background knowledge. If this is true, then diagrams have a twofold nature and
possess both cognitive and cultural features: they certainly exploits our spatial
cognition, but the spatial elements that can be visually identified are then open
to contextual interpretation. As a consequence, it is pointless to consider them
as visual tools in opposition to language; instead, the more appropriate and
interesting question concerns their computational efficacy, and how it happens
to emerge from the interaction between more spontaneous abilities and the
production of cultural artifacts.
82 Valeria Giardino
In the rest of the article, my objective will be to sketch out the elements for a
conceptual framework explaining diagrams’ inferential powers. This will bring
me at first far from diagrams, and then back to a richer landscape involving not
only diagrams but also writing and notation, i.e., cognitive tools in general. In
Section 2, I will present the more general cognitive background. First, I will
introduce the view according to which most of human cognition is embodied,
in a sense to define, and then I will discuss how this feature of cognition relates
to the use of space in representing information. Such a framework would pro-
vide some insights on the cognitive reasons why the appeal to diagrams is
so spontaneous and widespread across cultures. In Section 3, I will focus on
another aspect characterizing diagrams that is at one time cognitive and cultural,
which is their relation to writing and more in general to written signs. In Section
4, I will present another crucial element that has been almost neglected in con-
sidering diagrammatic reasoning, which is the analysis of the actions that
are performed on the diagrams. Finally, in Section 5, I will propose a possible
‘big picture’ to offer as an account of diagrammatic reasoning, based on the
assumption that diagrams are kinesthetic tools open to change and manipula-
tion, and I will introduce what I consider as the typically human ability of
diagramming. As I will show, the consideration of the role of action in and on
the space of diagrams will be crucial in order to make sense of their computa-
tional efficacy.
2 The space of the diagram: insights from

cognitive science
2.1 Grounding cognition
In this section, instead of addressing diagrams directly, I will ‘zoom out’ from
them and consider what kind of theory about human cognition would be appro-
priate to defend the claim that diagrams are computationally effective for the
reason that they externalize information in space and consequently they can
invite the user to act on them.
In the recent philosophical literature, various hypotheses about the nature of
our mind have been put forward that are possibly in line with the claim that rea-
soning is heterogeneous and that humans spontaneously refer to external cogni-
tive tools such as diagrams in order to stack and work with various information.
For example, Varela, Thompson and Rosch (1992) have proposed that the mind is
embodied; in other words, human cognition would be determined by various

characteristics of our body. More recently, Clark and Chalmers (1998) have sug-
gested that the mind is extended, i.e., the objects in the environment may
assume functions that would make them a genuine part of our mind. Their
much discussed example is the case of Otto, an Alzheimer patient, who lives
in NYC; one day, Otto decides to visit the Museum of Modern Art on the
53th street and writes down on his notebook the directions to go there, because
he is afraid of getting lost. According to the authors, Otto’s notebook would
be an extension of Otto’s mind serving as his memory. However, despite being
very inspiring, the metaphors of the mind as embodied or extended are
unfortunately not sufficient when it comes to clarify in the practice how humans
were able to create and are able to rely on ‘scaffolding’ structures such as dia-
grams in order to enhance their reasoning and inferential capacities. The crucial
issue is to define more clearly the relation between these ‘extensions’ of the
mind and more spontaneous and precocious abilities such as vision or action.
If the body really matters, as the view of cognition as embodied suggests, then
it will be necessary to specify which of its characteristics have a role in cognition
as well as what the limits in considering cognition as based on the body are.
Note that I do not want to take up here any particular metaphysical stance about
the location of the mind in the environment. More modestly, I will claim, follow-
ing Hutchins (2001), that cognitive processes do not happen exclusively in the
brain but extend themselves beyond the skin and skull of an individual. This is
not to say that external objects replace mental processes, but rather that cogni-
tion is distributed between internal and external, at least in three different ways:
first, cognition is distributed with respect to a specific social group, sharing the
same jargon and the same background knowledge; second, it is distributed in
time, since the production of previous events transform the nature of sub-
sequent events; third – and more importantly for the present topic – cognition
is distributed in space: various cognitive processes rely on the coordination
between different external structures, which can be elements of the environment
as well as material artifacts specifically introduced for the purpose of enhancing
inferential powers. Once we accept the claim that cognition is distributed, it
remains to evaluate the possible role of the body in it: how much of my experi-
ence of having the body I have would count for the way in which I perform
various cognitive tasks?
On this topic, I will follow Goldman’s (2012) suggestion and assume a
moderate approach to embodied cognitive science. Consider proprioception and
kinesthetics, which refer in the first place to the perception of the position and
the movements of the different parts of our body. According to Goldman, em-
84 Valeria Giardino
pirical studies show that human cognition can in fact be considered for the most
part as embodied because information obtained from proprioception and kines-
thetics happens to be often reused for other more abstract tasks. The empirical
part of Goldman’s claim is to assume the existence of so-called bodily represen-
tational codes, a subset of mental codes that are primarily or fundamentally
applied in forming interoceptive or directive representations of one’s own bodily
states and activities (see for reference Goldman and Vignemont 2009). The phil-
osophical part of his claim is to consider that the brain reuses or redeploys cog-
nitive processes having different original uses to solve new tasks in new con-
texts; when it comes to bodily representational codes, these appear to be
extremely pervasive. Goldman’s proposal leads to a “moderate” conception of
embodiment-oriented cognitive science precisely because it specifies the role of
the body in cognition: first, by defining what bodily representational codes are
and then by explaining how they happen to have an influence on some
cognitive processes. We will see later in Section 5 that these considerations are
crucial to consider the relevance of performing actions on diagrams and to
explain the cognitive advantages of their use in some particular tasks.
Other approaches in cognitive science may be of interest. Barsalou (1999),
for example, has focused on the question about the influence that perception
seems to have on cognition. In his reconstruction of human cognition, he
observes that symbolic operations can do many things and are in fact essential
for intelligent activity. By symbolic operations, he intends things such as bind-
ing types to tokens or arguments to values, drawing inductive inferences from
category knowledge, predicating properties and relations of individuals, com-
bining symbols to form complex symbolic expressions, or representing abstract
concepts that interpret metacognitive states. If this is the case, then what cogni-
tive mechanisms underlie these operations?
Standard theories in cognitive science have claimed that core knowledge
representations in cognition are amodal data structures that get processed inde-
pendently of the brain’s modal systems for perception (e.g. vision, audition),
action (e.g. movement, proprioception) and introspection (e.g. internal percep-
tion of motivational states, affective states, goals, beliefs, cognitive operations,
meta-cognition and so forth). According to Barsalou, the somewhat paradoxical
result of the Cognitive Revolution of the 50s has been to ban imagery from
psychology as not sufficiently scientific. Interestingly, in his view, this attitude
was mainly due to the major developments that were taking place at the same
time in logic, linguistic, statistics and programming languages. Mental images
were replaced by amodal language-like symbols deriving from predicate calculus
and propositional logic: the idea was that such symbols are transduced from
experience to mentally represent knowledge, and once they are established in

the brain, they can represent knowledge about categories across a wide range
of cognitive tasks. Barsalou’s claim is that this classical view of cognition,
despite surely having a strong explanatory power in accounting, for example,
for the role of language in human cognition, suffers from three flaws. First, it is
not sustained by empirical evidence; second, it cannot solve the so-called
grounding problem, that is the question about the interface between cognition
on the one hand and perception and action on the other; third, it is not clear
how the information would be stored in the brain in a way that is consistent
with neural principles of computation. For these reasons, Barsalou suggests
that we should think of cognition as grounded: in his view, cognition is under-
lined by modal simulations, situated action and occasionally bodily states.
Barsalou chooses the term “grounded” cognition and renounces to the label
“embodied”, because he does not want to produce the mistaken assumption
that to reject the standard theories of cognition means necessarily to claim that
bodily states are necessary for cognition and exclusively to focus on them: there
are cases in which cognition proceeds independently of the body, and many
researchers address other forms of grounding. Moreover, considering cognition
as grounded does not amount to rejecting the standard theories as irrelevant:
classic symbolic operations are still part of cognition, but one needs to under-
stand their possible implementation in the brain. Barsalou’s proposal is then
to consider cognition as inherently perceptual, i.e., as sharing systems with
perception at both the cognitive and the neural level. Cognition would be based
on Perceptual Symbol Systems, which are modal and analogical and, more
interestingly, are not holistic and static representations but not compositional
dynamic representations. A perceptual symbol contains only a schematic aspect
and it is multimodal. In Barsalou’s view, the dissociation perception/cognition is
artificial and invalid: perception and cognition share common neural systems,
and therefore they function simultaneously and cannot be divorced. If on the
one hand Goldman’s view helps understand what the role of the body could be
in cognition, Barsalou offers a way of explaining why cognitive external tools
such as diagrams can be good scaffolding structures for reasoning. However,
more discussion is needed in order to assess the scope and the consequences
of his suggestions.
On this point, let me mention briefly that very recently Stjernfelt (2014) has
criticized Barsalou’s view as an example of contemporary “psychologism”.
Despite the fact that it is interesting and promising to think of symbols as flexible,
dynamic and multi-modal, it is not that clear how abstractions such as categorical
knowledge is obtained within such a framework. A problem that has been
86 Valeria Giardino
commonly recognized in Barsalou’s proposal is that perception symbol theory

seems to be able to explain at most a rather small subset of abstractions, while
it is not obvious how it can make more complex concepts emerge, such as
for example scientific concepts or even the intersubjectivity of language. For
these reasons, in Stjernfelt’s view, Barsalou’s theory would be open to the usual
criticisms that have been traditionally put forward against other similar em-
piricist theories of symbols.
For reasons of space, I will not settle this issue here but only suggest that
the consideration of cognition as both “moderately embodied” and “grounded”
gives us some crucial elements for defining empirical and philosophical argu-
ments in favor of the idea that (1) diagrams are kinesthetic objects and (2) they
are genuine elements of our reasoning. Of course, more work needs to be done
in order to assess the plausibility of both these theses taken in isolation and of
their possible compatibility.
2.2 Space to reason

Let me now move one step back towards diagrams and consider a more specific
theory about the role of imagery and space in reasoning. As it is well known, in
the 1980s Johnson-Liard (1983) proposed a theory of human reasoning that was
based on mental models. According to his theory, human reasoning is a mental
simulation process in which models of the premises are constructed, inspected,
and validated. In a very recent book, Knauff (2013) has proposed what he defines
the space to reason theory of human reasoning, sharing essential assumptions
with Johnson-Liard but making more clear-cut hypotheses regarding the format
in which the models would be represented in the mind. In Knauff’s definition,
mental models are non-propositional, transcend language and are spatial in
nature. Finally and most importantly, they are definitely different from visual
images. This is a crucial issue in Knauff’s theory that relates to the discussion
in the Introduction. Visual images are not the basis of reasoning; on the con-
trary, in some cases, they can even impede the process of inference. Compatibly
with what already discussed relatively to Barsalou’s theory, Knauff as well suggests
that reasoning is based on supramodal spatial representations and proposes
a spatial theory of human reasoning involving spatial layout methods. Going
beyond the opposition between visual and sentential accounts, Knauff explores
what he calls a “third way”, according to which supramodal spatial representa-
tions are at the heart of human thought, and most importantly they are used
even to think about non-spatial relations in the world. These relations are effec-
tive to enhance inference precisely because on the one hand they are more
abstract than visual images and on the other hand they are more concrete than
propositional representations.
The interesting point in Knauff’s general framework is that he wants to
move against a set of dichotomies that he considers dangerous. As he explains,
one of his motivations is to overcome two common tendencies in cognitive
science: reductionism, claiming that unveiling the functioning of the brain
would be sufficient to understand the mind, and mentalism, endorsing the oppo-
site, that is that brain’s mechanisms would not have any role in explaining the
functioning of the mind. Knauff’s long-term objective is to give a contribution
to the study of human rationality, “barely scratching the surface” of this phe-
nomenon (Knauff 2013: 222): in his view, our rationality depends on the number
of models that we actually consider in reasoning, and this number varies accord-
ing to prior knowledge, previous beliefs, and one’s working memory capacity.
Note that Knauff’s theory has recently been considered in relation to Eucli-
dean diagrammatic reasoning. According to Hamami and Mumma (2013), a
cognitive account of Euclidean diagrammatic reasoning would be compatible
with Knauff’s framework. In fact, as it has been already mentioned, Euclidean
diagrammatic reasoning is not supported by visual images alone since one
needs to postulate the cognitive representations of the spatial information con-
tained in the premises of Euclidean diagrammatic inferences: it is the space of
the diagram that matters, not its visual appearance.
3 Diagrams are a cognitive technology

3.1 The power of writing
In the previous section, I presented some lines of research in cognitive science
that could be offered as elements for a theory about the features and functioning
of diagrams. Up to this point, I have considered how space and in particular the
space of our body may matter not only to orient ourselves in the environment
and interact with it, but also to enhance our inferential powers. The claim is
that our spatial cognition, together with proprioception and kinesthetics, are
recruited and reused in tasks that are specifically cognitive and therefore different
from the original ones in which they are employed and have most likely evolved.
There are therefore reasons to believe that we spontaneously refer to diagrams
because they are optimal tools exploiting space in a relevant matter and thus
creating scaffolding structures for our thoughts.
88 Valeria Giardino
Now I would like to focus on another feature characterizing diagrams. As

I said in the Introduction, diagrams are very sophisticated artifacts: on the
one hand, they rely on our “nature”, that is on our more precocious cognitive
abilities; on the other hand, they are the product of our “culture”. Seen from
this point of view, diagrams can be considered as ‘traces’: they are drawn on a
piece of paper or, nowadays, very often shown on a digital screen. The fact that
diagrams are traces allow us to inspect, explore and possibly change them, and
then inspect and explore them again. It is interesting then to ‘zoom out’ again
from diagrams, and consider this time a crucial aspect of human culture, that
is the role of writing intended as leaving (meaningful) traces. Moreover, I will
focus on a particular form of writing that is mathematical writing, by presenting
some recent works on the cognitive role of formal languages and notations. We
will see how a theory about diagrams cannot avoid to refer to signs in general,
in particular in the case in mathematics.
The introduction of writing has changed the character of our culture forever,
and from this point of view it can be considered as a real cognitive revolution.
As it has been proposed in anthropology, most of the aspects of linguistic
communication in written cultures are heavily influenced and in some cases
even determined by writing or by a variety of graphic representations, diagrams
being among them (Goody 2000). In Daniel Defoe’s Robinson Crusoe, once the
protagonist finds pen, ink and paper, he finally begins “keeping things very
exact”, which means above all telling the time. As Goody suggests by referring
to this example, time is measured, reckoned and even conceived in graphic
terms, and therefore it is given a visual and – more interestingly for our purposes
here – a spatial counterpart: “oral discourse takes place in time, written discourse
in space and time; the latter is seen, the former unseen” (Goody 2000: 65). More-
over, once we have pen, ink and paper, we can do even better than simply
writing down markers that mimics the cuts that we might have carved on a piece
of wood; we can shift to “abstract” markers, like letters or numerals. This is a
crucial improvement, both for science and for daily life, and will influence not
only the content of our exchanges but their very structure. Although these
changes are not directly due to writing per se, “they are dependent upon graphic
or conceptual lines being drawn (and the drawing of lines is a basic graphic
accomplishment)” (Goody 2000: 78). The crucial point here is to consider writing
as a cognitive technology, having a particular influence on our communication: it
not only changes they way in which we communicate, but also the very nature
of what we communicate, to the others and also to ourselves. Goody points out
that in writing there is always a visual component, but also for him this visual
component amounts to the presence of a graphical and spatial element. Writing
allows us to impose an order to things, and in most cases, as for example in the
timetable example given in the Introduction, this order is in fact cognitively

relevant. As Goody sums up, the mathematical table “is essentially the product
of writing, but one that can be taught to and learned by those who can neither
read nor write. Yet it provides those who use it with a special cognitive tool, a
technology of the intellect” (Goody 2000: 148). The “intellect”, we know it from
cognitive science, can rely on space for cognitive tasks; for this reason, it creates
new technologies such as a diagram or a table to the aim of improving its
powers.
In the previous sections, I gave an idea of what the source for these cogni-
tive technologies can be, that is our grounded mind and our ability to reuse
the space of the body and the space of the environment as tools to enhance our
reasoning. Now I will ‘zoom in’ and see how these cognitive technologies can
have an influence on our reasoning abilities in practice, so as to add another
element to the consideration of diagrams’ computational advantages.
3.2 Operative writing, notation and diagrams

In a recent and inspiring book, Dutilh Novaes (2012) offers an original analysis
of the role of formal languages in logic. Her motivation is to embrace a wider
conception of formal languages that would allow going beyond their alleged
nature of mathematical objects and investigating instead their main function:
being a tool for thought. She starts from the question about what exactly goes
on when someone uses formal languages. To reply to this question, she takes
into account both the history of notations in logic and the empirical results
that supports the view that cognition is extended. For this point of view, we
are clearly in the same line of thought, despite the fact that, being interested
in logic in particular, she specifically focuses on the literature in psychology of
reasoning while, for reasons that should be clear by now, I have considered
as more relevant studies about mental imagery and spatial representations in
reasoning. Also for Dutilh Novaes, endorsing an extended view of the mind
allows renewing the attention on the interactions between the subject who
reasons and the elements of the environment that he or she makes use of. Again,
to assume such an attitude does not necessarily imply to be forced to take
any particular metaphysical stance about where the mind would or should be.
Dutilh Novaes’ claim is that formal languages should be “viewed as cognitive
artifacts enhancing and modifying an agent’s reasoning processes” (Dutilh
Novaes 2012: 2): they are indispensable as calculative and computing devices
and they crucially rely on sensorimotor processing. As she explains, her main
90 Valeria Giardino
goal is to consider historical as well as empirical data to the aim of reaching

the conclusion that “a formalism is a powerful technology that allows humans
to reason in ways that would be otherwise virtually beyond their reach” (Dutilh
Novaes 2012: 3). We should by now see the connection between the issues
discussed in the previous sections and the ones that will be later introduced. I
started from diagrams, took a step back to consider human cognition in general,
and then I have pointed out that, thanks to writing, also other cognitive technol-
ogies have been created and they too, like diagrams, are useful computational
devices.
In order to support her claim, Dutilh Novaes, among other things, refers to
the notion of the formal as de-semantification, which has been proposed by
Krämer (2003). In Krämer’s view, formalism is in most cases computationally
convenient because it allows its user to remove or disregard from it the semantic
dimension of written signs and to treat them as ‘meaningless’. For this reason,
formal language is a particular form of writing: operative writing. In other words,
it is a tool that makes manipulations and mechanic calculations possible. As
she explains, “operative writing is not only a tool for describing, but also a
tool for cognizing, a technique for thinking that enhances intelligence” (Krämer
2003: 534, emphasis added). Note that Krämer points out as well that writing
has a fundamental iconic nature, since it takes place in space, typically on two-
dimensional surfaces; furthermore, Duthil Novaes discusses in details how the
iconicity of writing would be a crucial element to understand the history and
development of formal notation in logic. Could it then be possible to think that
all forms of writing as to some extent potentially ‘diagrammatic’ in the sense
defined in the Introduction?
More recently, Krämer (2014) took up the notion of operative writing again
by connecting it more closely to diagrammatic reasoning. According to her, in
the “diagrammatic”, “‘saying’ and ‘showing’ work together to create ‘operative
iconicity’” (Krämer 2014: 3; see also Krämer in this volume). In her view, we do
not merely think with paper, but we think on it. The epistemic use of inscribed
and illustrated surfaces is a cultural achievement of the highest order, and there
are reasons to believe that the invention of the surface has to be considered “for
the mobility and creativity of the mind as the invention of wheel was for the
mobility and creativity of the body” (Krämer 2014: 7). In line with what we have
discussed so far, this is due in particular to spatiality; even more interestingly,
Krämer distinguishes between two kinds of spaces of the inscriptions that are
interacting all the time: the structural space and the movement space. To clarify,
in a diagram we have on the one hand places and relations between them,
presented synchronically, and on the other hand a space on which the user can
carry actions in time. In Krämer’s framework, diagrams are so effective in reason-

ing because they put together spatiality and our “cartographic impulse”: one
draws a structural space to create a movement space, “to open up a complex
terrain to the possibility of movement and action” (Krämer 2014: 13–14). Here,
an element emerges that will become crucial for the ‘big picture’ I shall propose
later: action on the diagram.
These considerations are of course very relevant for mathematics. Macbeth
(2012) as well has expressed a similar view in favor of the “paper-and-pencil
reasoning in the mathematical practice”. According to her, signs in mathematics
are not used merely to record results; by contrast, their main function is to
embody the relevant bits of mathematical reasoning. In her words, “they put
the reasoning itself before our eyes in a way that is simply impossible in written
natural language” (Macbeth 2012: 60). As we saw in the example in the Intro-
duction, the diagram puts the very reasoning about the pulley system before
our eyes. Using Macbeth’s terminology, notation shows how the reasoning goes,
and this facilitates calculation. Her claim is that mathematical signs, such as
for example a Euclidean diagram, do not only picture something but formulate
the content of something “in a mathematically tractable way, in a way that ena-
bles reasoning in the system of signs” (Macbeth 2012: 63). If this is true, then we
can also make further distinctions, for example by considering two kinds of
diagrammatic reasoning. On the one hand, the mathematician reasons intra-
configurationally, as in some picture proofs such as Euler and Venn diagrams
and in general in Euclidean practice: he or she ‘stays in the diagram’ and iden-
tifies different possible configurations in it. In this case, there is only one
diagram, but it can be ‘seen’, i.e., spatially organized, in a different fashion. On
the other hand, the mathematician can also reason trans-configurationally, that
is by relying on new writing or drawing of new graphic elements, which is
what happens with notations in arithmetic, algebra and knot theory. It is impor-
tant to note that such a distinction concerns of course the use one can make of a
representational system and not the system as such. Again, as mentioned in the
Introduction, there is nothing in the format per se that may allow us to define it
as diagrammatic: what matters is the way in which such a format is used in a
particular problem-solving context.
In a recent introduction to the philosophy of mathematics, Colyvan (2012)
devotes one whole (but unfortunately brief) chapter to notation, and adds a
further aspect about mathematical signs that is worth noticing. According to
him, a “good” notation is far from being trivial, since it would not only facilitate
calculation but also play a role in prompting new ideas and inducing new
developments. He takes the example of topology, where a good notation
92 Valeria Giardino
composed of squares with arrows drawn on them indicating the gluings i.e.
the homeomorphisms would be able to make new and in a sense unexpected
objects such as the Klein bottle emerge. Such algebraic topology notation has
characteristics that are typical of algebra, because it is a calculation device,
but also of geometry, because it gives the instructions for constructing the
object. For this reason, “we have a powerful piece of notation here that does
some genuine mathematical work for us” (Colyvan 2012: 139, emphasis added).
A good notation facilitates the economization of cognitive resources, calcula-
tion, possible advances in mathematics and it also provides more perspicuous
mathematical explanations. Notation or, to go back to Krämer’s terminology,
operative writing, is thus a very relevant aspect of the mathematical practice
that has been unfortunately neglected by the standard views in philosophy of
mathematics. One reason among others could have been that too much attention
was put on the supposed descriptive function of mathematical signs as opposed
to their creative function, as Rotman (2000) has proposed. This has obscured
their crucial importance in the practice of mathematics. We can think of building
a “semiotics” for mathematics: in Rotman’s view thoughts and scribbles are
mutually constitutive, and “mathematicians at the same time think their
scribbles and scribble their thoughts” (Rotman 2000: 35). This is again related
to writing, of which we have already seen the constitutive nature; however,
the standard approaches to mathematics such as formalism, Platonism and
intuitionism, have failed to acknowledge “the intricate interplay of imagining
and symbolizing, familiar on an everyday basis to mathematicians within their
practice” (Rotman 2000: 48): mathematics is first of all “a certain kind of traffic
with symbols, a written discourse” (Rotman 2000: 51).
If this is the right picture, then diagrams in particular, be them drawn or
simply imagined, are:
the work of the body; they are created and maintained as entities and attain significance
only in relation to human visual-kinetic presence, only in relation to our experience of the
culturally inflected world. As such, they not only introduce the historical contingency
inherent to all cultural activity but also, more to the present point, call attention to the
materiality of all signs and of the corporeality of those who manipulate them” (Rotman
2000: 57).
For Rotman, and in line with what I have claimed so far, the attitude typical of
the standard approaches of avoiding diagrams “like the plague” (Rotman 2000:
56) has ended up in overshadowing the importance for mathematics of things
such as materiality, embodiment, corporality, as well as its historical and social
situatedness.
Our journey has brought us from formal languages to operative forms of

writings – typically iconic – and then to notation, and through notation to
diagrams again. In the next section, I will focus on the last element characteriz-
ing my ‘big picture’ that I have not yet addressed in detail: the actions that are
performed on the diagram.
4 The action on the diagram: insights from

embodied mathematics
4.1 Gestures embodying mathematics
Some proposals have been put forward to consider not only cognition but in
particular mathematics as embodied. According to the standard approaches,
mathematics is an abstract science, dealing with abstract objects described by
the signs that have nothing to do with ordinary objects we interact with every
day. However, some recent studies seem to question the existence of a sharp
distinction between abstract knowledge on the one hand and concrete world
on the other. Even when working with abstract objects, we seem to recruit
some competences and abilities that are typical of our relation with the world.
Again, I am endorsing here a “moderate embodied” approach to cognition.
Consider for example the work of Lakoff and Núñez (2000). In their view,
abstract mathematical concepts are rooted in embodied activities, such as for
example our ways of thinking about the world and describe it. In line with
previous researches on language (see for reference Lakoff and Johnson, 1980),
the authors propose that abstract scientific concepts as well as ordinary ones
can be reformulated in terms of metaphors, which would in fact be fundamental
for understanding mathematics. If one thinks of the typical mathematical jargon,
it is true that it contains many terms alluding to our relationship with the real
world: natural numbers “grow” indefinitely, points “lay” on a line, functions
“move” to zero. Conceptual metaphors and conceptual blending connecting
two different domains so as to obtain a third one being a projection of both,
would then be the main cognitive mechanisms to conceptualize mathematical
objects. The basis of these metaphors is bodily experience and they are not a
mere linguistic phenomenon, but crucial elements of thought. Mathematics is
thus embodied because it is possible to understand and explain it by making
appeal to embodied cognitive mechanisms, of which conceptual metaphor is
the main one.
94 Valeria Giardino
Recently, recalling Lakoff and Núñez’ work, Sinclair and Gol Tabaghi (2010)
have claimed that, both to improve mathematical education and to define mathe-
matical understanding, it is necessary to provide a detailed analysis of the
use that mathematicians make of many heterogeneous representations: speech,
gestures, diagrams, and so on. To this aim, they have interviewed six mathema-
ticians and asked them to explain the meaning of the mathematical concept
of “eigenvector”; these interviews were filmed, to the aim of evaluating the
mathematicians’ embodied way of reasoning. The videos show that the six
interviewed mathematicians make use of various representations, moving from
one to the other without difficulties, thus blurring the alleged boarders between
the mathematical and the physical world. They all come from different disciplinary
domains – pure and applied – ranging from number theory to discrete applied
mathematics and they all well know the formal “manual” definition of eigen-
vector, which is single, atemporal and static; nonetheless, with no exception,
they offer a description of eigenvectors that alludes to a very different interpre-
tation, including also temporal and kinesthetic elements, as shown both in the
terms and in the gestures they use. Metaphors strike back. Some of the mathe-
maticians focus on the transformations of the vectors: for example, they say and
show in their gestures how the vectors “shrink” or “turn”; others describe the
vectors’ personality: they “go in the same direction”, they “align”, and so on.
In other words, none of the mathematicians speaks of eigenvectors only in terms
of algebraic equalities. Moreover, some metaphors are perceptual (for example,
one mathematician thinks of the quadratic function as a “goblet”) but most
of the metaphors used have a movement component. Furthermore, since the
mathematicians come from different specializations, there is high variability in
their gestures that seems to depend on their respective education and com-
petence, i.e., on their relevant background knowledge. The authors of the study
conclude that gestures, compared to language, would give even more possibilities
than simple speech to express continuity, time and movement, thus confirming
the intuition of the French mathematician Châtelet (1993) who saw in a mathe-
matical diagram the “crystallization” of a gesture.
In this example, we have seen that gestures, which are actions, can be used
to represent abstract objects and to talk about them. Therefore, they can embody
the reasoning about them, thus being close to the diagrams and notations that
were discussed in the previous sections. Nonetheless, gestures are volatile,
while diagrams and notations are not: they do not leave traces, as it happens
when drawing a diagram or writing an equation. Moreover, gestures are not
artifacts, at least not in any intuitive meaning of the term. However, it is true
that gestures, at least the ones of the mathematicians interviewed in the
reported study, are produced from within a written culture. As Goody explains,
memory is surely different in oral and written cultures; if this is true, then it
would be possible to think that also spontaneous gesture can be affected by
writing. More work needs to be done on the relation between gesture and
writing, in particular in specific contexts such as mathematics, where writing
plays a crucial role. For the moment, let us focus our attention on actions, and
consider if they can still play a role when we take into account the way we deal
with mathematical signs. Once more, this would give some plausibility to the
moderate embodied approach: as we will see, human beings seem to be able to
recruit not only space but also action for new tasks that are different from the
original ones for which space and action are used and have a specific cognitive
nature.
4.2 Epistemic actions and kinesthetic diagrams

In the discussion so far, I have shown that diagrams exploit space in a relevant
way and they can be considered as ‘traces’. Both these aspects seem to enhance
inference and reasoning. Moreover, in the previous section, I have presented a
first way in which actions and diagrams can be connected, through the body
and in particular in gestures. In this section, I will focus on the role of action
on the diagrams, both the action of constructing a diagram and of manipulating
it in line with a specific practice. As Nyíri (2014: Ch. 1) has suggested in a recent
book, there is in fact an essential connection between the visual on the one
hand and the motor and tactile on the other.
Let us consider another definition coming from cognitive science that is
useful for characterizing the kind of actions that we will be dealing with when
considering diagrammatic reasoning: epistemic actions. To illustrate this notion,
Kirsh and Maglio (1994) famously took as an example the simple case of Tetris, a
popular videogame from the 1980s. In Tetris, some pieces that are shaped as
geometric figures and composed of four squared blocks fall one after the other
down the playing field. The objective of the game is to create a horizontal line
below, composed of ten blocks presenting no gaps. Once a whole line is created,
it disappears, and any block that is above the deleted line will fall down.
The game is over if the stack of pieces reaches the top of the playing field; by
contrast, if a certain number of lines is cleared, then the game enters a new level
(in which the pieces will go down faster). In order to create the line below to the
aim of having it cleared, the player can move the pieces sideways and rotate
them by 90-degree units. The experimenters’ observation is that Tetris players
act on the pieces of the game not only to directly achieve the result of arranging
them below in a line, but also to understand what the better move will be. In
96 Valeria Giardino
fact, they rotate and move them around to the aim of testing different alternative
solutions before letting them fall below. On this basis, the authors propose to
distinguish between pragmatic and epistemic actions: the aim of the actions of
the first kind is to bring the agent closer to his or her physical goal (in this
case, the one of creating a line without gaps); actions of the second kind instead
“use world to improve cognition” (Kirsh and Maglio 1994: 513), i.e. they are
“physical actions that make mental computation easier” (Kirsh and Maglio
1994: 513). An epistemic action is performed outside on the physical objects that
are available, and it is precisely the performance of this action that enhances
our inferential capacities. Note here again that the authors talk about facilitating
reasoning through actions on external tools.
Despite the obvious limits of the Tetris case in relation to more sophisticated
cognitive tasks, the concept of epistemic action can be extended to other more
complex contexts, for example to the use of signs in mathematics. As De Cruz
and De Smedt (2013) have recently proposed, mathematical symbols “enable us
to perform mathematical operations that we would not be able to do in the mind
alone, they are epistemic actions” (De Cruz and De Smedt 2013: 4). Of course,
such a view is not epistemologically innocuous, since it assumes once again
that mathematical signs are intimately linked to the concepts they represent –
or, in Macbeth’s view, of which they formulate the content – and vice versa.
Recently, together with De Toffoli, we have considered the practice of proof
in topology, in particular in knot theory and in low dimensional topology (De
Toffoli and Giardino 2014, 2015). We have proposed an interpretation of knot
diagrams as dynamic tools: in perceiving a diagram, one has to see the possible
moves that can be applied to it. For this reason, experts have developed a
specific form of enhanced manipulative imagination, which allows them to
draw inferences from knot diagrams by imagining and in some cases actually
performing epistemic actions on them. That is why, as Macbeth suggests, knot
theory implies a form of re-writing. Also the actual practice of proving in low-
dimensional topology involves a kind of reasoning that cannot be reduced to
formal statements without loss of intuition. The representations that are com-
monly used in low-dimensional topology are heterogeneous, being neither
entirely propositional nor entirely visual. This form of reasoning is shared by
experts: it is the kind of reasoning that one has to master to become a practi-
tioner. Moreover, the manipulations allowed on the representations as well as
the representations themselves are epistemologically relevant, because they are
integral parts both of the reasoning and of the justification provided. As a case-
study, we took Rolfsen’s demonstration of the equivalence of two presentations
of the Poincaré homology sphere (see, for reference, Rolfsen 1976) and explained
how most of the steps into which the proof can be broken down are transforma-
tions that lead from one picture to another, as clarified by the text giving the
instructions on how to interpret the pictures. Therefore, the representations give
a material form to the transformations (and in this sense they embody them),
thus allowing experts to perform epistemic actions (and in particular to draw
inferences) on them. These actions are controlled by the shared practice: the
set of legitimate transformations is limited and determined by the context.
As Larvor (2012) explained in a recent article by comparing different forms of
mathematical arguments:
if an argument includes an inferential action that manifests or manipulates the subject

matter, or a representation thereof, then formalising this argument in a general logical
language must either misrepresent or fail to include this action. Moreover, we can say
something in the direction of explaining how informal arguments work as arguments:
they are rigorous if they conform to the controls on permissible actions in that domain
(Larvor 2012: 724, emphasis added).
Inferences involving the manipulation of pictures and diagrams are permissible

only within a specific practice and in this sense they are context dependent: this
leads to the establishment of local criteria of validity. We see a difference with
gestures: instead of being idiosyncratic, diagrams and pictures are here part of
a solid and stable practice to which they refer.
I have now presented a certain number of elements that are necessary to
sketch a conceptual framework about diagrammatic reasoning. In the next and
final section, I will introduce my hypothesis for a ‘big picture’ about diagrams
encompassing all these various components and explaining the computational
advantages of thinking with diagrams and analogous cognitive artifacts.
5 The big picture: the diagramming ability

In the previous sections, I have discussed the role of space, writing and action in
relation to diagrams’ characteristics and functioning. Diagrams as well as other
kinds of cognitive tools can serve as convenient inferential shortcuts if the space
they display is correctly interpreted and permissible actions are performed on it.
If this is true, then I will propose a ‘big picture’ that will encompass all these
various features, based on the existence of a typically human ability that I label
diagramming.
Let me mention that Tversky (2015) has recently put forward a similar view
about human reasoning by referring to what she defines spraction: her project is
98 Valeria Giardino
to draw “connections between material representations of thought and ethereal

representations of thought created by gestures. Both are formed by the actions
of hands in space, so parallels abound.” (Tversky 2015: 100). Action in space
can be integrated with the abstractions that material representations create on
a page and gestures create in the air, thus developing the concept of spraction.
It is possible to begin historically and take into account external representations
of things in the wild. As for Goody’s Robinson Crusoe, human beings started
with ‘natural’ marks, but then created more and more sophisticated cognitive
artifacts. In Tversky’s view, spraction can be investigated by applying two
research paradigms, one coming from linguistics and the other from psychology:
evidence in linguistic is based on the analysis of the cognitive tools that are
produced across cultures and schematize or design marks in space in similar
ways; evidence in psychology is instead found in what can be called empirical
semiotics, that is “experiments examining regularities in the ways people create,
interpret, and use various external representations of meaning” (Tversky 2015:
100, emphasis added).
In the same spirit, the framework I propose aims to explain the reasons why
the numerous representations that I have discussed so far are commonly used to
externalize thought. First, I claimed that cognitive tools – diagrams of course,
but also other cognitive artifacts – are widespread because they amount to
cognitive advantages. These advantages are due to the way in which such
artifacts (i) exploit our cognition of space, (ii) are a form of writing and (iii) can
be subject to change and manipulation. Given all these elements, my conjecture
is that the cognitive advantages in using these artifacts depend on humans’
complex cognitive architecture, and in particular on the ability that I have called
diagramming (Giardino 2014). By diagramming, humans recruit several systems
that are already available for perception or action – such as the visuo-spatial
system, the conceptual system, and the motor system – and establish an external
connection between them, by means of a particular cognitive tool, the role of
which is to trigger such a connection thus enhancing inference and reason-
ing. The manipulative imagination that is used in topology – and possibly in
other areas of mathematics – would then be one instance of diagramming. My
hypothesis is that most of the systems of representation that are introduced in
mathematics and science in general are supposed to serve dynamically as spatial
interfaces where a connection among different cognitive systems is established
and experiments are performed. Moreover, diagramming can even be the source
of gestures, in particular spontaneous gestures that are performed not only to
communicate content but also ‘for oneself ’, namely to reason: it makes sense to
intend physical objects such as maps or tables as a sort of ‘crystallized’ gesture.
For example, we have recently shown that gestures can improve memory and
facilitate inferences about the features of an environment (Jamalian et al. 2013).
A small text illustrates the environments in two different alternative ways: as
in a topographical map (in a bird’s eye view), which is, in Krämer’s terminology,
the model for structural space, and as in a participant-oriented description of a
series of movements (“enter the room”, “go left”, “you see in front of you”, etc.),
which are instead the prototypical representational modality for movement
space. Recall that according to Krämer, thinking and understanding are in fact
inherently spatial. My suggestion is that even if we do not want to endorse
such a strong claim, we can accept that the power of cognitive tools resides in
their being an occasion for various cognitive systems to work concertedly. The
hypothesis about the existence of diagramming is in line both with the approach
of the moderate embodied cognition, according to which some abilities are re-
cruited for new tasks, and with the space to reason theory. Moreover, diagram-
ming and writing are clearly connected, in a non-trivial way: in fact, diagramming
seems to be the condition for the introduction of writing, but at the same time,
writing, once developed, can shape our forms of diagramming, as I have dis-
cussed above. The framework I propose will also be able to clarify how impor-
tant mathematics is from a cognitive point of view: in mathematics, we have
writing, we have manipulations and operations, we have space plus action,
all this in continuous interaction with the definition and the formulation of
concepts. By doing mathematics, we explore our ability of diagramming and we
invent new spaces, new objects to embed in them and new actions to perform
in and on the available representations
To sum up, I have first pointed out that, instead of assuming a sharp
distinction between visual and sentential thinking, it is more interesting to
differentiate two representations on the basis of their informational equivalence
and their computational differences. I have claimed that diagrams are in most
cases computationally advantageous in comparison to language, because our
cognition is (moderately) embodied and space can have a cognitive representa-
tional role. Moreover, diagrams are also ‘written’ objects, which are traced on a
piece of paper or shown on a computer screen. This brought me to the investiga-
tion of writing and its cognitive and cultural relevance. Finally, I have con-
sidered the role of action in diagrammatic reasoning, both in gestures and in
epistemic actions on mathematical signs, and proposed that diagrams’ function-
ing depends on diagramming, which is the typically human ability of facilitating
the interaction of different cognitive systems by means of some particular cogni-
tive tool, so as to enhance inference and reasoning.
As I said in my Introduction, the aim of this article was not to provide a full-
fledged theory on diagrammatic reasoning, but to sketch out the elements that
100 Valeria Giardino
are, in my view, necessary to take into consideration if the aim is to give a

comprehensive account to diagrams. Of course, much more work needs to be
done, both to better define diagramming and to solve the major problems that
are still open, which I will consider in turn in this final section. In some sense,
the definition of diagramming is more a starting point for new exciting research
than a conclusive result: many aspects of diagramming must still be investigated
in depth.
First, it would be necessary to clarify the role of the sensorimotor in dia-
gramming. Diagramming is based on an interconnection between spatial cogni-
tion, action and background knowledge. How does the body affect the possible
actions that are performed? Does perception have an influence on the shape of
the cognitive artifacts we create? Would different creatures with a different body
and/or living in a different environment create a different mathematics and a
different science? What are the importance and the limits of the consideration
of the sensorimotor in human cognition? Second, once this non-standard frame-
work for diagrammatic reasoning is accepted, then the question of the relation
between content and veridicality should be reformulated. Third, when talking
about permissible actions I alluded to a community of practitioners, but in a
very abstract fashion. It would be instead very interesting to consider not only
the cognitive/individual part of the story to tell about diagrams, but also the
cognitive/social one. As I said in the Introduction, diagrams are the product of
our culture: we use them to communicate, share ideas, we experiment on them
and test our hypotheses, by ‘reading off’ their consequences on them. Therefore,
once diagramming is more precisely defined, it will be necessary to consider
how it modulates in the encounter with other diagramming people. For example,
how important is it to acknowledge the intentions behind a diagram? If a diagram
has been designed for a specific task, shouldn’t it be necessary to recognize the
designer’s intention in order to correctly interpret its spatial structure? And if
acknowledging intentions is crucial for interpreting a cognitive artifact, would
people with deficits in theory of mind such as autistic individuals be able to
understand and use some artifact like a map? Moreover, if two people happen
to work with the same cognitive tool in solving some tasks, would their reason-
ing processes be the same as when they work with it on their own? Finally,
diagramming would be determinant to assess new educational strategies to
teach mathematics and science, and more in general to enhance the learners’
inferential abilities.
I hope I have succeeded in showing how many questions and issues are
hidden behind our use of diagrams and, more in general, of written signs, and
in giving some arguments in favor of the hypothesis about the existence of a
typically human ability, diagramming, that is the source both of our spontaneous
reference to external cognitive tools and of their cognitive advantages. Many

aspects need to be clarified, but that are reasons to believe that the path that
will bring to the definition of a full-fledged theory about diagrammatic reason-
ing will keep forcing the researcher to blur disciplinary boundaries: diagrams
and cognitive artifacts in general are the potential target of cognitive science,
history of science, contemporary mathematics and science; the challenge will
then be to work on a framework that will be capable of encompassing all these
approaches.
Frederik Stjernfelt and Svend Østergaard
4 Diagrammatic problem solving
1 Problem solving
In this paper, we shall look at types of problems and their interplay with dia-
grammatic representation. The general notion of diagram goes back to Peirce
(e.g. “PAP”, in Peirce 1976b, 316ff), see (Stjernfelt 2007, chapter 4) for an in depth
introduction. Here, we will focus upon prototypical diagrams, a two dimensional,
stylized topological (geometric) representation of some subject matter. In some
cases this representation is analogical, as for instance, in a map of a country,
but in other cases it is a geometric representation of quantitative relations
as, for instance, in a pie chart diagram. The difference is whether we map an
already existing geometric relation onto the diagram, or whether it is quantita-
tive or other relations that are geometrized. If we add time as a dimension to
the traditional spatial dimensions, we can also interpret temporal relations as
geometric. Hence, for instance, a flowchart diagram of information currents in
a company will qualify as a diagram of geometric relations. However, there are
other relations than spatial/temporal and quantitative; for instance, interpersonal
relations: a crime investigator might set up a diagram of possible interpersonal
relations between the different suspects and associates in order to get an over-
view of the problem. In short, a prototypical diagram is a two-dimensional geo-
metric representation of something we may qualify as “relations” which might
then be spatial/temporal relations, quantitative relations, interpersonal or other
relations.
When we have a diagram it is possible to explore it. This exploration can
take two forms. 1) the exploration does not add anything to the diagram but con-
sists in an examination of possible true statements that may be deduced directly
from the diagram. This is the case of most information in both the pie chart
diagram and the map of the country. From the map we can read the distance
between A and B (given the map scale), find the shortest route between A and
B etc. without adding anything further to the map.1 2) In the second type of
1 Even in this case, however, it could be argued that something is added – namely the points of
departure and destination and the line routes explored between them. In this sense, the distinc-
tion we are making is rather one of a continuum between less and more manipulation/addition.
Frederik Stjernfelt, Copenhagen

Svend Østergaard, Aarhus
104 Frederik Stjernfelt and Svend Østergaard
explorations there is a manipulation of the diagram. In the example with the

crime investigation, the detective might add some connections in the diagram
just to check if these connections are true and what then can be deduced.
In Peirce’s theory (e.g. “Minute Logic”, 1902, CP 4.233) and in Stjernfelt (2014,
ch. 10), these two types of diagrams are called theorematic when there is manip-
ulation or the addition of further elements, and corollarial when the informa-
tion can be read off the diagram directly. This terminology is well known from
mathematics where the theorems that the mathematician proves are the hard
stuff requiring information that is imported from outside the problem space,
often introduced as lemmas, and the corollary is the true statement that can be
read off directly from the theorems without further manipulation. It is clear from
Peirce’s definition and his examples of theorematic reasoning that by ‘manipu-
lation’ he does not just refer to external manipulations of the diagram but
also to any mental manipulation. Hence, if the problem solver confronted with
the diagram imagines some kind of operation on the diagram it qualifies as
theorematic reasoning, for instance, in the mundane case of a map one can
image things like “if I go this way I get to B” etc. For this reason, theorematic
reasoning constitutes a heterogeneous set that can be explored further for internal
differences and subtypes. An attempt to do this can be found in (Stjernfelt 2014,
ch. 10.3).
Solving a problem is, to a large degree, a question of focusing on the right
information, and a diagram is a representation of information relevant to the
situation. In what follows, we will briefly present a typology of types of problems
and the corresponding diagrams. The diagram is per definition a geometric/
topological structure. Hence, if the source is not itself geometric the diagram
is the result of a mapping from another source domain to a geometric domain.
Since this mapping is performed by humans, the diagram is not just the visual
representation but is in fact indicative of what ways the human mind may work.
In other words, thinking is already diagrammatic. This connects to Peirce’s
radical claim that solving any mathematical problem necessarily involves dia-
grammatic manipulation. This is because it is through diagrammatic representa-
tion and diagrammatic manipulation that the mind accesses the information
necessary to solve any given problem.
2 Information internal or external to the

problem space
The major distinction concerning problem solving is whether all information
is present in the diagram or whether one has to add information from outside
the problem space. The first is the prototypical case and includes all types
Diagrammatic problem solving 105
of dynamic operations on a given diagrammatic representation. They may be

mental or they may involve addition of elements as long as they belong to the
problem space. We find a case of a rule-governed manipulation in chess. The
position on the board can be considered as a diagram which you can only
manipulate following certain rules; the chess player reasons by imagining moving
a piece and estimating the possible opponent moves and in this way she might
find the optimal solution of the position.
Figure 25: Euclid’s proof of the sum of the angles in a triangle
The classical example mentioned by Peirce is Euclid’s proof that the sum of
the angles in a triangle is 180, a version of which is given above (Figure 25).
Given the triangle ABC, you extend the line BC to D, and from C you draw a
new line CE parallel with AB. The angle between this new line CE and AC is
identical to the angle A and the angle between the new line and the extended
line CD is identical to the angle B. So, the three angles meeting in C are the
same as the three angles of the triangle, and as BD is a straight line, the three
add up to 180°. Hence, the sum of the angles is 180°. Although two new ele-
ments are added, they do not come from outside the problem space, on the
contrary they, as well as the triangle, are part of the two dimensional space
and they are drawn according to the general axioms of Euclidean geometry –
which are analogous to the chess player who imagines moves in accordance
with the general rules of chess. In both cases, however, what is additionally
required for the problem solver is some strategic information. It is not sufficient
to know Euclid’s axioms and chess rules – in both cases, some sort of strategy
must be pursued: the relevant auxiliary lines to add must be chosen over an
infinity of others, and strategically clever moves sequences must be chosen
among the finite set of possible moves.
A subclass of problems in the general category of information extraction
from the diagram space is what we could call a perspective shift. This might
include all cases where there is a reorganization of the elements in a diagram

based on abductive reasoning, i.e. a manipulation that is not rule-governed.
For instance, the crime investigator might reorganize the relations shown in his
diagram of the possible connections between the criminal elements, this might
involve a perspective shift where instead of A, B now is considered the main
culprit. An example from science could be Copernicus’ reorganization of the
Ptolemaic model of planetary movements. Instead of taking the sun as the
center of the movements, Copernicus took an Earth-centered viewpoint, not based
on any rules but because he sensed that this could explain the empirical obser-
vations better. Another Peircean example from mathematics is discussed in
Stjernfelt (2014: 278) – a proof of Desargues’ theorem – where the reorganization
consists of embedding the elements in a three-dimensional space instead of two-
dimensional.
The type of problems described above contrasts the cases where the infor-
mation needed is not present in the problem space. For instance, the diagram
of the criminal investigator typically contains question marks. Empty slots in
the diagram show that the investigator tries to fill out by looking for clues he
does not yet know about. The typical examples of this group of problems are
mathematical problems, however. Mathematical problems will in many cases
need helping theorems, so-called lemmas. A lemma is itself a theorem of a
simpler kind and often not directly connected to the problem in question; for
instance, to know if one can find the solutions to an equation of nth degree one
has to know something about the possibilities of permuting a row of n letters,
which obviously belongs to another domain than solving an equation.
2.1 Insight problems

In the gestalt approach to problem solving the focus is on another distinction,
namely whether the solution requires insight or not (Köhler 1925; Duncker 1945;
Ohlsson 1992) – which may be seen as a correlate to the theorematic/corollarial
distinction. Insight can be defined by whether a specific target element has to be
accessed in order to solve the problem or not. Consider for instance the follow-
ing problem: Describe how to put 27 animals in 4 pens in such a way that there
is an odd number of animals in each pen. This problem can only be solved by
putting all the animals in one pen and place the others in concentric pens
around it. The target element is in this case the diagram of concentric circles,
without accessing that diagram the problem cannot be solved (Figure 26). We
can schematically represent the idea of a target element in the following way:
Figure 26: Gestalt approach to problem solving
To the left we have the different positions that the problem solver will move
between without getting the solution, but she might accidentally cross the
boundary and hit the target element on the right, in which case the solution
becomes trivial.
The insight problems are opposed, of course, to problems that do not require
insight. The border between the two classes is slippery, though; take for instance
a chess problem. One can make a program that can solve any problem instantly,
not by insight but by systematically going through all possibilities until it hits
upon the solution. For humans, however, the cognitive process that leads to the
solution will proceed along strategic schemata and have the characteristics of
insight which possibly is measurable in the problem-solving individual as an
increase of entropy in behaviour just before discovering the target move. For
instance, in Metcalfe and Wiebe (1987), people’s feeling of warmth are recorded
while they solve insight and non-insight problems. There was a progressive
increase in warmth during non-insight problems. But with insight problems
the warmth ratings remained at the same low level before suddenly increas-
ing dramatically shortly before the solution was reached. Similarly, Stephen,
Boncoddo, Magnuson and Dixon (2009) show that when people solve a gear
problem (see below), there is an increase in entropy just before they see the
optimal strategy of counting the gears. In this case, entropy is measured through
eye tracking, i.e., the gaze pattern is stable for a long time, but just before reach-
ing the solution it becomes instable. So it seems that, in terms of behavioural
dynamics, the insight problems are determined by a catastrophic point, namely
reaching the target element, whereas solving non insight problems is determined
by a continuous process reflecting that these problems are routine tasks and tasks
where information is used incrementally, as, e.g., in the Tower of Hanoi problem
(Ohlsson 1992).
We can now classify problems and their relation to diagrams according to

whether it is an insight problem or not. If it is not an insight problem, the
diagram is a representation of some subject matter and from this representation
it is possible to extract information by making mental operations without funda-
mentally changing the diagram, as mentioned above. For instance, a prototypical
diagram is a flow chart diagram, which is a representation of the temporal
dynamics of some domain: water flow in a heating system, information flow in
an institution, etc. Given such a diagram, one can read off possible routes in the
diagram; similarly, in the pie chart diagram, one can get direct information
about the relative size of expenditures in a single domain. However, in these
examples, as is often the case, the diagram is a representation of a source
domain so one can make experiments on the diagram in order to find alternative
organizations of the source. If the heating system doesn’t work properly, a look
at the diagrammatic representation can solve the problem. Whether the manipu-
lation is relying on insight or not is determined by the underlying process
as mentioned above. Thus, if we manipulate a map incrementally to get to the
solution it is not insight, but if we have to use some hitherto unknown property
of the diagram it may be insight.
The insight problems we propose to divide into three types. Firstly, we have
the insight that relies in information that is present in the problem space and
which, although not easy to find, does not violate entrenched schematic repre-
sentation of the field. Solving the problem of finding the sum of the angles in a
triangle is an example of this. Remember: insight problems hinges on a target
element that gives access to the solution. In the case of the sum of the angles
the target element is the line parallel with AB, cf. the figure above. When this
line is added to the diagram the problem becomes corollarial because this line
provides direct access to the solution. Although not easy to find, the parallel line
does not violate any schematic knowledge about geometry, cf. the example
above.
Secondly, we have the problems where the solution requires new elements
that are not directly accessible. This includes the cases mentioned above: the
criminal investigation where one clue might give the insight to the solution
and, of course, the mathematical problems where some external element might
be the clue.
Thirdly, we have an interesting class of problems where the target element
might or might not be part of the problem space but where it will in any case
constitute a violation of entrenched knowledge or entrenched schematic repre-
sentations of the world. Entrenched knowledge gives rise to fixation according
to the gestalt theorists (Duncker 1945). The mind is fixed in a specific represen-
tation, which blocks access to the solution. The problem of the pens mentioned
above is an example of fixedness of representation, since we tend to think

of pens with animals as disjoint. Manipulating the pens in order to solve the
problem is a diagrammatic manipulation. In this process one can only solve
the problem if one stumbles upon the target element, which in this case is the
concentric arrangement of the pens. In Stjernfelt (2007) we find an interesting
example of the same type of problem, namely the German geographer and
explorer Alfred Wegener’s discovery of the plate-tectonics of current geology.
Wegener was doing simple diagrammatic manipulation using the information
you have on a map of the world and noticing that the West coast of Africa
fit with the East coast of South America. This is more than just using the infor-
mation present in the diagram because it requires the breaking up of the
entrenched assumption that the continents were fixed. Geometrically, of course,
the manipulation is simple, but geologically, the manipulation broke with
entrenched assumptions about the structure of the Earth crust. One can also
mention the Copernican revolution as an instance of this phenomenon, in this
case breaking with the entrenched and religiously motivated assumption that
the Earth is the center of the universe. Another instance is the discovery of
non-Euclidean geometry breaking with the assumption that if we have a point
outside a line we can draw exactly one other line through the point that is par-
allel with the given one. Yet another is the definition of imaginary numbers.
When Cardano tried to find a solution of an equation of third degree, he intro-
duced the square root of a negative number in the formulas for the solution. In
order to solve a problem concerning real numbers he had to invent a wholly
new type of number outside the domain of real numbers. Hence, this is a case
in which the diagrammatic manipulation of the equation reaches an obstacle
which give rise to a redefinition of our very concept of numbers2. This is, in
many cases, a central purpose of diagrammatic reasoning, namely, to find the
problematic spots in the reasoning process.
To summarize: we have problems that imply simple diagrammatic manipu-
lations and problems that can only be solved by accessing one (or more) target
elements, such as, for instance, in finding the sum of the triangle there might be
more elements that could provide the solution. In the last case, the target ele-
ment might, in some cases, constitute a break with our entrenched knowledge
about the world and, finally, the target element might be part of the problem
space as is the case with the four pens and the sum of the triangle or it has to
be imported from outside the problem space as is often the case for mathematical
problems.
2 An equation is a diagram because the spatial arrangement determines what manipulation

can be done.
3 Diagrammatic re-description and diagrammatic

re-encoding
One of the characteristics of human cognition is the ability to represent a subject
matter in an external format, such as, for instance, representing the landscape
in a map or the country’s finance in a pie chart diagram. This is a re-description
of a subject matter in a representational format, i.e. a representational re-descrip-
tion. We find this notion in Karmiloff-Smith (1992) where the ability to redescribe
a subject matter in another symbolic representation is seen as a basic aspect of
the cognitive development of children. As the examples suggest, a diagram is a
representational re-description, hence the basis for this cognitive ability – and
thereby the capacity to make abstractions – is the diagrammatic thinking. The
diagrammatic re-description is the key to improve already existing representa-
tions. Take as a simple example the decimal number 5,5. In this number there
are two fives but they don’t mean the same thing. The meaning is determined
by the spatial position and for that reason it is an example of a diagram that
would replace a more cumbersome representation using fractions. We find
another prime example of diagrammatic re-description in the so-called algebraic
geometry. The basic idea is that curves can be represented in a symbolic format
by equations. For example x2 + y2 = 5 represents a circle of radius 5. One
diagram, a drawn circle, is represented by another diagram, an equation. It is
possible to make (algebraic) manipulations with the last diagram and thereby
prove any number of properties of curves easier than by the purely geometric
methods of the classical Greeks.
The notion of diagrammatic re-description is closely related to the gestalt
notion of re-encoding Ohlsson (1992). In the gestalt tradition, re-encoding means
that some aspect of the problem representation is reinterpreted. In a diagram-
matic representation, this means that elements in the diagram are given a new
interpretation; for instance, in Alfred Wegener’s discovery the continents are
re-encoded as being slowly floating instead of being stationary. As the example
suggests, re-encoding is mostly a case of insight. The situation is coded accord-
ing to some entrenched assumptions which block access to the solution. In
other words, access to what we have called the target element sometimes re-
quires a re-encoding. In terms of diagrammatic representation, this is of course
relevant for the problems whose solution relies on perspective shift and the
examples mentioned above would be typical for a recoding procedure: recode
the problem as embedded in 3D and recode the sun as the center for the plane-
tary moves.
3.1 A special case: the Cogwheel experiment

In an experimental setup that is described in more detail below, participants
organized in pairs look at a board with a string of connected cogwheels; they
then have to determine what way to turn the first cogwheel in order for the last
one to turn left or right. This example shows a dual aspect of diagrammatic
thinking. On the one hand, the problem is presented in the form of a diagram,
hence the board with the represented cogwheels is no different from a map of a
country, a pie chart diagram etc. It is then possible to perform bodily and/or
mental operations, i.e. experiments on the diagram. However, in this case the
representations the participants make in solving the problem are themselves
diagrams; for instance, one pair might make gestures of circles, another might
follow the contour of the cogwheels in a continuous curve, etc. We consider all
these gestures as diagrammatic representation of a procedure that will lead to a
possible solution. The transformations of the gestures during the trials corre-
spond to the manipulations of a diagram whereby the diagram might change
considerably. For instance, a participant might start by making circles, alternat-
ing between circles turning clockwise and circles turning anti-clockwise from
wheel to wheel, this may degenerate into a simple wriggling and then finally
just a pointing. This corresponds to a process where superfluous information is
discarded, and the implied diagram changes appropriately. For instance the
process full circle → wriggling → pointing corresponds to discarding first the
information about the size and turning of the wheels, leading to a simple
wriggling indicating direction, then discarding the information about the direc-
tion of turning leading to a simple pointing, often accompanied with speech
indicating the directions symbolized This corresponds to a process of abstrac-
tion, which is a way to make a representation of the subject matter that discards
unnecessary information.
Is this case of diagrammatic problem solving an instance of insight or not?
The answer is that it may be both. If the solver sticks to the mechanics of the
system of wheels, it is easy to solve the problem in a pedestrian way by keeping
track of how each individual wheel turns. This is mechanical and does not
require any insight; it’s like finding one road in a map. The insight requires a
perspective shift as in the cases mentioned above, namely from a system that
consists of n wheels to a system that only contains two wheels. Every second
wheel is categorized the same; consequently, every time we count an odd
number of wheels it is the same as the starting wheel, and every time we count
an even number it is the same as the second wheel. The final insight is then to
see that also the number of wheels is unnecessary information. The only neces-
sary information is whether there is an odd or even number of wheels (so the
counting may also proceed as “odd-even, odd-even, . . .” or “A-B, A-B, . . .”). The
purpose of the experiment is to see if the participants get to this insight, and
if so, how. If they do, to what extent can this be predicted from the gestures
(diagrams) they produce from the very beginning?
3.2 Cogwheel lessons

In what follows, we shall present some results from the Cogwheel experiment
that was realized at the Center for Semiotics in Aarhus in 2012–14. The overall
idea is to present a cogwheel problem as described above to two persons in
order to investigate their collaboration activity as they are trying to solve the
problem. The problem was presented to each pair of participants on a 60’’
screen on the wall with an empty space in front of it so that the pair is able to
go close to the screen, gesturing and touching it if they so wish. The activities of
each pair were then video-filmed and recorded, just like their gestural activity
was recorded by measurement devices on their wrists. A simple example of the
task is the following:
The task is to determine whether to pull the left or right lever in order
to give the rabbit access to the carrots rather than to give the lion access to the
rabbit. The cogwheel diagram requires an interpretation in which the single
wheels are able to move around their center and thus pass energy and resulting
movement to connected wheels. This interpretation of the diagram, in itself
static, as a dynamical physical system is so obvious that participants immedi-
ately adopt it, requiring no explicit instruction in diagram rules in order to begin
solving the problem.
In the experimental setup, 25 pairs of participants solved a series of cog-
wheel tasks. Each pair was subjected to 18 trials with different gear systems,
advancing from around 5–6 connected wheels to 13 gears. In a final trial, they
were presented with a problem containing 28 gears, in which their solution
time was measured. The initial idea behind the experiment was that two different
solution strategies are at hand (Stephen, Boncoddo, Magnuson and Dixon 2009) −
one continuous and one discrete. The continuous strategy follows the imagined
wheel movements from one wheel to the next − while the discrete strategy indi-
cates, in stepwise sequence, the movement of each wheel as the opposite direc-
tion of the former. Earlier experiments by Stephen, Boncoddo, Magnuson and
Dixon (2009) seem to show that the former strategy is the most immediate while
access to the latter, more efficient strategy takes the shape of a phase transition
in the conceptualization of the problem. By presenting the problem to collabo-
rative pairs of participants rather than to single participants, the idea was to
investigate whether the phase transition in solution behaviour was primed or
Figure 27: The Cogwheel experiment
accompanied by significant changes in communicative behaviour, gestural or

linguistic. Findings proved more complicated, however, indicating that there
are indeed several possible phase transitions in solution strategies, combining
verbal and gestural behaviour in characteristic ways.
The simplest and, to some participants, the most immediate strategy traces
the movement of the initial wheel with a circling finger and then goes on,
continuously, to successive wheels. This can be called the “continuous motor
strategy”, and in the respondents, this strategy is never accompanied by verbal
differentiation of direction (like “this way”/“that way”, “left”/“right”, “up”/“down”
etc.)
Group 13 (Figure 28) is an example of this strategy. Its final trial solution
time is a bit above average. Another strategy could be called the “wriggling
hand” strategy. Instead of continuously tracing movement from one wheel to
the next, the movement of wheels is indicated by alternate hand movements,
typically with the hand a bit removed from the screen. The “wriggling hand”
differentiates directions and often leads to effective (abstract) strategies, such
as the accompanying of wriggling with verbal differentiation of direction or
pointing accompanied by counting.
Figure 28: Group 13 – an example of continuous motor strategy.
Group 24 forms an example of the wriggling strategy. A peak in reaction tie

around trials 13–14 might indicate problems in solution strategy which are
subsequently solved, forming a phase transition to a better strategy − with a
resulting last trial significantly below average.
3.3 Solution strategies

Looking at the whole population, the types of behaviour are found to have con-
siderably more possibilities than the simple continuous/discrete phase transition
originally assumed.
Focusing upon gesture, we find no less than 5 qualitatively different hand/
arm movement patterns:
1. Drawing a full circle for each wheel, typically with just one finger on the
screen
2. Drawing the half arc of a circle before continuing to the next wheel, typically
with one finger.
3. Wriggling the open hand left or right over each wheel
4. Drawing one continuous curve following the outlines of the gears
5. Pointing sequentially to wheel after wheel, typically with one finger
As to verbal behaviours (apart from coordination talk, meta-talk and jokes

etc. among participants), we observe 4 qualitatively different types:
1. Causal reasoning:“if we pull this one, then this one goes up”, abbreviated to
“if this way then this way” or just “this way, this way”
2. Describing the alternating directions of movement of the wheels: “left,
right” or “clockwise, anti-clockwise”
3. Silence (while gesturing)
4. Counting: 1, 2, 3 etc.
These different gestural and verbal behaviours are found to combine in the
following sets of stable patterns:
gesture language
f Chaotic use of gestures f Causal
f Circle/half circle f Causal
f Circle/half circle f Direction
f Wriggle f Causal
f Wriggle f Direction
f Continuous curve f Silence
f Pointing f Direction
f Pointing f Counting
These behaviours combine sequentially in a series of typical developments

which may be indicated in the following flow chart diagram over the landscape
of different phase transitions between stable solution strategies (each arrow
indicating a shift in strategy):
Figure 30: Flowchart diagram of different phase transitions between stable solution strategies.
There is not any very simple pattern of phase transition between two states
to be found, as suggested by Stephen, Boncoddo, Magnuson & Dixon (2009).
Rather, a more complicated landscape of phase transition appears, with two
attractors: the continuous curve-following and the discrete marking of the two
directions, terminating in counting. Gestures accompanied by causal reasoning
are generally instable as a strategy and will develop into either of the two stable
states, the Continuous-silence and the Pointing-counting positions. Obviously,
not all pairs of respondents reach one of these end points, but it seems as
if those settling upon the less than optimal Continuous-silence strategy will
typically remain there. All the different strategies characterized by a verbal
indication of direction, however, may develop further to Pointing-direction or,
ultimately, to the Pointing-counting strategy.
There are some indications that early behaviour during the trials may indi-
cate later strategy choices. Early alternation between left-right movements might
be indicative of later discretization in terms of Wriggle-direction or Pointing-
counting, while, on the other hand, reliance upon causal arguments following
wheel contours may be indicative of settling for the Continuity-silence strategy.
It is interesting to compare the strategies by reaction time at the last trial:
Figure 31: Reaction time by strategy.
Causal reasoning is obviously the slowest strategy, followed by the more

efficient continuous strategy. After this come the three strategies, in order of
increasing efficiency, and involving verbal indication of wheel directions – the
Circle, Pointing and Wriggling strategies with direction indications, respectively.
It is remarkable that the Wriggle-direction strategy is almost as efficient as the
Pointing-counting strategy, making it understandable that strategy developments
may stop at the Wriggle-direction strategy.
4 Conclusion: Embodiment and collaboration –

two hypotheses
These results hold some important lessons as to the role of embodiment in
diagram reasoning. It has often been pointed to the fact that diagrams facilitate
reasoning by means of their spatial presentation of problems, making it possible
to solve them by means of real or imagined bodily manipulation with parts of
the diagram. Additionally, externalized diagrams on paper, board, screen or
elsewhere facilitate the simultaneous or consecutive collaboration on the diagram
by several persons. Both of these aspects, of course, are involved in the Cogwheel
experiment. As to the embodiment issue, an important result seems to be that
“more” embodiment does not equal better solution strategy. The strategies
which closely mimic the causal chain or the movement pattern of the wheels,
touching the screen, are the least effective. The more effective strategies are the
more abstract ones of Pointing-counting and Wriggle-direction which are by no
means completely disembodied but which use gesture (pointing, wriggling) to
address the discrete structure of the problem rather than its concrete, con-
tinuous materiality. We may consider the different gestures as diagrammatic
representations of possible solution strategies. Here, the causal arguments con-
tain, in a sense, too much information. This is also the case with the continuous
curve, which is more efficient than simple causal reasoning but remains a sub-
optimal solution. Getting to the optimal solution requires an instance of insight.
This step of abstraction seems to involve moving focus from local information
about how two wheels influence each other to a more abstract, global regularity
about every second wheel moving in the same direction. To get to the optimal
solution, then, it seems necessary to get away from a closely embodied interac-
tion with the target. Indeed, certain solvers successfully used the wriggling-
direction strategy at a distance of several meters from the screen.
This may give rise to the following hypothesis. Not all reasoning is based on
immediate embodied experience – but rather on abstraction from immediate,
concrete and local embodied experience. It should be added, however, that this
abstraction is by no means completely disembodied but accompanied with
specific embodied strategies supporting abstraction – in this case the wriggling
and pointing gestures.
As to the collaboration issue, the provisional results of the Cog Wheel
experiment seem to indicate that mixed-media strategies involving both language
and gesture are the more efficient. Moreover, comparisons with individuals solv-
ing the same task seem to show that pairs are more efficient than individuals.3
Figure 32: Reaction time on last trial showing the advantage of collaboration.
Why do pairs perform better? A hypothesis may be that several reasons com-
bine here. One explanation is that the task affords abstraction – and as pairs
speak, language may form a route to finding the more abstract solution strategies.
Another explanation is that individuals may tend to get stuck with their first
3 These provisional data were collected at a public performance at the “Science in the City”
festival at the ESOF conference, Copenhagen, June 2014.
solution, while pairs may bring different perspectives and strategies motivating
the intuition that there might be more than one solution strategy to be tried
out. A further explanation may be feedback between the two parties, in the
shape of one participant watching the other’s gesture and drawing further
conclusions; more generally in the shape of collaboration or competition or
both (the two should not be seen as mutually exclusive, rather as two feedback
aspects which may even enhance each other).

I What Is A Diagram and How Does It Function?

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

I What Is A Diagram and How Does It Function?

Uploaded by

Copyright:

Available Formats

All rights reserved.

I What is a diagram and how does it function?

1 Introduction: Peirce looks at paintings

2 Some Vagaries of Peirce’s thinking about

Michael Marrinan, Stanford

1 In the alternate text of “Prolegomena for an Apology to Pragmatism” Peirce writes: “A

because “reading rules” guide our “prescission” of “accidental characters that

comprehension in his 1906 description of how diagrammatic reasoning works:

3 Peirce’s Prescission: Diagrams of no Substance

He remarks a bit further in the main body of the same essay:

Nonetheless, a diagram’s ﬁeld of support is signiﬁcant even if its materiality is

discontinuous, involving the adding or erasing of whole discrete structures on

Not incidentally, this passage follows an aside in which Peirce remarks:

This is a vivid statement suggesting a problematic conception of the diagram-

horizon, which is nonetheless framed by the nearby arching branches. A village

4 Thinking with Materials

On his account, optical consistency makes it possible to work on paper with

Here Diderot describes – especially with his attention to “interconnections” and

measureable volume: there is no ground plane upon which we might imagine

one of Diderot’s many collaborators on the Encyclopédie (cf. Aldridge 1951).

6 Cognitive Turn: Thinking outside the Head

energy around each of us, luminous or mechanical or chemical energy, is not

7 Conclusion: Re-thinking Peirce’s Thing

‘Thinking’ is a fabled ‘operation of the mind’ by which an imaginary object is brought

Peirce goes on to describe the “operation of the mind” as an ens rationis – an

In cognitive science, the two antithetical camps – internalists and exter-

1 Explicative versus ampliative reasoning

Sun-Joo Shin, Yale

2 Classiﬁcation of explicative reasoning

Let me introduce another philosopher’s insight about diﬀerent kinds of

It [deduction] is either Corollarial or Theorematic. A Corollarial Deduction is one which

Kant’s analytic/synthetic distinction is drawn, more or less, in terms of the infor-

3 Searching for a deductive step

An immediate question is raised: Peirce says theorematic reasoning brings in a

Proposition 1: On a given straight line to construct an equilateral triangle.

Proof: Let AB be the given straight line.

(1) Draw a circle ABD, with A as a center and radius AB.

Figure 21: A theorhematic deductive step

4 Abductive elements in deduction

a crucial role of a hypothesis in a scientiﬁc theory, how to form a hypothesis

From given data to a hypothesis, there is no clear algorithm and it seems to

Multiple choices and lack of an algorithm go hand in hand as important features

It became axiomatical that a hypothesis adopted by abduction could only be adopted on

tion of how a conclusion follows from given premises. Deductive explanations

5 Abduction and representation

Proposition 10: To bisect a given a straight line. (Heath 1956)

(1) Draw the equilateral triangle, ABC. (Proposition 1)

For example, if we concluded D is bisecting the line AB by observation of the

Let A1, A2, A3, . . . be sentence symbols.

Figure 22: A truth table

1 Why diagrams matter

en example; its features are presented by relying on two diﬀerent formats: a

(3) 08:01 08:12 08:32

2 The space of the diagram: insights from

embodied; in other words, human cognition would be determined by various

experience to mentally represent knowledge, and once they are established in

commonly recognized in Barsalou’s proposal is that perception symbol theory

2.2 Space to reason

3 Diagrams are a cognitive technology

Now I would like to focus on another feature characterizing diagrams. As

timetable example given in the Introduction, this order is in fact cognitively

3.2 Operative writing, notation and diagrams

goal is to consider historical as well as empirical data to the aim of reaching