Lev Manovich The Engineering of Vision from Constructivism to Computers


1 8









The dissertation presents a history of modern ideas about vision. I believe that vision is not a timeless concept; rather, each period understands vision differently depending on how it is used. In the twentieth century, vision acquired new roles as the medium of mass communication and the instrument of labor, and, as any other productive tool, it was subjected to engineering, rationalization and automation. Such new disciplines as applied experimental psychology and cognitive science, communication engineering and film, robotics, and advertising design continue to search for ways to utilize vision productively. In the process, they generate new knowledge about vision, at the same time reducing it to a few disjoined and limited models. The dissertation chapters follow the development of four such models: vision as a code, vision as a means of logical reasoning, vision as a way to capture spatial information, and vision as information processing.

Let us consider a few definitions of vision that are representative of entire research paradigms and that were unthinkable before the middle of this century. David Marr's Vision, published in 1980, summarized a decade of investigations on human perception carried out at the MIT Artificial Intelligence Laboratory. This book has been the most influential account of the computational approach to vision, shared by computer scientists and psychologists. It opens with this statement: What does it mean, to see? The plain man's answer (and Aristotle's, too) would be, to know what is where by looking. In other words, vision is the process of discovering from images what is present in the world, and where it is.1

1 David Marr, Vision (New York: W.H. Freeman and Company, 1982), 3.

There is nothing "plain" about this definition of vision; it is functional and pragmatic. Visual perception is reduced to a number of computational processes for the recovery of limited information about the world: the identity of objects and their positions. In fact, this is the only kind of information which may be required for a robot or an automatic missile to perform its task. Marr projects these goals of machine vision onto human vision. Here, vision is reduced to the common denominator shared by humans and low level organisms: to detect an obstacle, a predator, a prey. In a special 1984 issue of Cognition (the leading periodical of cognitive science) devoted to visual representations, Steven Pinker outlined the understanding of vision commonly held by cognitive psychologists: Certain abstract problems could be best solved by translating their entities into imagined objects, transforming them using available image transformations, detecting the resulting spatial relations and properties, and translating those relations and properties back to the problem domain.2 In this definition, vision is valued merely for its topological properties -- the ability to represent such relations as inclusion, proximity, and relative positions. According to many cognitive scientists, these properties make it a more efficient medium for problem-solving and abstract thinking than language. And since the mind itself is imagined as an exemplary model of computational efficiency that evolved through evolution, scientists postulate the centrality of visual (read: topological) representations for various mental processes. To think productively and without waste, we might dispense with language and instead think through images, submerging ourselves into the silent movie theater of our minds. Here is another model of vision that narrows its definition to a particular productive property. In a 1951 overview of applied experimental psychology, the field which is today known as human factors, Paul Fitts writes:
2 Steven Pinker, "Visual Cognition: An Introduction," Cognition 18 (1984): 66.

Each sense modality has certain inherent advantages and disadvantages for the detection and analysis of different kinds of information. Audition is more nearly a continuous sense than vision; vision is basically selective and intermittent. As a consequence, audition is well adapted for the detection of warning stimuli that may arise at any moment from one of a variety of sources, whereas vision is well suited to the selection of and concentration on particular stimuli to the exclusion of others.3 What is the purpose of this research into the relative properties of the senses? In contrast to a manual worker, the job of an operator in a modern human-machine system, be it an airplane cockpit, radar display or an automated production line, is primarily perceptual in nature. His or her work is to monitor the displays and to detect those signals that require intervention. Starting in World War II, experimental psychologists have collaborated with engineers in the design of displays and perceptual strategies for their operators. One of the important questions in this research has been the relative advantages of visual, auditory and tactile displays for the detection and analysis of different kinds of signals. Consequently, Fitts describes vision as a sense which is most reliable and efficient for the constant surveillance of a single source -- it is "basically selective and intermittent." According to this model, in order to use vision most productively, one's eyes must be literally "glued to the screen." In each of these models, vision is understood in terms of its efficiency in performing specific tasks: recovery of three-dimensional information, logical reasoning, detection of signals. In fact, there is no single meaning of "vision" which is shared among them -- "vision" as such does not exist. The only thing that unites these modern approaches to vision is the quest for the efficient, reliable, and effective instruments of labor. In this respect, vision is just a set of separate tools which can be employed by either a human or a machine to get work done. It is this emphasis on the productive uses of vision that distinguishes my project from a number of recent histories of vision. In 1988 Hal Foster edited the anthology Vision and

3 Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook of Experimental

Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1314.

Visuality which brought together the groundbreaking essays united by the common theme of historicising modern vision, that is, rejecting it as a natural or a cultural constant.4 In the last few years, each of the five contributors -- Martin Jay, Jonathan Crary, Rosalind Krauss, Norman Bryson, and Jacqueline Rose -- have developed their essays into books which today largely define the direction of the discourse on the visual in humanities.5 At the center of this discourse is the experience of a seeing subject: how is what one sees influenced by psychic structures, by gender and power positions, and by the constraints of the body; how is one's subjectivity determined by the experience of seeing and being seen? At its boundaries are works of art and other cultural representations, which are interpreted as the reified traces of this changing experience. The accounts of these writers greatly expanded the horizons of our knowledge about modern vision. Yet, what remained unexamined is the employment of vision for work. Indeed, it is remarkable that the grounding philosophical and psychoanalytic models of vision, influential in this discourse, are theorized from the experience of a "vacationer." For Sartre, it was the person wandering in the deserted Parisian park; for Lacan, it was the young man (Lacan himself), "being observed" by a shining sardine can, while he was in a boat with some fishermen near a small port; for Freud, it was a young woman confined to isolation in a Viennese apartment or sent to a resort. Not only are these subjects far from a modern workplace, but they are also away from the prototypical perceptual spaces of modernity: the city street, the movie theater, the shopping arcade.

4 Hal Foster, ed., Vision and Visuality (Seattle: Bay Press, 1988). 5 Martin Jay, Downcast Eyes: the Denigration of Vision in Twentieth-century French Thought

(Berkeley: The University of California Press, 1993); Jonathan Crary, Techniques of the Observer: on Vision and Modernity in the Nineteenth Century (Cambridge: The MIT Press, 1990); Rosalind Krauss, The Optical Unconscious (Cambridge: The MIT Press, 1993); Norman Bryson, Looking at the Overlooked: Four Essays on Still Life Painting (Cambridge: Harvard University Press, 1990); Jacqueline Rose, Sexuality in the Field of Vision (London: Verso, 1986).

It is precisely the experience of these spaces that gave rise to a very different kind of theorizing of vision in the writings of Walter Benjamin. Scrutinizing these new spaces of modernity, Benjamin was among the first to notice the contiguity between the perceptual experiences in the workplace and outside of it: Whereas Poe's passers-by cast glances in all directions which still appeared to be aimless, today's pedestrians are obliged to do so in order to keep abreast of traffic signals. Thus technology has subjected the human sensorium to a complex kind of training. There came a day when a new and urgent need for stimuli was met by the film. In a film, perception in the form of shocks was established as a formal principle. That which determines the rhythm of production on a conveyer belt is the basis of the rhythm of reception in the film.6 Here Benjamin already notices that vision became a functional and pragmatic activity rather than a disinterested or contemplative one. The eye is trained to keep pace with the rhythm of industrial production at the factory and to navigate through the complex visual semiosphere beyond the factory gates. Among the contributors to Vision and Visuality, Jonathan Crary is the most sensitive to the primacy of work for the history of vision in the modern period. In Techniques of the Observer, he examines how such nineteenth century disciplines as physiology and psychophysics, and such techniques as optical apparatuses and illusions prepared vision for its later productive deployment. This far-reaching archeology is extremely revealing, yet, because the account ends in the middle of the nineteenth century, it can outline the future deployment of vision only in very broad terms: as participating in the process of the rationalization of labor and as adapting to the new flux of dematerialized images and signs. In order to understand what really happened to vision in the twentieth century, how it was actually deployed and investigated, Crary's Techniques of the Observer is necessary but not sufficient. This century has seen the emergence of disciplines and techniques of vision that simply cannot be traced to the period

6 Walter Benjamin, "On Some Motives in Baudelaire," in Illuminations, ed. Hannah Arendt

(New York: Schochen Books, 1969), 175.

analyzed by Crary. In order to understand their current directions, we need to consider social phenomena of more recent origin such as automation, computerization, and "the information revolution." Most importantly, what could not have been predicted from Crary's account is the radical shift in the nature of labor itself. Benjamin described the rhythm of perceptual shocks of the assembly line worker of the industrial era, who was nevertheless paid to perform physical work: lifting, hammering, filing, and molding. Today, however, in the post-industrial era, the same worker is paid for "perceiving" -- watching, detecting, scanning, and monitoring. This dissertation is an archeology of the currently prominent disciplines and techniques of vision -- computer vision and cognitive science, virtual reality and other new human-computer interfaces. In this archeology, I try to uncover the recently accumulated layers -- 1920s, 1940s, 1960s. Rather than postulating a single paradigm shift, I describe a number of distinct approaches to the rationalization of vision, which developed in parallel. Each chapter investigates one of these approaches. The first chapter is concerned with the model of vision as code of communication. Investigations of the psychological effects of basic colors and elemental forms, conducted in experimental psychology since the second half of the nineteenth century, made possible the idea of a rational visual language composed from simple elements. This idea acquired new importance in the 1920s, when artists found themselves in the role of the designers of mass communication. In Soviet Russia, artists collaborated with psychologists in order to create a fully rationalized visual code of mass communication where each visual element is capable of communicating a meaning, producing an emotion or causing a behavioral response. Visual reasoning is the topic of the second chapter. While classical philosophy considered vision unsuitable for reasoning, in the modern period this attitude is reversed. Freud's model as defined in The Interpretation of Dreams, Galton's composite photography, Eisenstein's intellectual montage, and the adaptation of graphic diagrams in logic and in popular visual

culture are all symptoms of this emerging understanding of vision: as the most efficient medium for logical thinking. With the centrality of cognition for the post-industrial workplace, the notion of visual reasoning acquired new importance. Consequently, the work on the models, techniques, and technologies of visual reasoning, which was previously pursued by a few individuals, now became a matter of systematic research on the industrial scale in such fields as cognitive psychology and scientific visualization. The third chapter addresses the role of vision as a way to capture spatial information. Historically, this role has been played by the techniques of perspectival representation. Between 1940 and 1960, these techniques were automated (computer vision, computer graphics) and also expanded beyond the realm of the visible with radar and other remote sensing instruments. The automation of these techniques necessitated a new paradigm of vision research, which defined vision as a computational system for the reconstruction of object positions in three-dimensional space. The last chapter takes up the understanding of vision in terms of information processing, which emerged after World War II. In the post-industrial society, vision became the principal instrument of labor, the most productive organ of an operator in a human-machine system. In order to monitor and improve the system's productivity, engineers and experimental psychologists have adopted a common framework of information theory for both human and machine components. Today, the number of new disciplines which study vision continues to expand: image science, computer graphics, image processing, computer vision, research on human-computer interfaces and so on. Why? Modern society relies on vision not only as a means for the production of subjectivity but first of all as a means of economic production. If the first use of vision has been extensively theorized, the second so far remained largely unexamined. I hope that this dissertation will begin filling this missing gap in histories of vision.

Chapter 1. Visual Atomism
1. Introduction

Let us compare two statements both related to the question of visual effectiveness but made in different periods. Charles Henry was a French writer who is mainly remembered today for his theory of scientific aesthetics which greatly influenced Georges Seurat and Paul Signac.7 Writing in the 1880s, he advocated that aesthetic responses to simple perceptual elements (which for him were brightness, color, and line) need to be studied scientifically: Art pursues the expression of the physiognomy of things, and aesthetics studies the conditions in which these things are satisfying; that is, when they are represented gay or sad, agreeable or disagreeable, beautiful or ugly...aesthetic things for us are reduced to forms, to colors, and to sounds.8 Half a century later, in 1927, in the article Photography in Advertisement, L‡szl— Moholy-Nagy states: A modern engineer, if his goal and the functional purpose of his work are clear, can without any great effort make a product that is formally adequate and perfect in its economic construction. But the photographic advertisements of our time are not so easy to define. They don't come with "user's instructions." Research into the physiological and psychological laws of visual effectiveness is still far behind the times, compared to the study of the physical laws.9 Both Henry and Moholy-Nagy argue that the creation of visual artifacts should be rationalized and grounded in scientific psychological knowledge. However, the impetus for this rationalization is fundamentally different. For Henry, the function of visual artifacts is artistic
7 See JosŽ ArgŸelles, Charles Henry and the Formation of a Psychophysical Aesthetics

(Chicago: University of Chicago Press, 1972). 8 Qtd. in Paul C. Vitz and Arnold B. Glimcher, Modern Art and Modern Science. The Parallel Analysis of Vision (New York: Praeger, 1984), 87. My analysis of experimental psychology and scientific aesthetics in this chapter is indebted to this groundbreaking book. 9 L‡szl— Moholy-Nagy, "Photography in Advertising," in Photography in the Modern Era, ed. Christopher Phillips (New York: Aperture, 1989), 87.

expression; for Moholy-Nagy, it is to induce effects in the viewer. For Henry, the artifacts are situated in the realm of the aesthetic, producing pleasure or displeasure, divorced from practical life; Moholy-Nagy is concerned with everything practical -- changing beliefs, attitudes, behavior. Henry calls for the rationalization of beauty while Moholy-Nagy wants to rationalize the process of communication. He believes that modern designers have to grasp "the laws of visual effectiveness" to be able to control the viewer's response -- precisely and predictably. And finally, while Henry's "scientific aesthetics" deals with the artistic expression and aesthetic response of an individual spectator, Moholy-Nagy's "laws of visual effectiveness" are to be employed by the newly emerged institutions of mass communication, such as advertising. In this chapter I will discuss one solution to the problem of the rationalization of vision advanced by the 1920s artistic avant-garde. Investigations of the psychological effects of basic colors and elemental forms, conducted by psychologists since the second half of the nineteenth century, made possible the idea of a rational visual language composed from simple elements -"atoms" of visual communication. This idea was already pursued in the nineteenth century by such artists as Seurat and such theoreticians as Henry. However, in the 1920s, when artists found themselves in the role of the designers of mass communication, the idea of an atomistic visual language acquired new urgency and importance. The comparison between the statements of Henry and Moholy-Nagy makes it clear that it was no longer a question of scientific aesthetics; of a work of art producing an aesthetic effect in an individual spectator. Now it became the question of rationalizing mass communication -- the question of economic and political importance. Therefore, Moholy-Nagy writes about the need to precisely "engineer" photographic advertisements, and Bauhaus welcomes experimental psychologists. The trend reaches its extreme in Soviet Russia, where artists thought that they could control not just the consumer habits of segments of society (as in the West), but the consciousness of the whole country. Here the program of research into the "atoms" of visual communication became most systematic,

translating into the establishment of a number of psychological laboratories at various art institutions.10 Their goal was to create a fully rationalized visual language of mass communication where each visual element is capable of communicating a meaning, producing an emotion or causing a behavioral response.

10 These institutions included INKhUK (Institute of Artistic Culture, Moscow 1920-24); GAKhN (State Academy of Artistic Sciences, Moscow 1921-30); VKhUTEMAS (State High Art-Technical Studios).

2. Experimental Psychology and the Science of Art

As other nineteenth century sciences, experimental psychology approached its subject, the human mind, by postulating the existence of further indivisible elements, the combination of which would account for perceptual or mental experience. If chemistry and physics postulated the levels of molecules and atoms and so forth, and if biology saw the emergence of the concepts of cell and chromosome, experimental psychology applied the same reductive logic to the human mind. Psychologists divided sensory consciousness into different modalities: vision, hearing, and tactility and proceeded to enumerate the elementary sensations of each.11 The first psychological laboratory was founded by Wilhelm Wundt at the University of Leipzig in 1879. The general program of Wundt's laboratory was to define the fundamental elements of each modality and to formulate the laws according to which these elements are combined. In 1896 former student of Wundt, E.B. Titchener, who brought experimental psychology to the U.S., proposed that there are 32,800 visual sensations and 11,600 auditory sensory elements, each just slightly distinct from the rest. Titchener summarized his research program as follows: "Give me my elements, and let me bring them together under the psychophysical conditions of mentality at large, and I will guarantee to show you the adult mind, as a structure, with no omissions and no superfluity."12 This analysis of visual experience into its elements seemed to have no practical utility. In fact, the new discipline of experimental psychology developed within philosophy, both intellectually and institutionally. In German universities of the 1870s, more lecture courses were devoted to psychology than any other branch of philosophy except logic.13 Wundt, formerly a
11 Julian Hochberg, "Sensation and Perception," in The First Century of Experimental

Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates, 1979), 94. 12 Qtd. in Eliot Hearst, "One Hundred Years: Themes and Perspectives," in The First Century of Experimental Psychology, 25. 13 Kurt Danziger, Constructing the Subject: Historical Origins of Psychological Research (Cambridge: Cambridge University Press, 1990), 211.

professor of philosophy, was quite content with the allocation of psychology to philosophy and had no interest in its practical applications.14 Yet, even before the discipline of experimental psychology was recognized officially, it had already found its first eager users -- artists and aestheticians. This was not simply a question of creating works of art which directly illustrated the new scientific theory, in this case, psychologist's atomistic theory of perception -- such as the works of Impressionists and Neoimpressionists which represented reality with dots and dashes, the painterly equivalents of sensory elements. Rather than illustrating an atomistic theory of perception, some art theorists and artists realized that they could adopt its approach as a basis of a truly scientific aesthetic and art. What made this idea possible? As experimental psychologists split visual experience into separate aspects (color, form, depth, motion) and subjected these aspects to a systematic investigation, from the outset they had been asking two questions. First, what physiological and psychological mechanisms are responsible for the perception of each aspect of vision? Secondly, what is the psychological effect on the subject of the stimuli, such as elemental colors or forms? It is this second question which was of great interest to aestheticians and artists. By relying on elemental colors or forms with known psychological effect, artists could now predict and anticipate viewers' emotions. Conversely, aestheticians could now develop an objective aesthetic, grounded in the scientific laws of the emotional effects of pictorial elements. As an example of this research, consider an aspect of visual experience such as form -"visual configuration considered as distinct from such things as color and pictorial content and from the representation of objects."15 While some psychologists were analyzing human perception of simple forms such as squares, circles, and straight lines of different orientations,

14 Ibid., 39. 15 Vitz and Glimcher, Modern Art and Modern Science, 144.

others were investigating their psychological effects. In the 1870s a French professor of art, Charles Blanc, developed a theory of the intrinsic psychological significance of vertical, straight, and oblique lines.16 Blanc popularized the ideas of the Dutch painter and theorist Humbert de Superville who around 1830 first claimed that simple lines conveyed emotions by having an inherent psychological response (fig. 1): Blanc argued that the straight line is a "symbol of unity" and the curved line of "variety." Of the straight lines, the most important is the vertical -- since man stands perpendicular to the horizon and since he is bilaterally symmetric. The horizontal line is next in importance and then the two obliques. Blanc discusses three types of human faces characterized by three orientations and by the associated effects of happiness, rest and sadness.17 In the 1880s, Charles Henry further advanced Blanc's ideas, developing the "aesthetic protractor" to measure the harmony of line angles (fig. 2). Henry's device was designed to measure "whether the angles between lines radiating in different directions from a single point are harmonious."18 It is significant that the first laboratory research on the aesthetics of simple forms was conducted in 1876 by Gustav Fechner -- the founder of mathematical methods for the measurement of sensations which became the basis for experimental psychology. Later, at the turn of the century, many proto-Gestalt psychologists started to investigate the aesthetics of simple forms, isolating preferences for squares, rectangles, ellipses, and triangles as well as different lines.19 Their publications became filled with "abstract" pictures consisting of simple forms. This psychological research into the effects of simple forms influenced Seurat, Signac, Wassily Kandinsky, Paul Klee, and Piet Mondrian, among others. For instance, Seurat, who was

16 Ibid., 170. 17 Ibid., 169. 18 Ibid., 170. 19 Ibid., 155.

familiar with the works of Blanc and Henry, advanced a similar theory of the intrinsic psychological effects of lines of different orientation (fig 3).20 In another example, Kandinsky, in Point and Line to Plane, advocated "microscopic" analysis of three basic elements of form (point, line, and plane) claiming that there exists reliable emotional responses to simple visual configurations.21 Equally telling of Kandinsky's program are the titles of the articles he published in 1919: "Small Articles About Big Questions. I. About Point," and "II. About Line."22 At first, artists would embed simple lines or forms in their otherwise representational compositions. A typical case is Seurat who based the orientations of lines in his major paintings, such as Le Cirque (1890-91) and La Parade (1887-88), on the theory which he derived from Blanc: "Gaiety...of line, lines above the horizontal; calmness...the horizontal...Sadness...of line, downward directions."23 Gradually, however, artists gave up representation altogether and began to compose works which would consist solely of the simple elements already studied by psychologists. It is, therefore, not accidental that the paintings of Mondrian, Klee, and Kandinsky look remarkably similar to the visual stimuli already widely used by psychologists in previous decades. They are also experiments, a result of a systematic investigation into what Kandinsky called "the science of art," the science which would allow the reliable communication of any emotional experience.

Proposing scientifically based aesthetics, Henry wrote:

20 Ibid., 170-71. 21 Wassily Kandinsky, (1926), Point and Line to Plane (New York: Solomon R. Guggenheim

Foundation, 1947). 22 Yu. A. Molok, "'Slovar simvolov' Pavla Florenskogo. Nekotorye margonalii" (Pavel Florensky's 'dictionary of symbols.' A few margins), Sovetskoe Iskusstvoznanie 26 (1990): 328. 23 Qtd. in Vitz and Glimcher, 170.

That which science can and must do is to expand the agreeable within us and outside of us, and from this point of view its social function is immense in this time of oppression and blind conflicts. It ought to spare the artist hesitations and useless attempts in assigning or indicating the way in which he can find ever more rich aesthetic elements; it ought to furnish the critic a rapid means of discerning the ugly, so often informulable, however much it is felt.24 Henry equates an artist with a scientist whose vocation is to provide the viewers with the means of escaping the pressures and the unpleasant experiences of everyday life through the beauty of art. The artist is the therapist of a society, and because of the importance of this role, he ought to follow a scientific method in order to make his works most effective in helping the viewers to forget the "oppression and blind conflicts" of modern existence. Moholy-Nagy, in the already quoted statement from the 1920s, also compares the artist to an applied scientist -- an engineer. The artist, however, is no longer removed from the realm of everyday life. Just as the modern engineer shapes every aspect of material reality -- the machines for living (architecture), the machines for transportation (airplanes, cars, trains), the machines to produce other machines (the tools of industrial production) -- an artist has the crucial role of shaping the psychical reality, controlling people's emotions and ideas through advertisements, publicity, posters, and other visual propaganda. Moholy-Nagy's appeal to designers and photographers brings forth the new purpose of the research into the elemental units of vision -- the need to optimize the process of mass communication. This new purpose became crystallized at the moment when modernist artists claimed the position of designers of mass propaganda in Soviet Russia in the 1920s. At this moment the two lines of inquiry -- artistic exploration of the visual elements and research in experimental psychology -- explicitly converge in Soviet discourses and institutions. In a number of Soviet art institutes of the 1920s such as INKhUK (Institute of Artistic Culture, Moscow 1920-24), GAKhN (State Academy of Artistic Sciences, Moscow 1921-30), and VKhUTEMAS (State High Art-Technical Studios), El Lissitsky, Alexander Rodchenko, Osip Brik, and others

24 Qtd. in Ibid., 87-88.

collaborated with experimental psychologists to investigate the effectiveness of visual elements and their combinations.25

The paradigm of research into the elements of visual communication and their effects can be conveniently traced through the history of INKhUK, which during the four years of its existence united the key groups of avant-garde artists and theoreticians most committed to this research. In the first year of INKhUK's existence the group led by Kandinsky was the most active. His program called for the cataloging of basic elements of art and to describe their emotional effects.26 Following Kandinsky's proposals, INKhUK established its own psychophysical laboratory in the Fall of 1920 but Kandinsky and his supporters were forced to leave a few months later.27 Kandinsky then became vice-president of the newly formed GAKhN and the head of its largest physical-psychological division. The method of Kandinsky's division was characterized as "the observation of and experimentation with the content and processes of artistic creation and reception as well the work of art in all its complexity, understood as a

25 Other Soviet research institutes of the 1920s whose programs included the development of a

scientific approach to art and art history include: The State Institute of Artistic Culture (headed by Kasimir Malevich, Leningrad 1923-27); The Institute of Literature, Art and Language, a part of Communist Academy (1918-36); The State Institute for the History of the Arts (Leningrad 1921-31); The Institute of Art History, a part of Moscow University; The Russian Association of Research Institutes of Social Sciences (Moscow 1924-31). 26 Wassily Kandinsky, (1920), "Programma raboty Instituta khudozhestvennoy kultury (The Program of the Institute of Artistic Culture)," in Sovetskoe iskusstvo za 15 let. Materialy i dokumentatsiia (Fifteen years of Soviet art: materials and documentation), ed I. Matsa (Moscow, IZOGIZ: 1933), 126-139. 27 The work of INKhUK during first year of its existence has been documented in S.O. KhanMagomedov, "INKhUK: vozniknovenie, formirovanie i pervyy period raboty. 1920" (INKhUK: appearance, formation and first period of its work, 1920), Sovetskoe Iskusstvoznanie (1981): 332-368. 3. Visual Atomism: a Code for Mass Communication

psycho-physical object."28 The members of the academy included art historians, practicing artists, physicists, psychologists, and physiologists. Meanwhile, Alexander Rodchenko became the new president of INKhUK. Rodchenko and his supporters formulated a new program in total opposition to Kandinsky. Against Kandinsky's "subjective" psychologism they emphasized the materiality of objects. They also opposed Kandinsky's "individualistic" composition in favor of construction and promoted the making of objects rather than the creation of art.29 However, the project of a science of visual elements was not abandoned in practice, only the emphasis shifted from the "subjective" analysis of individual aesthetic reactions to the "objective" analysis of the elements as manifested in material objects. INKhUK members, who started to teach at the newly formed VKhUTEMAS (State High Art-Technical Studios), had the opportunity to put these ideas into practice.30 For instance, in his course on graphics Rodchenko would start his students with systematic exercises to combine simple forms (circle, triangle, and square) within a given format in order to test the visual effectiveness of various combinations (fig. 4).31 In its third stage INKhUK became the center of activity of the promoters of the idea of production art -- Osip Brik, Boris Arvatov, Nikolai Tarabukin, and others.32 In 1923, this group founded the journal LEF (The Journal of the Left Front of Art, Moscow 1923-25) the goal of which was to unite artists of the Left dedicated to radical revolutionary activity in the face of what they perceived as the new flourishing of "bourgeois" art made possible by the NEP.
28 P.C. Kogan, "O zadachakh akademii i yeyo zhurnala" (About the goals of the academy and its journal), Iskusstvo 1, no. 1 (1923): 9. 29 A number of early constructivist manifestos are now available in English translation in Nicholas H. Allison, ed., Art Into Life: Russian Constructivism 1914-1932 (New York: Rizzoli International Publications, 1990), 61-82. 30 It was not accidental that the artists referred to their activities at INKhUK as "laboratory work." "Laboratory" connoted scientific labor, rather than artistic experimentation. "Institut Khudozestvennoy Kultury" (The Institute of Artistic Culture), Russkoe Iskusstvo 2-3 (1923): 85. 31 A.H. Lavrent'ev, "Propedevticheskaya distsiplina 'Grafika.' Vkhutemas. 1920-22 gody" (The discipline 'Graphics.' VKhUTEMAS. 1920-22), Tekhnicheskaya Estetika 7 (1984): 19. 32 "Institut Khudozestvennoy Kultury," 86.

What should this radical revolutionary activity be? Western art historians, critics, and artists glorified the constructivists' desire to give up the traditional roles of artists and instead submerge themselves in industry, to leave the artist's studio for the factory -- in other words, to become industrial designers.33 What received practically no attention was another role which constructivists and many other Soviet artists were claiming for themselves in the 1920s -- the role of theoreticians and practitioners of mass visual communication, be it propaganda or advertisement. And it was in this role, particularly as advertising designers, that VKhUTEMAS artists, as well as their counterparts from the Bauhaus, were able to accomplish the transition "from an easel to a machine" (to quote the title of a book by Tarabukin, one of the theoreticians of production art).34 These machines were not a lathe or an engine but a printing press, a film camera, a light projector, a radio transmitter -- the machines of mass communication rather than mass production. Left artists were united in rejecting art in favor of mass propaganda. In the first issue of LEF Sergei Tretyakov writes: "Revolution put forward new practical tasks [for artists] -affecting mass psyche, organizing the will of the class"; "The work of futurism is parallel to the work of communism -- futurism is fighting for the same dynamic organization of personality without which the movement toward communism is impossible."35 For LEF, this task of the revolutionary education of the masses required a scientific method and in the same article Tretyakov called for the creation of a "science of art as a means of emotional and organizing

33 In 1923, Lunacharsky wrote: "They [constructivists] play at being engineers, but they don't know as much of the essence of machinery as a savage." Qtd. in Brandon Taylor, Art and Literature under the Bolsheviks (London: Pluto Press, 1991), 1: 177. 34 Between 1923 and 1925 Rodchenko designed advertising for many Soviet government agencies; Lissitsky, who was sent to Germany to publicize the art of the first Socialist state, created brilliant designs for many German firms; so did Moholy-Nagy and other Bauhaus faculty. 35 Sergei Tretyakov, "Otkuda i kuda? (Perspektivu futurizma)" [Where from and where to? (Perspectives of futurism)], LEF 1 (1923): 197, 203.

influence on psyche, in connection with the task of class struggle."36 In fact, artists should become as scientific in their work as Marxists are in their political activity. To do so, they should rationalize the creative process and put art education on a scientific basis.37 This idea was most systematically promoted by Proletcult (Organization of the Representatives of Proletarian Culture). Thus, Tarabukin organized the artist's workshop aiming to bring the principles of Taylorism and Fordism into artistic practice: "We are analyzing processes of work, experiment with compressing work periods, aim to rationalize work and leisure. In its practice the workshop tries to get away from bohemian habits so common to artists."38 Artists should also rationalize viewers' reactions by developing a science which would enable them to produce predictable and measurable effects in the viewer. Thus, the author of a 1927 article which summarized the activities of the Art Section of Leningrad's Proletcult writes: Artist-communist, artist-revolutionary came to understand the necessity to perfectly master his weapons to be able to strike without a miss. The striving to conscious mastery was put forward as the main task ("it is necessary to know in order to produce"). Artists of Proletcult were sure that at the basis of art's effect on perception are to be found completely objective laws. From this came the task to master the "mechanics" of artistic work, in order to orient it toward new utilitarian tasks of revolutionary art.39 This dual goal of rationalization was symbolized by the popular metaphor of artist as "psycho-engineer," frequently encountered in the discourse of constructivism, Proletcult and

36 Ibid., 199. 37 LEF required the same from poets and writers who could make their work scientific by

relying on the advances made by Russian Formalists in studying the devices of literature. For LEF, these devices became the tools of verbal propaganda. In the first issue of LEF Osip Brik writes: "OPOYAZ studies the laws of poetic production... OPOYAZ will help the art of the proletariat, but not with vague conversations about the 'proletariat's spirit' and 'communist consciousness,' but with precise technical knowledge of the devices of contemporary poetic work." Osip Brik, "T.n. 'formalnyi metod'" (So called 'formal method'), LEF 1 (1923): 214-215. The meaningless experimental poetry of Futurists which originally made Formalists recognize the "independent value of the poetic sound" now became full of meaning; the techniques of "poetic instrumentation" were now seen as the techniques of organizing the consciousness of the masses. 38 I. Matsa, Sovetskoe iskusstvo za 15 let, 279. 39 Ibid., 281.

LEF. Aleksey Gastev, a leading figure of Proletcult as well as of the Soviet Taylorist movement who in 1921 organized the Central Institute of Labor coined the phrase "social engineering" to describe the process of the radical reconstruction of the human psyche.40 Tretyakov applied the metaphor to art: "Along with man of science the art worker should become psycho-engineer, psycho-constructor."41 The crucial question then becomes how to rationalize visual effectivity. In principle, different approaches are possible: to create particular conditions of reception; to limit the number of available images in the public sphere (this was done later in the period of Socialist Realism); and finally, to create a particular code, a new visual language guaranteed to communicate effectively and efficiently to mass audiences (often, in the case of Soviet Russia of the 1920s, illiterate). It is the last solution which was explored by Leftist artists such as Rodchenko or Lissitsky. Why did the notion of a visual language consisting of simple abstract forms become attractive? First, abstract geometric forms were neither associated with bourgeois figurative representation nor did they look like the "subjective" abstract improvisations of Kandinsky. Secondly, as Igor Golomstock points out, abstraction was justified by emphasizing its stylistic origin in truly popular decorative and religious art (for instance, icons).42 Thirdly, simple forms were proclaimed to be efficient in a situation of mass communication. Thus, Lissitsky argued: "The most unambiguous and immediately recognizable forms are geometric forms. No one will confuse a rectangle with a circle, or a circle with a triangle."43 Finally and most importantly, such forms were imagined to be perfectly suited for the purpose of controlling the effect in the viewer -- both the idea and the imagery being prepared by decades of investigations in the

40 Igor Golomstock, Totalitarian Art (New York: HarperCollins Publishers, 1990), 26. 41 Tretyakov, "Otkuda i kuda?," 202. 42 Golomstock, Totalitarian Art, 26. 43 Qtd. in Golomstock, Totalitarian Art, 24.

psychology of perception and in scientific aesthetics. The tradition of research into the effectiveness of visual elements transformed into the dream of mass "psychophysical culture" advanced in Soviet Russia in the 1920s. The most striking record of this dream is the paper "Engineerism" which was presented at one of INKhUK's meetings. The paper starts with an analysis of modernism as a reduction of visual experience to its elements: "In the final stage of its development art has renounced representation and has moved toward visual sensations as such."44 The trend continues with abstract art, the aim of which, according to the author, is "the organization of visual perceptions." Bourgeois art cannot go beyond this stage of analysis, but in the first proletarian state, this analytic work will become the basis of a new visual culture of social control: "Visual sensations as such will concertedly shape the human being as an organized unit, with the tempo of something which belongs only to its own time."45 From this perspective, we also should not be surprised to find in the same period the recurrent attempts to establish various graphic alphabets and dictionaries. The poet Velemir Khlebnikov in his 1919 article "Artists of the World" postulated two types of a universal visual code -- graphic signs and color equivalents. In the already mentioned INKhUK program, Kandinsky proposed to create a full dictionary of colors, forms and their combinations with descriptions of the psychological effect of each element. Pavel Florensky together with his students at VKhUTEMAS started to compose "Symbolarium" ("The Dictionary of Symbols") in 1922.46 This "international and ahistorical dictionary" should have contained all cultural meanings of every graphic symbol. In the introductory article of "Symbolarium" Florensky regretted that while whole academies were busy preparing and preserving the dictionaries of
44 Allison, Art Into Life, 79. 45 Ibid., 81. 46 P.A. Florensky, (1922), "Symbolarium (Slovar simvolov). Predislovie" [Symbolarium (The dictionary of symbols). Introduction], Trudy po znakovym sistemam V (Tartu: Uchebnye zapiski Tarturskogo Universiteta 284, 1971): 521-527.

national languages, no work yet has been done in composing a similar dictionary of ideograms -the "universal language of mankind."47 Similar projects of a universal graphic alphabet were advanced in the same years by Ivan Puni, Petr Miturich, and other Russian artists.48

To summarize: early Soviet attempts to theorize and put into practice a scientifically grounded visual language of mass communication bring together artistic exploration of visual elements and research in experimental psychology. While the desire of Soviet artists to emulate "scientific" Marxism and to adopt Taylorism and Fordism were already sufficient reasons to pursue the science of elements of art, the ultimate justification for such a science was provided by experimental psychology and scientific aesthetics (Blanc, Henry, and others). Their investigations of the psychological effects of basic colors and simple forms prepared the idea of a language of mass communication composed from simple elements. In theory, this model called to enumerate visual elements and to describe their effects in order to compose dictionaries of "visual effectiveness." In practice, it translated into the creation of works composed from distinct and simple abstract forms. From 1921 to 1929 the psychological studies of viewers' responses to visual forms were actively conducted in the central art institutes. In 1926 GAKhN created a psychophysiology laboratory, a psychophysics department, and laboratories for experimental aesthetics and art theory.49 In order to arrive at the general laws of the effectiveness of visual forms the investigators have systematically analyzed the basic dimensions of volume, line, color, and
47 Ibid., 527, 523. 48 Molok, "'Slovar simvolov' Pavla Florenskogo," 322. 49 E.L. Beliajeva, Arkhitekturno-prostranstvennaya sreda goroda kak obyekt zritelnogo

vospriyatiya (Architectural and spacial city environment as an object of visual perception) (Moscow: Stroyizdat, the Mind-Body Problem 4. Visual Atomism and1977), 13.

texture. It was hoped that the results of the analysis would put visual communication on a scientific basis. Jack Chen, a Communist British artist trained in the Soviet Union during the 1920s, wrote about this program of art production based on psychological research: Art must be a science, an industry. Pictures and sculptures should be constructed according to exact scientific principles after colors and forms had been classified according to their human reaction values...A picture according to the Left was really nothing but a "machine" for generating certain predetermined human reactions. Artists should be engineers of form and color.50 There does not appear to be a disagreement among the Left as to the function of art -- to create predetermined viewer responses. But how was this to be achieved? In other words, assuming that we have isolated the elements of a visual code, how should these elements affect the viewer? Do they induce emotions, do they communicate ideas or do they control behavior in a more direct way? To answer this question, Soviet artists of the 1920s relied heavily on the psychological theories of the day.51 It is possible to discern two distinct trends in how psychology was used. In the first trend, autonomy was granted to the human psyche. The second trend privileged physiology, considering the mind a neurological organ not different from the rest of the organism. This trend was represented by such influential scientists as Bekhterev and Pavlov, with their studies of physiology, reflexology, and conditioning. The latter model was also prevalent in Soviet Marxist psychology in the 1920s. In 1924 it was challenged by Lev Vygotsky, a young psychologist and a former literary critic. Recognized today as the forerunner of cognitive psychology, he argued that the higher cognitive functions are qualitatively different from the lower physiological processes and require different

50 Jack Chen, Soviet Art and Artists (London: The Pilot Press, Ltd., 1945), 58. 51 For the history of Soviet psychology in the 1920s, I have consulted David Joravsky, Russian

Psychology: A Critical History (Cambridge, Mass.: Basil Blackwell, 1989); Alex Kozulin, Psychology in Utopia: Toward a Social History of Soviet Psychology (Cambridge: The MIT Press, 1984).

investigative approaches. In his famous address to the Second Russian Psychoneurological Congress, Vygotsky declared human consciousness to be a fundamental problem in the psychology of behavior, claiming that it cannot be understood through the study of reflexes. Significantly, the turning point of Vygotsky's career from literary critic to psychologist was his 1924 dissertation Psychology of Art. Through the analysis of Hamlet Vygotsky attempted to demonstrate that art is a product of the capacities of the specifically human mind -- imagination, emotion, and symbolic representation. Crucial for Vygotsky's theory was the study of the formal organization of works of art: ...every work of art is considered to be a system of stimuli, which are organized consciously and intentionally in order to cause an aesthetic reaction. This way we reconstruct the structure of the reaction while analyzing the structure of the stimuli...The general direction of this method can be summarized in the following formula: from the form of the art work through the functional analysis of its elements and structure to the reconstruction of the aesthetic reaction and the discovery of its general laws.52 In Vygotsky's formula the given was the existing works of art and the unknown was the laws of aesthetic reaction. As we have already seen, for the artists, designers, and film directors of the time the formula was reversed. If for the psychologist Vygotsky, visual works represented a reservoir of knowledge about the human mind, artists, on the contrary, were hoping to utilize the objective psychological knowledge in order to create visual works which would produce predetermined responses in the viewer. Contemporary psychologists -- the supporters of Pavlov and Bekhterev on the one hand, and younger psychologists such as Vygotsky and A. Luria on the other hand, argued over whether the mind could be reduced to the laws of physiology. Similarly, the artists in theory and in practice drew on these two alternative psychological models. Sergei Eisenstein's attempts to ground his filmmaking methods in different psychological theories epitomizes the two models. In one of his latest written works Eisenstein summed up his work in film:
52 Lev Vygotsky, Psikhologija Iskusstva (Psychology of art) (Moscow: 1968), 39-41.

I never intended to "reflect" the existing reality. I had one task -- using the means of its influence -- to affect the feelings and thoughts, to influence the psyche and to shape the consciousness of the viewer in the desired, required, and chosen direction. 53 Although Eisenstein's goal remained the same, the means of achieving the desired effect were conceived of differently throughout his life. On the one hand, Eisenstein developed the concept of "intellectual montage," privileging a purely intellectual response. The goal of cinema, in this view, was thought to induce dialectical reasoning. The structure of the film itself was conceived of as dialectical thinking in visual form with montage being the means of representing the dialectical process through the contrast and juxtaposition of images. This preoccupation of Eisenstein with cinema as the visualization of the work of the intellect culminated in his project to create a screen adaptation of Marx's Capital.54 On the other hand, Eisenstein's second important concept developed in the Montage of Attractions takes the viewers' physiological reaction as a point of departure.55 Eisenstein based his concept on the psychological theories of Ludwig Klages and William James.56 According to Klages, in a human being emotional states are expressed through bodily movements. Klages also insisted that human expressivity is characterized by a unique quality -- the muscular contractions of one person are involuntarily repeated by the observer. James' theory was related to Klages' but causally reversed. He postulated that emotions were the effect of muscular contractions -- one does not cry because he is sad, but one becomes sad due to crying. Eisenstein combined the two theories: the emotional state of the actor translates into his muscular movements; these
53 Qtd. in Yu. A. Vasilieva, "Eisenstein: Kontseptsija 'Agressivnogo Iskusstva'" (Eisenstein: the concept of 'aggressive art'), Kinovedcheskije Zapiski 3 (1989): 207. 54 I will return to Eisenstein's "intellectual montage" in chapter 2. 55 Sergei Eisenstein, "Montazh atraktsionov" (Montage of attractions), LEF 3 (1923): 70-75. 56 Olga Bulgakova, "Sergei Eisenshtein i ego 'psikhologicheskiy Berlin' -- mezhdu psikhoanalizom i strukturnoy psikhologiey" (Sergei Eisenstein and his 'psychological Berlin': between psychoanalysis and structural psychology), Kinovedcheskie Zapiski 2 (1988): 178-80.

movements are involuntarily repeated by the viewer causing him to experience similar emotions. The important issue, then, was the training of actors in simulating precise gestures and facial expressions in order to produce a desired emotional response in the viewer. Eisenstein proposes to develop a collection of emotional stimuli which would be strung together in a film -"montage of attractions." The film becomes a script of the emotional responses of viewers. The two concepts -- the intellectual montage and montage of attractions -- are the rare summits in Eisenstein's intellectual career where the opposing tendencies of his thought concerning the modes of cinematic influence on the viewer can be clearly distinguished. Most of the time they seem to have been intertwined, appearing simultaneously in his changing conception of cinematic effectivity. Similarly, in the cultural imagination of the 1920s, the two tendencies appear to be running side by side. Is there a mind, a specifically human autonomous cognitive apparatus, with operational laws not reducible to simple reflexes? Or can human behavior be controlled by affecting the organism through a set of predictable physiological stimuli? This second position received its extreme elaboration in the theory developed by Emmanuel Enchmen, an early Socialist revolutionary.57 Enchmen's "theory of new biology" was based on Pavlovian reflexology and proposed a purely proletarian culture based directly on conditioned reflexes, without the necessity of conventional language or thought. The new proletarian language would consist of conditioned grunts and gestures and replace the verbal culture of bourgeois society. All in all, the co-existence of the two schools in Soviet psychology lead to the coexistence of the two kinds of models of artistic affectivity: affecting consciousness by communicating ideas and bypassing consciousness altogether. This dichotomy is reflected, for instance, in the following statement from "New Artistic Program of Proletcult" published in
57 Joravsky, Russian Psychology, 212.

1923: "Art should become the necessary part of everyday life both in its active-representational forms (poster, advertisement, agitational theater, cinema) and in its material-organizational forms (psychophysical culture, organization of mass happenings, processions and demonstrations, construction of things)."58 The opposition "representational" -- "organizing" reflects two views of affectivity: affecting the mind and affecting the body. These different models of affectivity were also used to understand the roles of visual elements which were being "isolated" and "purified" in the laboratories of the 1920s. The elements could have affected the viewer in more than one way. No longer just catalysts of basic emotional states, as in the tradition of experimental psychology and scientific aesthetics, they were now re-interpreted as capable of carrying complex messages addressed to the mind, and as simple signals designed to trigger physiological and behavioral responses.

I have discussed different models of affectivity which underlined the research into the elements of a visual code. But what kind of code can be formed from such elements? Semiotician Luis Prieto distinguished between codes with and without articulation.59 A code with articulation consists of a number of elements combined to produce messages. The examples are human language, playing cards, and maritime flag codes. The crucial feature of any of these codes is that they allow for the generation of a large or even infinite number of messages from a small set of elements. For instance, in the case of human language, its speaker can generate an infinite number of sentences. In contrast, in codes without articulation there are as many meanings as elements. For example, in the language of flowers, each flower signifies a

58 Matsa, Sovetskoe iskusstvo za 15 let, 261. 59 Luis Prieto, Messages et signaux (Paris, PUF, 1966). 5. Visual Esperanto

distinct meaning (for instance,"red rose" = "victory is yours"); a combination of flowers does not create any new meanings. In view of this distinction, the research into visual elements could justify both a code with articulation as well as a code without articulation. In other words, it is possible to understand the elements in two ways: as carrying a complete message (a verbal meaning, an emotion or a particular behavioral response), or as just a part of the message, like a word in a sentence, combined with other words through grammar. The artistic tradition of visual atomism, from Blanc to Kandinsky, appears to favor the first interpretation. In their writings, Blanc, Henry, and Seurat correlated the orientation of straight lines with different emotions such as gaiety or sadness. Kandinsky's famous questionnaire, compiled in 1920 while he was active in INKhUK, also indicates that he considered the elements to directly correspond to emotions or verbal meanings: "How do you see a triangle -- do you feel that it is moving, where, do you see it as more witty than square; is the sensation from a square similar to the sensation from a lemon; what is a canary's song more like - a triangle or a square."60 A code without an articulation requires as many signs as there are potential messages and therefore is not efficient. Yet, it is this inefficiency which makes it ultimately suitable for new communicative situations faced by artists and designers in the earlier decades of this century. Traffic signs, the labels on instrument panels, trademarks had to be unambiguous and instantly recognizable by an untrained eye. Not surprisingly, the designers in every field have adopted simple abstract forms. Thus, in 1909, when Mondrian and Kandinsky were moving toward their first abstractions, the first International Conference on the Regulation of Automobile Traffic took

60 Qtd. in Khan-Magomedov, "INKhUK," 346.

place in Paris. Following its recommendations, in the 1910s-20s, universal pictograms signifying main road situations were developed and employed.61 For the Soviet "psycho-engineers," the notion of a visual code without articulation, which correlates each message with a separate sign, had a particular attraction. The behaviorist psychologies of Pavlov and Bekhterev popularized the notion of directly controlling behavior through conditioned reflexes. Simple visual forms could have acted as such stimuli, controlling "the proletarian zombie" as the lights and whistles already controlled a dog in Pavlov's laboratory. In this interpretation, the dictionary of visual elements would contain not a set of meanings or emotions but a set of commands -- previously learned responses to stimuli. This is what the author of the "Engineerism" paper seems to have in mind when he writes of the future where "visual sensations as such will concertedly shape the human being as an organized unit." In other words, just as the motorists in the 1920s were beginning to memorize the international traffic signs, learning how to respond to them automatically (today, this is the closest most of us come to experience conditioning through a visual code), artists took this as a paradigm for controlling the masses through visual communication. Yet, while suitable for communication of basic emotional states or a set of behavioral stimuli, the code without articulation is not practical for communicating an arbitrary large number of messages. If visual language is to compete with a verbal language, and even to replace it, visual forms have to be interpreted as equivalents of letters or words. They also have to be supplemented by grammar, the rules of how these visual "words" are combined to form sentences. Indeed, contemporary debates on whether such animals as dolphins or primates can use human language center around the attempts to teach them to use visual signs in this way. It is

61 A.S. Sardarov, "5000 let evolutsii doroznogo znaka" (5000 years of the evolution of a traffic sign), Tekhnicheskaya Estetika 9 (1984): 14-19.

easy to train animals to respond to visual forms as distinct messages: a square means "take food," a circle means "jump," and so on. The question is whether animals can string such symbols together to form new messages, thus treating the individual symbols as words rather than as complete messages. At least a few researchers have claimed that they have taught some animals to do so, thus teaching them a visual code with articulation. What about humans? The idea of replacing particular verbal language by a universal visual language -- more efficient, capable of reaching new masses of immigrants (U.S.) or "proletarians of the whole world" (U.S.S.R.) was often expressed in the 1920s. Film, in particular, was seen as the prime candidate for a visual Esperanto. In a 1921 interview D.W. Griffith said: "A picture is the universal symbol, and a picture that moves is a universal language. Moving pictures, someone suggests, 'might have saved the situation when the Tower of Babel was built.'"62 Miriam Hansen writes about the metaphor of film as visual Esperanto in the U.S.: Griffith participated in the widespread celebration of film as a new "universal language" which accompanied its formation as an institution. Used by journalists, intellectuals, social workers, clergy, producers, and industrial apologists alike, the metaphor of film as a universal language drew on a variety of discourses (Enlightenment, nineteenth-century positivism, Protestant millennialism, the Esperanto movement, and the growing advertising industry) and oscillated accordingly between utopian and totalitarian implications.63 However, it was across the Atlantic, in the U.S.S.R., where the comparison between film and verbal language was explored most systematically by Victor Shklovsky and other literary theorists associated with the so-called formalist school as well as by film directors and theorists such as Eisenstein, Vsevolod Pudovkin, and Lev Kuleshov. In their writings we encounter

62 Qtd. in Miriam Hansen, "The Hieroglyph and the Whore: D.W. Griffith's Intolerance," The

South Atlantic Quarterly 88, no. 2 (1989): 363. 63 Ibid., 362.

frequent comparisons of a shot and a word, or a sequence and a sentence as well as references to a "grammar of film."64 Can we find similar kinds of analogies proposed for the elements of a single image? Just as the artists were comfortable with different mutually exclusive models of affectivity, they were equally at ease with two contradictory models of a code. If the visual elements were imagined to be capable of carrying complete messages, they were also thought of as the elements of a message. Thus, art historian S.O. Khan-Magomedov writes about the INKhUK program: "The point was to discover certain primary elements of artistic expressiveness, in themselves without signification and understood as an alphabet of artistic-compositional system."65 What we don't find in the 1920s, however, is equally serious thinking about the grammar of a still image. It was only in the 1960s, when the analogy between a verbal and a visual language was taken with new rigor in structural semiotics, that coherent theorizing about the grammar of visual elements began to appear.66

The understanding of vision in modernity is characterized by attempts to think of vision as a language. This is reflected in the popularity of the term "visual language," this term which is so familiar to us today that we tend not to ask about its historical specificity. What this term points

64 Paradoxically, the classical Hollywood film style, which matured by the 1920s, did become the universal visual language of the twentieth century, while the films of Soviet avant-garde directors, who were most systematic in theorizing film as a language, were rejected by mass audiences and continue to play art houses, as they did in the 1920s. 65 Khan-Magomedov, "INKhUK," 343. 66 The best critical review of visual semiotics available in English is Gšran Sonesson, Pictorial Concepts. Inquiries into the Semiotic Heritage and its Relevance for the Analysis of the Visual World (Lund, 6. Conclusion Sweden: Lund University Press, 1989).

to, in my view, is the new social role of vision as the medium of mass communication which it acquired in the earlier decades of this century. The attempts to rationalize vision in this role, to conceive of it as a code which can function independently of verbal language and which can be effective in the new communicative situations created by modernization have centered around the quest for the elements, the "atoms" of visual communication. This quest reaches its culmination in the work of a number of art institutes in Soviet Russia in the 1920s, where artists collaborate with experimental psychologists to isolate these elements and study their effectiveness. The search for the essential elements of each art which preoccupied early twentieth century culture is usually seen in the context of artistic modernism. This search is interpreted in the context of the rhetoric of the purity of medium. Indeed, especially in the 1910s-20s, artists tried to reduce every medium to its unique qualities. To do so, they gave up representation and concentrated on the material elements thought to be unique to each medium. Poets, such as Russian futurists, were experimenting with sounds; filmmakers proposed that the essence of cinema was movement and temporal rhythm (French film theory of the 1920s)67 or montage (Kuleshov's group in Russia); and painters were exploring pure colors and geometric forms. The following statement made in 1924 by Jean Epstein, a French avant-garde filmmaker and theoretician, is typical of modernist rhetoric of purity; countless statements like it appeared on the pages of avant-garde publications of the time: For every art builds its forbidden city, its own exclusive domain, autonomous, specific and hostile to anything that does not belong. Astonishing to relate, literature must first and foremost be literary; the theater, theatrical; painting, pictorial; and the cinema, cinematic. Painting today is freeing itself from many of its representational and narrative concerns...And any literature worthy of the name turns its back on those twists and turns of plot which lead to the detective's discovery

67 See especially Jean Epstein, "On Certain Characteristics of PhotogŽnie," in French Film

Theory and Criticism, ed. Richard Abel (Princeton: University of Princeton Press, 1988), 1: 314318; Germaine Dulac, "Aesthetics, Obstacles, Integral CinŽgraphie," in French Film Theory and Criticism, 1: 389-397.

of the lost treasure...The cinema must seek to become, gradually and in the end uniquely, cinematic; to employ, in other words, only photogenic elements.68 Yet, as this chapter has suggested, more was at stake in the quest for the visual "table of elements" than just the demand for purity of medium. While many artists searched for "ever more rich aesthetic elements" (Henry), trying to make their art into a "forbidden city" (Epstein), others welcomed the opportunity to take their art to the streets. For these artists, the science of visual elements has provided hope for the creation of a fully rationalized visual language of mass communication. The dream to rationalize communication was not unique to this project -- the proposals for a universal language go back to the seventeenth century. Descartes suggested that the lexicon of a universal language should be composed of primitive elements. By systematically combining these elements, one can generate "an infinity of different words."69 In the early eighteenth century Leibniz outlined a language in which grammatical and logical structure would coincide, thus making possible the automation of thinking. The basic elements of his ideal language were characters representing unambiguously a limited number of elementary concepts. Leibniz called the inventory of these concepts "the alphabet of human thought." Through the algebra of thought primitive concepts would be combined to form any complex idea. 70 The proposal of a universal language composed from graphic symbols and therefore suitable only for writing followed soon with the first pure pasigraphy ("writing for all") invented by Joseph de Maimieux in 1797.71 However, until the twentieth century these remained isolated philosophical ideas advanced by single individuals. With the rise of mass communications, the fantasy of a universal rational language attains new significance, extending into the realm of vision. Thus, what we now

68 Ibid., 314-15. 69 Qtd. in Winfried Nšth, Handbook of Semiotics (Bloomington and Indianapolis: Indiana

University Press, 1990), 272.
70 Ibid., 274. 71 Ibid., 269.

witness are systematic and extensive attempts to establish fully rationalized visual languages of mass communication. At the same time, the studies of behavior and conditioning in physiology, behaviorism and reflexology, lead to new interpretations of the functioning of the elements of a visual Esperanto. Not only would they be capable of communicating ideas thus affecting the intellect, but they would also directly control behavior bypassing the mind altogether, thus making possible an international "psychophysical culture."

In the sixth Meditation Descartes defined man "as a thing that thinks," to whom reasoning comes naturally. Imagination, on the other hand, requires a special effort and is in no way necessary for a human being. Distinguishing between imagination and pure intellection, Descartes demonstrated the inferiority of vision to reasoning: For example, when I imagine a triangle I not only conceive (intelligo) that it is a figure comprehended by three lines, but at the same time also I look upon (intueor) these three lines as present by the power and internal application of my mind (acie mentis), and this is what I call imagining. But if I desire to think of a chiliagon, I indeed rightly conceive that it is a figure composed of a thousand sides, as easily as I conceive that a triangle is a figure composed of only three sides; but I cannot imagine the thousand sides of a chiliagon as I do the three sides of a triangle, nor, so to speak, view them as present [with the eyes of my mind].72 Indeed, how can humans possibly reason through visual representations, if the latter are not capable of representing general concepts to begin with? In Treatise Concerning the Principles of Human Knowledge, George Berkeley insisted, for instance, that it is impossible to have a mental image of an idea, such as "man," as a generality; it is only possible to visualize a tall or a short man, but not man as such. Thinking deals with generalities and does not tolerate particular things. If a philosopher tries to reason about the nature of "man," any image of a particular man would mislead the reasoning process.73 The philosophical arguments against visual reasoning still continue in this century, but they are out of step with the modern productive use of vision: not only the concepts of "visual

72 RenŽ Descartes, "Meditations on the First Philosophy," in The Rationalists, trans. John Veitch

(Garden City, NY: Doubleday & Company, 1960), 160-161. 73 Rudolf Arnheim, "A Plea for Visual Thinking," in New Essays on the Psychology of Art (Los Angeles 2. I London: University of Chapter and See, Therefore I ThinkCalifornia Press, 1986), 136. 1. Introduction

thinking," "thinking through images," etc. have become respectable and commonplace,74 but they are also put into practice daily through new techniques and technologies of visual representation. While Descartes thought that it was impossible (and, at any rate, unwise) to imagine the thousands of sides of a chiliagon, today scientists stare at computer monitors, where computer imaging brings before their eyes not only chiliagons but geometric objects of ndimensions, objects composed not from thousands but from millions of sides, objects representing weather systems, diffraction patterns, atomic surfaces, and other processes inconceivable to the bare imagination (fig. 5).75 "Scientific visualization" is the name of the new field which claims to transform the nature of scientific reasoning in biology, chemistry, and even mathematics. Likewise, Berkeley's insistence that images cannot convey "man" as a generality has been superseded. Since the 1960s, numerous space stations sent to the Moon, Venus, Mars or simply into the infinity of the Universe have carried images destined to communicate to intelligent beings (should they happen to find the stations) the basics of human civilization and the "essence of man" in a "universal language of images." Long after the sun explodes and man disappears, space stations will continue to fly on their trajectories, carrying such images as the famous drawing by Leonardo da Vinci of an "ideal man." Intelligent (or perhaps, not so intelligent) beings will eventually discover these images, learning from them if not about human civilization in general, then at least about the artistic tastes of administrators of space programs, assuring Leonardo of inter-galactic fame. The images of general or abstract concepts can be encountered not only on space missions but on Earth, everyday and everywhere, in numerous logotypes and designs (for
74 As witnessed, for instance, by such titles as Thinking Eye (Paul Klee) or Visual Thinking

(Rudolf Arnheim).
75 Examples of scientific visualizations presented at the Showcase at SIGGRAPH 1992.

SIGGRAPH '92 Final Program (New York: The Association for Computing Machinery, 1992), 33-40.

instance, in symbols which distinguish men's from women's lavatories), or in the representation of unimaginable data sets. However, these examples do not tell us anything about the specificity of the new function of vision in modernity. After all, for centuries pictures have been employed as visualizations of the holy stories. Michael Baxandall cites a late thirteenth-century text which summarizes this purpose of images: Know that there were three reasons for the institution of images in churches. First, for the instruction of simple people, because they are instructed by them as if by books. Second, so that the mystery of the incarnation and the examples of the Saints may be more active in our memory through being presented daily to our eyes. Third, to excite feelings of devotion, these being aroused more effectively by things seen than by things heard.76 Therefore, what is different about the modern function of vision is not simply the use of images to represent abstract concepts concretely. What is different is the idea of reasoning through images, the idea unthinkable for Descartes or Berkeley. Thus, the field of scientific visualization is based on the assumption that visual observation of computer generated images of data sets and processes leads to scientific breakthroughs otherwise impossible. Richard Mark Friedhoff and William Benson propose that "using a computer simulation...even a beginning student of chemistry might be able to deduce benzene's structure" (a reference to Friedrich KebulŽ's 1865 discovery of the molecular stucture of benzene supposedly made after he saw a dream involving a snake biting its own tale).77 Similarly, Jaron Lanier, one of the most visible researchers/promoters of virtual reality, maintains that virtual reality will lead to the age of "postsymbolic communication," the communication of ideas and arguments solely through images.78 Finally, a prominent cognitive psychologist Philip Johnson-Laird claims that everyday logical

76 Michael Baxandall, Painting and Experience in Fifteenth Century Italy (Oxford: Oxford

University Press, 1972), 41.
77 Richard Mark Friedhoff and William Benson, The Second Computer Revolution:

Visualization (W.H. Freeman and Company, 1991), 13. 78 See, for instance, Timothy Druckrey, "Revenge of the Nerds. An Interview with Jaron Lanier," Afterimage (May 1991): 5-9.

reasoning takes place through images: when we are engaged in reasoning, we construct a mental model of a situation which represents its essential features in the form of topological relations.79

Before it became possible to send Leonardo's images to remote corners of the Universe to represent "humanity," before scientists accepted that computer-aided visualization would dramatically speed up scientific reasoning, before psychologists could claim that reasoning in humans is a matter of manipulating mental images, a profound change in the cultural attitude toward vision had to take place. This change occurred between the 1870s and the 1920s. Logical thinking entails the representation of abstract ideas and, also, of the logical relations between them. During this period, Galton, Venn, Freud, Eisenstein, and others put vision to use for both purposes. More importantly, for the first time, we can find in their work the explicit justifications for the very notion of reasoning through vision. In 1877 Galton's composite portraits visualized universal human types; in 1880, John Venn made public a method for solving problems in logic by using graphic diagrams (section 3). In The Interpretation of Dreams, published in 1900, Sigmund Freud developed a systematic theory of how abstract notions and logical forms can be visualized (section 5). In the late 1920s Eisenstein proposed that film can be used "to encourage and direct the whole thought process" (section 4). These different models of how vision can be used in reasoning represent the first stage in the reversal of attitude towards the inadequacy of vision. The second stage arrived when, in postindustrial society, the concern with the efficiency of the mind overwhelmed the concern with the efficiency of the body. Now, the questions of how information can be coded, stored, retrieved, and processed more efficiently give direction to the study of the mind, including the use of visual

79 Philip Johnson-Laird, Mental Models: Towards a Cognitive Science of Language, Inference,

and Consciousness (Cambridge: Cambridge University Press, 1983).

representations in mental processes. Cognitive scientists, among others, begin to approach vision from this new perspective, and they postulate the centrality of visual representations for human reasoning, because of their efficiency (sections 6 and 7).

To be able to appreciate the novelty of the modern paradigm of visual reasoning, I will begin by discussing the classic philosophical positions on the relationship between vision and reason. Many philosophers thought vision to be unsuitable for reasoning, first of all because throughout the history of philosophy, reason was closely associated with logic. If, following its formalization by George Boole and Gottlob Frege in the nineteenth century, logic has developed into a sophisticated branch of mathematics, today seemingly removed from our everyday discourse, originally it was conceived by Aristotle as a systematization of principles according to which rational debates should be conducted.80 Therefore, it was Aristotle who originated the tradition of discussing human reasoning and logic interchangeably. Locke, for instance defines reason as "that faculty whereby man is supposed to be distinguished from beasts, and wherein it is evident he much surpasses them." Claiming that the highest order of reason consists in "the discovering and finding out of proofs," Locke writes: Sense and intuition reach but a very little way. The greatest part of our knowledge depends upon deductions and intermediate ideas: and in those cases where we are fain to substitute assent instead of knowledge, and take propositions for true without being certain they are so, we have need to find out, examine, and compare the grounds of their probability.81

80 Antony Flew, ed., A Dictionary of Philosophy (London: Pan Books Ltd., 1984), 208-212. 81 John Locke, An Essay Concerning Human Understanding, ed. A.S. Pringle-Pattison (Oxford: Clarendon Press, 1924), Book IV, Ch. 17. 2. I See, Therefore I Think

The equation of reason with logic persisted late into the twentieth century. Piaget assumed that the final stage in the development of human thought processes was a kind of formal logic analysis. Equally, Levi-Strauss's binary oppositions and Greimas's semiotic square are based on stripped down formal logic. This logic is postulated as the structure of the "native mind" (Levi-Strauss) or the structure of all signification (Greimas).82 Only in the last decades have cognitive psychologists discovered that human judgments do not follow statistical probabilities and tend to ignore information about the prior probability of an event;83 that rather than having some uniform "laws of thought," humans employ a variety of heterogeneous heuristics; and that in general the normative rules of logic do not describe human thinking.84 However, the idea of a separation between logic as a normative or prescriptive discipline and human reasoning, which can be described and studied empirically and which does not follow any strict rules of logic, is quite recent, and would be unimaginable for Aristotle or Locke.85 Therefore, in classical philosophy to reason is to carry out logical operations. How then can one possibly reason through images? How can one represent visually "all," "some," "any" or

82 What is different about reason as imagined by Levi-Strauss and Greimas in contrast to Locke

is its new industrial efficiency: the concept of binary oppositions was brought by Jakobson from MIT, the birthplace of information theory which demonstrated that binary code is the most economical of all. According to Geoffrey Sampson, "what led Jakobson to the hypothesis that all parameters are 'binary' seems to have been the mathematical notion that a transmission-code is more efficient when it uses only independent binary choices." Geoffrey Sampson, Schools of Linguistics (Stanford, CA: Stanford University Press, 1980), 250. 83 Gillian Cohen, The Psychology of Cognition, 2nd ed. (London and New York: Academic Press, 1983), 190. 84 Michael I. Posner, and Gordon L. Shulman, "Cognitive Science," in The First Century of Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1979), 393. 85 A recent dictionary of philosophy still equates thinking with logic: "The mental activity of (a) theoretical contemplation directed toward some object with a view to reaching a propositional conclusion; or (b) practical deliberation directed toward some object with a view of reaching a decision to act." Flew, A Dictionary of Philosophy, 352. Emphasis mine - L.M. An encyclopedia of philosophy is equally categorical: "particular thoughts have some kind of logical form; they may be categorical, hypothetical, disjunctive, universal, particular, and the like." Paul Edwards, ed., Encyclopedia of Philosophy (New York: Macmillan, 1967), 8: 101.

other logical constants? How to represent truth and falsity? How to distinguish particular from general? Reason, of course, cannot proceed without the raw materials, collected for it by vision. However, the role of vision is limited to the passive and unintelligent collection of data (sensations), later to be interpreted by reason. Experimental psychology, emerging from empiricist philosophy in the nineteenth century, gave additional support to this conceptual hierarchy by providing a topological scheme: the outside world enters through the filters of the eyes and then is carried through visual pathways inside the head, to the brain, the site of "higher mental faculties." Even today, psychological textbooks are organized around the same hierarchy, inherited from empiricist philosophy: they start with chapters on sensation (vision and other senses) and end with cognition (thinking, memory, text comprehension, etc.)

These are some of the ways in which philosophy has demoted vision in relation to reason: it can't represent logical forms; it can't generalize; its only job is to collect the raw material reason needs.86 However, this philosophical tradition is counterbalanced and upset by other traditions. For, while subordinating vision to reason, philosophers also often claimed that reason in fact was born of vision. Moreover, it can be said that they could only conceive of reason in visual terms. For instance, while in terms of the temporality of everyday mental functioning, vision serves reason, its task being simply the collection of data for reason to work with, in terms of a much longer temporality of phylogenetic development of the human race, vision is often postulated as reason's origin. In particular, Locke, Herder, Vico, Nietzsche, and others claimed that abstract categories of language evolved from concrete visual (and also bodily) experience. Locke: "I doubt not but, if we could trace them [words] to their sources, we should find, in all
86 In his Downcast Eyes, unavailable to me at the time of this writing, Martin Jay discusses the

role of vision in modern French philosophy. Martin Jay, Downcast Eyes: the Denigration of Vision in Twentieth-century French Thought (Berkeley: The University of California Press, 1993).

languages, the names which stand for things that fall under our senses to have had their first rise from sensible ideas."87 Nietzsche: "Everything which makes man stand out in relief against the animal depends on this faculty of volatilizing the concrete metaphors into a schema and therefore resolving perception into an idea."88 Vico: "It is noteworthy that in all languages the greater part of the expressions relating to inanimate things are formed by metaphor from the human body and its parts and from the human senses and passions."89 Vico's ideas, in particular, have been recently revived in the work of the linguist George Lakoff. Lakoff argues that most, if not all, abstract semantic concepts and abstract human reasoning are based on metaphorical mapping of spatial concepts (such as inside-outside, partwhole, and others) which are inherently meaningful to human beings since they have bodies. Like Vico before him, Lakoff challenges three traditional hierarchies -- between concrete and abstract thought, between vision and reasoning, and between literal and metaphorical meanings. In contrast to the objectivist-computational paradigm, which underlies much of Anglo-American philosophy of language and cognitive science, in which thinking is conceptualized as the algorithmic manipulation of symbols meaningless in themselves, Lakoff claims that in thinking we project the structures of our sensory and bodily experience to abstract

87 Qtd. in Tzvetan Todorov, Theories of the Symbol (Ithaca: Cornell University Press, 1984),

88 Friedrich Nietzsche, "On Truth and Falsity in their Ultramoral Sense," in Early Greek

Philosophy and Other Essays (New York: The Macmillan Company, 1924), 181. 89 Thomas Goddard Bergin and Max Harold Fisch, The New Science of Giambattista Vico (Ithaca: Cornell University Press, 1968), 129. Discussing the metaphorical origin of abstract concepts, Vico goes on to claim that the common assumption "that prose speech is proper speech, and poetic speech improper" reverses the true history of human mind: "first nations" have spoken and thought in poetry, i.e. through the metaphorical tropes (131).

situations.90 The semantics of natural language is built upon the meaningful "image-schemes" which are the generalizations of what Lakoff calls our "basic human experience." We live inside our bodies; our bodies consist of parts which are linked. Therefore, we conceptualize all situations in terms of containers (for instance, an enormous number of expressions use in and out), part-whole relations (divorce is splitting up) and links (we make connections, break relations). For Lakoff, metaphors such as these are not the exception but the norm in language,


George Lakoff, "Cognitive Linguistics,"

Versus 44/45 (1986): 119-154.

which involves the extension of the bodily and sensory experience to the domain of abstract concepts. Because spatial (more precisely, topological) relations are an important part of our experience, spatial metaphors are at the core of the semantics of human languages; reaching goals, moving out, walking over an enemy, etc. Not only does the semantics of natural languages turn out to be based on spatial metaphors but abstract reason itself is the manipulation of spatially represented information: "natural language reasoning makes use of at least some unconscious and automatic image-based processes such as superimposing images, scanning them, focusing on part of them, etc."91 In fact, reason is a special case of more general mental operations which are needed for perception: "Abstract reasoning is a special case of image-based reasoning. Image-based reasoning is fundamental and abstract reasoning is image-based reasoning under a metaphorical projection to an abstract domain."92 With this claim, Lakoff transcends the earlier views of Locke, Vico, and Nietzsche according to whom abstract reason phylogenetically evolved from vision. It is one thing to suggest that abstract concepts have evolved from "sensible ideas" (Locke) or perceptions (Nietzsche). It is quite different to start with the assumption, as Lakoff does, that abstract ideas are nothing but a limited set of visual (more precisely, topological) forms (such as inside-outside, part-whole) extended into the abstract domain. Here the modern obsession with the visualization of reasoning leads to the most dramatic break with the traditional conceptualization of the relation between reason and vision. Modern writers such as Lakoff or Johnson-Laird, claim is that there is only one kind of cognitive operations, which reason and vision share. These operations are fundamental, original to vision: reason borrows these operations; it does not invent any of its own.
91 Ibid., 149. 92 George Lakoff, "The Invariance Hypothesis: Is Abstract Reason Based on Image-Schemas?"

Cognitive Linguistics 1, no. 1 (1990), 65. Perhaps even this very brief presentation of Lakoff's ideas already makes apparent a substantial problem of his theory -- the rather unquestioned notion of "basic human experience."

Johnson-Laird has proposed that when we are engaged in reasoning, we construct a mental model of a situation which represents its essential features in the form of topological relations.93 The conclusion then simply can be "read off" from the representation. For instance, how can we see the truth of a modus ponens: "If X is an A, and if all A's are B's, then X is a B"? We construct a mental model: X is inside A; A is inside B. If the model is mentally "scanned," the right conclusion is invariably arrived at. According to Johnson-Laird, we don't have to


Philip Johnson-Laird, Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness (Cambridge:

Cambridge University Press, 1983).

assume the existence of any special rational logical competence unique to humans. Logical reasoning is just one consequence of vision. But do these views of Lakoff and Johnson-Laird really represent a break? Perhaps this revolt against the condemnation of vision in the philosophical tradition was prepared by this tradition itself? For, while declaring vision unsuitable for reason, philosophers conceived of reason in visual terms all along: witness the persistence of the metaphors of eye, light, mirror, revelation and unveiling, from Plato to Descartes, Hegel, Heideger, and Lacan as standing in for logos, knowledge, and truth.94 "Idea" comes from the Greek verb "to see" and is frequently linked to ancient optics and theories of perception. According to Aristotle, in thinking ideas function to substitute for the absent objects as their images: "The reasoning mind thinks its ideas in the form of images; and as the mind determines the objects it should pursue or avoid in terms of these images, even in the


For gender implications of this tradition, see Evelyn Fox Keller and Christine R. Grontkowski, "The Mind's Eye," in Discovering Reality: Feminist Perspectives on Epistemology, Metaphysics, Methodology, and Philosopohy of Science, ed. Sandra Harding and Merrill B. Hintikka (London: D. Reidel Publishing Company): 207-224. For a discussion of Lacan's concept of the mirror stage in relation to the Western philosophical tradition with its concepts of knowledge and subjectivity grounded in vision see "The Statue Man," in M. Borch-Jakobson, The Absolute Master (Stanford: Stanford University Press, 1991).

absence of sensation, so it is simulated to action when occupied with them."95 For Berkeley and Hume thinking involved a sequential series of images; these images are tied to certain habits of the mind to move from one image to another. Reason was equated with an eye which compared all objects of thought laid out before it. As summarized by Jonathan Crary, But whether it is Berkeley's divine signs of God arrayed on a diaphanous plane, Locke's sensations 'imprinted' on a white page, or Leibniz's elastic screen, the eighteenth century observer confronts a unified space of order...on which contents of the world can be studied and compared, known in terms of a multitude of relationships. In Rorty's words, "It is as if the tabula rasa were perpetually under the gaze of the unblinking Eye of the Mind..."96

95 Qtd. in Jean Matter Mandler and George Mandler, eds., Thinking: From Association to

Gestalt (New York: John Wiley & Sons, Inc., 1964), 9. 96 Jonathan Crary, Techniques of the Observer: on Vision and Modernity in the Nineteenth Century (Cambridge: The MIT Press, 1990), 55.

It was Heidegger who, in What is Called Thinking?, insisted on the visual basis of Western philosophy (both epistemology and ontology) from Plato to Hegel most strongly.97 "The Greeks...conceived knowledge as a kind of seeing and viewing...Because Being means presence and permanence, 'seeing' is especially apt to serve as an explanation for the grasping of


Martin Heidegger, What is Called

Thinking? (New York: Harper & Row, Publishers,


what is present and what is permanent."98 For Plato, Being is idea, present as outward appearance. However, already in Plato the seeds of the split between object and subject are implanted: Being is presence but at the same time is what man brings before his eyes. From this moment on, idea is gradually transformed into perception and representation, that, according to Heidegger, characterizes the Cartesian cogito. Descartes laid a new foundation in privileging the thinking subject, but the certitude of the cogito derives from its visibility: as Heidegger explains, cogitatio means Vor-stellung, i.e. "the bringing-before-itself and what-is-brought-before-itself and made 'visible' in the widest sense."99 Indeed, Cartesian rationality is based on the notion of a subject which is outside himself, observing himself contemplating the subject. This is the price Descartes pays for his condemnation of images and imagination in relation to reason: he can only conceive of reason in visual terms. Before "I think therefore I am" comes "I see therefore I think."

98 Qtd. in Borch-Jakobson, The Absolute Master,

99 Qtd. in Ibid., 54.

Like Siamese twins, vision and reason are inseparable in classical philosophy, being connected in a complex network of dependencies, only a few of which have been discussed above. On the one hand, vision is but a servant of reason, unable to represent logical relations and abstract concepts and therefore, clearly unsuitable for reasoning. On the other hand, vision achieves its revenge, consistently slipping in as the phylogenetic origin of reason and as the basis of metaphors in which operations of reason are conceived. Regardless of whether Heidegger's brilliant account of some of these dependencies represents a break with this classical tradition or is still a part of it, he clearly lives in a different age. Vision and reason are no longer thought of as antagonists; on the contrary, "visual reasoning" is now taken for granted and even promoted as a new highly efficient means of decision making and/or propaganda, suitable for the age of mass mobilization, mass education, industrialized science, and bureaucracy. Heidegger is a contemporary of Eisenstein who, in line with Stalin's plans to industrialize Russia overnight, dreams of turning workers into dialectical thinkers by passing them through the darkness of special incubators of cognitive skills -- movie houses, thus shortening the duration of the learning process to hours. Heidegger is also a contemporary of Benjamin who wants to teach German workers dialectical thinking even without any external aids (such as film) -- by simply juxtaposing images, in montage-like fashion, in the mind. Finally, Heidegger's younger compatriot, Rudolf Arnheim, becomes the most outspoken champion of thinking in images, arguing that artistic skills should become the

3. First Signs of Revolt: Venn and Galton

basis of all education, since all, even the most abstract reasoning, involves representation of information in visual form.100 The signs of the arrival of the new age of visual reasoning start to appear in the last decades of the nineeteenth century. In 1880 English logician John Venn published an article "On the Diagrammatic and Mechanical Representation of Propositions and Reasoning."101 The

100 See

Arnheim, "A Plea for Visual

101 Martin Gardner, Logical Machines and Diagrams, 2nd ed. (Chicago: The University of

Chicago Press, 1982), 32.

article presented a method to use graphic diagrams to represent complex logical relations and it soon became the standard for logicians. Venn's method not only made it possible to illustrate the rules of inferential logic in visual forms; but much more importantly, now one could reason automatically by simply drawing pictures (fig. 6). Consider, for instance, the following inference: "If all A is contained in B, and if C is contained in A, then C must also be contained in B." Its validity becomes obvious once the terms are represented in a diagrammatic form. Likewise, such inferences as "All men are mortal. Socrates is a man. Therefore, Socrates is mortal" can be arrived at through similar diagrams. Images thought to be unsuitable for representing logical relations and thus unsuitable for reasoning turn out to be its perfect indeed its natural medium since they can represent topological relations with sufficient precision and complexity. If Venn's diagrams showed that vision can be used to arrive at logical relations, Galton's composite photographs proved that abstract concepts can be represented visually. Sir Francis Galton, a statistician and a cousin of Charles Darwin, a founder of eugenics (a project of social betterment through planned breeding), and the author of highly influential psychological texts, pioneered in 1877 a procedure of making composite photographs which proliferated widely in the next three decades.102 Fabricated by a process of successive registration and exposure of portraits onto a single plate, Galton's composites were thought to constitute true statistic averages, representing human types -- a criminal, a prostitute, an Englishman, a Jew, and others (fig. 7). Galton wrote about his composite pictures that they are "much more than averages; they are rather the equivalents of those large statistical tables whose totals, divided by the number of classes and entered on the bottom line, are the averages. They are real generalizations, because they include the whole of the material under consideration."103

102 Allan Sekula, "The Body and the Archive," October 39 (1987): 40. 103 Qtd. in Ibid., 47.

Galton not only claimed that "the ideal faces obtained by the method of composite portraiture appear to have a great deal in common with...so-called abstract ideas" but in fact he proposed to rename abstract ideas "cumulative ideas." In contrast to the human mind, "a most imperfect apparatus for the elaboration of general ideas," Galton championed his composite photographs, which, being mechanical and precise, were much more reliable for arriving at abstract representations.104 With his photographs, Galton not only proposed that universals may be represented through generic images; he actually objectified and materialized them. Plato's ideas were given concrete form: they could now be touched, copied, fabricated, multiplied, distributed, etc. If Venn's and Galton's inventions overturned the traditional claims about the impossibility of visual reasoning, they did that not through new philosophical arguments about vision but first of all by inventing new ways to use it. The latter invented a new visual technology: representing universals through composite photographs. The former introduced a new visual technique, a new way to employ visual representations, better yet, a new graphic sign system -- diagrams which signify not any concrete objects, but logical forms.105 In effect, Venn's technique and Galton's technology, which gave logical forms and abstract ideas material tangible form, externalized reasoning. What before was a mental process, a uniquely individual state, now became part of a public sphere. Unobservable and interior processes and representations were taken out of individual heads and put outside -- as drawings,

104 Qtd. in Ibid., 51. 105 I should clarify the distinction between the terms visual techniques and visual technologies as they are used in this chapter. By "techniques" of visual representation, I mean particular graphic and/or pictorial signifying systems which do not depend in any crucial way on any one medium or technological base. For example, Euler's diagrams can be drawn on paper, etched on metal, generated with computer graphics, etc. The term "visual technologies" refers to certain representational devices, which do depend on the particular technological configuration and would be hard or impossible to achieve without it. The examples include dissolve or traveling shots in film, which are only possible given a sequence of images in time generated by a film projector, video tape player or computer. Galton's composite photography is such visual technology, dependent, both in its design and its artifacts, on particular photographic technology.

photographs and other visual forms. Now they could be discussed in public, employed in teaching and propaganda, standardized, and mass-distributed. What was private became public. What was unique became mass-produced. What was hidden in an individual's mind became shared.

We should not be surprised, then, that the revolutionary new medium, the medium of mass society par excellence -- cinema -- was immediately proclaimed to be the machine for the externalization of private mental functions and states. In 1916 Hugo MŸnsterberg, a Professor of Psychology at Harvard University and one of the founders of the fields of industrial and applied psychology, published The Film: A Psychological Study, today canonized as one of the earliest

4. "To teach the worker to think dialectically."

theoretical treatments of cinema.106 According to MŸnsterberg, the essence of the new medium lies in its ability to reproduce, or "objectify" various mental functions on the screen: "The photoplay obeys the laws of the mind rather than those of the outer world."107 In contrast to the theater, where the action is constrained by the limitations of physical reality, film is free to shape arbitrarily its material, closely approximating flashes of memory, the flights of imagination, and


Hugo MŸnsterberg, The Photoplay: A Psychological Study (New York: D. Aplleton & Co., 1916).

107 Ibid., 41.

other mental acts. For instance, while in theater events have to follow each other corresponding to the progression of time, in film the action can suddenly jump back and forth, just as in an act of imagination. MŸnsterberg was not content to point out the analogy between film and mental life; in an astounding analysis, he correlated the main cinematic techniques to different mental functions such as attention and memory, one-to-one. For example, in the close-up, "everything which our mind wants to disregard has been suddenly banished from our sight and has disappeared," analogous to how our attention selects a particular object from the environment. Similarly, the "cut-back" technique objectifies the mental function of memory. "In both cases," MŸnsterberg wrote, "the act which in the ordinary theater would go on in our mind alone is here in the photography projected into the pictures themselves."108 The psychological laboratory became indistinguishable from the movie house; the textbook of experimental psychology -- from the cinematographer's manual. The mind was projected on the screen; the inside became the outside. MŸnsterberg admired the power of film to externalize the functions of consciousness. The next logical step was taken by a German psychologist Kurt Levin who, in 1924-25, was the first to use film in his experiments. He wrote that "fiction film attempts to objectify certain psychological processes for the viewer. Psychological (scientific) film studies to what extent these psychological processes can be objectified."109 Soviet psychologist A.P. Luria, who planned to establish a psychological laboratory in Moscow in cooperation with the State Film Academy, acquainted Levin with Eisenstein, who attended the shooting of one of Levin's films and advised him.110
108 Ibid., 41. 109 Qtd. in Olga Bulgakova, "Sergei Eisenshtein i ego 'psikhologicheskiy Berlin' -- mezhdu

psikhoanalizom i strukturnoy psikhologiey" (Sergei Eisenstein and his 'psychological Berlin': between psychoanalysis and structural psychology), Kinovedcheskie Zapiski 2 (1988): 187. 110 Ibid., 177.

The figure of Eisenstein is particularly important because it reveals the historical connection between the interest in visual reasoning and the rise of mass communication, of which film was a major vehicle. The emergence of new mass societies in the earlier part of this century dictated the necessity to communicate ideological concepts to mass populations which were often illiterate.111 In the 1920s Eisenstein boldly conceived a screen adaptation of Marx's Capital as a way to efficiently bring about the political enlightenment of Russian audiences, especially the peasants who would not sit through a political lecture but, attracted by the "novelty" of a movie projector, would come to see movies, regardless of what was shown.112 Unprecedented as his project was, its radicalism lay not only in the decision to visualize the abstract notions and logic of Capital but in the method employed, which, according to Eisenstein, would directly provoke dialectical thinking in audiences.113 Jacques Aumont concludes that for Eisenstein, "the object privileged in Marx's work is not a theoretical one, like any of the key concepts from Capital. It is at another level entirely that Eisenstein selects his true object -- the Marxist method itself."114 Thus it was not simply a matter of the modern redeployment of the directions of the 1492 sermon: "...Images of the Virgin and the Saints were introduced...on account of the ignorance of simple people, so that those who are not able to read the scriptures can yet learn by seeing the sacraments of our salvation and faith in pictures."115 The viewers of Capital were not only to learn the scriptures of the new atheistic religion; they were to learn the process of reasoning.

111 For instance, according to the 1926 census, out of every 1,000 citizens of the U.S.S.R., only 445 were literate. Peter Kenez, The Birth of the Propaganda State. Soviet Methods of Mass Mobilization, 1917-1929 (Cambridge: Cambridge University Press, 1985), 157. 112 Kenez, The Birth of the Propaganda State, 220. 113 The pioneering work of Annette Michelson was important in bringing my attention to Eisestein's Capital project. Annette Michelson, "Reading Eisenstein Reading 'Capital'," October 2 (1976): 27-38; October 3 (1977): 82-88. 114 Jacques Aumont, Montage Eisenstein (London and Bloomington: BFI Publishing and Indiana University Press, 1987), 163. 115 Qtd. in Baxandall, Painting and Experience in Fifteenth Century Italy, 41.

It is significant that the most categorical statement by Eisenstein on the possibility of "filmic reasoning," reasoning through images, appears in the context of his discussion of the sequence known as For God and Country from October (1929): Maintaining the denotation of "God," the images increasingly disagree with our concept of God, inevitably leading to individual conclusions about the nature of all deities...a chain of images attempted to achieve a purely intellectual resolution, resulting from a conflict between a preconception and a gradual discrediting of its purposeful steps. Step by step...power is accumulated behind a process that can be formally identified with that of a logical deduction...The conventional descriptive form for film leads to the formal possibility of a kind of filmic reasoning. While conventional film directs emotions, this suggests an opportunity to encourage and direct the whole thought process as well.116 Far from simply representing God or deities, as they did for centuries, here images serve a totally new function -- to provoke and direct reasoning, reasoning of a particular kind -- "Marxist dialectics." In accordance with its principles, as canonized by the official Soviet philosophy, Eisenstein wants to present the viewer with the visual equivalents of thesis and anti-thesis so that the viewer can then proceed to arrive at synthesis, i.e. the correct conclusion, pre-programmed by Eisenstein. "The content of CAPITAL (its aim) is now formulated: to teach the worker to think dialectically," Eisenstein writes enthusiastically in April of 1928.117 Schooled by the film, viewers would become self-sufficient thinkers, learning the skill of "Communist decoding of the world," each a walking camera, snapping pictures of visual thesis and anti-thesis, the brain automatically executing cognitive operations of montage, thinking through images, efficiently and effectively. Eisenstein claims the radical novelty of his concept of "filmic reasoning":

116 Sergei Eisenstein, "A Dialectical Approach to Film Form," in Film Form: Essays in Film

Theory, ed. Jay Leyda (New York: Harcourt Brace and World, 1949), 62. Emphasis mine -- L.M. 117 Sergei Eisenstein, "Notes for a Film of 'Capital,'" trans. Maciej Sliwowski, Jay Leuda, and Annette Michelson, October 2 (1976): 10.

The proclamation that I'm going to make a movie of Marx's Das Kapital is not a publicity stunt. I believe that the films of the future will be found going in this direction (or else they'll be filming things like The Idea of Christianity from the bourgeois point of view!) In any case, they will have to do with philosophy...the field is absolutely untouched. Tabula rasa.118 Yet, Einstein's theory was not an isolated development. Many in the artistic left of the 1920s shared a similar belief in the cognitive power of new visual forms such as montage. In the late 1920s Alexander Rodchenko promoted the use of montage sequences in graphic design and, like Eisenstein, he saw montage as being equivalent to "dialectical" reasoning. In this formulation, an individual image corresponded to a single concept, and thinking was thought to be provoked when a number of images were juxtaposed in a series.119 Walter Benjamin's notion of "dialectical seeing," central in his unfinished Passagen-Werk project, also depends on montage, but within a single frame, so to speak. "Dialectical seeing" was conceived by Benjamin as a way to grasp the forms of the present by looking to the past and to the future in the same instant, juxtaposing them in the same mental image.120
118 Michelson, "Reading Eisenstein," October 2: 28. 119 While at first Rodchenko practiced juxtaposition of many separate photographs and

fragments within the space of a single image, at the end of the 1920s his photomontages became multi-page layouts composed of a number of more "traditional" photographs, more like a film montage sequence. 120 On Benjamin's concept of "dialectical seeing" see

Susan Buck-Morss, The Dialectics of

Venn, Galton, MŸnsterberg, Eisenstein. Logical diagrams, composite photographs, cinematic devices of close-up, cut-back, and montage. Different, in effect, as they are, these developments are symptoms of a single social imaginary at work: to make reasoning more efficient and at the same time more controllable by externalizing it and rendering it visible. This imaginary can be glimpsed in the new techniques of visual representation, such as Venn's diagrams. It also governs the development and the promotion of new visual technologies, such as Galton's composites or Eisenstein's dialectical montage. Finally, this imagination can be also read in theoretical speculations about how abstract concepts and logical forms can be represented as images in principle, such as in Freud's psychoanalysis. Is there a single person whose work brings into focus the modern preoccupation with visual reasoning -- whether in the form of new theoretical models, new techniques of visual representation or new visual technologies? It is not Venn, or Galton, or MŸnsterberg. This person must be Sigmund Freud. He is not only more explicit in his defense of the possibility of visual reasoning; he develops a systematic theory of how abstract notions and logical forms can Seeing: Walter Benjamin and the Arcades Project (Cambridge: The MIT Press, 1989).

be visualized. This theory becomes one of the cornerstones of a new science, devoted, we should note, to externalization, bringing into the open, into public view the most intimate, private, and unobservable psychic phenomena -- the unconscious ideas, unreachable for consciousness. In this, the new science goes well beyond MŸnsterberg's notion of the externalization of memory and attention by the techniques of film -- after all, these mental functions can be voluntarily controlled and commanded by consciousness. Indeed, MŸnsterberg and his fellow experimental psychologists at the turn of the century scrutinized the mind through the technique of introspection -- which, by default, limited them to the study of only those mental acts and functions that could be voluntarily controlled and consciously observed. Instead, the new science postulates the existence of unobservable, uncontrollable psychic forces, which, lying deep below the level of consciousness, nevertheless control it. The latter hypothesis, it is often said, constitutes the most dramatic attack on Cartesian rationality launched in this century. However, it should be noted for the record that the new science also gives rationality and reason even more power than before -- and not only because it tamed the unconscious for the consciousness but also because it gained another ground for reason -- vision. Having postulated the existence of subterranean unconscious ideas and unconscious reasoning, this science immediately proposes a technique by which these phenomena can be brought to the surface, and rendered visible and harmless. It claims that unconscious ideas, groups of ideas, and even arguments are transformed into the form of dreamimages. By analyzing these images, an analyst can recover these ideas and arguments.

5. Freud's Theory of Visual Reasoning

Freud's The Interpretation of Dreams, published in the first year of this century, inaugurated the rise of visual reasoning.121 This work exemplifies a new understanding of vision -- as not inferior to reason but, on the contrary, as its equal, as a medium quite capable of representing abstract ideas and logical arguments. According to Freud, dreams can translate into visual form such concepts as "a high-placed official" (Freud 378) or "superfluous" (441). They can also represent logical connection (349), causality, negation (361, 372) or contradiction (470). In short,
121 All subsequent refererences in this chapter to Freud, unless overwize indicated, are to

Sigmund Freud, The Interpretation of

Dreams (New York: Avon Books, 1965).

the whole range of reasoning operations can be encoded in visual images. With this theory, vision is completely subsumed into the realm of logical thinking. According to Freud, the mind's contents, ideas and logical arguments, are securely hidden from its owner, from the psychoanalyst and from the public, lurking, invisible and unreachable, in the archaic cave of the unconscious. They travel through the darkness of association networks, like newly formed urban masses, riding to and from work in the underground subways, which in the end of the nineteenth century were rapidly constructed in major metropolitan centers. Like these urban masses, always a danger, potentially lurking in the darkness of the labyrinths of city streets at night, ideas are formed, dissolved and rearranged in the total darkness, which the limited "spotlight of attention" (Wundt) cannot reach. This, of course, cannot be tolerated. Streets are streamlined and, even more importantly, well lit. Indeed, the turn of the century is often called "the age of electricity" due to its obsession with electrical light. But does not this light provide the best way of fighting invisibility and assuring externalization? Similarly, ideas and reasoning need to be externalized, brought into the open, rendered visible. Luckily, Freud assures us, the psyche already possesses a process which transforms the invisible contents of the unconscious into images within our reach. All the psychoanalyst needs to do is to properly interpret these dream images by tracing them back to their underlying ideas. So how can dreams represent ideas visually? In the section "Consideration of Representability," after he establishes the requirement for visualization -- "considerations of representability in the peculiar psychical material of which dreams make use -- for the most part, that is, representability in visual images," (379) Freud provides a number of examples of how dreams transform abstract concepts and thoughts into pictures.122 Here are a few of them:

122 Mieke Bal's seminar on "Interpreting Culture" at the University of Rochester brought my attention to this point. For the original discussion of considerations of representability see Samuel Weber, The Legend of Freud (Minneapolis: University of Minnesota Press, 1982).

"A high-placed official" is represented by a high tower (378). "Working my way through" is represented by "a long knife under a cake, as though to lift out a slice" (380). "Superfluous" is represented by water everywhere -- "overflowing," "flowing over" (441). Legs (part of the body) are represented through pillars or columns (382).

Is there a single logic these examples follow? Yes, and we have already encountered it in Lakoff. Lakoff, as we have seen, claims that abstract concepts and abstract reasoning take shape in the process of extending the structures of human sensory and bodily experience to abstract situations; and that even the most abstract categories are metaphors, based on such topological relations as part-whole and inside-outside. To use Freud's examples, such concepts as "a highplaced official" or "working my may through" are meaningful in the first place because they involve the conceptualization of abstract situations ("position of people in society"; "solving a problem") in spatial terms. Lakoff postulates that behind all seemingly abstract concepts of natural language lie concrete "image-schemes." If we accept Lakoff's theory for a moment and read Freud's examples through its terms, it appears that in dreams abstract concepts and ideas turn back into the "imageschemes" from which they evolved in the first place. Thus, "a high-placed official" becomes again a concrete spatial representation, "high tower." Of course, the point is not that Lakoff's theory "explains" Freud or that Freud "anticipates" Lakoff; rather, it is that both implicitly rely on the notion which has always figured centrally in the speculations about the origin of language and reason (including, as we have seen, of such thinkers as Locke, Herder, Vico, and Nietzsche): that abstract concepts have evolved

from concrete ones, the latter directly connected with perceptual experience. Freud, however, adds a new and important notion: regression. If in dreams abstract concepts and abstract thinking itself return to their "origin" as images and "image-based reasoning" (Lakoff's term) this transformation would be in perfect correspondence with the regressive character of dreams. The notion of regression is the key for understanding Freud's theory of visual thinking. Not only does it explain why in dreams abstract concepts return to their visual origins, it also partly clarifies why dreams invariably rely on visual means of representation at all: "what is older in time is more primitive in form and in psychical topography lies nearer to the perceptual end" (587). Regression to the visual mode in dreams, then, operates in three ways: regression to an older representational mode (formal regression); topographic regression (to the perceptual system); and temporal regression to the older (in the evolutionist sense) psychical structures. There is also another, fourth meaning of regression to the visual mode: regression to the visual scenes witnessed in childhood ("primal scene"). Therefore, the visual means of representation employed by dreams are characterized by Freud as an archaic, "primitive mode of representation and expression" (587). Vision, in Freud's diagnosis, is seen as the primordial, original language of mankind. Should we be surprised that, according to Freud, Vienna's inhabitants circa 1900, modern and respectable by day, at night totally lose their modernity, shed away progress, and instead immerse themselves in archaic visual illusions? No, if we recall the view, frequently voiced in the earlier part of the twentieth century, that the new visual forms of mass culture represent a return, indeed, a regression to humanity's archaic stage. Most frequently, it was cinema which was seen (for instance, by Nabokov) as a time machine, where the metropolitan masses, stripped of their individuality, subjectivity, or even simply basic interiority, in total darkness (like that of prehistoric caves, even darker than the cave described by Plato), gave up the light of Enlightenment, Civilization and Reason, and instead were chanting in unison...a collective body,

mesmerized by visual forms, moving before them on the screen, made possible through the magic of the cinematograph. As his contemporaries, Freud believes that the visual cinematograph of dream-images represents the regression to the original language of mankind -- the language of pictures; however, he does not think that this entails the loss of rationality, logic, and reason. Freud not only claims, as we have just seen, that the dream-work has the means to represent singular ideas, even abstract ones, but he also confidently describes the mechanisms by which the dream-work visualizes the logical relations between dream-thoughts. "What representation do dreams provide for 'if,' 'because,' 'just as,' 'although,' 'either-or,' and all the other conjunctions without which we cannot understand sentences or speeches?," asks Freud. His initial answer is negative: "In the first resort our answer must be that dreams have no means at their disposal for representing these logical relations between the dream-thoughts" (347). But astonishingly, contradicting his own statement, Freud then proceeds to elaborate the mechanisms by which the whole range of logical relations can be represented in images:

Logical connection is represented by simultaneity in time (349). Spatial proximity is used to represent logical relation: "Whenever they [dreams] show us two elements close together, this guarantees that there is some specially intimate connection between what corresponds to them among the dream-thoughts" (349). Causality is reproduced by a time sequence (349) or by a metamorphosis. An alternative ("either-or") may be represented by dividing the dream into two pieces of equal length (353). Negation is usually discarded (353) -- but later Freud gives examples to the contrary (361, 372).

Similarity ("just as") is the "favorite" of dreams -- "this relation, unlike any other, is capable of being represented in dreams in a variety of ways" (354), including "composition" -- the creation of composites. In principle, composition is similar to metamorphosis: "The psychical process of constructing composite images in dreams is evidently the same as when we imagine or portray a centaur or a dragon in waking life" (359). "The temporal repetition of an act is regularly shown in dreams by the numerical multiplication of an object" (407). Contradiction is represented by absurdity (470).

Freud does not explicitly connect his descriptions of the dream-work's mechanisms of the visualization of logical relations with his view of the dream-work as regression to archaic mental structures. However, his characterization of these mechanisms only makes sense if we assume that, like everything connected with the dream-work, they owe their character to regression. The connection is provided in the work of another writer, Lucien LŽvy-Bruhl, a leading French anthropologist and Freud's contemporary. Like Freud, LŽvy-Bruhl was concerned with the nature of archaic mental structures. But while Freud located these structures in the unconscious of every modern civilized individual, LŽvy-Bruhl claimed to have found them in the mental language of present-day "savages." In a number of books LŽvy-Bruhl advanced the notion of "primitive" logic which enjoyed wide popularity in the first decades of the twentieth century. According to LŽvy-Bruhl, the thinking of primitive people follows a logic quite different from our own "normal" logic. First of all, primitive people confuse temporal succession with causality. If they witness the two events following each other, they assume that the two are causally related. Thus, LŽvy-Bruhl recounts that if one of them sees a snake fall down from the tree in front of him and learns soon

thereafter that his son is dead, he will relate these two facts.123 Second, primitive people believe that if two objects have been physically connected, they continue to influence each other when separated. For instance, some primitive people are careful to hide a babies' milk-teeth -- for if somebody finds them, then the new teeth will grow improperly. Summing up this feature of primitive logic, LŽvy-Bruhl writes: "Primitive people confuse the antecedent with the cause. This would be a very common error in reasoning that is known as post hoc, ergo propter hoc sophism."124 These "laws" of "primitive thinking," as described by LŽvy-Bruhl, are the same ones that Freud claims govern the representation of the relations between dream-thoughts. Savages think that two events following each other are causally related (LŽvy-Bruhl) -- dreams represent causality by temporal sequence (Freud). For savages, two objects physically connected are also logically related -- dreams signify a logical connection by the spatial closeness of elements or by the simultaneity of time (Freud, 349). Moreover, as "savage mind," the dream-work finds similarity everywhere, seeing logical relations where there are none: "One and one only of these logical relations is very highly favored by the mechanism of dream-formation; namely, the relation of similarity, consonance or approximation -- the relation of 'just as.' This relation, unlike any other, is capable of being represented in dreams in a variety of ways" (354). The visual language of the dream-work -- the archaic language of the psyche, as described by Freud - is the same as the primordial "mental language" still at work in modern savages -- as claimed by LŽvy-Bruhl.

123 Lucien LŽvy-Bruhl, Les functions mentales dans les sociŽtŽs primitives (Paris: 1910), 72. 124 Ibid., 73. 6. The Rise of the Diagram

Freud does not just insist that abstract ideas and logic can be visualized or provide a theoretical model of how this can be done in principle. He practices what he preaches. What makes The Interpretation of Dreams a truly modern text, compatible with the age of illustrated magazines, color printing, scientific documentaries, dialectical montage, and other techniques and technologies of visual communication is not the abundance of illustrations or figures (in fact, there are only a couple). Freud does not have to include illustrations, because many of his theories, and indeed his overall theory of the psyche (the famous topographic model), are based on simple visual models, easy to grasp and easy to remember. And this is what makes his text modern. The reader is invited to disregard the complexities of real anatomical data and instead to conceptualize the psyche "as resembling a compound microscope or a photographic apparatus, or something of the kind" (574). According to Freud, this apparatus is made up of just a few components (called by Freud agencies or systems), which are arranged in spatial order (574583). This "spatial analogy" (653) allows Freud to visualize any psychic process as a simple movement between a few parts of the model. It is as though Freud senses that the construction of such models is very effective in pushing the reader to "see" the self-evidence of the appropriate conclusions. Thus, after presenting the typographic model of the psyche, Freud uses it to explain the newly introduced concept of regression: "I believe that the name 'regression' is of help to us in so far as it connects a fact that was already known to us with our schematic picture, in which the mental apparatus was given a sense or direction. And it is at this point that picture begins to repay us for having constructed it. For an examination of it, without any further reflection, reveals a further characteristic of dream-formation" (582. Emphasis mine -- L.M.) To clarify the logical relation between the conscious and the unconscious, Freud again relies on the same technique of constructing a visual diagram. From the point of view of the ontogenetic and phylogenetic explanation of the formation of the psychical apparatus, primary

processes, as their name implies, "are present in the mental apparatus from the first" (642) while secondary processes appear only later and never achieve full domination. Thus, concludes Freud, the primary processes (and corresponding to them the system of the unconscious) constitute the essence of psychical life: "The unconscious is the true psychical reality" (651). To emphasize this point and to make sure the reader remembers it, Freud "visualizes" it in the form of a simple image of a sphere within a larger sphere: "The unconscious is the larger sphere, which includes within it the smaller sphere of the conscious" (651). He then "reads off" his conclusion from this visual model: "Everything conscious has an unconscious preliminary stage" (651). Freud's skillful use of visual models, analogous to Venn's diagrams, to represent the logical relations between the terms in his theories is unusual in that Freud painstakingly describes a visual model verbally, rather than directly presenting the reader (or should we say, the viewer?) of his text with a picture. It is as though Freud fully realized the power of diagrams to represent complex concepts, models and arguments but, circa 1900, he was still hesitant to "vulgarize" his scientific opus with images. In 1933 Freud finally includes a real diagram in The New Introductory Lectures on Psychoanalysis to illustrate the relation between two triads of concepts: id, ego, and superego, and unconscious, preconscious, and conscious. Freud's diagram, which effectively signifies a complex set of relations by just a few lines, is representative of a whole family of new techniques of visual representation, new to the modern period (fig. 8). These techniques have risen to fill what can be called the cognitive needs of modernization: to quickly communicate more and more complicated relations between various entities, to manipulate huge sets of data (whether in science -- which became more and more the problem of managing collected data -- or in the bureaucratic management of people), to depict in manageable forms endless hierarchies of relations -- in short, to represent reasoning visually and to reason with the help of vision. These techniques of visual representation comprise what semioticians call the graphic sign systems --

signifying systems, employing a particular set of graphic conventions to represent particular kinds of objects.125 We have already encountered one of these new graphic systems -- Venn's diagrams. These diagrams rely on the unique property of vision, which, along with touch, as Rudolf Arnheim points out, is "the only sensory medium that conveys such spatial properties as inclusion, overlap, parallelism, size, etc., with some precision."126 This property makes possible the precise representation of the logical relations between classes in graphic form. But, while in principle visual images are capable of representing spatial properties, not every image can effectively do so. For instance, continuously varying tonal gradients are useless, whereas discrete elements are effective. And these elements, to be easily readable, must have enough visual difference among them. Therefore, Venn's diagrams are typically drawn using simple lines. The classes are represented through circles (a representation in the form of rectangles is possible, but it would be ambiguous, leading the viewer to believe that the shapes of figures code some additional logical relation). If circles overlap, the common area (representing exclusive "and") is clearly distinguished through shading or color. In short, we are dealing with a distinct graphic system, employing particular kinds of graphic marks, optimized for the representation of particular references -- a logic of classes. This graphic system accentuates one property of images (the ability to represent spatial properties such as inclusion) at the expense of other properties, for instance, the ability to represent continuously varying quantities through continuously varying tone or color. Venn's diagrams is an example of a specialized graphic sign system. Their use is limited to a narrow professional domain of logic. In the same period, another graphic sign system for visualization of logical relations comes into use (fig. 9). In contrast to Venn's diagrams, the
125 The most comprehensive study of graphic sign systems is Jacques Bertin, Semiology of

Graphics (Madison, WI: The University of Wisconsin Press, 1983). 126 Arnheim, "A Plea for Visual Thinking," 143.

graphic marks of this sign system are now everywhere, seemingly the very air of which the contemporary visual semiosphere is composed. What I have in mind is the gradual emergence of greatly simplified graphic marks which came to form a notation, a sort of an erector construction set, whose parts can be used to quickly assemble a visualization of a network of relations, a representation of a process or a scheme of dependencies. This construction set includes boxes, arrows, vectors, solid and dotted lines, and so on. Arrows can denote such relations as "belongs to," "relates to," "is a member of a class of," "causes," etc. Boxes, on the other hand (or simply any area separated by lines) can represent a physical object, the state of a system, a stage in a process or any abstract entity. Freud's drawing circa 1933, where a few encircled areas together with dotted and solid lines represent a complex relationship between psychical agencies, illustrates the power of this graphic system, which I will call diagrammatic. My comparison of the diagrammatic graphic system with a construction set is not accidental. A construction set includes some parts with more or less fixed functions (for instance, wheels for moving models), while others (bars with holes) can be used for a variety of purposes - to hold other parts, to represent parts of a model, to be stationary or to move. Similarly, while some elements of the diagrammatic system have relatively fixed meanings (for instance, wavy brackets), others, such as arrows or dotted lines, can have numerous meanings, depending on their particular arrangement. This leads us to the second feature. In diagrammatic language, the same few symbols can represent anything: a box, for instance, may stand for an idea, a theory, a building, 200 buildings, a citizen, an army, or a state of a physical system. Unlike the users of verbal language who learn particular semantics, the users of the diagrammatic system learn the general conventions of syntax. It is enough to know that a box represents an entity -- regardless of what the entity may be. In this, diagrammatic language, used to represent a set of entities and the relations between

them, functions exactly like the symbolic language of logic: a properly executed logic diagram is isomorphic with a logical statement.127 But this power is achieved through the standardization of graphical conventions themselves. A diagrammatic system is a metalanguage, whose symbols represent not any particular ideas or objects, but rather the abstract relations evoked in any process of reasoning: "a part of;" "leads to;" "associated with." It is not accidental, therefore, that this language of arrows and boxes is what recent generations of computer programs, aiming to help reasoning or to imitate it (outliners, hypertext, project management tools, idea generators, and the like) provide their users with. The mental construction set became the graphic toolbox, always just a mouse click away. This computer implementation of already existing conventions of visual culture is accompanied by a further standardization of graphic symbols. This standardization is justified by the argument, popularized since the adaptation of Macintosh user interface, that it helps the user to recognize familiar symbols and commands regardless of the program she is using (fig. 10). The standardization of diagrammatic symbols went hand in hand with their simplification. While drawings were used for centuries to represent schematic information rather than simply to symbolize particular objects or to record their shapes, modernization is accompanied by the simplification of graphic symbols -- boxes, arrows, and the like. (This was also accompanied by the streamlining of typography.) Now, it becomes much easier to "read off" logical relations represented by the diagrams, since there are fewer graphical marks to scan. When do the symbols of diagrammatic language, as it exists today, first appear? Obviously, we would not find a sharp "break" in graphic paradigms, with a new graphic system appearing overnight. According to Ernst Gombrich, the arrow does not acquire its meaning as pointer or vector until the eighteenth century. And even then, he could only find it in a couple of specialized professional contexts: it appears in a French treatise on Hydraulic Architecture of
127 Gardner, Logic Machines and Diagrams, 28.

1737 and on some maps, where it was used to show the direction of the river's flow.128 Similarly, we can find other abstract graphic marks employed for centuries in graphic systems used in particular professions -- for instance, engineering drawing. Gradually, however, these symbols leave these professional contexts and enter visual culture at large. Finally, what is the relation between the development of the diagrammatic graphic system and the overall movement in modern visual culture towards the simplification, streamlining, and standardization of all visual forms? In the opening chapter of Techniques of the Observer, Jonathan Crary points out an inherent contradiction present in most standard accounts of modernism. On the one hand, these accounts postulate the emergence of new radical models of perception and representation, summed up in the notion of abstraction, whose first signs appear with Manet, impressionism or post-impressionism and which fully blossoms with Kandinsky, Kupka, Mondrian, and others. On the other hand, it is also postulated that in the same historical period new popular visual forms developed (first photography and then cinema), further solidifying realism. As Crary notes,

128 Ernst Gombrich, "Pictorial Instructions," in Images and Understanding, ed. Barlow Horace,

Colin Blakemore and Miranda Weston-Smith (Cambridge: Cambridge University Press,


, 28.

"Classical space is overturned, so it seems, on one hand, but it persists on the other. This conceptual division leads to the erroneous notion that something called realism dominated popular representational practices, while experiments and innovations occurred in a distinct (if often permeable) arena of modernist art making."129 Crary's way to solve this contradiction is to claim, throughout his book, that the so-called realistic forms of popular visual culture, based on photography, and abstract art are both grounded in the emergence of "subjective vision" in the early nineteenth century, in which the link between representational signs and external referents is severed. "I argue," writes Crary, "that some of the most pervasive means of producing 'realistic' effects in mass visual culture, such as stereoscope, were in fact based on a radical abstraction and reconstruction of optical experience, thus demanding a reconsideration of what 'realism' means in the nineteenth century."130 What his anti-realist argument leaves out, however, is any consideration of the representational power of new visual technologies such as photography. With photography, based on mechanized perspective, the ability to establish a "reciprocal, metrical relationship between the shapes of objects as definitely located in space and their pictorial representations" (William Ivins) became greatly amplified. In Ivins' words, photography is "a form of picturemaking that is not only precisely duplicable but one in which geometrical perspective is so inherent that today the camera is used as a surveying and measuring instrument."131 Rather than being anti-realistic, photography became the perfect technology of instrumental realism (Allan Sekula's term), literally allowing its users to have physical power over the represented subjects and objects. As Sekula points out in his important account of the instrumental uses of photography at the turn of the century, criminal identification photographs, for instance, "are

129 Crary, Techniques of the Observer, 4. 130 Ibid., 9. 131 William M. Ivins, On the Rationalization of Sight (New York: Da Capo Press, Inc., 1975),


designed quite literally to facilitate the arrest of their referent" and, similarly, military reconnaissance photographs make possible identification and destruction of the objects they capture.132 A different way to solve the contradiction, noted by Crary, between the opposition abstraction/high art versus realism/popular visual culture is to point out that in fact the development of abstract visual forms was not limited to modernist art but equally took over the realm of the popular -- first of all, in the adaptation of what I have called here diagrammatic visual language. But in order to see this it is necessary to include in the discussion more than images from the realms of art, or entertainment or leisure -- the realms to which the problematic of high-low still limits itself, and whose borders in Crary's argument are represented by modernist painting, on the one side, and photographs, stereoscopes, phenakistiscopes and other popular nineteenth century optical toys, on the other. Rather, it is necessary to consider the whole range of images employed in modern culture -- images employed not only for leisure but also for work; images through which, literally speaking, modern work gets done: aerial surveillance photographs and criminal identification shots, engineering and architectural drawings and plans, maps and typography, instrumentation displays, and scientific imagery. And, last but not least, we should consider the abstract graphic marks of diagrammatic visual language, designed to facilitate cognitive and communicative workloads of modernity, in the office, at school, in the control room, in the boardroom, and on the assembly line -- which is today replaced by equally monotonous rows of computer terminals, outputting not Model T, but bits of information; this steel and coal of the information -- or shall we better say, cognitive age.

Information age, knowledge economy, post-industrial society. All these terms come to designate the changing nature of society after World War II: the gradual displacement of manual labor by
132Allan Sekula, "The Body and the Archive," 7.

cognitive labor. With the centrality of cognition for the post-industrial workplace, the notion of visual reasoning acquired new importance. Consequently, the work on the models, techniques, and technologies of visual reasoning, which was previously pursued by a few isolated individuals (Galton, Freud, Eisenstein), has now become a matter of systematic research on the industrial scale in such fields as cognitive psychology, scientific visualization, and virtual reality. I will consider this research in the next two sections.

What is the relation between the psychological theories of the mind and their contemporary visual technologies? The recurrent claims that new visual technologies externalize and objectify reasoning, and that they can be used to augment or control it, are based on the assumption of the isomorphism of mental representations and operations with external visual effects such as dissolves, composite images, and edited sequences. This assumption, which I so far have not questioned, on a closer examination appears to be highly problematic. Whatever mental representations and operations really are, the mind surely does not contain pictures, photographs or film clips which some mental homunculus looks at. The external images presented to the mind are not magically transplanted inside it as ready-made ideas and arguments. Regardless of what visual forms can be presented before the eye -- diagrams, photographs, film images -- as any other visual input, they are subjected to the complicated processing by the nervous system which constructs its own internal representations. Yet, the assumption of such an isomorphism continues to persist in modern thinking about vision, ignited by every new round of visualization technology: photography, film, computer animation, and virtual reality. Consider the claims which surround the new field of scientific visualization -- visualization of data sets, their relationships and their dynamic behavior using computer graphics. Richard Mark Friedhoff and
7. Visual Technologies and the Mind

William Benson proclaim that computer visualization techniques constitute the second computer revolution because they act as the direct "extension of preconscious visual processes."133 They assume that the images on a computer screen do not simply function as an aid for reasoning but that they are equivalent to the mental representations the mind may construct while thinking -and this is the source of their power. Or consider the technology, which, even more so than scientific visualization, is seen as capable of completely objectifying, better yet, transparently merging with mental processes -virtual reality (VR). Again, the descriptions of its capabilities do not distinguish between internal mental functions, events and processes, and externally presented images. This is how, according to Jaron Lanier, VR can take over human memory: "You can play back your memory through time and classify your memories in various ways. You'd be able to run back through the experiential places you've been in order to be able to find people, tools."134 Lanier also claims that VR will lead to the age of "post-symbolic communication," communication without language or any other symbols. Indeed, why should there be any need for linguistic symbols, if everybody, rather than being locked into a "prison-house of language" (Jameson), will happily live in the ultimate nightmare of democracy -- the single mental space which is shared by everybody, and where every communicative act is always ideal (Habermas). This is Lanier's example of how post-symbolic communication will function: "you can make a cup that someone else can pick when there wasn't a cup before, without having to use a picture of the word "cup."135 Here, as with the earlier technology of film, the fantasy of objectifying and augmenting consciousness, extending the powers of reason, goes hand in hand with the desire to see in technology a return to the primitive happy age of pre-language, pre-misunderstanding.

133 Friedhoff and Benson, The Second Computer Revolution: Visualization, 13. 134 Druckrey, "Revenge of the Nerds," 9. 135 Ibid., 6.

Locked in virtual reality caves, with language taken away, we will communicate through gestures, body movements, and grimaces, like our primitive ancestors...136 What can one make of this apparently unsound, yet irresistible, assumption of isomorphism between the mental process of reasoning and external, technologically generated visual forms, haunting us at least since the end of the nineteenth century? The conflation of outside and inside is, of course, symptomatic of the desire to project the inside onto the outside, to make it objective and public. But is this all? To really understand the persistance of this assumption, we should turn to the history of ideas about the nature of mental processes. It is well known that technologies have historically provided and continue to provide the models according to which people imagine the mind. In the seventeenth century it was the clock, in the nineteenth -- the motor, in the second half of the twentieth -- digital computers. More precisely, the paradigms were provided not by the technologies themselves, but by theories which made them possible. To take the last paradigm as an example, cognitive psychology, born in the 1950s and gaining prominence ever since, approaches the mind as an informationprocessing system, as software which runs on the hardware of the brain. But what gave cognitive psychology its epistemological basis was not the new technology itself (computers), but the information theory accompanying it. It was this paradigm which substituted the discussions of mind and brain by the notion of "human information processing." And before information

136 At SIGGRAPH 1992, the premier annual conference on Computer Graphics and Interactive Techniques in the U.S., attended by nearly 30,000 people, about a dozen VR exhibits always had long lines of visitors. However, the lines to two of these exhibits were much longer than any of the others. One was called Dome, the other -- Virtual Reality Cave; in both cases, to see the show, the visitors had to go inside some cave-line structures. It did not matter that one of the exhibits featured a scientific visualization display, of interest only to specialists. Clearly, the fact that in order to see the spectacle one had to go inside a dark, cave-like space, different from normal space, provided enough attraction at the end of this century, just as it did at its beginning, when millions flocked into the dark caves of movie theaters.

theory, theories of the mind were influenced by thermodynamics (as in Freud) and mechanics (Hobbes).137 It can be also claimed, however, that human imagination about the mind's operations is limited by the current visual technologies. Lakoff's assertion that "natural reasoning makes use of at least some unconscious and automatic image-based processes such as superimposing images, scanning them, focusing on part of them" (1986: 149) as well as Johnson-Laird's proposal that logical reasoning is a matter of scanning visual models, would have been impossible before television and later, computer graphics. These visual technologies made operations on images such as scanning, focusing, and superimposition seem natural. Even more telling are the models of cognitive psychologists who, in the last two decades, have systematically scrutinized the role played by mental images in reasoning. These models, which define mental images in terms of such characteristics as spatial resolution, speed of access, basic graphic operations (rotation, translation, copy), seem to be describing first of all computer imaging systems. Psychologists argue among themselves which imaging systems better resemble mental processes, but they do not doubt the basic metaphor. As Paul Virilio notes, "now the virtual images of the computer screen seem to confirm not only the existence of certain forms of representation but, more immediately, the objective presence of mental images."138
137 While this theory is well known and widely accepted, other facts suggest that, at least,

sometimes, the influence runs in the opposite direction -- biological and psychological theories of body and mind providing paradigms for theories of mechanisms. For instance, it appears that Norbert Wiener's cybernetics was inspired by the concept of homeostasis developed in biology: "Physiologist Walter B. Cannon viewed the animal body as a self-regulating machine. Building on the work done by Claude Bernard in the nineteenth century, Cannon developed the concept of 'homeostasis' -- the process by which the body maintains itself in a state of internal equilibrium. Cannon's ideas were well known to Norbert Wiener. In fact, Cannon's Wisdom of the Body (1932) may be read as sort of an introduction to Wiener's Cybernetics (1948)." Charles Eames and Ray Eames, A Computer Perspective: Background to the Computer Age (Cambridge: Harvard University Press, 1990), 99. Another example is provided by the turn of Artificial Intelligence (AI) in the 1980s from trying to simulate the disembodied mind to the simulation of a collective of primitive organisms, having the functionality of insects. Drawing directly on research in biology, the researchers in AI hope that intelligence will emerge as a product of the collective behavior of machines simulating simple biological organisms. 138 Paul Virilio, Lost Dimension (New York: Semiotext(e), 1991), 114.

Similarly, in the earlier period, when Freud, in The Interpretation of Dreams, described the mechanisms by which dream-thoughts and the logical relations between them are represented in dreams, he and his fellow psychologists relied on available visual technology for their understanding of the mind. Not surprisingly, Galton's composites, the earliest form of image processing before digital computers, provided a particularly attractive model. Freud compared the process of condensation with one of Francis Galton's procedures which became especially famous: making family portraits by overlaying a different negative image for each member of the family and then making a single print.139 Writing in the same decade, the American psychologist Edward Titchener opened the discussion of the nature of abstract ideas in his textbook of psychology by noting that "the suggestion has been made that an abstract idea is a sort of composite photograph, a mental picture which results from the superimposition of many particular perceptions or ideas, and which therefore shows the common elements distinct and the individual elements blurred."140 He then proceeds to consider the pros and cons of this view. We should not wonder why Titchener, Freud and other psychologists take the comparison for granted rather than presenting it as a simple metaphor -- contemporary cognitive psychologists also do not question why their models of the mind are so similar to the computer workstations on which they are constructed. Thus, not only reason has always been conceived by philosophers in visual terms; as psychologists begin to furiously take on the questions philosophers only wondered about, subjecting mental processes to controlled, scientific, laboratory study, their models begin to reflect, more and more, the external visual forms made possible by whatever visual technology dominates the period. As before, vision is the mirror reason looks into.

139 Sigmund Freud, Standard Edition of the Complete Psychological Works (London: Hogarth

Press, 1953), 4: 293. 140 Edward Bradford Titchener, A Beginner's Psychology (New York: The Macmillan Company, 1915), 114.

Given the reliance of psychological theories of the mind on contemporary visual technologies, is there any "progress" between the turn of the century and today, except that the imagination of contemporary psychologists depends on the more sophisticated visual technologies of computer graphics? In fact, during this century, the assumption of an isomorphism between the mental and the objective became even more prominent; and externalization of reasoning has been taken much further, both technologically and theoretically. On the one hand, the refinement of various medical imagining techniques in the 1980s made possible an increasingly precise imaging of brain activity, including the visualization of reasoning -- in a literal sense. It is now possible to ask the experimental subject to concentrate on solving a problem and to see which parts of the cerebral cortex are active. The question of whether reasoning in fact depends on the operations normally involved in perception becomes more and more the question which, according to a number of researchers, can be answered through experimentation -- it is enough to show that the part of a cortex normally dedicated to the processing of visual information is activated in the process of reasoning.141 More importantly, in their theories, many cognitive psychologists have accepted as given the equivalence between internal mental processes and the operations which can be done with external, objectively existing visual representations and objects. Consider the debates about the nature and role of mental imagery, which have constituted one of the most active areas of research in cognitive psychology in the last two decades.142 On the one hand, there are those (such as Zenon Pylyshyn) who argue that mental imagery simply consists of the use of general thought processes to simulate physical perceptual events. In this view, if the subjects report the presence of mental imagery during reasoning and problem solving, this is simply a side effect, a

141 Martha Farah, "Is Visual Imagery Really Visual? Overlooked Evidence from

Neuropsychology," Psychological Review 95, no. 3 (1988): 307-317. 142 For a summary of different positions, see Ronald A. Finke, Principles of Mental Imagery (Cambridge: The MIT Press, 1989).

by-product of real mental computations which do not involve visual representations. On the other hand, there are those psychologists and neurophysiologists who, through experiments and direct imaging of brain activity, want to prove that reasoning takes place through the construction and manipulation of mental images (Alan Pavio, Roger Shepard, Stephen Kosslyn, Martha Farah). One of the most well-known experiments in defense of the latter view has been done by Roger Shepard and Jacqueline Metzler of Harvard University.143 They presented subjects with


S. Shepard and J. Metzler, "Mental Rotations of Three-dimensional Objects," Science 171 (1971): 701-


pairs of perspective line drawings of three-dimensional forms constructed from small cubes. The subjects' task was to determine whether or not the forms were identical in shape, despite the difference in orientation. Shepard and Metzler have found that the reaction time was proportional to the degree of rotation which is required to bring the two objects into a similar position. These results were taken as proof that in solving the problem, the subjects mentally rotated representations of three-dimensional objects, and that imagined rotations corresponded to actual physical rotations of objects: "Imagined rotations and physical transformations exhibit corresponding dynamic characteristics and are governed by the same laws of motion."144 Thus, a mental process was equated with an operation one would perform with real, objectively existing objects. Other experiments in defense of the position that many kinds of reasoning involve manipulation of mental imagery entail the comparison of abstract qualities. When subjects were asked to recall two animals and to judge which one was larger, the reaction time decreased proportionally to the difference in estimated size. In another experiment, one group of subjects was asked to rate animals in intelligence on a scale from one to ten, while another group had to compare the intelligence of pairs of animals. Again, the reaction time decreased as the distance in rated intelligence increased. It was concluded that when the subjects tried to discriminate between two objects, reaction time was shorter the greater the difference between two objects, regardless of whether the objects were really presented (for instance, two lines of different length), or were imagined (size of animals), or whether the qualities to be compared were abstract (intelligence of animals).145 We shall leave to psychologists the debates whether these and numerous related experiments indeed prove that internal mental processes involve the manipulation of pictures
144 Finke, Principles of Mental Imagery, 93. 145 John Robert Anderson, "Mental Imagery," in Cognitive Psychology and Its Implications, (W.H. Freeman and Company, 1980).

similar in their qualities to real images. But for my purpose it is significant in itself that in imagining what mental processes are like, contemporary psychologists have assumed, without any reservations, an equivalence between the internal and the external, between the mental objects and the real ones.

Modernization, visualization, externalization. In order to externalize the internal, the invisible, it was first equated with the visual. Once this was accomplished, it was simple and only logical to equate the visual inside with the visual outside, the objectivity and standardization of images drawn on a classroom blackboard, on the screen of a movie theater, or, most recently, on the computer terminal.

In this chapter I am concerned with the answer to the problem of how to represent, communicate, induce or perform reasoning more efficiently. This problem first became important with the emergence of mass society which required an efficient means to communicate abstract ideas to large audiences. It became even more important with the shift to post-industrial society when logical reasoning and other cognitive skills attained central economic importance. The answer was to adopt vision as a more efficient medium. It can be perceived in Freud's theoretical speculations of how ideas and logical arguments can be visualized, in the development of new techniques of visual representation (Venn's pictorial logic, diagrammatic graphic system), and in the claims made about new visual technologies (Galton, MŸnsterberg, Eisenstein). Vision, traditionally considered to be inferior to reason, is now seen as the proper, indeed, as the ideal medium for it. And this is not only a question of the efficiency
8. Analog Engine

and effectiveness of communication, although, with visual reasoning, these two demands are fully satisfied. Indeed, in Eisenstein's montage sequences, reasoning is speeded up -- to twentyfour frames a second. And, with pictorial statistics and diagrammatic visual language, reasoning becomes more effective: complex conceptual hierarchies are reduced to boxes and arrows, huge sets of data become marks on paper which can be easily manipulated. But what is equally important is that visual reasoning fits perfectly with the demand of mass society for standardization. The subjects have to be standardized, and the means by which they are standardized need to be standardized as well. Hence the objectification of internal, private mental processes, including ideas and logical connections, and their equation with external visual forms which can be easily manipulated, mass produced, and standardized on its own. Whether we are dealing with the conventions of Venn's diagrams, of signs of diagrammatic sign system, or of cinematic techniques, the private and individual is translated into the public and becomes regulated. This new use of vision as a means of reasoning is accompanied by its rationalization on a conceptual level. If previously vision was thought to be confused, incapable of forming generalizations and unsuitable for representing logical forms, now this is all forgotten. Vision is accepted as being fully capable of representing logical relations (Freud), abstract ideas (Galton), and even dialectics (Eisenstein). This equation of vision with logic, accomplished by the beginning of the twentieth century, is the first stage in vision's rationalization. When industrial society turned into the society of information, computers became the dominant technology, both economically and intellectually, and behaviorism (efficiency of the body) was replaced by cognitive psychology (efficiency of the mind), this rationalization entered into a second stage. Now, with the questions of how information can be coded, stored, retrieved, and processed more efficiently becoming a matter of economic survival, cognitive psychologists, together with computer scientists, began to

scrutinize vision from this new perspective. In the process, they came up with very precise descriptions of what vision is. However, what they describe is not some ahistorical, essential vision, but vision already conceived in a historically and socially specific way -- as an efficient computational system. Why is it that when cognitive psychologists started to wonder about the properties of data structures involved in human reasoning, vision began to take a central place in their models? Because every mental process is conceived as a computation, and it is assumed that this computation has to be efficient. The criterion of efficiency is a very general and pervasive notion in contemporary cognitive psychology. This criterion is the main factor in deciding which of the competing models should be favored. The criterion is justified by a reference to biological evolution -- it is said that in the process of evolution the most efficient method, or the most efficient representational code had to be adopted. However, the real justification of this criterion, in my view, is economic, coming from the close connection between cognitive psychology and its master discipline, to which cognitive psychology ultimately serves -- computer science, and behind it, the computer industry. Just as computer scientists would not want to build an inefficient computer, psychologists cannot imagine an inefficient mental process. Once psychologists realized that images possess some unique properties, which make them a very efficient medium for computation, it was only natural that they entered into their theories. How does Johnson-Laird justify his theory of mental models, according to which logical reasoning involves the construction and manipulation of spatial models, analogous to Venn's diagrams? A model represented in a dimensional space can be constructed, manipulated or scanned in any direction, while propositional representations can only be processed in one direction, making them much less efficient. Why do some cognitive psychologists argue that human long-term memory stores not only knowledge in the form of propositions, but also as images? Because if we store appearances

of objects then later we can compute those properties that could not be anticipated when we saw the objects initially.146 In contrast to propositional (linguistic) representation, images turn out to be capital which can be used over and over in the mind's economy. Finally, why all the debates in cognitive psychology about the role of mental imagery in reasoning in the last two decades? Because many problems can be solved more efficiently by analog rather than digital computer, and vision has unique properties that enable it to function as such a computer: If images are representations in a medium with certain fixed properties, and can be subjected to transformations such as rotation and scaling, then imagery could be used as an analog computer, to solve problems whose entities and relations are isomorphic to objects and spatial relations. That is, certain abstract problems could be best solved by translating their entities into imagined objects, transforming them using available image transformations, detecting the resulting spatial relations and properties, and translating those relations and properties back to the problem domain.147 What are these "certain properties" which make vision into an analog engine? On the one hand, images can represent continuously varying information. On the other hand, images can represent topological relations such as inclusion, "next to," and others. Taken together, these two properties make vision a very powerful computer, better suited for solving many problems than language. As Steven Pinker explains, in a visual medium, "it is impossible to represent the fact that one object is next to another without also committing oneself to which is to the left. (This is not true for abstract propositional representations, where the two-place predicate 'next to' can be asserted of a pair of objects with no such commitment.)"148 In principle, it is always possible to solve analog problems using propositional representations but this simply would not be as efficient. So efficiency becomes the final reason why vision, in spite of the arguments of Descartes, Locke or Leibniz, is today equated with

146 Steven Pinker, "Visual Cognition: An Introduction," Cognition 18 (1984): 65. 147 Ibid., 66. 148 Ibid., 66-67.

reasoning. However, in this equation, vision has little to do with what these philosophers meant by it -- it became an analog code, a "data structure" more proficient for reasoning than language. What remained the same, although recast in new terms, is the idea that vision came before verbal language. If for Freud vision was the original, primordial language of humankind, today cognitive psychologists speculate that the continuous, analog representational system of mental images emerged first, followed by the system for propositional representation (language). I once saw, therefore I think.

The year 1991 saw two events, of different importance and seemingly unrelated. One was the long awaited publication in English of one of the most influential essays of modern art history --

Chapter 3. Mapping 1. Visual Nominalism Space

Erwin Panofsky's Die Perspektive als 'symbolische Form'.149 The interest generated around the re-emergence of this legendary essay, written in 1924-1925,150 demonstrates that the problem of


Erwin Panofsky, Perspective as

Symbolic Form (New York: Zone Books,



perspective is still felt to be relevant to contemporary culture. Joseph Leo Koerner starts his review of Perspective as Symbolic Form with the following claim: "Perspective is again the rage."151 The second event was the Gulf War, the outcome of which was largely predetermined by Western superiority in the techniques of perspectival representation.

Erwin Panofsky, "Die Perspektive als 'symbolische Form'," Všrtrage der Bibliothek Warburg (Leipzig & Berlin: 1927), 258-330. 151 Joseph Leo Koerner, "The Shock of the View," The New Republic (April 26, 1993): 32.


The images extensively televised during the Gulf War, confirmed Paul Virilio's thesis that modern warfare has become a matter of the "logistics of perception."152 True, broadcasts have included more traditional views of soldiers, planes, and tanks as seen from the outside, by a reporter's video camera. But what we also saw were not just images of the war, but endless images through which the war was carried out: video images from an infrared camera mounted on a plane; video images from a camera installed on a weapon guided by a laser sensor; video in its role as "battle damage assessment" where a weapon equipped with an imaging device follows a weapon of destruction and records details of the damage. This was no longer a traditional reporter's view of a battle. We saw what the soldiers themselves saw: the images that were their only information about their targets. More often, in a strange case of identification, we witnessed what was "seen" by a machine, a bomb, or a missile. Television reports became catalogs of various vision technologies used to plan a battle, to fight it, and to assess the results. They included satellite imagery of Iraqi territory; diagrams of Iraqi radar activity over the course of a week; 3-D visualizations of weapons; a 3-D animation computed from satellite data of a mountain region of Iraqi thought to contain a secret sight of


Paul Virilio, War and Cinema: the

Logistics of Perception (London: Verso, 1989).

biological weapons production. We saw U.S. soldiers equipped with night vision devices as well as U.S. pilots and soldiers training in flight and battle simulators. We also saw military advisors scrutinizing and interpreting aerial photographs and satellite imagery. We were also shown Iraq's weak, antiquated resistance against the superior Western technologies of vision, image making, and image analysis. Iraq fought back with camouflage weapons, decoys, and finally, according to Western journalists, with oil fires set to confuse U.S. surveillance satellites and planes. The Gulf War was the combat of surveillance against camouflage, visibility against invisibility, human eye against computer eye. This warfare was indeed based on the "logistics of perception," but we can describe its visual techniques even more precisely. Vision was employed in a limited way as an instrument to capture and represent information about shapes of objects and their positions in three-dimensional space. The productive employment of this function of vision can be seen most dramatically in the example of radar. A radar screen shows nothing but distances between the radar and other objects in its field of vision. The colors and textures are not important. The guiding laser sensor of a "smart" missile tracks the same information -- the distance between the missile and the target. Similarly, it is the layout of a scene, distances between points, dimensions of objects which are recorded by a satellite photograph. The multitude of vision technologies which only "see" distances and dimensions are paralleled by the reduction of human vision to the same single function. Thus, the images from traditional or infrared video cameras, which contain information about various features of the visual world, are used by a soldier or a pilot in a much more limited way -- as a source of information about distances and shapes in a scene. What is the shape of an object on the horizon -- is it an enemy or a friend? What is the distance between me and the enemy? Finally, the technologies (such as radar) and techniques (using human sight in a function of a radar) of obtaining information about shapes and distances in the real world are

complemented by 3-D computer graphics that simulate the real world on the basis of such information. In their turn, simulated images generated with 3-D computer graphics are used in flight and battle simulators, where the military eye undergoes its training to learn how to reduce the world to geometry and topology. The use of technologies which capture and visualize information about shapes and distances today extends beyond warfare into all spheres of industry and science. The pilots of commercial airlines utilize the same radar screens and are trained in the same flight simulators as Air Force pilots.153 Engineers and architects use 3-D graphics to visualize their designs. Oil-rich

153 As I write this in early 1993, many companies which yesterday supplied very expensive simulators to the military are busy converting them into entertainment arcade-based systems. In fact, one of the first such systems already commercially operating in a number of major cities, including Chicago and Tokyo -- Battletech Center from Virtual World Entertainment, Inc. -- is directly modeled on SIMNET (Simulation Network) developed by DARPA (Defense Advanced Research Projects Agency). SIMNET can be thought of as the first model of cyberspace, the very first collaborative VR environment. SIMNET consists of a number of individual simulators, networked together, each containing a copy of the world database and the virtual representation of all other participants in the conflict such as the Kuwaiti theater of operations. Similarly, a Battletech Center comprises a networked collection of futuristic cockpit models with VR gear. For $7 each, seven players can fight each other in a simulated environment. In another example, in 1992 Lucas Arts has teamed up with Hughes Aircraft, combining the expertise in computer games of the former with the expertise in building actual flight simulators of the latter, in a joint venture aimed at theme-park type rides. On the connection between SIMNET and Battletech

Centers, see Tony Reveaux, "Virtual Reality Gets Real," New Media (January 1993): 36-41. On VR entertainment systems in the

countries pay a high price for satellite images of their territories to help them discover potential oil fields. And, in automated factories, tireless robotic arms move parts under the control of computerized vision systems trained to recognize the shapes of these parts.

In this chapter I will discuss twentieth century automation of the use of vision to record and visualize geometric and topographic information -- to chart the distance and shape of real and imaginary scenes and objects. This function of vision complements another function which was the subject of the previous chapter: the visual representation of abstract ideas and their logical relations. Here we will deal with the opposite problem -- how to represent a single, concrete object or concrete scene. How, for instance, to represent not an idea of a chair but a particular chair with a unique shape and dimensions. These two kinds of referents -- abstract ideas and arguments, on the one hand, and concrete three-dimensional shapes, on the other hand, can be related to two long-standing philosophical positions on what constitutes reality. For Plato, sensible particulars were but pale reflections of Ideas or Forms. Aristotle criticized Plato, declaring the primary substances were not Ideas but the individual things such as particular men or animals. These opposing views have continued to be debated in scholastic philosophy, with Plato's view giving rise to realism and Aristotle's to nominalism.154

context of location-based entertainment -- arcades and theme parks -- see Richard Cook, "Serious Entertainment," Computer Graphics World (May 1992): 40-48. 154 Allan Sekula relies on this distinction between the philosophical positions of realism and nominalism in his discussion of different systems of police photography in the end of the

In scholastic philosophy the terms "realism" and "nominalism" refer to two opposing ontological positions: realism claiming the primacy of Ideas and nominalism -- the primacy of individual things.155 Although in contemporary thought the meanings of "realism" and "nominalism" are somewhat different, the original meanings are helpful in distinguishing two fundamental kinds of representations: abstract ideas versus concrete objects. I have suggested in the previous chapter that the desire to visualize abstract ideas and logical relations was new to modernity. But it would not be reasonable to propose that the function of vision which can be called visual nominalism -- to capture the identity of individual objects and spaces by recording distances and shapes -- originates in the twentieth century as well. After all, maps were used for many centuries; and the sophisticated graphic language of engineering drawings was well developed by the nineteenth century. Yet, as this chapter will try to demonstrate, in this century the automation of visual nominalism, which began with the Renaissance perspective, entered a new stage. The sign of this automation is the multitude of new technologies used to capture and visualize three-dimensional reality that have emerged since

Allan Sekula, "The nineteenth century. Body and the Archive," October 39 (1987): 3-64. 155 Antony Flew, ed., A Dictionary of Philosophy (London: Pan Books Ltd., 1984), 250, 299.

the middle of the twentieth century, such as radar, infrared imaging, laser sensors, CAT scans, magnetic resonance imagining, 3-D computer graphics, and computer holography. Finally, since the early 1960s work has been under way to automate vision completely, to create computer vision systems that will recognize objects and interpret scenes automatically. The development of these technologies has been accompanied by massive research into the general problems of visual nominalism in computer science, experimental psychology, and neuroscience. New formal mathematical techniques were developed to analyze images as a source of depth information and, vice versa, to transform this information into realistic images. The work on the automation of visual nominalism has also lead to new attention to particular aspects of human perception. A new paradigm for the study of human vision emerged during the 1970s at MIT associated with the name of David Marr.156 Within this paradigm, the goal of human perception is taken to be the recognition of shapes, leading researchers to study algorithms by which the brain "computes" shapes of objects from retinal input in the hope that these algorithms can then be used by computer vision systems. This chapter brings together seemingly unrelated visual technologies and techniques: radar, 3-D computer graphics, and computer vision. I will suggest that they are ultimately related by the function they all serve -- visual nominalism. Until this century, this function has been performed by techniques of perspectival representation such as perspective and descriptive geometry and, later, photography. Rather than thinking of them as artistic techniques, such writers as William Ivins and Bruno Latour emphasize their significance for the development of science, modern technology, and the economy. What these techniques allow best is to represent precisely three-dimensional information on a two-dimensional plane, thus recording quantitatively and efficiently the knowledge of the surrounding space. New twentieth century techniques further extend the usefulness of these kinds of representations. Radar allows to obtain
156 David Marr, Vision (New York: W.H. Freeman and Company, 1982).

spatial information without the limitations of visibility (section 3); 3-D computer graphics add the dimension of interactivity (section 4); and computer vision automates visual nominalism altogether (section 5).

While there is no history of the burgeoning techniques of visual nominalism in the twentieth century, the earlier history of these techniques has been studied in detail by art historians. The problem of representing three-dimensional space upon a two dimensional plane has been a subject of art historical research from at least the beginning of the century. It is quite remarkable that the people who are today recognized as the founders of modern art history (Alois Riegl, Heinrich Wšlfflin, and Erwin Panofsky), all defined the field as the history of the representation of space. Working within the paradigms of cyclic cultural development and racial topology, they related the representation of space in art to the spirit of entire epochs, civilizations and races. In his 1901 Die SpŠtršmische Kunstindustrie, Riegl characterized humankind's cultural development as the oscillation between two extreme poles, two ways to understand space, which he called "haptic" and "optic." Haptic perception isolates the object in the field as a discrete entity, while optic perception unifies objects in a spatial continuum.157 Riegl's contemporary, Heinrich Wšlfflin, similarly proposed that the temperament of a period or a nation expresses itself in a particular mode of seeing and representing space. Wšlfflin's Principles of Art History (1913) plots the difference between Renaissance and Baroque on five dimensions: linear -painterly, plane -- recession, closed form -- open form, multiplicity -- unity, and clearness -unclearness. Finally, another founder of modern art history, Erwin Panofsky, contrasted the

157 Michael Ann Holly, Panofsky and the Foundations of Art History (Ithaca: Cornell University

Press, 1984), 73. 2. "The most important event of the Renaissance."

"aggregate" space of the Greeks with the "systematic" space of the Italian Renaissance in the essay Perspective as a Symbolic Form (1924-1925).158 Panofsky established a parallel between the history of spatial representation and the evolution of abstract thought. The former moves from the space of individual objects in antiquity to the representation of space as continuous and systematic in modernity; in Panofsky's neologisms, from "aggregate" space to "systematic" space. Correspondingly, the evolution of abstract thought progresses from ancient philosophy's view of the physical universe as discontinuous to the post-Renaissance understanding of space as infinite, ontologically primal in relation to bodies, homogeneous, and isotropic -- in short, as "systematic." Panofsky's 1924-25 essay has been recognized as the first in what became an evergrowing series of interpretations of perspective.159 These interpretations related perspective to every known characteristic of the modern period: economic, social, and philosophical. Some of these interpretations acquired the character of dogmas, self-evident truths. For instance, the correlation between Cartesian ideas of rational subjectivity in philosophy and Renaissance perspective in visual art has been presented as one of the central metaphors in interpreting

158 Panofsky, "Die Perspektive als 'symbolische Form.' 159 The recent very important work on perspective is Hubert Damisch, L'origine de la

perspective (Paris: Flammarion, 1987).

Western modern culture. This concept became so prevalent that in his critical analysis Martin Jay feels justified to name it "Cartesian perspectivalism."160 Most interpreters have singled out an aspect of perspective, which then become for them the core of its symbolic function. For instance, Pierre Francastel, and following him, John Berger, and many film theorists of the 1960s and 1970s, focused on the mechanical character of perspective and its capacity to organize the representation of reality around a single point of view. They theorized a connection between Renaissance perspective and the development of early capitalism, the emergence of instrumental consciousness which measures everything, which is concerned with solidity and extension, with numbers and equivalents -- the consciousness of market and accounting. Representations of cosmic, religious, unmeasurable space were replaced by the pictures of cultivated, appropriated landscapes or of rooms cluttered by precisely drawn objects signifying wealth. Perspective presented the world as ready to be mastered, consumed, colonized -- the world originating in the eye of the spectator.161

Martin Jay, "Scopic Regimes of Modernity," Vision and Visuality, ed. Hal Foster (Seattle: Bay Press, 1988), 4. 161 Andrew Dudley, Concepts in Film Theory (Oxford: Oxford University Press, 1984), 31.


Other writers foregrounded different characteristics of perspective in their interpretations. For instance, for Rudolf Arnheim, a pyramidal world portrayed by perspective signifies a hierarchical conception of human existence.162 On the other hand, for Fernande Saint-Martin, perspectival representation which can only show objects at a distance, corresponds to the "conceptual rather than sensorial, 'ideological' rather than concretely experienced" relation of a subject to the world.163 Panofsky's own account of perspective is more subtle than many of the consequent interpretations it inspired. Panofsky correlates a mode of visual representation with a philosophical system, but he is careful not to read one as an expression of the other. His conclusion is open: perspective can be equally interpreted as subjective and objective, since it presents space as an objective phenomenon but inevitably perceived from a subjective point of view. Calling perspective "a two-edged sword," Panofsky writes: Perspective subjects the artistic phenomenon to stable and even mathematically exact rules, but on the other hand, makes the phenomenon contingent upon human beings, indeed upon the individual: for these rules refer to the psychological and physical conditions of the visual impression, and the way they take effect is determined by the freely chosen position of a subjective "point of view."164

162 Rudolf Arnheim, Art and Visual Perception. The New Version (Berkeley: University of California Press, 1974), 295. 163 Fernande Saint-Martin, Semiotics of Visual Language (Bloomington and Indianapolis: 1990), 141. 164 And while modern perspective, as a coherent unity, is contrasted by Panofsky to the modes of representation in antiquity and the Middle Ages, he also suggests that within the perspective mode there exist many distinct variations. Working within the same mode of perspective, different cultures and epochs take its meaning to be different: the Dutch represent "near space," Germans favor "oblique space," Italians explore "high space."

While Panofsky's essay had to wait almost six decades before its publication in English, another equally important essay on perspective, "Obratnaya Perspektiva" (The Inverted Perspective) still remains untranslated and unknown to Western art historians.165 In his article Joseph Leo Koerner briefly refers to the essay and regrets its unavailability in English (this is the only reference I am aware of).166 The essay was written by the Russian philosopher, linguist, mathematician and art historian Father Pavel Florensky, who headed the department of the

Ervin Panofsky, Perspective as Symbolic Form, 68-70. 165 Pavel Florensky, "Obratnaya Perspektiva (The Inverted Perspective)," Sobranie Sochineniy, ed. N.A. Struve (Paris: YMCA-PRESS, 1985), 1: 117-192. 166 Koerner, "The Shock of the View," 37-38.

Analysis of Spatial Representation in the Arts at VKhUTEMAS from 1921-1924.167 Florensky's essay can be justly considered as the counterpart to Panofsky's work, completed in 1922, two years before Panofsky's work. Both Panofsky and Florensky interpret any perspectival system as convention and oppose the view that any particular system is natural.168 At the same time, their arguments tend to favor antithetical perspectival systems and world views: linear perspective and Renaissance individualism in the case of Panofsky; inverted perspective and medieval religious collectivism in the case of Florensky. Panofsky narrates the evolution of the representation of space as an inevitable progression towards the "systematic" yet subjective space of the Renaissance; thus, the Renaissance perspective is given a privileged status, its final triumph as inevitable as the progress of history itself. The discussion of medieval representation of space becomes for

167 VKhUTEMAS (State High Art-

Technical Studious) was the leading school of art and design in the USSR in the 1920s and the center of avant-garde artistic culture. 168 Florensky writes: "Is it true that perspective, as it is claimed by its supporters, expresses the true nature of things and therefore should be everywhere and always understood as the absolute condition of artistic truth? Or is it only a scheme, only one among many schemes of representation, corresponding not to the universal world view but only to one possible understanding of the world, connected with a particular sensibility and cognizance?" Florensky, "Obratnaya Perspektiva," 123.

Panofsky nothing more than a brilliant narrative device, a dramatic detour in what otherwise is presented as a linear course from Antiquity to Renaissance: "When work on certain artistic problems has advanced so far that further work in the same direction, proceeding from the same premises, appears to bear fruit, the result is often a great recoil, or perhaps better, a reversal of direction."169 In Panofsky's interpretation, in other words, medieval representation of space serves as a dialectical antithesis between ancient and Renaissance representations. Thus, Panofsky claims, while it may appear that medieval representation gave up the advance of Antiquity in repressing three-dimensionality, in fact it approached the Renaissance understanding of space as infinite and homogeneous: "For if Romanesque painting reduced bodies and space to surface, in the same way and with the same decisiveness, by these very means it also managed for the first time to confirm and establish the homogeneity of bodies and space." 170 Florensky, on the other hand, is mostly interested in the medieval representation of space as manifested by the spatial constructions of Russian icons between the fourteenth and sixteenth centuries. According to Florensky, seemingly heterogeneous spatial constructions of icons were neither a result of the lack of artistic skill nor a step towards the Renaissance perspective.171

169 Panofsky, Perspective as Symbolic Form, 47. 170 Ibid., 51. 171 The view of medieval art as unsystematic representation of space, against which Florensky

revolted, continues to persist. Thus, Samuel Edgerton writes in 1975: "Unlike the Renaissance painter depicting his scene in perspective, the medieval artist viewed his world quite subjectively. He saw each element in his composition separately and independently, and thus

Rather, they are result of a conscious and coherent system for representing reality which corresponds to a coherent world view. It is not hard to see that for the orthodox and slavophile Florensky, this world view is superior to Western post-Renaissance individualism. Russian artists, like the artists of ancient Egypt and China, were aware of perspective but consciously refused its power, choosing instead "religious objectivity and super-personal metaphysics." For

paid little attention to anything in the way of systematic spatial relationship between objects."

Samuel Edgerton, The Renaissance Rediscovery of Linear Perspective (New York: Basic Books, 1975), 21.

Florensky, the Western Renaissance represents the beginning of the Fall: "When religious stability of the world view begins to disintegrate, and the sacred metaphysics of collective popular consciousness is segmented by individual vision of an individual subject from an individual point of view, and only in a particular moment of time -- only then perspectivalism, symptomatic of disunited consciousness, emerges." 172 Florensky's Slavophilism strikes us today as being politically biased; yet, this bias makes us realize that Panofsky's seemingly more objective account is also biased in its privileging of what Florensky labeled "bourgeois individualism." While Florensky offered a theoretical critique of linear perspective, for his compatriots at the time this critique became a practical problem: what new visual forms shall be suitable for the "first people's state"? Similarly identifying linear perspective with "bourgeois individualism," El Lissitsky and Kazimir Malevich relied in their designs on other types of perspectival construction such as parallel projection. In this type of projection, all projection rays are parallel to each other, instead of originating from a single point of view. Parallel projection, with its absence of a single point of view, was presented as a more suitable way to symbolize the vision and knowledge of a collective. Paradoxically, this "objective" point of view of a collective was expressed in the image (commonly encountered in

172 Florensky, "Obratnaya Perspektiva," 125-126. Justifying the value of reverse perspective as a symbol of "super-personal metaphysics," Florensky offered a critique of the arguments later to be found in Panofsky's "Die Perspektive als 'symbolische Form.'" For instance, he precisely articulates the notion of "systematic" space, which, according to Panofsky, represented the triumph of the Renaissance and was still absent in Medieval thought: "If we summarize everything which is said in a formal sense against Medieval art, they are reduced to one criticism: 'There is no understanding of space.' This criticism means that there is no spatial unity, no scheme of Euclid's and Kant's space, the latter reduced, within the framework of painting, to linear perspective and proportionality."

Malevich's writings) of the individual artist looking at the Earth from an infinitely far away point in outer space, so that the convergence of rays is replaced by a parallel projection.173


A.G. Rappoport, "El Lissitsky i ego 'Pungeometiya'" (El Lissitsky and his 'Pan-geometry'), Sovetskoe Iskusstvoznanie 25 (1989): 118119. Four decades later, LANDSAT satellites start bringing back images of Earth objects in axonometric projections (because they were shot from high orbits) thus fulfilling Malevich's designs.

The refusal of linear perspective by such artists as Malevich or Lissitsky is usually seen as just one episode in what by now became a widely accepted narrative -- the shattering and negation of perspectival space by modernist artists. According to this narrative, perspective was already dead by the time art historians such as Panofsky had begun writing its history. Such narrative is announced, for instance, in the very title of Pierre Francastel's Painting and Society. Birth and Destruction of Plastic Space from the Renaissance to Cubism (1952). The opening section of The Production of Space by Henri Lefebvre is equally authoritative: The fact is that around 1910 a certain space was shattered. It was a space of common sense, of knowledge (savoir), of social practice, or political power...a space, too, of classical perspective and geometry, developed from the Renaissance onwards on the basis of the Greek tradition (Euclid, logic) and bodied forth in Western art and philosophy, as in the form of the city and town.174 Yet, if perspective disappeared from modern art, it survived as one of techniques of the visual nominalism, a method for precisely representing the three-dimensional world on a twodimensional surface. In this role, it extended into many new domains and became the foundation of new kinds of automated technologies of remote sensing and image synthesis. To consider perspective in its role as a technique of visual nominalism we should turn to another interpreter of perspective -- William Ivins and his short 1939 essay On the Rationalization of Sight.175 If Panofsky connects the development of perspective with the idea of infinite space, abstract space existing prior to objects, Ivins on the contrary emphasizes that perspective allows the creation of precise maps of three-dimensional reality, to record the shapes of concrete objects and the layout of concrete spaces. It is the tool of a businessman and a scientist rather than an artist. In Ivins' definition, perspective is "a practical means for securing a rigorous two-way, or reciprocal, metrical relationship between the shapes of objects as definitely located in space and
174 Henri Lefebvre, The Production of Space (Oxford: Blackwell Publishers, 1991), 25. 175 William M. Ivins, On the Rationalization of Sight (New York: Da Capo Press, Inc., 1975).

their representations."176 This definition sums up a number of unique characteristics of perspectival projection. First, the size of graphic marks in a perspectival image is proportionally related to the dimensions of represented objects. For instance, a foot-long object would be represented by a line of three inches, while an object twice as long would become a line of six inches, and so on. Second, the ratio of sizes between these lines is proportional to the ratio of sizes between the corresponding objects. Third, the perspectival image does not only record the shapes and dimensions of objects but also contains information about their distance from one another (in contrast to parallel projection, for instance). Finally, all these relationships between the perspectival symbols on the one hand, and between the symbols and the corresponding objects on the other hand, are controlled by a single rule or, to use the language which would become common within a decade after the publication of On the Rationalization of Sight, by a single algorithm. Given the coordinates of the point of view and the projection surface, the projection of any point in space can be determined by mechanically following a sequence of steps. The most important quality of perspective foregrounded in Ivins' definition is the precise and reciprocal relationship it establishes between objects and their symbols. We can go from objects to symbols (two-dimensional representations); but we can also go from such symbols to three-dimensional objects. Using the rules of perspective, an architect could create a visual representation of an already existing building, change this representation, and the workers would execute the specified changes in the building itself. Better yet, our architect can design a brand new building -- not by painstakingly constructing a small model, where each change takes time to implement -- but by creating a perspectival drawing, where each form is just a line and change requires only a single movement of an eraser (or a single press of a key on a computer keyboard, if the architect is working with a CAD system).
176 Ibid., 9.

Bruno Latour extended Ivins' idea by pointing out that a reciprocal relationship made possible by perspective allows us not only to represent reality but also to control it.177 Latour sees perspectival representations as the "most powerful instrument of power," defined as the ability to mobilize resources across space and time, to manipulate these resources at a distance. For instance, we cannot measure the sun in space directly, but we only need a small ruler to

177 Bruno Latour, "Visualization and Cognition: Thinking with Eyes and Hands," Knowledge and Society: Studies in the Sociology of Culture Past and Present 6 (1986): 1-40.

measure it on a photograph (the perspectival image par excellence).178 And even if we could fly around the sun, we would still be better off studying the sun through its representations which we can bring back from the trip -- because now we have unlimited time to measure, analyze, and catalog them. We can also represent absent things and plan our movement through space by working on representations: "One cannot smell or hear or touch Sakhalin Island, but you can look at the map and determine at which bearing you will see the land when you send the next fleet."179 We can move objects from one place to another by simply moving their representations: "You can see a church in Rome, and carry it with you in London in such a way as to reconstruct it in London, or you can go back to Rome and amend the picture." Finally, as Latour points out, "the two ways become a four-lane freeway! Impossible palaces can be drawn realistically, but it is also possible to draw possible objects as if they were utopian ones." Real and imagined objects can meet on a flat space of perspectival representation. All in all, perspective is more than just a sign system, reflecting reality -- it makes possible the manipulation of reality through the manipulation of its symbols.

178 Ibid., 22. 179 Ibid., 8.

Citing perspective as a perfect example of a new instrument of power and domination, Latour proposes that the history of science and technology should be understood as a cascade of such innovations in representation. Yet, his argument is already anticipated by Ivins. Ivins concludes his essay by stating that the beginning of the rationalization of sight through the discovery and the development of perspective "was the most important event of the Renaissance."180 It was more important than the fall of Constantinople or the Reformation or the Counter Reformation because it made all other discoveries possible by launching a constantly accelerating series of innovations.181 The invention of perspective propelled modern empirical science, for instance biology, which could now represent forms of nature with mathematical precision. It also stimulated the rise of modern engineering and manufacturing by making feasible the distribution of identical designs to far away places.182 Ivins' approach stands in sharp contrast to the more traditional art historical analyses of perspective by Panofsky and Francastel. They are concerned with perspective as an artistic form and do not look beyond its history in art. Ivins, on the contrary, is concerned with visual culture - the techniques and technologies of visual representation available to a society at a given moment and the fundamental role they play in shaping every aspect of society. Because he sees these techniques and technologies183 as ultimately determining social development, his approach is deeply materialistic. Perspective, for Ivins, is not an artistic form reflecting social or philosophical modernity; it is the very condition which makes modernity and modernization possible.184 For instance, he writes: "Many reasons are assigned for the mechanization of life
180 Ivins, On the Rationalization of Sight, 13. 181 Ibid., 7, 12. 182 Ibid., 13. 183 Ivins focuses on two fundamental developments: first, the invention of perspective and its development into descriptive and perspective geometry; second, the invention of technique for printing pictures which made possible easy duplication of exact copies of the same image. 184 This is the theme which Latour also greatly extends, proposing "a new kind of materialism" which will account for social and scientific history through the analysis of available forms of representation.

and industry during the nineteenth century, but the mathematical development of perspective was absolutely prerequisite to it."185 Ivins is also a materialist in the most extreme Marxist sense because he suggests that the available forms of representation, such as perspective, determine the development of abstract thought. He points out that as important as the development of perspective was for picture making, "it is doubtless even more important to general thought, because the premises on which it is based are implicit in every statement made with its aid."186 This idea is repeated again at the end of the essay: The constant extensions of the fields of usefulness of the pictorial symbol that is precisely duplicable [through printing] and of the grammars of its use [development of perspective and descriptive geometry] have had a most astonishing effect not only upon knowledge but upon thought and its basic assumptions or intuitions...Relativity, which now in one form or another runs throughout contemporary thought and practice, is in large measure a development of ideas that were evolved through the study and use of projective transformations.187 It is Ivins' general focus on visual culture, rather than art, that makes him attentive to the current use of perspective and its continuing development and expansion. While Panofsky or Francastel write about perspective in the past tense aware of its demise in modern art, Ivins on the contrary notes "that today there are few sciences and technologies that are not predicated in one way or another upon this power of invariant pictorial symbolization."188 Modern designers, scientists or engineers, of course, do not simply use perspective as it was formulated by Alberti in the fifteenth century; they use much more sophisticated techniques. According to Ivins, the rationalization of perspectival sight proceeded in two directions. On the one hand, perspective became the foundation for the development of the techniques of descriptive and perspective geometry which became the standard visual language of modern engineers and architects (fig. 11). On the other hand, the photographic technologies automated the creation of perspectival
185 Ivins, On the Rationalization of Sight, 12. 186 Ibid., 9. Emphasis mine - L.M. 187 Ibid., 13. 188 Ibid.

images. Both were accomplishments of the nineteenth century; in fact, both were developed more or less simultaneously. Indeed, as Ivins points out, Ni pce and Talbot, the founders of photography, were contemporaries of Monge and Poncelet, decisive figures in the development of descriptive and perspective geometry.

Writing On Rationalization of Sight between 1936 and 1938, Ivins mentions such examples of the contemporary use of perspective as aerial photographic surveillance, classification in the field of archeology, and criminal detection.189 However, all these applications of perspectival techniques already existed in the nineteenth century and, by the 1930s, did not represent the "latest" developments. While photo reconnaissance was first employed systematically on a mass scale during World War I, the interest in using photography for aerial surveillance existed since its invention. FŽlix Tournachon Nadar, one of the most eminent photographers of the nineteenth century, famous for his portraits, succeeded in exposing a photographic plate at 262 feet over Bi vre, France in 1858. He was soon approached by the French Army to attempt photo reconnaissance but rejected the offer. In 1882, unmanned photo balloons were already in the air; a little later, they were joined by photo rockets both in France and in Germany. The only innovation of World War I was to combine aerial cameras with a superior flying platform: the airplane.190

189 Ibid., 12, 13. 190 Beaumont Newhall, Airborne Camera (New York: Hastings House, Publishers, 1969). For

critical histories of photo reconnaissance see Allan Sekula, "The Instrumental Image: Steichen at War," in Photography against the Grain: Essays and Photo Works, 1973-1983 (Halifax: The Press of the Nova Scotia College of Art and Design, 1984); Paul Virilio, War and Cinema: the Logistics of Perception (London: Verso, 1989); 3. Radar: Seeing Without Eyes

In 1858, Albrecht Meydenbauer, a director of the German Government Building Office, published a proposal to use photographs for scale measurement. His proposal was based on the existence of a geometrical relationship between the photographic image and the object being photographed. Why, for instance, climb a facade of a cathedral in order to measure it (as Meydenbauer had to do, nearly once getting killed) when it is much safer to measure a photograph? Additionally, wrote Meydenbauer, "some may find it hard to believe, but

Manuel De Landa, "Policing the Spectrum," in War in the Age of Intelligent Machines (New York: Zone Books,



experience has proven than one can see, not everything, but many things, better in scale measurement than on the spot." In 1885 the Royal Prussian Institute for Scale Measurement was founded and the measurement of photographs of historic monuments became a frequent practice.191 In the field of criminal detection, perspectival technology of photography was routinely employed decades before the time of Ivins' writing. In his 1844 Pencil of Nature William Fox Talbot, one of photography's founders, comments on a beautiful calotype depicting several shelves of china: "should a thief afterwards purloin the treasures -- if the mute testimony of the picture were to be produced against him in court -- it would certainly be evidence of a novel kind."192 In 1883, Alphonse Bertillon, director of the Identification Bureau of the Paris Prefecture of Police, developed a system for classification and identification of criminals that relied on photographs. By 1893, Bertillon's system was already employed in the United States, Belgium, Switzerland, Russia, much of South America, Tunisia, the British West Indies, and Romania.193

Modern perspective and descriptive geometry were fully developed in the first half of the nineteenth century. Photography was already utilized by a number of professions for measurement, identification, and classification by the end of the century. What was the next step in the "rationalization of sight"? In fact, while Ivins was writing his essay on perspective, across the Atlantic, in England, work was already underway to install twenty radar stations on the east and southeast coasts to provide surveillance of these air approaches. These radar installations turned out to be absolutely essential in the coming war, allowing for the severely outnumbered Royal Air Force to defeat the
191 Harun Farocki, "Reality Would Have to Begin," Documents 1/2 (1992): 136-146. 192 Qtd. in Sekula, "The Body and the Archive," 6. 193 Sekula, "The Body and the Archive," 25, 35.

Luftwaffe in the Battle of Britain. Radar, the latest technology of visual nominalism, became Britain's most important weapon.194 The following anecdote about the "discovery" of radar, regardless of its historical accuracy, is useful in understanding how radar works.195 In 1922, two civilian scientists employed by the Navy set out to conduct a routine communication experiment along the arm of the Potomac river. As usual, they set a short-wave radio receiver and transmitter along the two banks of the river. This time, however, a passing steamer interrupted the signal, creating an electronic echo. The scientists immediately realized that such echoes could be used to deduce the locations of ships entering a harbor. This anecdote illustrates the main principle of radar operation. Radar is an acronym for Radio Detection and Ranging. Like sound waves, radio waves create echoes when they are interrupted by objects in their path. Radar transmits a radio wave in a particular direction. The signal reflected back from the objects is picked up by an antenna. The time between the transmission and the reception of the echo indicates the distance to the object; the direction the antenna is pointing when the echo is received reveals the object's position in relation to the radar. Detected objects appear as bright spots on a display watched by a radar operator.196 Radar exemplifies the further rationalization of visual nominalism. All it sees and all it shows are the positions of objects, 3-D coordinates of points in space, points which correspond to submarines, aircraft, birds or missiles. Color, texture, even shape are disregarded. Instead of Alberti's window, opening onto the full richness of the visible world, a radar operator sees a
194 For the history and technology of radar, I mainly relied on two sources: Echoes of War (Boston: WGBH Boston, n.d.), videotape; McGraw-Hill Encyclopedia of Science & Technology: an International Reference Work in Twenty Volumes Including Index (New York: McGraw-Hill, 1992.) 195 Echoes of War (Boston: WGBH Boston, n.d.), videotape. 196 Numerous variations of basic radar technology exist. For instance, in addition to active radars which send a signal and detect energy reflected by objects there are also passive radars which do not send a signal themselves. However, all radars have in common the use of electromagnetic radiation (radio waves) to detect and measure objects in their vicinity.

screen, a dark field with a few bright spots. Here, the function of visual nominalism, which perspectival image performed along with many other functions (representation of light, color, shade, texture, etc.), is isolated and abstracted. Radar serves a single function -- but it performs it more efficiently than any previous perspectival technique or technology. First, the detection of objects' positions in space is no longer limited by conditions of visibility. Vision is no longer confined by the distance resolvable by the human or camera eye, since radar can detect objects hundreds of miles away, and at any time of day or night. Second, this recording takes place instantly. Previously, military commanders had to wait until pilots come back from surveillance missions and the film was developed. The inevitable delay between the time of the observation and the delivery of the finished image limited its usefulness: since by the time the image was produced, enemy positions could have changed. Now, the imaging is instantaneous. With radar, its vision not limited by visibility, its image literally tracking the referent, rationalization of visual nominalism reaches a new stage. The usefulness of vision and of imaging are greatly extended. This expansion is achieved by fundamentally redefining the very foundations of vision and imaging technologies. While radar came to supplement perspectival representation and photography in terms of its function, it also represents a fundamental break with older technologies. Like the photographic camera, radar captures the positions of objects and displays them on a flat surface. But how it sees has little resemblance to what was commonly meant by vision as defined by older optical apparatuses -- from human eyes to photography. Instead of "looking" in a particular direction, radar sees all around, its antenna scanning the area in a circular sweep. Instead of passively receiving light reflected by objects, radar sends energy into the environment. Its vision is active; it can be compared to exploration of the

environment by a blind person.197 Instead of relying, like photography, on the small region of the electromagnetic spectrum to which our eyes are sensitive, it uses other regions, sending and receiving waves of different lengths. Vision is no longer limited by the spectral capacity of the human or camera eye; it is extended to include the whole of electromagnetic spectrum. The visible becomes a small part of a larger field of sensory exploration of the environment. Images produced by radar are also fundamentally different than those of previous imaging technologies, from drawing paper to photographic film. The photographic image is an imprint corresponding to a single referent or to a limited time of observation. With the invention of the radar screen, the image surface is no longer static. For the first time it becomes constantly updatable in real time. Here, with radar, we see for the first time the mass employment of a fundamentally new kind of display, which soon comes to dominate modern visual culture, found today in video or computer screen. What is new about such a display is that its image can change in real time reflecting changes in the referent, be it the position of an object in space (radar), any change in visible reality (live video) or changing data in computer memory (computer screen). The radar screen changes, tracking the referent. But while it appears that the element of time delay always previously present in imaging is eliminated, in fact time enters radar image in a new way. With older photographic technologies, all parts of an image are exposed simultaneously. Now the image is produced through sequential scanning, which means that its contiguous parts in fact correspond to different moments in time. Like in an audio record, moments in time become circular tracks on a surface.

Along with radar, many other technologies of visual nominalism came into existence following the advances in electronics and computers during World War II: ultrasonic imaging,
197 There also exist passive radars which have no transmitters but are equipped to measure

signals from targets themselves. The active radar can be thought as a historical continuation of projectors previously used by the military to artificially light up the scene of a battle to increase visibility.

multispectral photography, multispectral imaging, infrared, sonar, magnetic resonance imaging, and so on.198 As radar, these technologies are effectively used to record distance, position, layout, shape, and volume. Sonar, for instance, detects objects in the water by using sound waves. Ultrasonic computer tomography uses sound waves and computer graphics to construct images of body tissues. Multispectral photography isolates energy reflected from surfaces in a number of given wavelength bands. Coming to supplement perspectival techniques and technologies, the new technologies of visual nominalism are more efficient but also more specialized. If older optical technologies indiscriminately capture a number of dimensions of visual reality (color, shape, texture, tonality), now a single dimension can be isolated. More precisely, a single dimension, in this case 3-D information, is broken into distinct components recorded by different imaging technologies. The technique of perspective allows one to record both the shape of objects and their relative positions in relation to each other. Radar only records positions, it is blind to shape. On the other hand, a laser or ultrasound range finder produces a depth map, treating as shapes both the actual objects and the empty space between them. This specialization can be seen as a side effect of the automation of visual nominalism. This automation requires a priori assumptions about the nature of reality which become embedded in different technologies. In the case of radar, reality is reduced to a set of geometric points which have no extension. In the case of a range finder, reality is treated as if a skin, a membrane, a continuous 3-D form. We may recall here the far-seeing prediction of the effects of photography published in 1859 by Oliver Wendell Holmes: "Every conceivable object of Nature and Art will soon scale off its surface for us. Men will hunt all curious, beautiful, grand objects,

198 See McGraw-Hill Encyclopedia of Science & Technology.

as they hunt cattle in South America, for their skins and leave the carcasses as of little worth."199 Holmes' words seem to come true not only metaphorically (modern culture which values only images) but also literally, as range finders methodically skin reality. Along with the separation between technologies that record positions and technologies that record shapes, another specialization emerges. Depending upon whether the position of the observer in relation to objects in space is important or not, the technologies of visual nominalism specialize in creating two different types of representations which can be called viewer-oriented and viewer-independent.200 The first kind of representation is exemplified by the radar screen, a record of distances between enemy targets and the subject -- the point in the center of the screen. Here, even more so than in the traditional perspective, the world is represented in relation to the observer who becomes the center of the coordinate system. In the second type of representation, the distance between the observer and the object is unimportant and in fact, the position of the observer cannot even be deduced from the image. These are images which employ types of projection other than perspective: maps, engineering and architectural plans, and aerial and satellite photography, whether conventional, infrared or multispectral. A third specialization, a third division of labor brought about by new technologies is that between vision and imaging, which now become distinct processes. Engineering textbooks and encyclopedias group many new technologies of visual nominalism under the term "remote sensing," defined as the gathering and imaging of information without actual physical contact with the object or area being investigated.201 This definition clearly separates the two
199 Oliver Wendel Holmes, "The Stereoscope and the Stereograph," in Photography, Essays and

Images: Illustrated Readings in the History of Photography, ed. Beaumont Newhall (New York: Museum of Modern Art, 1980), 60. 200 This distinction is related to but not synonymous with the distinction between "viewercentered" and "object-centered" representations, common since David Marr's Vision (New York: W.H. Freeman and Company, 1982). It is also related, but again, not equivalent to, the difference between perspectival and orthographic projections. The first records reality from a single point of view, the second does not have a single point of view. 201 McGraw-Hill Encyclopedia of Science & Technology, 15: 311.

operations: the gathering of information (vision) and its presentation (imaging). The first operation may have nothing to do with what is visible to the human eye, but in the second operation the eye eventually comes into play since the gathered information has to be presented to the human observer in visual form in order to be useful. Technologies of remote sensing convert the data obtained by technologically augmented biological senses (for instance, we can think of radar, which sends waves in order to detect objects, as an augmented sense of touch) into information perceivable by the human eye. They map other parts of the electromagnetic spectrum into a small visible part. The invisible (sound waves, microwaves, ultraviolet waves) is converted into the visible. This is why these technologies are also often called imaging technologies -- multispectral imaging, ultrasound imaging, infrared imaging, and so on.

I have emphasized the continuity of function between perspectival drawing, photography, and newer technologies such as radar and ultrasound imaging. Yet, not only these newer technologies of visual nominalism serve the same function as the old -- to capture the identity of individual objects and spaces by recording distances and shapes -- but they rely on the same principle of perspective. We can justifiably refer to them as perspectival technologies if we understand perspective as extending beyond the domain of the visible. In his seminar "Of the Gaze as Object Petit a" Jacques Lacan argued that this is how perspective should be understood.202 He starts by reminding us that an image is anything defined "by a point-by-point correspondence of two unities in space." To obtain an image of something we do not have to rely on light or to operate in the domain of the visible. Nor do we have to limit images to 2-D representations of 3-D reality. We can represent an object by another

202 Jacques Lacan, "On the Gaze as Objet Petit a," in The Four Fundamental Concepts of

Psycho-Analysis, ed. Jacques-Alain Miller (New York: W.W. Norton & Company, 1981), 67122.

object or represent a 2-D form by another form. All that is required is a rule to establish the correspondence between the points of the object being imaged and the points on the image. Similarly, says Lacan, "what is an issue in geometric perspective is simply the mapping of space, not sight."203 Perspective is one such rule, a particular method to establish a correspondence between the object and its image. The method of perspective consists of connecting a single point in space (usually referred to as subject's point of view) with a number of points on the object by straight lines; the intersection of these lines with a plane creates an image. It is coincidental that perspective, whether as a part of the human sight apparatus or as a part of the photographic apparatus, works through light. Light travels in straight lines, therefore it can be used to create perspectival images. But one can construct such images without light: "In Descartes, dioptrics, the action of the eyes, is represented as the conjugated action of two sticks."204 As Lacan points out further on in the seminar, this idea that perspective is not limited to sight alone but functions in other senses as well defines the classical discourse on perception: "The whole trick, the key presto!, of the classic dialectic around perception, derives from the fact that it deals with geometric vision, that is to say, with vision in so far as it is situated in a space that is not in its essence the visual."205 Lacan's clarification that the principle of perspective is not limited to the visible helps us understand that the technologies of remote sensing function on the principle of perspective. Regardless of their lengths, all waves travel in straight lines, and therefore points in space are connected by straight lines to a point of reception (such as radar antenna) or recording (such as a photographic camera). Radar, infrared imaging, sonar, and ultrasound are all part of what Lacan called "geometric vision," perspectival vision extending beyond the visible.

203 Ibid., 86. 204 Ibid., 87. 205 Ibid., 94.

The technologies of remote sensing made it possible to map space without the limitations of visibility. In the next section I will discuss the technology of 3-D computer graphics, which further extended the usefulness of perspectival representations by making them interactive.

From the moment of adaptation of perspective, artists and draftsmen have attempted to aid the laborious manual process of creating perspectival images.206 Between the sixteenth and the nineteenth century various "perspectival machines" were constructed. They were used to construct particularly challenging perspectival images, to illustrate the principles of perspective, to help students learn how to draw in perspective, to impress artists' clients, or to serve as intellectual toys. Already in the first decades of the sixteenth century, DŸrer described a number of such machines.207 One device is a net in the form of a rectangular grid, stretched between the artist and the subject. Another uses a string representing a line of sight (fig. 12). The string is fixed on one end, while the other end is moved successively to key points on the subject. The point where the string crosses the projection plane, defined by a wooden frame, is recorded by two crossed strings. For each position, a hinged board attached to the frame is moved and the point of intersection is marked on its surface. It is hard to claim that such a device, which gave rise to many variations, made the creation of perspectival images more efficient, however the images it helped to produce had reassuring mechanical precision. Other major types of perspectival machines that appeared subsequently included the perspectograph, pantograph, physionotrace, and optigraph.
206 For a survey of perspectival instruments, see Martin Kemp, The Science of Art (New Haven:

Yale University Press, 1990), 167-220.
207 Ibid., 171-172. 4. 3-D Computer Graphics: Interactive Perspectivalism

Why manually move the string imitating the ray of light from point to point? Along with perspectival machines a whole range of optical apparatuses was in use, particularly for depicting landscapes and in conducting topographic surveys. They included versions of camera obscura from large tents to smaller, easily transportable boxes. After 1800, the artist's ammunition was strengthened by camera lucida, patented in 1806.208 Camera lucida utilized a prism with two reflecting surfaces at 135û. The draftsman carefully positioned his eye to see both the image and the drawing surface below and traced the outline of the image with a pencil. Optical apparatuses came closer than previous perspectival devices to the automation of perspectival imaging. However, the images produced by camera obscura or camera lucida were only ephemeral and considerable effort was still required to fix these images. A draftsman had to meticulously trace the image to transform it into the permanent form of a drawing. With photography, this time-consuming process was finally eliminated. The process of imaging physical reality, the creation of perspectival representations of real objects was now mechanized. However, this mechanization did not affect other uses of perspectival representation. According to Latour, the greatest advantage of perspective over other kinds of representations is that it establishes a "four-lane freeway" between physical reality and its representation. We can combine real and imagined objects in a single geometric model and go back and forth between reality and the model. By the twentieth century, the creation of a geometric model of both existing and imagined reality still remained a time consuming manual process, requiring the techniques of perspectival and analytical geometry, pencil, ruler, and eraser. Similarly, if one wanted to visualize the model in perspective, hours of drafting were required. And to view the model from another angle, one had to start all over again. The mechanization and automation of geometrical modeling and display were yet to come.

208 Ibid., 200.

Nothing perhaps symbolizes mechanization as dramatically as the first assembly lines installed by Henry Ford in U.S. factories in 1913. The assembly line relied on two crucial principles. The first was the standardization of parts, already employed in the production of military uniforms in the nineteenth century. The second, newer principle, was the separation of the production process into a set of repetitive, sequential, and simple activities that could be executed by workers who did not have to master the entire process and could be easily replaced. It seemed that mechanical modernity was at its peak. Yet, in the same year the Spanish inventor Leonardo Torres y Quevedo had already advocated the industrial use of programmed machines.209 He pointed out that although automatons existed before, they were never used to perform useful work: The ancient automatons...imitate the appearance and movement of living beings, but this has not much practical interest, and what is wanted is a class of apparatus which leaves out the mere visible gestures of man and attempts to accomplish the results which a living person obtains, thus replacing a man by a machine.210 With mechanization, work is performed by a human but his or her physical labor is augmented by a machine. Automation takes mechanization one step further: the machine is programmed to replace the functions of human organs of observation, effort, and decision. The term "automation" was coined in 1947; and in 1949 Ford began the construction of the first automated factories. Mass automation was made possible by the development of digital computers during World War II and thus became synonymous with computerization. A decade later, the automation of the process of constructing perspectival images of both existent and nonexistent objects and scenes was well underway.211 By the early 1960s Boeing designers already
209 Charles Eames and Ray Eames, A Computer Perspective: Background to the Computer Age

(Cambridge: Harvard University Press, 1990), 65-67.
210 Qtd. in ibid., 67. 211 I am not aiming here by any means to provide a full account of the history of 3-D computer graphics or its various uses. I am concerned with computer graphics as one development, among others, in the general move toward the rationalization of perspectival imaging. For a more

relied on 3-D computer graphics for the simulation of landings on the runway and of pilot movement in the cockpit (fig 13).212 By automating perspectival imaging, digital computers completed the process which began in the Renaissance. This automation became possible because perspectival drawing has always been a step-by-step procedure, an algorithm involving a series of steps required to project coordinates of points in 3-D space onto a plane. Before computers the steps of the algorithm were executed by human draftsmen and artists. With a computer, these steps can be executed automatically and, therefore, much more efficiently. The details of the actual perspective-generating algorithm which could be executed by a computer were published in 1963 by Lawrence G. Roberts, then a graduate student at MIT.213 The perspective-generating algorithm constructs perspectival images in a manner quite similar to traditional perspectival techniques. In fact, Roberts had to refer to German textbooks on perspectival geometry from the early 1800s to get the mathematics of perspective.214 The algorithm reduces reality to solid objects, and the objects are further reduced to planes defined by straight lines. The coordinates of the endpoint of each line are stored in a computer. Also stored are the parameters of a virtual camera -- the coordinates of a point of view, the direction of sight, and the position of a projection plane. Given this information, the algorithm generates a perspectival image of an object, point by point.

comprehensive account of 3-D computer graphics techniques, see J. William Mitchell, The Reconfigured Eye: Visual Truth in the Post-Photographic Era (Cambridge, The MIT Press, 1992), 117-162. 212 Jasia Reichardt, The Computer in Art (London and New York: Studio Vista and Van Nostrand Reinhold Company, 1971), 15. 213 L.G. Roberts, Machine Perception of Three-Dimensional Solids, MIT Lincoln Laboratory TR 315, 1963; L.G. Roberts, Homogeneous Matrix Representations and Manipulation of NDimensional Constructs, MIT Lincoln Laboratory MS 1405, 1965. 214 "Retrospectives II: The Early Years in Computer Graphics at MIT, Lincoln Lab, and Harvard," in SIGGRAPH '89 Panel Proceedings (New York: The Association for Computing Machinery, 1989), 72.

The subsequent development of computer graphics can be seen as the struggle to automate other operations involved in producing perspectival stills and moving images. The computerization of perspectival construction made possible the automatic generation of a perspectival image of a geometric model as seen from an arbitrary point of view -- a picture of a virtual world recorded by a virtual camera. But, just like with the early perspectival machines described by DŸrer, early computer graphics systems did not really save much time over traditional methods. To produce a film of a simulated landing, Boeing had to supplement computer technology with manual labor. As in traditional animation, twenty-four plots were required for each second of film. These plots were computer-generated and consisted of simple lines. Each plot was then hand-colored by an artist. Finished plots were filmed, again manually, on an animation stand.215 Gradually, throughout the 1970s and the 1980s, the coloring stage was automated as well. Many algorithms were developed to add the full set of depth cues to a synthetic image -- hidden line and hidden surface removal, shading, texture, atmospheric perspective, shadows, reflections, and so on.216 Today, these algorithms make possible the simulation of almost any object in such a way that its computer image is indistinguishable from the photograph. Yet, this by itself does not represent a radical break with older techniques of visual nominalism. A painter can paint the

215 This mixture of automated and pre-industrial labor is characteristic of the early uses of computers for the production of images. In 1955 the psychologist Attneave was the first to construct an image which was to become one of the icons of the age of digital visuality -random squares pattern. A pattern consisted of a grid made from small squares colored black or white. A computer generated table of random numbers has been used to determine the colors of the square -- odd number for one color, even number for another. Using this procedure, two research assistants manually filled in 19,600 squares of the pattern. Paul Vitz and Arnold B. Glimcher, Modern Art and Modern Science (New York: Praeger Publishers, 1984), 234. Later, many artists, such as Harold Cohen, used computers to generate line drawings which they then colored by hand, transferred to canvas to serve as a foundation for painting, etc. 216 For further discussion of the problem of realism in computer graphics, see Lev Manovich, "'Real' Wars: Esthetics and Professionalism in Computer Animation," Design Issues 6, no. 1 (Fall 1991): 18-25; Lev Manovich, "Assembling Reality: Myths of Computer Graphics," Afterimage 20, no. 2 (September 1992): 12-14.

same image, although it would take much longer. What does represent a radical break is interactive computer graphics. In 1962 Ivan Sutherland designed his now legendary Sketchpad program. With Sketchpad, a human operator could create graphics directly on the computer screen by touching the screen with a light pen. In the same year, ITEK began marketing its Electronic Drafting Machine, an apparatus similar to Sketchpad.217 Although both programs dealt only with 2-D graphics, they introduced a new paradigm of interactive graphics: by changing something on the screen, the operator changed the data in the computer's memory. 218 When this paradigm of interactive editing was combined with the algorithms of 3-D graphics, a radically new way of using perspectival images emerged. This development was more revolutionary than the automation of perspective construction per se. Indeed, traditional draftsman could have accomplished what the computer at Boeing was doing -- generating plots in perspective given 3-D database -- only more slowly. But now it became possible to change the point of view of a virtual camera and see the corresponding changes in the perspectival image after a short delay. It also became possible to build and modify 3-D models interactively and observe the changes on the screen. The emergence of interactive 3-D computer graphics started the race to eliminate the time delay between the action of an operator and the displayed results. To be able to move through virtual world in real time a computer has to generate 30 frames a second. The amount of

217 "Retrospectives II: The Early Years in Computer Graphics at MIT, Lincoln Lab, and

Harvard," 51. 218 In fact, interactive computer graphics technology appeared earlier, although it was not publicized. Already in the 1950s the Air Force used interactive CRT displays and light pens in order to more efficiently process information obtained by radar. Both CRT displays and light pens were designed at Lincoln Laboratory as part of the SAGE project. Using this technology, Lincoln researchers created a number of computer graphics programs. They included programs which made posible the display of brain waves (1957), of simulated planet and gravitational activity (1960), as well as the creation of 2-D drawings (1958). "Retrospectives II: The Early Years in Computer Graphics at MIT, Lincoln Lab, and Harvard," 42-54.

calculations involved increases proportionally as the model becomes more complex and as more depth cues are added such as shading and shadows. In this race for speed, which accelerated in the 1970s as synthetic images began to be utilized in flight simulators, the algorithms of 3-D graphics were gradually transported from software into hardware, each algorithm becoming a special computer chip. In 1966 Ivan Sutherland and his colleagues started research on the first head-mounted display, a prototype of Virtual Reality -- the ultimate interface for 3-D computer graphics. The research was cosponsored by ARPA (Advanced Research Projects Agency) and the Office of Naval Research.219 Describing the project, Sutherland wrote in 1968: The fundamental idea behind the three-dimensional display is to present the user with a perspective image which changes as he moves. The retinal image of the real objects which we see is, after all, only two-dimensional. Thus if we place suitable two-dimensional images on the observer's retinas, we can create the illusion that he is seeing a three-dimensional object...The image presented by the three-dimensional display must change in exactly the way that the image of a real object would change for similar motions of the user's head.220 The images seen in 1970 by the first users of Sutherland's system, such as a wireframe cube or a square room, looked remarkably similar to those utilized by Renaissance artists in their demonstration of perspective. Demonstrations of perspective always favored such simple geometrical objects because they most clearly showed the achievement of perspective -- the creation of a 2-D image of a 3-D object with foreshortening that matches our visual experience.221 The difference was that now the perspectival image of a cube was not fixed on a sheet of paper; it appeared to an observer as floating in the air in front of her, changing its view

219 Howard Rheingold, Virtual Reality (New York: Simon & Schuster, Inc., 1991), 105. 220 Qtd. in ibid., 104. 221 Later, Renaissance artists would deliberately insert checkerboard floors in perfect perspective in their paintings to impress their clients -- a move repeated in computer graphics, where such checkerboard background receding towards infinity became a favorite motif.

in correspondence with the slightest movement of the observer's head. Perspectival imaging became interactive. In order to achieve this interactivity, special hardware was built. To speed up the calculations necessary to update the synthetic image 30 times a second, Sutherland's group broke down the process of 3-D image synthesis into a series of discrete steps. Each step was delegated to a special electronic circuit: a clipping divider, a matrix multiplier, a vector generator. Later on, such circuits became specialized computer chips, connected together to achieve real-time, high resolution, photorealistic 3-D graphics. Silicon Graphics, one of the major manufacturers of computer graphics hardware, labeled such a system "geometry engine." The term appropriately symbolizes the second stage of the automation of perspectival imaging. At the first stage, the photographic camera, with perspective physically built into its lens, mechanized the process of creating perspectival images of existing objects. Now, with the perspectival algorithm and other necessary geometric operations embedded in silicon, it becomes possible to display and interactively manipulate models of non-existent objects as well. In the next section, I will discuss the final stage in the automation of perspectivalism -the development of computer vision.

In his papers, published between 1963 and 1965, Roberts formalized the mathematics necessary for generating and modifying perspective views of geometric models on the computer. This, writes William J. Mitchell, was "an event as momentous, in its way, as Brunelleschi's perspective

5. Computer Vision: Automation of Sight

demonstration."222 However, Roberts developed techniques of 3-D computer graphics having in mind not the automation of perspectival imaging but another, much more daring goal -- "to have

222 Mitchell, The Reconfigured Eye

, 118.

the machine recognize and understand photographs of three dimensional objects."223 Thus, the two fields were born simultaneously: 3-D computer graphics and computer vision, automation of imaging and of sight. The field of computer vision can be seen as the culmination of at least centuries-long histories. The first is the history of mechanical devices designed to aid human perception, such as Renaissance perspectival machines. This history reaches its final stage with computer vision,


"Retrospectives II: The Early Years in

Computer Graphics at MIT, Lincoln Lab, and



which aims to replace human sight altogether. The second is the history of automata, whose construction was especially popular in the seventeenth and eighteenth centuries. Yet, despite similarity in appearance, there is a fundamental difference between Enlightenment automata which imitated human's or animal's bodily functions and the modern day robots equipped with computer vision systems, artificial legs, arms, etc. As noted by Leonardo Torres, old automata, while successfully copying the appearance and movement of living beings, had no economic value. Indeed, such voice synthesis machines as Wolgang von Kempelen's 1778 device which directly imitated the functioning of the oral cavity or AbbŽ Mical's T tes Parlantes (1783) operated by a technician hiding offstage and pressing a key on a keyboard were used only for entertainment.224 When in 1913 Torres called for automata that would "accomplish the results which a living person obtains, thus replacing a man by a machine" he was expressing a fundamentally new idea of using automata for productive labor. A few years later, the brother of the Czech writer Karel Capek coined the word robot from the Czech word robota, which means "forced labor."225 Capek's play R.U.R. (1921) and Fritz Lang's Metropolis (1927) clearly demonstrate this new association of automata with physical industrial labor. Therefore, it would be erroneous to conclude that, with computer vision, twentieth century technology simply added the sense of sight to eighteenth century mechanical statues. But even to see computer vision as the continuation of Torres', Capek's or Lang's ideas about industrial automation which replaces manual labor would not be fully accurate. The idea of

224 Remko Scha, "Virtual Voices," MediaMatic 7, no. 1 (1992): 33. Scha describes two

fundamental approaches taken by the developers of voice imitating machines: the genetic method which imitates the physiological processes that generate speech sounds in the human body and the gennematic method which is based on the analysis and reconstruction of speech sounds themselves without considering the way in which the human body produces them. While the field of computer vision, and other fields of artificial intelligence, first clearly followed gennematic method, in the 1980s, with the growing popularity of neural networks, there was a shift towards the genetic method -- direct imitation of the physiology of the visual system. In a number of laboratories, scientists begin to build artificial eyes which move, focus, and analyze information exactly like human eyes. 225 Eames and Eames, A Computer Perspective, 100.

computer vision became possible and the economic means to realize this idea became available only when automation entered its second stage after World War II. The attention turned from the automation of the body to the automation of the mind, from physical to mental labor. This shift to the automation of mental functions such as vision, hearing, reasoning, problem solving is exemplified by the very names of the two new fields that emerged during the 1950s and 1960s -artificial intelligence and cognitive psychology. The latter gradually replacing behaviorism, the dominant psychology of the "Fordism" era. The emergence of the field of computer vision is a part of this cognitive revolution, a revolution which was financed by the military escalation of the Cold War.226 This connection is solidified in the very term "artificial intelligence" which may refer simultaneously to two meanings of "intelligence": reason, the ability to learn or understand, and information concerning an enemy or a possible enemy or an area. Artificial intelligence: artificial reason to analyze collected information, collected intelligence. In the 1950s, faced with the enormous task of gathering and analyzing written, photographic, and radar information about the enemy, the CIA and the NSA (National Security Agency) began to fund the first artificial intelligence projects. One of the earliest projects was a Program for Mechanical Translation, initiated in the early 1950s in the attempt to automate the monitoring of Soviet communications and media.227 The work on mechanical translation was probably the major cause of many subsequent developments in modern linguistics, its move towards formalization; it can be discerned in Noam Chomsky's early theory which, by postulating the existence of language universals in the domain of grammar, implied that translation between arbitrary human languages could be automated. The same work on mechanical translation was also one of the catalysts in the development of the field of pattern recognition, the precursor to computer vision. Pattern recognition is concerned with

226 De Landa, "Policing the Spectrum," 194-203. 227 Ibid., 214.

automatically detecting and identifying predetermined patterns in the flow of information. A typical example is character recognition, the first stage in the process of automating translation. Pattern recognition was also used in the U.S. for the monitoring of Soviet radio and telephone communication. Instead of listening to every transmission, an operator would be alerted if computer picked up certain words in the conversation. As a "logistics of perception" came to dominate modern warfare and surveillance and as the space race began, image processing became another major new field of research. Image processing comprises techniques to improve images for human or computer interpretation.228 In 1964, the space program for the first time used image processing to correct distortions in the

228 The first paper on image processing was published in 1955.

L.S.G. Kovasznay, and H.M. Joseph, "Image Processing," Proceedings of IRE 43 (1955): 560-570.

pictures of the Moon introduced by a on-board television camera of Ranger 7.229 In 1961, the National Photographic Interpretation Center (NPIC) was created to produce photoanalysis for the rest of the U.S. intelligence community and, as Manual De Landa points out, by the end of the next decade computers "were routinely used to correct for distortions made by satellite's imaging sensors and by atmospheric effects, sharpen out-of-focus images, bring multicolored single images out of several pictures taken in different spectral bands, extract particular features while diminishing or eliminating their backgrounds altogether..." De Landa also notes that computer analysis of photographic imagery also became the only way to deal with the pure volume of intelligence being gathered: "It became apparent during the 1970s that there is no hope of keeping up with the millions of images that poured into NPIC...by simply looking at them the way they had been looked at in World War II. The computers therefore also had to be taught to compare new imagery of a given scene with old imagery, ignoring what had not changed and calling the interpreter's attention to what had." 230

229 Rafael C. Gonzalez, and Paul Wintz,

Digital Image Processing (Reading, Mass.: Addison-Wesley Publishing Company, 1977), 2.
230 Qtd. in De Landa, "Policing the Spectrum," 200.

The techniques of image processing, which can automatically increase an image's contrast, remove the effects of blur, extract edges, record differences between two images, and so on, greatly eased the job of human photoanalysts. And the combining of image processing with pattern recognition made it possible in some cases to delegate the analysis of photographs to a computer. For instance, the technique of pattern matching used to recognize printed characters can also be used to recognize objects in a satellite photograph. In both cases, the image is treated as consisting of two-dimensional forms. The contours of the forms are extracted from the image are then compared to templates stored in computer memory. If a contour found in the image matches a particular template, the computer signals that a corresponding object is present in a photograph. A general computer vision program has to be able to recognize not just two-dimensional but three-dimensional objects in a scene taken from an arbitrary angle.231 Only then it can be used to recognize an enemy's tank, to guide an automatic missile towards its target or to control a robotic arm on the factory floor. The problem with using simple template matching is that "a two-dimensional representation of a two-dimensional object is substantially like the object, but a two-dimensional representation of a three-dimensional object introduces a perspective projection that makes the representation ambiguous with respect to the object."232 While pattern recognition was working for images of two-dimensional objects, such as letters or chromosomes, a different approach was required to "see" in 3-D. Roberts' 1965 paper "Machine Recognition of Three-dimensional Solids" is considered to be the first attempt at solving the general task of automatically recognizing three-dimensional

231 Within the field of computer vision, a scene is defined as a collection of three-dimensional objects depicted in an input picture. David McArthur, "Computer Vision and Perceptual Psychology," Psychological Bulletin 92, no. 2 (1982): 284. 232 Paul R. Cohen and Edward A. Feigenbaum, eds., The Handbook of Artificial Intelligence (Los Atlos, CA: William Kaufmann, Inc., 1982), 3: 139.

objects.233 His program was designed to understand the artificial world composed solely of polyhedral blocks -- a reduction of reality to geometry that would have pleased CŽzanne. Using image processing techniques, a photograph of a scene was first converted into a line drawing. Next, the techniques of 3-D computer graphics were used: Roberts' program had access to three-dimensional models of objects: a cube, a rectangular solid, a wedge, and a hexagonal prism. They were represented by coordinates (x, y, z) of their vertices. A

L.G. Roberts, "Machine perception of three-dimensional solids," Optical and Electo Optical Information Processing, ed. J.T. Tippett


(Cambridge: The MIT Press, 1965)


program recognized these objects in a line drawing of the scene. A candidate model was selected on the basis of simple features such as a number of vertices. Then the selected model was rotated, scaled, projected, and matched with the input line drawing. If the match was good, the object was recognized, as were its position and size. Roberts' program could handle even a composite object made of multiple primitive shapes; it subtracted parts of a line drawing from the drawing as they were recognized, and the remaining portions were analyzed further.234 Was this enough to completely automate human vision? This depends upon how we define vision. The chapter on computer vision in The Handbook of Artificial Intelligence (1982) opens with the following definition: "Vision is the information-processing task of understanding a scene from its projected images."235 But what does "understanding a scene" mean? With computer vision research financed by the military-industrial complex, the definition of understanding becomes highly pragmatic. In the best tradition of the pragmatism of James and Pierce, cognition is equated with action. The computer can be said to "understand" a scene if it can act on it -- move objects, assemble details, destroy targets. Thus, in the field of computer vision "understanding a scene" implies two goals. First, it means the identification of various objects represented in an image. Second, it means reconstruction of three-dimensional space from the image. A robot, for instance, need not only recognize particular objects, but it has to construct a representation of the surrounding environment to plan its movements. Similarly, a missile not only has to identify a target but also to determine the position of this target in threedimensional space. It can be seen that Roberts' program simultaneously fulfilled both goals. His program exemplified the approach taken by most computer vision researchers in the following two decades. A typical program first reconstructs the three-dimensional scene from the input image and then matches the reconstructed objects to the models stored in memory. If the match is good, the program can be said to recognize the object, while simultaneously recording its position (fig. 14).
234 Cohen and Feigenbaum, The Handbook of Artificial Intelligence, 3: 129. 235 Ibid., 127.

A computer vision program thus acts like a blind person who "sees" objects (i.e., identifies them) by reading their shapes through touch. As for a blind person, understanding the world and the recognition of shapes are locked together; they cannot be accomplished independently of one another. With computer vision, visual nominalism, which, after all, is just a particular use of vision, attains unprecedented importance. Now, to see means to see shapes and distances. Visual nominalism and vision itself have become equated.

I have presented the new twentieth century technologies of visual nominalism as building upon perspective, extending its powers in space and time. Technologies of remote sensing, such as radar, extended perspective beyond the realm of the visible. 3-D computer graphics speeded up and automated design and perspectival display of the models of both real and imagined objects. With the emergence of the field of computer vision, perspectival sight reaches its apotheosis and at the same time begins its retreat. At first computer vision researchers believed that they could invert the perspective and reconstruct the represented scene from a single perspectival image. Eventually, they realized that it is often easier to bypass perspectival images altogether and use other means as a source of three-dimensional information. Latour points out that with the invention of perspective it became possible to represent absent things and plan our movement through space by working on representations. To quote him again, "one cannot smell or hear or touch Sakhalin island, but you can look at the map and determine at which bearing you will see the land when you send the next fleet."236 Roberts' program extended these abilities even further. Now the computer could acquire full knowledge of the three-dimensional world from a single perspectival image! And because the program determined the exact position and orientation of objects in a scene, it became possible to see the reconstructed scene from another viewpoint. It also became possible to predict how the scene
236 Latour, "Visualisation and Cognition," 8.

would look from an arbitrary viewpoint.237 Finally, it also became possible to guide automatically the movement of a robot through the scene. Roberts' program worked using only a single photograph -- but only because it was presented with a highly artificial scene and because it "knew" what it could expect to see. Roberts limited the world which his program could recognize to simple polyhedral blocks. The shapes of possible blocks were stored in a computer. Others simplified the task even further by painting all objects in a scene the same color. However, given an arbitrary scene, composed from arbitrary surfaces of arbitrary color and lighted in an arbitrary way, it is very difficult to reconstruct the scene correctly from a single perspectival image. The image is "underdetermined." First, numerous spatial layouts can give rise to the same two-dimensional image. Second, "the appearance of an object is influenced by its surface material, the atmospheric conditions, the angle of the light source, the ambient light, the camera angle and characteristics, and so on," and all of these different factors are collapsed together in the image.238 Third, perspective, as any other type of projection, does not preserve many geometric properties of a scene. Parallel lines turn into convergent lines; all angles change; equal lines appear unequal. All this makes it very difficult for a computer to determine which lines belong to a single object. Thus, perspective, which until now served as a foundation of new technologies of visual nominalism, becomes the drawback which needs to be overcome. Perspective, this first step towards the rationalization of sight (Ivins) has eventually become a limit to its total rationalization -- the development of computer vision. The realization of the ambiguities inherent in a perspectival image itself came after years of vision research. In trying to compensate for these ambiguities, laboratories began to scrutinize

237 Cohen and and Feigenbaum, The Handbook of Artificial Intelligence, 3: 141. 238 Ibid., 128.

the formal structure of a perspectival image with a degree of attention unprecedented in the history of perspective. In 1968 Adolpho Guzman classified the types of junctions that appear in line representations after he realized that a junction type can be used to deduce whether regions of either side of a junction line were part of the same object. 239 In 1986 David Lowe presented a method to calculate the probability that a particular regularity in an image (for instance, parallel lines) reflects the physical layout of the scene rather than being an accident due to a particular viewpoint.240 All other sources of depth information such as shading, shadows or texture gradients were also systematically studied and described mathematically. Despite these advances, a single perspectival image remained too ambiguous a source of information for practical computer vision systems. An alternative has been to use more than one image at a time. Computer stereo systems employ two cameras which, like human eyes, are positioned a distance apart. If the common feature can be identified in both images, then the position of an object can be simply determined through geometric calculations. Other systems use a series of continuous images recorded by a video camera. But why struggle with the ambiguity of perspectival images at all? Instead of inferring three-dimensional structure from a two-dimensional representation, it is possible to measure depth directly by employing various remote sensing technologies. In addition to video cameras, modern vision systems also utilize a whole range of different range finders such as lasers or ultrasound.241 Range finders are devices which can directly produce a three-dimensional map of an object. The same basic principle employed in radar is used: the time required for an electromagnetic wave to reach an object and reflect back is proportional to the distance to the object. But while radar reduces an object to a single point and in fact is blind to close-by objects, a range
239 Ibid., 131. 240 David Lowe, Three-Dimensional Object Recognition from Single Two-Dimensional Images,

Robotics Report 62 (New York: Courant Institute of Mathematical Sciences, New York University, 1986). 241 Cohen and Feigenbaum, The Handbook of Artificial Intelligence, 3: 254-259.

finder operates at small distances. By systematically scanning the surface of an object, it directly produces a "depth map," a record of an object's shape which can be then matched to geometric models stored in computer memory thus bypassing the perspectival image altogether (fig. 15).

What began as separate developments -- to automate the recording of three-dimensional information, the generation of perspectival views, and the identification of objects -- eventually converged. Remote sensing, 3-D computer graphics, and computer vision form a closed circle. As we have seen, algorithms of 3-D computer graphics were first developed by Roberts in order to solve the general problem of computer vision. From that point on, the two fields developed in parallel. The field of computer graphics was gradually learning to simulate more and more "realistic" images of reality, adding shading, shadows and texture to initial wireframe drawings. Meanwhile, the field of computer vision was learning to deal with shading, shadows, and texture as sources of depth information. In effect, the goal of computer vision is to undo what 3-D computer graphics aims to achieve. The former tries to reconstruct a scene from its photographic image, to deduce viewer independent information about the objects: their shapes and dimensions. The goal is to "produce an object centered representation that is independent of the particular details of the viewing angle or photographic process."242 The aim of 3-D computer graphics is exactly the opposite: given the objective information about the scene (shapes and positions of objects, their materials, direction of light) to produce its image which is virtually indistinguishable from a photograph. At the foundation of this mutual dependence is the shared geometrical model of reality as a set of surfaces having a particular orientation, position, and curvature. The circle is completed by the technologies of remote sensing that, by producing depth maps of real scenes, can

242 David Peat, Artificial Intelligence: How Machines Think (New York: Baen Enterprises,

1985), 230.

automatically reduce them to geometric models. Portable shape acquisition cameras are already being developed that, instead of taking snapshots of a scene, will automatically generate geometric models which then can be used for computer vision or 3-D computer graphics.243 Technologies of remote sensing can automatically create a depth map of reality. Using such depth map as input, 3-D computer graphics can be used to generate an image which, in its turn, can act as input to a computer vision system. Computer vision then can reconstruct "objective" information from an image, a model which can act as a source for further 2-D computer graphics synthesis. A series of mirrors, constructions and reconstructions are triggered by each other, with speed only limited by the processing power of "geometry engines."

In this chapter I discussed the twentieth century automation of the function of vision which I have called visual nominalism -- capturing the information about shapes and distances and representing this information in two-dimensional displays. Radar and other technologies of remote sensing extend the capacities of human vision beyond the limits of visibility, making it possible to track positions of objects in real time. Techniques of 3-D computer graphics allow the interactive display and construction of geometric models of both real and non-existent objects. Finally, computer vision automates the recognition of objects and the reconstruction of threedimensional scenes from their images. In this discussion, the concept of perspective played a special role for a number of reasons. I relied on the familiarity of this concept to explain the principles of operations of newer twentieth century technologies of visual nominalism. I have also relied on Ivins' account of perspective and its extension by Latour to suggest that the Renaissance's adaptation of
243 Rheingold, 6. Conclusion Virtual Reality, 229, 252.

perspective represented the first step in the automation of visual nominalism. While other cultures used sophisticated methods of space mapping, the importance of perspective lies not in its representational superiority but in its algorithmic character. This algorithmic character enabled the gradual development of visual languages of perspective and descriptive geometry and, in parallel, of perspectival machines and technologies, from a simple net described by DŸrer to photography and radar. And when digital computers made possible mass automation in general, automation of perspectival vision and imaging followed soon. It is this automation of vision and imaging which gave the West superiority in the "logistics of perception," and contributed to its victory in the Gulf War. A two-dimensional representation containing information about geometry and topography allows one to control reality across space and time. The advantage of perspective is that it makes it possible to present this information in an easily readable form within a single image. In the West, until this century, perspective formed the foundation of techniques and technologies of visual nominalism. New twentieth century technologies of visual nominalism extend perspective, utilizing to the extreme its inherent qualities such as the algorithmic character and the reciprocal relationship it establishes between reality and representation. For instance, the perspective algorithm, a foundation of both computer graphics and computer vision, is used to generate perspectival views given a geometric model and to deduce the model given a perspectival view. In another example, technologies of remote sensing, such as radar, extend perspectival vision beyond the visible, relying on the fact that "what is an issue in geometric perspective is simply the mapping of space, not sight" (Lacan). Yet, while giving rise to new technologies of "geometric vision," perspective also becomes a limit to the final automation of sight -- recognition of objects by a computer. Finally, it is displaced from its privileged role, becoming just one among other techniques of space mapping and visualization.

In order to automate sight, to replace human vision with machine vision, a new understanding of vision became necessary. David Marr's Vision (1980) has been the most influential account of this paradigm, shared by computer scientists and psychologists. The book opens with this statement: What does it mean, to see? The plain man's answer (and Aristotle's, too) would be, to know what is where by looking. In other words, vision is the process of discovering from images what is present in the world, and where it is.244 There is nothing "plain" about this definition of vision. Marr projects the goals of computer vision onto human vision: the identification of objects represented in an image and the reconstruction of their positions in three-dimensional space. The larger part of the book is devoted to the discussion of algorithms by which the human nervous system may accomplish this reconstruction. In this way, nature was redefined as the most useful instrument in its own replacement. What David Marr, and many others have assumed, is that the key to successful computer vision systems lies in emulating the algorithms hidden in human nervous system. If only these algorithms, created by nature during millions of years of evolution, could be understood, the way to replace human vision by computer vision will be open. The automation of vision is a part of the overall process of industrial automation after World War II. Automation affects the modern understanding of vision in two crucial ways. On the one hand, automation entails the replacement of human cognitive functions by a computer, such as the substitution of vision by computer vision. Here, visual nominalism becomes the dominant paradigm for the study of vision. On the other hand, the automation also involves the integration of human and machine in new man-machine systems, such as radar installations or pilot cockpits. The need to describe the performance of human and machine components in the
244 Marr, Vision, 3.

same terms leads to the emergence of another paradigm -- vision as information processing -which will be discussed in the following chapter.

Writing in 1927, L‡szl— Moholy-Nagy makes a comparison between engineering and design: A modern engineer, if his goal and the functional purpose of his work are clear, can without any great effort make a product that is formally adequate and perfect in its economic construction. But the photographic advertisements of our time are not so easy to define. They don't come with "user's instructions." Research into the physiological and psychological laws of visual effectiveness is still far behind the times, compared to the study of the physical laws.245 In the opening chapter, where this quotation already appeared, I have traced one line of research into "the physiological and psychological laws of visual effectiveness," suggesting that investigations of the psychological effects of basic colors and elemental forms, conducted in experimental psychology since the second half of the nineteenth century made possible the idea of a rational visual language composed from simple elements. In theory, this model called for the enumeration of visual elements and the description of their effects in order to compose dictionaries of "visual effectiveness." In practice, it translated into visual works which at first only contained and then gradually became fully composed of simple abstract forms. Such theory and practice were already pursued in the nineteenth and early twentieth centuries by a number of artists (Seurat, Signac, Kandinsky, Klee, Mondrian) and critics interested in "scientific aesthetics" (Blanc, Henry). However, in the 1920s, when artists found themselves in the roles of the designers of mass communication, the idea of a scientifically grounded visual language acquired new urgency and importance. It was no longer a question of scientific aesthetics, of a work of art producing an aesthetic effect in an individual spectator. Now it became a question of rationalizing mass communication -- a question of economic and political importance. The design of mass communication was too important to be left to an artist's intuition. Therefore,
245 L‡szl— Moholy-Nagy, "Photography in Advertising," in Photography in the Modern Era, ed. Christopher Engineers Chapter 4: The Phillips (New of Vision from INKhUK87. MIT 1. Not Artists butEngineering York: Aperture, 1989), to

Moholy-Nagy wrote about the need to precisely "engineer" photographic advertisements, and the Bauhaus welcomed experimental psychologists. The trend reached its extreme in Soviet Russia, where artists thought that they could control not just the consumer habits of segments of society (as in the West), but the consciousness of the whole country --"affecting mass psyche, organizing the will of the class" (Tretyakov). Here the discourse of engineering visual communication became most systematic, translating into the establishment of a number of psychological laboratories at various art institutions, such as VKhUTEMAS, GAKhN, and the State Institute of Artistic Culture. Their goal was to create a fully rationalized visual language of mass communication where each visual element is capable of communicating a meaning, producing an emotion or causing a behavioral response. In the words of a paper entitled "Engineerism" presented at INKhUK: "visual sensations as such will concertedly shape the human being as an organized unit."246 This is one development, one trajectory -- from Blanc's theories and Fechner's psychophysical experiments to INKhUK and VKhUTEMAS -- where the dream of controlling the masses through perception was systematically pursued. This trajectory leads from Wundt's psychological laboratory to the "laboratory" explorations of constructivists and rationalists; and from the laboratory to the sites of mass communication -- a movie hall, a printed page, an architectural space, a stadium, a city street with posters, advertisements, and placards. In this concluding chapter I will follow the next part of this trajectory, the next stage in the rationalization of visual communication: from the avant-garde's fantasies of psychophysical control of the masses to information theory, engineering psychology, and human-machine interfaces. The visual language advocated by Rodchenko or Lissitsky -- elemental abstract forms, each having a predictable effect -- seemed the most effective solution to the problem of

246 Nicholas H. Allison, ed., Art Into Life: Russian Constructivism 1914-1932 (New York:

Rizzoli International Publications, 1990), 81.

rationalizing vision. But after World War II, a new kind of rationalization became necessary. A new paradigm was now needed to encompass not only human communication but also communication between a human and a machine, the two linked in a human-machine system. It is the emergence of this paradigm which is the subject of this chapter. The new paradigm, called information theory, emerged in the 1920s in response to the growth of modern telecommunications. The theory was originally developed by engineers in order to measure the efficiency of communication systems, such as telephone, radio, and television. However, in the late 1940s, the theory left its engineering context and was embraced by aestheticians, linguists, social scientists, and semioticians. Most importantly, it was taken up by psychologists faced with the practical task of designing human-machine systems which increasingly came to dominate both the battlefield and the workplace after World War II (radar screen, aircraft controls, computer terminals of the automated factory). As human vision became the main channel of communication between human and machine, it also came to be understood in terms of information theory which now provided a new quantitative model of a human observer. Thus the dreams of the avant-garde for an engineering approach to visual communication were fulfilled.

The avant-garde was concern with rationalizing vision, with creating visual communication according to scientific principles. In this avant-garde artists likened themselves to modern engineers, as can be seen from Moholy-Nagy's statement. But what is a modern engineer? The engineering profession emerged in the 1870s as the liaison between science and industry. The job of an engineer was to put any industrial or corporate process on a scientific basis in order to achieve the maximum output with a minimum investment of time, materials, and energy. In other words, the job of an engineer was to make a process efficient, regardless of what industry s/he worked in. A 1933 book on engineering defines it in this way: "Engineering is

the science and art of efficient dealing with materials and forces...it involves the most economical design and execution of a vast number of important undertakings, assuring, when properly performed, the most advantageous combination of accuracy, safety, durability, speed, simplicity, efficiency and economy possible for the design and service."247 The avant-garde artists aimed to apply these principles of engineering to the field of mass visual communication. In 1921, during a heated discussion at INKhUK (Institute of Artistic Culture) over how to turn their fellow artists into "constructivists," who would utilize modern engineering methods for construction -- be it the construction of consumer goods or mass psyche, industrial design or mass propaganda -- participants expressed doubts about whether traditional artists could accomplish this transition from "art into life." Boris Arvatov suggested that accomplished artists should become political activists, and the young ones would have to go to the polytechnic.248 More categorical was the opinion of Sternberg, who, after asking the same question, "What are artists to do?," answered it in these words, "They're good for nothing, they should be dealt with the way the Cheka deals with counterrevolutionaries."249 A few minutes later in the discussion he suggested a more humane solution, the solution which was repeatedly put into practice in regard to Soviet non-conformist artists decades later -- their confinement to a mental institution. "All the artists who conduct youth groups, and who are going to continue their work and perhaps teaching the way they have in the past, we should really stuff a hospital with these artists."250 Arvatov's request that artists should get their training in an engineering school, and Sternberg's warning that those who refuse to comply would be shot, illustrate the grave seriousness with which many Soviet avant-garde artists of the 1920s took the ideal of engineering as a model for their own practice. Consider the term constructivism itself. In
247 Qtd. in Theodore Hoover and Charles Fish, The Engineering Profession (Stanford: Stanford

University Press, 1941), 416. Emphasis mine -- L.M.
248 Allison, Art Into Life, 74. 249 Ibid. 250 Ibid., 77.

Russian, konstruktor (the one who constructs) is a synonym for engineer. This allowed a number of artists to call themselves konstruktors and to refer to their movement as constructivism. The term conveniently emphasized that, like engineers, they would construct utilitarian objects or propagandistic works on the basis of scientific, rational principles, in contrast to traditional artists who were said to rely on inspiration. Translated into English, the word constructivism loses its close association with engineering, leading to a somewhat different interpretation of the entire movement. In the early 1920s, constructivist artists competed to be the best at embodying the ideal of an engineer. Vertov announces his desire to become "kinok-engineer, controlling cameras over distance."251 Eisenstein, who, like Sternberg, graduated from an engineering school before the 1917 Revolution, compares the theatrical apparatus (actors' performances, sound effects, lighting) to the machinery (orudie obrabotki) which a director should use to form the viewer -- as a milling machine forms machinery parts.252 Later, he likens the effect of a film to the work of an engine -- a significant analogy since engineering was historically identified with the design of engines.253 Tatlin photographs himself next to his already legendary design for the monument to the Third International. The monument, with its two intertwining lattice spirals 400 meters high, looks like an Eiffel Tower put at an angle, eternally moving into the sky, a symbol appropriate to signify the idea of permanent revolution, the (future) industrial might of the new state and, at the same time, the engineering skills of its designer. Finally, Lissitsky and his students design another triumph to engineerism -- an inclined lattice girder entitled "Lenin Tower."
251 Dziga Vertov, "Kinoki. Perevorot" (Kinoki. A revolution), LEF 3 (1923): 143. 252 Sergei Eisenstein, "Montazh atraktsionov (Montage of attractions)," LEF 3 (1923): 71. 253 A book on the engineering profession published in 1941 quotes these dictionary definitions

of engineering: "engineering: originally, the art of making engines; in its modern and extended sense, the art and science by which mechanical properties of matter are made useful to man in structures and machines;" "engineering: the art and science of making, building, or using engines and machines, or of designing and constructing public works or the like requiring special knowledge of materials, machinery, and the laws of mechanics." Hoover and Fish, The Engineering Profession, 416.

Here is another device: similar to traditional portraits where the identity of a person was established by depicting the instruments of his or her craft, constructivists' portraits and selfportraits prominently feature engineering instruments. Lissitsky's self portrait: on the background of graphed paper -- his face and through it -- a hand holding a compass, its leg touching a perfectly drawn square. The title of this photomontage is Konstruktor. Even more dramatic is another Lissitsky's montage, a portrait of Tatlin (fig. 16). Tatlin is standing on a chair, holding a long pole -- a ruler. By his feet are mathematical symbols (an integral, a square root), a curve crossing axis lines. Behind him is a blackboard with more symbols. And, a monstrosity: a huge compass, widely open, growing straight from Tatlin's eye. A man-compass -- measuring, calculating, constructing. Constructing what? "My reaction to the participation of the artist in production is positive," declared architect Vesnin during one INKhUK discussion, "but an artist must look after his own affairs, and his affair is the [psycho-physiological] effect of form on [human] consciousness." In other words, Vesnin is saying: let us leave the engineering of machines and mass-produced objects to real engineers. Our business, the business of designers is the engineering of perception, the engineering of visual communication -- the creation of architecture, mass spectacles, posters, designs, photomontages, which would produce calculated bodily responses, emotions, and meanings in the viewers. "The modern engineer has created brilliant objects: the bridge, the steam engine, the airplane, the crane,...the modern artist must create objects that are equal to them in power, intensity, and potential in the context of their psycho-physiological impact as an organizing element in man's consciousness."254 The job of an artist is to engineer vision, and to engineer vision means to affect the viewer with engineering precision, predictability, and effectiveness.

254 Qtd. in Anatole Senkevitch, Jr., "The Sources and Ideals of Constructivism in Soviet Architecture," in Art Into Life: Russian Constructivism 1914-1932, ed. Nicholas H. Allison (New York: Rizzoli International Publications, 1990), 173-174.

In this statement from the early 1920s, Vesnin envisions an engineer as the designer of physical objects -- the bridge, the steam engine, the airplane, the crane. But already in the same decade a new kind of engineer appears, whose job is to design not objects but communication systems -- telephone, radio, television. A new paradigm appears as well -- information theory which becomes the scientific basis of communication engineering. In the next section, I will follow its emergence.

In 1922 Moholy-Nagy created his famous "Telephone Pictures." Here is his account: In 1922 I ordered by telephone from a sign painter five paintings in porcelain enamel. I had the factory's color chart before me and I sketched my paintings on graph paper. At the other end of the telephone the factory supervisor had the same kind of paper, divided into squares. He took the dictated shapes in correct position. (It was like playing chess by correspondence.)255 There is no conclusive proof that this event actually took place. At any rate, the story helped Moholy-Nagy get a teaching job at the Bauhaus the next year, where he could fully pursue his vision of using the latest communication systems as artistic tools. His colleague wrote of the Bauhaus atmosphere: "There is incessant talk of cinema, optics, mechanics, projection and continuous motion...Is this the atmosphere in which painters like Klee and some others of us can go on developing? Klee was quite depressed yesterday when talking about Moholy."256 The story of "Telephone Pictures" became one of the myths of modern art, or rather, of the modern struggle against the Romantic notion of art: to refuse subjectivity, the unique touch

255 L‡szl— Moholy-Nagy, The New Vision (New York: Wittenborn, 1947), 79. The three

existing accounts of this episode are discussed in Eduardo Kac, "Aspects of the Aesthetics of Telecommunications," in SIGGRAPH '92 Visual Proceedings, ed. John Grimes and Gray Lorig (New York: The Association for Computing Machinery, 1992), 52-53. 256 Qtd. in Otto Stelzer, "Moholy-Nagy and His Vision," in L‡szl— Moholy-Nagy, Painting, Photography, Theory: an Engineer Analyzes Communication 2. Information Film (1925; reprint, Cambridge: The MIT Press, 1973), 146.

of the artist's hand and the artist's eye (exemplified in Bauhaus by Klee) and to embrace the mass-produced, the industrial, the mechanical, electrical, electronic, and digital -- from Duchamp's ready-made to Op Art, Kinetic Art, copy art, telecommunication art, fax art, computer art -- in short, the tradition of industrial art, throughout the twentieth century feeding on the latest technical developments in communication and imaging. And Moholy-Nagy intended the story this way, as a myth, a beginning more important than Duchamp's readymades. Indeed, not just art as chess, as a logical game (as with Duchamp), but -- art as "playing chess by correspondence," a communication over distance between a sender and a receiver who do not occupy the same physical space. Let us analyze Moholy-Nagy's story from the point of view of telecommunication -- the communication over distance. It mentions three different communication systems. First, the most ancient: communication with the help of signs, such as traffic or street signs (Moholy-Nagy orders his pictures at the sign factory). Second, more modern: the postal service. Third, still a recent development for the 1920s: the telephone. As communication systems developed, they made possible the transmission of more information over longer distances in a shorter period of time. The effectiveness of road signs was limited by the distance at which they were visible, and the amount of information they carried was minimal. With the postal service, this distance limit was overcome, but a significant time delay remained. This delay was finally eliminated by the telephone. In the story, Moholy-Nagy communicated his design to the factory supervisor over the telephone, dictating coordinates and colors of squares. He was able to do this only by fitting his design to a pre-existing factory grid, treating the grid's squares as minimal units of his composition. (Moholy-Nagy, in other words, fully embraced digital visuality and computer art long before digital computers.) But could he, in 1922, transmit this design as a picture -- and in

real time? And what if the picture was an arbitrary image, for instance a photograph of a real scene, rather than a by-product of the communication code? The rapid transmission of photographic pictures was just becoming possible, utilizing the same method employed by Moholy-Nagy -- digital representation. In 1921, digitized newspaper photographs began to be regularly sent between London and New York over a submarine cable in less than three hours. On one end, a picture was segmented using a grid, and the tone of each cell was coded; on another end, the received code would drive a special printer fitted with typefaces simulating different tones.257 A few years later, Moholy-Nagy included the examples of this "wireless telegraphed photography" in his Painting, Photography, Film, proudly putting them at the very end of the photographic section of the book to represent the latest achievement in mechanized tele-seeing (fig. 17). Yet, this system was not fast enough for the age of "the film; the electric sign, simultaneity of sensorily perceptible events."258 The speed of wireless telegraphed photography was limited by the channel capacity of an older communication system -- telegraphy. Images contained much more information than telegraphic cable was originally designed to carry, so speed had to be sacrificed. Throughout the 1920s, engineers in Russia, the U.S., England, and Germany were working on a new system to transmit images in real time called television -vision over distance. The explosive development of modern communication systems -- telegraphy, telephone, radio, and television, along with their growing economic importance -- urged the creation of a general communication theory.259 With telephone, radio, and television becoming the foundations of whole new industries in the earlier decades of the century, the question of
257 M. D. McFarlane, "Digital Pictures Fifty Years Ago," Proc. IEEE 60, no. 7 (1972). 258 Moholy-Nagy, Painting, Photography, Film, 39. 259 For a history of the developments leading to the creation of mathematical theory of communication, see Colin Cherry, On Human Communication, 2nd ed. (Cambridge: The MIT Press, 1968), 41-52.

efficiency came to the foreground. Engineers needed a quantitative theory which could define and measure the commodity that was communicated over telegraph, telephone, and radio networks. Quantitative measures were also required in order to compare the capacities of various communication systems (for instance, wire versus radio transmission) and to evaluate their performance. The distant precursor to modern communication theory can be found in the centuries-old interest in inventing secret codes for military and diplomatic communication. But with the appearance of modern telecommunications, the problem of designing codes to make the deciphering of messages more difficult for the enemy was joined by a new problem, that of designing codes which could make everyday communication more efficient. Already by 1825, telegraphists designated certain commonly used messages, such as commercial expressions and greetings, by single numbers, thus making coded messages shorter. The famous dot-dash code, introduced by S.F.B. Morse in 1832, associated the most commonly used letters with the shortest dot-dash symbols, thus making coded messages use even fewer symbols.260 The problem of how to devise the most efficient methods of encoding, transmitting, and decoding signals was greatly intensified by the development of television in the 1920s. Television, the "Far Seer," required the simultaneous transmission of a much greater amount of information than in previous communication technologies: "masses of information had to be read off at high speed at the camera end, transmitted, and reassembled in the receiver."261 The great channel capacity required for television spurred new theoretical studies, and by the end of the 1920s, the fundamental ideas of the modern mathematical theory of communication (which also came to be called information theory or statistical communication theory) were formulated. (Significantly, it was the great promise and also the great difficulty in the development of a

260 Cherry, On Human Communication, 36-37. 261 Ibid., 42.

system for visual telecommunication -- television -- which was decisive in the rise of the new paradigm of a quantitative approach to communication.) The essential ideas of information theory were presented in 1927 at the International Congress of Telegraphy and Telephony by R.V.L. Hartley of the Bell Telephone Laboratories in a paper entitled "Transmission of Information." Hartley proposed "to set up a quantitative measure whereby the capacities of various systems to transmit information can be compared" and to "discuss its application to systems of telegraphy, telephony, picture transmission and television over both wire and radio paths."262 Hartley's theory conceptualized the transmission of a message as the successive selection of signs from a set, known to both the sender and the receiver. If the receiver knows beforehand which of the symbols will be transmitted, no new knowledge (no information) is communicated. The amount of information is proportional to the uncertainty about which sign can be received. If a particular message is made up from N signs which can be chosen from a total set of S signs, then there are SN distinct possibilities. Hartley proposed that the "quantity of information" (H) of such a message is H=N log 2S. Hartley also established one of the basic laws of information theory: the total amount of information a system can transmit "is proportional to the product of the frequency range which it transmits by the time available for transmission."263 In other words, the more signals a system transmits simultaneously (frequency range, or bandwidth) and the longer the time available for transmission, the greater its information capacity. With Hartley's paper the two fundamental tools of information theory were now in place: a quantitative measure of information and an equation specifying the information capacity of a communication system. It may seem that the theory defined information in an abstract and counter-intuitive way by associating it with the freedom to make a choice (on the part of the

262 R.V.L. Hartley, "Transmission of Information," Bell System Tech. Journal 7 (1928): 535. 263 Ibid., 554.

sender) and with uncertainty (on the part of the receiver). Yet, the definition of information is quite pragmatic from the point of view of an engineer concerned with making a communication system efficient. What the measure of information (H) defines is the minimum number of binary choices which enables the receiver to reconstruct the message, or conversely, which is required to transmit it.264 In other words, "it is a measure of the minimal effort by which message can be transmitted."265 The development of information theory was further motivated by the work of scientists and engineers on communication and electronic systems during World War II, the most important of which was radar. In its modern form, the theory was formulated by Claude Shannon; in 1949, his papers, together with a non-technical overview by Warren Weaver, were published in what became the bible of the coming information age -- Mathematical Theory of Communication.266 In his contribution, Weaver extended Shannon's model of information transmission, created to describe physical systems of telecommunications, such as telegraphy or radio, to a more general model of any communication situation (fig. 18). To emphasize this generality, his description of the model deliberately juxtaposes examples of telecommunication systems and human communication: The information source selects a desired message out of a set of possible messages...The selected message may consist of written or spoken words, or of pictures, music, etc. The transmitter changes this message into the signal which is actually sent over a communication channel from the transmitter to the receiver. In the case of telephony, the channel
264 Each binary choice constitutes one bit of information. Weaver writes: "if one has available

say 16 alternative messages among which he is equally free to choose, then since 16=24 so that log 216=4, one says that this situation is characterized by 4 bits of information." Claude E. Shannon and Warren Weaver, The Mathematical Theory of Communication (Urbana: The University of Illinois Press, 1949), 100-101. 265 D. Gabor, "A Summary of Communication Theory," in Proceedings of a Symposium on Applications of Communication Theory, London, 1952, ed. Willis Jackson (London: Butterworth Scientific Publications, 1953), 2. 266 Shannon and Weaver, The Mathematical Theory of Communication.

is a wire, the signal a varying electrical current on this wire; the transmitter is the set of devices (telephone transmitter, etc.) which change the sound pressure of the voice into the varying electrical current. In telegraphy, the transmitter codes written words into sequences of interrupted currents of varying lengths (dots, dashes, spaces). In oral speech, the information source is the brain, the transmitter is the voice mechanism producing the varying sound pressure (the signal) which is transmitted through the air (the channel). In radio, the channel is simply space (or the aether, if any one still prefers that antiquated and misleading word), and the signal is the electromagnetic wave which is transmitted. The receiver is a sort of inverse transmitter, changing the transmitted signal back into a message, and handing this message on to the destination. When I talk to you, my brain is the information source, yours the destination; my vocal system is the transmitter, and your ear and the associated eighth nerve is the receiver. In the process of being transmitted, it is unfortunately characteristic that certain things are added to the signal which were not intended by the information source. These unwanted additions may be distortions of sound (in telephony, for example), or static (in radio), or distortions in shape or shading of picture (television), or errors in transmission (telegraphy or facsimile) etc. All of these unwanted changes in the transmitted signal are called noise.267 It is not accidental that the principal examples in this description -- telegraphy, telephony, and oral speech -- all have to do with written or verbal communication. By the middle of the century, both the engineer and the public still associated electronic communication with telephone and radio, and this is why the example of human communication which naturally comes to Weaver is speech. It would be few decades before electronic communication would become synonymous with the communication of images (television, teleconferencing, multimedia, teleoperators, ISDN, cyberspace); before the terms of information theory, such as channel, bit, bandwidth will enter the speech of journalists -- telling us everyday about yet another development in visual telecommunication -- as well as the speech of policymakers, endlessly debating who will control these channels and how wide their bandwidth will be.

The Mathematical Theory of Communication appeared in 1949 and within years the theory was taken up by linguists, psychologists, social scientists, and even art historians. Thus, Roman

267 Ibid., 98-99. Emphasis in Theory or the 3. The Influence of Informationthe original. Ideology of the Code

Jakobson, who as a professor at MIT was close to the source of the paradigm, adopted the model for linguistics and then semiotics.268 Even quicker to embrace the paradigm were experimental psychologists.269 As information theory was transformed from a tool of the communication engineer into a broad intellectual paradigm, three crucial developments took place. First, the theory was extended to understand not just communication within an electronic system, but also communication between humans as well as between humans and machines. Second, the theory was adopted to describe the meaning and effect of communication -- its semantic and pragmatic aspects. Third, the basic assumption of the theory, that communication is a one-way process, was extended to theorize social communication. In this section, I will first consider these three developments in turn. The first development is connected with the figure of Norbert Wiener, a professor of mathematics at MIT who supervised Shannon's research. In 1948 Wiener, published Cybernetics or Control and Communication in the Animal and the Machine which proposed that seemingly different entities -- a machine, an animal, a human, an animal collective, a human collective -can be conceived of as communication systems. Wiener also equated communication and control, thus opening a way to think of social, political, and economic problems in informationcybernetic terms. He justified this equation in this way: When I communicate with another person, I impart a message to him, and when he communicates back with me he returns a related message which contains information primarily accessible to him
268 Roman Jakobson, (1956), "Metalanguage as a Linguistic Problem," Presidential address

delivered at the Annual Meeting of the Linguistic Society of America, 1956, published in The Framework of Language (Ann Arbour: Michigan Studies in the Humanities, 1980): 81-92; Roman Jakobson, (1960), "Closing Statement: Linguistics and Poetics," in Semiotics. An Introductory Anthology, ed. Robert Innis (Bloomington: Indiana University Press, 1985). 269 G.A. Miller, Language and Communication (New York: McGraw-Hill Book Co., Inc., 1951); D.E. Broadbent, Perception and Communication (Oxford: Pergamon Press, 1958); H. Quastler, ed., Information Theory in Psychology: Problems and Methods (Glencoe, IL.: Free Press, 1955); F. Attneave, Applications of Information Theory to Psychology (New York: Holt, 1959).

and not to me. When I control the actions of another person, I communicate a message to him, and although this message is in the imperative mood, the technique of communication does not differ from that of a message of fact. Furthermore, if my control is to be effective I must take cognizance of any messages from him which may indicate that the order is understood and has been obeyed.270 In this explication Wiener talks about human communication, but his realization that the "problems of control engineering and of communication engineering were inseparable" emerged during the war when he worked on anti-aircraft artillery systems. Wiener realized that the key to successful control lies in feedback -- the modification of behavior of a system (be it human or a machine) by taking into account the results of the actual (rather than predicted) performance. The results of the performance are communicated back to the control mechanism, making it adjust the system's behavior. A prototypical example of a feedback mechanism is a self-guided shell: "When we desire a motion to follow a given pattern, the difference between this pattern and the actually performed motion is used as a new input to cause the part regulated to move in such a way as to bring its motion closer to that given by the pattern."271 The authors of information theory always insisted that the theory was only concerned with the conditions for the correct and efficient transmission of signals constituting the message, and not with its meaning, interpretation, or effect. Hartley points out that "in estimating the capacity of the physical system to transmit we should ignore the question of interpretation."272 Shannon also begins by insisting that "semantic aspects of communication are irrelevant to the engineering problem."273 Finally, Weaver starts his essay included in The Mathematical Theory of Communication by distinguishing between the three levels of a communication problem: technical (how accurately the signs can be transmitted), semantic (how accurately signs convey

270 Norbert Wiener, (1950), The Human Use of Human Beings. Cybernetics and Society (New York: Avon Books, 1967), 24-25. 271 Norbert Wiener, Cybernetics. Of Control and Communication in the Animal and the Machine (New York: John Wiley & Sons, Inc., 1948), 13. 272 Hartley, "Transmission of Information," 538. 273 Shannon and Weaver, The Mathematical Theory of Communication, 3.

the desired meaning), and pragmatic (how effective the message is in affecting the behavior of the receiver). Weaver refers to these levels as A, B, and C.274 He next emphasizes that the mathematical theory of communication as developed by Shannon applies only to the first level, yet also immediately suggests that it is deeply relevant for the second and third. Impressed by the fact that the technical aspect of a communication process is finally controlled by a precise mathematical theory, Weaver clearly hopes that the theory can somehow be extended to handle the elusive questions of semantics and pragmatics with the same quantitative precision. Throughout the essay he promises a detailed discussion of how this can be done. Weaver never keeps his promise; instead, in a number of places he suddenly announces that perhaps no modifications will be necessary. The announcements appear abruptly at the end of sentences, explosions in what is otherwise the dry rhetoric of a scientific discourse. "It is the purpose of this concluding section to review the situation, and to see to what extent and in what terms the original section was justified in indicating that the progress made at Level A is capable of contributing to levels B and C, was justified in indicating that the interrelation of the three levels is so considerable that one's final conclusion may be that the separation into the three levels is really artificial and undesirable." "It is almost certainly true that a consideration of communication on levels B and C will require additions to the schematic diagram on page 97 [fig. 18], but it seems equally likely that what is required are minor additions, and no real revision."275 The temptation to extend information theory to handle the meaning and effect of communication proved too great to resist. This was understandable given the historical context of the Cold War during which the theory matured and attracted public attention. This context is not hard to see in Weaver's descriptions of different levels of communication: "The semantic

274 Ibid., 96. 275 Ibid., 114, 115. Emphasis mine -- L.M.

problem has wide ramifications if one thinks of communication in general. Consider, for example, the meaning to a Russian of a U.S. newsreel picture." "The problem of effectiveness involves...all the psychological and emotional aspects of propaganda theory."276 Even more revealing is the opening paragraph of the essay, where within a few lines we are transported from artistic communication, painting, theater, and ballet, to the theater of the new cybernetic warfare, which emerged during World War II and now protected the U.S. against a Soviet attack: The word communication will be used in a very broad sense to include all of the procedures by which one mind may affect another. This, of course, involves not only written and oral speech, but also music, the pictorial arts, the theatre, the ballet, and in fact all human behavior. In some connections it may be desirable to use a still broader definition of communication, namely, one which would include the procedures by means of which one mechanism (say automatic equipment to track an airplane and to compute its probable future positions) affects another mechanism (say a guided missile chasing this airplane).277 In 1952 Y. Bar-Hillel, who was working in the field of automatic language translation (which, as I mentioned in the previous chapter, also developed during Cold War with the promise to automate the surveillance and analysis of Soviet military communication and media), extended information theory to define and measure "semantic information" of the propositions in a language system. In the same years, the Shannon-Weaver model (fig. 18) was adopted in the U.S. by the growing field of mass communication studies as the theoretical foundation of the field. Weaver's hope that the model, originally designed to deal solely with the technical aspects of telecommunication, could be applied to describe semantic and pragmatic levels as well, was fully realized: the model was adopted to describe the process of mass communication: sender (film studio, TV station, publisher) -- message (a film, a television program, a newspaper story) - receiver (film viewer, television viewer, newspaper reader).278

276 Ibid., 97. 277 Ibid., 95. 278 Wilbur Schramm, "How Communication Works," in The Process and Effects of Mass

Communication (Urbana: University of Illinois Press: 1954): 3-26.

In this transfer, the basic postulate of the information theory became an ideology. Shannon begins his exposition of information theory by stating that "the fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point."279 American scholars of mass communication applied the same postulate to the semantic level. Even a textbook on mass communication published in the 1970s, still states: "if the meaning of the destination is isomorphic with the meaning of the source which originated the act, then communication can be said to have taken place."280 With such a definition, any discrepancy between the codes of a sender and a receiver becomes undesirable "noise." As Stuart Hall pointed out, the ideology lies in assuming that encoding and decoding codes are or should be the same: "What are called 'distortions' or 'misunderstandings' arise precisely from the lack of equivalence between the two sides in the communication exchange."281 Thus, while American social scientists contrasted American democracy with Soviet totalitarianism, they simultaneously adopted the theoretical model according to which mass communication was synonymous with following the prescribed meaning. In short, communication was defined as control.

Despite the fact that modern mass communication crucially depends on the visual channel, American scholars of mass communication never seriously confronted its visual aspects. Others, however, have tried to use information theory to understand visual communication. I will next

279 Shannon and Weaver, The Mathematical Theory of Communication, 3. Shannon includes "approximately" because one of the main concerns of a communication engineer is to increase the efficiency of a communication system by making it transmit only the information which is absolutely essential for the correct identification of a message by the receiver. For instance, the telephone does not carry the whole range of frequencies of the human voice but only those sufficient for understanding what is said. 280 Melvin L. De Fleur and Sandra Ball-Rokeach, Theories of Mass Communication (New York: Longman, 1975), 127. 281 Stuart Hall, "Encoding/Decoding," in Culture, Media, Language, ed. Stuart Hall et al. (Hatchinson, 1980), 131.

discuss these attempts which took place in the late 1950s and in the 1960s in semiotics, experimental aesthetics, and art history. In the early 1960s Roland Barthes published The Photographic Message and Rhetoric of the Image where he applied a semiotic apparatus to the analysis of mass media images, in his case, publicity photographs.282 However, it may be surprising to discover that The Photographic Message opens by formulating the situation of mass communication in terms of information theory, exactly in the same way as it was done in communication studies: The press photograph is a message. Considered overall this message is formed by a source of emission, a channel of transmission and a point of reception. The source of emission is the staff of the newspaper, the group of technicians certain of whom take the photo, some of whom choose, compose and treat it, while others, finally, give it a title, a caption and a commentary. The point of reception is the public which reads the paper. As for the channel of transmission, this is the newspaper itself... Barthes, however, was not going to propose, as communication researchers did, that the system of mass communication also contains a feedback mechanism -- the right of readers to write letters to newspapers with complaints or suggestions -- thus assuring that mass communication truly serves the purposes of Democracy and Free Speech, rather than unquestioned ideology. In fact, Barthes' whole purpose in his articles on the semiotics of mass media images was to reveal the mechanisms by which images communicate or subvert an ideological meaning. Communicate or subvert: Barthes' position oscillates between these two mutually exclusive positions. On the one hand, Barthes proposes in The Rhetoric of the Image that the specificity of a photograph's codes is what allows it to naturalize the ideological message in a particularly transparent manner; on the other hand, in the same article he also claims that an image is essentially polysemous and therefore a caption is required to anchor a single meaning. In a later article The Third Meaning Barthes makes an even stronger claim for the subversive
282 Roland Barthes, (1961), "The Photographic Message," in Image, Music, Text, ed. Stephen Heath (New York: Hill and Wang, 1977); "Rhetoric of the Image," (1964), in Image, Music, Text.

nature of the visual code in relation to both ideology and interpretation by suggesting that a film still contains an excess, an "obtuse meaning" not reducible to its "informational" or "symbolic" meanings.283 The code. More than any other, it was this particular concept of information theory that entered semiotics in the 1960s. And for semioticians concerned with visual signs, it became the main preoccupation of their research: is there a unique visual code, an autonomous "language of pictures," and if so, what is its specificity? Now, after we have followed the development of information theory, we can see the historical specificity of approaching vision as a code by Barthes, J.M. Floch, Felix ThŸrlemann, the members of the Groupe Mu, Fernande Saint-Martin, and other semioticians.284 The notion of the code, as information theory itself, has emerged in response to the practical problem faced by engineers: how to make telecommunication systems more efficient and how to measure this efficiency. Encoding is one of the principal ways to achieve this efficiency. Morse code condensed transmitted messages by associating the shortest combination of dots and dashes with the most frequently used letters. Later, communication engineers were able to achieve even greater economy by encoding only as much information as necessary for the receiver to decode the message with sufficient accuracy.285 When carried over to visual semiotics, the notion of the code retained its original engineering meaning, its association with the issue of efficiency. Indeed, to ask what kind of information can be obtained through seeing and/or represented as images; how visual codes differ from other kinds of codes; whether visual representations have any unique qualities in terms of their effect on the viewer -in short, to propose the uniqueness of the visual code or to deny this uniqueness is already to
283 Roland Barthes, (1970), "The Third Meaning," in Heath, Image, Music, Text. 284 For a critical discussion of visual semiotics, see Gšran Sonesson, Pictorial Concepts.

Inquiries into the Semiotic Heritage and its Relevance for the Analysis of the Visual World (Lund, Sweden: Lund University Press, 1989). 285 A principal contribution of Shannon was his famous Sampling Theorem which specified the absolute minimum of the signal which needs to be retained before information is irreversibly lost.

conceive of vision as productive, as potentially a special way of transmitting information, to conceive of it as a communication engineer would.286 The semioticians did not go as far as to apply the mathematical apparatus of information theory to the analysis of visual signs. However, the field of experimental aesthetics, which, from its origins in the nineteenth century always tried to reduce the effect of an image to a mathematical formula, welcomed the quantitative aspect of the theory. But how to apply a theory developed for the purpose of engineering communication to a very different question of aesthetic pleasure? Experimental psychologists interested in aesthetics assumed that a measure of information in an image could represent its aesthetic effect. In the 1960s the German aesthetician Max Bense developed a theory of "information aesthetics" and tried to measure the "information" contained in such classical images as Rembrandt's etchings. It was not at all obvious, however, how to measure the "information" in an etching or a painting, so other researchers went on to construct simple abstract patterns where "information" was easier to quantify. They reasoned that since the amount of information in a message is directly proportional to its unpredictability and inversely proportional to its redundancy, a pattern made from small squares, where the tone of each square is determined randomly, would be the most unpredictable and therefore would contain the most information. On the contrary, in a pattern like Mondrian's or Malevich's painting there is much more redundancy (more predictability) and consequently, the amount of information is low. In a number of experiments, subjects rated their "aesthetic pleasure" when looking at different patterns (fig. 19); psychologists then tried to
286 This carrying over of assumptions of information theory into semiotics reaches its extreme in Jurij Lotman's Structure of the Artistic Text (1970). Lotman was strongly influenced by cybernetics which in the 1960s was more popular in the Soviet Union than it ever was in the West and which became as important for Soviet semioticians as Saussure or Peirce. (In part, it was used to justify semiotics in the eyes of the authorities by presenting it as a part of cybernetics, concerned with the study of different artificial languages, from computer languages to the languages of art.) Lotman claimed that the artistic text is a message coded in a particular way and that art is the most economical way to transmit information because of the unique features of the artistic code. Jurij Lotman, (1970), The Structure of the Artistic Text (Ann Arbour: Department of Slavic Languages and Literatures, The University of Michigan, 1977).

determine the level of visual organization that was biologically most pleasurable.287 Psychologists, turned aestheticians. Aestheticians, turned communication engineers. Finally, one of the most influential texts of post World War II art history is also heavily indebted to information theory.288 In Art and Illusion (1960) Ernst Gombrich provides a revealing analysis of Greek vases and mosaics as precursors to modern digital codes; he compares the reading of pictures to the decipherment of codes; and he uses various concepts of information theory throughout the book.289 However, Gombrich does more than simply pay tribute to a fashionable theory. During the war, Gombrich was employed for six years by the British Broadcasting Corporation to listen for radio transmissions.290 He was confronted daily with the central problem of modern information theory -- the identification of signal in the presence of noise. As he himself admits, it was this experience which led to his deep interest in information theory as a paradigm for understanding perception. Rather than being accidental, the paradigm of information theory plays a central role in Art and Illusion. One of Gombrich's key concepts is "the beholder's share": the importance of inference in interpreting the image. According to Gombrich, a representational image is essentially incomplete.291 As a representation of three-dimensional space, a two-dimensional

287 See, for instance, D.E. Berlyne, "Novelty, Complexity, and Hedonic Value," Perception and

Psychophysics 8 (1970): 279-86; D. Dorfman and H. McKenna, "Pattern Preference as a Function of Pattern Uncertainty," Canadian Journal of Psychology 20 (1966): 143-53; P.C. Vitz, "Preference for Different Amounts of Visual Complexity," Behavioral Science 2 (1966): 105-14. It is interesting to compare this work in experimental aesthetics with Lotman's view of art as communication (see footnote 278). Lotman proposed that a unique feature of art is that it carries more information than any other kind of communication, while experimental aestheticians proceeded under the assumption that there is some optimal level between the complete lack of information and the complete saturation by it which characterizes the most pleasurable works of art. 288 I am grateful to Michael Ann Holly and Norman Bryson who brought to my attention the relevance of Art and Illusion to the problematic of this chapter. 289 E.H. Gombrich, Art and Illusion (Princeton: Princeton University Press, 1960), 39-40, 88, 205. 290 Ibid., 204. 291 Ibid., 208, 211.

image is always ambiguous, potentially corresponding to numerous spatial configurations. It is equally ambiguous as a representation of other aspects of visual reality, substituting a few brush strokes for a field of grass, a few spots of color for a human form. The job of a beholder is to supplement the partial information in the image with her or his own projection. In other words, Gombrich understands visual communication in terms of information theory: as the decoding of a signal in the presence of noise. According to information theory, this decoding is always probabilistic: there can never be the absolute certainty than an original signal is correctly reconstructed. Similarly, Gombrich describes interpretation as the process of forming and rejecting hypotheses, a process which is never finished. For an engineer, this uncertainty of communication is a problem to be solved. Gombrich, however, interprets it as the source of our aesthetic pleasure. We derive pleasure from art precisely because in trying to "identify a signal" we have to use our imagination. There is an unexpected agreement here between Gombrich and experimental aestheticians of the 1960s, such as Bense. Both start from the assumption that the aesthetic pleasure is proportional to the amount of information in a message, i.e. its unpredictability. As Gombrich explains it, "the greater the probability of a symbol's occurrence in any given situation, the smaller will be its information content."292 Because of their ambiguity, representational images in Western art carry significant information -- and, correspondingly, aesthetic pleasure.

Mass communications, visual semiotics, and aesthetics are just some of the fields that adopted concepts of information theory in order to understand visual communication. The avant-garde of the 1920s, which dreamed of engineering visual communication, would welcome all these applications of information theory. The theory, which was developed by a communication engineer in response to the growth of television, was now used to rationalize the idea that mass
292 Ibid., 205.

communication is synonymous with control, and that to understand is to decode an intended meaning (mass communication studies). The theory, designed to measure only the efficiency of telecommunications networks, was now adopted to describe all other levels of communication as well. Visual semiotics and experimental aesthetics extended information theory to the levels of meaning and effect, the former addressing the specificity of visual signification, the latter attempting to quantify the effect of an image by measuring its "information" content. However, it was the field of experimental psychology that was most transformed by information theory. After World War II, two crucial developments took place. The first is that psychologists, by adopting the approach of information theory, essentially became engineers who approach human vision as a communication system, studying its channel capacity, bandwidth, and other engineering characteristics. The second is that work, more and more, became a matter of the mental processing of visually presented information. And this made experimental psychology, for the first time since the discipline emerged, practically important; in fact, as important as communication engineering. In the next three sections we will see how these developments came about and what their impact was on modern understanding of vision.

Why was it that even though the essential ideas of information theory were articulated in the 1920s, it was only in the late 1940s, after the publication of The Mathematical Theory of Communication, that the theory was suddenly widely accepted in so many fields? Is there a relationship between this acceptance and the social and economic changes which were taking place in modern society after World War II? The best way to answer these questions is to compare the role of information theory in the second half of the twentieth century to the role

4. From "Human Motor" to "Human Information Processing"

previously occupied by thermodynamics. First, a historical parallel. Hermann von Helmholtz formulated the universal law of the conservation of energy in 1847. Exactly a century later, the theory of information in its modern form was developed by Shannon and Wiener. Helmholtz was instrumental in turning the theory of thermodynamics into a more general paradigm which treated the work of nature, the work of machines, and human labor on the same terms. Similarly, Wiener extrapolated the concepts of information theory and the techniques of control engineering into the science of cybernetics which described the organization and behavior of human, machine, and society using a single set of terms as well -- information, feedback, and entropy. The concept of entropy, in fact, directly connects thermodynamics and information theory. In 1929 the physicist Leo Szilard identified entropy with information, and the measure of information with the negative of the measure of entropy.293 Entropy is the central notion of thermodynamics, referring to the tendency of a closed system to change from an organized, differentiated, and less probable state to a chaotic, undifferentiated, and more probable state. Since entropy is the measure of randomness, i.e. the unpredictability of a thermodynamic system, and since information similarly measures the unpredictability of a message, the equations for entropy and information are the same. As Weaver explains, "for a communication source one can say, just as he would also say it of a thermodynamic ensemble, 'This situation is highly organized, it is not characterized by a large degree of randomness or of choice -- that is to say, the information (or the entropy) is low.'"294 But beyond these connections, there is a more crucial parallel. Just as thermodynamics provided the model for understanding human work in the latter nineteenth and first half of the

293 Howard Resnikov, The Illusion of Reality (New York: Springer-Verlag New York Inc.,

1989), 5.
294Shannon and Weaver, The Mathematical Theory of Communication, 103.

twentieth century, information theory has been crucial for a new post-industrial model of work, which gradually emerged after 1940.

In The Human Motor: Energy, Fatigue and the Origins of Modernity Anson Rabinbach demonstrated how the scientific ideas of thermodynamics, formulated in the middle of the nineteenth century, became central for the conception of work in modernity. Helmholtz, who discovered the law of the conservation of energy, promoted this law as the universal principle which equally applies to nature, machines, and humans. Helmholtz "portrayed the movements of the planets, the forces of nature, the productive force of machines, and of course, human labor power as examples of the principle of conservation of energy."295 All work was understood as the expenditure of energy, with a crucial consequence of redefining human labor as labor power, the expenditure of the energy of a body. Thus a worker was redefined as a "human motor." This, in turn, lead to the emergence, towards the end of the century, of the movement which Rabinbach calls the European science of work, "the search for the precise laws of muscles, nerves, and the efficient expenditure of energy centered on the physiology of labor."296 In manual labor, the energy stored in the body where it was accumulated through the intake of food, sleep, and rest is transferred into muscular force -- hammerer striking a blow, filer filing a machine part, and so on. Therefore, psychologists, physiologists and industrial experts searched for methods to maximize both the accumulation of a worker's energy (through proper nutrition, shorter working hours, appropriate breaks) and its expenditure in labor. Just as an engineer designing an engine was concerned with the most efficient transfer of fuel energy into movement, European work experts aimed to maximize worker efficiency and to eliminate possible waste. Central to the quest for the efficiency of the human motor was the struggle
295 Anson Rabinbach, The Human Motor: Energy, Fatigue, and the Origins of Modernity (Basic

Books, Inc., 1990), 3. 296 Ibid., 10.

against fatigue, understood as the equivalent of entropy. "As entropy revealed the loss of energy involved in any transfer of force, so fatigue revealed the loss of energy in the conservation of Kraft to socially useful production. As energy was the transcendental, 'objective' force in nature, fatigue became the objective nemesis of a society founded on labor power."297 The European science of work may appear to be very similar to the American scientific management movement pioneered by Frederick Winslow Taylor, a former engineer turned management consultant. As a part of his program, Taylor aimed to minimize and standardize the time required by a worker to perform each operation. He employed the method of time studies whereby the best workers were timed and the results became the norm to be followed by the rest.298 Later, Frank and Lilian Gilberts (he -- an engineer, she -- a psychologist) popularized another method of motion study.299 They argued that maximizing worker productivity is best achieved by the elimination of unnecessary movements and making the necessary more efficient. Although both time and motion studies and the European science of work were concerned with the efficiency of manual work, there was a fundamental difference between the two approaches.300 Taylorism aimed for maximum productivity, and had no concern for the exhaustion and deterioration of the human motor. In contrast, European scientists aimed for optimum productivity, and therefore were concerned not only with the rationalization of the workplace, but also with the workers' health, nutrition, safety, and the optimal length of a workday. In short, Taylorism had no reservations about replacing one exhausted human motor with another -- the philosophy which in the U.S. seems to go hand in hand with the emerging ethics of the consumer society and with immigration policies which assured the constant supply

297 Ibid., 68. 298 Frederick Winslow Taylor, The Principles of Scientific Managment (New York, 1967). 299 William R. Spriegel and Clark E. Myers, eds., The Writings of the Gilbreths (Homewood,

IL., 1953). I am grateful to Lisa Cartwright for introducing me to the work of the Gilbreths and its relevance for the history of modern vision. 300 Rabinbach, The Human Motor, 117, 277.

of a cheap labor force. Europeans, on the other hand, were committed to caring for and servicing the human motor. The two paradigms converged after World War I, when European industrialists partly adopted the more brutal, but ultimately more effective Taylorist methods, while U.S. managment experts became more sensitive to workers' physiology and psychology. Taylorism reduced the worker's body to a mechanical machine and had no concern for her or his mind. Indeed, as Marta Braun points out, Taylorism aimed to systematically rob the worker of any degree of independence or even understanding of the overall work process by "separating responsibility for the execution of work from its planning or conception."301 This disdain for the mind was shared by behaviorism, which matured at the same time as the European science of work and Taylorism, and which equally well characterizes the imaginary of hard-edged social engineering of the first half of the twentieth century. In 1913, J.B. Watson, the founder of behaviorism, explicitly defined it as the science of social control: "Psychology as the behaviorist views it is a purely objective experimental branch of natural science. Its theoretical goal is the prediction and control of behavior."302 Behaviorism approached the human subject as an input-output system of stimulus and response to be controlled through conditioning. Concerned with controlling the body, it almost completely suppressed any studies of perceptual or mental processes between 1920 and 1950 in the U.S. It was a psychology well suited for controlling the subject already reduced to the brainless human motor.

301 Marta Braun, Picturing Time: the Work of Etienne-Jules Marey (1830-1904) (Chicago: The

University of Chicago Press, 1992), 337.
302 Qtd. in Eliot Hearst, "One Hundred Years: Themes and Perspectives," in The First Century

of Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1979), 27. 5. Communication Engineer Analyzes Human Vision

After World War II, the references to time and motion studies disappear. And by 1960, the dominance of behaviorism is over as well. Industrial society became post-industrial society. In this shift, the concepts of manual labor, production of goods, and fatigue were replaced by new concepts of cognitive labor, information processing, and noise. Taylorism, Gilberts' motion studies, and behaviorism gave way to engineering psychology, "human information processing," and cognitive science. In short, with the transformation of industrial society into post-industrial society, the disciplines of the efficiency of the body were replaced by the disciplines concerned with the efficiency of the new instrument of labor -- the mind. One obvious and perhaps the earliest sign of this transition was the speed with which information theory was disseminated into so many fields by the end of the 1940s and early 1950s. But, along with being a symptom of post-industrial society which was yet to fully manifest itself, information theory also provided actual techniques for defining and studying the human subject as an "information processing unit," and its visual apparatus as a crucial part of "human information processing." Information theory therefore is central for understanding the new ideas and techniques of vision that emerged after the World War II. The significance of the adaptation of information theory as a new model of the human observer, which took place in the 1950s, can be best seen through the following analogy: Industrial society: manual labor, energy, fatigue. Post-industrial society: mental labor, information processing, noise.

1. Manual versus mental labor. The point is not whether corporeal labor was indeed universally displaced by mental labor: this is different from country to country, from industry to industry. What is important is that the obsession with the rationalization of corporeal work (Taylorism, European science of work, psychotechnics) disappeared, displaced by new obsession with the rationalization of the mind (cognitive psychology, artificial intelligence, cognitive

engineering). Regardless of the percentage of the work force that still may be engaged in manual labor, society is no longer concerned with spending more intellectual resources to perfect workers' movements. What comes under scrutiny since the 1950s, when cognitive psychology begins to displace then dominant behaviorism, are mental functions: perception, attention, text comprehension, memory, and problem solving. This replacement of manual work by cognitive work is directly related to automation. Already in 1961, in an influential study of automation in French industry, Pierre Naville and his fellow sociologists had described the transition from the "work of the laborer to the work of communication," work which became primarily "cognitive or semiotic."303 In his summary of this study Rabinbach writes, "The appearance of the cerebral worker whose material and product is 'information' is emblematic of the vast distance traversed between the worker who surveys complex technologies of communication and the 'man-beef' of Taylor."304 It is important to note that automation does not lead to the replacement of human by machine. Rather, the worker's role becomes one of monitoring and regulation: watching displays, analyzing incoming information, making decisions, and operating controls. And it is the corresponding human functions of perception, attention, memory, and problem solving which become the subject of research by new cognitive sciences. What Taylor's scientific management was for the age of industrialization, cognitive sciences are for the age of automation. In the 1940s, Herbert Simon worked on theories of management, the field of research originated by Taylor. Having recognized the increasing importance of mental skills in the corporate workplace, Simon became one of the pioneers of cognitive science with his work on automatic reasoning by computer. In 1964 he wrote that "the

303 Qtd. in Rabinbach, The Human Motor, 298. 304 Ibid., 298.

bulk of productive wealth consists of programs...stored in human minds."305 Another pioneer of cognitive science was Jerome Bruner. Reflecting back on his work in the 1950s, he noted in 1983: "It seems plain to me now that the 'cognitive revolution'...was a response to the technological demands of the 'post-industrial revolution.' You cannot properly conceive of managing a complex world of information without a workable concept of mind."306

2. Just as thermodynamics in the nineteenth century provided a model for the laboring body, information theory provided a new model for the mental processes, becoming particularly influential in the research on human senses, including vision. In 1958 the English psychologist D.E. Broadbent published Perception and Communication, which was the first attempt to understand human sensory-motor performance as information processing, or, as he put it, to describe it "in terms originally developed for telephone engineering."307 This statement may at first remind us of the comparison between nerves and telegraphy, frequently evoked by nineteenth century physiologists and psychologists, including Helmholtz.308 Yet, the approach of Broadbent and other psychologists who followed him was fundamentally new. It was less concerned with human physiology than with the limits of human performance understood as a communication system. In other words, just as a modern telephone engineer is concerned with the capacity of the telephone system, a psychologist is concerned with the amount of visual or auditory information that can be adequately processed by a human system; sensed, recognized, stored in memory, recalled, acted upon. What for Helmholtz was a metaphor became a systematic program of research a century later.
305 Qtd. in Douglas Noble, "Mental Materiel: The Militarization of Learning and Intelligence in U.S. Education," in Cyborg Worlds: the Military Information Society, ed. Les Levidov and Kevin Robins (London: Free Association Books, 1989), 34. 306 Qtd. in Ibid., 34-35. 307 Broadbent, Perception and Communication, 36. 308 Jonathan Crary, Techniques of the Observer: on Vision and Modernity in the Nineteenth Century (Cambridge: The MIT Press, 1990), 93-94.

For a communication engineer, information theory provides both a standard set of terms to specify quantitatively the performance of a communication system and a set of principles which specify the limits of this performance. For instance, the capacity of a system depends on the code used. Morse code makes transmitted messages shorter by associating the most frequently used letters with the shortest dot--dash symbols. The time saved in transmitting, however, is offset by the additional time now required to encode and decode the message. Another example is the principle, already established by Hartley, that the maximum amount of information a system can transmit is the product of its bandwidth and the time available for transmission. In the 1920s, it was possible to transmit images over telegraph wires -- a small bandwidth system -- by sacrificing time. In addition to the adoption of the terms of information theory, such as bandwidth, information capacity, and filter, psychologists also applied its principles to understanding the limits of human performance. One of Broadbent's contributions was the idea that the nervous system cannot process all of the sensory stimulation which may be available and that there is "a filter at the entrance to the nervous system which will pass some classes of stimuli but not others"309 (fig. 20). In other words, because of its limited bandwidth, a nervous system can only process a certain amount of information in a given unit of time. How much can be processed depends on the nature of this information since different human senses such as vision and hearing, now understood as separate communication channels, have different information capacities. Once filtered through the senses, the information is further processed through a series of stages. Each stage is understood as a specific communication system with its own capacities determined by its codes and bandwidth. Figure 21, taken from 1986 Handbook of Perception and

309 Broadbent, Perception and Communication, 42.

Performance, summarizes the current knowledge of the temporal capacities of different stages of sensory and mental processing. So just as the productivity of a laboring body was limited by the available energy, it is now assumed that the productivity of the mind is limited by its information-processing capacity. In fact, the notion of a limited reservoir of the energy in the body has found a direct correlatation in the notion of there being a limited amount of human information-processing resources which can be used in different ways to deal with the incoming information. For instance, the subject may concentrate all his/her resources on processing visual information, thus sacrificing his or her attending to auditory stimuli.

3. Fatigue versus noise. Just as fatigue is the opposite of energy, the limit to the body's productivity, noise, always present in a communication system, whether natural or man-made, is the limit to perfect communication and, consequently, the limit to faultless human information processing. Noise is the source of error. And error is the worst enemy of the human-machine system in post-industrial cybernetic society. The late nineteenth century science of work came about as a struggle against fatigue, this "subversive element in the human motor, the body's fifth column...fatigue was the physiological limit of even the most perfectly executed work, the horizon of the metaphor of the human motor."310 As physiologists and psychologists were preoccupied with fighting the body's fatigue, cognitive psychologists became preoccupied with preventing human cognitive and perceptual errors. A textbook on human factors begins a chapter on error by optimistically asking: "Are errors a basic part of human nature that cannot be changed, or is there some way to eliminate or reduce error and the damage errors cause?" The experts on human factors classified

310 Rabinbach, The Human Motor, 118.

various types of human errors, came up with mathematical formulas to calculate human reliability, and compiled tables of Human Error Probability (HEP) for different tasks.311 What are the sources of human error? In contrast to a manual worker of the industrial age, an operator in a human-machine system is primarily engaged in the observation of displays which present information in real time about the changing status of a system or an environment, real or virtual: a radar screen tracking a surrounding space (see chapter 3); a computer screen updating the prices of stocks; a video screen of a computer game presenting an imaginary battlefield; a control panel of an automobile showing its speed, etc.312 From time to time, some information causes an operator to make a decision and to intervene in the system's operation: tell the computer to track an enemy bomber noticed on the radar screen; buy or sell a stock; press a joystick; change gears. It is not essential that in some situations these interventions may be required every second (a pilot engaged with an enemy, a computer game player, a financial analysis monitoring stock prices), while in others they are needed very rarely (a technician monitoring an automated plant, power station, a nuclear reactor; a radar operator monitoring a radar screen, waiting for potential enemy planes). This is why a human-machine system is defined as "an equipment system, in which at least one of the components is a human being who interacts with or intervenes in the operation of the machine components of the system from time to time."313 In a famous passage Walter Benjamin characterized modern experience as a constant periodic rhythm of perceptual shocks; the experience shared by an assembly line worker, by a

311 Barry H. Kantowitz and Robert D. Sorkin, Human Factors: Understanding People-System

Relationships (John Wiley & Sons: 1983), 30-57. 312 A 1965 textbook on human-machine systems calls an automobile "a first rate example of a true man-machine system...a highly complex system in which the operator plays a commanding role or actively intervenes in the system from time to time." Alphonse Chapanis, Man-Machine Engineering (Bemont, CA: Wadsworth Publishing Company, Inc., 1965), 16. 313 Chapanis, Man-Machine Engineering, 16. Emphasis mine -- L.M.

pedestrian, and by a film viewer. 314 This experience, characteristic of modernization, finds a direct continuation in one type of cybernetic workplace: the constant overwhelming amount of information; the constant cascade of cognitive shocks which require immediate interventions (a pilot engaged with an enemy, a player of a computer game). Now, however, these shocks arrive exclusively through the visual channel (dials, computer screen, head-mounted display). Thus of the roles mentioned by Benjamin, it is the film viewer rather than the assembly line worker who directly anticipates the experience of an operator in this type of human-machine situation. However, there is also a second type of work experience, new to post-industrial society: work as waiting for something to happen. A radar operator waiting for a tiny dot to appear on the screen; a technician monitoring an automated plant, power station, or nuclear reactor, knowing that a software bug will eventually manifest itself, making a pointer on one of numerous dials shoot into the red... If a manual worker is eventually overcome by fatigue, an operator monitoring displays in situations where nothing may happen for hours eventually loses vigilance. In terms of the information processing approach, when a signal occurs infrequently and randomly over time, an operator is more likely to miss the signal.315 This is one source of human error. Another source

314 "Whereas Poe's passers-by cast glances in all directions which still appeared to be aimless,

today's pedestrians are obliged to do so in order to keep abreast of traffic signals. Thus technology has subjected the human sensorium to a complex kind of training. There came a day when a new and urgent need for stimuli was met by the film. In film, perception in the form of shocks was established as a formal principle. That which determines the rhythm of production on a conveyer belt is the basis of the rhythm of reception of film." Walter Benjamin, "On Some Motives in Baudelaire," in Illuminations, ed. Hannah Arendt (New York: Schochen Books, 1969), 175. 315 In the extreme case of sensory deprivation the subject becomes totally disoriented, experiences illusions and even faints. It is interesting that in the 1950s, when automation was making this new work-as-waiting-for-signal commonplace, experimental psychologists began to conduct experiments in artificial sensory deprivation. In 1953 L.A. Riggs and his colleagues discovered that if the image is stabilized on the retina, thus eliminating miniature eye movements normally accompanying vision, the image vanishes. In 1954-57, W. Heron constructed an experimental cubicle to study the effects of total perceptual isolation. Subjects, deprived of all visual, auditory, and tactile stimuli, experienced extreme boredom, restlessness and, eventually, visual hallucinations. L.A. Riggs, et al., "The Disappearance of Steadily Fixated Visual Tests

is visual illusions, limits to the reliability of human vision, which preoccupied experimental psychologists from the early days of the discipline. Still another source is the inherent limitations of a human as a decision maker. When cognitive psychologists began to study human reasoning, they made a grave discovery: human judgments do not follow statistical probabilities and ignore information about the prior probability of an event.316 So even if the operator detects a signal in time, s/he may still make a wrong decision.

Along with providing a model that allowed psychologists to conceive of human vision as a communication system where performance may be described in quantitative terms, information theory also provided a general concept to specify an inherent limit to this performance -- noise. The incorporation of the concept of noise is what distinguishes modern information theory, as developed by Shannon, Wiener, Kolmogoroff, and others, from the original formulations by Hartley. In any real telecommunication system noise is always present due to the random motion of electrons in all electrical conductors. Therefore, in practice, the signal at the receiving end is always to some degree different from the signal originally sent. One of Shannon's principal contributions was an equation for determining the maximum capacity of a communication channel to transfer information in the presence of noise. As Colin Cherry notes, "This channel capacity concept is somewhat analogous to the notion of conservation of energy; it is a definite limit which no practical system can exceed, and its principal value is that it provides a standard against which efficiencies may be assessed."317 Just as entropy limits the performance of any thermodynamic system (such as a motor), leading to some of the energy being lost unproductively, noise sets the limit on the efficiency of Objects," J. Optical Soc. America 43 (1953): 495; W. Heron, "The Pathology of Boredom," Scientific American 196 (1957): 52-56. 316 Gillian Cohen, The Psychology of Cognition, 2nd ed. (London and New York: Academic Press, 1983), 190. 317 Cherry, On Human Communication, 52.

communication. And just as attempts to rationalize manual performance stumbled across the problem of fatigue, the "physiological limit" to the efficiency of work, psychologists recognized noise as the inherent limit in the performance of a human information processing system. If the source of noise in a telecommunication apparatus is the random motion of electrons, the source of noise in the nervous system is the random firing of neurons. Always present, these random firings are assumed to generate a small but constantly present level of neural activity which interferes with human information processing. The observer may miss a signal (for instance, a flash of light in a psychophysical experiment or a dot on a radar screen in a real situation) or, on the contrary, detect a signal where none was present; or s/he may make a mistake in estimating the signal's strength or its position, and so on. In fact, the noise in man-made components and biological noise become combined. Due to the noise in its electronic parts, radar may display a dot when nothing is present or the location of the dot may be inaccurate. Additional noise is added by the imperfections of a display, such as glare, inadequate contrast, or placement. Finally, neural noise may cause the radar operator to miss the dot, to misjudge its size and, consequently, either to miss an enemy plane or to send an intercepting weapon towards a friendly aircraft. In this way, the overall performance of a humanmachine system, such as a radar installation, is equally limited by the performance of its electronic and human parts.

A radar operator? A human-machine system? Was not I talking about information theory as a paradigm for experimental psychology, a purely academic discipline? A discipline whose practitioners, according to William Estes, "developed the tradition and the reputation of being

6. Human Engineering

among the purest of pure scientists" during its first decades?318 Practitioners who for decades never left the seclusion of the university laboratory while their colleagues -- physiologists, fatigue experts, behaviorists -- seemed to hold the knowledge necessary to maximize the productivity of industrial society? The fact that after 1950 experimental psychologists displaced their colleagues in importance, of course, fits in with the larger shift from industrial to post-industrial society and the new image of work: visual and mental processing of information rather than corporeal labor. The problem with such global explanations is that they explain things too well. It is easy to connect the interest of experimental psychologists in human information processing to the shift to post-industrial society. In fact, it is as easy as to connect the very emergence of experimental psychology in the nineteenth century to the processes of modernization, industrialization, or rationalization. For instance, discussing Fechner's psychophysics, developed in the 1850s, Jonathan Crary writes that Fechner's equations were "a means of rendering a perceiver manageable, predictable, productive and above all consonant with other areas of rationalization."319 Similarly, in his discussion of reaction time experiments, which became standard in experimental psychology by the 1890s, Didier Deleule points out that "the measured time is only the sign of the living machine's capacity for increasing its productivity."320 Both Crary and Deleule are right in seeing the social utility of experimental psychology. It should be also noted, however, that this social utility remained unfulfilled for many decades. In fact, the enormous amount of knowledge accumulated by psychologists did not find its systematic application until World War II.

318 William Estes, "Experimental Psychology: an Overview," in The First Century of

Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1979), 629. 319 Crary, Techniques of the Observer, 147. 320 Didier Deleule, "The Living Machine: Psychology as Organology," in Incorporations, ed. Jonathan Crary and Sanford Kwinter (New York: Zone Books, 1992), 215.

The gradual expansion of the practical applications of experimental psychology provides a precise map of the new occupations and new conditions of modern experience which call for perceptual skills. During World War I, England, Germany, and France utilized experimental psychologists to design and administer tests for aviation pilots, aeronautical, and airplane observers, hydrophone operators, and submarine "listeners-in."321 During peacetime, a number of psychologists published papers on the readability of written text and of highway signs and on the visibility of lights at sea.322 However, in the industrial world which conceived of the worker as a human motor and was largely concerned with the productivity of manual rather than perceptual labor, these studies were an exception rather than the mainstream rule. It was World War II which finally put to use the expertise of experimental psychologists. Why did this happen? The first textbook on applied experimental psychology (1949) opens by describing the recent origins of the field: For years experimental psychologists have worked diligently in academic laboratories studying man's capacities to perceive, to work, and to learn. Only very slowly, however, have the facts and methods which they have assembled been put to use in everyday life. A particularly glaring gap in modern technology, both industrial and military, is the lack of human engineering -- engineering of machines for human use and engineering of human tasks for operating machines. Motion-andtime engineers have been at work on many of these problems, but the experimental psychologist is also needed for his fundamental knowledge of human capacities and his methods of measuring human performance. The recent war put the spotlight on this gap. The war needed, and produced, many complex machines, and it taxed the resources of both the designer and operator in making them practical for human use. The war also brought together psychologists, physiologists, physicists, design engineers, and motion-and-time engineers to solve some of these problems. Though much of their work began too late to do any real good, it has continued on a rather large scale into the piece. Today, there are many groups busy with research on man-machine problems. They use different names to describe the work in its various aspects: biotechnology, biomechanics, psychoacoustics, human engineering, and systems research. Other names may be appropriate and may appear in the future. In casting about for a title for this book, we tried to select one that would describe the subject matter without the restrictive connotations attaching to some of the names
321 Morris Viteles, Industrial Psychology (New York: W.W. Norton & Company, Inc., 1932),

322 Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook of Experimental

Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 12871340.

mentioned above. Applied Experimental Psychology seems best to fill these requirements, because the traditional data and subject of experimental psychology are fundamental to this field.323 Already before the war, experimental psychologists assisted in selecting military personnel for such jobs as pilot or airplane observer by administering special aptitude tests. During the war, a much greater number of pilots, radar operators and other similar personnel became needed. The emphasis was shifted, therefore, from selecting personnel with particularly good perceptual and motor skills to designing the equipment (controls, radar screens, dials, warning lights) to match the sensory capacities of an average person.324 And it was the field of experimental psychology that possessed the knowledge about the sensory capacities of an average, statistical person: how visibility and acuity vary between day and night; how the ability to distinguish colors and brightness vary with illumination or distance; what the smallest amount of light is which can be reliably noticed; and so on.325 All this data was now utilized for designing better displays and controls of the first modern human-machine systems such as high-speed aircrafts or radar installations. The development of these new human-machine systems during the war pushed human perceptual and mental performance to the limit and this was the second reason why experimental psychologists were called in. The performance of a human-machine system was limited by human information capacity to process information. In the words of the authors of Applied Experimental Psychology, We can make a machine that will do almost anything, given enough time and enough engineers. But man has limits to his developments, at least as far as we can see it. When we think how much a single radar can do in a small fraction of a second, and then realize by comparison that even the simplest form of reaction for a human being requires about a fifth of a second, we realize what we are up against... The full potential of radar, for example, lagged far behind physical developments

323 Alphonse Chapanis, Wendell R. Garner, and Clifford T. Morgan, Applied Experimental

Psychology (New York: John Wiley & Sons, Inc., 1949), v. 324 Ibid., 8. 325 Estes, "Experimental Psychology: an Overview," 630.

because human operators could not master the complex operation of this machine system. We had to worry about such things as a new kind of visual signal -- very small and not very bright.326 Considering that the authors described the work of time-and-motion engineers as directly leading to applied experimental psychology, this rhetoric can be expected. Taylor was impatient with the limitations of the body; now there was a similar impatience with the limitations of human information processing. With Taylor, it was the question of the speed of muscular movements; now, it became the question of reaction time: the minimum time in milliseconds required for an operator to detect a signal, to identify it, to press a control. In order to measure normal human sensory capacities, experimental psychologists have always put subjects in, so to speak, boundary conditions. They measured sensory thresholds, such as the least amount of light which can be detected. They also measured just noticeable differences (j.n.d.), the smallest difference between two stimuli which can be detected. Finally, they measured reaction times, the measure which became the main tool to deduce the time taken by different mental processes. In order to measure these characteristics, a number of standard experiments were designed, and they remained largely unchanged from the times of Weber, Fechner, and Wundt. In a detection experiment, the task of an observer is to detect the presence of barely visible stimuli, for instance a tiny light briefly flashed in the dark (did I see something?). In an identification experiment, the task is to identify which of possible stimuli was presented, for instance which of two colors (which one did I see?). In a recognition experiment, the task is to not only detect something, but to recognize what it is, for instance: what was the shape that briefly appeared (what did I see?) During World War II, the radar operator, the anti-aircraft gunner, the aircraft pilot found themselves in the same situations in which nineteenth century psychologists put their experimental subjects. The setups of psychophysical experiments became, in all details, the

326 Chapanis, Applied Experimental Psychology, 7-8.

conditions of military work; the tasks devised by psychologists to study human vision became the actual tasks faced by the operators of human-machine systems. Like the subject of a detection experiment, a radar operator scans the radar screen for a barely noticeable dot of light.327 Like the subject of an identification experiment, he has to try to guess whether this dot is the same or different from another dot which from his previous experience he knows to correspond to a friendly airplane. An anti-aircraft gunner is subjected to a recognition experiment, trying to identify a plane by its shape. And all of them, especially the pilot, are engaged in a sort of reaction time experiment. Thus, nineteenth century psychophysical setups became the military, and soon, civilian workplaces of post-industrial society; from there, they traveled back into laboratories, leading to such close interrelations between basic research in experimental psychology and its practical applications that they were no longer separable.328 The terms "applied experimental psychology," "human engineering" and "man-machine engineering" were replaced by another term standard today -- "human factors." The radar operator who in the 1940s and 1950s was the prototypical example of a human-machine system, was replaced by the 1980s by a new prototypical figure, the computer user. Thus, references to "human-machine systems" became references to "human-computer systems." The same amount of intellectual energy and research which in the middle of the century went into theorizing the performance of a radar operator and adapting him and radar display to each other, today goes into the work on computer interfaces. In
327 As Paul Fitts notes in his 1951 overview of engineering psychology, "radar operators are often forced to search for weak signals at near-threshold levels." Fitts, "Engineering Psychology and Equipment Design," 1290. 328 For example, a 1947 article in American Psychologist describes the work of Naval Research Laboratory as following these three directions: "the design of gun fire control and missile control instruments from the point of view of ease and efficiency of operation; the design and evaluation of synthetic gunnery and missile control trainers; and basic psychological research." But what is meant here by "basic research"? We read that "at present, all basic research studies are aimed at the eye-hand coordination problem involved in target tracking." "Target tracking" is just one example of a military task which traveled into a psychological laboratory, and gradually become a standard psychophysical experiment. Franklin Taylor, "Psychology at the Naval Research Laboratory," American Psychologist 2, no. 3 (1947): 87, 91.

retrospect then, we should recognize the radar operator as the central figure standing at the origins of post-industrial society, the figure which put directly into motion the new disciplines of the efficiency of the mind: engineering psychology, human information processing, and cognitive science. We are now in a position to see that the ready acceptance of information theory by psychologists in the 1950s was not simply a part of a "Zeitgeist" of the coming information age. Rather it was a practical necessity. The 1986 Handbook of Perception and Human Performance describes the rationale for adopting the information processing approach in psychology in this way: Because the language of information processing provides an objective and quantitative way of describing the basis of human performance, it has proven useful in applications. Indeed, much of the impetus for the development of this kind of empirical study stemmed from the desire to integrate description of the human within the overall systems. In the design of cockpits, control systems for weapons, and other military applications, it has been useful to have descriptions of the speed, accuracy, and reliability of the human in terms comparable to those used for other mechanical parts of the system.329 As psychologists became involved in designing human-machine systems, they needed a language to characterize human performance quantitatively. But not just any language. If the human is conceived of as part of a human-machine system (for instance, in the role of a radar operator), it is convenient to use the same language to describe the performance, capacities, and errors of both the electronic and human components. Put differently, if the human-machine system is conceived of as a machine, and its performance is to be characterized in engineering terms, the same terms need to be applied to its human components if the performance of the overall machine is to be characterized at all. Since information theory already provided a language to

329 Michael I. Posner, "Section V: Information Processing. Overview," in Handbook of

Perception and Human Performance, ed. Kenneth Boff, Lloyd Kaufman, and James P. Thomas (New York: John Wiley and Sons, 1986), II: V-6.

characterize the performance of electronic components, it was logical to apply it to the human component as well.

Modernization brought with it a special discipline concerned with efficiency -- engineering. The job of an engineer was to ensure maximum performance with a minimum investment of energy, materials, and time, be it the performance of machines (mechanical engineering), communication systems (communication engineering) or human bodies (scientific management, time and motion studies). Inspired by modern engineering, the avant-garde of the 1920s tried to systematically apply its principles to vision. To engineer vision meant to eliminate waste, to use minimal material resources. Thus, constructivist graphic design streamlined typography, eliminating complicated typefaces in favor of block letters consisting of straight lines; it also eliminated illustrations and "wasteful" decorations by making type itself the main element of design. The goal: maximum impact with minimum use of ink. To engineer vision also meant to minimize the psycho-physical resources required of the viewer. Vertov writes in his famous 1923 manifesto: "The least advantageous, the least economical communication of a scene is theatrical communication."330 In contrast, montage forces the eye to see the right thing at the right time, thus eliminating the visual waste of theater, ballet, painting, and other traditional forms. In montage, "camera drags the eyes of a film viewer from hands to legs, from legs to eyes and the rest in the most advantageous order..."331

330 Vertov, "Kinoki," 139. 331 Ibid., 139. Emphasis in the original 7. Conclusion: the Labor of Perception -- L.M.

To engineer vision also meant to ensure perception in the shortest possible time. Here as well, the avant-garde promoted montage as an example of possible economy, in this case economy of time. Maud Lavin describes the 1930 manifesto of the group of leading German designers headed by Kurt Schwitters: "Walter Dexel writes that modern man has the right to expect communications in the shortest possible time. Willi Baumeister points out that photomontage is efficient, allowing for the quick grasp of several images at once...Similarities between photomontage and film are often emphasized, with photomontage being considered a quicker, more efficient medium."332 Finally, to engineer vision also meant to be able to measure its efficiency, or, to use the language of a communication engineer, to measure "system performance." Eisenstein, fresh from engineering school, invented his first theory of artistic communication, the famous "montage of attractions": "...Laboratory analyses and diagrams...Mendeleev's Periodic Table and the laws of Gay-Lussac and Boyle Mariotte in the realm of art!...Let us therefore search for the unit which will measure the influence exerted by art! Science has its "ions," its "electrons," its neutrons." Art will have -- attractions!"333

In its desire to engineer vision, the avant-garde was ahead of its time. The systematic engineering of vision took place only after World War II when vision become the major instrument of labor, the most productive organ of a worker in a human-machine system. To ensure the maximum performance of such a system, it became necessary to engineer it around the capacities and the limitations of human vision. It also became necessary to understand vision in a new way: as information processing.

332 Maud Lavin, "Photomontage, Mass Culture, and Modernity. Utopianism in the Circle of New Advertising Designers," in Montage and Modern Life: 1919-1942, ed. Matthew Teitelbaum (Cambridge: The MIT Press, 1992), 54. 333 Qtd. in Jacques Aumont, Montage Eisenstein (London and Bloomington: BFI Publishing and Indiana University Press, 1987), 41. Emphasis mine -- L.M.

This is the history of the engineering of vision from INKhUK to MIT: from the Soviet art institute of the 1920s which gathered the avant-garde artists united by the idea of engineering vision -- to the premier engineering school where much of this engineering later occurred through the work on information theory, on radar and computer graphics, on new humanmachine interfaces, on algorithms of human vision.

This dissertation begins at the scene of heated debates at the Soviet art institutes in the early 1920s. It ends with the recent research by cognitive scientists and engineers at MIT. What justifies this temporal continuity, on the one hand, and disciplinary shift, on the other hand -from the Soviet artistic avant-garde to one of the premier science institutions in the world, which today is in the forefront of research on robotics, artificial intelligence, and human-computer interfaces? What can be in common between Soviet constructivism and American cognitive science? Both are the product of the age of engineering. Engineering, the quintessentially modern profession, appeared in the 1870s. The job of an engineer was to ensure maximum performance - of machines, human bodies or communication systems -- with minimum investment of energy, materials, and time. The engineer was the new specialist in the efficiency of any productive process, regardless of its nature. And this ideology of "engineerism" became the modern myth, the religion of a society aspiring to total rationalization.334 In order to employ vision efficiently in its new roles, whether as a medium of propaganda and advertising or as the channel of communication between human and machine, artists, designers, and scientists adopted the engineering approach as well. It is not surprising that Soviet constructivists, the pioneers of modern propaganda design, insisted on calling themselves engineers rather than artists. It is also not surprising that the archeology of twentieth century vision would uncover numerous similar solutions to the problem of its rationalization in such seemingly distant fields as film and robotics, graphic design and experimental psychology. For instance, let us compare two statements: one by a designer, the other by a psychologist. El Lissitsky wrote in the 1920s regarding his new geometric style of poster design: "The most unambiguous and immediately
334 Max Weber proposed rationalization as an umbrella concept to describe the social processes

of modernity. See, for instance, From Max Weber: Essays in Sociology (New York: Oxford University Press, 1958). Max Weber, Conclusion General Economic History (New York, 1927).

recognizable forms are geometric forms. No one will confuse a rectangle with a circle, or a circle with a triangle."335 In 1951, the already mentioned experimental psychologist Paul Fitts echoed practically the same words in his summary of research on the use of graphic symbols for humanmachine interfaces: "Geometrical figures differ considerably in legibility. Straight lines have been found to be more legible than curved ones. The triangle, the rectangle, and the square have been reported to be more easily recognized under conditions of low illumination and in peripheral vision than circular and hexagonal forms."336 Both Lissitsky and Fitts were concerned with the same problem -- reliability and speed in the recognition of visual symbols -- and, therefore, it is logical that they recommended the same solution. Whether filmmakers or psychologists, engineers or interface designers, twentieth century professionals have understood vision as a medium of communication and have attempted to put its use on a scientific basis. If in the early part of the century this research focused on human communication (the human as the subject of mass propaganda and mass entertainment), after World War II, the focus shifted to human-machine interface. Human vision became the key instrument of post-industrial labor as the channel of communication between human and machine. This history can be summarized by three images. The first image: a portrait of Tatlin by Lissitsky (fig. 16). A compass, extending straight from Tatlin's eye, a metaphor of vision for work. The second image, a popular icon in the late 1980s and early 1990s, promoted virtual reality interface designed at NASA/Ames Human Factors Research Center (fig. 22).337 Instead

335 Qtd. in Igor Golomstock, Totalitarian Art (New York: HarperCollins Publishers, 1990), 24. 336 Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook of Experimental

Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1298. 337 On NASA/Ames virtual reality research in the 1980s, see Scott S. Fisher, "Virtual Interface Environments," in The Art of Human-Computer Interface Design, ed. Brenda Laurel (Reading, Mass.: Addison-Wesley Publishing Company, 1990): 423-438.

of the metaphor of the eye-compass, a reality: video monitors strapped to the eyes. The notion of vision as work is now fully realized: the operator wearing the gear works by mentaly processing visually presented information. The gear is designed using all the available knowledge accumulated by experimental psychology about human vision. In the photograph we see the last leftover from the age of manual labor -- an arm in a DataGlove. It will soon disappear since through gaze tracking the operator can control the system by merely looking at different points in virtual space. What is the third image which stands between these two? Let me quote from the description of the history of engineering psychology found in an 1965 overview of the field: The primary emphasis in time-and-motion engineering has been on man as a worker; that is, as a source of mechanical power. It was not until World War II that a new category of machines appeared -- machines that made demands not upon the operator's muscular power, but upon his sensory, perceptual, judgmental, and decision-making abilities. The job of a radar operator, for example, requires virtually no muscular effort, but makes severe demands on sensory capacity, vigilance, and decision-making ability. This new class of machines raised some intricate and unusual questions about human abilities: How much information can a man absorb from a radar screen?338

The World War II radar operator is this third image, a historical intermediary between Lissitsky's man-compass (1920s) and NASA's virtual reality user (1980s). The figure of a radar operator stands at the gates to the post-industrial society of perceptual labor, at the origins of research on human-machine interface, and the understanding of human vision as information processing (fig. 23). "I am a mechanical eye," wrote Dziga Vertov in 1923.339
338 Alphonse Chapanis, Man-Machine Engineering (Bemont, CA: Wadsworth Publishing

Company, Inc., 1965), 9-10.
339 Dziga Vertov, "Kinoki. Perevorot" (Kinoki. A revolution), LEF 3 (1923): 141.

Allison, Nicholas H., ed. Art Into Life: Russian Constructivism 1914-1932. New York: Rizzoli International Publications, 1990. Anderson, John Robert. Cognitive Psychology and Its Implications. W.H. Freeman and Company, 1980. ArgŸelles, JosŽ. Charles Henry and the Formation of a Psychophysical Aesthetics. Chicago: University of Chicago Press, 1972. Arnheim, Rudolf. "A Plea for Visual Thinking." In New Essays on the Psychology of Art. Los Angeles and London: University of California Press, 1986. Arnheim, Rudolf. Art and Visual Perception. The New Version. Berkeley: University of California Press, 1974. Attneave, F. Applications of Information Theory to Psychology. New York: Holt, 1959. Aumont, Jacques. Montage Eisenstein. London and Bloomington: BFI Publishing and Indiana University Press, 1987. Barthes, Roland. "Rhetoric of the Image." In Image, Music, Text, edited by Stephen Heath. New York: Hill and Wang, 1977. Barthes, Roland. "The Photographic Message." In Image, Music, Text, edited by Stephen Heath. New York: Hill and Wang, 1977. Barthes, Roland. "The Third Meaning." In Image, Music, Text, edited by Stephen Heath. New York: Hill and Wang, 1977. Baxandall, Michael. Painting and Experience in Fifteenth Century Italy. Oxford: Oxford University Press, 1972. Beliajeva, E.L. Arkhitekturno-prostranstvennaya sreda soroda kak obyekt zritelnogo vospriyatiya (Architectural and spacial city environment as an object of visual perception). Moscow: Stroyizdat, 1977. Benjamin, Walter. "On Some Motives in Baudelaire." In Illuminations, edited by Hannah Arendt. New York: Schochen Books, 1969. Bergin, Thomas Goddard and Max Harold Fisch. The New Science of Giambattista Vico. Ithaca: Cornell University Press, 1968. Berlyne, D.E. "Novelty, Complexity, and Hedonic Value." Perception and Psychophysics 8 (1970): 279-86. Borch-Jakobson, M. The Absolute Master. Stanford: Stanford University Press, 1991. Braun, Marta. Picturing Time: the Work of Etienne-Jules Marey (1830-1904). Chicago: The University of Chicago Press, 1992. Bibliography

Brik, Osip. "T.n. 'formalnyi metod'" (So called 'formal method'). LEF 1 (1923): 214-215. Broadbent, D.E. Perception and Communication. Oxford: Pergamon Press, 1958. Bryson, Norman. Looking at the Overlooked: Four Essays on Still Life Painting. Cambridge: Harvard University Press, 1990. Buck-Morss, Susan. The Dialectics of Seeing: Walter Benjamin and the Arcades Project. Cambridge: The MIT Press, 1989. Bulgakova, Olga. "Sergei Eisenshtein i ego 'psikhologicheskiy Berlin' -- mezhdu psikhoanalizom i strukturnoy psikhologiey" (Sergei Eisenstein and his 'psychological Berlin': between psychoanalysis and structural psychology). Kinovedcheskie Zapiski 2 (1988): 174-191. Chapanis, Alphonse. Man-Machine Engineering. Bemont, CA: Wadsworth Publishing Company, Inc., 1965. Chapanis, Alphonse, Wendell R. Garner, and Clifford T. Morgan. Applied Experimental Psychology. New York: John Wiley & Sons, Inc., 1949. Chen, Jack. Soviet Art and Artists. London: The Pilot Press, Ltd., 1945. Cherry, Colin. On Human Communication. 2nd ed. Cambridge: The MIT Press, 1968. Cohen, Gillian. The Psychology of Cognition. 2nd ed. (London and New York: Academic Press, 1983). Cohen, Paul R. and Edward A. Feigenbaum, eds. The Handbook of Artificial Intelligence. Los Atlos, CA: William Kaufmann, Inc., 1982. Cook, Richard. "Serious Entertainment." Computer Graphics World (May 1992): 40-48. Crary, Jonathan. Techniques of the Observer: on Vision and Modernity in the Nineteenth Century. Cambridge: The MIT Press, 1990. Damisch, Hubert. L'origine de la perspective. Paris: Flammarion, 1987. Danziger, Kurt. Constructing the Subject: Historical Origins of Psychological Research. Cambridge: Cambridge University Press, 1990. De Fleur, Melvin L. and Sandra Ball-Rokeach. Theories of Mass Communication. New York: Longman, 1975. De Landa, Manuel. War in the Age of Intelligent Machines. New York: Zone Books, 1991. Deleule, Didier. "The Living Machine: Psychology as Organology." In Incorporations, edited by Jonathan Crary and Sanford Kwinter. New York: Zone Books, 1992. Descartes, RenŽ. "Meditations on the First Philosophy," In The Rationalists. Garden City, NY: Doubleday & Company, 1960. Dorfman, D. and H. McKenna. "Pattern Preference as a Function of Pattern Uncertainty." Canadian Journal of Psychology 20 (1966): 143-53.

Druckrey, Timothy. "Revenge of the Nerds. An Interview with Jaron Lanier." Afterimage (May 1991): 5-9. Dudley, Andrew. Concepts in Film Theory. Oxford: Oxford University Press, 1984. Dulac, Germaine. "Aesthetics, Obstacles, Integral CinŽgraphie." In vol. 1, French Film Theory and Criticism, edited by Richard Abel. Princeton: University of Princeton Press, 1988. Eames, Charles and Ray Eames. A Computer Perspective: Background to the Computer Age. Cambridge: Harvard University Press, 1990. Echoes of War. Boston: WGBH Boston, n.d. Videotape. Edgerton, Samuel. The Renaissance Rediscovery of Linear Perspective. New York: Basic Books, 1975. Eduardo, Kac. "Aspects of the Aesthetics of Telecommunications." In SIGGRAPH '92 Visual Proceedings, edited by John Grimes and Gray Lorig. New York: The Association for Computing Machinery, 1992. Edwards, Paul, ed. Encyclopedia of Philosophy. New York: Macmillan, 1967. Eisenstein, Sergei. "A Dialectical Approach to Film Form." In Film Form: Essays in Film Theory, edited by Jay Leyda. New York: Harcourt Brace and World, 1949. Eisenstein, Sergei. "Montazh atraktsionov" (Montage of attractions). LEF 3 (1923): 70-75. Epstein, Jean. "On Certain Characteristics of PhotogŽnie." In vol. 1, French Film Theory and Criticism, edited by Richard Abel. Princeton: University of Princeton Press, 1988). Estes, William. "Experimental Psychology: an Overview." In The First Century of Experimental Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1979. Farah, Martha. "Is Visual Imagery Really Visual? Overlooked Evidence from Neuropsychology." Psychological Review 95, no. 3 (1988): 307-317. Farocki, Harun. "Reality Would Have to Begin." Documents 1/2 (1992): 136-146. Finke, Ronald A. Principles of Mental Imagery. Cambridge: The MIT Press, 1989. Fisher, Scott S. "Virtual Interface Environments." In The Art of Human-Computer Interface Design, edited by Brenda Laurel. Reading, Mass.: Addison-Wesley Publishing Company, 1990. Fitts, Paul. "Engineering Psychology and Equipment Design." In Handbook of Experimental Psychology, edited by S.S. Stevens. New York and London: John Wiley & Sons, Inc., 1951. Flew, Antony, ed. A Dictionary of Philosophy. London: Pan Books Ltd., 1984. Florensky, Pavel. "Obratnaya Perspektiva" (The Inverted Perspective). In vol. 1, Sobranie Sochineniy, ed. N.A. Struve. Paris: YMCA-PRESS, 1985.

Florensky, P.A. "Symbolarium (Slovar simvolov). Predislovie" [Symbolarium (The dictionary of symbols). Introduction]. Trudy po znakovym sistemam V. Tartu: Uchebnye zapiski Tarturskogo Universiteta 284, 1971. Foster, Hal, ed., Vision and Visuality. Seattle: Bay Press, 1988. Freud, Sigmund. The Interpretation of Dreams. New York: Avon Books, 1965. Freud, Sigmund. Standard Edition of the Complete Psychological Works. London: Hogarth Press, 1953. Friedhoff, Richard Mark and William Benson. The Second Computer Revolution: Visualization. W.H. Freeman and Company, 1991. Gabor, D. "A Summary of Communication Theory." In Proceedings of a Symposium on Applications of Communication Theory, London, 1952, edited by Willis Jackson. London: Butterworth Scientific Publications, 1953. Gardner, Martin. Logical Machines and Diagrams. 2nd ed. Chicago: The University of Chicago Press, 1982. Golomstock, Igor. Totalitarian Art. New York: HarperCollins Publishers, 1990. Gombrich, E.H. "Pictorial Instructions." In Images and Understanding, ed. Barlow Horace, Colin Blakemore and Miranda Weston-Smith. Cambridge: Cambridge University Press, 1990. Gombrich, E.H. Art and Illusion. Princeton: Princeton University Press, 1960. Goodman, Nelson. Languages of Art. Indianapolis: Hackett, 1976. Hall, Stuart. "Encoding/Decoding." In Culture, Media, Language, ed. Stuart Hall et al. Hatchinson, 1980. Hansen, Miriam. "The Hieroglyph and the Whore: D.W. Griffith's Intolerance." The South Atlantic Quarterly 88, no. 2 (1989): 361-392. Hartley, R.V.L. "Transmission of Information." Bell System Tech. Journal 7 (1928): 535-563. Hearst, Eliot. "One Hundred Years: Themes and Perspectives." In The First Century of Experimental Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates, 1979. Heidegger, Martin. What is Called Thinking? New York: Harper & Row, Publishers, 1968. Heron, W. "The Pathology of Boredom." Scientific American 196 (1957): 52-56. Hochberg, Julian. "Sensation and Perception." In The First Century of Experimental Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates, 1979. Holly, Michael Ann. Panofsky and the Foundations of Art History. Ithaca: Cornell University Press, 1984.

Holmes, Oliver Wendel. "The Stereoscope and the Stereograph." In Photography, Essays and Images: Illustrated Readings in the History of Photography, edited by Beaumont Newhall. New York: Museum of Modern Art, 1980. Hoover, Theodore and Charles Fish. The Engineering Profession. Stanford: Stanford University Press, 1941. "Institut Khudozestvennoy Kultury" (The Institute of Artistic Culture). Russkoe Iskusstvo 2-3 (1923): 85-88. Ivins, William M. On the Rationalization of Sight. New York: Da Capo Press, Inc., 1975. Jakobson, Roman. "Closing Statement: Linguistics and Poetics." In Semiotics. An Introductory Anthology, edited by Robert Innis. Bloomington: Indiana University Press, 1985. Jakobson, Roman. "Metalanguage as a Linguistic Problem," Presidential address delivered at the Annual Meeting of the Linguistic Society of America, 1956. In The Framework of Language. Ann Arbour: Michigan Studies in the Humanities, 1980. Jay, Martin. Downcast Eyes: the Denigration of Vision in Twentieth-century French Thought. Berkeley: The University of California Press, 1993. Jay, Martin. "Scopic Regimes of Modernity." In Vision and Visuality, ed. Hal Foster. Seattle: Bay Press, 1988. Johnson-Laird, Philip. Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness. Cambridge: Cambridge University Press, 1983. Joravsky, David. Russian Psychology: A Critical History. Cambridge, Mass.: Basil Blackwell, 1989. Kandinsky, Wassily. Point and Line to Plane. New York: Solomon R. Guggenheim Foundation, 1947. Kandinsky, Wassily. "Programma raboty Instituta khudozhestvennoy kultury" (The program of the Institute of Artistic Culture). In Sovetskoe iskusstvo za 15 let. Materialy i dokumentatsiia (Fifteen years of Soviet art: materials and documentation), ed I. Matsa (Moscow, IZOGIZ: 1933). Kantowitz, Barry H. and Robert D. Sorkin. Human Factors: Understanding People-System Relationships. John Wiley & Sons: 1983. Keller, Evelyn Fox and Christine R. Grontkowski. "The Mind's Eye." In Discovering Reality: Feminist Perspectives on Epistemology, Metaphysics, Methodology, and Philosopohy of Science, edited by Sandra Harding and Merrill B. Hintikka. London: D. Reidel Publishing Company. Kemp, Martin. The Science of Art. New Haven: Yale University Press, 1990. Kenez, Peter. The Birth of the Propaganda State. Soviet Methods of Mass Mobilization, 19171929. Cambridge: Cambridge University Press, 1985.

Khan-Magomedov, S.O. "INKhUK: vozniknovenie, formirovanie i pervyy period raboty. 1920" (INKhUK: appearance, formation and first period of its work, 1920). Sovetskoe Iskusstvoznanie (1981): 332-368. Koerner, Joseph Leo. "The Shock of the View." The New Republic (April 26, 1993): 32-38. Kogan, P.C. "O zadachakh akademii i yeyo zhurnala" (About the goals of the academy and its journal). Iskusstvo 1, no. 1 (1923). Kovasznay, L.S.G. and H.M. Joseph. "Image Processing." Proceedings of IRE 43 (1955): 560570. Kozulin, Alex. Psychology in Utopia: Toward a Social History of Soviet Psychology. Cambridge: The MIT Press, 1984. Krauss, Rosalind. The Optical Unconscious. Cambridge: The MIT Press, 1993. Lacan, Jacques. "On the Gaze as Objet Petit a." In The Four Fundamental Concepts of PsychoAnalysis, edited by Jacques-Alain Miller. New York: W.W. Norton & Company, 1981. Lakoff, George. "The Invariance Hypothesis: Is Abstract Reason Based on Image-Schemas?" Cognitive Linguistics 1, no. 1 (1990): 39-74. Lakoff, George. "Cognitive Linguistics." Versus 44/45 (1986): 119-154. Latour, Bruno. "Visualization and Cognition: Thinking with Eyes and Hands." Knowledge and Society: Studies in the Sociology of Culture Past and Present 6 (1986): 1-40. Lavin, Maud. "Photomontage, Mass Culture, and Modernity. Utopianism in the Circle of New Advertising Designers." In Montage and Modern Life: 1919-1942, edited by Matthew Teitelbaum. Cambridge: The MIT Press, 1992. Lavrent'ev, A.H. "Propedevticheskaya distsiplina 'Grafika.' Vkhutemas. 1920-22 gody" (The discipline 'Graphics.' VKhUTEMAS. 1920-22). Tekhnicheskaya Estetika 7 (1984): 16-21. Lefebvre, Henri. The Production of Space. Oxford: Blackwell Publishers, 1991. LŽvy-Bruhl, Lucien. Les functions mentales dans les sociŽtŽs primitives. Paris: 1910. Locke, John. An Essay Concerning Human Understanding. Edited by A.S. Pringle-Pattison. Oxford: Clarendon Press, 1924. Lotman, Jurij. The Structure of the Artistic Text. Ann Arbour: Department of Slavic Languages and Literatures, The University of Michigan, 1977. Lowe, David. Three-Dimensional Object Recognition from Single Two-Dimensional Images. Robotics Report 62. New York: Courant Institute of Mathematical Sciences, New York University, 1986. Mandler, Jean Matter and George Mandler, eds. Thinking: From Association to Gestalt. New York: John Wiley & Sons, Inc., 1964.

Manovich, Lev. "Assembling Reality: Myths of Computer Graphics." Afterimage 20, no. 2 (September 1992): 12-14. Manovich, Lev. "'Real' Wars: Esthetics and Professionalism in Computer Animation." Design Issues 6, no. 1 (Fall 1991): 18-25. Marr, David. Vision. New York: W.H. Freeman and Company, 1982. Matsa, I., ed. Sovetskoe iskusstvo za 15 let. Materialy i dokumentatsiia (Fifteen years of Soviet art: materials and documentation). Moscow, IZOGIZ: 1933. McArthur, David. "Computer Vision and Perceptual Psychology." Psychological Bulletin 92, no. 2 (1982): 283-309. McFarlane, M. D. "Digital Pictures Fifty Years Ago." Proc. IEEE 60, no. 7 (1972). McGraw-Hill Encyclopedia of Science & Technology: an International Reference Work in Twenty Volumes Including Index. New York: McGraw-Hill, 1992. Michelson, Annette. "Reading Eisenstein Reading 'Capital'." October 2 (1976): 27-38; October 3 (1977): 82-88. Miller, G.A. Language and Communication. New York: McGraw-Hill Book Co., Inc., 1951. Mitchell, J. William. The Reconfigured Eye: Visual Truth in the Post-Photographic Era. Cambridge, The MIT Press, 1992. Mitchell, W.J.T. Iconology. Image, Text, Ideology. Chicago: The University of Chicago Press, 1985. Moholy-Nagy, L‡szl—. "Photography in Advertising." In Photography in the Modern Era, edited by Christopher Phillips. New York: Aperture, 1989. Moholy-Nagy, L‡szl—. The New Vision. New York: Wittenborn, 1947. L‡szl— Moholy-Nagy. Painting, Photography, Film. 1925. Reprint. Cambridge: The MIT Press, 1973. Molok, Yu. A. "'Slovar simvolov' Pavla Florenskogo. Nekotorye margonalii" (Pavel Florensky's 'dictionary of symbols.' A few margins). Sovetskoe Iskusstvoznanie 26 (1990): 322-343. MŸnsterberg, Hugo. The Photoplay: A Psychological Study. New York: D. Aplleton & Co., 1916. Newhall, Beaumont. Airborne Camera. New York: Hastings House, Publishers, 1969. Nietzsche, Friedrich. "On Truth and Falsity in their Ultramoral Sense." In Early Greek Philosophy and Other Essays. New York: The Macmillan Company, 1924. Noble, Douglas. "Mental Materiel: The Militarization of Learning and Intelligence in U.S. Education." In Cyborg Worlds: the Military Information Society, edited by Les Levidov and Kevin Robins. London: Free Association Books, 1989.

Nšth, Winfried. Handbook of Semiotics. Bloomington and Indianapolis: Indiana University Press, 1990. Panofsky, Erwin. Perspective as Symbolic Form. New York: Zone Books, 1991. Peat, David. Artificial Intelligence: How Machines Think. New York: Baen Enterprises, 1985. Pinker, Steven. "Visual Cognition: An Introduction." Cognition 18 (1984): 1-63. Posner, Michael I. and Gordon L. Shulman. "Cognitive Science." In The First Century of Experimental Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1979. Posner, Michael I. "Section V: Information Processing. Overview." In vol. 2, Handbook of Perception and Human Performance, edited by Kenneth Boff, Lloyd Kaufman, and James P. Thomas. New York: John Wiley and Sons, 1986. Quastler, H., ed. Information Theory in Psychology: Problems and Methods. Glencoe, IL.: Free Press, 1955. Rabinbach, Anson. The Human Motor: Energy, Fatigue, and the Origins of Modernity. Basic Books, Inc., 1990. Rappoport, A.G. "El Lissitsky i ego 'Pun-geometiya'" (El Lissitsky and his 'Pan-geometry'). Sovetskoe Iskusstvoznanie 25 (1989): 113-130. Reichardt, Jasia. The Computer in Art. London and New York: Studio Vista and Van Nostrand Reinhold Company, 1971. Resnikov, Howard. The Illusion of Reality. New York: Springer-Verlag New York Inc., 1989. Reveaux, Tony. "Virtual Reality Gets Real." New Media (January 1993): 36-41. Rheingold, Howard. Virtual Reality. New York: Simon & Schuster, Inc., 1991. Riggs, L.A. et al. "The Disappearance of Steadily Fixated Visual Tests Objects." J. Optical Soc. America 43 (1953): 495. Roberts, L.G. "Machine perception of three-dimensional solids." In Optical and Electo Optical Information Processing, edited by J.T. Tippett. Cambridge: The MIT Press, 1965. Roberts, L.G. Homogeneous Matrix Representations and Manipulation of N-Dimensional Constructs. MIT Lincoln Laboratory MS 1405, 1965. Roberts, L.G. Machine Perception of Three-Dimensional Solids. MIT Lincoln Laboratory TR 315, 1963. Rose, Jacqueline. Sexuality in the Field of Vision. London: Verso, 1986. Saint-Martin, Fernande. Semiotics of Visual Language. Bloomington and Indianapolis: 1990. Sampson, Geoffrey. Schools of Linguistics. Stanford, CA: Stanford University Press, 1980.

Sardarov, A.S. "5000 let evolutsii doroznogo znaka" (5000 years of the evolution of a traffic sign). Tekhnicheskaya Estetika 9 (1984): 14-19. Scha, Remko. "Virtual Voices." MediaMatic 7, no. 1 (1992): 27-43. Schramm, Wilbur. "How Communication Works." In The Process and Effects of Mass Communication. Urbana: University of Illinois Press: 1954. Sekula, Allan. "The Instrumental Image: Steichen at War." In Photography against the Grain: Essays and Photo Works, 1973-1983. Halifax: The Press of the Nova Scotia College of Art and Design, 1984. Sekula, Allan. "The Body and the Archive." October 39 (1987): 3-64. Shannon, Claude E. and Warren Weaver. The Mathematical Theory of Communication. Urbana: The University of Illinois Press, 1949. Shepard, S. and J. Metzler. "Mental Rotations of Three-dimensional Objects." Science 171 (1971): 701-703. SIGGRAPH '89 Panel Proceedings. New York: The Association for Computing Machinery, 1989. SIGGRAPH '92 Final Program. New York: The Association for Computing Machinery, 1992. Sonesson, Gšran. Pictorial Concepts. Inquiries into the Semiotic Heritage and its Relevance for the Analysis of the Visual World. Lund, Sweden: Lund University Press, 1989. Spriegel, William R. and Clark E. Myers, eds. The Writings of the Gilbreths. Homewood, IL., 1953. Stelzer, Otto. "Moholy-Nagy and His Vision." In L‡szl— Moholy-Nagy, Painting, Photography, Film. 1925. Reprint, Cambridge: The MIT Press, 1973. Taylor, Brandon. Art and Literature under the Bolsheviks. London: Pluto Press, 1991. Taylor, Franklin. "Psychology at the Naval Research Laboratory." American Psychologist 2, no. 3 (1947): 87-92. Taylor, Frederick Winslow. The Principles of Scientific Managment. New York, 1967. Titchener, Edward Bradford. A Beginner's Psychology. New York: The Macmillan Company, 1915. Todorov, Tzvetan. Theories of the Symbol. Ithaca: Cornell University Press, 1984. Tretyakov, Sergei. "Otkuda i kuda? (Perspektivu futurizma)" [Where from and where to? (Perspectives of futurism)]. LEF 1 (1923): 192-203. Vasilieva, Yu. A. "Eisenstein: Kontseptsija 'Agressivnogo Iskusstva'" (Eisenstein: the concept of 'aggressive art'). Kinovedcheskije Zapiski 3 (1989), 205-209. Vertov, Dziga. "Kinoki. Perevorot" (Kinoki. A revolution). LEF 3 (1923): 135-143

Virilio, Paul. Lost Dimension. New York: Semiotext(e), 1991. Virilio, Paul. War and Cinema: the Logistics of Perception. London: Verso, 1989. Viteles, Morris. Industrial Psychology. New York: W.W. Norton & Company, Inc., 1932. Vitz, P.C. "Preference for Different Amounts of Visual Complexity." Behavioral Science 2 (1966): 105-14. Vitz, Paul C. and Arnold B. Glimcher. Modern Art and Modern Science. The Parallel Analysis of Vision. New York: Praeger, 1984. Vygotsky, Lev. Psikhologija Iskusstva (Psychology of art). Moscow, 1968. Weber, Max. From Max Weber: Essays in Sociology. New York: Oxford University Press, 1958. Weber, Samuel. The Legend of Freud. Minneapolis: University of Minnesota Press, 1982. Wiener, Norbert. The Human Use of Human Beings. Cybernetics and Society. New York: Avon Books, 1967. Wiener, Norbert. Cybernetics. Of Control and Communication in the Animal and the Machine. New York: John Wiley & Sons, Inc., 1948.

FIGURES Figure 1. Humbert de Duperville's "Synoptic Table," 1827-32. Figure 2. Charles Henry's aesthetic protractor, 1988. Figure 3. Diagram illustrating Georges Seurat's theory of expression. Figure 4. The exercises developed by Rodchenko for his course on graphics at VKhUTEMAS, 1921-22. Top: examples of basic forms given by Rodchenko. Bottom: example of a student work. Figure 5. Example of computer generated visualization. Figure 6. Representations of propositions as Euler's diagrams, an earlier form of Venn's diagrams. Max MŸller, The Science of Thought (London: Longmans, Green, and Co., 1887), 543. Figure 7. Composite photography. Frontispiece from Francis Galton, Inquiries into Human Faculty (London: Macmillan, 1883). Figure 8. Freud's diagram illustrating the relation between two triads of concepts: id, ego, and superego, and unconscious, preconscious, and conscious. Figure 9. Top: Sigmund Freud's schematic picture of sexuality. Bottom: example of a modern diagram. Figure 10. Example of diagrammatic graphic language. Figure 11. Figure from Monge's GŽomŽtrie descriptive, 1799. Figure 12. Two draughtsmen plotting points for the drawing of a lute in foreshortening, from Durer's Underweysung, 1525. Figure 13. Wireframe computer graphics by Boeing, early 1960s. Figure 14. ACRONYM computer vision system processes an aerial photograph, late 1970s. Figure 15. Laser range finder, early 1980s. Figure 16. El Lissitsky. Tatlin at Work. Photomontage. Illustration for Ilia Erenburg's Six Tales with Easy Endings. 1922. Figure 17. Wireless telegraphed photography. Figure 18. Schematic diagram of a general communication system. Figure 19. Example of a stimulus used in experiments on preference as a function of the amount of information. Figure 20. One of the earliest models of a human as information processing system. Figure 21. The model of a human as information processor which summarizes the available data (by 1986) about human sensory, cognitive and motor performance. Figure 22. NASA Ames Virtual Reality Environment Workstation, late 1980s. Figure 23. SAGE (the "Semi-Automatic Ground Environment") -- the first human-machine interactive display system, mid 1950s.

Sign up to vote on this title
UsefulNot useful