You are on page 1of 321

r e a di n g i m a g e s

Thissecondeditionof the landmarktextbookReadingImages


buildson its reputationasthe irst systematicand
comprehensive accountof the grammarof visualdesign.
Drawingon an enormous rangeof examples from children,s
drawingsto textbookil lustrations,photo-journalism
to fineart,
aswell asthree-dimensional formssuchas sculpture andtoys,
the authorsexamine the waysin whichimagescommunicate
meantng.

Features
of this fully updatedsecondeditioninclude:
. newmaterialon movingimagesand on colour
. a discussion of howimagesandtheir useshavechanged
t h r o u g ht i m e
. websites andweb-based images
. ideason the futureof visualcommunication.

ReadingImagesfocuses on the structuresor'grammar'of visual


design- colour,perspective,framingandcomposition - and
providesthe readerwith an invaluable'tool-kit'for
reading
images,whichmakesit a mustfor anyoneinterested in
communication, the mediaandthe arts.

GuntherKressis Professor of Englishat the Instituteof


Education, University of London.Theo van Leeuwenhas
workedas a film andtelevision producerin the Netherlanos ano
Australiaandas Professor in the Centrefor Language &
Communication Research at CardiffUniversity. He is currently
Deanat the Facultyof Humanities and SocialSciences,
Universityof Technology, Sydney. Theyhavebothpublished
widelyin the fieldsof language andcommunication studies.
Pr ai s ef o r t h e
f i r s t e di t i o n
'ReadingImagesis the mostimportantbookin visual
c o m m u n i c a t i os ni n c eJ a c q u eB s e r t i n ' s e m i o l o goyf
i n f o r m a t i ognr a p h i c sI t. i s b o t ht h o r o u g ha n dt h o u g h t -
provoking; a remarkable breakthrough.'
l(evinG. Barnhurst,SyracuseUniversity,USA
' F r e s ha n ds t i m u l a t i n gT.h es o c i o c e n t rai cp p r o a c ihs b y f a r
the mostpenetrating
approachto the subjectcurrently
available.'
PaulCobley,LondonGuildhallUniversity
'A usefultext for al I studentswho are involvedin areas
w h i c hr e l yo n b o t hl a n g u a gaen dv i s u a il m a g e fso r t h e i r
e x o r e s s i oann da r t i c u l a t i oonf i d e a s . '
Catriona Scott, M iddlesexUniversity
'Thisis the bestdetailedandsustained
development
of the
" s o c i a sl e m i o t i ca" p p r o a ctho t h ea n a l y s iosf v i s u a l sC. l e a r ,
informative andtheoretical ly developmental.'
Dr S. Cottle,Bath HE College
' E x c e l l e n-t w i d er a n g i n -g a c c e s s i b-l et u t o r s '" B i b l e " . '
Jan Mair, EdgeHill UniversityCollegeof
Higher Education
'Extremelyattractiveandwell laid out.Veryuseful
bibliography.'
Dr IVl.Brottman,East LondonUniversity
'Veryclearlywritten- it makesgoodconnections between
differentareasof visualpractice- especially usefulfor
studentsfrom a varietyof backgrounds attempting
" m i x e d "c o u r s e w o r k . '
Amy Sargeant,PlymouthUniversity
r e a d i n gi m a g e - s
G U N T H E RK R E S S a n d
T H E 0 v a n L E E U W E N

T H E 6 R A M M A R
O F V I S U A LD E S I 6 N

S E C O N DE D I T I O N

Hltiy"'l:s,g:",
LONDON AND NEW YORK
Firstoublished
1995
by Routledge
2 ParkSquare,MiltonPark,Abingdon,0xon
0X144RN
published
Simultaneously in the USAandCanada
by Routledge
270 MadisonAve,NewYork,NY 10016

editionpublished
Second 2006
2007(twice),2008
Reprinted

Routledgeis an imprint of the Taylor & Francis Group,an informa business

@ 1996,2006GuntherKressandTheovanLeeuwen

Typesetin Bell Gothicby RefineCatch


Ltd, Bungay,Suffolk
Printedandboundin GreatBritainby
TJ InternationalLtd, Padstow,Cornwall

No partofthisbookmaybe reprinted
All rightsreserved. or reproduced or
utilizedin anyformor byanyelectronic,mechanical,
or othermeans/ now
knownor hereafter invented, photocopying
including andrecording, or in any
withoutpermission
informationstorageor retrievalsystem, in writingfrom
the oublishers.
British Library Cataloguingin Publication Data
A catalogue recordfor thisbookisavailable fromthe BritishLibrary

Library of CongressCatalogingin Publication Data


l(ress.GuntherR.
Reading images: the grammarof visualdesign / Guntherl(ressandTheo
vanLeeuwen. - 2nded.
0 .c m .
Includes bibliographical
references andindex.
1. Communication in design. I. VanLeeuwen,Theo,I94T- II. Title.
Nl(1510.K64
2006
70I-dc22 2006002242

ISBN10:0-415-31914-5 (hbk)
ISBN10:0-415-31915-3 (pbk)
I SBN10: 0-203-61972-2 Gbk)

ISBN13: 978-0415-3L9I4-0 (hbk)


ISBN13: 97844I5-3L915-7 (pbk)
ISBN13: 978-0-203-6L972-!(ebk)
CONTENTS

vii P r e f a c teo t h e s e c o n e
dd i t i o n
ix Prea
f c et o t h ef i r s t e d i t i o n
xi Acknowledgements

I I n t r o d u c t i otnh:eg r a m m aor f v i s u a l
design
16 T h es e m i o t i cl a n d s c a p el a: n g u a g ae n d
v i s u a cl o m m u n i c a t i o n
45 N a r r a t i v ree p r e s e n t a t i o ndse: s gi n i n g
s o c i a al c t i o n
79 C o n c e p t u ar e l p r e s e n t a t i o ndse: s i g n i n g
s o c i a lc o n s t r u c t s
tt4 4 Representatio a n di n t e t a c t i o n :
d e s i g n i n tgh e p o s i t i o no f t h e v i e w e r
t54 5 g o d e l so f r e a l i t y
M o d a l i t yd: e s i g n i n m
L75 T h em e a n i n go f c o m p o s i t i o n
2t5 7 M a t e r i a l i t ay n d m e a n i n g
239 B T h et h i r d d i m e n s i o n
266 9 C o l o u r f utlh o u g h t s( a p o s t s c r i p t )

27t References
287 Index
P r e fa c e t o t h e s e c o n de d i t i o n

Thefirst editionof ReadingImageshashada positivereception amonga widegroupfrom


t h e p r o f e s s i o nasn d d i s c i p l i n ewsh i c hh a v et o d e a lw i t h r e a l p r o b l e m a s nd real issues
involving i m a g e sT. h i s h a s g o n ea l o n gw i t h a b r o a d e ra g e n d ao f c o n c e r nw i t h ' m u l t i -
m o d a l i t ya ' , r a p i d l yg r o w i n gr e a l i z a t i otnh a t r e p r e s e n t a t i iosna l w a y sm u l t i p l eW. ed o n o t
think for a moment that this book represents anything like a settled approach, a definitive
'grammar'of images, andat timeswe havebeenworriedby attemptsto treat it in that way.
We seeit as an earlyattempt/oneamongmanyothers,andwe wouldliketo seeit treated
v e r ym u c ha s a r e s o u r cfeo r b e g i n n i ntgo m a k ei n r o a d si n t o u n d e r s t a n d i tnhge v i s u a a l s
r e p r e s e n t a t iaon dc o m m u n i c a t i o- ni n a s e m i o t i fca s h i o n- a n da l s oa s a r e s o u r cien t h e
d e v e l o p m eonftt h e o r i eas n d' g r a m m a r so' f v i s u acl o m m u n i c a t i oI n .t h a ts p i r i tw ew a n tt o
stressthat we seeeverything we havewrittenheresimultaneously as our fully seriousand
y e te n t i r e l yp r o v i s i o n sael n s e o f t h i sf i e l d .
When we completedthe first editionof this book we were aware of a numberof
' o m i s s i o n s ' t-h i n g sw e f e l t s t i l l n e e d e d o i n g S . o m eo f t h e s ew e h a v et a k e nu p i n o t h e r
ways, for instance in our attempt to develop a theory of multimodality; otherswe havetried
to addressin this second edition. Foremost among these have been the quite different
issuesof the movingimage and of colour. The first of these has been constantly raisedby
thosewho haveusedthe book, and rightly so. We hope that what we have said herecan
beginto integratethe field of moving images into our social semiotic approach to visual
communication. Theissue colourof was less frequently raised, yet constituted for us a l<ind
of theoreticaltest case, as much to do with the issue of colour itself as to do with a theory
o f m u l t i m o d asl o c i a sl e m i o t i cm s u c hm o r ew i d e l yc o n s i d e r e H d .e r et,o o ,w e f e e lt h a t w e
have provided just a first attempt for a different approach. In additionwe haveaddeda
n u m b eo r f n e we x a m p l e s f r o m CD-R0Ms a n d w e b s i t e s
d ,
o m a i n s f v i s u a cl o m m u n i c a t i o n
o
that had hardlybegunto develop when we wrote the first edition, and are now of central
importance for many users of this book.
0ne persistent criticismof the first editionfrom a groupof readershasbeenthat the
b o o kw a s( t o o )l i n g u i s t i cT.h ef i r s tc o m m e nwt ew o u l dm a k ei st o s a yt h a tf o r u s ' f o r m a l i t y '
i n t h ed o m a i no f r e p r e s e n t a t i iosnn o t i n a n yw a yt h es a m ea s ' b e i n gl i n g u i s t i cS' .ot o s o m e
extentwe think that that criticismrestson that kind of misunderstanding. We alsothink
thatthere is a difference betweenexplicitness and formality. We certainly have aimedfor
theformer,andoften (but not always)for the latter. Nor do we think that either explicitness
or formalityare the enemiesof innovation, creativity,imagination: often all theselatter
rest on the former.It is the case that our starting point has been the systemicfunctional
grammarof Englishdeveloped by MichaelHalliday,though we hadand haveattemptedto
useits generalsemioticaspectsratherthan its specificlinguistically focusedfeaturesas
the grounding for our grammar. As Ferdinand de Saussure had done at the beginning of
the last century, we see linguistics as a part of semiotics; but we do not see linguistics as the
vil| Preface to the second edition

disciplinethat can furnisha ready-mademodelfor the descriptionof semioticmodes


otherthan language. Thenwe had thought,in our first attempt,that to showhow visual
c o m m u n i c a t i owno r k si n c o m p a r i s ot no l a n g u a gm e i g h tb e h e l p f uiln u n d e r s t a n d i e n igt h e r
and both - but that, too, was misunderstood maybeas an attemptto imposelinguistic
categories on the visual.We havethereforetried to refineand clarifythosesectionsof the
bookthat dealwith the relationbetweenlanguage andvisualcommunication, andto delete
or reformulatematerialwhichwe think mighthavegivenriseto thesemisunderstandings,
h o p e f u l lw
y i t h n o l o s so f c l a r i t yA
. c a r e f u rl e a d i n go f t h i s s e c o n d
e d i t i o no f o u r b o o kw i l l
show,we trust,that we are as concerned to bring out the differences betweenlanguage
a n d v i s u a lc o m m u n i c a t i oans w e a r e t h e c o n n e c t i o ntsh,e b r o a d e rs e m i o t i cp r i n c i p l e s
t h a t c o n n e c tn, o t j u s t l a n g u a g a e n d i m a g e b, u t a l l t h e m u l t i p l em o d e si n m u l t i m o d a l
communication.
In our growingunderstanding of this domain,reffectedin the reworkingof this book,
we owea debtof gratitudefor support,commentand critiqueto manymorepeoplethan
we can mentionor eventhanwe actuallyknow.But the namesof somefriends,colleagues,
students, fellowresearchers andcriticswho werenot alreadyacknowledged in our preface
to the first editionhaveto be mentioned. Amongtheseare CareyJewitt,Jim Gee,Ron
Scollon,Paul Mercer,Brian Street, Radan Martinec,Adam Jaworski,David Machin,
l(las Prytz,TealTriggs,AndrewBurn,Bob Ferguson, PippaStein,DeniseNewfield,Len
Unsworth,LesleyLancasterand the many researchers whosework has both givenus
confidence and new ideas,and extendedour understanding of this field - and of course,
and crucially, we acknowledge the supportfrom our publishers and editorsat Routledge,
L o u i s aS e m l y eann dC h r i s t a b el (l i r k p a t r i c k .
P r e fa c e t o t h e f i r s t e d i t i o n

T h i sb o o l < g r e wo u t o f d i s c u s s i o anb s o u tv i s u a cl o m m u n i c a t i ownh i c hs p a n n ead p e r i o do f


sevenyears.Bothof us hadworkedon the analysisof verbaltexts,and increasingly felt the
needof a betterunderstanding of all the thingsthat 9o with the verbal:facialexpressions,
gestures, images,music,and so on. Thiswas not only because we wantedto analysethe
wholeof thetextsin whichthesesemioticmodesplaya vital roleratherthanjusttheverbal
part, but alsoto understand language better.Just as a knowledge of other languages can
0pennewperspectives on one'sown language, so a knowledge of othersemioticmodescan
0pennewperspectives on language.
I n 1 9 9 0 w e p u b l i s h ead f i r s t v e r s i o no f o u r i d e a so n v i s u a lc o m m u n i c a t i oR n ,e a d i n g
Images,with DeakinUniversityPress.It waswrittenfor teachers, andwe concentrated on
c h i l d r e n 'dsr a w i n gas n ds c h o otle x t b o o ki l l u s t r a t i o nasl,t h o u g h
w e a l s oi n c l u d eedx a m p l e s
from the massmedia,suchas advertisements and magazinelayout.Sincethen we have
expandedour researchto other fieldsof visualcommunication: a muchwider rangeof
massmediamaterials;scientific(and other)diagrams,mapsand charts;and the visual
arts.We havealso madea beginning with the studyof three-dimensional communication:
sculpture, children'stoys,architectureand everydaydesigned objects.The presentbook
thereforeoffers a much more comprehensive theory of visualcommunication than the
e a r l i e rb o o k .
I n A u s t r a l i aa, n d i n c r e a s i n gel yl s e w h e roeu, r w o r k h a sb e e nu s e di n c o u r s e so n c o m -
municationand mediastudies,and as a methodology for researchin areassuchas media
representation, film studies, children'sliteratureandthe useof illustrations and layoutin
schooltextbooks. The presentbook hasbenefitedgreatlyfrom the suggestions and com-
mentsof thosewho haveusedour work in theseways,and of our own undergraduate and
postgraduate students, initiallyat the University of Technology and MacquarieUniversity
i n S y d n e yl ,a t e r a t t h e I n s t i t u t eo f E d u c a t i o na n d t h e L o n d o nC o l l e g eo f P r i n t i n gi n
London,andalsoat the TemasekPolytechnic in Singapore.
W e b e g a no u r w o r k o n v i s u a lc o m m u n i c a t i oinn t h e s u p p o r t i v ea n d s t i m u l a t i n g
environmeo n ft t h e N e w t o w nS e m i o t i cC s i r c l ei n S y d n e yd;i s c u s s i own si t h o u r f r i e n d st ,h e
m e m b e r os f t h i s C i r c l eh, e l p e ds h a p eo u r i d e a si n m o r ew a y st h a nw e c a n a c k n o w l e d g e .
If any two peoplefrom that first periodwereto be singledout, it would be Jim Martin,
who gaveus meticulous, detailed,extensive and challenging commentson severalof the
chaptersof the earlierbook,and FranChristie,who had urgedus to write it. But herewe
wouldalsoliketo makea specialmentionof Bob Hodge,whoseideasappearin this bookin
manyways/evenif not alwaysobviously so.
0f thosewho usedour book in teachingand research, and whosecommentson the
e a r l i e rb o o k h a v eh e l p e du s r e t h i n ka n d r e f i n eo u r i d e a s w , e wouldlike to mention
the research teamof the Disadvantaged Schools Programme in Sydney, in particularRick
Iedema,SusanFeez,PeterWhite, RobertVeeland Sally Humphrey;Staffan Selander,
x . Preface to the first edition

throughwhoseCentrefor TextbookResearch in Hdrnosand our work cameto betakenup


by researchers in the field of textbookresearchin Swedenand severalother European
countries;the membersof the 'Languageand Science'researchteam at the Institute
o f E d u c a t i o nI s, a b e lM a r t i n s J, o n 0 g b o r na n d l ( i e r a nM c G i l l i c u d d yP;h i l i pB e l l ; B a s i l
B e r n s t e i nP;a u lG i l l e na n dT e u nv a n D i j k .
T h r e ew r i t e r si n f f u e n c eodu r i d e a si n d i f f e r e nat n d f u n d a m e n t awl a y s . O n ei s R o l a n d
B a r t h e sA. l t h o u g h w e s e eo u rw o r ka s g o i n gb e y o n h d i ss e m i n awl r i t i n go n v i s u asl e m i o t i c s
in severalways,he remainsa stronginspiration. Thereis not a subjectin semioticson
y .e h a sp r o v i d efdo r u s a m o d e l
w h i c hB a r t h e sh a sn o t w r i t t e no r i g i n a l l ay n d i n s p i r i n g l H
of what semioticscan be, in the rangeof his interests, in the depthof his work, and in
h i se n g a g e m ew n ti t h t h e s o c i a a
l n dc u l t u r aw l o r l d .E q u a l l ys i g n i f i c a nf ot r u s i s M i c h a e l
H a l l i d a yH. i s v i e w o f l a n g u a g a e s a s o c i a ls e m i o t i ca, n d t h e w i d e r i m p l i c a t i o nosf h i s
theories, gaveusthe meansto go beyondthe structuralistapproachof 1960sParisSchool
s e m i o t i c sa,n d o u r w o r k i s e v e r y w h e rien f f u e n c ebdy h i s i d e a s T . h e nt h e r e i s R u d o l f
Arnheim.The morewe readhis work,the morewe realizethat most of what we haveto
say hasalreadybeensaid by him, often betterthan we havedoneit, albeit it usuallyin
commentaries on individual worksof art ratherthan in the form of a moregeneraltheory.
H e i s c o m m o n lay s s o c i a t ewdi t h G e s t a lpt s y c h o l o gwy e : w o u l dl i k et o c l a i mh i ma s a g r e a t
s o c i asl e m i o t i c i a n .
W ew o u l dl i k et o t h a no u r e d i t o rJ, u l i aH a l l ,f o r h e re n c o u r a g e m eanntd i n v a l u a b lhee l p
i n p r o d u c i ntgh i s b o o k .J i l l B r e w s t ear n d L a u r aL o p e z - B o n i lw l ae r ei n v o l v e d in various
s t a g e so f t h e b o o k t; h e i re n c o u r a g e m eanntdh e l pm a d et h ew o r kp o s s i b laen de n j o y a b l e .
Acltnowledgements

P l a t e3 J o s h u aS m i t h b yW i l l i a mD o b e l l1, 9 4 3 , @D A C S2 0 0 4 .

Plate5 Cossacks by Vassilyl(andinsl<y, 1910-1911.@ ADAGP,Parisand DACS,London


2 0 0 4 .P h o t o g r a p h@
y T a t eL
, o n d o n2 0 0 5 .

P l a t e6 H i s t o r i cC o l o u r sb y C o l i nP o o l er,e p r o d u c ebdy k i n d p e r m i s s i oonf P h o t o W o r d


S y n d i c a t i oLnt d .

P l a t e7 P a l g r a vceo l o u rs c h e m ree p r o d u c ewdi t h p e r m i s s i oonf P a l g r a v M


e acmillan.

1.1+1.13 'My bath' from Baby's First Book by B. Lewis,illustratedby H. Wooley,


c o p y r i g h@
t L a d y b i r dB o o k s1 t d . , 1 9 5 0 .

L 2 B i r d i n t r e e f r o m 0 n M y W a l l gb y D i c k B r u n a ,1 9 8 8 .I l l u s t r a t i o nD i c k B r u n a@
M e r c i sb 4 1 9 7 2 .

1 . 4 M a g a z i n ceo v e rw i t h n a t u r a l i s t ipch o t o g r a p h
co, v e ro f N e w s w e eA k ,p r i l 9 , 2 0 0 4 @
2 0 0 4 N e w s w e e kI n, c . P h o t o g r a p b h y l ( a r i m S a h i b - A F P - G e t It m
y a g e s .R e p r i n t e db y
p e r mi s si o n .

1.5 Magazine coverwith conceptual photograph, coverof Newsweek, 12,200I


November
O 2 0 0 1 N e w s w e eIkn,c .R e p r i n t ebdy p e r m i s s i o n .

1.6 Imagefrom 'lnteractivePhysics'.


Courtesyof MSC Software.

2.5 Communication J. and Hill, A. (1980) A Dictionaryof Communi-


modelf rom Watson,
cationand MediaStudiegLondon,Arnold,p.I43. Reproduced by permission of Hodder
H e a d l i nPeL C .

2 . 6 T w o C o m m u n i c a t i oMno d e l sf r o m W a t s o nJ, . a n d H i l l , A . ( 1 9 8 0 ) A D i c t i o n a r yo f
Communication and MediaStudieqLondon,Arnoldp.I47. Reproduced by permission of
H o d d eHr e a d l i nPeL C .

2.IO Beatthe Whiteswith the Red Wedgeby El Lissitzky,I9I9-20.@ DACS2004.


( 1 8 7 8 - 1 9 3 5 )" S u p r e m a t i sCt o m p o s i t i o R
2 . 1 1 l ( a s i m i rM a l e v i c h n :e dS q u a r ea n d B l a c k
S q u a r e "1, 9 1 4 ,N e wY o r k ,M u s e u mo f M o d e r nA r t ( M o l V l A@ ) 2 0 0 4 ,D i g i t a il m a g eT, h e
Museum o f M o d e r nA r t , N e wY o r l < / S c a lFal ,o r e n c e .

2.I7 Gulf War Diagram,SydneyMorning Herald,14 February,1991 reproducedby


permissionof SydneyMorning Herald.

2.18 Speechcircuitfrom Saussure's Coursein GeneralLinguistics,I9T4, F.de Saussure,


t r a n s l a t ebdy R o yH a r r i sb y p e r m i s s i oonf G e r a l dD u c k w o r t &
h Co.
xii . Acknowledgements

t sp r o d u c ebdy k i n dp e r m i s s i oonf N e s t l 6G r o u p .
2 . 2 0 V i t t e la d v e r t i s e m e nr e
2.22 Communication Model from Watson,J. and Hill, A. (1980) A Dictionaryof
Communication and MediaStudieELondon,Arnoldp.I47. Reproduced by permission
of
H o d d eH
r e a d l i nPeL C .
2.23 Arctictundrasystem/fi1.7.5, p.1,72f rom Sale,C.,Friedman,
B. and Wilson, G. )ur
ChangingWorld,Book1, PearsonEducationAustralia.Reproduced by permission of the
publisher.

2.24 Communication Modelfrom Watson,J. and Hill, A. (1980) A Dictionaryof Com-


municationand MediaStudies,London,Arnoldp.54. Reproduced by permission
of Hodder
H e a d l i nP
eL C .
3 . 1 G u i d ei n t e r f a c fer o m ' D a n g e r o uCs r e a t u r e s ' , 7 9 9S by permis-
4 .c r e e n s h or et p r i n t e d
s i o nf r o m M i c r o s o fC t orporation.
3 . 2 S e k o n da d v e r t i s e m ernetp r o d u c ebdy k i n dp e r m i s s i oonf S e k o n d a / T i mPer o d u c t s .
3.3 Sourcesof signsfrom Eco,U. (1976) A theoryof semiotics,Bloomington, Indiana
U n i v e r s i tPy r e s sp, . 1 7 7 .R e p r o d u c ebdy p e r m i s s i oonf t h e p u b l i s h e r .
3.5 Semanticfield diagramfrom Eco, U. (I976) A theoryof semiotics,Bloomington,
IndianaUniversityPress,p.78.Reproducedby permission
of the publisher.
3 . 7 N e t w o r kf r o m S h a r p l e sM, . a n d P e m b e r t o n , ' R e p r e s e nwt irni tgi n g e
: x t e r n arle p r e s e n -
tationsand the writing process'inN. Williamsand P. Holt,eds Computers and Writing.
Reproduced by kind permission of IntellectLtd,www.intellectbooks.com
3.13 Resortwear,AustralianWomen'sWeekly,December
1987.@ AustralianWomen's
Weekly/ACP with permission.
Reproduced
Syndication.
3.19 Electricalcircuitdiagramfrom J. Hill, 1980, IntroductoryPhysics.Reproduced
by
p e r m i s s i oonf T a y l o r& F r a n c i G
s roup.
, . A . l ( .( 1 9 7 8 )
3 . 2 0 T h e p l a c eo f l i n g u i s t i cosn t h e m a p o f k n o w l e d gfer o m H a l l i d a yM
Languageas Social Semiotic,London,Arnold. Reproduced by permission of Hodder
H e a d l i nPeL C .
3.21 'Womenat work', Fig.III-6, p.29,from Pictographs and Graphs:How to Makeand
Use Themby RudolfModleyand DynoLowenstein. @ 1952 by Harper& Brothers. Copy-
r i g h tr e n e w e 1
d 9 8 0 b y P e t e rM . M o d l e ya n d M a r i o nE . S c h i l l i n gR. e p r i n t ebdy p e r m i s s i o n
o f H a r p e r C o l l i nPsu b l i s h e rI n
sc.
3.28 'Fun with fungi',SydneyMorningHerald,18 June1992, reproduced
by permission
of SydneyMorning Herald.
4 . 2 A T Ms c r e e nr e p r o d u c ebdy k i n dp e r m i s s i oonf N a t i o n aAl u s t r a l i aB a n k .
4.4 The murder of Dr Chang,SydneyMorning Herald,5 July 1991, reproducedby
permissionof SydneyMorning Herald.
Acknowledgements xiii

4 . 5 P l a y s t a t i owne b s i t er e p r o d u c ebdy k i n dp e r m i s s i oonf S o n yC o m p u t eEr n t e r t a i n m e n t


E u r o p eL t d .

r ekp r o d u c ebdy l < i n dp e r m i s s i oonf F o r d .


4 . 7 N e wl o o kF o r dM o n d e of r o mw w w . f o r d . c o . u

4 . 8 F i e s t a ' R o c ks o l i d ' w e b s i tree p r o d u c ebdy k i n dp e r m i s s i oonf 0 g i l v yG r o u pH o l d i n g s


Ltd and Ford.

4.16 'PrisonGuard'by DannyLyon,1969,fromConversations with the dead.@ Danny


L y o n . M a g n uP
mh o t o sR. e p r o d u c ewdi t h p e r m i s s i o n .

by permission
4.19 Gulfwar map,SydneyMorningHerald,22January1991,reproduced
of SydneyMorning Herald.
by
4.20 An increasein tourism,SydneyMorningHerald,22 January1991, reproduced
permissionof SydneyMorning Herald.

4.22 DeIail from a fourteenth-century Spanishnativity,from Rudolf Arnheim,Art


and Visual Perception: A Psychologyof the Creative Eye. The New Version.@ I974
T h e R e g e n tos f t h e U n i v e r s i toyf C a l i f o r n i aR. e p r o d u c ebdy p e r m i s s i oonf U n i v e r s i toyf
CaliforniP a ress.

5.1 Speechcircuitfrom Saussure's Coursein GeneralLinguistics,7974,F. de Saussure,


t r a n s l a t ebdy R o yH a r r i sb y p e r m i s s i oonf G e r a l dD u c k w o r t &
h Co.

5.2 Schematized speechcircuit from Saussure's Coursein GeneralLinguistics,1974,


F . d eS a u s s u r e , t r a n s l abt e
ydR o yH a r r i sb y p e r m i s s i oonf G e r a l dD u c k w o r t &
h Co.

5.6 Card-players(Van Doesburg,1916-17) photograph/picture: Tim l(oster,ICN,


R i j s w i j k / A m s t e r d aRme.p r o d u c ewdi t h p e r m i s s i oonf I n s t i t u tC o l l e c t i N
e ederland.

5.7 Composition 9 (YanDoesburg, I9I7). Collection DenHaag.


of the Gemeentemuseum
R e p r o d u c ewdi t h p e r m i s s i o n .

r e s i d e n c( G
5 . 8 C o l o u rP r o j e cfto r t h e S c h r o d eR e e r r i tR i e t v e l d1,9 2 3 - 4 )@ D A C S2 0 0 5 .

5 . 9 P h o t o g r a pohf t h e S c h r o d eR
r e s i d e n cGee, r r i tR i e t v e l d@, D A C S2 0 0 4 .

5.11 @ OxfordUniversityPressfrom The YoungGeographer Investigates: Mountainsby


T e r r yJ e n n i n g(s0 U P ,1 9 8 6 ) ,r e p r i n t ebdy p e r m i s s i oonf 0 x f o r dU n i v e r s i tPy r e s s .

5.12 Drawing b y N e w t o nB. y p e r m i s s i oonf t h eW a r d e na n d F e l l o w sN, e wC o l l e g e , O x f o r d ,


a n dT h eB o d l e i a n , n i v e r s i toyf 0 x f o r d .M S 3 6 1 ,v o l .2 , f o l . 4 5 V .
L i b r a r yU

5.13 Drawingof Stretton'sexperiment, figure 8.1 (p.141) from The Eye and Brain:
Psychology of Seeing5/eby RichardGregory,1998, reproduced by permission
of 0xford
UniversityPress.Gregory,Richard,Eyeand Brain.Reprinted by permission
of Princeton
U n i v e r s i tPy r e s s .

6.2 Gold-diggers, AustralianWomen'sWeekly,November 1987.@ AustralianWomen's


W e e k l y i A CS n .e p r o d u c ewdi t h p e r m i s s i o n .
Py n d i c a t i o R
xiv . Acknowledgements

6 . 3 S o n yl Vild d l eE a s t w e b s i tree p r o d u c ebdy k i n dp e r m i s s i oonf S o n yG u l f F Z E .

6.9 Gerbner'scommunicationm f r o dmeW


l atson,J.andHill,A.(198
A0D)i c t i o n a r y o f
Communication and MediaStudies,London,Arnold.Reproduced by permission of Hodder
H e a d l i nPeL C .

6 . 1 0 R o y a lc o u p l e .H . M . T h e Q u e e n ' w s e d d i n gp, h o t o g r a p bh y B a r o n ,C a m e r aP r e s s ,
L o n d o nH. . M .T h eQ u e e a n n d P r i n c eP h i l i pp, h o t o g r a pbhy H . R . H .P r i n c eA n d r e wC, a m e r a
. e p r o d u c ewdi t h p e r m i s s i o n .
P r e s sL, o n d o nR

6.12 Buddhistpaintingfrom RudolfArnheim,Art and VisualPerception: A Psychology


of the CreativeEye.TheNew Version.@1974 The Regents of the University
of California.
R e p r o d u c ebdy p e r m i s s i oonf U n i v e r s i toyf C a l i f o r n i P
a ress.

6.13 'Going on holiday'from Prosser,R. (2000) Leisure,Recreationand Tourism,


L o n d o nC, o l l i n sE d u c a t i o n aRl .e p r i n t ebdy p e r m i s s i oonf H a r p e r C o l l i nPsu b l i s h e rLst d @
R .P r o s s e r , 2 0 0 0 .

n o d e lf r o m W a t s o n J, . a n d H i l l , A . ( 1 9 8 0 )
6 . 1 4 A n d e r s c he t a l . ' s c o m m u n i c a t i om
A Dictionary of Communicationand Media Studies,London,Arnold. Reproducedby
p e r m i s s i oonf H o d d e H
r e a d l i nP
e LC.

by kind permis-
6.16 Verticaltriptychfrom the websiteof 0xford Universityreproduced
sionof 0xford University.

6 . 2 3 S c r e e n s h of rt o m C D - R 0 M ' 3 D B o d y A d v e n t u r e 'l,( n o w l e d g A
e d v e n t u r e1, 9 9 3
provided courtesyof l(nowledge Adventure, Inc.

7.1 Roy Lichtenstein,


Big Painting,1965. @ The Estateof Roy Lichtenstein/DACS
2004.

8.I Jacob and the Angel Qacob Epstein,1940) O The Estate of Jacob Epstein/Tate,
L o n d o n2 0 0 5 .I m a g es u p p l i ebdy a n dr e p r o d u c ebdy p e r m i s s i oonf G r a n a d T
a V.

8.2 Peoplein the Wind (l(ennethArmitage,1952). Reproduction


courtesyof l(enneth
ArmitageEstate.Photography @ Tate,London2005.

8 . 5 W o m a nb y J o a nM i r 6 , I 9 7 O@ S u c c e s s M
i oi r o ,D A C S2 0 0 4 .

8 . 6 L e s H e u r e s d e s T r a c e s ( H o u r o f t h e T r a c e l b y A l b e r t o G i a c1o9m3e0t@
t i ,A D A G P ,
Parisand DACS,London2004. Photography @ Tate,London2005.

8.7 Jacoband the Angel Qacob Epstein,1940) O The Estateof Jacob Epstein/Tate,
L o n d o n2 0 0 5 .I m a g es u p p l i ebdy a n dr e p r o d u c ebdy p e r m i s s i oonf G r a n a d T
a V.
8 . 8 P l a y m o b 'i fl a m i l ys e t ' a n d ' e t h n i cf a m i l y ' f r o mP l a y m o b icl a t a l o g u eR. e p r o d u c ebdy
k i n dp e r m i s s i oonf P l a y m o b iUl l <L t d .

8.9 Recumbent Figureby HenryMoore,1938.Illustratedon p. 250;hasbeenreproduced


b y p e r m i s s i oonf t h e H e n r yM o o r eF o u n d a t i o P
n .h o t o g r a p h@
y T a t eL
, o n d o n2 0 0 5 .
Acknowledgements . xu

o n p . 2 5 3 ; h a sb e e nr e p r o -
B . I 0 l ( i n g a n d Q u e e nb y H e n r yl V l o o r e1,9 5 2 - 3 . l l l u s t r a t e d
d u c e db y p e r m i s s i oonf t h e H e n r yM o o r eF o u n d a t i o P n .h o t o g r a p h@ yTateL , o n d o n2 0 0 5 .

8.11 Churchof SantaMaria DellaSpinafrom RudolfArnheim,Art and VisualPercep-


tion: A Psychologyof the CreativeEye. The New Version.@ 7974 The Regentsof the
U n i v e r s i toyf C a l i f o r n i aR. e p r o d u c ebdy p e r m i s s i oonf U n i v e r s i toyf C a l i f o r n i P
a ress.
8 . 1 2 C o n n e c t eadn d d i s c o n n e c t enda r r a t i v ep r o c e s sp/p . 8 3 - 8 4 f r o m S . G o o d m a na n d
D. Graddol,RedesigningEnglish- new texts, new identities,London,Routledge,1997.
R e p r o d u c ebdy p e r m i s s i oonf t h e p u b l i s h e r .

8.13 Overshoulder shotin computergame,DeltaForce.Image


courtesyof NovaLogic
Inc.
@ 2 0 0 4 .A l l r i g h t sr e s e r v e d .

8.14 Dynamicinterpersonalrelationsin the openingsceneol The Big Sleep(Howard


Hawks,1947),pp.9I-92 from S. Goodmanand D. Graddol,Redesigning English- new
texts,newidentities,
London,Routledge,l997. Reproduced ofthe publisher.
by permission
I n t r o d u c t i o nt :h e g r a m m a o
r fvisual
design

T h es u b t i t l eo f t h i s b o o ki s ' t h e g r a m m a ro f v i s u a ld e s i g n 'W . e h e s i t a t e od v e rt h i s t i t l e .
E x t e n s i o nosf t h e t e r m ' g r a m m a r ' o f t e ns u g g e s t ' r u l e sI'n. b o o k sw i t h t i t l e s l i k e T h e
Grammarof Television Productionone learns,for instance, aboutthe rulesof continuity;
l < n o w i nt h g e s er u l e si st h e nw h a ts e t st h e' p r o f e s s i o n a l ' a p a f rrot mt h e' a m a t e u r W ' . h a tw e
wish to expressis a little different.In our view,most accountsof visualsemioticshave
' w o r d s ' - w h a t l i n g u i s tcsa l l
c o n c e n t r a t eodn w h a tm i g h tb e r e g a r d eads t h e e q u i v a l e notf
'lexis'- ratherthan 'grammar',andthenon the 'denotative' and'connotative', the 'icono-
g r a p h i c a l ' a n d ' i c o n o l o g i c a l ' s i g n i f i coaf nt hceee l e m e n tisn i m a g e st h , e i n d i v i d u aple o p l e ,
p l a c e sa n dt h i n g s( i n c l u d i nagb s t r a c t ' t h i n g sd' )e p i c t etdh e r e I. n t h i sb o o k b , yc o n t r a s t , w e
will concentrate on 'grammar' and on syntax,on the way in which theseelementsare
c o m b i n e di n t o m e a n i n g f uwl h o l e sJ. u s t a s g r a m m a r so f l a n g u a g d eescribe howwords
' g r a m m a r ' w i l dl e s C r i bt e h ew a y i n
C o m b i nien C l a u s esse, n t e n c eaSn dt e x t s s, o o u r v i s u a l
'statements'of
w h i c hd e p i c t e d e l e m e n t-s p e o p l ep, l a c e sa n dt h i n g s- c o m b i n ei n v i s u a l
greateror lessercomplexity andextension.
We are by no meansthe first to dealwith this subject.Nevertheless, by comparison to
the studyof visual'lexis',the studyof visual'grammar'hasbeenrelativelyneglected, or
dealt with from a differentperspective, from the point of view of art history,or of the
formal,aestheticdescriptionof composition, or the psychology of perception, or with a
focuson morepragmaticmatters,for instance the way composition can be usedto attract
the viewer'sattentionto onethingratherthananother, e.g.in suchappliedenvironments as
advertisingor packaging. All theseare validapproaches, and in many placesand many
wayswe havemadeuseof the insightsof peoplewritingfrom thesedifferentperspectives.
Yet the resulthas beenthat, despitethe very largeamountof work doneon images,not
muchattentionhasbeenpaidto the meanings of regularities in theway imageelements are
used- in short,to their grammar- at leastnot in explicitor systematic ways.It is this
focuson meaning that we seek,aboveall,to describe andcapturein our book.We intendto
p r o v i d eu s a b l ed e s c r i p t i o nosf m a j o rc o m p o s i t i o nsatlr u c t u r ew s h i c hh a v eb e c o m e s t a b -
l i s h e da s c o n v e n t i o ni n s t h e c o u r s eo f t h e h i s t o r yo f W e s t e r nv i s u a ls e m i o t i c sa,n d t o
analysehowtheyare usedto producemeaningby contemporary image-makers.
W h a t w eh a v es a i da b o u t v i s u a' g l r a m m a r ' i s t r u ea l s oo f t h e m a i n s t r e a m of linguistic
g r a m m a rg : r a m m a rh a s b e e n ,a n d r e m a i n s , ' f o r m a lI' t. h a s g e n e r a l l b y e e ns t u d i e di n
isolation f r o m m e a n i n gH. o w e v et[h e l i n g u i s tas n dt h e s c h o ool f l i n g u i s t itch o u g h ft r o m
w h i c h w e d r a w p a r t oofu r i n s p i r a t i o n - l i n g u i s t s f o l l o w i n g t h e wM o ri ck ho af eHl a l l i d a y -
havetaken issuewith this view,and see grammaticalforms as resources for encoding
i n t e r p r e t a t i o nosf e x p e r i e n caen d f o r m s o f s o c i a l( i n t e r ) a c t i o nB. e n j a m i nL e e W h o r f
t o l a n g u a g ef rso m d i f f e r e nct u l t u r e sI n
a r g u e dt h e p o i n ti n r e l a t i o n . w h a th ec a l l e d' S t a n d -
'summer', 'winter','September', 'morning',
ard AverageEuropean'languages, terms like
' n o o n ' , ' s u n s eat 'r e c o d e da s n o u n sa/ s t h o u g ht h e yw e r et h i n g s H . e n c et h e s el a n g u a g e s
Introduction

make it possibleto interprettime as something you can count/use/save/etc. In Hopi,a


N o r t h A m e r i c a nI n d i a nl a n g u a g et h, i s i s n o t p o s s i b l eT.i m e c a n o n l y b e e x p r e s s eads
'subjective duration-feeling'. you haveto sav
Youcannotsay'at noon',or'threesummers,.
s o m e t h i nlgi k e' w h i l et h es u m m epr h a s ei s o c c u r r i n g( ,W h o r f 1 , 950.
T h ec r i t i c a ll i n g u i s tos f t h e E a s tA n g l i aS c h o o lw, i t h w h o mo n eo f u s w a sc o n n e c t e d ,
haveshownthat suchdifferentinterpretations of experience can alsobe encoded usingthe
resources of the samelanguage, on the basisof differentideological positions. TonyTrew
( 1 9 7 9 : 1 0 6 - 7 ) h a sd e s c r i b ehdo w ,w h e nt h e H a r a r ep o l i c e- i n w h a t w a s i n 1 9 7 5 s t i l l
Rhodesia - fired into a crowdof unarmedpeopleand shotthirteenof them,the
Rhodesia
Heraldwrote,'A politicalclashhas led to death and iniury,,whilethe TanzanianDaily
N e w sw r o f e , ' R h o d e s i aw ' sh i t e s u p r e m a t i spt o l i c e. . . o p e n e df i r e a n d k i l l e dt h i r t e e n
unarmedAfricans.'In otherwords,the politicalviewsof newspapers are not onlyencoded
t h r o u g hd i f f e r e nvt o c a b u l a r i e( so f t h e w e l l - k n o w 'nt e r r o r i s t ' v s' f r e e d o mf i g h t e r ,t y p e ) ,
but also throughdifferentgrammaticalstructures;that is, throughthe choicebetween
c o d i n ga n e v e n ta s a n o u n( ' d e a t h ' , ' i n j u r y o ' )r a v e r b( ' k i l l ' ) ,w h i c hf o r i t s g r a m m a t i c a l
c o m p l e t i orne q u i r eas n a c t i v es u b j e c(t' p o l i c e 'a) n da n o b j e c t( , u n a r m eA df r i c a n s , ) .

Grammargoesbeyondformal rulesof correctness. It is a meansof representing


p a t t e r n so f e x p e r i e n c e . . . . l te n a b l e h
s u m a nb e i n g st o b u i l da m e n t a lp i c t u r eo f
reality,to makesenseof their experience of what goeson aroundthem and inside
them.
( H a l l i d a y , 1 9 8t50:t )

T h e s a m ei s t r u e f o r t h e ' g r a m m a ro f v i s u a ld e s i g n 'L. i k e l i n g u i s t i sc t r u c t u r e sv,i s u a l


structurespointto particularinterpretations of experience andformsof socialinteraction.
To somedegreethesecan also be expressed linguistically. Meaningsbelongto culture,
ratherthanto specificsemioticmodes.And the way meanings are mappedacrossdifferent
semiotim c o d e st ,h ew a ys o m et h i n g sc a n , f o ri n s t a n c eb,e ' s a i d ' e i t h evri s u a l l yo r v e r b a l l y ,
o t h e r so n l yv i s u a l l ya,g a i no t h e r so n l yv e r b a l l yi s, a l s oc u l t u r a l l ay n dh i s t o r i c a l lsyp e c i f i cI n.
the courseof this book we will constantlyelaborateand exemplifythis point.But even
whenwe can express what seemto be the samemeanings in eitherimage-form or writing
or speech, they will be realizeddifferently.For instance, what is expressed in language
throughthe choicebetweendifferentword classesand clausestructures,may,in visual
communication, beexpressed throughthe choicebetween differentusesof colouror differ-
e n tc o m p o s i t i o nsatlr u c t u r e sA.n dt h i sw i l l a f f e c tm e a n i n gE. x p r e s s i nsgo m e t h i nvge r b a l l y
or visuallm y a k e sa d i f f e r e n c e .
As for otherresonances of the term 'grammar'(grammar,asa set of rulesonehasto
obey if one is to speakor write in'correct', sociallyacceptable ways),linguistsoften
protestthat they are merelydescribing what peopledo,and that othersinsiston turning
d e s c r i p t i o ni nst or u l e s B
. u t o f c o u r s et o d e s c r i bies t o b e i n v o l v e idn p r o d u c i nkgn o w l e d g e
whichotherswill transformfrom the descriptive intothe normative, for instancein educa-
t i o n .W h e na s e m i o t i cm o d ep l a y sa d o m i n a nrto l e i n p u b l i cc o m m u n i c a t i oint s, u s ew i l l
inevitablybe constrainedby rules,rulesenforcedthrougheducation,for instance, and
I ntroduction

t h r o u g ha l l k i n d so f w r i t t e na n d u n w r i t t e ns o c i a sl a n c t i o n s0.n l y a s m a l le l i t eo f e x p e r i -
m e n t e r iss a l l o w e dt o b r e a kt h e r u l e s- a f t e ra l l ,b r e a k i n g r u l e sr e m a i n n s e c e s s a troy k e e p
o p e nt h e p o s s i b i l i t y
o f c h a n g eW . ebelieve t h a t v i s u a c
l o m m u n i c a t i o i s
n c o m i n gt o b e l e s s
a n d l e s st h e d o m a i no f s p e c i a l i s tasn, d m o r ea n d m o r e c r u c i a li n t h e d o m a i n so f p u b l i c
c o m m u n i c a t i oInn.e v i t a b l tyh i s w i l l l e a dt o n e w a
, n d m o r e r u l e s a
, n d t o m o r ef o r m a l /
n o r m a t i v et e a c h i n gN . o t b e i n g ' v i s u a l l Iyi t e r a t e ' w i l lb e g i n t o a t t r a c t s o c i a ls a n c t i o n s .
' V i s u a ll i t e r a c y ' w i lbl e g i nt o b ea m a t t e ro f s u r v i v ael ,s p e c i a l il n y t h ew o r k p l a c e .
We are well awarethat work suchas ourscan or will helppavethe way for develop-
mentsof this kind.Thiscan be seennegatively, as constraining the relativefreedomwhich
v i s u acl o m m u n i c a t i oh na ss of a r e n j o y e da,l b e i t a t t h ee x p e n s oe f a c e r t a i nm a r g i n a l i z a t i o n
by comparison to writing;or positively, as allowing more people greateraccess to a wider
rangeof visualskills.Nor doesit haveto stand the in way of reativity. Teaching the rulesof
writinghasnot meanttheendof creativeusesof language in literature and elsewhere, and
t e a c h i n vgi s u a sl k i l l sw i l l n o t s p e l tl h e e n do f t h e a r t s . Y e t j
, u s t a s t h e g r a m m a r
c r e a t ively
employedby poetsand novelistsis, in the end,the same grammar we use when writing
l e t t e r sm, e m o sa n d r e p o r t ss, o t h e ' g r a m m a ro f v i s u a ld e s i g n ' c r e a t i v eel ym p l o y e d by
artistsis,in the end,the same grammar we needwhen producing attractive layouts, images
a n dd i a g r a mfso r o u rc o u r s eh a n d o u t sr e, p o r t sb, r o c h u r ecso, m m u n i q u 6asn,ds o o n '
It is worth askingherewhat a linguisticgrammaris a grammarof.fhe conventional
' E n g l i s h ' o r ' D u t c h ' o r ' F r e n c h ' - t hr uel e st h a t
a n s w eirs t o s a yt h a t i t i s a g r a m m aor f
d e f i n eE n g l i s a h s ' E n g l i s hD ' , u t c ha s ' D u t c h 'a, n ds oo n .A s l i g h t l yl e s sc o n v e n t i o naanl s w e r
w o u l db e t o s a yt h a t a g r a m m a ri s a n i n v e n t o roy f e l e m e n tasn d r u l e su n d e r l y i ncgu l t u r e -
s p e c i f i fco r m so f v e r b a cl o m m u n i c a t i o n . ' U n d e r l y i n g ' ihsear e s h o r t h a ntde r mf o r s o m e -
t h i n gm o r ed i f f u s ea n dc o m p l e xm, o r el i k e ' k n o w l e d gs e h a r e d r eo r l e s sb y m e m b e rosf a
m o
g r o u pe, x p l i c i t layn di m p l i c i t l yT' .h i sb r i n g si n s u b t l em a t t e r so f w h a tl < n o w l e d igsea n dh o w
i t i s h e l da n de x p r e s s eadn, da b o v ea l l t h e s o c i aql u e s t i oonf w h a ta ' g r o u p ' i s .T h a tm a k e s
definitions of grammarvery mucha socialquestion, oneof the knowledges and practices
s h a r e db y g r o u p so f p e o p l e .
We might now ask,'What is our "visual grammar" agraffimar of?'First of all we
w o u l ds a yt h a t i t d e s c r i b eass o c i a rl e s o u r coef a p a r t i c u l a9r r o u pi,t s e x p l i c iat n di m p l i c i t
knowledge aboutthis resource/ and its usesin the practicesof that group.Then,second, we
wouldsaythat it is a quitegeneral grammart because we need a term that can encompass
o i l p a i n t i n ga s w e l l a s m a g a z i n lea y o u tt,h e c o m i cs t r i pa s w e l l a s t h e s c i e n t i f idci a g r a m .
Drawingthesetwo pointstogether, and bearingin mindour socialdefinitionof grammar,
w e w o u l ds a y t h a t ' o u r ' g r a m m a r i s a q u i t eg e n e r a g l Y a m m aorf c o n t e m p o r a rvyi s u a l
d e s i g ni n ' W e s t e r n ' c u l t u r easn, a c c o u not f t h e e x p l i c iat n d i m p l i c i tl < n o w l e d a gn e dp r a c -
t i c e sa r o u n da r e s o u r c e co , n s i s t i nogf t h e e l e m e n tasn d r u l e su n d e r l y i n ag c u l t u r e-specific
f o r mo f v i s u acl o m m u n i c a t i oW n .eh a v e q u i t edeliberately m a d e o u r d e f i n i t i o an s o c i aol n e ,
b e g i n n i nwgi t h t h e q u e s t i o n ' W h ai st t h e g r o u p ?W h a ta r e i t s p r a c t i c e s ? ' a n d f r omthere
attemptingto describe the grammarat issue, ratherthan adopting an approach which says,
' H e r e i so u r g y a m m a r ; d o t h e p r a c t i c e s a n d l < n o w tl e hd i sggersooufp c o n f o r m t o i t o r n o t ? '
In the bookwe have,by and large,confinedour examples to visualtext-objects from
, W e s t e r n ' c u l t u raens da s s u m etdh a t t h i sg e n e r a l i z a t i o hn a ss o m ev a l i d i t ya s i t p o i n t st o a
Introduction

communicational situationwith a longhistorythat hasevolvedoverthe pastfivecenturies


or so/alongside writing (quitedespitethe differences betweenEuropeanlanguages), as a
'languageof visualdesign'.
Its boundaries are not thoseof nation-states, althoughthere
are,andverymuchso,cultural/regional variations. Rather/ this visualresource hasspread,
alwaysinteracting with the specificities of locality,whereverglobalWesterncultureis the
d o m i n a nct u l t u r e .
This means,first of all, that jt is not a'universal'grammar. Visuallanguageis not -
despite a s s u m p t i o tnost h e c o n t r a r y- t r a n s p a r e nat n du n i v e r s a lul yn d e r s t o o idt ;i s c u l t u r -
ally specific.We hopeour work will continueto providesomeideasand conceptsfor the
studyof visualcommunication in non-Western formsof visualcommunication. To givethe
mostobviousexample, Westernvisualcommunication is deeplyaffectedby our convention
o f w r i t i n gf r o m l e f t t o r i g h t ( i n c h a p t e r6 w e w i l l d i s c u s tsh i s m o r ef u l l y ) .T h e w r i t i n g
directionsof culturesvary:from rightto left or from leftto right,from top to bottomor in
circularfashionfrom the centreto the outside.Consequently differentvaluesandmeanings
are attachedto suchkeydimensions of visualspace.Thesevaluations and meanings exert
their inffuencebeyondwriting, and inform the meaningsaccordedto differentcom-
positionalpatterns, the amountof usemadeof them,andso on.In otherwords,we assume
t h a t t h e e l e m e n t s ,u c ha s ' c e n t r e ,o r , m a r g i n ,',t o p , o r ' b o t t o m , ,w i l l p l a y a r o l e i n t h e
visualsemioticsof any culture,but with meaningsand valuesthat are likelyto differ
d e p e n d i nognt h a t c u l t u r e ' hs i s t o r i eosf u s eo f v i s u asl p a c ew, r i t i n gi n c l u d e dT.h e' u n i v e r s a l ,
aspectof meaningliesin semioticprinciples and processes, the culture-specific aspectlies
in their applicationoverhistory,and in specificinstances of use.Herewe merelywant to
signalthat our investigations havebeenrestricted,by and large,to Westernvisualcom-
m u n i c a t i o nE.v e nt h o u g ho t h e r sh a v eb e g u nt o e x t e n dt h e a p p l i c a t i o nosf t h e p r i n c i p l eosf
this grammar, we makeno specificclaimsfor the application of our ideasto othercultures.
Within Westernvisualdesign,however, we believethat our theoryappliesto all formsof
v i s u a cl o m m u n i c a t i oW n .e h o p et h a t t h e w i d er a n g eo f e x a m p l ews e u s ei n t h e b o o kw i l l
c o n v i n cree a d e ros f t h i sp r o p o s i t i o n .
0 u r s t r e s so nt h e u n i t yo f W e s t e r vni s u acl o m m u n i c a t i o dn o e sn o te x c l u dteh ep o s s i b i l i t y
o f r e g i o n aal n ds o c i avl a r i a t i o nT.h eu n i t yo f W e s t e r d n e s i g ni s n o ts o m ei n t r i n s i fce a t u r eo f
v i s u a l i t yb,u t d e r i v efsr o m a l o n gh i s t o r yo f c u l t u r acl o n n e c t i oann di n t e r c h a n gaes,w e l la s
now from the globalpowerof the Westernmassmediaand cultureindustriesand their
technologies. In manypartsof the world,Westernvisualcommunication existssideby side
with localforms.Westernformsmight be used,for instance, in certaindomainsof public
c o m m u n i c a t i osnu,c ha s p u b l i cn o t i c e ss,i t e so f p u b l i ct r a n s p o r t h , e p r e s sa, d v e r t i s i nagn, d
t h ev i s u aal r t s ,a sw e l la s i n s o m e w h amt o r e ' p r i v a t e , d o m a i innst ,h e h o m ea, n di n m a r k e t s
a n ds h o p s , f ol rn s t a n c e . O f ttehner e l a t i o ni s h i e r a r c h i c a l , w o i t nh ef o r mo v e r l a i od n a n o t h e r
( s e eS c o l l o na n dS c o l l o n2, 0 0 3 ;l ( r e s s2, o o 3 ) ,a n do f t e n- a s i n a d v e r t i s i n g ,
f o r i n s t a n c-e
the two are mutuallytransformed andfused.WhereWesternvisualcommunication begins
t0 exertpressure on localforms,therearetransitionalstagesin whichthe formsof thetwo
c u l t u r e sm i x i n p a r t i c u l aw r a y s I. n l o o k i n ga t a d v e r t i s e m e ni nt sE n g l i s h - l a n g u amgaeg a -
zinesfrom the Philippines, for instance, we were struck by the way in which entirely
conventional Westerniconographical elements wereintegratedinto designsfollowingthe
Introduction

r u l e so f a l o c a lv i s u asl e m i o t i cI n. a d v e r t i s e m e notnst h e I V I T R i n H o n gK o n gs, o m ea d v e r t -


i s e m e n tcso n f o r mt o t h e ' E a s t e r n ' d i r e c t i o n a loi ttyh,e r st o t h e W e s t e r ny,e to t h e r sm i x t h e
two. As with the Filipinoadvertisements, discourses and iconography can be 'Western',
mixedin variouswayswith thoseof the 'East', whilecolourschemes can/at the sametime,
be distinctlynon-Western. The situationthereis in any casecomplicated (as it is, differ-
ently,in Japan)by the fact that directionalityin the writing systemhas becomecompli-
catedin severalways:by the adoption,in certaincontexts, of 'Western'directionality and
t h e R o m a na l p h a b eat l o n g s i dteh e c o n t i n u e u d s eo f t h e m o r et r a d i t i o n adl i r e c t i o n a l i t i e s
andformsof writing.And as economic(andnow oftencultural)poweris re-weighted, the
trend can go in both or more directions: the influence of Asianforms of visualdesignis
b e c o m i nm g o r ea n d m o r ep r e s e nitn t h e ' W e s t ' S . u p e r i m p o s oe nd a l l t h i sa r et h e i n c r e a s -
i n g l yp r o m i n e ndt i a s p o r i c o m m u n i t i e- s o f G r e e k sL, e b a n e s T e ,u r k so, f m a n yg r o u p so f
t h e I n d i a ns u b c o n t i n e not f, n e w a n d o l d e r C h i n e s e c o m m u n i t i e(sf o r i n s t a n c eH, o n g
l(ongChinese aroundthe PacificRim) whichseemingly - affectonlythe membersof this
diaspora, andyet in realityare havingdeepinfluences well beyondthem.
W i t h i nE u r o p ei n , c r e a s i nrge g i o n a l i tcyo u n t e r b a l a n ci ne cs r e a s i nggl o b a l i z a t i oSno. l o n g
asthe European nationsand regionsstill retaindifferentwaysof life anda differentethos,
t h e y w i l l u s et h e ' g r a m m a ro f v i s u a ld e s i g n ' d i s t i n c t lIyt . i s e a s yf,o r e x a m p l et ,o f i n d
examplesof the contrastinguse of the left and right in the compositionof pagesand
imagesin the Britishmedia.It is harderto findsuchexamples in,for instance, the Greekor
the Spanishor the Italian media.as studentsfrom thesecountrieshaveassuredus and
demonstrated in their worl<- aftertryingto do the assignments we hadsetthemat home
d u r i n gt h e i r h o l i d a y si n. t h e c o u r s eo f o u r b o o kw e w i l l g i v es o m ee x a m p l eosf t h i s ,f o r
instancein connection with newspaper layoutin differentEuropean countries. However,we
a r e n o t a b l et o d o m o r et h a nt o u c ho n t h e s u b j e c ta; n dt h e i s s u eo f d i f f e r e n t ' d i a l e c t s ' a n d
' i n f f e c t i o n s ' n e et d
o sb ee x p l o r e md o r ef u l l y i n t h ef u t u r e .
In any case,the unity of languages is a socialconstruct,a productof theoryand of
s o c i aa l n dc u l t u r a lh i s t o r i e sW. h e nt h e b o r d e r so f ( a ) l a n g u a gaer e n o t p o l i c e db y a c a d -
emies,and when languages are not homogenized by educationsystemsand massmedia,
p e o p l eq u i t ef r e e l yc o m b i n e l e m e n tfsr o m t h e l a n g u a g et h s e yk n o wt o m a k et h e m s e l v e s
understood M.i x e dl a n g u a g e(s' p i d g i n s 'd) e v e l o pi n t h i sw a y ,a n d i n t i m e c a n b e c o m e the
l a n g u a g eo f n e w g e n e r a t i o n(s' c r e o l e s ' )V. i s u a lc o m m u n i c a t i o n o , t subject o such
policing,has developed more freely than language, but there has nevertheless beena
d o m i n a nlta n g u a g e , ' s p o k e n ' ad ne dv e l o p ei n d c e n t r e os f h i g hc u l t u r ea, l o n g s i dlee s sh i g h l y
v a l u e dr e g i o n aal n ds o c i a vl a r i a n t s( e . 9 . ' f o l ka r t ' ) . T h ed o m i n a nvt i s u a l a n g u a gies n o w
c o n t r o l l ebdyt h eg l o b a cl u l t u r a l / t e c h n o l o gei cmapl i r e o s f t h e m a s sm e d i aw, h i c hd i s s e m i n -
ate the examples set by exemplary designers and,Ihrough the spreadof imagebanksand
c o m p u t e r - i m a g i tnegc h n o l o g ye,x e r ta ' n o r m a l i z i n gr'a t h e rt h a n e x p l i c i t l y' n o r m a t i v e '
i n f f u e n coen v i s u a cl o m m u n i c a t i oanc r o s st h ew o r l d .M u c ha s i t i s t h e p r i m a r ya i m o f t h i s
b o o kt o d e s c r i bteh e c u r r e n st t a t eo f t h e ' g r a m m a ro f v i s u a d l e s i g nw ' , e w i l l a l s od i s c u s s
t h e b r o a dh i s t o r i c a ls, o c i a la n d c u l t u r a lc o n d i t i o ntsh a t m a k ea n d r e m a k et h e v i s u a l
'language'.
I ntroduction

A S O C I A LS E M I O T I CT H E O R YO F R E P R E S E N T A T I O N

0ur work on visualrepresentation is setwithinthetheoretical frameworkof 'socialsemiot-


ics'.It is importantthereforeto placeit in the contextof the way'semiotics'has developed
during,roughly, the past75 years.In Europe, threeschoolsof semiotics appliedideasfrom
t h e d o m a i no f l i n g u i s t i ctso n o n - l i n g u i s tm i co d e so f c o m m u n i c a t i oTnh. e f i r s t w a s t h e
P r a g u eS c h o ool f t h e 1 9 3 0 sa n de a r l y1 9 4 0 s I. t d e v e l o p et h d ew o r ko f R u s s i a F normalists
by providing it with a linguistib c asisN . o t i o n ss u c ha s ' f o r e g r o u n d i nw g ,e r ea p p l i e dt o
l a n g u a g(ee . 9t.h e ' f o r e g r o u n d i nfgo'r,a r t i s t i cp u r p o s eosf/ p h o n o l o g i coarl s y n t a c t ifco r m s
t h r o u g h' d e v i a t i o n ' f r o m standard f o r m s f, o r a r t i s t i cp u r p o s e sa)s w e l l a s t o t h e s t u d yo f
a r t ( M u k a r o v s k yt )h,e a t r e( H o n z l )c, i n e m a( J a k o b s o na)n dc o s t u m e ( B o g a t y r e vE ) .a c ho f
t h e s es e m i o t iscy s t e mcso u l df u l f i l t h es a m ec o m m u n i c a t i fvuen c t i o n(st h e ' r e f e r e n t i aaln, d
t h e ' p o e t i c ' f u n c t i o n sT)h. es e c o n d w a st h e p a r i sS c h o ool f t h e 1 9 6 0 sa n d 1 9 7 0 s w , hich
a p p l i e di d e a sf r o m d e s a u s s u r e a n d o t h e r I i n g u i s ttso p a i n t i n g( S c h e f e r )p, h o t o g r a p h y
( B a r t h e sL, i n d e l < e n fsa) s, h i o n( B a r t h e s )c, i n e m a( M e t z ) ,m u s i c( N a t t i e z )c, o m i cs t r i p s
( F r e s n a u l t - D e r u e lel e t c) .,T h ei d e a sd e v e l o p ebdy t h i s S c h o oal r e s t i l l t a u g h ti n c o u n t l e s s
coursesof mediastudies, art anddesign, etc.,oftenunderthe heading,semiology,, despite
the fact that they are at the sametime regardedas havingbeenovertakenby post-
structuralism E .v e r y w h e rset u d e n tasr e l e a r n i n ag b o u t ' l a n g u e ' a n, d p a r o l e ,t ;h e ' s i g n i f i e r ,
'signified',''arbitrary' ,motivated, ,icons,, ,indexes,
andthe and signs; and'symbols,(these
t e r m sc o m ef r o m t h e w o r k o f t h e A m e r i c a np h i l o s o p h e a rn ds e m i o t i c i aCnh a r l e sS a n d e r s
Peirce,but are often incorporated in the frameworkof 'semiology,), and so on. Generally
this happens withoutstudentsbeinggivena senseof, or accessto, alternativetheoriesof
s e m i o t i c(so r o f l i n g u i s t i c sW ) .ew i l l c o m p a r e a n dc o n t r a stth i s k i n do f s e m i o t i cws i t h o u r
o w na p p r o a c hi n, t h i s i n t r o d u c t i oans w e l la s e l s e w h e ri en t h e b o o k .T h i st h i r d ,s t i l lf f e d g -
l i n g ,m o v e m e ni tn w h i c hi n s i g h t sf r o m l i n g u i s t i chsa v eb e e na p p l i e dt o o t h e rm o d e so f
representation hastwo sources/ bothdrawingon the ideasof MichaelHalliday, onegrow-
i n g o u t o f t h e ' c r i t i c a l L i n g u i s t i c s ' oaf g r o u po f p e o p l ew o r k i n gi n t h e 1 9 7 0 sa t t h e
U n i v e r s i toyf E a s tA n g l i a l,e a d i n g t o t h e o u t l i n eo f a t h e o r yt h a t m i g h te n c o m p a sost h e r
semioticmodes(Hodgeand l(ress),the other,in the later 1990s,as a development of
H a l l i d a y asny s t e m i c - f u n c t i olni nagl u i s t i cbsy a n u m b e or f s c h o l a r isn A u s t r a l i ai n , semiot-
i c a l l y o r i e n t e ds t u d i e so f l i t e r a t u r e( T h r e a d g o l dT,h i b a u l t )v, i s u a ls e m i o t i c s( 0 , T o o l e ,
o u r s e l v e sa )n dm u s i c( v a nL e e u w e n ) .
T h ek e yn o t i o ni n a n ys e m i o t i ciss t h e ' s i g n ' 0 . u r b o o ki s a b o u ts i g n s-
o r ,a s w e w o u r o
r a t h e rp u t i t , a b o u ts i g n - m a k i nW g .e w i l l b e d i s c u s s i nf o g r m s( ' s i g n i f i e r ss' )u c ha s c o l o u r ,
perspective and line,as well as the way in whichtheseformsare usedto realizemeanings
( ' s i g n i f i e d si 'n) t h e m a k i n go f s i g n sB . u t o u r c o n c e p t i oonf t h es i g nd i f f e r ss o m e w h af rt o m
that of 'semiology', andwe wishthereforeto comparethe two viewsexplicitly. In doingso
we usethe term 'semiology'to referto the way in whichthe ParisSchoolsemioticsis
g e n e r a l ltya u g h ti n t h e A n g l o - S a x owno r l d t, h r o u g ht h e m e d i a t i o on f i n f f u e n t i at el x t b o o k s
s u c ha s t h e s e r i e so f m e d i as t u d i e st e x t b o o k e s d i t e db y J o h n F i s k e( F i s k ea n d H a r t l e y ,
r 9 7 9 ; D y e r , r 9 B 2F; i s l < e , r 9 B 2 H;a r i l e y , 1 9 B 20;' s u l l i v a ne t a \ . , 1 9 9 3 )I.n d o i n gt h i sw e d o
not seekto repudiatethosewho went beforeus.We seea continuitybetweentheir work
I ntroduction

andours,as shouldbeclearfrom our maintitle, ReadingImages, whichechoes that of the


first volumein Fiske'sseries,ReadingTelevision (Fiskeand Hartley,I979).
W e w o u l dl i k et o b e g i nw i t h a n e x a m p l e o f w h a tw e u n d e r s t a nbdy ' s i g n - m a k i n gT' h .e
drawingin figure 0.1 was made by a three-year-old boy.Sitting on his father'slap,he
t a l k e da b o u t h ed r a w i n ga s h ew a sd o i n gi t : ' D o y o uw a n tt o w a t c hm e ?I ' l l m a k ea c a r . . .
gottwowheels...andtwowheelsattheback...andtwowheelshere...that'safunny
w h e e l . . . . ' W h e nh e h a df i n i s h e dh ,e s a i d , ' T h i iss a c a r . ' T h i sw a st h e f i r s tt i m eh e h a d
nameda drawing,andat first the namewaspuzzling. Howwasthis a car?0f coursehe had
p r o v i d e tdh e k e y h i m s e l f : ' H e r e 'as w h e e l . A ' c a r ,f o r h i m ,w a s d e f i n e db y t h e c r i t e r i a l
characteristic of 'havingwheels',and his representation focusedon this aspect.What he
r e p r e s e n t ewda s ,i n f a c t , ' w h e e l n e sW t o c h o o s feo r t h r e e -
s ' h. e e l sa r ea p l a u s i b lcer i t e r i o n
year-olds, and the wheel'saction/on toy cars as on reai cars/is a readilynoticedand
describable feature.In otherwords,thisthree-year-old's interestin carswas,for him,most
p l a u s i b lcyo n d e n s ei nd t oa n de x p r e s s eads a n i n t e r e sitn w h e e l sW . h e e l si ,n t u r n ,a r e m o s t
plausiblyrepresented by circles,both because of their visualappearance and because of
t h e c i r c u l a rm o t i o no f t h e h a n di n d r a w i n g / r e p r e s e ntthi negw h e e l ' a s c t i o no f ' g o i n gr o u n d
a n dr o u n d ' .
To gatherthis up for a moment,weseerepresentation as a process in whichthe mal<ers
of signs, whether child or adult, seek to make a representation of some objector entity,
w h e t h e rp h y s i c aol r s e m i o t i ca, n d i n w h i c ht h e i r i n t e r e s itn t h e o b j e c t a , t t h e p o i n to f
m a k i n gt h e r e p r e s e n t a t i i
o sn a, c o m p l e o
x n e a
, r i s i n o
g u t o f t h e c u l t u r a s
l , o c i aal n dp s y c h o -
logical h i s t o r yo f t h e s i g n - m a k e ar n
, d f o c u s e db y t h e s p e c i f i c o n t e x i
t n w h i c ht h e s i g n -
maker p r o d u c et s
h e s i g n T
. h a t ' i n t e r e s its' t h e s o u r c e
o f t h e s e l e c t i o on f w h a ti s s e e na st h e
criterial aspect of the object, and this criterial aspect is then regarded as adequately
given words, it is never the 'wholeobject'
representative of the object in a context. In other
but onlyeverits criterialaspects whichare represented.
Thesecriterial aspects are represented in what seemsto the sign-maker, at the moment
of sign-mal<in t hge, m o s ta p t a n d p l a u s i b lfea s h i o na , n d t h e m o s ta p t a n d p l a u s i b lree p r e -
s e n t a t i o n aml o d e ( e . g d. r a w i n g L, e g ob l o c k s p
, a i n t i n gs ,p e e c h S
) . i g n - m a k e trhsu s ' h a v e ' a

O
co g
O child
Fig 0.1 Drawingby a three-year-old
Introduction

meaning/ the signified, whichtheywishto express, andthenexpressit throughthe semiotic


mode(s)that make(s)availablethe subjectively felt, mostplausible, mostapt form,as the
s i g n i f i eTr .h i sm e a n s t h aitn s o c i asl e m i o t i c s t hsei g ni s n o t t h ep r e - e x i s t i nc g o n j u n c t i oonf a
signifieranda signified, a ready-made signto be recognized, chosenandusedas it is,in the
waythat signsare usuallythoughtto be 'availablefor use'in ,semiology,. Ratherwe focus
on the processof sign-making, in which the signifier(the form) and the signified(the
meaning)are relativelyindependent of eachotheruntil they are broughttogetherby the
sign-maker in a newlymadesign.To put it in a differentway,usingthe examplejust above,
the processof sign-makingis the processof the constitutionof a sign/metaphor in two
s t e p s : ' ac a r i s ( m o s tl i k e )w h e e l s ' a n d ' w h e ealrse ( m o s tl i k e )c i r c l e s , .
Puttingit in our terms:the sign-maker's interestat this momentof sign-making has
s e t t l e do n ' w h e e l n e s s ' atsh e c r i t e r i a lf e a t u r eo f ' c a r ' . H e c o n s t r u c t sb,y a p r o c e s o sf
analogy, two metaphors/signs: first,the signified 'wheel'is aptlyrepresented by the signifier
' c i r c l e ' t om a k et h e m o t i v a t e d
s i g n ' w h e e ls' ;e c o n dt h , e s i g n i f i e d ' c a ri s' a p t l yr e p r e s e n t e d
by the signifier'many wheels'to makethe motivatedsign 'car'. The resultingsign,the
drawingglossed'this is a car', is thusa motivatedsignin that eachconjunction of signifier
and signifiedis an apt, motivatedconjunctionof the form which best reoresents that
w h i c hi s t o b e m e a n tT. h i ss i g ni s t h u st h e r e s u l to f a d o u b l em e t a p h o r ipcr o c e sisn w h i c h
analogyis the constitutive principle. Analogy,inturn,is a process of classification: x is like
y ( i n c r i t e r i aw . h i c hm e t a p h o r(sa n d , ' b e h i n d ' t hmee t a p h o rw
l a y s )W s ,h i c hc l a s s i f i c a t i o n s )
carrythe day and passinto the semioticsystemas conventional, and thenas naturalized,
andthenas'natural',neutralclassifications, is governed by socialrelationsof power.Like
adults,childrenare engagedin the construction of metaphors. Unlikeadults,they are,on
the one hand,lessconstrictedby cultureand its already-existing and usuallyinvisible
metaphors, but, on the otherhand,usuallyin a positionof lesspower,so that their meta-
phorsare lesslikelyto carrythe day.
It followsthat we seesignsas motivated- not as arbitrary-conjunctionsof signifiers
( f o r m s )a n ds i g n i f i e d(sm e a n i n g sI)n. ' s e m i o l o g y ' m o t i v a t i iosnu s u a l l yn o t r e l a t e dt o
the
act of sign-making as it is in our approach,but definedin termsof an intrinsicrelation
b e t w e etnh e s i g n i f i ear n dt h e s i g n i f i e dI t. i s h e r et h a t p e i r c e ' s ' i c o n ' , , i n d eaxn,d ' s y m b o l ,
m a k et h e i r a p p e a r a n cien/c o r p o r a t ei d n t o ' s e m i o l o g y ' ian w a y w h i c hi n f a c t c o n t r a d i c t s
s o m eo f t h e k e yi d e a si n P e i r c e 'sse m i o t i c sT.h e ' i c o n 'i s t h e s i g ni n w h i c h, t h es i g n i f i e r -
signifiedrelationshipis one of resemblance, likeness'(Dyer,r9g2: r24) - i.e. objective
likeness, ratherthananalogymotivatedby'interest',establishes the relation.The,index,is
the sign in which'thereis a sequential or causalrelationbetweensignifierand signified,
(Dyer,1982: 125);that is,a logicof inference, ratherthananalogymotivatedby ,interest,.
Thethird term in the triad,'symbol',by contrast,is relatedto signproduction, as it'rests
on convention, or "contract" ' (Dyer,r9g2:125), but this veryfact makesit ,arbitrary,,
'unmotivated'/ a caseof meaningby decreeratherthan of activesign-making.
I n o u r v i e ws i g n sa r e n e v e a r r b i t r a r ya, n d ' m o t i v a t i o n ' s h o ubl e
df o r m u l a t e idn r e l a t i o n
to the sign-maker andthe contextin whichthe signis produced, and not in isolationfrom
the act of producing analogies andclassifications. Sign-makers usetheformstheyconsider
apt for the expression of their meaning, in anymediumin whichtheycanmakesigns.when
I ntroduction

childrentreat a cardboardbox as a pirate ship,they do so becausethey considerthe


materialform (box)an apt mediumfor the expression of the meaningthey havein mind
(pirateship),and because of their conception of the criterialaspectsof pirateships(con-
t a i n m e n tm , o b i l i t ye, t c . ) .L a n g u a g ies n o e x c e p t i otno t h i s p r o c e s o s f sign-making A.l l
l i n g u i s t ifco r m i s u s e di n a m e d i a t e dn,o n - a r b i t r a rm y a n n e ri n t h e e x p r e s s i oonf m e a n i n g .
For childrenin their early,pre-school yearsthereis bothmoreand lessfreedomof expres-
sion:more,because theyhavenot yet learnedto confinethe makingof signsto the cultur-
ally and sociallyfacilitatedmedia,and because they are unawareof established conven-
tionsand relativelyunconstrained in the makingof signs;less,because they do not have
suchrich culturalsemioticresources availableas do adults.So whena three-year-old boy,
labouring to climba steephill,says,'Thisis a heavyhill',heis constrained by not havingthe
word'steep'asan availablesemioticresource. Thesameis the casewith the resources of
syntacticandtextualforms.
' H e a v y 'i,n ' h e a v yh i l l ' ,i s ,h o w e v ear ,m o t i v a t esdi g n t: h ec h i l dh a sf o c u s e o dnparticular
a s p e c tos f c l i m b i n ga h i l l ( i t t a k e sa l o t o f e n e r g yi;t i s e x h a u s t i n ga)n d u s e sa n a v a i l a b l e
form whichhe seesas apt for the expression of thesemeanings. Theadultwho correctsby
offering'steep'('Yes,it's a verysteephill') is,from the child'spointof view,not so much
offeringan alternative as a synonym for the precisemeaning whichhe hadgivento'heavy'
in that context.Boththe childand the parentmakeuseof 'what is available';it happens
that differentthingsare available to each.But to concentrate on this is to missthe central
a s p e cot f s i g n - m a k i negs, p e c i a l tl yh a t o f c h i l d r e n . ' A v a i l a b i l i tsyn' o t t h e i s s u eC. h i l d r e n ,
likeadults,maketheir own resources of representation. Theyare not'acquired',but made
by the individual sign-maker.
I n ' s e m i o l o g yc'o, u n t l e ssst u d e n tasc r o s tsh ew o r l da r e i n t r o d u c et d o t h et e r m s ' l a n g u e '
and'parole',with 'langue'explained, for instance, as'the abstractpotentialof a language
system. . . the sharedlanguagesystemout of which we make our particular,possibly
unique, statements' (O'sullivanet a|.,1983:t27) or,in our terms,as a systemof available
f o r m sa l r e a d yc o u p l e tdo a v a i l a b lm e e a n i n gas n , dw i t h ' p a r o l ed' e f i n e a ds:

of the potentialof langue.. . .


that is a particularrealization
an individualutterance
By extensionwe can arguethat the total systemof televisionand film conventions
andpractices constitutesalangue, andthe waytheyare realizedin eachprogramme
or filma parole.
( 0 ' S u l i v a ne t a l . ,l 9 B 3 i I 2 7 )

W e c l e a r l yw o r kw i t h s i m i l a rn o t i o n sw, i t h ' a v a i l a b lfeo r m s 'a n d' a v a i l a b lcel a s s i f i c a t i o n s '


( ' l a n g u e 'a) n di n d i v i d u aalc t so f s i g n - m a k i n(g' p a r o l e ' )a,n dw ea g r e et h a ts u c hn o t i o n cs a n
usefullybe extended to semioticmodesotherthan language. But for usthe ideaof'poten-
tial' (whatyoucan meanand howyoucan'say' it, in whatevermedium)is not limitedby a
s y s t e mo f ' a v a i l a b l me e a n i n g s ' c o u p lw , n dw e w o u l dl i k et o u s ea
e idt h ' a v a i l a b lfeo r m s ' a
slightlylessabstractformulation: a semiotic'potential'is definedby the semioticresources
availableto a specificindividualin a specificsocialcontext.0f course,a descriptionof
semioticpotentialcan amalgamate the resources of manyspeakers and manycontexts.
10 Introduction

B u t t h e r e s u l t i n 'gl a n g u e(' t h e l a n g u eo f ' E n g l i s h ' o ro f ' w e s t e r nv i s u a d l e s i g n ,i)s i n t h e


end an artefactof analysis. What exists,and is thereforemorecrucialfor understanding
representation andcommunication, arethe resources available to realpeoplein realsocial
c o n t e x t sA.n d i f w e c o n s t r u cat ' l a n g u e 'a, m e a n i n g p o t e n t i aflo r ' W e s t e r nv i s u a d
l esign,,
then it is no moreand no lessthan a tool which can serveto describea varietyof sign-
makingpractices, within boundaries drawnby the analyst.It followsthat we would not
d r a wt h e l i n eb e t w e e n ' l a n g u e ' a n d ' p a r o l es' ahsa r p l ya s i t i s u s u a l l yd o n e .D e s c r i b i nag
' l a n g u e ' i sd e s c r i b i nag
s p e c i f i cs e t o f s e m i o t i cr e s o u r c easv a i l a b l e for communicative
a c t i o nt o a s p e c i f isco c i agl r o u p .
Hereare someantecedents of the car drawing.Figure0.2 is a drawingmadeby the
s a m ec h i l ds, o m et e n m o n t h se a r l i e rI .t s c i r c u l a rm o t i o ni s e x p r e s s i voef t h e c h i l d ' se x u b e r -
a n t ,e n t h u s i a s tai cn de n e r g e t iacc t i o n si n m a k i n gt h e d r a w i n gI.n f i g u r e0 . 3 ,m a d ea b o u t
three monthslater,the circular motion has becomemore regular.The exuberance and
e n e r g ya r es t i l l t h e r eb, u t t h ed r a w i n gh a sa c q u i r e m d o r er e g u l a r i t ym, o r ei n t e r e si tn s h a p e :
' c i r c u l a rm o t i o n 'i s b e g i n n i n g
t o t u r n i n t o' c i r c l e ' I. n o t h e rw o r d st,h e m e a n i n gosf f i g u r e

O Fig 0.2 Drawingby a two-year-old


chitd
I ntroduction II

O Fig0,3 Drawingbyatwo-yeaholdchild

0.2 persistin figure0.3,transformed, yetwith significant figure0.3 gathersup,


continuity:
so to speak,the meanings of figure0.2, andthentransformsandextendsthem.
showsa seriesof circles,eachdrawnon a separate
Figure0.4,finally, sheet,onecircleto
eachsheet.The movementfrom figure 0.2 to figure 0.4 is clear enough,as is the con-
ceptualandtransformative work doneby the childovera periodof fourteenmonths(figure
0.4 datesfrom the sameperiodas figure0.1). Together the drawingsshowhowthe child
developedthe representational resources available to him,andwhy circlesseemed suchan
apt choiceto him:the expressive, energetic physicality of the motionof figure0.2 persisted
as the childdeveloped this representational resource/ sothat the circularmotionremained
part of the meaningof circle/wheel. But something was addedas well:the transformation
of representational resources wasalsoa transformation of the child'ssubjectivity, from the
emotional, physicalandexpressive disposition expressed in the act of representing'circular
motion'tothe moreconceptual andcognitivedisposition expressed in the act of represent-
i n ga ' c a r ' .
, k et h e i r ' o w n ' r e p r e s e n t a t i orneaslo u r c easn, dd o s oa s
C h i l d r e nl i,k ea l l s i g n - m a k e rms a
12 . Introduction

O Fig 0.4 Drawingby a three-year.old


child

part of a constantproductionof signs,in which previouslyproducedsignsbecomethe


signifier-material to be transformedinto new signs.This processrestson lhe interestof
sign-makers. Thistransformative, productivestancetowardssign-making is at the same
time a transformation of the sign-makers'subjectivity - a notionfor whichtherewas little
place in a 'semiology'which describedthe relationbetweensignifiersand signifiedsas
restingon inference or objectiveresemblance, or on the decrees of the social'contract'.
We haveusedchildren'sdrawingsas our examplebecause we believethat the produc-
p r o v i d etsh e b e s tm o d efl o r t h i n k i n ga b o u ts i g n - m a k i nIgt .a p p l i e s
t i o n o f s i g n sb y c h i l d r e n
also to fully socializedand acculturatedhumans,with the exceptionof the effectsof
'convention'. As maturemembersof a culturewe haveavailablethe culturallyproduced
semioticresources of our societies, andareawareof the conventions andconstraints which
are sociallyimposedon our making of signs.However, as we havesuggested, in our
approachadult sign-makers, too,are guidedby interest,by that complexcondensation of
c u l t u r a la n d s o c i a lh i s t o r i e as n d o f a w a r e n e sosf p r e s e nct o n t i n g e n c i e s . ' M a t u r e ' s i g n -
makersproducesignsout ol that interest,alwaysas transformations of existingsemiotic
materials, thereforealwaysin somewaynewlymade,andalwaysas motivatedconjunctions
of meaningandform.Theeffectof convention is to placethe pressure of constantlimita-
tions of conformityon sign-making; that is, the way signifiershavebeencombinedwith
signifieds in the historyof the culture,actsas a constantlypresentconstrainton how far
one might movein combiningsignifierswith signifieds. Convention doesnot negatenew
making;it attemptsto limit andconstrainthe semioticscopeof the combinations.
v i s - d - v i s ' E u r o p e a n ' s e m i o lwohgeyr: ed e S a u s s u rhea d( b e e n
T h i s , t h e ni s, o u r p o s i t i o n
assumed to have)saidthatthe relationof signifierandsignifiedin the signis arbitraryand
conventional, we wouldsaythat the relationis alwaysmotivatedand conventional. Where
he had seemingly placedsemioticweightand powerwith the social,we wishto assertthe
effectsof the transformativeroleof individual
agents, yetalsothe constantpresence of the
social:in the historicalshapingof the resources,
in the individualagent'ssocialhistory,in
Introduction

the recognitionof presentconventions,


in the effectof the environment in whichrepresen-
tationandcommunication happen.Yet it is the transformative along
actionof individuals,
the contoursof socialgivens,
whichconstantlyreshapes and mal<es
the resources, possible
the self-making of socialsubjects.
0ne of the now taken-for-granted insightsof sociallyorientedtheoriesof languageis
the variationof language with the variationof socialcontext.Theaccounts ofthis variation
differ,rangingfrom correlation('language form x relatesto socialcontexty') to determin-
a t i o n( ' l a n g u a gfeo r m x i s p r o d u c e b dy s o c i a al c t o r sy o r i n s o c i a cl o n t e x yt ' ) . A s o c i a l
s e m i o t i ac p p r o a ctha k e st h e l a t t e rv i e wa, l o n gt h ef o l l o w i n gl i n e s .

( 1 ) C o m m u n i c a t i or n e q u i r e st h a t p a r t i c i p a n tms a k et h e i r m e s s a g em s a x i m a l l yu n d e r -
standablein a particularcontext.Theythereforechooseforms of expression which
t h e yb e l i e v e t o b e m a x i m a l l tyr a n s p a r e nt o t o t h e rp a r t i c i p a n t s . 0tnh e o t h e rh a n d ,
communication takesnlacein socialstructures whichare inevitably markedby power
differences, andthis affectshoweachparticipantunderstands the notionof'maximal
understanding'. Participantsin positionsof powercan forceother participantsinto
greatereffortsof interpretation, andtheir notionof'maximalunderstanding' is there-
fore differentfrom that of participantswho do their bestto producemessages that
w i l l r e q u i r ea m i n i m a le f f o r t o f i n t e r p r e t a t i oonr, f r o m t h a t o f p a r t i c i p a n tw s ho,
throughlack of commandof the representational system,producemessages that are
, a r n e r os f a f o r e i g nl a n g u a g eT) .h eo t h e rp a r t i c i -
h a r d e rt o i n t e r p r e(te . g .c h i l d r e nl e
pantsmaytheneithermakethe effort requiredto interpretthesemessages or refuse
to do so,whetherin a schoolor in a railwaystationin a foreigncountry.
( 2 ) Representation requiresthat sign-makers chooseforms for the expression of what
t h e yh a v ei n m i n df,o r m sw h i c ht h e ys e ea s m o s ta p t a n dp l a u s i b lien t h eg i v e nc o n t e x t .
The examplesaboveinstantiatethis: circlesto standfor wheels,and wheelsto stand
for cars; heavyto stand for significanteffort, and significanteffort to stand for
climbinga steeps/ope.Speakersof a foreign languageuseexactlythe samestrategy.
Theychoosethe nearest,most plausibleform they knowfor the expression of what
t h e yh a v ei n m i n d T . h er e q u i r e m e not sf c o m m u n i c a t i oanr e n o d i f f e r e nitn m o r eu s u a l
circumstances, they are simply lessapparent.The interestof sign-makers, at the
momentof makingthe sign,leadsthemto choosean aspector bundleof aspects of the
objectto be represented as beingcriterial,at that moment,for representing whatthey
want to represent, and then choosethe most plausible, the most apt form for lts
representation. Thisappliesalsoto the interestof the socialinstitutions withinwhich
messages are produced, and thereit takesthe form of the (historiesof) conventions
andconstraints.

APPLICATIONS

In the previous
sectionwe havefocusedon thetheoreticalbackgroundof our work,but our
aimsare not just theoretical. andpractical.We seekto developa
Theyare alsodescriptive
14 . Introduction

descriptive frameworkthat can be usedas a tool for visualanalysis. Sucha tool will have
its usefor practicalas well as analyticaland criticalpurposes. To givesomeexamples of
the former,educationalists everywhere havebecomeawareof the increasing roleof visual
c o m m u n i c a t i oi nnl e a r n i n m g a t e r i a losf v a r i o u ks i n d sa, n dt h e ya r ea s k i n gt h e m s e l v e wsh a t
kind of maps,charts,diagrams,picturesand forms of layoutwill be most effectivefor
learning. To answerthis questionthey needa language for speakingaboutthe formsand
m e a n i n gosf t h e s ev i s u a l e a r n i n m g a t e r i a l sW. i t h i nt h e m e d i av, i s u adl e s i g ni s l e s sa n dl e s s
the provinceof specialists who had generally seenlittle needfor methodicaland analytic-
ally explicitapproaches, and had reliedinsteadon creativesensibilities honedthrough
experience. But wheremediaformsare relativelyrecentlyintroduced - as is the case.for
example, with advertising in EasternEuropeandpartsof Asia- thereis no suchresistance
to combiningsystematic analysisand practice.And with the advanceof easyto usesoft-
warefor desktoppublishing, the productionof diagramsand charts,imagemanipulation,
etc.,visualdesignbecomeslessof a specialistactivity,somethingmany peoplewill do
alongside otheractivities. Thishasalreadyledto rapidgrowthin the numberof coursesin
t h i s a r e a- a n d d e s i g n i nsgu c hc o u r s e rse q u i r e m s o r eo f a n a n a l y t i c agl r a s po f p r i n c i p l e s
than learningon the job by exampleandosmosis. Last,and maybeat bottomat the root of
m u c ho f t h i sc h a n g ei s, ' g l o b a l i z a t i o n w'h, i c h- m a y b en e a r l yp a r a d o x i c a -l l yd e m a n dtsh a t
the culturalspecificities of semiotic, social,epistemological andrhetoricaleffectsof visual
communication mustbe understood everywhere, sincesemioticentitiesf rom anywnere now
appearandare'consumed' everywhere.
A n a l y s i n vgi s u a lc o m m u n i c a t i oi sn, o r s h o u l db e ,a n i m p o r t a n p t art of the 'critical'
d i s c i p l i n eAs l.t h o u g hi n t h i s b o o kw e f o c u so n d i s p l a y i ntgh e r e g u l a r i t i eosf v i s u a lc o m -
munication, ratherthan its ('interested', i.e.political/ideological) uses,we seeimagesof
whateverkind as entirelywithinthe realmof the realizations and instantiations of ideol-
ogy/as means- always- for the articulationof ideological positions. Theplainfact of the
matteris that neitherpowernor its usehasdisappeared. It hasonlybecomemoredifficult
to locateand to trace.In that contextthere is an absoluteneedin democratictermsfor
makingavailablethe meansof understanding the articulationsof poweranywhere, in any
f o r m .T h es t i l l g r o w i n ge n t e r p r i soef ' c r i t i c a ld i s c o u r saen a l y s i s ' s e e tkos s h o wh o w l a n -
guageis usedto conveypowerandstatusin contemporary socialinteraction, and howthe
apparently neutral,purelyinformative (linguistic) textswhichemergein newspaper report-
ing, government publications, socialsciencereports,and so on, realize,articulateand
disseminate 'discourses' as ideologicalpositionsjust as much as do texts which more
explicitlyeditorializeor propagandize. To do so we needto be ableto'read betweenthe
lines',in orderto get a senseof what discursive/ideological position,what'interest',may
havegivenrise to a particulartext, and maybeto glimpseat leastthe possibilityof an
alternativeview.It is this kind of readingfor which critical discourseanalysisseeksto
providethe waysand means.So far, however, criticaldiscourse analysishas mostlybeen
confinedto language, realizedas verbaltexts,or to verbalparts of textswhich also use
othersemioticmodesto realizemeaning. Weseeour bookas a contribution to a broadened
criticaldiscourse analysis, and we hopethat our examples will demonstrate its potential
for this kindof work.
Introduction . \5

0 u r e x a m p l eisn c l u d e ' t e x t - o b j e c t sm ' oaf n yI < i n d fsr,o m w o r k so f a r t t o e n t i r e l yo r d i n -


ary, banal artefactssuchas maps,charts,pagesof differentkinds,includingthose of
websites, etc.We haveincludedworl<sof art notjust because of their keyrolein the history
o f c o n v e n t i o nasn dc o n s t r a i n ths e , n c ei n t h e f o r m a t i o no f t h e ' g r a m m a ro f v i s u a d l esign',
b u ta l s ob e c a u steh e yt,o o ,a r t i c u l a t e i d e o l o g i cp aol s i t i o nosf c o m p l eax n d p o t e n t
k i n d sa, n d
t h e yt,o o ,s h o u l db e a p p r o a c h ef rdo mt h e p o i n to f v i e wo f s o c i acl r i t i q u e .
As is perhapsalreadyobviousfrom what we havesaid so far, we believethat visual
d e s i g nl i,k ea l l s e m i o t i m c o d e sf ,u l f i l st h r e em a j o rf u n c t i o n sT.o u s eH a l l i d a y 'tse r m se, v e r y
s e m i o t i fcu l f i l sb o t ha n ' i d e a t i o n a l ' f u n c t i oanf,u n c t i o no f r e p r e s e n t i n g ' t w h eo r l da r o u n d
a n di n s i d eu s ' a n da n ' i n t e r p e r s o n a l ' f u n c t i o n ,
a f u n c t i o no f e n a c t i n sgo c i a il n t e r a c t i o nass
- -
s o c i a rl e l a t i o n sA.l l m e s s a geen t i t i e s t e x t s a l s oa t t e m p t o p r e s e nat c o h e r e n t ' w o r o l df
-
t h e t e x t ' ,w h a t H a l l i d a yc a l l st h e ' t e x t u a l ' f u n c t i o n a w o r l di n w h i c ha l l t h e e l e m e n tosf
the text cohereinternally, andwhichitselfcoheres with its relevantenvironment. Whether
we engagein conversation, producean advertisement or play a pieceof music,we are
s i m u l t a n e o u scloym m u n i c a t i ndgo, i n gs o m e t h i ntgo , o r f o r ,o r w i t h ,o t h e r si n t h e h e r ea n d
nowof a socialcontext(swapping newswith a f riend;persuading the readerof a magazine
to buy something; entertaining an audience)and representing someaspectof the world
'out there',be it in concreteor abstractterms (the contentof a film we haveseen;the
q u a l i t i eos f t h e a d v e r t i s epdr o d u c ta; m o o do r m e l a n c h o sl ye n t i m e notr e x u b e r a netn e r g y
c o n v e y em d u s i c a l l ya) ,n dw e b i n dt h e s ea c t i v i t i etso g e t h eirn a c o h e r e nt e t x to r c o m m u n i -
cativeevent.The structureof our book reffectsthis. Chapters2 and 3 deal with the
patternsof representation which the 'grammarof visual design'makesavailable,and
hencewith the wayswe can encodeexperience visually.Chapters4 and 5 dealwith the
patternsof interactionwhichthe'grammarof visualdesign'mal<es available, and hence
with the thingswe can do to, or for,eachotherwith visualcommunication, and with the
'texts'which this entails.Chapter6
relationsbetweenthe makersand viewersof visual
dealswith the 'textual'f unction,with the way in whichrepresentations andcommunicative
-
a c t sc o h e r ei n t om e a n i n g f uwlh o l e sC. h a p t e7r d e a l sw i t h t h e m a t e r i a l i toy f v i s u a sl i g n s
the tools we makethem with (ink, paint,brushstrokes, etc.) and the materialswe make
them on (paper,canvas/computerScreens/ etc.);these,too, contributeto the meaningof
visualtexts.Chapter8 extends the previous chaptersintothe domainofthree-dimensional
visualsand movingimages.Again we assumethat there is somethinglil<ea Western
'grammao r f t h r e e - d i m e n s i ovniaslu adl e s i g n a ' , s e to f a v a i l a b lfeo r m sa n d m e a n i n guss e d
i n s c u l p t u raesw e l la s ,f o r i n s t a n c ei n, t h r e e - d i m e n s i osncaile n t i f im c o d e l so,r i n c h i l d r e n ' s
toys- anda Western'grammarof the movingimage'.
Wewill begin,however, by discussing someof the broaderthemeswe havetouchedon in
t h i si n t r o d u c t i o n .
I T h e s e m i o t i cl a n ds c a p e :
languaga e n d v i s u a lc o m m u n i c a t i o n

In the earlyyearsof schooling, childrenare constantly encouraged to produceimages, and


to illustratetheir writtenwork.Teachers commenton theseillustrations as muchas they
do on the writtenpart of thetext,thoughperhapsnot quitein the samevein:unlikewriting,
i l l u s t r a t i o nasr en o t ' c o r r e c t e d ' n osru b j e c t et do d e t a i l e cdr i t i c i s m( ' t h i sn e e d ms o r ew o r k , ,
'not clear','spelling!','poor
e x p r e s s i o na'n, d s o o n ) . T h e ya r e s e e na s s e l f - e x p r e s s i o n ,
ratherthan as communication - as something which the childrencan do arreaoy,spon-
taneously, ratherthan as something theyhaveto betaught.
By the time childrenare beyondtheir first two yearsof secondary schooling, illustra-
tionshavelargelydisappeared from their ownwork.Fromhereon,in a somewhatcontra-
dictorydevelopment, writing increases in importanceand frequencyand imagesbecome
s p e c i a l i z eTdh. i s i s m a d em o r ep r o b l e m a t ibcy t h e f a c t so f t h e p r e s e npt e r i o di,n w h i c h
w r i t i n ga n d i m a g ea r e i n a n i n c r e a s i n gul yn s t a b l er e l a t i o nW . e m i g h tc h a r a c t e r i zteh e
situationof saytwentyor thirty yearsago in thisway:textsproduced for the earlyyearsof
schoolingwere richly illustrated,but towardsthe later yearsof primaryschoolimages
beganto givewayto a greaterandgreaterproportionof writtentext.In as muchas images
continued, they had becomerepresentations with a technicalfunction,maps,diagramsor
photographs illustratinga particularlandformor estuaryor settlementtype in a geog-
raphytextbook,for instance. Thuschildren'sown productionof imageswas channelled in
the directionof specialization - awayfrom 'expression'and towardstechnicality. In other
words,imagesdid not disappear, but theybecamespecialized in their function.
In manyways the situationin schoolremainsmuchthe same,with two profoundly
i m p o r t a n tp r o v i s o s . 0 nt h e o n e h a n da l l s c h o o sl u b j e c t sn o w m a k em u c h m o r e u s eo f
images,particularlyso in the yearsof secondaryschooling. In many of thesesubjects,
c e r t a i n l yi n t h e m o r et e c h n i c a l / s c i e n tsi fui cb j e c tssu c ha s ( i n E n g l a n dS ) c i e n c eI n, f o r m a -
tion Technology or Geography, imageshave becomethe major meansof representing
c u r r i c u l acr o n t e n tI.n t h e m o r eh u m a n i s t iscu b j e c t-s f o r e x a m p l eH, i s t o r yE , n g l i s ha n d
R e l i g i o uSs t u d i e s- i m a g e sv a r y i n t h e i r f u n c t i o nb e t w e e ni l l u s t r a t i o nd,e c o r a t i o a nn d
information. Thistrendcontinues, and it is the casefor worksheets, in textbooksand in CD-
R 0 M s . 0 nt h e o t h e rh a n dt,h e r ei s n o t e a c h i n g or'instruction'in t h e ( n e w )r o l eo f i m a g e s
(thoughin England,in the schoolsubjectInformationTechnology, there is teachingin
desktoppublishing). Most importantly, assessment continues to be basedon writingas the
m a j o rm o d e S . t u d e n tas r e c a l l e du p o nt o m a k ed r a w l n g isn S c i e n c eG, e o g r a p hayn d H i s -
tory; but, as before,thesedrawingstend not to be the subjectof the teacher,sattention,
i u d g i n gb y t h e i r ( w r i t t e n )c o m m e n t o s n t h e c h i l d r e n 'w s o r k . I n o t h e rw o r d s ,m a t e r i a l s
providedforchildrenmakeintenserepresentational useof images;in materialsdemanded
from children- in variousformsof assessment particularly- writingremainsthe expected
a n dd o m i n a nm t ode.
Outsideschool,however, imagesplay an ever-increasing role,and not just in textsfor
The semiotic landscape . 1,7

c h i l d r e nW . h e t h e irn t h e p r i n to r e l e c t r o n im c e d i aw , h e t h eirn n e w s p a p e rmsa, g a z i n eC s ,D -


R 0 M s o r w e b s i t e sw, h e t h e ra s p u b l i cr e l a t i o n sm a t e r i a l sa, d v e r t i s e m e not sr a s i n f o r -
mationalmaterialsof all kinds,mosttextsnowinvolvea complexinterplayof writtentext,
imagesand othergraphicor soundelements, designed as coherent(oftenat the first level
v i s u a lr a t h e rt h a n v e r b a l )e n t i t i e sb y m e a n so f l a y o u t B . u t t h e s k i l lo f p r o d u c i nm g ulti-
modaltexts of this kind,howevercentralits role in contemporary society,is not taught
i n s c h o o l sT.o p u t t h i s p o i n th a r s h l yi n , t e r m so f t h i se s s e n t i anle wc o m m u n i c a t i oanb i l i t y ,
this new'visualliteracy',institutionaleducation, underthe pressure of often reactionary
politicaldemands, produces illiterates.
0f course,writing is itselfa form of visualcommunication. Indeed,and paradoxically,
t h e s i g no f t h e f u l l y l i t e r a t es o c i a lp e r s o ni s t h e a b i l i t yt o t r e a t w r i t i n gc o m p l e t e lays a
v i s u a lm e d i u m- f o r i n s t a n c eb y n o t m o v i n go n e ' sl i p s a n d n o t v o c a l i z i nw g h e no n e i s
r e a d i n gn,o te v e n' s u b v o c a l i z i n(ga's i l e n t ' s p e a k i n agl o u di n t h e h e a d 't,o b r i n go u tt h ef u l l
paradoxof this activity).Readers who movetheir lipswhenreading, who subvocalize, are
r e g a r d eadsc u l t u r a l lay n di n t elle c t u al yl t a i n t e db y h a v i n gt o t a k er e c o u r steo t h ec u l t u r a l l y
l e s sv a l u e dm o d eo f s p o l < el n a n g u a gwe h e nr e a d i n g v i s u a sl c r i p t T
. h i s ' o l d ' v i s u al li t e r a c y ,
writing,hasfor centuriesnow beenoneof the mostessentialachievements and valuesof
Westernculture,and one of the most essentialgoalsof education, so muchso that one
majorandheavilyvalue-laden distinctionmadeby Westerncultureshasbeenthat between
literate(advanced) andnon-literate (oralandprimitive)cultures.Nowonderthat the move
towardsa newliteracy, basedon imagesandvisualdesign, cancometo be seenas a threat,
a s i g no f t h e d e c l i n e o f c u l t u r ea, n dh e n c ea p a r t i c u l a r lpy o t e n st y m b oal n d r a l l y i n gp o i n t
for conservative andevenreactionary socialgroupings.
The fadingout of certainkindsof texts by and for children,then, is not a straight-
f o r w a r dd i s v a l u a t i oonf v i s u a lc o m m u n i c a t i obnu, t a v a l u a t i o nw h i c h g i v e sp a r t i c u l a r
p r o m i n e n ct oe o n ek i n do f v i s u acl o m m u n i c a t i ownr,i t i n ga, n dt o o n ek i n do f v i s u a l i t e r a c y ,
t h e ' o l d ' v i s u a l i t e r a c yO. t h e rv i s u a cl o m m u n i c a t i oi sne i t h e rt r e a t e da s t h e d o m a i no f a
v e r ys m a l le l i t eo f s p e c i a l i s tosr, d i s v a l u eads a p o s s i b lfeo r m o f e x p r e s s i of onr a r t i c u l a t e ,
r e a s o n ecdo m m u n i c a t i osne/e na sa ' c h i l d i s h ' s t a goen eg r o w so u t o f .T h i si s n o ta v a l u a t i o n
of language as suchovervisualcommunication, because evennowthe structures, meanings
a n d v a r i e t i e so f s p o k e nl a n g u a g e a r e l a r g e l ym i s u n d e r s t o oadn, d c e r t a i n l yn o t h i g h l y
valuedrn their varietyin the educationsystem(with someexceptions, suchas in formal
' d e b a t i n g 'o) r i n p u b l i c
f o r u m so f p o w e r .
To sumup:the opposition to the emergence of the visuafas a full meansof representa-
t i o n i s n o t b a s e do n a n o p p o s i t i otno t h e v i s u a al s s u c hb, u t o n a n o p p o s i t i oinn s i t u a t i o n s
whereit formsan alternative to writingand canthereforebe seenas a potentialthreatto
the presentdominance of verballiteracyamongelitegroups.
In this bookwetal<ea freshlool<at the questionof the visual.Wewantto treatformsof
communicatio en m p l o y i nigm a g e as ss e r i o u s a l ys l i n g u i s t ifco r m sh a v eb e e nW . e h a v ec o m e
to this positionbecauseof the now overwhelming evidenceof the importanceof visual
c o m m u n i c a t i oann, dt h e n o w p r o b l e m a t iacb s e n coef t h e m e a n sf o r t a l k i n ga n dt h i n k i n g
a b o u tw h a ti s a c t u a l l yc o m m u n i c a t ebdy i m a g e a s n db y v i s u a dl e s i g nI .n d o i n gs o , w eh a v e
to moveawayfrom the positionwhich RolandBarthestook in his 1964 essay'Rhetoricof
I8 Thesemiotic landscape

the image'(1977i32-51).ln thisessay(andelsewhere, as in the introduction to Elements


of Semiology;Barthes,1967d, he arguedthat the meaningof images(and of other
semioticcodes,like dress,food,etc.) is alwaysrelatedto and,in a sense,dependent on,
verbaltext. By themselves, imagesare,hethought,too 'polysemous', too opento a variety
of possiblemeanings. To arriveat a definitemeaning,languagemustcometo the rescue.
V i s u am l e a n i n igst o o i n d e f i n i t ei t;i s a ' f f o a t i n gc h a i no f s i g n i f i e d sH' .e n c eB, a r t h e s a i d , ' i n
everysocietyvarioustechniques are developed intended to fx the ffoatingchainof signi-
fiedsin sucha way asto counterthe terrorof uncertainsigns;the linguisticmessage is one
of thesetechniques' i977'.39). He distinguished betweenan image-textrelationin which
the verbaltext extendsthemeaningof the image,or viceversa,as is the case,for example,
withthespeech b a l l o o n si n c o m i cs t r i p s a, n d a n i m a g e - t e xrte l a t i o ni n w h i c ht h e v e r b a l
t e x t e l a b o r a t e s t h e i m a g e , o r v i c e v el nr st hae. f o r m e r c a s e , w h i c h h e c a l lreedl a y , n e w a n d
differentmeanings are addedto completethe message. In the lattercase,the samemean-
ingsare restatedin a different(e.g.more definiteand precise)way,as is the case,for
example, whena captionidentifiesand/orinterprets what is shownin a photograph. 0f the
t w o ,e l a b o r a t i oi nsd o m i n a n tR. e l a ys,a i dB a r t h e si s, ' m o r er a r e ' .H ed i s t i n g u i s htewdot y p e s
of elaboration, onein whichthe verbaltext comesfirst,sothat the imageformsan illustra'
tionof it, andonein whichthe imagecomesfirst,sothat the text formsa moredefiniteand
p r e c i s ree s t a t e m e n o tr ' f i x i n g ' o fi t ( a r e l a t i o nh ec a l l sa n c h o r a g e ) .
Beforeapproximately 1600 (thetransitionis,of c0urse, verygradual),Barthesargued,
' i l l u s t r a t i o n ' w adso m i n a n tI .m a g e se l a b o r a t etde x t s ,m o r es p e c i f i c a ltl hy e f o u n d i n tge x t s
o f t h e c u l t u r e- m y t h o l o gtyh, e B i b l et,h e ' h o l yw r i t ' o f t h e c u l t u r e- t e x t st,h e r e f o r ew,i t h
whichviewerscouldbeassumed to befamiliar.Thisrelation,in whichverbaltextsformeda
sourceof authorityin society, and in which imagesdisseminated the dominanttexts in a
p a r t i c u l a rm o d et o p a r t i c u l a rg r o u p sw i t h i ns o c i e t yg,r a d u a l l yc h a n g e tdo o n e i n w h i c h
nature,ratherthandiscourse, becamethe sourceof authority.In the eraof science, images,
evermorenaturalistic, beganto functionas'thebookof nature',as'windowson theworld',
as'observation', andverbaltext servedto identifyandinterpret, to'load the image,burden-
i n gi t w i t h a c u l t u r ea, m o r a l a , n imagination'.
T h i sp o s i t i o n
d o e se x p l a i ne l e m e n tosf c o m m u n i c a t i oAnn. y o n eo f t h e i m a g e - t e xrte l a -
tions Barthesdescribes may at timesbe dominant,althoughwe feelthat todaythereis a
m o v ea w a yf r o m ' a n c h o r a g eC' .o m p a r ef ,o r e x a m p l et ,h e ' c l a s s i c ' d o c u m e n t afri yl m i n
whichthe vieweris first confrontedwith'imagesof nature',then with the authoritative
'current
voiceof a narratorwho identifiesand interpretsthe images,with the modern
affairs'item,in whichthe vieweris first confrontedwith the anchorperson's verbaldis-
c o u r s ea n d ,e i t h e rs i m u l t a n e o u sol ry f o l l o w i n go n f r o m t h e v e r b a li n t r o d u c t i o w n ,i t h t h e
' i m a g e so f n a t u r e ' t h a ti l l u s t r a t ee,x e m p l i fayn d a u t h e n t i c a t h e e d i s c o u r s eB.u t B a r t h e s '
accountmissesan importantpoint:the visualcomponentof a text is an independently
organized andstructuredmessage, connected with the verbaltext,but in no way dependent
o n i t - a n ds i m i l a r l tyh e o t h e rw a ya r o u n d .
One importantdifferencebetweenthe accountwe developin this book and that of
e a r l i e rs e m i o t i c i a ni s o u r u s eo f w o r k i n l i n g u i s t i tch e o r i e sa n d d e s c r i p t i o nTsh. i si s a
difficultargumentto make,but worth makingclearly.We think that this bookwouldnot
The semiotic landscape 19

havebeenpossible withoutthe achievements of linguistics, yet we do not,in the way some


criticsof our approachhavesuggested, seeour approachas a linguisticone.So what have
we usedfrom linguistics, and how havewe usedit? And,equally,what havewe not used
from linguistics? To start with the latterquestion, we havenot importedthe theoriesand
m e t h o d o l o g io e fs l i n g u i s t i cdsi r e c t l yi n t o t h e d o m a i no f t h e v i s u a la, s h a sb e e nd o n eb y
othersworkingin thisfield.For instance, we do not makea separation of syntax,semantics
a n d p r a g m a t i cisn t h e d o m a i no f t h e v i s u a lw ; e d o n o t l o o kf o r ( t h e a n a l o g u eosf) s e n -
tences,clauses,nouns,verbs,and so on, in images.We take the viewthat languageand
v i s u a lc o m m u n i c a t i ocna n b o t h b e u s e dt o r e a l i z et h e ' s a m e ' f u n d a m e n t as ly s t e m so f
meaningthat constituteour cultures,but that eachdoesso by meansof its own specific
forms,doesso differently, and independently.
To givean example, the distinctionbetween'subjective'and 'objective'meanings has
playedan importantrole in Westerncultureeversincethe physicalsciences beganto
developin the sixteenthcentury.Thisdistinctioncan be realized(that is, givenconcrete,
m a t e r i ael x p r e s s i ohne, n c em a d ep e r c e i v a bal en dc o m m u n i c a b lw e i)t h l i n g u i s t iacs w e l l a s
visualmeans.Theterms'subjective' and 'objective'canthereforebe appliedto both:they
belongto the meaningpotentialof a cultureand its society.But the way the distinctionis
realizedin language is quitedifferentfrom the way it is realizedin images.For example, in
l a n g u a gaen i d e ac a nb e r e a l i z e sd u b j e c t i v ebl yy u s i n ga ' m e n t a lp r o c e svse r b 'l i k eb e l i e v e
in the first person(e.9. We believethat there is a grammar of image); or objectively
throughthe absence of sucha form (e.9.Thereis a grammarof image). Visualrepresen-
tation,too,can realizebothsubjectivity, throughthe presence of a perspectival angle,and
o b j e c t i v i t tyh, r o u g hi t s a b s e n c ea, p o i n tw h i c hw i l l b e d i s c u s s em d o r ef u l l y i n c h a p t e 4 r.
M e n t a lp r o c e scsl a u s eas n d n o m i n a l i z a t i oanr e u n i q u et o l a n g u a g eP.e r s p e c t i vi seu n i q u e
to images.But the kindsof meanrngexpressed are from the samebroaddomainin each
case;andthe forms,differentastheyare/weredeveloped in the sameperiod,in response to
t h e s a m ec u l t u r a lc h a n g e sB. o t h l a n g u a g ae n d v i s u a lc o m m u n i c a t i oenx p r e s m s eanings
belonging to and structuredby culturesin the onesociety;the semioticprocesses, though
n o t t h e s e m i o t i cm e a n sa/ r e b r o a d l ys i m i l a r a; n dt h i s r e s u l t si n a c o n s i d e r a bdl e g r e eo f
congruence between the two.
At the sametime,however, eachmediumhas its own possibilities and limitationsof
meaning. Not everything that can be realizedin language can alsobe realizedby meansof
images, or viceversa.As well as a broadculturalcongruence, thereis significant difference
between the two (andothersemioticmodes,of course).In a language suchas Englishone
needsto usea verbin orderto makea full utterance(believe,rs);and language hasto use
namesto refer to whateveris to be represented (a grammar of images,believe,wd.But
language doesnot haveor needanglesof visionto achieveperspective, nor doesit haveor
needspatialdispositions of elements to achieve the meanings of syntacticrelations: images
haveand needboth.The meaningpotentialsof the two modesare neitherfully conflated
nor entirelyopposed. We differfrom thosewho seethe meaningof language as inherentin
the forms and the meaningof imagesas derivedfrom the context,or the meaningsof
l a n g u a gaes ' c o n s c i o uasn' dt h e m e a n i n gosf i m a g e a s s' u n c o n s c i o r . l s ' .
-
To returnto the first of our two questions What havewe usedfrom linguistics, and
20 . The semiotic landscape

how havewe usedit? - perhapsthe mostsignificantborrowingis our overallapproach, an


'attitude'whichassumes that,as a resource for representation, images,like language, will
d i s p l a yr e g ul a r i t i e sw, h i c hc a nb e m a d et h es u b j e cot f r e l a t i v e lfyo r m a ld e s c r i p t i oW n .ec a l l
t h i sa ' g r a m m a r ' t od r a wa t t e n t i o tno c u l t u r a l l yp r o d u c erde g u l a r i t yM. o r es p e c i f i c a l w l ye,
haveborrowed'semioticorientations', featureswhichwe takento be generalto all human
meaning-making, irrespective of mode.For instance, we thinkihat the distinctionbetween
' o b j e c t i v i t y ' a n d ' s u b j e c t i v i t ya' i sg e n e r acl u l t u r a l / s e m i o ti si cs u ew h i c hc a n b e r e a l i z e d
l i n g u i s t i c a lal ys w e l l a s v i s u a l l yt ,h o u g hd i f f e r e n t l ys o ,a s w e h a v es a i d . 0 r ,a s a n o t h e r
instance, we havetaken MichaelHalliday'ssocialsemioticapproachto languageas a
model,as a sourcefor thinkingaboutgeneralsocialandsemioticprocesses/ ratherthanas
a mine for categories to apply in the descriptionof images.His modelwith its three
functionsis a startingpointfor our accountof images,not because the modelworkswell
for language (whichit does,to an extent),but because it workswell asa sourcefor thinking
a b o u ta l l m o d e so f r e p r e s e n t a t i o n .
M a y b em o s tt o t h e p o i n ti s t h i s :o u r a p p r o a c tho c o m m u n i c a t i osnt a r t sf r o m a s o c i a l
base.In our viewthe meanings expressed by speakers, writers,printmakers, photographers,
designers, paintersand sculptorsare first and foremostsocialmeanings/ eventhoughwe
acknowledge the effectand importanceof individualdifferences. Giventhat societies are
not homogeneous, but composed of groupswith varying,andoftencontradictory, interests,
the messages producedby individuals will reffectthe differences, incongruities andclashes
which characterize sociallife. It is likely,and in our experience oftenthe case,that the
differentmodesthroughwhichtextsare constructed showthesesocialdifferences, so that
in a multimodaltext usingimagesandwritingthe writing maycarry oneset of meanings
and the imagescarry another. In an advertisement, for instance, it may be that the verbal
text is studiously'non-sexist',while the visualtext encodes overtlysexiststereotypes. Given
the still prevalentsenseaboutthe meaningof images,it is possibleto pretendthat the
m e a n i n cg a r r i e di n t h e i m a g ei s t h e r eo n l y ' i n t h e e y eo f t h e b e h o l d e rs' ,o m e t h i ntgh a t i t
w o u l dn o t b e p o s s i b lteo a s s e rat b o u tv e r b a l l yr e a l i z e m d eanings.
Our examplesin this bookare quitedeliberately drawnfrom very manydomains,and
from differenthistoricalperiods.We hopethat our ideaswill helpanyoneinterestedin
communication to seein imagesnot onlythe aestheticand expressive, but alsothe struc-
t u r e ds o c i a l p , o l i t i c aa
l n d c o m m u n i c a t i vdei m e n s i o nW s .e w i l l d r a w e x a m p l efsr o m t h e
kindsof textswhichare alreadyfully basedon the newvisualliteracyand playa dominant
rolein anypublicsphere, magazine articles,advertisements, textbooks, websites andso on.
Thisis not because we wantto promotethesetextsasa kindof modelwhichshouldreplace
otherkindsof texts,but because their rolein the livesof childrenandadultsis so important
that we simplycannotaffordto leavethe abilityto ihink andtalk aboutthem (and,indeed,
to producethem)to a handfulof specialists. We havea particularinterestin the placeof
t h ev i s u a il n t h e l i v e so f c h i l d r e na,n dw e h o p et o s h o wt h a t c h i l d r e vn e r ye a r l yo n ,a n dw i t h
v e r yl i t t l eh e l p( d e s p i tael l t h ee n c o u r a g e m e ndte) ,v e l oap s u r p r i s i nagb i l i t yt o u s ee l e m e n t s
of the visual 'grammar'- an ability which,we feel, shouldbe understoodbetter and
developed furlher,ratherthan beingcut off prematurelyas is, too often,the caseat
presena t ;n da n a b i l i t yt h a t s h o u l da l s ob ea v a i l a b lteo a d u l t s .
The semiotic landscape 27

A N U N C O N V E N T I O N AHL I S T O R YO F W R I T I N G

T h ed o m i n a n coef t h e v e r b a lw , r i t t e nm e d i u mo v e ro t h e rv i s u a m l e d i ai s f i r m l yc o d e da n d
b u t t r e s s ei d
n c o n v e n t i o nhails t o r i eos f w r i t i n g T . h e s eg o s o m e t h i nlgi k et h i s .L a n g u a gien
i t s s p o k e fno r m i s a n a t u r a pl h e n o m e n ocno,m m o nt o a l l h u m a ng r o u p sW . r i t i n gh, owever,
i st h ea c h i e v e m eonfto n l ys o m e( h i s t o r i c a l lbyy, f a r t h e m i n o r i t yo f ) c u l t u r e sA. t a p a r t i c u -
lar stagein the historyof certaincultures,theredeveloped the needto makerecordsof
transactions of variouskinds,associated usuallywith trade,religionor (governing) power.
Theserecordswere initiallyhighlyiconic;that is, the relationbetweenthe objectto be
recordedand the formsand meansof recordingwas closeand transparent. For instance,
the numberof notchesin a stickwouldrepresent the numberof objectsstoredor tradedor
owed.The representation of the objectwould usuallyalso be transparent:a wavy line
e v e n t u a l lbye c a m et h e C h i n e s ied e o g r a m f o r ' w a t e r ' ;t h e h i e r o g l y p hi m
c a g eo f t h e o x ' s
h e a dw hi c hi n i t i ally ' s t o o df o r ' ' o x 'e v e n t u a l yl b e c a mteh el e t t e ra l e p h( N ) ,a l p h a( a ) ,a .T h i s
exampleillustrates what in thesehistoriesis regarded asthe rarestof all achievements,the
i n v e n t i oonf a l p h a b e t iwc r i t i n g .
A l p h a b e t iwc r i t i n gd e v e l o p eidt ,s e e m cs l e a ro, u t o f i c o n i ci,m a g e - b a s secdr i p t sI.n t h e s e
originalscriptforms,an objectwas initiallyrepresented by an imageof that object.Over
t i m e ,i n t h e u s eo f t h e s c r i p tb y d i f f e r e ngt r o u p ss,p e a k i ndgi f f e r e nlta n g u a g e s , t ihme a g eo f
the objectcameto standfor the nameof the objectandthenfor its initialletter.Aleph,'ox'
in Egyptianhieroglyphics, after centuriesof traveland constanttransformation through
the culturesand languages of the easternMediterranean, becamethe letter alpha,and
eventually the lettera in the Romanalphabet.Clearlythis was a process whereeachstep
involvedconsiderable abstraction, so muchso that,seemingly, alphabetic writinghasbeen
i n v e n t eodn l yo n c ei n t h e h i s t o r yo f h u m a nc u l t u r e sA. l l p r e s e nat l p h a b e t iscc r i p t sf,r o m
I n d i at o t h e M i d d l eE a s tt o E u r o p ea, r e d e v e l o p m e not fst h a t i n i t i a ls t e pf r o m E g y p t i a n
( o r p o s s i b l yS u m e r i a n i)c o n i ch i e r o g l y p h ri ce p r e s e n t a t i ot on t h e P h o e n i c i aanl p h a b e t ,
and from there westwardto the Greek-speaking world, and eastwardto the Indian
s u b c o n t i n e notr, , i n t h e r e g i o no f i t s o r i g i n ,d e v e l o p i nign t o t h e A r a b i cv e r s i o no f t h e
alphabet.
This is indeedan impressive culturalhistory,impressive enoughto havestoodas the
acceptedhistoricalaccountof the achievement of (alphabetic) writing,unquestioned for
centuries. Withinthis account,all cultureswith formsof visualrepresentation that are not
directlyconnected to language aretreatedascultureswithoutwriting.However, it is worth
investigating this history,and in particularthe crucialstepfrom visualrepresentation to
t h e l i n kw i t h l a n g u a g e a , l i t t l e m o r ec l o s e l yP. r i o rt o t h i s s t e p( i n r e a l i t ya d e v e l o p m e n t
s p a n n i nm g i l l e n n i at )h e r ew e r et w o s e p a r a taen di n d e p e n d em n to d e so f r e p r e s e n t a t i o n . 0 n e
was language-as-speech; the other,the visualimage,or visualmarks.Eachserveda par-
ticular set of purposes suchas the construction of historiesand myths,the recordingof
genealogies andtransactions, andthe recording andmeasurement of objects.In thecaseof
somecultures,however, the oneform of representation 'took over'the other,as a means
of recording; that is,visualrepresentation becamespecialized - onecouldsay,reduced - to
functionas a meansof the visualrepresentation of speech, perhapsin highlyorganized and
22 The semiotic landscaPe

bureaucratized societies. At this pointthe visualwassubsumed, takenover,by the verbalas


p
i t s m e a n so f r e c o r d i n gC.o n s e q u e nittlsyf o r m e r u b l i uses, sibilitie
c p o s a sn dp o t e n t i a lfso r
independent representation disappeared, declinedandwithered away'
In the caseof othercUltures, however, this development did not occur.Herethe visual
continued, alongwith the verbalmeansof representation. Instances of this abound:from
the oneextremeof the Inca quipustrings (sensorily the tactilemode representation)
of to
AustrallanAboriginaldrawings,sand-paintings and carvings.These encode, in a manner
'translation'of, verballanguage, meaningsof the
not at all directlydependentontor a
culturewhichare deemedto be bestrepresented in visualform. They are connected with
language, or language with them,sothat wall-paintings or sand-paintings, for instance, are
accompanied by verbalrecounts of geographical features, journeyst ancestor myths, and so
on. However, in thesecasesthere is no questionof the priority of the one over the other
mode,and the visual has certainlynot becomesubsumed to the verbalas its form of
representation.
In this connection it is interesting to consider the historyof two wordswhichin a sense
are synonymous with Westernnotionsof literacy,the words grammar and syntax' Gram-
mar derivesfrom the Greekgrammatike('the art of readingand writing','qrammar',
'letter','alphabet'), grammatikosCliter-
'alphabet'),' relatedwordSweregramma('Sign',
ate,,,(primary)teacher','grammarian'). Thisetymologyrecordsthe state of thingsin the
H e l l e n i s t ipce r i o d( f r o ma p p r o x i m a t e3l y0 0 8 c ) ;i n e a r l i e rt i m e st h e m e a n i n g ' s i g na' ,s i n
'paintedor drawn[etc.] mark'wasthe primarymeaning. In Homer, for example, the verb
grapheinstill means'Scratch','scratch in', as in engraving/ and fr6m there it comes to
'drawing', 'painting'.Syntaxis,in pre-Hellenistic times, meant
mean both'writing'and
'Contract','Wage','OrganizatiOn','System','battle fOrmatiOn', with Syntagmat fOr inStanCe,
' c o n t i n g e notf t r o o p s ' , ' c o n s t i t u t i(oonf a s t a t e ) ' , ' b o ook r t r e a t i s e 0
'. n l y i n t h e Hellenistic
(among 'grammatical construc-
perioddoessyntaxiscometo mean its othermeanings)
'arrange 'concentrate
tion'. The verbsyntasso, again/meansboth battleformations'and
( o n e ' st h o u g h t s ) ' , ' o r g a n i z e ' , ' w r i t e ' , ' c o m p o s e ' .
W h i l ew ed o n o tw i s ht o p l a c et o o m u c he m p h a s iosn e t y m o l o gny e, v e r t h e l et shseh i s t o r y
of thesetwo wordswhichare so crucialto our notionsof literacypointsto formsof social
'markings'on the other'Together
organizationandorder,on the one hand,and to visual
they indicatethe initiallyquite independent organization of the modeof imagesand the
modeof verballanguage. At the sametime,the subsequent historyof the word grammar
b r i n g so u t c l e a r l yt h e s u b o r d i n a t i oonf t h e v i s u a lm e d i u mt o t h e m e d i u mo f v e r b a l a n -
guage.Cultureswhichstill retainthe full useof bothmediaof representation are,from the
ooint of view of'literate cultures',regardedas illiterate,impoverished, underdeveloped,
when in fact they havea richer arrayof meansof representation than that overtlyand
consciously availableto literatecultures.Nevertheless, as we pointed out earlier,literate
culturesdo makeuseof meansof visualcommunication otherthan writing, be it that they
areseenas uncoded replicasof realityor as a meansof individual expression by childrenor
artists.In otherwords,they are not treatedas eitherthe expressions of, or accessible to
meansof readingbasedon,articulated,rationalandsocialmeanings'
0ur unconventional historyof writing is onethat treatsthe comingtogetherof visual
Thesemiotic landscape 23

a n dv e r b a rl e p r e s e n t a t i ao sno n l yo n ep o s s i b i l i tayn/ do n e f/ u r t h e r m o r e t h, a t b r i n g sw i t h i t
n o t j u s tt h o s eb e n e f i tosf w r i t i n gw h i c ha r ew e lI e n o u g hu n d e r s t o o bd u, t a l s ot h e n e g a t i v e
aspectsincurredin the lossof an independent form of representation, the diminutionof
modesof expression and representation. Fromthat point of viewculturessuchas Austral-
i a n A b o r i g i n acl u l t u r e sa r e s e e na s h a v i n gb o t h m o d e so f r e p r e s e n t a t i ot h ne: v i s u a l( o r
p e r h a pa s w h o l es e t o f v i s u a fl o r m so f r e p r e s e n t a t i oann) d t h e v e r b a lT . h e o i n to f t h i s
p
h i s t o r yi s n o t o n l yt h e p o l i t i c a o l n e o f u n d e r m i n i nt h
g e n o t i o no f i l l i t e r a t ec u l t u r e ' ( o r
' m e r e l yo r a l c u l t u r e ' )b, u t a l s ot h e a t t e m p t o s e et o w h a te x t e ntth e c o n v e n t i o nhails t o r y
b l i n d su st o t h ef a c t sa n du s e so f v i s u acl o m m u n i c a t i oi nns o - c a l l eldi t e r a t ec u l t u r e s .
I n t h i s b o o kw e d e v e l o p t h e h y p o t h e sti hs a t i n a l i t e r a t ec u l t u r et h e v i s u a lm e a n so f
c o m m u n i c a t i oanr e r a t i o n a le x p r e s s i o nosf c u l t u r a lm e a n i n g sa,m e n a b l et o r a t i o n a l
accountsand analysis. The problemwhichwe face is that literatecultureshavesystem-
aticallysuppressed meansof analysisof the visualformsof representation, so that thereis
not, at the moment,an established theoreticalframeworkwithin which visualforms of
reoresentation can be discussed.

T H E ' O L D ' A N DT H E ' N E W ' V I S U A LL I T E R A C YI N B O O K SF O RT H E V E R Y


YOUNG

S o f a r w e h a v ed i s t i n g u i s h tewdo k i n d so f v i s u a l i t e r a c yo: n ei n w h i c hv i s u acl o m m u n i c a -


tion hasbeenmadesubservient to language and in whichimageshavecometo be regarded
a s u n s t r u c t u r erde p l i c a o s f r e a l i t y( t h e ' o l dv i s u a l i t e r a c y ' , i no u r t e r m s ) a ; n da n o t h e ri n
w h i c h ( s p o k e n l)a n g u a g e x i s t ss i d eb y s i d ew i t h , a n d i n d e p e n d e o n ft , f o r m s o f v i s u a l
representation which are openlystructured,ratherthan viewedas more or lessfaithful
d u p l i c a t eosf r e a l i t y( t h e ' n e w ' ,i n o u r t e r m s ) W . e h a v el o o k e da t t h e s ea s h i s t o r i c aal n d
culturalalternatives. But they also existside by side,at leastin contemporary Western
c u l t u r ea, n dw e s u g g e st h t a t w e a r e i n t h e m i d d l eo f a s h i f ti n v a l u a t i o a n n du s e sf r o mt h e
o n em o d et o t h e o t h e rf,r o mt h e ' o l d ' t o t h e ' n e w ' v i s u al li t e r a c yi n, m a n yi m p o r t a nst o c i a l
contexts.The exampleswe will now discusssuggestthat the very first bookschildren
e n c o u n t emr a ya l r e a d yi n t r o d u cteh e mt o p a r t i c u l akr i n d so f v i s u a l i t e r a c y .
Figure1.1 showsa typicaltwo-pagespreadfrom Baby'sFirst Book,a bookwhich,on
i t s i n s i d ec o v e rd, e c l a r etsh a t ' t h e t e x t a n d i l l u s t r a t i o ntsh,o u g ho v e r s i m p tl e o grown-ups,
w i l l s a t i s f yt h e i rt i . e .t h e t o d d l e r s 'cl r a v i n gfso r t h e r e p e t i t i o on f w h a tt h e ya l r e a d yk n o w ,
andwill helpthemassociate thewordswith the objects'.Whenwewrotethefirst versionof
t h i s c h a p t e ri ,n 1 9 8 9 , i t w a s s t i l l w i d e l yd i s t r i b u t e da,n d t o d a y i t i s a l r e a d ym a k i n ga
c o m e b a caks a n o b j e c o t f nostalgia.
Figure1.2 showsa typicalpagefrom Dicl<Bruna's0n My Walk.Thisbookis oneof a
setof four,the othersbeingIn My Home,In My ToyCupboardand0n the Farm.lt consists
of eightpagesand,with the exception of the front and backcovers, the pagescontainno
wordswhatsoever.
Comparedto the pictureof the bird in the tree,the pictureof the bath is realistic,
detailedand complex.If we were to analyseit into its components, if we were to try
Thesemiotic landscape

{-
l- ,
i . - _y *
' ---,,.-,
i'1d
-,J
,
f: i,;: i: i
\, -"i
i i'lii !ii:l f-Tt il
i
!-:* I i
i;*iir*o** 3
r !,-,,t E
"r

i'i:u **:,::1

O Fig l.l My bath(from Bahls Fitst Boo&Ladybird)

a n d i d e n t i f ya l l t h e d i f f e r e net l e m e n tosf t h i s p i c t u r ew
, e m i g h te n c o u n t eprr o b l e m sA.r e
the ripplesin the water to be countedas components? Are the shadows, cast by the tub
and towel?And if we wereto try and identifythe relationsbetweenthesecomponents,
what wouldwe haveto say,for example,about the relationbetweenthe duck and the
soap?We ask thesequestionsbecausethey are the kindsof questionswith which one
might start if one wantedto show that imagesare structuredmessages, amenable
to constituentanalysis.Isn't the structureherethat of the cultural object'bathroom',
r a t h e rt h a no n ei m p o s ebdy t h ec o n v e n t i o no sf a v i s u acl o d e ?I s n ' tt h i sp i c t u r eu n p r o b l e m -
atically,transparently readable(recognizable), providedone knowswhat bathroomslook
like?
T h i si s t h e l i n eP a r i sS c h o osl e m i o t i c i a nssu c ha s R o l a n dB a r t h e sa n d C h r i s t i a nM e t z
t o o k i n t h e 1 9 6 0 sC . o m m e n t i nogn p h o t o g r a p hBya, r t h e s a i d :

In orderto movefrom the realityto its photograph it is in no way necessary to divide


up this realityinto unitsandto constitutetheseunitsas signs,substantially different
f r o m t h e o b j e c t t h e yc o m m u n i c a t e . . .C. e r t a i n l y , t hi e
m a g ei s n o t t h er e a l i t yb u t a t
leastit is its perfectanalogonand it is exactlythis analogicalperfectionwhich,to
commonsense, definesthe photograph. Thuscan be seenthe specialstatusof the
photographicimage:.it is a messagewithouta code.
( B a r t h e s1. 9 7 7 : l - 7 )
The semiotic landscape . 25

Q fig f.Z Bird in tree (Bruna,t988)

A n d h ee x t e n dtsh i sa r g u m e nt to o t h e rp i c t o r i am
l o d e sa, l b e i tw i t h a q u a l i f i c a t i o n :

Are there other messages without a code?At first sight,yes:precisely the whole
rangeof analogicalreproductions of reality- drawings,paintings, cinema,theatre.
However, each of those messages developsin an immediateand obviousway a
s u p p l e m e n t am
r ye s s a g e
. . . w h i c h i s w h a t i s c o m m o n l yc a l l e dt h e s t y l e of the
reoroduction.
( B a r t h e s1,9 7 7 : 1 7 )

The pictureof the bird in the tree,on the otherhand,is muchlessnaturalistic, muchless
d e t a i l e ad n dm u c hs i m p l etrh a nt h ep i c t u r eo f t h eb a t h r o o mI t. i s s t y l i z e ad n dc o n v e n t i o n a l ,
a n d q u i t ec l e a r l ya ' c o d e d ' i m a g e .N o d e p t h ,n o s h a d o w sn,o s u b t l en u a n c e o sf colour:
s p l a i na n db o l da n ds i m p l eA. n dt h es t r u c t u r o
e v e r y t h i ni g e f t h e i m a g ew, i t h i t s o n ec e n t r a l
a n df o u r m a r g i n ailm a g e sd,o e sn o t i m i t a t ea n y t h i n g i n t h e r e a lw o r l d .I t i s a c o n v e n t i o n a l
visualarrangement, basedon a visualcode.As a resultthe components of the wholestand
out as separate, distinctunits,and the picturewouldseemquiteamenable to constituent
a n a l y s i sT. h i s i s n o t j u s t a m a t t e ro f s t y l e :t h e s t r u c t u r eo f t h i s p i c t u r ec o u l da l s o b e
realizedin more detailedstyles.Bruna'sbook datesfrom 1953, well beforethe era of
c o m p u t e' ri m a g i n gb' ,u tt h ep i c t u r eo f t h e b i r di n t h et r e ec o u l dh a v eb e e nc o m p o s ewdi t h a
c o m p u t ear ,l i g n i n gr e a d y - m a dsei m p l ei c o n si n a c o m p o s i t i o ncaol n f i g u r a t i o- ni t i s i n f a c t
q u i t es i m i l a tro t h e c o m p u t e r - d r a w
d inn n e ri n v i t a t i o inn f i q u r e1 . 3 .
The semiotic landscape

irrulNer{1\'

@ fig f.: Computer-drawn


dinnerinyitation

Second, the pictureof the bathroomis part of a two-pagelayout,and accompanied by


words.Languagecomesfirst, authoritatively imposingmeaningon the image,turningit
into a typicalinstanceof a bathroomby meansof the genericlabel'Bath'.As a resultthe
picturecould be replacedby other imagesof bathroomswithout much lossof meaning
( o n ev e r b a lt e x t , m a n y i m a g e sm , a n y p o s s i b l ei l l u s t r a t i o n sH
) .e r el a n g u a g e is general,
bestowingsimilarityand order on the diverse,heterogeneous world of images.Thusthe
book p r e s e n t o n t h e o n eh a n d a, n ' u n c o d e dn' ,a t u r a l i s t irce p r e s e n t a t i (o' n
s , t h ew o r l da s i t
i s ' - e m p i r i c a lf ,a c t u a l ,s p e c i f i ca) n d ,o n t h e o t h e rh a n d ,a s p e c i f i ca,u t h o r i t a t i v epl yr e -
s c r i b ewd a yo f r e a d i n tgh i s ' u n c o d e d n 'a t u r a l i s t ipci c t u r eW
. ew i l l s h o wl a t e rt h a t ,c o n t r a r y
to what Barthesand othersarguedin the 1960s, pictures of this kindare alsostructured,
whetherthey are photographs, drawings,paintingsor other kinds of pictures.For the
moment,however, the importantpointis that theyare not usuallyinterpreted as such,that
awareness of the structuredness of imagesof this kindis,in our society, suppressed andnot
o a r to f t c o m m o n sense'.
In Dick Bruna's0n My Walk,by contrast,thereare no wordsto authoritatively impose
meaning o n t h e i m a g ea, n dt h e i m a g ei s n o l o n g e ra n i l l u s t r a t i o nt h: e i m a g ec a r r i e st h e
m e a n i n g , t hweo r d sc o m es e c o n dP. a r e n tw s h o r e a dt h i sb o o kw i t h t h e i rc h i l d r e n c o u l da l l
tell a differentstory,couldevenusedifferentlanguages (one image,manyverbaltexts).
The semiotic landscape

The world of '0ne image,many differentverbaltexts' ('commentariesi) imposesa new


mode of control over meaning,and turns the image,formerlya recordof natureor
a
p l a y g r o u nfdo r c h i l d r e na n d a r t i s t s i, n t o a m o r ep o w e r f u b l , u t a l s om o r er i g o r o u s lcyo n -
t r o l l e da n d c o d i f i e dp u b l i cl a n g u a g ew,h i l ei t g i v e sl a n g u a g ef o, r m e r l yc l o s e l yp o l i c e d
in
m a n ys o c i a il n s t i t u t i o nas ,m o r ep r i v a t ea n d l e s sc o n t r o l l e db,u t a l s ol e s sp o w e r f u l ,
s t a t u s.
The'readings'which parentsproducewhenthey read0n My Walkwith their childrenmay
all be different,yet thesedifferentreadings will necessarily havecommonelements, deriv-
ing from their commonbasis- the elementsincludedin the image,and the way
these
e l e m e n tasr ec o m p o s i t i o nl yabl r o u g htto g e t h e r .
Whateverstoryparentswill tell aboutthe pagewith the bird in thetree,it will necessar-
ily haveto be a story that createsa relationbetween, for instance, birdsand aeroplanes
( n a t u r ea n dt e c h n o l o g ya)n d b i r d sa n dc a t s( p r e y
a n dp r e d a t o r )I .t w i l l a l s oh a v et o b e a
storyin whichthe bird,safelyin itstree,isthe centralcharacter,literally andfiguratively. In
how manywayscan catsand birdsbe related?Not that many,at leastnot if one
assumes
that bookslike 0n My Walkserveto introducechildrento the world aroundthem,rather
t h a nt o t h e p o s s i b lw e o r l d so f f a n t a s i eas n d u t o p i a sC . a t sc a n ' h u n t , , ' t o r t u r e.,k, i l l , a n d
'eat' birds.Birds
can 'escape'cats or fail to do so. Thereare not that manystoriesto
choose f r o m . 0 nt h e o t h e rh a n d p , a r e n t sa n dt h e i rc h i l d r e nc a nc h o o s teh e o r d e ri n w h i c h
theywant to dealwith the variouselements: the pageis'non-linear,. It doesnot imposea
sequential structure.And theycanchoosewhetherto tell the storyof the bird andthe cat
as a politicaistory,a story of powerfulpredatorscomingfrom anothercontinentand
n a t i v eb i r d sk i l l e da n dt h r e a t e n ewdi t h e x t i n c t i o n( a sm i g h tb e d o n ef,o r i n s t a n c ei n, A u s -
tralia),or as a storythat legitimizes the survivalof the fittest.Thestoryof the bird andthe
a e r o p l a n se i,m i l a r l ym, a y b e t o l d f r o m a n e n v i r o n m e n t a lpi soti n to f v i e w o , r a s a s t o r yo f
e v o l u t i o n a rtyr i u m p h sa n d h u m a nt e c h n o l o g i c a p lr o g r e s sE. v e nw h e r es u c hd i s c o u r s e s
are not explicitlyinvoked, they will still communicate themsejves to childrenthrouqhthe
parents'attitudes towardsthe characters andthe actions.
Not only the elementson the individualpages,but alsothe pagesthemselves must be
broughtin relationto eachother.The book as a whole must be readableas a conerent
sequence. Thisis promptedby the title (0n My Walb aswell as by the pictureon the front
cover,which showsall the elements together. We haveinvestigated this a little further in
connection with anotherbool<in the Bruna series,0nthe Farm.This book containsthe
followingcentralpictures:house, farmer,cat,dog,appletree,rooster, lamb,cow.Listingthe
w a y si n w h i c ht h e s ep i c t u r e cs a np l a u s i b lbye l i n k e dt o e a c ho t h e rw, e f o u n dt h a t s o m e( e . g .
t h e a p p l et r e ea n dt h e h o u s ec) a n o n l yb e l i n k e di n s p a t i a ll,o c a t i v e t e r m s( e . g t. h e a p p l e
t r e ei s n e x t o t h eh o u s e )O. t h e r s( e . gt.h ea n i m a l a s n dt h eh o u s ec) a nb er e l a t e db y v e r b so f
'dwelling'(e.9.
the cow liesunderthe appletree) or by theverbsof 'motion,(e.g.the cat
c l i m b su p t h e a p p l et r e e ) .T w oo f t h e a n i m a l s( t h ec a t a n dt h e d o g )c a nr e l a t et o t h e o t h e r
animalsand to eachotherby meansof antagonistic or co-operative actions(e.g.the dog
barksat the cow;the dog leadsthe sheep).only the farmercan relateto all the other
elementsin an agentiveway. He can buy them,own them,buildthem,grow them,keep
them,raisethem,harvestthem,shearthem,slaughterthem,and so on. In otherwords,
whateverway the parentsreadthesepictures,they will, in the end,haveto dealwith the
The semiotic landscape

themeof spatialorder,the themeof socialinteraction(projectedon to animals)and the


themeof humanmasteryovernature(aswell as,via the marginalpictures, with the theme
of procreation), andtheywill haveto do all this in termsof the elements pre-selected by the
book.An analysisof the way the elementscan be opposedto each other showsthat,
whateverthe classifications parentsmayconstruct, theywill not be ableto avoidengaging
with the Westernculturaldistinctions between 'untamednature','domesticated/cultivated
n a t u r e ' a n d ' h u m a nt e c h n o l o g yA' .n d t h e y w i l l a l s o h a v et o r e c o g n i zteh e d i s t i n c t i o n
betweenanimateand inanimate, flora andfauna,and betweenpets,farm animalsandwild
animals.
It shouldbe noted,however, that everypagein the book (and in Bruna'sotherbooks)
containsat leastonerelationthat doesnot easilyfit the received classifications, that forms
somewhat of a challengeandapuzzle.What,for example, is the relationbetween a rabbit
and a basketof ffowers?A beetleanda fence?Suchvisualenigmascanchalienge parents
a n dc h i l d r e tno e x e r c i steh e i ri m a g i n a t i otno, i n c l u d ei n t h e i rt h i n k i n ge l e m e n ttsh a t d o n o t
easilyfit in with the traditionalorder of things,to toleratesomeambiguity, to allowthe
i n c l u s i oonf t h e ' o t h e r 'i n t h e i rc o n s t r u c t i oonf t h ew o r l d .
Thetwo books,then,are very differentin their stancetowardsthe image.The Bruna
stancepresents highlyprocessed/ essentialized and idealized representations, and provides
parentsand,later,childrenwith the opportunity to talk aboutthe imagesin wayswhichare
or seemappropriate to them,to applyspecificvalues, specificdiscourses to theserelatively
abstractimages. The Ladybirdstancepresents ostensibly lessprocessed, morenaturalistic
visualrepresentations andprovides parents(and,lateron,children)with a specificverbally
realizedway of readingthe image.The Ladybirdbook is openand interactive from the
perspective of the image,and authoritarianfrom the perspective of writing;the Bruna
b o o kw o r k si n t h e o p p o s i t e w a y .T h ec l o s u r ei n t h e B r u n ab o o k l i e si n t h e l i m i t sw h i c h
selection,form andstructureof the imagesimposeon the apparently openreadings-these
e n t e rt h e d i s c o u r s ewsh i c ha r e a l r e a d y ' i n ' t h es o c i a l i z epda r e n t ss,o t h a t t h e w h o l eo, n c e
o r a l l yt r a n s m i t t etdo t h e c h i l d r e nw, i l l a p p e a rs p o n t a n e o u a sn d ' n a t u r a l ' t op a r e n t sa n d
c h i l d r e na l i k e .A r e t h e yn o t ,a f t e ra l l , m e r e l ye n g a g e idn a n i n n o c e nrte a d i n go f ' w h a t i s
there'in the pictures?Thusthe two booksreoresent two differentformsof socialcontrol
o v e rm e a n i n g . 0 nies o p e n l ya n de x p l i c i t l lyo c a t e di n t h e t e x t i t s e l f t; h e o t h e rl i e s p
, erhaps
more covertlyand implicitly,in the way the book presentsitself lessas a text than as
an organizedresourcefor makingtexts,jointly wiih the parentaldiscourses that will
inevitably enterthe text aswell.
Thesediscourses, however, are not themselves part of Bruna'sbooks,of the publictext
meantto transcend their diversity. Insteadtheyare relegated to the realmof the private,of
'lifestyles'where they do not threatenthe orderof the largersocialworld.Thereis never
j u s t ' h e t e r o g l o s s(ima 'a n ym e a n i n g sn)o, re v e rj u s t ' h o m o g l o s s i(ao'n ea u t h o r i t a t i vm ee a n -
i n g ) .I n s t e a tdh e r ei sa r o l ed i s t r i b u t i oanm o n gt h ed i f f e r e nst e m i o t i cas ,r o l ed i s t r i b u t i oinn
which somesemioticsare givena great deal of socialpower,but at the price of being
s u b j e c t etdo g r e a t e ri n s t i t u t i o n a( la n dt e c h n o l o g i c acl o ) n t r o lw
, h i l eo t h e r sa r e a l l o w e d
relativefreedomfrom control,but pay for this with diminished power.Today,we seemto
movetowardsa decreaseof control over language(e.9.the greatervarietyof accents
Thesemiotic landscaPe ' 29

a l l o w e do n t h e p u b l i cm e d i a , t h e i n c r e a s i npgr o b l e misn e n f o r c i nngo r m a t i vsep e l l i n ga) ,n d


towards an increase in codification andcontroloverthe visual(e.g.the useof imagebanks
from which ready-made images can be drawnfor the constructionof visualtexts,and,
generally, the effect of computer imaging technology).
The two forms of control over meaning can be found elsewhere also.Compare,for
e x a m p l et h, e c l a s s i d
c o c u m e n t a f
r i
y l m - i n w h i c h a n a u t h o r i t a t i v e ' v o i oc feG o d 'n a r r a t o r
e x p l a i nas n d i n t e r p r e t ism a g e so f r e c o r d e r
d e a l i t y- t o t h e m o r e m o d e r n' d i r e c tc i n e m a '
d o c u m e n t a riyn, w h i c hc o n t r o lo v e rm e a n i n gl i e si n t h e s e l e c t i o o n f i m a g e sa n d i n t h e
sometimes hardly noticeable ways in which these images are edited together' 0r thinkof the
'cultural on analysing 'whatthetext says'
way in which, in the field of studies', an emphasis
i s g r a d u a l lbye i n gr e p l a c ebdy a n e m p h a s losn ' h o wd i f f e r e nat u d i e n c er e s a dt h es a m et e x t ' ,
an emphasis/ other in words, on the apparent freedom of interpretation which, by diverting
attentionawayfrom the text itself, allows the limitations which the text imposes on this
'freedomof reading'to remaininvisible, and therefore,perhaps, all the moreefficacious
a n dp o w e r f u l .
I n t h i sc o n n e c t i ot h n eb a c k g r o u nodf t h e B r u n ab o o ki s w o r t hb r i e fm e n t l o nI t. w a sf i r s t
p r i n t e di n 1 9 5 3 ,i n A m s t e r d a ma,n dr e p r i n t e m d a n yt i m e si n i t s c o u n t r yo f o r i g i nT. h ef i r s t
British p r i n t i n gw a s in 1 9 7 8 . T h e t i m e l a g i s p e r h a pn s o a c c i d e n tU. n l i k et h e B r i t i s ht,h e
Dutchhad,early in the twentieth century, recognized that their countrydid not havea
'commonculture',but wasdividedintogroupscharacterized by differentandoftenoppos-
i n g l d e o l o g i ezsu, i l e n ( l i t e r a l l y ' c o l u m n s ' , ' p i l l a r as '
s )t,h e D u t c hh a v ec a l l e dt h e m .D u t c h
broadcasting, for example, had from its inception in the late 7920sa systemin which
differentlevensbeschouwelijke groups (i.e.'groups orientated towards a particularviewof
life,) ran broadcasting organizations which were allotted air time according to the size
o f t h e i r m e m b e r s h iT p .h u s t h e s a m e e v e n t w
s o u l d ,o n r a d i o a n d l a t e r o n t e l e v i s i o nb,e
interpreted from a variety different of discursive ideological positions, while most other
E u r o p e acno u n t r i ehs a d c e n t r a l i z e d , u s u a l l yg o v e r n m e n t - r ub n
r ,
o a d c a s t i o
n gr g a n i z ations
with oneauthoritative message. For a message to reach, in this context,the whole popula-
tion,it hadto beadaptable to a varietyof culturaland ideological constructions and,aswe
haveseen,the Brunabool<susethe visual medium to achieve exactly that. Perhaps the
belatedsuccess of the seriesin countries like Brltaln and Australia shows that there is now,
' c o m m o nC u l -
i n t h e s ec o u n t r i etso o ,a n i n c r e a s i nagw a r e n e st hs a t t h e yn o l o n g e rh a v ea
ture', andthat, instead,they havebecomecomplex,diverseand discursively divided,and
thereforein needof newformsof communication (although the Dutch zuilen system went
i n d e c l i n fer o mt h e 1 9 6 0 so n w a r d s ) .
T h ec h a n g l ndgi s t r i b u t i o nosf m e a n i n b g e t w e e lna n g u a gaen di m a g e , w h i cwhe s u g g e s t
is now in full flow,was foreshadowed by various experiments in the SovietUnionof the
earlyI920s. While linguistsand literary scholars lil<e Voloshinov and Bakhtinwrote of
' m u l t i - a c c e n t u a
a t n' d ' h e t e r o g l o s s icco' ,n s t r u c t i v i a s tr t i s t s
l a n g u a gaes s o c i a l l yd i v i d e d ,
l i k e M a l e v i c hE, l L i s s i t z kay n d R o d c h e n k a o n, d f i l m - m a l < e lri k
s e E i s e n s t e irne , j e c ted nat-
uralismanclbeganto elaboratea new vlsual language, capable of communicating new,
revolutionary ideasvisually. Then,as now,imagesbecamemorestylized, moreabstractand
more obviouslycoded:the new visual language was explicitly compared with language,
30 The semiotic landscape

with hieroglyphic writing,with the stylizedmasksof kabukitheatre.Then,as noWvisual


communication was alsoseenas transparent: coloursand shapeswerethoughtto havea
d i r e c t ,u n m e d i a t e d , ' p s y c h o l o g i c a l ' i map ancotn, - s e m i o tci ca p a c i t yf o r s t i r r i n gt h e e m o -
tions of the 'masses'.Then,as now,visualcommunication was to be removedfrom the
sphereof art, to becomepart of the morepowerfuland more publicsphereof industrial
production, of typography, design,architecture. Thissemioticrevolutionwas alliedto the
p o l i t i c arl e v o l u t i o nc:o n s t r u c t i v i ps ot s t e ras n df i l m sh a da p r o p a g a n d i s tpi u c r p o s -e t h e y
s o u g htto h e l pb r i n ga b o u ta c u l t u r a rl e v o l u t i o na,n dt h e yh a dt o g e tt h e i rm e s s a gaec r o s s
to a socialla y n d l i n g u i s t i c a lhl ye t e r o g e n e opuosp u l a t i o nT.h ev i s u a lt,h o u g h t o b e a b l et o
produce a n e m o t i v ei m m e d i a cw y ,a st o b et h e m e d i u mt h a t c o u l da c h i e v teh i s .I n t h e e n d ,
t h e n e ws e m i o t i o c r d e rf a i l e dt o e s t a b l i siht s e l fp e r m a n e n t Il yt w. a sc r u s h e b d y S t a l i n0. l d -
fashioned centralistandrepressive controlovermeaning(andwith it a returnto naturalist,
'bourgeois'arp t )r e v a i l eodv e rc o n t r o b l y m e a n so f a f o r mo f p r o p a g a n dt h a a t c o u l da l l o w
pluralism a n d i d e o l o g i c caol h e s i otno c o e x i s T t . h i st i m e- t h o u g h w i t h v e r yd i f f e r e npt o l i t -
i c a l ,s o c i a lt,e c h n o l o g i caanl de c o n o m icco n d i t i o n-si t m a yn o tf a i l .
Thesemioticshiftswe haveexemolified in our discussion of thetwo children's bookscan
be observed elsewhere, too. The shift from 'uncoded'naturalisticrepresentations to styl-
ized,conceptual imagescan be seen,for instance, on the coversof newsmagazines, which
usedto be dominatedby documentaryphotographs - photographs recordingevents,or
p o r t r a y i nnge w s w o r t hpye o p l e . O c c a s i o ntahlilsys t i l lh a p p e n as s, i n f i g u r e1 . 4 ,b u t i n c r e a s -
inglythe photographs on magazinecoversare contrivedand posed,usingconventional
symbolsto illustratethe essence of an issue,ratherthan documenting newsworthy events,
as in figure1.5,wherea padlockand a UnitedStatesflag,againsta neutralbackground,
i l l u s t r a t et h e i s s u eo f t i g h t e n e d b o r d e rc o n t r o l .U n l i k et h e p i c t u r eo f t h e ' B i r d i n T r e e '
( f i g u r e1 . 2 ) , t h e s a e r es t i l l p h o t o g r a p hiim c a g e sb,u t t h e ym i g h ta s w e l lb e d r a w i n g s .
As an exampleof the changingrelationbetweenlanguageand image,consideran
e x t r a c tf r o m a S c i e n c eC D - R O Mf o r t h e l o w e ry e a r so f h i g hs c h o o l( f i g u r e1 . 6 ) . H e r e
l a n g u a ghea sh e r eb e e nd i s p l a c ebdy t h ev i s u a al s d e c i s i v eal ys i n t h e B r u n ab o o k .I n s t e a d
o f t h e m a j o r m e d i u mo f i n f o r m a t i o nw,i t h t h e v i s u a la s ' i l l u s t r a t i o n 'i ,t h a s b e c o m ea
m e d i u mf o r c o m m e not r l a b e l l i n gw,i t ht h ev i s u aal st h ec e n t r asl o u r c eo f i n f o r m a t i o T n .w o
q u e s t i o nnse e da s k i n go: n ei s t h e q u e s t i o o n f i m p l i c i ct h a n g eisn n o t i o n sa n d p r a c t i c eos f
reading,and of readingsciencein particular;the other is the questionof changesin the
constitutionof what is represented here,scienceitself.The students/viewers/users of
the science C D - R 0 Ma r e n o l o n g e ra d d r e s s evdi a t h e h i e r a r c h i c a lcl yo m p l e xs t r u c t u r e s
of scientificwriting,with its specificdemandsfor cognitiveprocessing, and its needto
'translate'verba f ol r m st o t h e i r t h r e e - d i m e n s i o o n ravl i s u a le q u i v a l e n (t sa s o n t h e p a g e
r e p r o d u c eidn f i g u r e1 . 7 ) .T h e ya r e a d d r e s s elda r g e l yi n t h e v i s u a lm o d e a , n d e i t h e ra s
' s c i e n t i s t s ' w huon d e r s t a nadb s t r a c t i ofnr o mt h ee m p i r i c a l lrye a l ,o r a s p e o p l ef o c u s i nogn
t h e e m p i r i c a l l rye a lw i t h t h e i n t e n t i o n t o u n d e r s t a ntdh e r e g u l a r i t i el sy i n g ' b e h i n d ' t h a t
reality.In otherwords,eventhoughthe visualmodemightseemto providedirect access
t o t h ew o r l d ,i t i s a s a m e n a b lteo r e a l i z i ntgh e o r e t i c aplo s i t i o nass i s t h ev e r b a l .
M o r e c o m p l e xi s t h e q u e s t i o n w h e t h e ri ,n t h i s d r a m a t i cs h i f t f r o m t h e v e r b a lt o t h e
visual,the veryconstitutionof the schoolsubjectScienceis undergoing a transformation.
The semiotic landscape . 3l

@ fiq f,l Magazinecoverwith naturalisticphotograph(Newsweek,lgApril2004)

Can everything that was communicable in the formationof scientificwriting be said in


t h e s ev i s u a l l cy o n s t r u c t ef d o r m s ?C o n v e r s eal yr ,et h e r ep o s s i b i l i t i e
o sf s c i e n t i f icco m m u n i -
c a t i o ni n t h ev i s u aw l h i c hw e r en o t a v a i l a b lien t h e m o d eo f w r i t i n g ?A n dw h i c ho f t h e s ei s
a m o r ea p t m e d i u mf o r s c i e n t i f itch e o r y ?W i l l s c i e n t i f itch e o r i e sc h a n g ea s t h e f o r m o f
expression shiftsfrom the writtento the visualmode?We cannottal<eup thesequestions
here,but if we are to makeourselves conscious of the far-reachingimplications of these
changes in the semioticlandscape, theyneedat leastto be asked.
I m p l i c i ti n t h i si s a c e n t r aql u e s t i o n , w h i n che e dtso b ep u to p e n l ya,n dd e b a t esde r i o u s l y :
is the movefrom the verbalto the visuala lossor a gain?0ur answerat this stagein our
t h i n k i n gi s m u l t i p l eT. h e r ea r e l o s s e sa,n dt h e r ea r e g a i n s . 0 u ra r g u m e ntth r o u g h o ut th i s
bookis that differentsemioticmodes- the visual,the verbal,the gestural, etc.- eachhave
their p o t e n t i a l i t i e s
a n dt h e i r l i m i t a t i o n A
s . m o v ef r o m a c e n t r arl e l i a n c oe n o n em o d et o a
centralrelianceon anotherwill thereforeinevitablyhaveeffectsin both directions.But
that is notthe endof the story.We alsohaveto consider what is represented. It maybethat
visualrepresentation is moreapt to the stuff of sciencethan language everwas/or even
t h a t a s c i e n cw e h i c hi s c o n s t r u c t evdi s u a l l yw i l l b e a d i f f e r e nkt i n do f s c i e n c eT.h ew o r l d
represente v ids u a l l o y n t h e s c r e e nosf t h e ' n e wm e d i a ' i sa d i f f e r e n t lcyo n s t r u c t ewdo r l dt o
that whichhad beenrepresented on the denselyprintedpagesof the print mediaof some
The semiotic landscape

@ fi9 f.S Magazinecoverwith conceptualphotograph(ltewsweek,l2November


200I)

thirty or forty yearsago.Theresources it offersfor understanding andfor meaning-making


differfrom thoseof the world represented in language, andso do the citizensit produces.
Theseare far-reachingquestionsand they can only be answeredby considering the
i n t e r c o n n e c t i obnest w e e tnh e c h a n g i n p g o l i t i c a le, c o n o m i ac n d c u l t u r a lc o n d i t i o ngsa t h -
e r e d u p u n d e rt h e l a b e lo f g l o b a l i z a t i oann d t h e n e w p o s s i b i l i t i ef so r r e p r e s e n t a t i o n
affordedby the newmediaof production anddissemination. We havebarelyhintedat these
k i n d so f q u e s t i o ni sn o u r d i s c u s s i oonf t h e B r u n aa n d L a d y b i r db o o k sC . o u l di t b et h ec a s e
that informationis now so vast,so complex,that perhapsit, hasto be handledvisually,
because the verbalis no longeradequate?
M e r en o s t a l g i am/ e r es o c i aal n dc u l t u r a rl e g r e t so r p e s s i m i scma n n o ht e l ph e r eW . e ,a l l
of us, haveour particularstandpoints and our particularvaluescarriedforwardfrom
yesterdayor from the day beforeyesterday. The first most importantchallengeis to
u n d e r s t a nt h
d i ss h i f t ,i n a l l o f i t s d e t a i la, n di n a l l o f i t s m e a n i n gF.r o mt h a t u n d e r s t a n d i n g ,
we can hopeto beginthe task of constructing adequate newvaluesystems.
T os u m m a r i z e :

(1) Visualcommunication is alwayscoded.It seemstransparentonly because we know


at leastimplicitly- but withoutknowingwhat it is we know,without
the codealready,
The semiotic landscape

O fin f.O Contemporary


scienceCD-ROM

(e)

Q fig f.Z Earlytwentieth-century


sciencetextbook(McKenzie,Ig3g)

havingthe meansfor talkingaboutwhat it is we do whenwe readan image.A glance


at the 'stylized'artsof otherculturesshouldteachusthat the mythof transparency is
i n d e e da m y t h .W e m a ye x p e r i e n ct hee s ea r t s a s ' d e c o r a t i v e , , , e x o t i c , , ' m y s t e roiro u s ,
The semiotic landscape

'beautiful',but we cannotunderstand them as communication/ as formsof 'writing'


unlesswe are,or become, membersof thesecultures.
(2) Societies tendto developexplicitwaysfor talkingonlyaboutthosesemioticresources
w h i c ht h e yv a l u em o s th i g h l ya, n dw h i c hp l a yt h e m o s ti m p o r t a nrt o l e i n c o n t r o l l i n g
the commonunderstandings theyneedin orderto function.Untilnow,language, espe-
ciallywrittenlanguage, hasbeenthe mosthighlyvalued, the mostfrequentlyanalysed,
the mostprescriptively taughtandthe mostmeticulously policedmodein our society.
If, as we haveargued,this is now changingin favour of more multiplemeansof
representation, with a strongemphasison the visual,then educationalists needto
r e t h i n kw h a t w i l l n e e dt o b e i n c l u d e di n t h e c u r r i c u l ao f ' l i t e r a c y 'w, h a t s h o u l db e
i n s c h o o l sa,n dc o n s i d et rh e n e wa n ds t i l l c h a n g i n pg l a c eo f
t a u g h tu n d e ri t s h e a d i n g
writingas a modewithinthesenewarrangements.

If schoolsare to equipstudentsadequately for the newsemioticorder,if they are not to


producepeopleunableto usethe new resources of representation activelyand effectively,
t h e nt h e o l d b o u n d a r i ebse t w e etnh e m o d eo f w r i t i n go n t h e o n eh a n da, n dt h e ' v i s u aal r t s '
on the other,needto be redrawn.Theformerhad traditionallybeenthat form of literacy
withoutwhich peoplecouldnot adequately functionas citizensor as workers;the latter
had beeneithera marginalsubjectfor the speciallygifted,or a subjectwith limitedand
s p e c i a l i z eadp p l i c a t i o nas s, i n ' t e c h n i c adl r a w i n g 'T. h e n e w l yd e f i n e da r e aw i l l h a v et o
involve t h e t e c h n o l o g i eosf t h e ' n e ws c r e e n s ' -t h e e l e c t r o n itce c h n o l o g i eosf i n f o r m a t i o n
a n d c o m m u n i c a t i ocne, n t r a n l o wt o t h e s e m i o t i cl a n d s c a p Be .u t a b o v ea l l , s u c ha c u r r i c -
u l u mi s c r u c i a l l yd e p e n d e notn h a v i n gt h e m e a n so f a n a l y s i st h, e m e a n sf o r t a l k i n ga b o u t
the'new lileracy',aboutwhat it is we do whenwe produceand readimages.As Iedema
(I99 4 : 64) notes,in the'post-Fordist' workpIace,

W o r k e r sm u s tb e m u l t i - s k i l l e d
a ,r t i c u l a t ea n d ' s e l f - s t e e r i n .g.' . t T h e y Jn e g o t i a t e
t h e i rj o b sa s m e m b e rosf ' q u a l i t yc i r c l e s ' a n cdo n s u l t a t i vceo m m i t t e eTs h . i sr e q u i r e s
that workersare not merelycapableoI doingtheir work, but also that they are
capableof talkingandthinkingabouttheir work and its effectiveness.

Elsewhere (l(ress,2000;l(ressandvan Leeuwen,200L) we havetalkedof the needfor the


introductionof the conceptof design,both as a categorywith generalsignificance in
r e p r e s e n t a t iaon dc o m m u n i c a t i oann,da s a c r u c i a cl a t e g o rfyo r d e v e l o p i nt h g ec u r r i c u l a
o f i n s t i t u t i o n a l i z e d u c a t i o nw,h e t h e ri n t h e t r a d i t i o n asl c h o o ol r o t h e rf o r m a ls i t e so f
learning. Thisis implicitalsoin the description we havegivenearlierin this chapterof the
newformsof reading.This is not the placeto developthat point,thoughit is essential to
draw attentionto its unavoidable significance as part of the urgentneedfor developing
a d e q u a tw e a y sa n dt a l k i n ga b o u t h ev i s u a l .
Thesemiotic landscape ta

T H E S E M I O T I CL A N D S C A P E

Theplaceof visualcommunication in a givensocietycanonlybe understood in the context


of, on the onehand,therangeof formsor modesof public communication available in that
societyand,on the otherhand,their usesand valuations. We refer to this as 'the semiotic
landscape'. Themetaphoris worth exploringa little,as is its etymology. Thefeaturesof a
( a
l a n d s c a p e f i e l da , w o o da , c l u m po f t r e e sa, h o u s ea, g r o u pof b u i l d i n g so)n l ym a k es e n s e
in the contextof their wholeenvironment and of the historyof its development ('waste
l a n d ' h a sm e a n i nogn l yi n t h a t c o n t e x a h a
t ,s h a s ' f i e l d ' o r ' t r a c k ' ; ' v i l l a g e 's m e a n i nogn l ya s
a group of buildings that is part of a historyof waysof workingthe land). In the same way,
p a r t i c u l a fr e a t u r e sa n d m o d e so f c o m m u n i c a t i osnh o u l db e s e e ni n t h e h i s t o r yo f t h e i r
develooment. and in the environmentof all the other modesof communication which
surroundthem.The useof the visualmodeis not the samenow as it was even fifty years
ago in Westernsocieties; it is not the samefrom onesocietyto another;and it is not the
samefrom onesocial group or institutionto another.
Eachfeatureof a landscape hasits history,as doesthe landscape as a whole,andeach
is subjectto constantremaking.It is herethat the etymologyof the word landscape is
r e v e a l i n gT.o t h e c a s u a b l e h o l d ear l a n d s c a psei m p l yi s , a n d m a y e v e nh a v e t i m e l e s s a
a p p e a r a n c( 'et h et i m e l e sbse a u t yo f t h e E n g l i s ho,r S p a n i s hc ,o u n t r y s i d eY ' )e. ti t i s i n f a c t a
productof socialactionand of a socialhistory,of humanwork on the land,on nature:
-scape,withits relationto shapein Englishandschaffen(both'to work'and'to create')in
G e r m a ni n, d i c a t et sh i s .A n dt h i sa p p l i eas l s ot o t h e' s e m i o t i lca n d s c a p eM' .e t a p h o r iecx c u r -
s i o n so f t h i sk i n dc a nb es t r e t c h etdo o f a r ; h o w e v e r , wwei l l a l l o wo u r s e l v eosn eo t h e rp o i n t
of comparison. Landscapes are the result,not just of humansocialwork, but alsoof the
characteristics of the land itself.Theflat landby the riveris mostsuitablefor the grazing
of cattleor the growingof wheat;the hillsides for vineyards or forestry.At the sametime,
the characteristic valuesof a culturemaydetermine whichof the potentialusesof the land
are realized,whetherthe hillsidesare usedfor vineyardsor forestry,for example.And
c u l t u r avl a l u e sm a ye v e ni n d u c ep e o p l e t o g o a g a i n stth e g r a i no f t h e l a n dt,o u s et h es t e e p
h i l l s i d ef o r g r o w i n gr i c e ,f o r e x a m p l ew, h i c ho p p o s etsh e ' n a t u r a lp o t e n t i a l ' o ft h e l a n d
a l m o st o t h el i m i t .
Semioticmodes, similarly, areshapedbothby the intrinsiccharacteristics andpotential-
i t i e so f t h e m e d i u ma n d b y t h e r e q u i r e m e n thsi,s t o r i eas n d v a l u e so f s o c i e t i easn d t h e i r
c u l t u r e sT.h ec h a r a c t e r i s t iocfst h e m e d i u mo f a i r a r en o tt h e s a m ea st h o s eo f t h e m e d i u m
of stone,andthe potentialities of the speech organsare notthe sameasthoseof the human
h a n dN . e v e r t h e l e s s , c u l t au nr adsl o c i avl a l u a t i o nasn ds t r u c t u r esst r o n g l ay f f e c t t h eu s e so f
thesepotentialities. It is not an accidentthat in Westernsocieties writtenlanguage hashad
the p l a c ew h i c hi t h a sh a df o r t h e l a s tt h r e eo r f o u r m i l l e n n i aa,n dt h a t t h ev i s u am l o d eh a s
in effect becomesubservient to language, as its modeof expression in writing.Western
s a v em o r eo r l e s sn a t u r a l i z et dh e v i e wt h a t t h e u s eo f a i r a n dt h e v o c a l
l i n g u i s t itch e o r i e h
organsis the natural,inevitablesemioticmeansof expression. But evenspeechis, in the
e n d ,c u l t u r a lW . e a r e n o t b i o l o g i c a l lpyr e d i s p o s et od u s es p e e c ha s o u r m a j o r m o d eo f
communication. Thepartsof the bodythat we callthe tspeech organs'arean adaptation of
The semiotic landscape

p h y s i c aol r g a n si n i t i a l l yd e v e l o p et d o p r e v e nht u m a n sf r o m c h o k i n gw h i l eb r e a t h i n a gn d
eating.Whenthe needarises,we cananddo useothermeansof expression, as in the highly
articulated evelopmen o tf g e s t u r ei n s i g n l a n g u a g eas n , d a l s o i n t h e a t r i c am l i m ea n d
certain Easternforms of ballet.And, while theseare at presentrestrictedto relatively
m a r g i n adl o m a i n sw, h o i s t o s a yt h a t t h i sw i l l a l w a y sr e m a i ns o i n t h ef u t u r ed e v e l o p m e n t
o f h u m a n k i n dI?t i s s a l u t a r yt o c o n s i d ehr o wo t h e rc u l t u r e s ' r a n k ' m o d eosf c o m m u n i c a -
tion, and to bring that knowledgeinto the mainstreamof 'Western'thinking(see,for
instance, Finnegan, 2002).
T h e n e wr e a l i t i e o s f t h e s e m i o t i cl a n d s c a paer e b r o u g h a t b o u tb y s o c i a lc, u l t u r a al nd
e c o n o m i cf a c t o r s :b y t h e i n t e n s i f i c a t i oonf l i n g u i s t i ca n d c u l t u r a ld i v e r s i t yw i t h i nt h e
boundaries of nationstates;by the weakening of theseboundaries withinsocieties, dueto
m u l t i c u l t u r a l i seml e, c t r o n imc e d i ao f c o m m u n i c a t i ot e n c, h n o l o g i oe fst r a n s p o rat n dg l o b a l
e c o n o m idce v e l o p m e nG t sl.o b afl l o w so f c a p i t a a l n d i n f o r m a t i oonf a l l k i n d so, f c o m m o d -
i t i e s ,a n d o f p e o p l ed, i s s o l v ne o t o n l yc u l t u r a a l n d p o l i t i c a bl o u n d a r i ebsu t a l s os e m i o t i c
boundaries. Thisis alreadybeginning to havethe mostfar-reaching effectson the charac-
teristico s f E n g l i s h( a n dE n g l i s h e g s )l o b a l l ya,n de v e nw i t h i nn a t i o n abl o u n d a r i e s .
T h ep l a c eu, s ef,u n c t i o na n dv a l u a t i o on f l a n g u a gien p u b l i cc o m m u n i c a t i oi snc h a n g i n g .
I t i s m o v i n gf r o m i t s l o r m e ru, n c h a l l e n g e r odl ea s t h em o d eo f c o m m u n i c a t i o n ,at or o l ea s
one modeamongothers,to the function,for instance, of beinga modefor comment,for
r a t i f i c a t i o no,r f o r l a b e l l i n ga,l b e i tm o r es o i n s o m ed o m a i n st h a n i n o t h e r sa, n d m o r e
r a p i d l yi n s o m ea r e a st h a ni n o t h e r sA. l t h o u g h t h i si s a r e l a t i v e lnye wp h e n o m e n oi nnp u b l i c
c o m m u n i c a t i ocnh,il d r e nd o i t q ui t e ' n a t u r al yl ' i n t h e i rt e x t - m a k i n g .
N e w w a y so f t h i n k i n ga r e n e e d e di n t h i s f i e l d .H e r ew e u s e /o n c em o r e ,c h i l d r e n , s
representation as a metaphorto suggestsomedirections.The drawingsreproducedin
f i g u r e1 . 8 w e r em a d eb y a f i v e - y e a r - o bl do y . 0 n a s u m m e rS u n d a ya f t e r n o o nw, h i l eh i s
parentswereentertaining friends,the childtook a small,squarenotepadfrom nearthe
t e l e p h o naen dd r e wa p i c t u r eo n e a c ho f s i x p a g e sH. i s f a t h e rh a dn o t n o t i c e d t h i s u n t i lh e
c a m ea c r o s sh i m i n t h eh a l lo f t h e i rh o u s ew, h e r et h ec h i l dw a sp u t t i n gt h ec a r d s' i n o r d e r ' ,
as shownin figure1.8. Askedwhat he was doing,the child'saccountwas as follows:for
picturesI and2 together'Meandthe dog are in life,so they'rein the correctorder,;on
picture3 s a n d4 ' T h ef f y i n gb o m bi s i n t h e a i r a n dt h e p l a n ei s i n t h e a i r ,s o t h e y , r ei n t h e
correctorder';andon 5 and6'The patternsare in the correctorder,.
The wholeprocess/ involvingsign-making, representation and classification, had pro-
c e e d e dt h r o u g ht h e v i s u a lm e d i u mI.t w a s o n l y w h e nt h e p a r e n tc a m ea l o n gw i t h h i s
questionthat the childwasforcedto usewords.Themetaphoric processes of sign-making,
the actsof representation and classification, eachinvolving quite complexanalogies, took
place in the visual mode.Language, as speech,enteredwhen communication with the
parentbecamenecessary. Speech wasthe modeusedfor'ratifying'andfor describing what
hadtakenplacewithoutit.
Sometwo weekslater,at the end of the summerterm of his primaryschool,the child
b r o u g h ht o m es o m eo f h i se x e r c i sbeo o k sA. m o n gt h e s ew a st h e p a g es h o w ni n f i g u r e1 . 9 .
Clearly, herethe taskwasoneof classification, and it hadbeenundertaken at school,prior
t o t h e m a k i n ga n d o r d e r i n go f t h e d r a w i n g si n f i g u r e1 . 8 ,a t h o m e A . w h o l es e q u e n coef
The semiotic landscape 37

4,,.
/ t t p
l*gL
\a'r
" fu uu^'
. . M EA N O
T H E D O GA R EI N L I F E ,S OT H E Y ' R EI N T H E C O R R E CO
TRDER"

. . T H EF L Y I N GB O M B
I S I N T H EA I R A N DT H E P L A N ET SI N T H EA I R ,S OT H E Y ' R EI N T H E C O R R E CO
TRDER''

Y-

/U,N r,zN!,1

, , T H EP A T T E R N S
A R EI I { T H E C O R R E CO
TRDER''

O Fig 1.S Six drawingsby a five-year-old


boy

semioticactivitiesis thus involved, a sequence of production, transformation and develop-


ment,movingfrom the initialtask of joiningimagesof the sameobjects- a classificatory,
cognitive, conceptual, semioticand manualtask - to that of producingcomplexand dis-
s i m i l a ri m a g e sa, n d f i n d i n gl i k e n e sisn t h e m ( o r i m p o s i n g l i k e n e sosn t h e m )t h r o u g ha n
intermediary task of abstractionand generalization. If we think aboutthis periodof two
w e e k st,h e c h i l d ' sp r o d u c t i oonf s i g n si n v o l v e ad s e r i e so f d i s t i n c st e m i o t i cm o d e sa, n d o f
translations betweensuchmodes.Firstthe teachersnokewith the childrenaboutthe task
(mode:language as speech); thensheintroduced the bookand showedthemwhat was at
i s s u e( m o d e : 3 Dp h y s i c aol b j e c ta, n dv i s u a m l o d e )t;h e nt h e c h i l d r e nu s e dt h e i rp e n c i ltso
d r a wt h e c o n n e c t i nlgi n e s( m o d e m : anuaa l c t i o na n d v i s u a lm o d eo f d r a w i n g )t;h e nt h e
teacherengaged the childrenin spokendiscussion, and madeevaluative comments on their
38 The semiotic landscape

15 { /- \- \ 1/s.

.\\L ll APt
Drowo linet6 jbin thethingswhich

Q fig f.e Schoolexercise


bookof afive-year-otd
boy

work.Thiswasfollowedby a long periodof 'silence',a fortnightor so whennothingwas


s e e no r h e a r db, u t w h e n , w a e s s u m e , t hsee r i e o s f t r a n s f o r m a t i vaec t so f t h e c h i l dc o n t i n u e d
'internally','mentally'.
F i n a l l yt h e i n t e r n aal c t i v i t yb e c a m ev i s i b l el,i t e r a l l yt ,h r o u g ht h e
child'sunpromptedproductionof the drawings,his unpromptedclassificatoryactivity
( s p a t i a l lsyh o w na) n dh i ss p o k e n c o m m e n t a riyn r e s p o n st e o h i sf a t h e r , q s uestion.
0 f c o u r s ew, h i l ea l l t h i s t o o k p l a c et h e c h i l d ,a s d o a l l o f u s ,w o u l dn o d o u b th a v e
experienced constantly shiftingaffective, emotionalstates.Hemighthavebeenenthused by
thetask in the classandpraisedby theteacherfor hissuccess; he mighthavehada difficult
t i m ew i t h h i sf r i e n d si n t h e p l a y g r o u n o d r, a t h o m ea, n ds o o n ,a n da l l o f t h i sw o u l dh a v e
i n f f u e n c ehdo wh e ' r e a d ' t h ea c t i v i t ya n dh o wi t w a s ' t a k e nu p ,b y h i m .I f w e s e ei t l i k et h i s ,
The semiotic landscape

it makesit impossible to think of affect and cognitionas distinct,as separable. In other


words,here- asalways- the affectiveaspects arealwaysonewith,andact continuously as
a ' m o d a l i t yo' n ,c o g n i t i vsee m i o t i p crocesses.
In part in response to the representational, semioticand cognitiveresources made
availableby the teacher, and her demandsmadein the class,thoughafterwardsprompted
by hisown interests, the childuseda seriesof differentrepresentational modes(including,
of course,'internal representations') in a constantly productive sequence of semioticactiv-
i t i e s .S o m eh a p p e n ewdi t h i nt h e s a m em o d e( l i n k i n gt h e i m a g e o s f t h e o b j e c tb y a l i n ef,o r
instance), sometook placeby a shift acrossmodes(theshiftfrom the spatiallyperformed
classification to the spol<en commentary on it). Suchprocesses are constantly transforma-
tive (thenamewe usefor suchorocesses withinonemode)andtransductive (our namefor
such processes acrossmodes).All these,we assume/haveeffectson 'inner resources',
whichconstantlyreshape(transform)the subjectivity of the child.
As we haveindicated, the visual,actionalandspatialmodes,ratherthanspeech/ seemed
to be the centralrepresentational andcognitiveresources. Speechwas usedfor communi-
cationwith adults,as a meansfor translation, for commenlandfor ratification. It maywell
b e t h a t t h e c o m p l e x i t i er es a l i z e di n t h e s i x i m a g e sa n d t h e i r c l a s s i f i c a t i owne r ei n i t i a l l y
beyondthe child'scapacityof spokenexpression, conception andformulation, but that the
v i s u am l o d eo f f e r e dh i ms e m i o t i a c n dc o g n i t i v ree s o u r c ewsh i c hw e r en o t a v a i l a b lteo h i m
in the verbalmode.Howeverl onceexpressed in the visualmode,onceclassified throughthe
visual/spatia ml o d e ,t h e m e a n i n gw s h i c ht h e c h i l d h a d p r o d u c e db e c a m ea v a i l a b l ea s
externalized, objectiveexpression; this in turn may havemadethem differentlyavailable
for verbalexpression, for the verbalratificationof semiotic,affective/cognitive processes
that had alreadytaken place.
This incessantprocessof 'translation',or 'transcoding' - transduction - betweena
rangeof semioticmodesrepresents, we suggest, a better, a moreadequate understanding of
representation andcommunication. In the examplewe havediscussed here,language is not
at the centre.In manyareasof publiccommunication the sameis eitherthe casealready,
o r r a p i d l yc o m i n gt o b et h e c a s eA. n dc l e a r l yi t m a t t e r sw h i c hs e m i o t i m c o d e so f r e p r e s e n -
t a t i o na n dc o m m u n i c a t i oa nr ed o m i n a n m t , o s tf r e q u e n mt , o s tv a l u e di n t h e p u b l i cd o m a i n s
in whichwe act.
F i g u r e1 . 1 0 c o m e sf r o m w o r k d o n eb y t w o ( 1 3 - y e a r - o l ds)t u d e n t isn S c i e n c ei n, t h e
earlyyearsof secondary schoolin England. Twoquestions canbe asked.Thefirst is,What
is the effectof the modeof representation on the epistemology of science?, andthe second:
'Do differentmodesof representation facilitate,or rule out, differentaccountsof natural
phenomena?' To answerboth,we needto comparetwo differingmodalrepresentations of
'the same'issue.
T h i st i m eo u rq u e s t i oins n o t ' W h a ti st h es t a t u so f w r i t t e nl a n g u a gien t h e s et e x t s ?b' u t
r a t h e r ' W h a ti s t h e e f f e c to f t h e d i f f e r e nm s t e r m so f e p i s t e m o l o gi n,
t o d a lr e a l i z a t i o ni n
termsof the students'perspective on knowledge?' If we comparefigure1.10 with just one
examplefrom anotherexercise for assessment - the task'to write a storyof the journeyof
a r e db l o o dc e l la r o u n dt h e b o d y ' - w e c a ns e et h e r u d i m e n tosf t h a t d i f f e r e n c eH.e r ei s a
briefextractfrom onesuch'story',in this casewrittenin the genreof'diary':
Thesemiotic landscape

sr\4oKrNG
y" Drooogoes
\
\
/
/- the blood rerums t;;]
to the hean th@sh
I I 1Yi{1'9T
the hean \
\ /
/
rhe verns
In aneites

o'oo'""" - +
\
I HEARrI F;;;l
EilI
Y

rroNoxrpE
\
rhe head
pumps rhe
\ blood around
,/
,/ ..
1,

ne
I,
..
cELLs

oo@ conrarns
'I |-;I
ooav
I I \v. ,,/
I maKe
\ '""un" I ruruesI Y
I l----.-.-- | eLooo| -=>

[;;I
Y i n
serslxysenr,om /a-=
n;ii,;:a:"
the lunos
//
*
/ |
I contains
| *",r. u.*o I
|
cELis
I F'--I
---a ,/
/ I
{"on,u,n"
+ I ,*" f "n"ro"
maoJin
\ ,, ,o,'"0 broodsetsl
i o t ',1""Yrfl"
l"*tto I
intestines
I t--_l

lnNrraooresl |;;;I
[;;;]
+
G;t
I
..
DroxrDE w;;
I
d ' ' productof
respiration

@ fiq f.fO Conceptmapr'bloodcirculation'

DearDiary,I havejust leftthe heart.I hadto comefrom thetop ofthe rightchamber


of the heart( Rightatrium)andsqueeze my waythroughto the Rightventriclewhere
the heartbeatgot stronge4 and I left the heart.
Dear Diary,I am currentlyin the lungs,it is terriblycrampedin hereas the
capillaries aretiny andthereare millionsof us.We havejust droppedoff oxygenand
w e p i c k e du p s o m ec a r b o nd i o x i d e .
DearDiary,we haveenteredthe Iiverwherewe hada thoroughwash.
DearDiary,we havejust left the kidneywherewe droppedoff somewaterwhich
w i l l b et u r n e di n t ou r i n e .
Dear Diary,I havefinishedmy journeyaroundthe bodyby stoppingoff at the
heart.

In the diary,thefundamental organizationalprinciple, or logic,isthat of 'sequencein time,,


andthefundamental representational principleis that of actionor eventor,lessfrequently,
statesof affairs.Objectsare relatedto other objectsby actionsrepresented by verbs('we
haveenteredthe liver','wehavejust left the kidney').Theactionsand eventsthemselves
are arrangedin temporalsequence/ mirroringthat of the imagined eventsastheyhappened
in the world. In the conceptmap,on the other hand,the fundamentalorganizational
principleis that of a conceptualorder,realizedby the spatial arrangements of the
The semiotic landscape 4t

individualconcepts. Hereobjectsare related,not by actions,but by hierarchy, by signifi-


c a n c ed e r i v i n fgr o m r e l a t i o no f ' p r i o r i t y o
' f v a r i o u sk i n d s .
All theseexamples revealwhat has in fact alwaysbeenthe case:language, whetherin
speechor writing,hasalwaysexistedas just onemodein the ensemble of modesinvolved
in the productionof texts,spokenor written.A spokentext is neverjust verbal,but also
visual,combiningwith modessuchas facial expression, gesture,postureand otherforms
of self-presentation. A written text, similarly,involvesmore than language:it is written
on something, on somematerial(paper,wood,vellum,stone,metal,rock,etc.) and it is
written with something(gold,ink, (en)gravings, dots of paint,etc.);with lettersformed
as typesof font, influenced by aesthetic,psychologicai, pragmaticand other consider-
ations;and with layout imposedon the materialsubstance, whetheron the page,the
computerscreenor a polishedbrassplaque.Yet the multimodalityof writtentexts has,
by and large,beenignored,whetherin educational contexts,in linguistictheorizingor in
p o p u l a rc o m m o ns e n s eT. o d a yi,n t h e a g e o f ' m u l t i m e d i ai' t, c a n s u d d e n lby e p e r c e i v e d
again.

W e c a n s u m m a r i zteh i s d i s c u s s i oi n t h e f o r m a s e t o f h y p o t h e s e(sa:) h u m a ns o c i e t i e s
usea varietyof modesof representation; (b) eachmodehas,inherently, differentrepre-
sentationalpotentials,different potentialsfor meaning-making; (c) each mode has
specificsocialvaluationin particularsocialcontexts;(d) differentpotentialsfor meaning-
m a k i n gm a y i m p l yd i f f e r e npt o t e n t i a lfso r t h e f o r m a t i o no f s u b j e c t i v i t i e(se;) i n d i v i d u a l s
usea rangeof representational modes,and thereforehaveavailablea rangeof meansof
meaning-making, eachaffectingthe formationof their subjectivity; (f) the differentmodes
of representation are not held discretely, separately, as strongly boundedautonomous
d o m a i n isn t h e b r a i no, r a sa u t o n o m o ucso m m u n i c a t i o nr a e ls o u r c ei ns c u l t u r en, o ra r et h e y
deployed discretely, eitherin representation or in communication; (g) affectiveaspectsof
humanbeingsand practicesare not discretefrom othercognitiveactivity,and therefore
neverseparateor absentfrom representational and communicative behaviour; (h) each
modeof representation hasa continuously evolvinghistory,in whichits semanticreachcan
contractor expandor moveinto differentareasof socialuseas a resultof the usesto
w h i c hi t i s p u t .
None of these hypotheses would, we imagine,attract significantdisagreement,
e s p e c i a lwl yh e np u t s i n g l yJ.o i n t l yt h e yr e p r e s e natc h a l l e n gteo t h ee x i s t i n cgo m m o ns e n s e
on the relationsbetweenlanguage andthoughtand in mainstream theoriesand practices
i n a l l a r e a so f p u b l i cc o m m u n i c a t i o T nh. i s i s a c r u c i a lf e a t u r eo f t h e n e w s e m i o t i c
landscape.

A N O T EO N A S O C I A LS E M I O T I CT H E O R YO F C O M M U N I C A T I O N

i n o r d e rt o f u n c t i o na s a f u l l s y s t e mo f c o m m u n i c a t i ot h
ne, v i s u a ll,i k ea l l s e m i o t i m
c odes,
hasto serveseveralrepresentational andcommunicational requirements. have We adopted
t h e t h e o r e t i c anl o t i o no f ' m e t a f u n c t i o n ' f r o m
t h e w o r k o f M i c h a e lH a l l i d a yf o r t h i s
42 . The semiotic landscape

purpose. Thethreemetafunctions whichhe positsare the ideational,the interpersonal and


t h e t e x t u a l . l nt h e f o r m i n w h i c hw e g l o s st h e m h e r et h e ya p p l yt o a l l s e m i o t i cm o d e s ,
andare not specificto speechor writing.

The ideationalmetafunction

Any semioticmodehasto be ableto represent aspectsof the worldas it is experienced by


s n dt h e i rr e l a t i o nisn a w o r l d
. o t h e rw o r d si,t h a st o b ea b l et o r e p r e s e notb j e c t a
h u m a n sI n
outsidethe representational system.Thatworld may of coursebe,and mostfrequentlyis,
alreadysemiotical ly represented.
In doingso,semioticmodesofferan arrayof choices, of differentwaysin whichobjects,
andtheir relationsto otherobjectsandto processes, can be represented. Twoobjectsmay
be represented as involvedin a processof interactionwhichcouldbe visuallyrealizedby
vectors:

O Fiq l.tl vector

But objectscanalsorelatedin otherways,for instancein termsof a classification.


They
not by a vectorbut,for instance,
wouldbe connected, by a'tree'structure:

O rlg f.fZ Treestructure

In chapters2 and3 we will investigate precisely


whichideationalchoicesare available
f o r v i s u asl i g n - m a k i ni n
g t h i sw a y .

The interpersonalmetafunction
Any semioticmode has to be able to projectthe relationsbetweenthe producerof a
(complex)sign,andthe receiver/reproducer of that sign.Thatis,anymodehasto beableto
representa particularsocialrelationbetweenthe producer, the viewerand the object
represented.
As in the caseof the ideationalmetafunction, modesoffer an array of choicesfor
r e p r e s e n t i nd gi f f e r e n t ' i n t e r p e r s o nr ealla' t i o n s ,o m eo f w h i c hw i l l b ef a v o u r e d
i n o n ef o r m
of visual representation (say,in the naturalisticimage),othersin another(say,in the
The semiotic landscape . 43

d i a g r a m )A. d e p i c t epde r s o nm a yb es h o w na sa d d r e s s i nvgi e w e rds i r e c t l yb,y l o o k i n ga t t h e


camera.This conveysa senseof interactionbetweenthe depictedpersonand the viewer.
But a depictedpersonmayalsobeshownasturnedawayfrom the viewer, andthisconveys
the absenceof a senseof interaction. It allows the viewer to scrutinize the represented
characters as thoughtheywerespecimens in a display case.
In chapters4 and5 we will discuss theseandotherinterpersonal choices, bothin terms
of the kindsof interactions that canbe represented, and in terms of the visual featuresthat
realizetheseinteractions.

The textual metafunction

Any semioticmodehasto havethe capacityto form fexfs,complexes of signswhichcohere


both internallywith eachotherand externally with the contextin and for whichtheywere
produced. Here,too,visualgrammarmakesa rangeof resources available: differentcom-
positionalarrangements to allow the realizationof differenttextualmeanings. In figure
1 . 1 ,f o r e x a m p l et h, e t e x t i s o n t h e l e f t a n dt h e p i c t u r eo n t h e r i g h t .C h a n g i n g e l a y o u t
t h
(figure1.13) would completelyalter the relationbetweenwritten text and imageand
the meaningof thewhole.Theimage,ratherthanthewrittentext,wouldnowserveas point
o f d e p a r t u r ea,s ' a n c h o r ' f o rt h e m e s s a g eI n. c h a p t e r6 w e w i l l d i s c u s s u c hl e f t - r i g h t
r e l a t i o n s h ia
pns do t h e rc o m p o s i t i o nrael s o u r c e s .

: e . ,
'
" , i" : , " .j t- ' "i .. , '

.j
Y eli*hr
i ir*,r* lY:.y ** il:
h*{* r*. 1 qr:
I* **lj

@ fig f.fl Alteredlayoutof tigurel.I (left-right reversal)


44 . The semiotic landscape

Ourfocusis on the description of theseideational, interpersonal andtextualresources


as they are realizedin the visualmode.We recognizethat in doingthis work we are
engagedin morethan 'meredescription', and participateourselves in the reshaping of the
s e m i o t i cl a n d s c a paen; dw e r e a l i z ea l s ot h a t t h i s i s a h i g h l yp o l i t i c ael n t e r p r i s e .
2 N a r r a t i v er e p r e s n
etations:
d e s i g n i n sg o c i a a
l ction

INTRODUCTION

T h e p i c t u r e ss h o w ni n f i g u r e2 . 1 a r e t a k e nf r o m a n A u s t r a l i a np r i m a r y - s c h osool c i a l
s t u d i e tse x t b o o l(<Q a k l e ye t a 1 . , 1 9 8 5 ) . 0 n er e p r e s e n t h s e t r a d i t i o n atle c h n o l o goyf t h e
n b o r i g i n etsh,e o t h e rt h e s u p e r i otre c h n o l o goyf t h o s ew h o i n v a d etdh e i rt e r r i -
A u s t r a l i aA
tory ('The Britishhad a technology that was capableof changingthe face of the earth.
Theirtoolswereableto work fasterthanthoseof the Aborigines and their weapons were
muchmorepowerful'). Theformerhasthreemainelements (an axe,a basketanda wooden
s w o r d ) t, h e l a t t e rf o u r ( t h e ' B r i t i s h ' ,a s t h e y a r e c a l l e di n t h e c a p t i o nt,h e i r g u n s t, h e
A b o r i g i n easn dt h e l a n d s c a p eB) .u t t h e t w o p i c t u r e sd i f f e rn o t o n l yi n w h a te a c hi n c l u d e s
and excludes(the left picture,for instance, excludes the usersof the technology, the right
pictureincludesthem),they differ also in structure:they relatetheir elementsto each
other differently. The elementsof the left pictureare arrangedsymmetrically, againsta
neutralbackground: axe, basket and wooden sword are represented as equal in size, placed
at equaldistance from each other and oriented in the same way towards the horizontal
andthe verticalaxes,sothat the pictureas a wholecreatesa relationof similaritybetween
the threeelements. The picturesays,as it were,that this axe,this basketandthis wooden

O Fig2.1TheBritishusedguns(0akleyeta/.,1985)
Narrative representati ons

swordall belongto the sameoverarching category(a category, incidentally, which is only


implied,andwhichconflates the notionof 'tools,andthe notionof 'weaoons,).
Theright picturerepresents technology in action.Wherethe left pictureis impersonal,
t h i s p i c t u r ei s p e r s o n aW l . h e r et h e l e f t p i c t u r ei s s t a t i c t, h i s p i c t u r ei s d y n a m i cW . here
t h e l e f t p i c t u r ei s d r y a n d c o n c e p t u atlh, i s p i c t u r ei s d r a m a t i c I. t r e l a t e st h e B r i t i s h
a n dt h e A b o r i g i n etsh r o u g ha t r a n s a c t i o n a s cl h e m ai n w h i c ht h e B r i i i s hp l a yt h e r o l eo f
'Actor',the oneswho the deed,
do andthe Aborigines the role of 'Goal,,the onesto whom
the deedis done-the British stalktheAborigines, onecouldsay.It alsorelatesthe land-
s c a p et o t h e B r i t i s ha n dt h e A b o r i g i n ei sn a ' l o c a t i v e ' w a y( t h e B r i t i s ha n dt h e A b o r i g i n e s
are in lhe landscape), and the gun to the British in an 'instrumental'wav (the British
stalk the Aborigineswith their gun).
Theserelationscan be transformedinto linguisticform,as we havejust done,but the
pointis that heretheyare realizedby visualmeans.Thetransactional relationbetween the
B r i t i s ha n dt h eA b o r i g i n ei s r e a l i z e bd y t h ev e c t o r t h alti n k s t h e mn,a m e l y t hoeb l i q u el i n e s
formedby the glancesandoutstretched armsof the Britishand by their guns.Thelocative
relationis realizedby overlapping, by the gradientsof focus,the degreesof coloursatur-
ation and so on,whichcreatethe contrastbetweenforegroundand background. And the
i n s t r u m e n t rael l a t i o ni s r e a l i z e b
d yt h e g e s t u r e o f h o l d i n gw, h e r et h e o b j e c th e l di s a t o o l .
The importantpointat this stageis not the detailof the analysis, but the observation
that the semioticmodesof writing and visualcommunication eachhavetheir own quite
p a r t i c u l a rm e a n so f r e a l i z i n gw h a t m a y b e q u i t e s i m i l a rs e m a n t i cr e l a t i o n sW . hat in
I a n g u a gies r e a l i z e bd y w o r d so f t h ec a t e g o r y ' a c t i ovne r b si's v i s u a l l rye a l i z e b d ye l e m e n t s
that can be formallydefinedas vectors. What in languageis realizedby locativepreposi-
tions is visuallyrealizedby the formal characteristics that createthe contrastbetween
foregroundand background. This is not to saythat all the relationsthat can be realized
l i n g u i s t i c a lcl ya na l s ob e r e a l i z e vdi s u a l l -y o r v i c ev e r s at,h a t a l l t h e r e l a t i o ntsh a t c a nb e
realized v i s u a l l yc a na l s ob e r e a l i z e dl i n g u i s t i c a l lRy a
. t h e ra, g i v e nc u l t u r eh a sa r a n g eo f
g e n e r a pl ,o s s i b lree l a t i o nw s h i c hi s n o tt i e dt o e x p r e s s i oi n a n yp a r t i c u l asr e m i o t i cm o d e ,
although s o m er e l a t i o ncsa no n l yb er e a l i z evdi s u a l l a y n do t h e r so n l yl i n g u i s t i c a lol yr ,s o m e
m o r ee a s i l yv i s u a l l ya n d o t h e r sm o r ee a s i l yl i n g u i s t i c a l lTyh. i sd i s t r i b u t i oonf r e a l i z a t i o n
possibilitie a sc r o s tsh es e m i o t i m c o d e si s i t s e l fd e t e r m i n ehdi s t o r i c a l layn ds o c i a l l a y sw e l l
as by the inherentpotentialities and limitationsof a semioticmode.
To returnto thetwo picturesin figure2.I,they canbesaidto represent an aspectof the
experiential world,technology. But throughthe differentdesignpatternsselectedin each,
t h r o u g ht h e m a n n e ri n w h i c he a c hb r i n g si t s i n d i v i d u aell e m e n ttso g e t h e irn t oa c o h e r e n t
and meaningful whole,they represent the technologyof Aborigines very differentlyfrom
the technologyof the British.Nothingabout Aboriginaltechnologynecessitates it to be
represented as a static,conceptual taxonomy, nor is thereanythingintrinsicabout British
technology that requiresit to be represented in a personalized anddramatized way.British
technologyis just as capableof beingrepresented by a classificatory schemeas is Abo-
r i g i n atl e c h n o l o gayn, d i t i s j u s t a s p o s s i b lteo t e l l d r a m a t i cs t o r i e so f A b o r i g i n esst a l k i n g
their invaders with spearsor woodenswordsas it is to tell suchstoriesaboutthe British.
evenif the former kind of story doesnot usuallyform part of mainstreamAustralian
Narrativerepresentations' 47

history.In otherwords,the representation is mediated, visually, throughtwo distinctdis-


- ' k n o w n o h i s t o r y 'a; n dt h a t o f
c o u r s e st h: a t o f a n t h r o p o l o gf oy r A b o r i g i n apl e o p l e w h o
-
historyfor the whites who are not subjectsof anthropology.
imaginea reversalof thoserelations.Imagineon the left a catalogueof Britishtools
a n dw e a p o n sa ,n do nt h e r i g h ta p i c t u r ei n w h i c hA b o r i g i n epso i n t h e i rw o o d e ns w o r d sa t a
s m a l lg r o u po f B r i t i s hi n t h e b a c k g r o u nS du. d d e n lay r e p r e s e n t a t ioofnc o l o n i z a t i oanst h e
t r a n s i t i o nf r o m a f i x e d ,s t a b l e( ' p r i m i t i v e 'o) r d e ro f t h i n g st o t h e d y n a m i cu n f o l d i n g of
h i s t o r yi s c h a n g e idn t os o m e t h i nlgi k et h e r e v e n goef t h e ' p r i m i t i v e ' o tnh e W e s t ' t
s e c h n o-
logicalorder.This may be suitablefor a fictionfilm, perhaps, set safely in an apocalyptic
future,but not for a primary-school textbookin contemporary Australia.
Thetwo designpatternsin figure2.I,the classificatory patternand the transactional
pattern,are only two of severalpossiblepatterns.In the courseof this chapterwe will
introduceothers,andtry to givean overviewof the visualstructuresthat can realizeways
of representing theworld.0uremphasis is not on depiction, nor on the questionof recogni-
tion,on how we cometo seeconfigurations of pencilmarksor brushstrol<es or pixelsas
p i c t u r e so f t r e e s ,o r o n h o w p i c t u r e so f t r e e sm a y c o n n o t eo r s y m b o l i z e a n i n gasn d
m e
valuesoverandabovewhatthey literallyrepresent. Thisaspectof the pictorialhasalready
r e c e i v ead g o o dd e a lo f a t t e n t i o ni n t h e w r i t i n g so f p h i l o s o p h e(res . g 'G o o d m a n1,9 6 9 ;
H e r m e r e n1, 9 6 9 ) , s e m i o t i c i a n(se . g . E c o , I 9 7 6 a ; B a r t h e s ,I 9 7 7 ) , m e d i a a n a l y s t s
( W i l l i a m s o n1,9 7 8 ) a n d a r t h i s t o r i a n (se . g .P a n o f s k yI 9, 7 O ) .A t t h i s s t a g eo f o u r w o r k
on visualcommunication we havelittleto addto what hasbeensaidin theseareas.
T h eq u e s t i oonf v i s u asl t r u c t u r i n g o ,n t h eo t h e rh a n dh, a s i, n o u ro p i n i o nb,e e nd e a l tw i t h
lesssatisfactorily. Visualstructuringhas either beentreatedas simplyreproducing the
structuresof reality(e.g.Metz,I974a,7974b), ratherthan as creatingmeaningful pro-
p o s i t i o nbs y m e a n so f v i s u a ls y n t a xo, r i t h a s b e e nd i s c u s s ei d n f o r m a lt e r m so n l y ( e ' 9 .
A r n h e i m1, 9 7 4 t 1 9 8 2 ,w h o i n h i s a c t u a la n a l y s eosf f e r sm a n yi n s i g h t so n t h e s e m a n t i c
d i m e n s i oonf v i s u a ls t r u c t u r i n g ) . 0 uer x a m p l eo f t h e r e p r e s e n t a t i o nf A b o r i g i n aal n d
Britishtechnology has,we hope,madeclearwhy neitherof theseapproaches satisfiesus.
Visualstructuresdo not simplyreproduce the structuresof'reality'.0n the contrary,they
produceimagesof realitywhichare boundup with the interestsof the socialinstitutions
w i t h i nw h i c ht h e i m a g e sa r e p r o d u c e dc,i r c u l a t e a d n d r e a d .T h e ya r e i d e o l o g i c aVl .i s u a l
structuresare nevermerelyformal:theyhavea deeplyimportantsemanticdimension.

PARTICIPANTS

In chaptert we definedthe ideationalmetafunction as the abilityof semioticsystemsto


represent objectsand their relationsin a world outsidethe representational systemor in
the semioticsystemsof a culture.In the introduction to the presentchapterwe tried to
showhow two designpatternscan producetwo differentrepresentations of broadlythe
' o b j e c t s ' e i l l ,f r o m n o wo n ,u s et h e
o r ' e l e m e n t s ' ww
s a m ea s p e cot f t h e w o r l d .I n s t e a do f
'particip anfs'er,moreprecisely, 'represented participants'. Thishastwo advantages:
term
'participantrn something';and it draws
it pointsto the relationalcharacteristicof
48 . Narrative representations

attentionto the fact that therearetwo typesof participantinvolvedin everysemioticact,


interactiveparticipantsand represented participants.The former are the participantsin
the act of communication - the participants who speakand listenor write and read,make
imagesor viewthem,whereasthe latter are the participantswho constitutethe subject
matter of the communication; that is, the people,placesand things(includingabstract
'things') represented
in and by the speechor writing or image,the participantsabout
w h o mo r w h i c hw e a r es p e a k i nogr w r i t i n go r p r o d u c i nigm a g e s .
Thesituationis of coursemorecomplexthan this,for the real interactive participants,
the real lmage-producers and -viewers, cannotbe takento be identicalwith the'implied,
p r o d u c ewr h o' s i l e n t l yi n s t r u c tus s ,t h r o u g ht h ed e s i g no f t h ew h o l e (' c h a t m a nr,9 7 B : r 4 B )
and the 'implied'viewer.It may alsobe that the producers and/orviewersare themselves
explicitlyrepresented in the image,causingthe two categories to shadeinto eachother,
complexities which havebeenstudiedextensively in the field of literarynarratology(e.g.
Iser,l97B; Bal,1985; Rimmon-l(enan,l-9B3). we will returnto thesein the nextchapter;
f o r t h e p u r p o s eosf t h i sc h a p t etrh e b a s i cd i s t i n c t i ow nill suffice.
In the caseof abstractvisualssuchas diagrams,it doesnot seemtoo difficult to
determine who or what the represented participantsare. Shannonand Weaver'sfamous
' c o m m u n i c a t i omno d e l (' f i g u r e
2 . 2 ) ,f o r i n s t a n c ei s, m a d eu p o f b o x e sa n da r r o w s( S h a n -
non and Weaver, 1949). The boxesrepresentparticipants(peopleand/orthings,the dis-
t i n c t i o nb e i n gb l u r r e db y o b j e c t i f y i nlga b e l sl i k e ' i n f o r m a t i osno u r c e ' a n d ' d e s t i n a t i o n ' ) ;
the arrowsrepresent the processes that relatethem.If we wantedto translatethis into
languagw e ,e c o u l ds a yt h a t t h e b o x e sa r e l i k en o u n st,h e a r r o w sl i k ev e r b s( e . g . ' s e n do,r
'transmit'),andthat,together,
theyform clauses(e.g.'aninformationsourcesendsIinfor-
mationlto transmitter').
In the caseof moredetailednaturalisticimages, however, it maybedifficult,evenfutile,
to try and identifythe represented participants. Take'TheBritishusedguns',for instance
( f i g u r e2 . 1 ) .D ow e i n c l u d e t h e h a t sa n dk e r c h i e fwso r nb y t h et w o m e n ?E v e r ys i n g l et r e e ,
everysingleoneof the rocksstrewnaboutin the foreground? Theanalogywith language
losesits relevancehere.In language, wordssuchas man/ gun, tree,rocky groundabstract
awayf rom detailsof this kind.In naturalisticimagesthis doesnot happen. Theyare'worth
a t h o u s a nw d ords'.

O Fig2.2 ShannonandWeayer'scornmunicationmodel
Nar rati verepresentati ons 49

Yet we think that naturalisticimagescan be analysedinto participantsand processes


muchin the samewayas diagrams. Therearetwo different,but in the endcompatible, ways
o f a r g u i n gt h i s .T h ef i r s t i s t h e w a y o f f o r m a la r t t h e o r y( e . 9 .A r n h e i m1, 9 7 4 , 1 9 8 2 ) . f h e
l a n g u a g oe f t h i s k i n d o f t h e o r yi s , f o r t h e m o s t p a r t ,f o r m a l i s t i ca, n d g r o u n d e di n t h e
p s y c h o l o goyf p e r c e p t i o n P.a r t i c i p a n tasr e c a l l e d ' v o l u m e s ' o r ' m a s s eesa' ,c hw i t h a d i s -
t i n c t ' w e i g h t ' o r ' g r a v i t a t i o npaull l ' .P r o c e s s e a sr e c a l l e d ' v e c t o r s ' o r ' t e n s i o n s ' o r
'dynamicforces'.But,andthis is what mattersfor the purposeof identifyingparticipants,
t h e s e ' v o l u m e s ' a rpee r c e i v eads d i s t i n c e t n t i t i e sw h i c ha r e s a l i e n (t ' h e a v y 't)o d i f f e r e n t
degreesbecause of their differentsizes,shapes, colour,and so on. Thusthe two men in
figure 2.1 stand out as a distinctentity becauseof the tonal contrastbetweentheir
s i l h o u e t t easn dt h e l i g h to f t h e f i r e .A n d w h a t i s m o r e w , e r e c o g n i zteh e i r s h a p e o snthe
b a s i so f v i s u a sl c h e m anso t u n l i k et h o s et h a t a r e r e a l i z e di n d i a g r a m sA. r t i s t sh a v el o n g
l e a r n e tdh e i rc r a f tb y r e d u c i ntgh e v i s i b l ew o r l dt o s i m p l eg e o m e t r ifco r m s( s e eG o m b r i c h ,
1 9 6 0 ) .A c c o r d i n g t o A r n h e i m( 1 9 7 4 : c h . 4 ) , c h i l d r e nl e a r nt o d r a w i n t h e s a m ew a y ,
b u i l d i n gu p a r e p e r t o i r eo f b a s i cf o r m s ,a n d t h e n ,g r a d u a l l y' f,u s i n gt h e p a r t s ' .I f t h e
perceptionof picturesdoes indeedoperateon the basisof the sameprinciplesas the
p r o d u c t i oonf p i c t u r e sw, e m i g h tg r a s pt h e o v e r a lsl t r u c t u r eo f a p i c t u r el i l < e ' T h e British
u s e dg u n s ' a c c o r d i ntgo a s c h e m at h a t i s n o t s o d i f f e r e n ft r o m t h a t o f S h a n n o na n d
Weaver's communication model,as shownin figure2.3.
T h es e c o n dw a y o f i d e n t i f y i npga r t i c i p a n tiss t h a t o f f u n c t i o n asl e m i o t i ct h e o r y( s e e
H a l l i d a y1,9 7 8 ,1 9 8 5 ) .T h ec o n c e p t u a lp p a r a t uosf t h i s k i n do f t h e o r yh a s ,s o f a r ,b e e n

'd\
O Fig 2.3 Schematicreductionof tigure2.1,showingvector
50 . Narrative reoresentations

appliedonlyto language, the mostfrequentlyand methodically analysed semioticsystem.


It is orientedtowardsthe semanticf unctionsratherthantowardsthe formsof the partici-
p a n t sI.t u s e st e r m sl i k e ' A c t o r ' , ' G o a l ' a n' d R e c i p i e n t ' r a t htehra nt e r m sl i k e ' v o l u m e ' a n d
' m a s s 'Y . e tt h et w o a p p r o a c h easr e c o m p a t i b l e T.h em o s ts a l i e n t ' v o l u m ei sn'' T h e B r i t i s h
u s e d g u n s ' a r e n o t o n l y p e r c e p t u a l l y m o s t c o n s p i c u o u s , t h e y a l s o p l a y t hr oelm e so s t c r u c i a
in the grammaticalstructurethat constitutes the meaningof the picture:the two men(the
participantfrom whichthe vectoremanates)havethe role of Actor, andthe Aborigines
(the participantat which the vector points)havethe role of Goal in a structurethat
represents their relationas a Transactio4as somethingdoneby an Actor /o a Goal.The
s a m et e r m s( ' A c t o r ' , ' G o a l ' , ' T r a n s a c t i oanr'e) u s e di n f u n c t i o n alli n g u i s t i cT s .h i si s p o s -
s i b l eb e c a u steh e y a r e s e m a n t i c - f u n c t i o nr a lt,h e rt h a n f o r m a l ,t e r m s . 0 u ru s eo f t h e s e
termsdoesnot implythat imagesand diagramswork in the sameway as language; only
that they can 'say' (someof ) the samethingsas language- in verydifferentways:what in
languageis realizedby meansof syntacticconfigurations of certainclassesof nounsand
certainclassesof verbsis visuallyrealized,madeperceivable and communicable, by the
v e c t o r i arl e l a t i o n bsetween v o l u m e sI .n A r n h e i m ' w s o r d s , ' W es h a l ld i s t i n g u i sbhe t w e e n
volumesandvectors,betweenbeingandacting'(1982:1,54).
Thetransactional structureis not the only kind of structurethat can be realizedvisu-
ally.We havealreadydiscussed an exampleof a classificatory structure(a subjectwhich
w e w i l l t a k eu p i n m o r ed e t a i li n a l a t e rs e c t i o ns; e ep p . 7 9 - 8 7 ) . l n t h e p i c t u r ei n f i g u r e
2.4,takenfrom the samesocialstudiestextbookas figure2.1 (Oakleyet al., 1985),the
s t r u c t u r ei s ' a n a l y t i c a lH ' . e r et h e p a r t i c i p a n thsa v et h e r o l e sn o t o f ' A c t o r ' a n d' G o a l 'b u t
of ' C a r r i e r ' a n d ' A t t r i b u t eT' .h i sp i c t u r ei s n o t a b o u ts o m e t h i nw g h i c hp a r t i c i p a n tasr e
doingto otherparticipants, but abouttheway participantsfit togethertomakeup a larger
whole.It hasthe structureof a map.Just as in mapsa largerparticipant,the 'Carrier',
represents the 'whole'(say,Australia),anda numberof otherparticipants, the 'Possessive
Attributes',represent the 'parts' (say, the statesof Australia),so the Antarcticexplorer
functionsas'Carrier',andthe balaclava, thewindprooftop,thefur mittens,etc.functionas
'Possessive Attributes',asthe partsthat makeup the whole.Theclosestlinguistictransla-
tion here- werewe to attemptone- wouldnot be an actionclauselike'The Britishpoint
their gunsat the Aborigines', but a 'possessive attributive'clauselike tTheoutfit of the
Antarcticexplorerconsistsof a balaclava, a windprooftop,fur mittens. . . [etc.]'
We can now look at the abundance of detail in naturalisticimagesin a newway.The
naturalisticimage,whateverelseit maybe about,is alwaysalsoaboutdetail.It Contains a
multitudeof embedded 'analytical'processes. It may,at the most salientlevel,say,'The
B r i t i s hp o i n t h e i rg u n sa t i h e A b o r i g i n e sb'u, t i t w i l l a l s o a, t l e s si m m e d i a t ecl yo n s p i c u o u s
l e v e l ss,a yt h i n g sl i k e' T h em e n ' so u t f i t sc o n s i sot f h a t s ,k e r c h i e f.s. . [ e t c . ] ' a n d ' T h et r e e s
h a v ec l u m p so f l e a v e s . ' I nl a n g u a g ep,r e p o s i t i o n p ahl r a s e (st w o m e nw i t h h a t sa n d k e r -
chiefs)and subordinate clauses (two men,wearinghats and kerchiefs)fulfil the same
functionof addingdetailat a'secondary'or evenmoredeeplyembedded level.
E m b e d d i ncga na l s oo c c u ri n d i a g r a m sT.a k et h e ' c o m m u n i c a t i m o no d e li' n f i g u r e 2 . 5 , a
modeldrawnup,not by two telecommunication engineers, as in the caseof Shannonand
W e a v e r( f i g u r e2 . 2 ) , b u t b y t w o s o c i o l o g i s tR s ,i l e ya n d R i l e y( 1 9 5 9 ) .A s a w h o l e t, h e
Narrati verepresentations 5l

r11' rdqqv *i3-:


llr-t
"r.

i::t !rf1*{},:x :iq l';!* *a $$:!jh*


t1]!i1*:a1

;
;'.'.:. .i:";: 'r . trjs':h; F"!,..,,;i
.1:;;'r.: :-;-.,i. ! i
n
:l;
i
i1
.,*.J

O Fig 2.4 Antarcticexplorer(oakleyet at,,LgPjs)

O model(from Watsonand Hill,1980:143)


Fig 2.5 C0mmunication

d i a g r a mi s ' a n a l y t i c a l 'i;t i s a k i n d o f a b s t r a c m t a p .I t s h o w st h a t t h e ' o v e r - a l sl o c i a l


h t u r n c o n s i sot f ' p r i m a r yg r o u p s 'I.t
s y s t e m ' c o n s i sot sf ' l a r g e rs o c i a sl t r u c t u r e s ' w h i ci n
a l s o f e a t u r e st w o i n d i v i d u a l s , ' C('' C o m m u n i c a t o ra' )n d ' R ' ( ' R e c i p i e n t ' )T.h e s ea r e
depictedas half in, half out of the 'larger socialstructures', and they are connected to,
t h o u g hn o t p a r t o f , t h e ' p r i m a r yg r o u p s 'E . m b e d d ewdi t h i nt h i s a n a l y t i c asl t r u c t u r ei s a
Narrativerepresentati ons

O models(trom Watsonand Hill,1980:147)


Fig 2.5 Twocommunication

transactiona s tl r u c t u r et :h e ' l a r g e rs o c i a sl t r u c t u r e sa/n d t h e i n d i v i d u a 'l C


s ' a n d' R ' a r e
represented as involvedin an activeprocess of communication, realizedby vectors.
W h e nw e l o o ka t ' T h eB r i t i s hu s e dg u n s (/ i n f i g u r e2 . 1 ) a sa t r a n s a c t i o nsatlr u c t u r e , t h e
two menform oneparticipant:togethertheyhavethe roleof 'Actor'.Whenwe lookat the
two menasan 'analytical'structure, theyformtwo distinctparticipants, linkedby the lines
formedby the handof the manon the right andthe gunof the man on the left. Diagrams
a l l o wf u r t h e rp o s s i b i l i t i eass,c a n b e s e e ni n t h e t w o c o m m u n i c a t i omno d e l si n f i g u r e2 . 6 ,
b o t hd r a w nb y S c h r a m m( 1 9 5 4 ) ,a s o c i apl s y c h o l o g iwsrt i t i n ga b o u tm a s sc o m m u n i c a t i o n .
In the first model,'source'and 'encoder'are separate entities,conjoinedby a line,just as
a r e t h e t w o m e ni n ' T h e B r i t i s hu s e dg u n s ' W . e w i l l a r g u el a t e rt h a t l i n e sw i t h o u ta r r o w
h e a d sr e a l i z ea p a r t i c u l akr i n do f ' a n a l v t i c asl 't r u c t u r e :

O Fig 2.7 Conjoiningparticipants

I n t h e s e c o n dm o d e l , ' s o u r c e ' a n' edn c o d e ra' r e compounded,


weldedtogetheryet still
d i s t i n cct o m p o n e not sf t h ew h o l e :

O of participants
Fig 2.8 Compounding
ions
Narrative representat 53

T h et h i r d p o s s i b i l i twy o u l db e a c o m p l e t e f u s i o nb e t w e e \ns o L r r c e ' a n d ' e n c o d T ehr 'e.


s h a p eo f R i l e ya n d R i l e y ' s ' o v e r aslol c i asl y s t e m(/f i g u r e2 . 5 ) c a nb e i n t e r p r e t eads s u c ha
fusion- a fusionof two circlesand a box.Apparentlyparticipants can losetheir separate
identityto differentdegrees. Whenthey are conjoined, the process, the act of connecting
t h e m ,i s s t i l le x p l i c i tr,e a l i z e b
d y a l i n eW
. h e nt h e ya r ec o m p o u n d et dh ,e i ri d e n t i t i erse m a i n
distinct,but thereis no longeran explicitlyexpressed process to connectthem.Whenthey
are fused,eventheir separateidentitieshavedisappeared. In speechand writing,with
somewhatdifferentmeans- for instance, stressand intonation- we can movefrom, say,
Thebird is black,whichhastwo distinctparticipants aswell as a connecting process ('is'),'
to the blackbird,whichhasblack andbird still as differentwords,but removesthe process,'
to the blackbird,in whichtwo wordshavebeenfusedto becomeonesemanticentity/noun.
Eachsuccessive stepfurtherobscures the act of predication,the explicitact of bringingthe
t w op a r t i c i p a nt o s g e t h eur ,n t i lt h es t r u c t u r e
i s n o l o n g e r ' a n a l y t i c anlo' ,l o n g ear n a l y s eodr
a n a l y s a b lW e .e m a k et h e p o i n ta t s o m el e n g t hb e c a u soef t h e ( i d e o l o g i c asl i)g n i f i c a n coef
t h i ss e m i o t i rce s o u r cien c o n f i g u r i nt g herepresente wdo r l d .

O Fig 2.9 Fusionof participants

A s w i t h m a n yo t h e rk i n d so f d i a g r a mt,h e c o m m u n i c a t i omno d e l sw e h a v eu s e dt o
illustratethis sectionare explainedor paraphrased in the writtentextsthat accompany
them.But by no meanseverything that is expressed in the diagramsis alsoexpressed in the
written texts.Not all of the meanings conveyed visuallyare also conveyed verbally.The
m e a n i n gosf t h ev i s u asl h a p e st h, eb o x e a s n dc i r c l e sa n dt r i a n g l etsh a t g i v et h ep a r t i c i p a n t s
t h e i r v o l u m ef,o r e x a m p l ea, r e a l m o s ta l w a y sl e f t u n e x p l a i n e d . 0 l dceor ,m m o ns e n s eo r
t h e o r e t i c anl o t i o n ss, u c ha s ' i l l u s t r a t i o n('i m a g e s ' i l l u s t r a t i n g ' v e r bt eaxl t s )o r ' e x p l a n -
ation' (words'explaining'diagrams)are no longeran adequateaccountof the relations
between wordsand pictures,hereas in otherinstances. Why,in figure2.6, is the 'signal'a
circle,the 'source' a rectangle, the 'encoder' a triangle?Why,in figure2.5, is the 'primary
g r o u p ' ar e c t a n g lw e ,h i l et h e ' l a r g e sr o c i asl t r u c t u r e ' a nt dh e i n d i v i d u a l s ' C ' a n d ' R ' a r e
circlesW ? h yd o S h a n n o an n dW e a v e (r f i g u r e2 . 2 ) p r e f e ra n g u l a r i t w y ,h i l eR i l e ya n d R i l e y
prefercurvature (figure2.5)?
Therecanbe littledoubtthat suchchoicesarechargedwith meaning. Basicgeometrical
shapes havealwaysbeena sourceof fascination, evenof religious awe.0ur scientificageis
no exception. Circles,squaresand triangleshavebeenregardedas pure/quasi-scientific
' a t o m s ' o ft h e v i s i b l ew o r l d ,a ' p u r e m a n i f e s t a t i o n
o f t h e e l e m e n t st 'h, e ' u n i v e r s a l - a s - t h e -
mathematical', as Mondriansaid (quotedin Jaff6, 1967: 54-5). And they have been
thoughtto havethe powerto directlyaffect our nervoussystem,for instanceby the
c o n s t r u c t i v ia
s tr t i s t G a b o : ' T h ee m o t i o n aflo r c eo f a n a b s o l u t e s h a p ei s u n i o u ea n d n o t
ons
Narrati verepresentati

replaceable by any othermeans.. . . Shapes exultandshapesdepress, theyelateand make


desperate'(quotedin Nash,I974i 54). As we are here primarilyconcernedwith the
relationsbetweenparticipants, this subjectfalls somewhatoutsideour main concern;it
deserves a separate study.But giventhe semioticand ideological (mythical)significance of
theseaspects,we will at least indicatethe issueswith which such a study might be
concerned.
In contemporary Westernsociety, squares and rectangles arethe elements of the mech-
anical,technological order,of theworldof humanconstruction. Theydominate the shapeof
our cities,our buildings, our roads.Theydominatethe shapeof manyof the objectswe use
in daily life, includingour pictures,which nowadaysrarelyhavea roundor oval frame,
thoughotherperiodswerehappyto usetheseto framemoreintimateportraitsin particu-
lar. Unlikecircles,which are self-contained, completein themselves, rectangularshapes
canbestacked, alignedwith eachotherin geometrical patterns: theyform the modules, the
buildingblockswith whichwe constructour world,and they are thereforethe dominant
c h o i c eo f b u i l d e ras n de n g i n e e rasn, do f t h o s ew h ot h i n kl i k eb u i l d e ras n de n g i n e e r s . Ianr t ,
theyarethe choiceof geometrical abstractionists, artistsfor whomart hasto be,aboveall,
w r o t ei n t h e 1 9 2 0 s ,
r a t i o n a lA. s M o n d r i a n

I n a l l f i e l d sl i f eg r o w si n c r e a s i n gal yb s t r a c t w h i liet r e m a i n rse a l .M o r ea n dm o r et h e


machinedisplaces naturalpower.In fashionwe seea characteristic tensingof form
and intensification of colour,signifying the departuref rom the natural.
In moderndancesteps(boston,tango,etc.)the sametensingis seen:the curved
lineof the old dance(waltzetc.)hasyieldedto the straightline,andeachmovement
is immediately neutralized by a counter-movement - signifyingthe searchfor equi-
librium.)ur social/ifeshowsthis too: autocracy, imperialism with its (natural)rule
- -
o f p o w e ri ,s a b o u t o f a l l i f i t h a sn o t f a l l e na l r e a d y a n dy i e l d st o t h e ( s p i r i t u a l )
oowerof law.
Likewise the newspiritcomesstronglyforwardin logic,scienceandreligion.The
i m p a r t i n go f v e i l e dw i s d o my i e l d st o t h e w i s d o mo f p u r e r e a s o na; n d k n o w l e d g e
showsincreasingexactness. The old religion,with its mysteriesand dogmas,is
i n c r e a s i n gt lhyr u s ta s i d eb y a c l e a rr e l a t i o n s hti op t h e u n i v e r s a l .
( q u o t e di n J a f f 6 ,1 9 6 7 : 6 4 )

G l o s s e isn ' d i c t i o n a r i eos f v i s u a ls y m b o l s ' a n ds i m i l a rp u b l i c a t i o nt e


s n d t o e x p r e stsh e
meaningof geometrical shapesin termsof intrinsic,abstractqualities, but theypointin the
samedirection.Accordingto Dondis(1973:,44),thesquarerepresents'honesty, straight-
n e s sa n dw o r k m a n l i km e e a n i n ga' ;c c o r d i ntgo T h o m p s oann d D a v e n p o r(t1 9 8 2 : 1 1 0 ) ,i t
'represents the worldanddenotesorder'.
C i r c l e sa r e g l o s s e dv e r y d i f f e r e n t l yi n s u c h d i c t i o n a r i e sa,s d e n o t i n g ' e n d l e s s n e s s ,
w a r m t hp , r o t e c t i o n( 'D o n d i sI,9 7 3 : 4 4 ) , o r a s ' t h et r a d i t i o n asl y m b ool f e t e r n i t ya n dt h e
h e a v e n s( T' h o m p s oann d D a v e n p o r t , I 9 S 2 : 1 1 0B) .u t s u c hd e s c r i p t i o nc sa nb e m u l t i p l i e d
endlessly: the moreabstractthe sign,the greaterits semanticextension; or,to put it in our
terms,the greaterits potentialrangeof usesas a signifierin signs.We needto lookfor the
Narrativerepresentations. 55

principles that unitethesemeanings, and for the fundamental oppositions betweensquare


and circle,betweenthe angularand the curved.In nature,squareness doesnot exist.
Mondrianadmittedthat hismethodof 'abstracting the curve'madeit difficultto represent
nature:'1npaintinga tree,I progressively abstracted the curve;you can understand that
v e r yl i t t l e" t r e e " r e m a i n e d ' ( q u o t e i ndJ a f f 6 ,1 9 6 7 : 1 2 0 ) .C i r c l e sa n dc u r v e df o r m sg e n e r -
ally are the elementswe associate with an organicand naturalorder,with the world of
organicnature- and suchmysticalmeanings as may be associated with themderivefrom
t h i s .A n g u l a r i t yw e a s s o c i a tw e i t h t h e i n o r g a n i cc,r y s t a l l i nwe o r l d ,o r w i t h t h e w o r l d o f
technology, whichis a worldwe havemadeourselves, andthereforea worldwe can/at least
in principle,understand fully and rationally.The world of organicnaturels not of our
making,and will alwaysretain an elementof mystery.Curvedforms are thereforethe
d o m i n a nct h o i c eo f p e o p l ew h o t h i n k i n t e r m so f o r g a n i cg r o w t hr a t h e rt h a n m e c h a n i c a l
construction, in termsof what is naturalratherthan in termsof what is artificial.In art, it
i st h ec h o i c eo f w h a ti s s o m e t i m ecsa l l e d' b i o m o r p h iacb s t r a c t i o n a l i s mt h' -ec u r v e sb, l o b s
a n db u l g e si n t h e p a i n t i n gosf H a n sA r p o r t h e s c u l p t u r eosf H e n r yM o o r e .
The valuesattachedto thesepolesof meaning- that is, the actual signsproduced
w i t h t h e s i g n i f i e r os f t h e ' t e c h n o l o g i c a l ' a ntdh e ' n a t u r a l ' - d o , o f c o u r s ed, i f f e r .T h e
squarecan connotethe 'technological' positively, as a sourceof powerand progress/ or
n e g a t i v e lay s, a s o u r c eo f o p p r e s s i ownh i c h ,l i t e r a l l ya n d f i g u r a t i v e l y , ' b o xuess i n ' . I n
R i l e ya n d R i l e y ' sc o m m u n i c a t i omno d e l( f i g u r e2 . 5 ) s o c i e t yi s r e p r e s e n t eads a n a t u r a l
o r d e ro, r g a n i c a l leyv o l v e d r a t h e rt h a nh u m a n l yc o n s t r u c t e B d .u t t h e ' p r i m a r yg r o u p s ' a r e
depictedas rectangles. Perhapsthis betraysan unconscious bias in favour of modern
u r b a ns o c i e t ya, v i e wi n w h i C ht h e s m a l l ,c l o s e - k n ci to m m u n i t iyn w h i c he v e r y o nken o w s
everythingabout everyoneis seenas oppressive, and the 'larger social structure'as
l i b e r a t i n gp,r o v i d i ntgh e i n d i v i d u awl i t h a n o n y m i t ya,n dt h e r e b yw i t h a u t o n o m ya n d s e l f -
c o n t a i n m e nct ,h o i c ea n d f r e e d o m F . o r R i l e ya n d R i l e y i, n d i v i d u a lhsa i l f r o m ' p r i m a r y
g r o u p s (' a n da r e s t i l l c o n n e c t etdo t h e m ,a l b e i tt e n u o u s l yb) ,u t t h e n g o t h e i r o w n w a y ,
l e a v i n gt h e ' p r i m a r yg r o u p s ' b e h i n da ,n d m o v i n gf r e e l yi n a n d o u t o f t h e ' l a r g e rs o c i a l
structures',in a sociallymobileworld.This exampleshowsthat diagrams,rationaland
scientificas they may seem/can conveymeanings visuallythat are not necessarily also
conveyed verbally.
T h et r i a n g l ei s a n g u l a rl ,i k et h e s q u a r e- a n e l e m e not f t h e m e c h a n i c at e l ,c h n o l o g i c a l
o r d e rB . u t ,u n l i k et h e s q u a r et h/ et r i a n g l ee, s p e c i a lw l yh e nt i l t e d ,i s a ( f u s e ds t r u c t u r eof a)
participantanda vector,because it canconveydirectionality, pointat things.Themeanings
i t a t t r a c t sa r e t h e r e f o r el e s s l i l < e ' q u a l i t i eos f b e i n g ' t h a n l i l < ep r o c e s s eas s/ i n t h e
well-known r e v o l u t i o n a rpyo s t e rb y E l L i s s i t z k y( f i g u r e2 . 1 0 ) ,i n w h i c ht h e r e v o l u t i o n ,
represented by a red triangle,is an active,dynamicforce,wedgingitself into the inert,
s e l f - c o n t a i n e' odr,g a n i cs' o c i e t oy f W h i t eR u s s i a .
I n d i a g r a m st r,i a n g l ecsa ns i m i l a r l yi n t r o d u cae s e n s e of procesT s .h et r i a n g l e isn f i g u r e
2 . 6 , f o ri n s t a n c ce o, u l db e s e e na s ' f o c u s i n g ' o r ' a i m i n g ' t h e ' m e s s(awgeeh' a v ea l r e a d y
i n d i c a t ew d h y i t i s s o d i f f i c u l t o g i v ev e r b a tl r a n s c o d i n gosf v i s u a p l r o c e s s easn dw e w i l l
d i s c u stsh i sm o r ef u l l yl a t e r ) N , s s eosf t h em e a n i n gosf t r i a n g l e isn v i s u a l
. o ts u r p r i s i n g gl yl o
dictionariesreflectthis dynamicquality.Trianglesare 'a symbolof generativepower'
Narrative representations

&i,'..i.,

O Fig2.LO BeattheWhiteswiththeRed ryedge(ElLissitzky,lglg-20)(fromilash,1974)

( T h o m p s oann d D a v e n p o r1t ,9 8 2 : 1 1 0 ) ,a n d r e p r e s e n' at c t i o nc, o n f f i c t ,e n s i o n(' D o n d i s ,


I973:44).
The meanings of the basicgeometrical shapes, then,are motivatedin two ways.First,
they derivefrom the propertiesof the shapesort rather,from the valuesgivento these
propertiesin specificsocialand culturalcontexts.The straightline,for instance, means
what it literallyis:'straight'.This'straightness'may thenbe usedto carry anyoneof a vast
rangeof meanings compatible with that.It maybe positively valuedin onecontext(e.g.the
'straightand narrowpath',or Mondrian'sassociation of straightness with the'spiritual
p o w e ro f l a w ' ) ,l e s sp o s i t i v e liyn a n o t h e (r e . 9 . ' t h es t r a i g h m
t a n ' a so p p o s etdo t h e ' f u n n y
man',or 'straight'as opposed to 'gay').Theproducers of an imagehavetheir interestsin
makingthe visualsign,and this makesthe meaningof the imagequite specificfor the
producer;it coloursin and makesspecificthe abstractmeaningsthat derivefrom the
inherentproperties of the shapesand from the historiesof their culturaluses.Rectangles
can be stacked- and,again,this may be positivelyvaluedin one context,say in urban
planning,or in geometricabstractionism; and lesspositivelyin another,say in counter-
c u l t u r e tsh a t d r e a mo f l i v i n gi n g e o d e s idco m e so, r i n b i o m o r p h iacb s t r a c t i o n i s m .
Second,these meaningsderivefrom the commonqualitieswe may detect in such
o b j e c t si n o u r e n v i r o n m e n a ts w o u l db e c i r c u l a ro r r e c t a n g u l awr h e na b s t r a c t etdo t h e i r
underlying basicshape,and from the valuesattachedto thesequalitiesin differentsocial
Narrativerepresentations. 57

contexts. Thesun,the moon,the bellyof the pregnantwoman,are curved.Theskyscraper,


the executive desk,the expensive briefcase, are rectangular. Suchcommonqualitiesas we
m a ys e ei n t h e s e g r o u p s
of objects ( s a y , ' n a t u r e ' s
c y c l e s ' a n d ' m apl eo w e r 'w ) i l l e v i d e n t lbye
readandvalueddifferentlyin differentsocialcontexts andthe - groupings of objectsfrom
which we derivetheserneanings are likelyto be made selectively, so as to obtainthe
common q u a l i t i e s
sought.
F i n a l l yo, u r a r g u m e nst u g g e s ttsh a t t h e s e m i o t i ca e n e s ios f d i a g r a m sl i e s ,n o t j u s t i n
technicaldrawing,but also in art, and specifically in the abstractart movements of our
c e n t u r yw, h i c hn o l o n g efri l l o u t a n dc o r p o r e a l i zt hees c h e m a t i c r e d u c t i o nws h i c hn a tural-
i s t i c a r t i s t sh a v eu s e df o r c e n t u r i e sa,n d w h i c h ,i f t h e G e s t a l t i s tasr e r i g h t ,u n d e r l i ea l l
visualreoresentation.
From the basicshapesother geometricalshapescan be derived:square/circle and
t r i a n g l ec a nb eh o r i z o n t a l o l yr v e r t i c a l ley l o n g a t et do d i f f e r e ndt e g r e e sa ;n dt h es q u a r e , t h e
triangleandall elongated shapes canbetilted,eithertowardsthe right or towardsthe left.
Verticalelongationcreatesa more pronounced distinctionbetweentop and bottom,and
hencea biastowardshierarchy, andtowards'opposition'generally (what is most import-
ant or otherwisedominant goes on top,what is lessimportantor dominantis relegated to
the bottom).Horizontalelongation causes a shapeto leantowardsthe kindof structurein
w h i c hw h a t i s p o s i t i o n e d
o nt h e l e f ti s p r e s e n t eads ' G i v e n 'a, s i n f o r m a t i otnh a t i s a l r e a d y
f a m i l i a rt o t h e r e a d e ra n d s e r v e sa s a ' d e p a r t u r ep o i n t ' f o r t h e m e s s a g w e /h i l ew h a t i s
positioned on the right is presented as'New',as informationnot yet knownto the reader,
and hencedeserving his or her specialattention.Theshapeof Schramm's'fieldof experi-
ence'and 'signal' in the seconddiagramof figure 2.6, for instance, suggests that these
participantsare,at leastpotentially, endowed with suchan informationstructure.Tilting,
finally,createsobliquelinesand hencea senseof vectoriality. In Malevich'sSupremacist
Composition: RedSquareand BlackSquare(fi9ure2.11)the participants are represented
a s s q u a r e sB.u t b e c a u steh e r e ds q u a r ei s t i l t e d t, h e p a i n t i n gi s s t r u c t u r a l lm y o r es i m i l a tro
El Lissitzky'sBeat the Whiteswith the Red Wedge(figure2.10) than,for instance, to
M o n d r i a n ' cs o m p o s i t i o nosf r e d ,y e l l o wa n d b l u es q u a r e si :t i s a b o u td y n a m i c ' a c t i o n ,
w h e r e aM s o n d r i a n 'cso m p o s i t i o n a sr ea b o u ta s t a b l eo r d e ra, ' s e a r c hf o r e q u i l i b r i u m 't-h e
redsquareseemsto moveawayfrom the oppressively largeblacksquare.t
Finally,it is importantto stressthe essentialinterchangeability of visualand verbal
p a r t i c i p a n ti sn d i a g r a m sa,n d ,i n d e e di n , m a n yo t h e rv i s u a gl e n r e sA. l t h o u g ht h e p r o c e s s e s
and structuresin diagramsare alwaysvisual,the participantswhichthey relateto each
othermay be of differentkinds:pictures,naturalisticor schematic; abstractshapes, with
o r w i t h o u tv e r b a l a b e l sw; o r d se, i t h e re n c l o s eodr n o t e n c l o s eidn b o x e so r o t h e rs h a p e s ;
letters;and so on.Thesamethingcan be seenin pagelayout:the participantsare hetero-
geneous. Theycanbeverbal(headlines, blocksof copy,etc.),but the semioticmeanswhich
bring them togetherinto a coherentsemanticstructureare alwaysvisual.The key to
understanding suchtextsthereforeliesaboveall in an understanding of the visualsemiotic
meanswhichare usedto weldtheseheterogeneous elementsinto a coherentwhole,into a
text.Visualstructuresrelatevisualelements to eachother;thesevisualelements, however,
maythemselves be heterogeneous - a word as a visualelement, a blockof writtentext as
Narrati verepresentati ons

O Fig 2.ll SuprematistComposition:Red Sqaareand BlackSquare(Kasimir Matevich,I9l4) (from Nash,1974)


Narrative representations . 59

a visualelement,an imageas a visualelement,a numberor an equationas a visual


element.

N A R R A T I V EP R O C E S S E S

Whenparticipantsare connected by a vector,theyare represented as doingsomething to


or for eachother.Fromhereon we will call such vectorial patterns narrative- in l(ress and
van Leeuwen(1990) we usedthe term 'presentational'-andcontrastthemto conceptual
patterns(seefi9ure2.12).Whereconceptual patternsrepresentparticipantsin termsof
their class,structureor meaning/ in otherwords,in termsof their generalized and moreor
lessstableandtimelessessence/ narrative patterns serve to present unfolding actionsand
events, processes of change, transitoryspatialarrangements.
The hallmarkof a narrativevisual'proposition'is the presence of a vector:narrative
structuresalwayshaveone,conceptual structures never do. In pictures, thesevectorsare
formedby depictedelements that form an obliqueline, often a quite strong, diagonalline,
asin ' T h e B r i t i s hu s e d g u n s '( i n f i g u r e2 . 1 ) ,w h e r et h e g u n sa n d t h e o u t s t r e t c h eadr m so f
t h e B r i t i s hf o r m s u c ha l i n e .T h ev e c t o r sm a y b e f o r m e d b y b o d i e so r l i m b so r t o o l s ' i n
action',but thereare manyotherwaysto turn represented elements into diagonal linesof
action.A road runningdiagonallyacrossthe picture space, for instance, is also a vector,
a n dt h e c a r d r i v i n go n i t a n ' A c t o r ' i n t h e p r o c e s s
of'driving'. I n a b s t r a c i
t m a g e s u c ha s
diagrams,narrativeprocesses are realizedby abstract graphic elements - for instance,
l i n e sw i t h a n e x p l i c i ti n d i c a t oor f d i r e c t i o n a l i tuys, u a l l ya n a r r o w h e a dS.u c hf e a t u r e so f
directionality mustalwaysbe presentif the structureis to realizea narrativerepresenta-
t i o n : c o n n e c t i n lgi n e sw i t h o u ta n i n d i c a t o o r f d i r e c t i o n a l i tfyo r m a p a r t i c u l a rk i n d o f
a n a l y t i c aslt r u c t u r ea,n dm e a ns o m e t h i nlgi k e' i s c o n n e c t et d o ' , ' i sc o n j o i n etdo ' , ' i s r e l a t e d
t0'.
T h e ' A c t o r ' i st h e p a r t i c i p a nf tr o mw h o mo r w h i c ht h e v e c t o rd e p a r t sa, n dw h i c hm a y
be fusedwith the vector to different degrees.In Beat the Whites with the Red Wedge
(figure2.10),for instance, the redtriangleis both participantand vector,and represents
b o t ht h e ' w e d g ea' n dt h e a c t o f w e d g i n g i n ( o r ' b e a t i n ga' ,st h et i t l e h a si t ) . I n f i g u r e2 . I 3 ,
t a k e nf r o ma f a c t u a cl h i l d r e n 'bso o ka b o u tF r a n c e( B e n d e r , 1 9 B B t h) e
, A c t o r sa r er e a l i z e d
( t h e' G u l f ' M
S t r e a mv' e c t o ri s r e d ,t h e i s t r a l 'v e c t o rb l u e )a, n d
b y t h ec o l o u ro f t h e a r r o w s
by their volume (they are not just thin lines),and the narrativeprocesses are realizedby

r
I
Reoresentational I
structures I
classificatorv
I f
-----*t
L conceptuar
ilfl;:f::

O Fig 2.12 Main types of visual representationalstructure


60 Narrativerepresentatrons

E n g l i s hC h a n n e l

Biscay

ffi {
Mediterranean

,'-.,',]::'
i:,,i",:,:i,rll!'t'
Gulf Stream
ftrtistrat
LEso+,f*,
il
O Fig 2.13 culf Streamand Mistrat(Bender,t988)

t h e i rv e c t o r i a l i tIyn. S h a n n o an n dw e a v e r ' si 9 4 9 ) c o m m u n i c a t i omno d e (l f i g u r e2 . D , o n


the otherhand,Actor and process are realizedby separate visualelements,the Actor by a
b o x ( ' i n f o r m a t i osno u r c e ' )t h
, e p r o c e sbs y a n a r r o w .P i c t u r e sl i k e ' T h eB r i t i s hu s e dq u n s '
( i n f i g u r e2 . 1 ) o c c u p yp,e r h a p sa,n i n t e r m e d i apt eo s i t i o n .
In the caseof 'realist'images, the contextusuallymakesclearwhat kind of actionthe
v e c t o r sr e p r e s e n t . ' T hBer i t i s hu s e dg u n s /( i n f i g u r e2 . 1 ) c a n b e t r a n s l a t e-d s h o u l dw e
wishor needto do so- notso muchwith 'used',asthe captionhasit, aswith something like
' T h e B r i t i s hs t a l kt h e A b o r i g i n e s
w i t h t h e i rg u n s ' . O t h epro s s i b i l i t i e sx i s t b, u t i h e f i e l di s
limited.Thevectorsin abstractpicturesare moredifficultto transcode.One first needsto
formulatewhat they,literally,formally,do. Thiscan then helpto circumscribe the field of
possiblereadings.The trianglein El Lissitzky'sBeat the whites with the Redwedge
(figure2.10) literallywedges itselfintothewhitecircle.Thisopensup a perhapslarge,but
b y n o m e a n si n f i n i t er,a n g eo f p o s s i b l e r e a d i n g st h: e t r i a n g l ec a n b e s a i dt o , p i e r c e ,o r
'infiltrate'or'destabilize'the
c i r c l eI.n f a c t i t d o e sa l l t h e s et h i n g s- t h e p r o c e srse p r e s e n t s
a field of possiblemeanings. In the caseof Malevich,sSuprematistComposition:Red
Squareand Black Square(frgure2.7I), the red squareliterally tilts away from the black
squareaboveit. Howcanwe transcode this?Doesthe redsquare'ffeefrom,or 'pointaway
Narrativerepresentations. 6I

f r o m ' t h e b l a c ks q u a r e ?I s i t ' e j e c t e df r o m ' o r ' d i s c a r d e b d y ' i t ? A l l t h e s er e a d i n gas r e


l e g i t i m a t eT.h e p o i n ti s t h a t e a c hr e a d i n gw i l l h a v et h e s m a l lr e d s q u a r ea s t h e m o b i l e
participant,howevermuch it is dominatedby the big, heavyblacksquareiand the black
s q u a r ea s s t a t i c ,m o t i o n l e sasn d m o n o l i t h i cT.h e p i c t u r et e l l st h e s t o r yo f a n ' u n d e r l i n g '
e s c a p i nfgr o m ,o r b e i n ge x p e l l efdr o m ,t h e m o n o l i t h ipc o w e ro f t h e b l a c ks q u a r eW . hoor
w h a tt h i s u n d e r l i nogr t h e s ef o r c e sa r et h e r e a d e rw s i l l p r o d u c ei n t h e p r o c e sosf r e a d i n g ,
a l t h o u g ht h e s h a p e sa n d c o l o u r sw i l l p o i n tt h e m i n a c e r t a i nd i r e c t i o nm , a l < ec e r t a i n
r e a d i n gm s o r ep l a u s i b l e m, o r el i k et h o s eo f t h e p r o d u c e rt' sh a no t h e r s .
I n t h e c a s eo f d i a g r a m ist i s a l s od i f f i c u l t o s a yi n w o r d sj u s tw h a t k i n do f a c t i o nt h e
v e c t o r sr e p r e s e nTt .h ec o m m o ns e n s e v i e wo f t h e f u n c t i o no f i m a g e sa s ' i l l u s t r a t i o n ' a n d
' e x p l a n a t i o n ' w o ubl d et h a t t h e a c c o m p a n y i n veg r b atl e x te x p l a i nw s h a t i s n o t m a d ec l e a r
v i s u a l l yB. u t u s u a l l tyh i s i s n o t s o .U s u a l l tyh e p r o c e siss r e p r e s e n t eodn l yv i s u a l l ya,n dt h e
writtentext eitherdoesnot paraphrase it at all, or provides contradictory or evenmislead-
i n g g l o s s e sT.h e S h a n n o n a n d W e a v e cr o m m u n i c a t i omno d e lh a sb e e n ' q u o t e d ' a nvde r -
b a l l ye x p l a i n eidn m a n yb o o k sa n da r t i c l e s , f oirn s t a n cien W a t s o na n d H i l l ' sD i c t i o n a royf
Communication and MediaStudies(1980),whichhasa 250-wordentryaboutthe model,
and in a ScientificAmericanarticleby Pierce,a telecommunications engineer (I972).The
first givesthe followingindications of the meaningof the vectors:the authorssaythat the
model'can be appliedto any informationtransfersystem'and they call it a'process
c e n t r e dm o d e l (' 1 9 8 0 ; I 4 9 ,o u r i t a l i c s -) t w o r a t h e ro b l i q u er e f e r e n c eosn, ea b r o a dg l o s s
( ' p r o c e s s 't)h, e o t h e ro n l y s l i g h t l ym o r e s p e c i f i c( ' t r a n s f e r ' )P. i e r c e ' se x p l a n a t i o nasr e
c o n t r a d i c t o r y . 0t hneo n eh a n d h, ec a l l st h e m o d e al ' s y s t e m(' a sd o W a t s o na n d H i l l i n t h e
dictionary)and paraphrases it in termsof an analyticalratherthan a narrativestructure:
'The systemconsists of an informationsource,a transmitter, a communication channel,a
n o i s es o u r c ea, r e c e i v ear n da m e s s a gdee s t i n a t i o n0 '9 7 2 ' . 3 2 , o u r i t a l i c s -) n o t et h a t h e
listsonlythe participants, not the processes. 0n the otherhand,whenhe givesan example,
he paraphrases the vectorsby meansof activeverbs:

A humanbeingmay type a message consistingof the lettersand spaceson the


keyboard of a teletypewriter. Theteletypewriter servesas a transmitterthat encodes
e a c hc h a r a c t earsa s e q u e n coef e l e c t r i c aplu l s e sw, h i c hm a yb e ' o n 'o r ' o f f ' , ' c u r r e n t '
or'no current'.Theseelectricalpulsesare transmittedbya pair of wiresto another
teletypewriter that actsas a receiverandprintsout the lettersandspaces.
( P i e r c eI ,9 7 2 13 3 , o u r i t a l i c s )

Thesearejust two examples whereothersare possible. Themeaningpotentialof diagram-


maticvectorsis broad,abstractand hencedifficultto put into words.Theaccompanying
textstendto be muchmoreexplicitaboutparticipants, aboutthingsexistingin space(or
represented as thoughthey are),than aboutprocesses/ eventsand actions.Scientificand
bureaucratic writing,and manyforms of expositorywriting generally, put most of their
meaningin the nounsratherthan in the verbs.Verbs,in theseformsof language, remain
r e s t r i c t etdo a r e l a t i v e lsym a l ls e to f ( l o g i c a l c) o n n e c t o r(s' i s ' , ' h a s ' , ' l e a dtso ' , ' c a u s e s ' ,
' g e n e r a t e s ' , ' d e v e l ionpt o
s ' ,a n ds o o n ) ,a l m o s ta s t h o u g ht h e yw e r e ' f u n c t i o w n o r d s ' l,i k e
Narrative representat
i ons

articlesand pronounst ralherthan 'contentwords/.In his book Factual Writing (1985:


4 0 ) , M a r t i ng i v e sa n e x a m p l e :

Thereis little doubtthat televisioncoverage of a domesticslaughtering


operation,
conducted in a government approved abattoir,whichinvolved
the slaughterof lambs,
calvesandswine,wouldgenerate a gooddealof publicrevulsionand protest.

In this extract most of the specificactionsappearin nounIorm, and somehavebeen


nominalized; that is,turnedinto nounsfrom prior full clausalforms ('doubt','coverage',
' s l a u g h t e r i nogp e r a t i o n ' , ' s l a u g h t e r ' , ' r e v u l s i o n ' , ' p r oI n
t eastte' )x. t o f t h i r t y - s e v ewno r d s ,
thereare onlytwo mainverbs('is' and'generate'), bothverygeneral.Doingsand happen-
ingshavebeenturnedintothings.Thedynamicsof actionhasbeenchangedintoa staticof
relations. Diagrams do something similar.Theyrepresent eventswhichtakeplaceovertime
as spatialconfigurationg and so turn 'process'into 'system'- or into something ambigu-
ouslyin between, something that canbecalledeither'system-centred'or'process-centred'.
I n t h i s r e s p e cdt i a g r a m as r e a k i nt o c e r t a i nf o r m so f n o m i n a l i z i nwgr i t i n gw , h i l en a t u r a l -
istic images, with their humanparticipants andtheir moreconcrete/ specificprocesses, are
more akin to story-writing.Like many naturalisticimages,storiesare about humanor
animatebeingsandthe thingstheydo,and in storiesmuchmoremeaningis put intoverbs
than in mostnon-narrative formsof writing.
Becausetheir meaningis so abstract and general,vectorscan representfunda-
mentallydifferentprocesses as thoughtheywerethe same(for instance,'humans typing
lettersand spaceson a keyboard'and'teletypewriters transmittingelectricalpulses').
Diagramsof the Shannonand Weavertype can imposetwo modelsof interpretation on
one situationor perhapsone modelon many,'herethe two modelsare 'transport'and
'transformation'. Figure2.2 represents what is goingon eitheras transport,movement
from one placeto another,or as the more or lesscausallydeterminedtransformation
from one thing into another.And becauseone sign,the arrow,can representboth,the
two meaningsoften becomeconffated:movement, transportrs transformation; mobility
rs the causeof, and conditionfor,change,growth,evolution,progress. The Shannonand
Weavermodel,for instance, represents communication as transport,as movinginforma-
tion from one placeto another,but it also and at the sametime represents communi-
cation as the transformationof messagesinto signals,of 'letters and spaces'into
' e l e c t r i c apl u l s e s ' .
The arrowsin the 'systemnetworks'of systemic-functional grammar(our diagramin
f i g u r e2 . I 2 i s s u c ha ' s y s t e mn e t w o r k 'a) r e u s u a l l tyr a n s c o d ebdy ' c h o o s eo' r ' s e l e c t (' e . g .
' ').
" C o n c e p t u a ls" e l e c t "sC l a s s i f i c a t i o n a"l "A, n a l y t i c a l "o r " S y m b o l i c a l " B u t v i s u a l l y
the processis, again,a combinationof transportand transformation. And when such
networksare turnedinto computerprograms/ as indeedthey havebeen,the visualmeta-
p h o r b e c o m eas r e a l i t yi n w h i c ht h e r ea r e n o t p e o p l e ' c h o o s i n g ' b e t w e e n ' o p t i obnust ' ,
pulsestransported to pointsat whicha changeof stateoccurs.At that pointthe schematic
reductionof one semioticreality has turned into a blueprintfor another,new semiotic
reality,and peoplewill havebeenreducedto the role of'source' and 'destination'in an
Narrati verepresentations 63

autonomized and exteriorized just as they are in Shannonand Weaver's


semioticprocess,
c o m m u n i c a t i omno d e l .
Differentkindsof narrativeprocesscan be distinguished on the basisof the kindsof
vectorandthe numberand kindof participants involved,

I A c t i o np r o c e s s e s

TheActor is the participantfrom whichthe vectoremanates, or whichitself,in wholeor in


p a r t ,f o r m st h e v e c t o r( a sw i t h t h e t r i a n g l ei n f i g u r e2 . 1 0 ) .I n i m a g e tsh e ya r e o f t e na l s o
t h e m o s t s a l i e n tp a r t i c i p a n t st h, r o u g hs i z e ,p l a c ei n t h e c o m p o s i t i o nc ,o n t r a s a t gainst
b a c k g r o u n cd o, l o u rs a t u r a t i o no r c o n s p i c u o u s n esshsa, r p n e sosf f o c u s ,
and t h r o u g ht h e
' p s y c h o l o g i csaal l i e n c ew' h i c hc e r t a i np a r t i c i p a n t(se . 9 t. h e h u m a nf i g u r ea n d ,e v e nm o r e
so,the humanface)havefor viewers. In figure2.l,for instance, the Britishare largerthan
t h e A b o r i g i n eas n, dp l a c e di n t h e f o r e g r o u n d I n. t h e S h a n n o a n n dW e a v ecr o m m u n i c a t i o n
m o d e l( f i g u r e2 . 2 ) , t h e ' i n f o r m a t i o sno u r c ea' n d t h e ' n o i s es o u r c ea' r e A c t o r s( w e w i l l
commenton their differentpositionin the diagram,and on the differentdirectionality of
t h e i ra r r o w si,n c h a p t e6r ) :
W h e n i m a g e so r d i a g r a m sh a v eo n l y o n e p a r t i c i p a n t h , i s p a r t i c i p a nits u s u a l l ya n
Actor.Theresultingstructurewe call non-transactional.The actionin a non-transactional
p r o c e s hs a s n o ' G o a l ' , i s n o t ' d o n e t o ' o r ' a i m e d a t / a n y o n eo r a n y t h i n gT. h e n o n -
transactional actionprocess is thereforeanalogous to the intransitive verbin language (the
verbthat doesnot take an object).The processes in figure 2.I3 are non-transactional:
the waterof the Gulf Streamdoesnot movesomething, it just moves;andthe wind of the
just
Mistraldoesnot blowsomething,it blows.Thisvisualrepresentation is akinto theway
meteorological processes are represented in English;it rains,orit snows.As Hallidayhas
pointed o u t ( 1 9 8 5 :1 0 2 ) ,o t h e rl a n g u a g edso n o tn e c e s s a r d y t h i s .I n o n eC h i n e sdei a l e c t ,
i lo
for instance, one has to say somethinglike'the sky is droppingwater'; in otherwords,
raininghasto be represented as a transactive process. In figure2.15,thegestureof the old
manformsa vector,but hedoesnot gesture towardsanyoneor anything, at leastnot so far
as we can seein this picture.As a result,the vieweris left to imaginewho or what he may
b e c o m m u n i c a t i nwgi t h .I s h e a l r e a d yi n t o u c hw i t h w h a t l i e sb e y o n dl i f e ?I s t h a t w h y t h e
youngboy looksat him with suchconcentrated fascination?

O Fig2.t4Actors
Narrative representati ons

O Fig2.I5 NewYork,1955(RobertFrank)

At othertimes,thereis onlya vectorand a Goal(figure2.16).TheGoalis the partici-


pantat whomor whichthe vectoris directed,henceit is alsothe participantto whomor
w h i c ht h e a c t i o ni s d o n eo, r a t w h o mo r w h i c ht h ea c t i o ni s a i m e d .
R e p r e s e n t a t i oonf as c t i o n w
s h i c hi n c l u d e
o n l yt h e G o a lw ew i l l c a l l E v e n t ss:o m e t h i nigs
h a p p e n i nt o g someonb eu , tw e c a n n ost e ew h oo r w h a tm a k e si t h a p p e nF. i g u r e2 . 1 7s h o w s
a diagramwhich appearedin the SydneyMorning Heraldduringthe first Gulf War.A
vectorrepresents the actionof movingtowardsthetownof l(hafji,andthe Goalisthetown
of l(hafji itself,represented by a blackdot. But nothingrepresents the war planeswhich
are movingtowardsl(hafji.Closelyrelatedis the casein which just a small part of the
Actor ls visible,a hand,or a foot, so that the Actor becomes anonymous. In both cases
there rs in fact an Actor,as in figure 2.I7, but the Actor is either deletedfrom the
r e p r e s e n t a t ioornm a d ea n o n y m o uas v/ i s u aal n a l o g u ep ,e r h a p so,f ' p a s s i vaeg e n td e l e t i o n ' ,
a l i n g u i s t ifco r m o f r e p r e s e n t a t itohna t p l a y sa n i m p o r t a nrt o l ei n c r i t i c a ll i n g u i s t i casn d
c r i t i c a ld i s c o u r saen a l y s i sa,s w h e na n e w s p a p ehre a d l i n sea y s , ' F i f t e e n Demonstrators
Shot in Riots',therebyomittingto mentionthat they were shot by police(frew, I979:
eTff).
Whena narrativevisualproposition hastwo participants, oneis the Actor,the otherthe
Goal.fheActor in sucha transactionalprocess is not so muchthe participantwhichmoves
(as in the non-transactional process)as the participantwhich instigates the movement,
and if we hadto givea verbalparaphrase of a transactional process we wouldprobablyuse

O Fig2.16Goat
Nar rat i verePresentati ons 65

O Fig 2.17 Gulf War Diagnn (SydneyMoming Herald,I4 Fehruary1991)

a transitiveverb,a verbthat tal<es an object(e.g.'transport' or'send'insteadof Iintransi-


tivel'move').
In'The Britishusedguns/(figure2.1),thetwo menarethe Actor,theAborigines arethe
Goal.Thereis, in fact, a second transactional process: there are also vectors formed by
linesthat can be drawn from the heads of the Aborigines to the fire, and so constitute a
p r o c e s si n w h i c ht h e A b o r i g i n easr e t h e A c t o r a n d t h e f i r e t h e G o a l . ' T h eA b o r i g i n e s
s u r r o u ntdh ef i r e ' ,o n ec o u l dt r a n s c o d eB.u tt h e p o i n ti s n o tt o f i n da v e r b ael q u i v a l e nt th;e
point is to establishthat the Aborigines are represented as Actor and the fire as Goal.
Because of its smaller sizeand placement further towards the background, this process is a
, m i n o rp r o c e s se, ,m b e d d eidn t h e m a j o rp r o c e s sT.h ew h o l ec o u l db e t r a n s c o d eads ' T h e
B r i t i s hs t a l kt h e A b o r i g i n ews ,h os u r r o u ntdh ef i r e ' .
it is possible to arguethat structuressuchasthe ShannonandWeaverdiagram(figure
2.2) havebeenaffectedby the fact that Westernculturegivessuchcentralityto language
that the structureof English,wlth its lexicaldistinctionof verbs/processes and nouns/
objects,may haveacted as a model for a semiotic schema. So arrows as vectors/processes
and boxesas participants/nouns may be a more or lessunconscious translationfrom
Narrati verepresentati
ons

language intothe visual.However, it is importanthereto insiston the distinctorganization


of the two modes.The visualstructureof arrowsand boxesconveysa strongsenseof
' i m p a c t i n go' r ' t a r g e t i n gw
' , h i c hi s q u i t ea b s e nitn t h ev e r b atlr a n s l a t i o nwsh i c hc o m em o s t
immediately to mind.Thevisualstructureforegrounds procedure oversubstantive content,
the act of impacting'overwhat makesthe impact,more or lessin the way that, for
instance, marketingexpertsare oftenmoreconcerned aboutstrategies for reachingcon-
sumersthan about the goodsand servicesthat shouldreachthem, or that pedagogic
expertsare more concerned about the format of classroominteractionthan about the
contentof lessons.

i--._:--* ,- . z-: - -
-\\- F /a--
------'--"

O Fig2.l8 Speechcircuit(deSaussurelgT4tlgf6D

S o m et r a n s a c t i o n satl r u c t u r easr e b i d i r e c t i o n aela, c hp a r t i c i p a npt l a y i n gn o wt h e r o l e


of Actor,nowthe roleof Goal,as for instancein de Saussure's well-known'speechcircuit'
diagram ( f i g u r e2 . 1 8 ) i n w h i c h ' A ' a n d ' B ' a r e n o ws p e a k enr ,o w l i s t e n eIrt. i s n o t a l w a y s
c l e a rw h e t h ebr i d i r e c t i o ntarla n s a c t i o nasr e r e p r e s e n t eads o c c u r r i n sgi m u l t a n e o u solryi n
succession, althoughthereis a tendency to useonearrowwith two headsto signifysimul-
taneity,and two arrowspointingin differentdirectionsto signifysequentiality. In figure
2.I8,for instance, sequentiality is realizedby two separatedottedlines(line and arrow-
head,connection anddirectionality, are separate elements in this diagram).

O Fig 2.19 Simultane0us


and sequentialbidirectionality

We will refer to the participantsin suchstructuresas Interactors.to indicatetheir


d o u b l er o l e .
Narrativerepresentations. 67

2 Reactionalprocesses

Whenthevectoris formedby an eyeline, bythe directionof the glanceof oneor moreof the
represented participants, the process is reactional, andwe will speaknot of Actors,but of
Reacters, and not of Goals,bul of Phenomena.The Reacter, the participantwho doesthe
-
, u s tn e c e s s a r ibl ye h u m a no, r a h u m a n - l i kaen i m a l a c r e a t u r e
l o o k i n gm w i t h v i s i b l ee y e s
that havedistinct pupils, andcapableof facialexpression. The Phenomenon maybeformed
eitherby another participant, the participant at whomor which the Reacter is looking,or
b y a w h o l ev i s u a lp r o p o s i t i o n , s t
f o r e x a m p l ea, t r a n s a c t i o n a l r u c t u r eI .
n f i g u r e2 . ! 5 , f o r
instance, the old man and his gesture form the Phenomenon, whilethe young boy is the
Reacter. In figure2.20,an advertisement for mineralwater,the man is Actor in a trans-
actionalaction process in which the water is Goal ('The man drinl<swater', one might
transcode): the wholeangleof hisbodyformsa strongvectorbetween the two represented
participants. Thisprocess('Man drinkswater')thenbecomes the Phenomenon of a reac-
-
tional structurein whichthe womanis Reacter a vectotformedby the directionof her
glanceandthe angleof her left arm,leadsfrom herto the drinl<ing man.Shereactsto his
actionwith a smileof approval (the precise natureof reactionsis colouredin by facial
expression). Themanas doer,thewomanas faithfuladmirerof hisactions,is a distribution
of roleswhich,as Goffmanhasshowninhis GenderAdvertisements (I976), is verycom-
( b u t
m o n i n a d v e r t i s e m e n t s n o t o n l v i n a d v e r t i s e m e n t s ) : ' W hae nm a n a n d a w o m a n

T..-::

lfffi tjesi?;s

i"*r'-*,tfiil.,
es-i*$-ir-;:,i.i}fri i
;T .*.*"'*a{&,:s
f;w$$$ryrrwr-i;r*;Fi{i

O Fig2.2O Vittel advertisement(/Verlde4 5 Decenber 1987)


N arrat i ve representati ons

c o l l a b o r a tien a n u n d e r t a k i n tgh, e m a n i s l i k e l yt o p e r f o r mt h e e x e c u t i vreo l e ' ( G o f f m a n ,


7976:3D.
Like actions,reactionscan be transactionalor non-transactional. In the latter case
t h e r ei s n o P h e n o m e n oans,i st h ec a s ew i t ht h e l o o ko f t h eo l dm a ni n f i g u r e2 . 1 5 .I t i st h e n
l e f tt o t h e v i e w e tro i m a g i n w
e h a t h e o r s h ei s t h i n k i n ga b o u to r l o o k i n ga t , a n dt h i s c a n
createa powerfulsenseof empathyor identification with the represented participants.
Sometimesphotographers or picture editors crop photosback to close-upsof non-
transactionalReacters who look bored,or animated,or puzzled,at something we cannot
see.This can becomea sourceof representational manipulation. A caption may,for
instance, suggestwhat the Reacteris lookingat, but, needless to say,it neednot be what
t h e R e a c t ewr a sa c t u a l l yl o o k i n ga t w h e nt h e p i c t u r ew a st a k e n .S t u a r tH a l l ( 1 9 8 2 )h a s
d e s c r i b ehdo w t h i sk i n do f m a n i p u l a t i oi sn u s e di n p r e s sp h o t o g r a p hosf p o l i t i c i a n s .

3 S p e e c hp r o c e s sa n d m e n t a lp r o c e s s
A s p e c i akl i n d o f v e c t o rc a n b e o b s e r v eidn c o m i cs t r i p s t: h e o b l i q u ep r o t r u s i o nosf t h e
t h o u g h tb a l l o o n a s n dd i a l o g u e b a l l o o ntsh a t c o n n e cdt r a w i n g o s f s p e a k e rosr t h i n k e r st o
t h e i rs p e e c h o r t h o u g h tU. n t i l r e c e n t l tyh e yw e r ec o n f i n e d t o t h e c o m i cs t r i p s a , lthough
therehave,of course, alsobeenspeech processes in medievalart,for instancein the form of
r i b b o n se m a n a t i n fgr o m t h e s p e a k e r 'm s o u t h .T o d a yt h e y i n c r e a s i n gcl yr o p u p i n o t h e r
contexts, too; for instance, in connection with quotesin schooltextbooksor on the screens
of automaticbanktellers.Liketransactional reactions, theseprocesses connecta human
(or animate)beingwith 'content',but wherein transactional reactionsit isthe contentof a
p e r c e p t i o inn,t h e c a s eo f t h o u g h b t u b b l eas n ds i m i l a rd e v i c eist i s t h e c o n t e not f a n i n n e r
mentalprocess(thought,fear,etc.),and in the caseof speechvectorsthe contentof the
s p e e c hH. a l l i d a y( 1 9 8 5 :2 2 7 f f . )c a l s t h i s k i n do f s t r u c t u r e ' p r o j e c t i vTeh' .e P h e n o m e n o n
o f t h e t r a n s a c t i o n aRle a c t i o na,n dt h e c o n t e not f t h e d i a l o g u e b a l l o o no r t h o u g h b t alloon
are not represented directly,but mediatedthrougha Reacter, a'Senser'(in the caseof a
t h o u g hb t a l l o o no) r a ' S p e a k e r(' i n t h ec a s eo f t h e d i a l o g u b ea l l o o n ) .

processes
4 Conversion

The Shannonand Weavercommunication model (figure2.2) forms a chain of trans-


a c t i o n apl r o c e s s eTsh. i sc h a i n i n rge s u l t si n a t h i r d k i n do f p a r t i c i p a nat ,p a r t i c i p a nwt h i c h
is the Goalwith respectto oneparticipantandthe Actor with respectto another. Shannon
a n dW e a v e r ' s ' t r a n s m i t t e rs' iusc ha p a r t i c i p a nat ,c t i n ga s G o a lw i t h r e s p e ct to t h e ' i n f o r -
m a t i o ns o u r c e ' a nadsA c t o rw i t h r e s p e ct to t h e ' r e c e i v e rW ' . ew i l l c a l lt h i sk i n do f o a r t i c i -
panta Relay(our useof thisterm obviously differsfrom that of Barthesll977l, who uses
it to denotean image-textrelationin which the text extends,rather than elaborates
[ ' a n c h o r s ' tJh, e v i s u a li n f o r m a t i o n R s o n o t j u s t p a s so n / i n u n c h a n g ef d
) .e l a y d o r m ,w h a t
they receive;they alwaysalsotransformit - for instance, in the caseof communication
models, from 'lettersandspaces'into'electricalpulses', or,as in figure2.23,from 'grasses,
sedges andffowers'into'urine,droppings anddeadcarcasses'.
Narrat i verepresentatIons

O Fi92.2l Retay

W h i l et h e S h a n n o an n dW e a v em process
r o d e lr e p r e s e nct so m m u n i c a t i oans a c h a i n e d
with a beginningand an end,henceas an agentiveprocess/a processset into motionby an
Actor,othermodels, suchasthat shownin figure2.22,represent communication as a cycle,
a n d i n t h a t c a s ea l l t h e p a r t i c i p a n tasr e R e l a y sa,n da g e n c yi s m o r ew e a l < lsyi g n i f i e dT'h i s
k i n d o fp r o c e s s , w h i c h w e wci a l ll l a C o n v e r s i o n p r o c e s s , i s e s p e c i a l l y c o m m o n i n r e p r e s e n -
tationsof naturalevents;for instance, food chaindiagramsor diagrammatic representa-
. u t ,a s f i g u r e2 . 2 2s h o w si,t c a na l s ob e a p p l i e dt o h u m a n
t i o n so f t h e h y d r o l o g i c cayl c l e B
(inter)action, andwhenthis happens human(inter)actlon is represented asthoughit wasa
naturalprocess.

("..")

/en"oo") /o""oo")
Interpreter Interpreter

t"'o0"7 \=*"0"7
t*"?
O F i g 2 . 2 2C o m m u n i c a t i o n m o d e l ( f r o m w a t s o n a n d H i l l , l 9 S 0 : 1 4 7 )
70 . Narrative representations

!. il

i'atitilt t'i* !:i3t*IeS


i ! / " -
ilr*r****rrs l?,i i r+' i:i)3::r!i,,p,1i*::*
..-..-..-..-..._.,"_.-,-,".-.-.-..,"_..-_,1 !:ji-1{ta,!1jir!:ll

.si:i]#*+'d*ihan. &
i! *;t
'q]wsG
i,i:|,$irn iltiii$i #\4 AiSSal
l;trlf,i! lr: :14v* riii4 qr$ii!{
ia!t*.i 1+!s Y*rf$ ls.P.**sl
n.!,i. :-3iirriis s:!*,Sr#!!{q
*"'***uusll*.
ilillf"
ff" i*'-'q * x l ' :

---------"----l ., 'lri&#4:S!ilqt:l
i----" '' :'cq$e{ &6aS:$
I Ptr{.*s$*s$fs l3'.: i
i-----,_.--.--.---.:-i

O Fig 2.23 Arctictundrasystem(Saleetal.,1980)

5 G e o m e t r i c asl y m b o l i s m
F i g u r e2 . 2 4 ,a n o t h e r ' c o m m u n i c a t im oon d e l /d/ o e sn o t i n c l u d ea n y p a r t i c i p a n t Ts .h e r ei s
o n l ya v e c t o ri,n d i c a t i ndgi r e c t i o n a l i b
t yy m e a n so f a n ' i n f i n i t y ' s i g nr ,a t h e rt h a nb y m e a n s
of an arrowhead.Dance'sdiagramis in fact not so much a communication modelas a
'metadiagram'which showsusa process in isolation, in orderto discuss why helicalvectors
are moresuitablefor the representation of communication than,for instance, straightor

O model(from Dance,1967)
Fig 2.24 Communication
Narrative representations . 7l

c u r v e da r r o w sD. a n c ed o e st h i sb y p o i n t i n g c e a n i n gosf i n t r i n s i p
a t t h es y m b o l im croperties
o f t h e h e l i xA
. c c o r d i ntgo D a n c et h
, es h a p eo f t h e h e l i x :

c o m b i n etsh e d e s i r a b lfee a t u r e o s f t h e s t r a i g h lt i n ea n do f t h e c i r c l ew
, h i l ea v o i d i n g
t h e w e a k n e sosf e i t h e r . . . . l t g i v e st e s t i m o n tyo t h e c o n c e ptth a t c o m m u n i c a t i o n ,
whilemovingforward,is at the sametimecomingbackuponitselfandbeingaffected
by its pastbehaviour, for the comingcurveof the helixis fundamentally affectedby
t h e c u r v ef r o mw h i c hi t e m e r q e s .
(Dance,1967)

Imagesof this kind usepictorialor abstractpatternsas processes whosemeanings are


c o n s t i t u t ebdyt h e i rs y m b o l ivca l u e sa,n ds oe x t e n d t h ev e c t o r i avl o c a b u l a rbyy d r a w i n go u r
a t t e n t i o tno p o s s i b i l i t i eb se y o n tdh e d i a g o n aal c t i o nl i n eo r t h es i m p l ea r r o w :c o i l ss, p i r a l s ,
heIixes.
Variantsof the arrow may affectthe meaningof the processin narrativediagrams. A
curvedarrow,for instance, partakesof the symbolicvalueof the circle,so that the process
is represented as'natural'and'organic'(seefigure2.22).Vecforsmayalsolrc attenuated,
b y t h e u s eo f d o t t e dl i n e sb, y m a k i n gt h e a r r o w h e asdm a l l e ro r l e s sc o n s p i c u o ui ns o t h e r
w a y so/ r b y p l a c i n gi t i n t h e m i d d l er,a t h e tr h a na t t h ef r o n to f t h e l i n ew , h i c hd i m i n i s h et hse
senseof impacting'and 'targeting',and causesthe meaningof the vectorto movein the
direction o f m e r ec o n n e c t i v i (t vs e ef i q u r e2 . 2 5 ) .

Attenuatedvectors

O Fig 2.25 Attenuatedvectors

The vectorialrelationmay also be amplified,by meansof bolderarrows(seefigure


2.26),whichperhaps a certaindensityof 'traffic',as in figure2.17,the diagramof
suggest

#
A m p l i l i e d v !c t o r Ampliliedvector
i nd icati ng i nd icatj ng
cenSlty frequency

O Fig 2.26 Amplitied vectors indicating density and frequency


72 . Narrative representations

the attackon l(hafji,or by the useof a numberof arrows,whichmaysuggest the f requency


o r m u l t i p l i c i tw
y i t h w h i c ht h e p r o c e sosc c u r s .
In imagea s s i m i l a re f f e c tc a nb e a c h i e v ebdy m a k i n gt h e d i a g o n aal c t i o nl i n e sm o r eo r
lessconspicuoust moreor lessdominantin the composition as a whoie.

6 Circumstances

A s w e i n d i c a t eidn o u r d i s c u s s i oonf ' T h e B r i t i s hu s e dg u n s ' ( f i g u r 2


e . I ) , n a r r a t i v ei m a g e s
may containsecondary participants, participantsrelatedto the main participants, not by
meansof vectors,but in otherways.Wewill referto theseparticipants, followingHalliday
(1985), as Circumstances.They are participantswhichcould be left out without affecting
the basicproposition realizedby the narrativepattern,eventhoughtheir deletionwouldof
courseentaila lossof information.
LocativeCircumstances relateotherparticipants to a specificparticipantwe will call
Setting.Thisrequiresa contrastbetweenforegroundand background, whichcan be real-
izedin oneor moreof the followingways:(1) the participantsin the foregroundoverlap,
and hencepartiallyobscurethe Setting;(2) the Settingis drawnor paintedin lessdetail
(or, in the caseof photography, has softer focus),'G) the Settingis more muted and
desaturated in colour,with the variouscoloursall tendingtowardsthe samehue,usuallythe
blueof distance; (4) the Settingis darkerthanthe foreground, or lighter, sothat it acquires
an 'overexposed', ethereallook.Theseformalfeaturescan occurin variouscombinations,
andtheyare all gradients -'more-or-less,, ratherthan 'either-0r,,features.
As we will discussin moredetailin chapter5, settingshaveimportance for the realiz-
ation of visual modality.The Settingsthemselves can of coursebe read as embedded
analyticalprocesses ('Thelandscape consistsof grass,treesand rocks,).
Thetoolsusedin actionprocesses are oftenrepresented as Circumstances of Means.If
this is the case,thereis no clearvectorbetween the tool and its user.Thetoolsthemselves
may,of course/ constitutethe vectorswhichrealizethe actionprocesses/ aswith the gunsin
' T h eB r i t i s hu s e dg u n s (' i n
f i g u r e2 . r ) , a n dt h e yn e e dn o tb eo b j e c t sF. o ri n s t a n c e , wweo u l d
i n t e r p r e t t hgee s t u r eo f t h e o l d m a ni n f i g u r e2 . 1 5a s a n o n - t r a n s a c t i o n aac tl i o n( ' T h eo l d
man addresses an unseenparticipant'),and his handsas a circumstance of Means('The
old manaddresses an unseen participantwlthhishands,).
Figure2.27 showsa penguinwith her baby.Thereis againno vectorto relatethe two.
Yetthe penguinand her babyclearlyform two distinctparticipants: this is a pictureof a
penguinwith ababy.In sucha casewe will interpretthe relationas a Circumstance of
AccompanimeA n ts. t h e p i c t u r ec o n t a i n ns o v e c t o ra n dd i s p l a ytsh e p e n g u i n m o r eo r l e s s
frontally,againsta de-emphasized background, we interpretit as'analytical,, asthe kindof
picturemorelikelyto illustratea text givingdescriptive information aboutpenguins thana
storyaboutwhat penguins do.
Narrativerepresentations' 73

O Fig 2.27 Penguinwith baby(oakleyefal.,1985)

SUMMARY

F i g u r e2 . 2 8 s u m m a r i z et sh e d i s t i n c t i o nwse h a v ei n t r o d u c eidn t h i s s e c t i o n .F' o l l o w i n g


H a l l i d a y( 1 9 8 5 ) ,w e h a v ec a l l e dp r o c e s s et hsa t c a nt a k ea w h o l ev i s u a l( o r v e r b a l )p r o p -
osition as their 'object' proiective,and the othersnon-proiective.The squarebrackets
i n d i c a t ea s i n g l ec h o i c ef;o r i n s t a n c e , ' a t r a n s a c t i o n a lc t i o ni s e i t h e ru n i d i r e c t i o n oa rl
b i d i r e c t i o n a l ' . 0 ucrl a i m i s t h a t t h e ' c h o i c e s ' i nf i g u r e2 . 2 8 c h a r tt h e p r i n c i p aw l a y s in
i n o f ' d o i n g ' a n d
w h i c h i m a g e sc a n r e p r e s e nt th e w o r l d ' n a r r a t i v e l y ' -t h a t i s , t e r m s
' h a p p e nn ig ' .
74 , Narrativerepresentations

- Unidirectional
Transactional -----N
I
Action -----N L Bidirectional
L Non-transactional
lon-lrolective
{ ( Non-transactional)reaction

^**,*{
-*l
,a"a"rra, P.oj(tiw-{ Mental process

VerbaI process

.u"rr",'""
*f
Non agentive: Conversion

structures
Setting

L Cirauartunaar{ lvleans

Accompaniment

O Fig 2.28 Narrativestructuresin visualc0mmunication

REALIZATIONS

UnidirectionaI transactional A v e c t o rf ,o r m e db y a ( u s u a l l dy i a g o n a l )
action depictedelement, or an arrowlconnects two
p a r t i c i p a n tas n
, A c t o ra n da G o a l .
Bi di rectionaI transacti onaI A v e c t o rf ,o r m e db y a ( u s u a l l dy i a g o n a l )
action depictedelement, or a double-he adedarrow,
connects two Interactors.
Non-transact
i ona I acti on A v e c t o rf ,o r m e db y a ( u s u a l l dy i a g o n a l )
depictedelement, or an arrowtemanates from
a participant, the Actor,but doesnot pointat
anyotherparticipant.
Actor Theactiveparticipantin an actionprocessis
the participantfrom whichthe vector
emanates or whichis fusedwith the vector.
Goal Thepassive participantin an actionprocess
is the participant at whichthe vectoris
di rected.
Interactors Theparticipantsin a transactional action
process wherethe vectorcouldbe saidto
emanatef rom,andbe directedat, both
participants.
TransactionaI reacti on An eyeline vectorconnects two participants, a
Reacteranda Phenomenon.
Non-transacti onaI reacti on An eyeline vectoremanates from a participant,
the Reacter, but doesnot pointat another
participant.
Narrativerepresentati ons

Reacter Theactiveparticipantin a reactionprocessis


the participantwhoselookcreatesthe eyeline.
Phenomenon Thepassive participantin a (transactional)
r e a c t i o ni s t h e p a r t i c i p a nat t w h i c ht h ee y e l i n e
is directed;in otherwords,the participant
whichformsthe objectof the Reacter's look.
Thesameterm is usedfor the participant
( v e r b aoi r n o n - v e r b ael )n c l o s ebdy a ' t h o u g h t
b u b bl e ' .
Conversion A process in whicha participant, the Relay,is
the Goalof oneactionandthe Actor of
another. Thisinvolves a chanqeof statein the
participant.
Mental process A v e c t o rf o r m e db y a ' t h o u g h tb u b b l e ' o a r
s i m i l a rc o n v e n t i o ndael v i c ec o n n e c ttsw o
p a r t i c i p a n ttsh,e S e n s ear n dt h e P h e n o m e n o n .
Senser T h ep a r t i c i p a nf rt o mw h o mt h e ' t h o u g h t
bubble'vectoremanates.
Verbal process A vectorformedby the arrow-likeprotrusion
of a'dialogue b a l l o o no' r s i m i l a rd e v i c e
connects two participants, a Sayerandan
Utterance.
Sayer Theparticipantin a verbalprocess from whom
t h e ' d i a l o g ubea l o o n 'e m a n a t e s .
Utterance T h e( v e r b a lp) a r t i c i p a netn c l o s eidn t h e
' d i a l o g u be a l l o o n ' .
Qailinn TheSettingof a process is recognizable
because the participantsin the foreground
overlapandhencepartiallyobscureit; because
it is oftendrawnor paintedin lessdetail,or,in
the caseof photography, hasa softerfocus;
and because of contrastsin coloursaturation
and overalldarkness or lightness between
f o r e g r o u nadn db a c k g r o u n d .
Means The Meansof a process is formedby the tool
with whichthe actionis executed.
It usuallyalsoformsthe vector.
Accompaniment A n A c c o m p a n i m eins ta p a r t i c i p a ni tn a
narrativestructurewhichhasno vectorial
relationwith otherparticipants andcannot
be interpreted as a SymbolicAttribute(see
c h a p t e3r ) .
76 . Narrative representations

V I S U A LS T R U C T U R E S
A N D L I N G U I S T I CS T R U C T U R E S

We havedrawnattentionto the fact that,whilebothvisualstructures andverbalstructures


canbe usedto express meanings drawnfrom a commonculturalsource, the two modesare
n o t s i m p l ya l t e r n a t i vm e e a n so f r e p r e s e n t i n g ' t sh ae m et h i n g ' .I t i s e a s yt o o v e r e m p h a s i z e
e i t h e rt h e s i m i l a r i t o
y r t h e d i f f e r e n cbee t w e etnh e t w o m o d e s . 0 n lay d e t a i l e d comparison
canbringout howin somerespects theyrealizesimilartypesof meaning, thoughin different
ways,while in other,perhapsmost respectsthey representthe world quite differently,
allowingthe development of the differentepistemologies we discussed in the previous
chapter.In this brief final sectionwe wishto explorethis in somedetailwith respectto
narrativevisualstructures.
By comparison to the structures we will discuss in chapter3,narrativevisualstructures
are comparatively easyto'translate';though,as we will see/therecertainlyis no one-to-
o n ec o r r e s p o n d e nLciek.e ' n o n - t r a n s a c t i oancatli o n s ' , ' o n e - p a r t i c i pma an t e r i apl r o c e s s e s '
( Halliday,1985: 103ff.) represent eventsas thoughthey bearno relationto, and haveno
consequences for, participantsotherthan the Actor.In Many peoplemigrated onecannot
add a secondparticipantto this clauseand say,for instance,Many peoplemigratedtheir
relatives,althoughone can of courseadd circumstances: Many peoplemigrated to Aus-
tralia. And,like 'transactionalactions','two-participantmaterialprocesses' (Halliday,
1985: 103ff.) involvetwo participants(e.9.MigrantsinvadedAustralid. But linguistic
'Events'and v i s u a 'lE v e n t s ' a r eq u i t ed i f f e r e n tL. i n g u i s t i E
c v e n t sh a v ep r o c e s s et hsa t a r e
'happenings'which cannothavean Actor,as in Many of my relativesdied.In the caseof
visualEvents,the Actor is left out, but couldhavebeenused.Theyare the equivalent of
passives with agentdeletion,of clauseslike Many of my relativeswerekilled, ratherthan
of clauseslike Many of my relativesdied.Toshowsomeonedying ii is necessary to show
someone beingkilled,or to showsomeone performingan actionthat represents hisor her
death.Also,whilein Englishmanyprocesses cantakea third participant, the 'Beneficiary'
(traditionally'indirect object'in,e.9.,Mary gavehim theboob,in imagesthe possibility of
sucha third participantdoesnot exist.What is a Beneficiary in Englishbecomes a Goalin
images('shemessage-sends him' insteadof 'shesendshim a message').
0n the other hand, Englishlacksthe visual mode'sstructuraldevicesto represent
e v e n t sa s c y c l i c a(l a l t h o u g h l i n g u i s t ipca r t i c i p a n tcsa nh a v ea d o u b l er o l ei n E n g l i s -h f o r
instance, in exampleslike He madethemdo it,wherethemis Goalof makedo as well as
A c t o r o f d o ;c f . H a l l i d a y1,9 8 5 : 1 5 3 ) . N o r i s t h e r ea n ' i n t e r a c t i o n a l ' p r o c eTs o s .r e a l i z e
what we havecalled'lnteractors',Englishwouldhaveto makeuseof reffexive pronouns.
Consider, for instance,the problemof trying to 'translate'figure 2.18, de Saussure,s
' s p e e c hc i r c u i t ' d i a g r a mi,n t o E n g l i s hA. s i n g l ev i s u a lp r o c e s s
i n d i c a t es o m e t h i n fgo r
w h i c h i,n E n g l i s hw, e n e e da t l e a s ft o u r c l a u s e s : ' A s p e a ktso B ' , ' B s p e a ktso A , , ' A l i s t e n s
to B','B listensto A'. Howcan onerenderthis in onec{ause? 'A and B communicate with
eachother'? But that causes and 'A' 'B'to losetheir separateidentity,transforminga
reciprocal, bidirectional transactionintoa jointlyauthorednon-transactional action.
W h a t w e h a v ec a l l e dt h e ' n o n - t r a n s a c t i o nr ae la c t i o n ' i si n s o m ew a y sa k i n t o w h a t
H a l l i d a y c a l l s t bh e h a v i o u r a l p r o c e s s ( 1 9 8 5 : I 2 B ) , a p r o c e s s t y p e w h i c h c a n t a k e o n l y o n
Narrativerepresentations. 77

participant(whomustbe human)andservesto realizea restrictedfieldof action,the field


o f ' p h y s i o l o g i c aaln d p s y c h o l o g i cbael h a v i n g('1 9 8 5 : 1 2 8 ) . B u t t h e m e a n i n gosf v i s u a l
'non-transactional reactions'form a morerestricted field,tied up astheyarewith onekind
o f b e h a v i o u- r l o o k i n g .
Projectiveprocesses -that is,mentalandverbalprocesses - playan importantpart in
E n g l i s ha,n di t i s p o s s i b lteo d i s t i n g u i sahn u m b eor f d i f f e r e ntty p e so f e a c ho n t h e b a s i so f
f o r m a lg r a m m a t i c ac lr i t e r i a( H a l l i d a y1,9 8 5 :1 0 6 f f . ,1 2 9 ) .M e n t a p l r o c e s s ef o s /r i n s t a n c e ,
includeprocesses of perception('see','hear',etc.),processes of affection('like','fear',
'wish'e , t c . )a n dp r o c e s s eosf c o g n i t i o n ( ' k n o w ' , ' t h i n k ' , ' b e l i e veet c' ,. ) .E a c hh a sa S e n s e r ,
t h e p e r s o nw h o d o e st h e s e e i n go, r l i k i n g ,o r k n o w i n g( t h i s h a s t o b e a p e r s o no/ r a
p a r t i c i p a nrte p r e s e n t eads h u m a n )a, n d a P h e n o m e n osno, m e o n oe r s o m e t h i nsge e no/ r
l i k e d ,o r k n o w n ,b y t h e S e n s e rP. h e n o m e nm a a y b e r e a l i z e db y p a r t i c i p a n tos r w h o l e
structures,just as in the caseof images.In Manypeoplewant to migrate to Australia,the
clausefo migrate to Australiais Phenomenon; this sets mentalprocesses apart from
a c t i o n sa n dt r a n s a c t i o nwsh, i c hc a n n o ht a v ea c l a u s ea s G o a l .B u tw h a tw e h a v ec a l e dt h e
'transactional reaction'can,if onewishesor needsto, be relatedto only a subsetof the
p e r c e p t i opnr o c e s sb ,e c a u sneo n - v i s u aPlh e n o m e ncaa n n odt i r e c t l yb e r e a l i z e idn t h ev i s u a l
s e m i o t i cM . e n t a lp r o c e s s ef os r m ,a s w e h a v es h o w no, n l ya m i n o rc a t e g o r yi n t h e v i s u a l
semiotic;as far as we cansee,thereare no structuralvisualdevices for makingthe strong
d i s t i n c t i obne t w e e n ' c o g n i t i o n ' a' a
n fdf e c t i o np' r o c e s s e t hsa t h a sc o m et o c h a r a c t e r i zt hee
ideationalresources of English. Thecinema,however, hasdeveloped afairlyextensive setof
projectiveconventions for realizingdifferentkindsof mentalprocesses suchas memories,
d r e a m sh, a l l u c i n a t i o nasn,ds o o n .
In English,verbalprocesses differ from mentalprocesses in that they do not needa
human'Sayer'(onecan say Thedocumentsaid that..., but not Thedocumentthought
t h a t . . . ) . 0 n t h e o t h e rh a n d ,l i k em e n t a p l r o c e s s et sh,e yc a nt a k ew h o l ec l a u s eas s t h e i r
object,andthis in two differentways- in theform of Reported SpeechGs in He saidtthatl
he had no ided and in the form of QuotedSpeech(e.sinHe said'l haveno idea).fhere
s e e m st o b e n o d i r e c t s, t r u c t u r awl a y o f e x p r e s s i nt hgi s k i n d o f d i f f e r e n cvei s u a l l y . ' D i a -
l o g u eb a l l o o n sa' l w a y sq u o t e .
We haveidentified onlythreedifferenttypesof circumstance in images:location,means
a n da c c o m p a n i m e n t . A l l t h reexei s ti n E n g l i s h( H a l l i d a y , 7 9 8 5 : 1 , 3 7 1bfu. )t ,t h e r et h e ya r e
b y n o m e a n tsh eo n l yt y p e sE . nglisa h l l o w sa l l k i n d so f i n f o r m a t i otno b ea d d e dt o t h e b a s i c
narrativepropositionconveyed by the process('What happened?') and the participants
( ' W h oo r w h a tw a si n v o l v e d ? 'i )n;f o r m a t i oanb o u t i m e( ' W h e nd i d i t h a p p e n ? ' ; ' H olw ong
d i d i t l a s t ? ' )a; b o u tp u r p o s (e' W h a td i d i t h a p p e n f o r ? ' ) ;c a u s e( ' W h yd i d i t h a p p e n ?a' )n d
s o0 n .
Thefollowingtablegivesan overview of someof the correspondences betweenlinguistic
andvisualnarrativeorocesses:
Narrat ive representati ons

Table2.7 Nafiatiue processin languageand yisual communication

VisuaI narrati ve processes L i nquisti c narrative cIauses

Non-transactional action (Actor)materialprocess


0ne-participant ('action')
Unidirectional transactionalaction materialprocess
Two-participant
Event Passive clausewith agentdeletion
transactional
Bidirectionaltransactionalaction
Non-transactional reaction process
Behavioural (fieldof looking)
Transactional reaction Mentalprocess:perception (visualonly)
Mentalprocess Mentalprocess(cognition andaffection)
Verbalprocess (quotation)
Verbalprocess
(affection
Verbalprocess )
Conversion

Comparisons suchas thesecan highlightwhichwaysof representing the world can be


realized l i n g u i s t i c a lw y n dw h i c h( m o r eo r l e s s )i n b o t hw a y sT. h i s i,n t u r n ,i s
l yh, i c hv i s u a l l a
usefulas a background for analysingrepresentation in multimodaltexts:photographs and
their captions,diagramsand their verbalglosses, storiesand their illustrations. If, for
instance, a diagramshowsan arrowemanatingfrom a participantlabelled'environment'
a n dd i r e c t e a d t a n o t h epr a r t i c i p a nl ta b e l l e'dm e s s a g e ' , t h ae 'nl i t e r a l ' t r a n s l a t i owno u l db e
'the environment acts uponthe message/. If the accompanying text saysthat the'com-
m u n i c a t i o np r o c e s s ' i s ' i n t e r a c t i n w g i t h f a c t o r s ( o r s t i m u l i )f r o m t h e e n v i r o n m e n t '
( W a t s o na n d H i l l ,1 9 8 0 : 1 4 ) ,i t ' m i s t r a n s l a t e sa' n, d t h e m i s t r a n s l a t i oi sn n o t d u et o t h e
l i m i t a t i o nosf e i t h e rE n g l i s h o r v i s u acl o m m u n i c a t i oAn .l i t e r a 'lt r a n s l a t i o n ' o' tf h es o u r c e
sendsa message to the receiver', on the otherhand,is not possible: the spatialrepresenta-
tion of verballyconceived ideaschanges the ideasthemselves, and viceversa.In the next
c h a p t ew r e w i l l s h o wf u r t h e re x a m p l eosf t h i sp r o b l e m .

Notes

1 We are gratefulto the Danisha r t h i s t o r i a nL i s e M a r k f o r p o i n t i n go u t t h a t , i n t h e


'on its side',i.e.tilted by 90
p r e v i o ues d i t i o no f t h i s b o o k ,t h i s paintingwas reproduced
degrees.
2 Wewouldliketo thank BenteFogedMadsenfor someusefulcomments that haveledto
improvements of this diagram.
3 C o n c e p t u ar le p r e s e n t a t i o n s :
d e s i g n i n sg o c i a lc o n s t r u c t s

CLASSIFICATIO
NAL PROCESSES

In the previouschapterwe notedthat visualstructuresof representation can either be


narrative, presenting unfoldingactionsand events,processes of change, transitoryspatial
arrangements, or conceptual, representing participantsin termsof their moregeneralized
and moreor lessstableandtimelessessence/ in termsof class,or structureor meaning.
It is to the lattercategoryof representational structuresthat we nowturn, beginning
with classificational processes. Classificational processes relateparticipantsto eachother
i n t e r m so f a ' l < i n do f ' r e l a t i o na, t a x o n o m ya: t l e a s to n es e t o f p a r t i c i p a n tws i l l p l a yt h e
role of Subordinafeswithrespectto at leastoneother participant,the Superordinafe. We
havealreadycomeacrossan examplein the left-handpictureof figure2.1,wherethethree
participants - the axe,the basketandthe woodensword- wererepresented as 'species' of
t h e s a m e ' g e n u sa' ,s a l l b e l o n g i ntgo t h e s a m eo v e r a r c h i ncga t e g o r yI n . t h i se x a m p l e the
overarching categorywas not shownor named.Thestructurewasa CovertTaxonomy, a
taxonomyin whichthe Superordinate is inferredfrom suchsimilaritiesas the viewermay
perceive to existbetween the Subordinates, or only indicatedin the accompanying text,as
i n f i g u r e3 . 1 .
0 n ev i s u acl h a r a c t e r i s tiiscc r u c i ailn t h er e a l i z a t i oonf c o v e rtta x o n o m i etsh:ep r o p o s e d
e q u i v a l e n cbee t w e etnh e S u b o r d i n a t ei ssv i s u a l l yr e a l i z e db y a s y m m e t r i c ac lo m p o s i t i o n .
The Subordinates are placedat equaldistancefrom eachother,giventhe samesizeand
the same orientationtowardsthe horizontaland vertical axes.To realizethe stable,
timelessnatureof the classification, the participantsare often shownin a more or less
objective, decontextualized way.The backgroundis plain and neutral.Depthis reduced
or absent.The angle is frontal and objective.And frequentlythere are words inside
the picturespace.Thesefeatureswill be discussed in a later chapterunderthe heading
'modality'.
Classificatio pn r o c e s s ed so n o t ,o f c o u r s es,i m p l yr e f f e c t ' r e a l ' , ' n a t u r a l ' c l a s s i f i c a t i o n s .
Forparticipants to beput togetherin a syntagmwhichestablishes the classification means
that theywere judged to be members of the sameclass,andto be readassuch.'Naturaliza-
t i o n ' i s n o t n a t u r a lw
, h e t h e ri n i m a g e so r i n l a n g u a g eT.h e o r d e r i n gi n t h e i m a g ei t s e l f
producesthe relations.This makesit possiblefor the producerof an imageto classify
b a s k e t sa n d w e a p o n s( f r g u r e2 . I ) , o r z o o l o g i s t w
s ,i l d l i f ep h o t o g r a p h ear sn d A b o r i g i n a l
storytellers (figure3.1) as beingof the sameorder.
Coverttaxonomiesare often usedin advertisements, wherethe photographs may,for
instance, showarrangements of bottlesthat representthe varietyof productsmarketed
undera brandname,or arrangements of differentpeoplewho all usethe sameproduct.
Figure3.2 is a pagefrom Cosmopolitan magazine. What do thesewatcheshavein com,
mon?Theyall belongto the sameXposerange.
Conceptual representations

@ rig f.f cuide intertace(Microsoft,1994)

O t h e rt a x o n o m i essh o wa h i g h e d r e g r e eo f ( e x p l i c i to) r d e r i n gi n c l u d ea S u p e r o r d i n a t e .
Theyrepresentand namethe Superordinate within somekind of tree structure.In that
structurethe orientationis vertical,and the Superordinate is placedaboveor belowthe
S u b o r d i n a t eass,i n f i g u r e3 . 2 . f h e p a r t i c i p a n tm
s a y b e r e a l i z e vd e r b a l l yv,i s u a l l yo,r b o t h
verballyandvisually, but the processis alwaysvisual.
Taxonomies do not haveto be represented by formaldiagramswith simplelines;they
may be realizedin more realistfashion,for instance, by an actualtree in a'family tree'.
0 v e r tt a x o n o m i easr e u s u a l l y ' c h a i n e ds 'o, t h a t t h e ' i n t e r m e d i a t e ' p a r t i c i p a (net s. 9 .t h e
' i n o r g a n i sc u b s t a n c e s ' a n d ' o r g asnui cb s t a n c e s ' o
f i fg u r e3 . 3 ) w i l l b e S u p e r o r d i n awt ei t h
respectto someof the other participants, and Subordinate with respectto others.To
indicatethis we will coin the term Interordinate. In otherwords,overttaxonomieshave
levels,and participantsat the samelevelare represented as being,in somesense,'ofthe
s a m ek i n d ' .
Treestructures,however, are not only usedto realize'kind of' relations.'Reporting
d i a g r a m s 's, h o w i n gt h e h i e r a r c h i c aslt r u c t u r eo f c o m p a n i eas n d o t h e r o r g a n i z a t i o n s ,
and genealogical or evolutionary trees,usethe samestructure.This meansthat visual
grammarconflates, or at leastrepresents as verycloselyrelated,what would,in language,
be expressed by differentmeans.Conceptualclassification is represented by the same
representations. 8),
Conceptual

r F r r ' " & p * { f ' s A r , { $ c d F v . " . 1 ' ; . . s { ;fi i y s


.{.r.'",;s :r ; *tr i:?}J;1s1f.a.6+r I I r"5i1'a+lsr

@ fig f,Z Xposerange (Cosmopafifau,November2O0l: g4)


82 . Conceptual representations

Sourcesof siqns
I

-
Inorganic substances t l
organic substances

Natural Manufactuied Extraterrestrial Terrcstrial

I
Homo sapiens Sp!echlesscreatures

I
componetrts organisms components ofgdisms

O fis f.: Sourcesof signs(Eco,l976b:177)

structureas socialhierarchy; that is, the more generalidea is represented as similarto


greaterpower.As VirginiaWoolfhassaid,'General ideasare alwaysalsoGenerals' ideas.'
Furfher,hierarchies of concepts and hierarchies of socialpowerare represented as similar
to genealogies. In otherwords,the identityof an individual(or a species) is represented as
b e i n g ' s u b o r d i n a t e 'ittos ' o r i g i n s ' o r ' a n c e s t o ri sn 't h e s a m ew a y a s s p e c i f i c o n c e p tasr e
subordinated to moregeneraland abstractconcepts, and loweremployees or localbodies
to managers or centralorgans.
Diagrammatic tree structurescan take differentforms.The branches of the tree may
b e p a r a l l eol r o b l i q u es, t r a i g h to r c u r v e da, n d s o o n ( s e ef i g u r e3 . 4 ) . O b l i q u eb r a n c h e s

tre structure with


parallel branches

-\

/n\
tree slructure wilh
cuded bianches

AA AA

AAAA
piSf.l Treestructureswith parallel(straight)0bliqueand curyedbranchet
O
Conceptualrepresentations. 83

abstractsomewhatlessfrom the shapeof the treethan parallel branches, so that moreof


the symbolicmeaningof the tree can be preserved. Hencethey are commonin contexts
w h e r ea s e n s e o f ' g e n e r a t i o n ' a n d ' g r o w t hc' iosn n o t e da,sf o r i n s t a n cien g e n e a l o g i o e rsi n
the diagramsusedin'generativegramrnar', a form of linguistics whichpositsthat'surface
structures'aregenerated from (possiblyinnateand maybeuniversal)'deep structures'.
The contrastbetweenstraightand curvedbranchesis perhapssimilarto that between
participantsrepresented as boxesand participantsrepresentedas circles or ovals
( s e ec h a p t e 2r ) , a c o n t r a sbt e t w e etnh e ' m e c h a n i c a l ' a n d ' t e c h n o l o g i c a lt'haen'dn a t u r a l '
a n d' o r g a n i c ' .
Manytreediagramsare inverted('bottom-up'),andwhenthe branches are obliquethe
overallshapewill tend towardsthat of a pyramid.Such structuresare concerned with
hierarchyand hierarchicaldifferenceratherthan with clarity about levels:a readingof
levelsis possible, but not readilyfacilitated.Not all trees,however, are inverted. Sometimes
the specificor,in the caseof genealogies and evolutionary trees,the present,is placedon
h ,u m a n k i nadst h ep i n n a c loef e v o l u t i orna t h e tr h a na sb e i n g
t h et o p ,d e p i c t i n gf o, r i n s t a n c e
foreverdominated by its lowlyorigins.
Althoughclassificational structuresrepresent participantsin termsof their placein a
staticorderithe verballabelsand explanations whichmayaccompany themdo not always
do so.Theterm 'reportingdiagram',for instance, usesan activeprocess('report')rather
t h a na s t a t i co n es u c ha s ' i s s u b o r d i n a t o e ' . G e n e a l o g iaensde v o l u t i o n a tr rye e ss, i m i l a r l y ,
m a y b e g l o s s e db y v e r b sl i k e ' g e n e r a t e s ' , ' e v o l vienst o ' , ' b e g e t s a ' ,n d s o o n . V i s u a l l y ,
however, a hierarchicalorder is signified,a system.Thusthe visual representation can
b l u r t h e b o u n d a r i ebse t w e e n t h e d y n a m i ca n d t h e s t a t i c .I s t h e d y n a m i ci n r e a l i t yt h e
instantiationor enactmentof an underlyingsystem?0r is the static the systemization
and objectification of a dynamicand ever-changing reality?Suchquestionsbecomedif-
ficult to answerin a modewhich haserasedthe boundaries betweenthe schemaand the
b lu e n r i n t .
A s i m i l a rb l u r r i n gm a yo c c u rb e t w e e a n n a l y t i c a(l' p a r t o f ' ) a n dc l a s s i f i c a t i o n(a' kl i n d
o f ' ) p r o c e s s eTsh. e ' s t a c kd i a g r a mi'n f i g u r e3 . 5 c o u l db e c a l l e dc l a s s i f i c a t i o naanl ,d i s i n
fact so renderedin the accompanying text:'to the Latin word /mus/ correspondtwo
d i f f e r e ntth i n g sw h i c hw e s h a l lc a l l x , a n dx r ' ( E c o ,r 9 7 6 b : 7 8 ) .I n o t h e rw o r d sa, ' m o u s e '
i s a k i n do f ' m u s ' .B u t i t c a na l s ob e s e e na s a n a l y t i c a0l .n ec a na l s os a yt h a t t h e m e a n i n g
'mouse'is part of the meaningof 'mus'.And,mostimportantly, Ecohaschosenthe form of
the analyticaI diagram.

Mous

Mus

Rat

O fis f.l Semantictield diagram(Eco,l976b:78)


84 . Conceptual representations

Finally,classificational diagramsmay be rotatedthroughninetydegreesso that their


main orientationis alongthe horizontalaxis.Theyhavethen the orientationtypical of
narrativediagrams, and hencea dynamicconnotation. But they retainthe structureof the
classificational diagram.Theystill representthe relationbetweenthe participantsas a
system.Features from differenttypesof structureare abstracted andrecombined to create
patternsthat are ambiguously in betweenthe dynamicand the conceptual. When such
diagramsacquirearrows- as, for example,in systemnetworks(seefigure 3.8) - they
becomein fact dynamicand narrative.Yettheystill movefrom the generalto the specific,
in contrast(for example)to flowcharts.
Taxonomies and ffowchartsclearlyprovidetwo differentkindsof knowledge. The one
represents the world in termsof a hierarchical order.Its main concernis the rankingof
phenomena from the perspective of a singleunifyingterm,be it that of the originof things,
the most generalizing generalization, or that of the highestpower.The other describes
the world in termsof an activelypursuedprocesswith a clear beginning and an end (or
wa t e r i a l s ' a n'df i n i s h e d
' i n p u t ' a n d' o u t p u t ' , ' s o u r c e ' a n' d e s t i n a t i o n ' , ' r am p r o d u c t ' )I.t
hasa sequential progression and is goal-oriented. And,as we havenotedalready,system
networkssuchas we usein this bookattemptto combinethe two perspectives.
R e c e n t layn o t h e kr i n do f d i a g r a mh a sb e g u nt o g a i na s c e n d a n -c et h e ' n e t w o r k 'N . et-
worksseekto showthe multipleinterconnections betweenparticipants. Any participantin
a network('node')can form an entry-pointfrom which its environment can be explored,
and the vectorsor lines('links') betweentheseparticipantscan take on manydifferent
v a l u e st ,h e v a l u eo f s i g n i f i c a t i o(n' a m e a n sb ' ) , o f c o m b i n a t i o(n' a g o e sw i t h b ' ) , o f c o m -
position('a containsb'): the essence of the link between two participantsis that theyare,
in somesense,nextto eachother,or closeto eachother,associated with eachother.To
demonstrate the difference with a linguisticexample, a taxonomywouldshow,for instance,
a hierarchy of words,a 'ffowchart',for instance, a way of generating a clauseby followinga
precisesequence of instructions', anda networkmightshowlhe collocation of words- the
otherwordswith which any givenword typicallycombines/ regardless of the structural
relationsbetween the words(seefigure3.6).
Figure3.7 showsa'linear' (ffowchart)and'non-linear'(network)representation from
an articleon a'writer's assistant'computer program.Thenetwork,its authorssay/allows
t h e w r i t e r ' t o f o r m i d e a si n t o a n a s s o c i a t i vnee t w o r k ' ( S h a r p l easn d P e m b e r t o nI 9, 9 2 i
2 2 ) . T h ep r i n c i p l e b e h i n ds u c hn e t w o r kcsl e a r l yr e l a t e tso t h e i d e ao f t h e ' n o n - l i n e a r ' t e x t
t o w h i c hw e h a v ea l l u d e di n o u rd i s c u s s i oonf D i c kB r u n a ' s0 n M y W a l k( f i g u r e1 . 2 )a n dt o
whichwe will returnin greaterdetail in chapter6. In discussing a pagefrom Brunawe
stressed that suchpageson the onehandprovidethe readerwith manychoices, manypaths
to follow,but on the other hand tend to obscurethe fact that the range of choices
is ultimatelypre-designed and limited.As such,networksare, in the end,just as much
modelledon formsof socialorganization as taxonomies and ffowcharts. Thetaxonomyis
modelledon a static,hierarchical organizationin which everythinghas its pre-ordained
placein a grandschemeunifiedby a singlesourceof authority.Theflowchartis modelled
on the principleof authoritatively prescribed, structured,goal-oriented activity.The net-
work is modelledon a form of socialorganization whichis a vastlabyrinthof intersecting
Concept uaI representati ons

affection

Pleasure displeasure

physical psychological
\ (etc
pleasure pleasure

/ l
/
(etc.)
r--+---r
contentment joy delight

(D fis f,O Taxonomy,flowchart


and network

localrelationsin whicheachnodeis relatedin manydifferentwaysto othernodesin its


immediateenvironment, but in which it is difficult,if not impossible,toform a coherent
viewof the whole.Perhapsit is not accidental that this kind of networkis comingto the
fore in an age of increasingsocialfragmentationand regionalization. But regionsare
n e v e r t h e l ecsosn n e c t etdo t h e w h o l ea n dt h e n e t w o r km o d e lm a y o b s c u r teh e g l o b a l i z i n g
t e n d e n c i ews ,h i c ha r e a l s oa n d s i m u l t a n e o u sal tyw o r k i n c o n t e m p o r a rsyo c i e t yT. h i s i s
recognizedby those who pioneerthe applicationof this mode of representation, for
instancein connection with hypertexts, whichare alsonetworks:

Readers who browsehypertextnetworksbecome'lost',unawareof wherethey are


in relationto the document,and thus unableto achievea senseof text, i.e. an
86 Concept uaI representati ons

lrtii:ri !4 iJl iXf;r?;i il

@ fig f.Z Network(from Sharplesand Pemberton,1992)

intellectual feelfor the substance of the document. Thisis analogous to startingat


f r o m o n e s i m i l a ra s s o c i a t i vweo r d
a p a r t i c u l a rw o r d i n a t h e s a u r uasn d p a s s i n g
to another.Soon it is likelythat the readerwould be examininga word that is
completely differentfrom the originalword,with no notionof any meaningto the
derivation that hasjust takenplace.
(Ghaouiet al.,1992: 7I0)

In the pronouncements of networkguru l(evinl(elly,as reportedby Jim McClellanin the


of the networkbecomeevenclearer:
)bserver,thepoliticalimplications

The imageof the atom . . . stoodfor powerand knowledge. Its sure,regularorbits


a n dd e f i n e d s p a c e rse p r e s e n t e d ' l a w - a b i dsionlga rs y s t e mos f e n e r g y ' u n d ecre n t r a l
-
direction.In contrastthe Net - a tangleof apparentdisorder was an icon of no
b e g i n n i n ng o, c e n t r en, o e n d ( o r a l l b e g i n n i nagl,l c e n t r ea, l l e n d ) .I t w a st h e e m b l e m
of our newunderstanding of the complexlogicof natureand computers; the banner
of systemswhich,in somesenses, organizedthemselves. The atom had beenthe
i c o no f t h e 2 6 i h c e n t u r y ( t h e A t o m A g e ) ,h e c o n c l u d e db ,u t t h e N e t w o u l db e t h e
archetype of the comingnetworkcultureof the 2lst.
(McClellan,0bserverLife Magazine, 21 September 1994,p. 62)

i n f i g u r e3 . 8 w e s u m m a r i zteh e d i s t i n c t i o nwse h a v em a d ei n t h i s s e c t i o nN. e t w o r k a s re


n o t i n c l u d e di n t h e s u m m a r ya s t h e y a r e ' a n a l y t i c a l ' r a t h etrh a n ' c l a s s i f i c a t i o n a l h' .e
T
d i f f e r e n cbee t w e e n ' s i n g l e - ' a n d ' m u l t i - l e v e l l emda' i rsk e db y t h e a b s e n coer p r e s e n coef
'lnterordinates'. 0ur discussion abovehas,we hope,madeit clearthat we seethesedistinc-
tionsas toolswith whichto describevisualstructuresratherthan that specific,concrete
visualscannecessarily alwaysbedescribed exhaustively anduniquelyin termsof anyoneof
o u rc a t e g o n e s .
Conceptua I representat i ons a7

Covert taxonomy

- Single-levelled
Overttaxonomy
-
L Ntulti-leve|ed

O fig f.g Chssificationat


imagestructures

REALIZATIONS

Coverttaxonomy A s e to f p a r t i c i p a n t(s, s u b o r d i n a t e si s, )
d i s t r i b u t esdy m m e t r i c a lal yc r o s tsh e p i c t u r e
space,at equaldistancefrom eachother,
equalin size,and orientedtowardsthe
verticaland horizontalaxesin the same
way.
Single-leveIIed overt taxonomy A participant('Superordinate,) is
c0nnected to two or moreotherparticipants
( ' S u b o r d i n a t e tsh' )r o u g ha t r e es t r u c t u r e
with two levelsonly.
M ulti- |eve| |ed overt taxonomy A p a r t i c i p a n(t' S u p e r o r d i n a t e i s, )
connected to otherparticipants througha
tree structurewith morethantwo levels.
Theparticipants whichoccupyintermediate
levelsare interordinates, whilethosewhich
o c c u p tyh e l o w e slte v e (l i f t h e
S u p e r o r d i n aitseo n t o p ) o r t h e h i g h e slte v e l
(if the Superordinate is at the bottom)are
Subordinates.

A N A L Y T I C A LP R O C E S S E S

Analytical processesrelate participantsin terms of a part-whole


structure.They
involvetwo kindsof participants:oneCarrier(thewhole) and any
numberof possessive
Attributes(the parts).We havealreadygivenan exampleof an analytical
structure,the
pictureof the Antarcticexplorer(figure2.4), which
analysedthe explorerin terms of
his'outfit'. Fashionshots,too, are analytical.Like the pictureof
the Antarcticexplorer,
they clearlydisplaythe parts of an 'outfit', and label both the
Carrier('easy-wearing,
inexpensive cottonsteamedwith the rightaccessories'- seefigure3.9) andthe possessive
Attributes('Laura Ashleytrenchcoat,Stuart Memberysweater,Benettonjodhpurs,),
albeit in a caption, rather than insidethe picturespace,and in the
ffowery languageof
fashion w r i t i n gw h i c hh a sb e e nd e s c r i b esdow e l l b y B a r t h e s( r 9 6 7 b ) .
ntations
I represe
Conceptua

O C0ttons(yogue,November1987)
fis f.S Easy-wearing

Differentas they may seemat first sight,mapshavethe samestructure:there is a


Carrier,for instance'Australia', and there are Possessive Attributes,for instance'the
statesof Australia',and bothare labelled,eitherinsidethe picture spaceor in a legendor
caption.Mapsmay providequitedistinctanalyses of what seemsto be the samecarrler'
Somemapsfocuson geographical featuressuchas waterways/ altitude,etc.,whileothers
s .n a l y s i sa l w a y si n v o l v esse l e c t i o nS.o m e
l n d p o l i t i c a bl o u n d a r i eA
c o n c e n t r a toen s o c i a a
attributesor characteristics of the Carrierare singledout as criterialin the givencontext
orlgeneralltwhileothersare ignored, treatedas non-essential and irrelevant'
Thedifferencebetweenthe mapand the fashionshot liesnot in their ideationalbut in
their interpersonal structures- for instance,in their modality(seechapter5)' Many
analyticalvisualshavelow modality,from the naturalisticpoint of view.Too much life-
, o u l dd i s t r a c ft r o m t h e i ra n a l y t i c apl u r p o s e0' n l yt h e e s s e n t i a l
l i k e n e stso, o m u c hd e t a i lw
featuresof the possessive Attributesare shown/and for this reasondrawingsof various
degrees of schematization are oftenpreferredoverphotographs or highlydetailedartwork'
The representation of depthis reducedor absent,as is the detailedrepresentation of llghi
and shadow, and of subtletonal distinctions. Colour,if it is usedat all, is restricted to a
reducedpalette,or usedconventionallyfor instance, - to distinguish participants, such as
differentsocioeconomic groups/or landforms.Backgroundis left out, or only sketched
in lighly.And the Possessive Attributesare labelled.Notethat the arrowsin figure2.4 do
not realizenarrativeprocesses but a relationof identitybetweena verbaland a visual
ConceptuaI representat i ons

realizationof the samePossessive Attribute.Yet photographs, particularlyposedphoto-


graphs,can alsobe analytical,as in the caseof fashionshots,or of advertisements which
g i v ea d e t a i l e d e p i c t i oonf t h ea d v e r t i s epdr o d u c to, r o f p r e s s ' m u g s h o t s p ' oof l i t i c i a nasn d
othernewsworthy persons. The schoolsocialstudiestextbookfrom whichwe took figure
2.4 shows- on the pageoppositeto that showingthe drawingin figure2.4 -the picture
r e p r o d u c ei nd f i g u r e3 . 1 0 .
It is an analyticalpicture;there is neithera vector (narrativeprocess)nor com-
positionalsymmetryand/oratree structure(classificational process).ltservesto identify
a Carrierand to allow viewersto scrutinizethis Carrier'sPossessive Attributes.Some
minordegreeof loweredmodalityis alsopresent: the background is plain,andthefact that
the pictureis posedaddsfurther artificiality.Yet,althoughit is analytical,its purposeis
more interactionaland emotivethan representational. The interactionalsystemof the
gazedominates: the gazeof represented participantsdirectlyaddresses the viewersand
s o e s t a b l i s h easn i m a g i n a r rye l a t i o nw i t h t h e m ,w h i l em o r es c h e m a t iacn a l y t i c apl i c t u r e s
i n v i t ei m p e r s o n adl ,e t a c h e sdc r u t i n yA. s i m i l a ra r g u m e nct a n b e m a d ea b o u ta d v e r t i s e -
mentsshowingthe advertisedproduct- the overall impressionof an abundanceof
p a r t s( o r i n g r e d i e n tosr, v a r i e t i e o
s f t h e p r o d u c t )o, r t h e a l l u r i n gs e n s o r q
y u a l i t yo f t h e
a d v e r t i s e pd r o d u c ta s a w h o l e( t h e s t r e a m l i n esdh e e no f t h e c a r t h e v i v i dc o l o u ra n d

@ fiq f.fO Sir DouglasMawson(0akleyetal.,1985)


9O . rePresentations
Conceptual

textureof the ingredients of a cannedsoup)take precedence over more dispassionate


scrutiny of the Possessive Attributes. Persuasionis foregrounded,instructionand
3.10
expositioa n r e b a c k g r o u n d eTdh. e t e x t b o o ki n c l u d e sb o t h f i g u r e 2 . 4 a n d f i g u r e
it seeksnot onlyto teachchildrenobjectivefactsaboutAntarcticexploration but
because
hero.Until recentlyr this then
alsoto makethememotionallyidentifywith an adventurous
decreased in the later yearsof education. Advancedtextbooksaddressed their readers
as
as'no longerneedingpictures',as havingbeenweanedoff everydaynaturalism,and
havingacquiredthe abstractand impersonal attitudethat characterized higher learning
and higherart appreciationin Westernculture.However, the tendencyis now for the
-
interpersonal to enterthe textbooksof lateryears,as well throughimageand through
of
writing.In the languageof the seniorsecondaryschooltextbook,as in the languaqe
scienceand bureaucracy,the passive voicehad beennormal,realizinga more impersonal
form of address:
anddetached

mapsandstreetdirectories.
from topographic
Theseoutlinesmaybe abstracted

textbook,asalsoin advertising
of the primary-school a
language,'you'was
In the language
keyword, andthe readerwasalwaysdirectlyaddressed:

how wouldyou
Hereis a pictureof Antarctica.If you landedherein a spaceship,
d e s c r i bteh i sP l a c e ?

Today,the personaland the informalenter increasingly many domainswhich formerly


werecharacterized by impersonal andformalmodesof address,verballyaswelI asvisually'
0f course,some photographs remainalmost as objectiveand detachedas traditional
and,for instance, of
diagramsand maps.This is still true of manyscientificphotographs
aerialphotographs: their top-downangleensuresthe absenceof depthand background
and,depending on the heightfrom whichthey are taken,detail is moreor lessremoved.
with
B u t , a s i n o u r p r i m a r y - s c h otoelx t b o o ke x a m p l et /h e yw i l l n o w o f t e nb e c o m b i n e d
less formal pictures,even in the PowerPointconference presentations of scientists
( R o w l e y - J o l i v2e0t ,0 Q .
Abstract art may also be analytical.The structureof Theo van Doesburg'sPure
Painting,for instance(figure3.11), is like the structureof a map' It analysesreality
in terms of possessive Attributes,highlyabstractones:rectangles of differentsizeand
colour.But it doesnot labeleitherthe Carrieror the Possessive Attributes.It leavesit
up to the viewerto do so,and as a resultthe paintingcan be readin manydifferentways'
We couldseeit, quite concretely, as the map of the moderncity (greenfor recreational
-
areas,yellowfor housing,red for industrialareas,and so on) figure3.12 ls in fact a
map of a city,and looksquitesimilar,if we ignorethe writing.we couldalsoseeit, more
(greenfor contemplation,
abstractly, as a map of desirablehumanqualitiesor activities
point is,
red for passion,etc.), and of the spacethey shouldoccupyin our lives.The
the paintingrepresents boththesethings,and manymore.It is opento many readings, and
that constitutesits powerto shapereality,a powerwhich,however, cannot be unleashed
i ons
Conceptual representat 91

I
I
t

Q fig f.ff (from Jaff6,1967)


PurePainting(Theovan Doesburg,1920)

until the boxesare givenconcretereference, so that the schemacan be turned into a


bIu e p r i n t .
Lowerednaturalisticmodality,however, is not a definingcharacteristic of analytic
. o d a l i t yf o r m s a s e p a r a t e( i n t e r a c t i o n asl )y s t e mw h i c h i s p r e s e n it n v i s u a l s
v i s u a l sM
simultaneously with the kindsof structurewe aredescribing in thischapter. At mostwe can
saythat, in specificsocialcontexts(cartography, educationat differentlevels,advertising
fashion),there is a tendencyfor certain modalitychoicesto go togetherwith certain
representational choices,certain kindsof processes. The definingcharacteristic of the
analyticalprocess is in fact a'default'one.It liesin the absence of vectorsandthe absence
of compositional symmetryand/ortree structures, and also in the absence of the features
t h a t m a r k t h e s y m b o l i cp r o c e s s ewse w i l l d e s c r i b e i n t h e n e x ts e c t i o nT. h e r ea r e m o r e
positiveand specificcharacteristics, but these pertainto specifickinds of analytical
process. As a whole,the analyticalprocessis the usual,the 'unmarked', andthereforealso
the most elementaryoption in the visualsystemof representation - a visual'this is'.
92 Concept uaI representati ons

WGww
w
wrc
ffi
ffiWffi
iWuffiffiffiW
ffiffiffiffiffi
Wwffiffiffiffiffi}rcry]I 1
13r::irf;*y* ffi*u.,."
W**.,,'",,.,",,o I'.'.**.*q4::.:'i.ni'i,"rnl.r1 m
ffi81*.o**o-*.*0"*, n.,''-,"' t:

Q district of Melbourne(Paskand Bryant,1982)


fig Z.tZ Map of the centralbusiness

Experimental studiesof the productionof drawingswould seemto bear this out: the
principaltask that drawersmustmasteris the representation of objectsin termsof their
m i n i m a ld e f i n i n gc h a r a c t e r i s t i casn, d t h e p r i n c i p a p
l u r p o s ef o r w h i c h n o n - s p e c i a l i s t s
actually use drawing in everydaylife is the productionof pragmaticallymotivated
'descriptions', sketchesof localities,clothesand hairstyles,mechanicaldevices,and so
on, as well as the productionof doodles, which often are also analytical(van Sommers,
I9B4 234ff.).

1 Unstructuredanalyticalprocesses

Someanalyticalprocesses areunstructured;that is,theyshowusthe Possessive Attributes


of the Carrier,but not the Carrieritself,theyshowusthe parts,but not the way the parts
fit togetherto makeup a whole.Thepagereproduced in figure3.).3,Iorinstance, is a kind
of unstructured map,accuratein scale,but without any visualindicationof the location
of the Possessive Attributesrelativeto eachother.We seethe parts of the garment,but
n o t t h e w a y t h e ya r e t o b e a s s e m b l e-d a l t h o u g hv e r b a l a b e l s( ' c o a tf r o n t ' , ' b i k i n ti o p
back',etc.)providean indication. Thepagedoesalsoincludea photograph of the finished
garment.This photograph,however, would have to be seenas a separateanalytical
structure, with highernaturalisticmodalitythan the patternitself.The magazineusesthe
representations. 93
Conceptual

- ' ' ' ' - ' - ' .


" '
" l ' .
- t " . " : " '::
:1 :" :.- .;. I ::: _.

Q fiq f.ff Escape(Austnlian Women'sWeekl!,December1987)

samestrategyas the socialstudiestextbool<: onepictureto enticeyou/andanotherto give


youa m0reobjectiveanalysisof the Carrier.
E s p e c i a lw l yh e nt h e C a r r i e ri s a b s t r a c t ,h e r em a yn o t b e a s i n g l ep r i n c i p l feo r t h ew a y
i n w h i c hP o s s e s s iAv tet r i b u t e s h o u l db ea s s e m b l eTdh. eC a r r i e cr a n n o bt ev i s u a l i z ei d nan
assembled state,andanyarrangement of the Possessive Attributesis thereforepossible: an
u n s t r u c t u r eadn a l y t i c apl r o c e siss l i k ea m o r eo r l e s su n o r d e r eldi s t .T h i sc a n b e s e e nf,o r
i n s t a n c ei n, a d v e r t i s e m e nwths i c hd i s p l a ya l l t h e p a r t st h a t m a k eu p t h ee n g i n e of acar,or
Conceptual representati ons

the full rangeof productsmanufactured in orderto impressthe viewer


by an advertiser,
w i t h t h e i rs h e ear b u n d a n c e .

2 Temporalanalyticalprocesses

Theexamples we havediscussed so far represent the thingstheyrepresent as objects.They


focuson spatiality.But thereis alsothe categoryof the timeline,a process whichseemsto
occupyan intermediate positionbetween the narrativeandthe analytical. Timelines involve
the temporaldimension, and this suggestsnarrative.Yet these linesare not vectorial
and,ratherthan representing historyas a gradualunfoldingof events, theyanalyseit into
successive stageswith fixedandstablecharacteristics, stageswhichcanthenbetreatedas
thoughtheywerethings.Thetypicalevolutiontimeline,showinga seriesof figuresstarting
with an ape 0n the left and endingwith homo sapienson the right,would be analytical
in this sense,'saying':'the evolutionof humankindconsistsof the "Ape" stage,the
"Apeman"stage,the "Australopithecus" stage(etc.)',with 'the evolutionof humankind'
as Carrier, andthe stagesas Possessive Attributes.
Theessential characteristic of temporalanalyticalprocesses isthat theyare realizedby
t i m e l i n e st :h e p a r t i c i p a n t (ss o m e t i m ews h o l es t r u c t u r e s' s, c e n e s 'a) r e a r r a n g e do n a n
a c t u a lo r i m a g i n a rlyi n e u, s u a l l yh o r i z o n t asl ,o m e t i m evse r t i c a lT. h et i m e l i n em a yb et o p o -
graphical, drawnto scale,or topological, assembling the participantsin the rightsequence,
b u t n o td r a w i n gt h et i m e i n t e r v a l s ' t soc a l e ' .
T i m e l i n ense e dn o tb es t r a i g h ta, n dt h e ym a yi n v o l v a e l l k i n d so f g e o m e t r i c a
s yl m b o l i s m .
F i g u r e3 . I 4 , t a k e nf r o m a S w e d i s jhu n i o rh i g h - s c h o h o il s t o r yt e x t b o o ki,s a p a r t i c u l a r l y
interesting example.Heretime doesnot progresslinearly,but takesturns,goesdownhill,
movestowardsthe viewer,and changescolou[ from a crystallineblueto a drab brown
( t h ec o l o u rc o d ee m p l o y e bd y t h e s e r i e so f f o u r b o o k so f w h i c ht h e b o o kc o n t a i n i ntgh i s
i l l u s t r a t i o ins p a r t ,h a sb l u ea s t h e c o l o u ro f h i s t o r ya n d b r o w na s t h e c o l o u ro f s o c i a l
studies).The resultis a ratherethnocentric and patriarchalstorywhichcouldbe saidto
reveal,behindthe surfaceof Sweden's modernand egalitarianwelfarestate,a deeplong-
ingfor the primitiveuncertainlife of the nomadichunter. When'M an' first appeared on the
horizonof time,sothisstorygoes,hewasa hunteranda makerof tools.Hethenacquireda
wife,but she,the womanof 'homoerectus',was not yet 'erectus'herselfand,whereher
manfixedhisgazeuponthe future,shelookedbacktowardsthe past.Then'Man'invented
fire - a maleachievement, for womanis not shownto be part of it, not evenas an admiring
onlooker.Next,he beganto coverhis bodyand begota child.But at this point,the point
wherehistorytakesa backwardturn, a curiousinversionoccurred.It was now he who
crouched and shewhowas'erectLrs', hewho lookedbackwards and shewhofixedhergaze
upona futurewhich,however, was no longerthere,as time hadalreadyturnedbackwards.
B u ta l l w a sn o t l o s t .T h eu n i o no f t h et w o p r o v i d eadv e r y ' e r e c tv' ,e r ym a l ea n dv e r yN o r d i c
homosapiensas the outcomeof evolution, as'our'ancestor,hereplacedin the centreof
the composition andcloseto the viewer. ThisstoryaboutSwedishidentityis told by visual
meansonly,and it illustrates a text whichis told,not as Swedishhistory,but as a historyof
t h e w o r l d .T h et w i s t i n ga n dt u r n i n go f t h et i m e l i n ei n f i g u r e3 . 1 4 d o e so, f c o u r s eg, i v ei t a
ConceptuaI representations

l*

.H

- *ffi
ffiffiffi -&ssa*lffiqqr
iisq!{i!r4ejd;ii*ii&etd6
* a iR
a'drd#W#q@'S
eFe4;"
.:q3?@ 4aq"e.!!@?:
Wqiftir*r4!!&ltsi!&
s- L*i*lw*:&:*-

dts 4'. 'h

!M
*{"!

dlb
bFr
@

"i
* * #4+'*F;
jk;s.4*
sS
q *aw
" #&
ffi;6htu"tutrF,

{i;!i!{saii,iid.istd;.'l
'
,r: i r' _,.. r" .:
rs4
$d{.1

. :r'

@ rig f.f+ Evolution(ohman,1989)

d y n a m i cq u a l i t ya n dr e m i n d o
s f t h e h e l i xi n f i g u r e2 . 2 4 . T h ec a t e g o r i eosf v i s u a lg r a m m a r
do not haveclear-cutedges, andspecificrepresentations canmergetwo or morestructures
- for instance, the narrativeandthe analytical.

3 Exhaustiveand inclusiveanalyticalprocesses
(Spatial)structuredanalyticalprocesses canbe exhaustive,that is,theycan exhaustively
represent the Possessive Attributesof a Carrier,so that all of the Carrieris accounted for,
all of its spacetakenup by Possessive Attributes.To put it in anotherway,in an exhaustive
analyticalstructurethere is assembly: the Possessive Attributesare joinedtogetherto
makeup a complexshape.Structuredanalyticalprocesses can also be inclusive, that is,
they can showus only someof the Possessive Attributesof a Carrier,leavingmuch of
the Carrierunaccounted for, as blank spacenot taken up by Possessive Attributes.The
c i r c l e so n t h e l e f t a n dr i g h to f S c h r a m m 'tsh i r dc o m m u n i c a t i omno d e l( f i g u r e 2 . 2 2f)o r m
e x h a u s t i vaen a l y t i c asl t r u c t u r e se,m b e d d eidn t h e l a r g e rn a r r a t i v se t r u c t u r et :h e ' s e n d e r '
andthe 'Receiver'are hereanalysed as beingmadeup out of the samethreecomponents
( ' E n c o d e r ' , ' l n t e r p r e t e r ' a' D
ned c o d e r 'a) ,s s e m b l ei nd d i f f e r e nwt a y sa, n de v e r yp a r t o f t h e
spaceof the circlesis coveredby Possessive Attributes.Thepointis not,of course, that the
a n a l y s i iss i n f a c t e x h a u s t i v e . ' S e n d e r s ' a n d ' R e c e im ve a rysw,e l l h a v eo t h e rc o m p o n e n t s
besidethesethree.The point is that in the analysisthe world is treatedas thoughit is
exhaustively represented, as thoughthe Carriershavethesemajor components and no
96 . Conceptual representations

others.Just as we haveto assumethat a map showingthe statesof Australiashowsus


a//the statesof Australia,anda pictureof the outfit of an Antarcticexplorera//the parts
of this outfit, so we haveto assumethat this diagramshowsus all the components of
t h e ' S e n d e r ' a nt dh e ' R e c e i v e r ' . c0 of u r s e , ' e n c o d i n g ' , ' i n t e r p r e t i n g ' a n d ' d e c o d i n g ' m i g h
be seenas activitiesrather than as components. However, Schrammhas represented
them hereas thoughthey were parts of a machine, as though'Senders'and 'Receivers'
havethese elements, or consistoI theseelements. As sooftenin diagrams, doingshavebeen
turnedintothings,spatialized andobjectified.
In inclusive analyticalstructures, the Possessive Attributesdo not exhaustively divide
up the spaceof the Carrier. Theyarecontainedwithinthe Carrier, andsotakeup part of its
space,but not all - otherpartsare left blank,unanalysed. Thisstructurealsoentailsthe
possibilityof partial inclusionand exclusionIn figure 3.15, A and B are Possessive
Attributesof the Carrierformedby the largestcircle,C is a Partial Possessive Attribute
(in part belongingto the Carrier,in part outsideit), and D is excluded,hencenot a
Possessive Attribute.In other words,where exhaustivestructuresare formed by the
weldingtogetherof Possessive Attributes,inclusivestructuresare formedby the full or
partial overlappingof the participants.And the structure is recursive'. a Possessive
Attributecan becomethe Carrierof other Possessive Attributes,as in the casewith B in
fi9ure3.15: it is a Possessive Attributewith respectto the largercircle,but a Carrierwith
r e s p e ct to E . W e c a ns e et h i ss t r u c t u r e a t w o r k i n R i l e ya n d R i l e y ' sc o m m u n i c a t i omno d e l
(figure2.5). It doesnot tell us that 'over-allsocialsystems'exhaustively consistof two
'larger socialstructures'and 'larger socialstructures'of two and not more than two
' p r i m a r yg r o u p s 'b, u t t h a t ' o v e r - a lsl o c i a sl y s t e m s ' c o n t a(i bn e s i d teh e m a n yo t h e rt h i n g s
t h e y m a y a l s o c o n t a i n )a t l e a s tt w o ' l a r g e r s o c i a ls t r u c t u r e sa' ,n d t h a t ' l a r g e r s o c i a l
s t r u c t u r e s ' c o n t aai tn l e a s t w o ' p r i m a r yg r o u p s 'I.n d i v i d u a sl su c ha s ' C ' a n d ' R ' a r e
partial Possessive Attributes of the CarrierlAttribute'larger social structure' and
f u l l y P o s s e s s iAvet t r i b u t e so f t h e C a r r i e r ' o v e r - asl lo c i a sl y s t e m 'T. o g i v ea g l o s so n t h i s
s t r u c t u r ea, c c o r d i n gt o R i l e ya n d R i l e y i, n d i v i d u a lcsa n n o te s c a p et h e ' o v e r - a l ls o c i a l

@ fi9 f.fS Inclusiveanalyticalstructures


representations' 97
Conceptual

system// but theycan/at leastin part,freethemselves from the constraints of'largersocial


structures'.
Manymapshavea similarstructure.A mapof a state0r nationwhichshowscitiesand
towns,for instance,is not read as meaningthat everycity and everytown has been
i n c l u d e d , o r t h a t t h e s t a t e o r n a t i o n c o n t aPi n osn se os s i v e A t t r i b u t e s o t h e r t h a n c i t i e s a n d
towns.It is readas meaning that these are some (the major) citiesandtownsof that state
or nationor,ralher,that theseare the cities and towns of interest and relevance to you,the
r e a d e ro f t h i s p a r t i c u l a r
map. I n c l u s i v s
e t r u c t u r e as r e h e r e e m b e d d ewd i t h i ne xhaustive
: e mapas a whole
s t r u c t u r e st h m a y b e p r i m a r i l y
c o n c e r n ewd i t h t h e b o u n d a r i et sh a t
exhaustively dividethe Carrier into Possessive Attributes (Europe into nations, Australia
into states,etc.),but these Possessive Attributesthemselves are Carriersin embedded
inclusive structureswith Possessive Attributes such as major cities, rivers,lakes,etC'

4 C o n j o i n e da n d c o m p o u n d e de x h a u s t i v se t r u c t u r e s

In conjoinedexhaustive structures,Possessive Attributesare eitherconnected, by a line


lackinga featureof directionality, or disengaged,by a layoutof the Possessive Attributes
whichseparates them,yet clearlyshowshow they fit together. The latter is the case,for
instance,of the 'pie chart' in figure 3.16, wherethe disengagement of the Possessive
Attributesacquiressymbolicvalue,showing,literallyand figuratively, the rifts that divide
t h e n a t i o nT. h ef o r m e ri s t h e c a s ei n t h e c o m m u n i c a t i omno d e o l n t o p o f f i 9 u r e2 . 6 , w h e r e
. u c hc o n n e c t i nlgi n e sm a y b e p u r e l ya b s t r a c t
a l l t h e p a r t i c i p a n tasr e c o n n e c t ebdy l i n e sS
as in figure2.6,or havea featureof conductivity:
connectors,

*&{x*v*
xs",4
&*hi*q*s*
*?YE

ryffi* Srs$lti*eal
*{*%
tl*ar*igncd1S%

Q fig f,fe Australiarthe segmenls(Builetin,lo January1989)


98 Conceptual representations

Conductors are connectors that alsorepresent a Possessive Attribute,a physicalentity


- for instance,wiring, a pipeline, a road,a railwaytrack- andtheymayalsobeabstract,as
i n s o m ec o m m u n i c a t i omno d e l sR. e a l i z ebdy a d o u b l el i n e c, o n d u c t o risn d i c a t a e potential
for dynamicinteractionbetweenthe Possessive Attributesthey connect.As such,they
are both participantand process,connectedelementand connector, compounded and
conjoining.
In compounded structuresthe Possessive Attributesare weldedtogether, whileat the
s a m et i m e r e t a i n i ntgh e i rd i s t i n c it d e n t i t i e sT.h i si s a s m u c ht h e c a s ei n s i m p l ep i ec h a r t s
as in a technicaldrawing,for instance, the drawingof a machinefor crushingore, in
f i g u r e3 . 1 8 .

5 Topographicaland topologicalprocesses
When analyticalstructures are topographicalthey are read as accuratelyrepresenting
the physicalspatialrelationsandthe relativelocationof the Possessive
Attributes.AII of

Q fig f.fZ Connectors


and conductors

,:#qr
; i:,:

lr:!1i:
1:i;gli

O fiS f.fg Fourth.stage


crusher(Merritt,1984)
rePresentations' 99
Conceptual

thetypesof structuredanalyticalprocesses we havediscussed sofar maybetopographical:


we read the picture of the ore crusher in figure 3.18 as accurately scalingdown the
dimensions and relativelocationof the partsof the machine, just aswe readtopographical
maps as accurately scaling down, for example, the dimensions of a lake,and its distance
from other Possessive Attributes (mountains, rivers) and from the boundariesof the
Carrier.
Whenanalyticalstructuresaretopological,they are readas accurately representing the
' l o g i c a l ' r e l a t i o nbse t w e e n p a r t i c i p a n t tsh, e w a y i n w h i c hp a r t i c i p a n tasr e c o n n e c t etdo
e a c ho t h e r( w h e t h etrh e y h a v ec o m m o nb o u n d a r i eos r, a r e p a r t i a l l yo r w h o l l yi n c l u d e d
in eachother,in whichsequence theyare connected, etc.),but not the actualphysicalsize
of the participantsor their distancefrom eachotherorl in the caseof inclusive structures,
from the boundaries of the Carrier. An electrical circuit diagram, for instance, is
'above' and 'at
topological(figure3.19). It doesnot signify,say,that lampsa and b are
battery, and it does not accurately scale down the distance between the
the right of,the
batteryand a lamp a, or betweenthe two lamps.But it doessignifythat they are con-
n e c t e di n t h i sp a r t i c u l asr e q u e n cjeu,s ta s e v o l u t i otni m e l i n essi g n i f yt h a t ' A p e ' , ' A p e m a n ' ,
'Australopithecus', etc.madetheir first appearance in historyin the ordershown'
Maps,too,may either be topographical or topological, as,for instance, in mapsof urban
transportsystems. The digital networks we discussed in the previous section are als0
topologicaldiagrams, abstract mapsr as they are based on adjacency, co-location' In other
words,abstract diagrams, too, can be either topological or topographical. Schramm's
the
c o m m u n i c a t i omno d e l( f i g u r e2 . 6 ) , f o ri n s t a n c ei s, t o p o l o g i c a l . ldt o e sn o t t e l l u st h a t
, S o u r c ei ,s o n t h e l e f t ,o r t o t h ew e s t o , f t h e ' E n c o d e ri' t; t e l l su so n l yt h a t t h e p a r t i c i p a n t s
the
a r ec o n n e c t eidn t h i ss e q u e n cTeh. i si s n o tt o s a yt h a t t h e p l a c e m e notf t h e ' S o u r c e ' t o
left of the 'Encoder' is not significant, only that its significance derives, not from the
f r o m t h e t e x t u a ls t r u c t u r e o f t h e d i a g r a m ( s e ec h a p t e r6 ) . H a l l i d ay's
i d e a t i o n a bl ,u t
of linguistic structures and their relations to other fields of
diagramof the'nature
s c h o l a r s h i p| 1' - 9 7 8 : 1 0 ) , o n t h e o t h e rh a n d ,i s a n i n t r i c a t ep i e c e o f a b s t r a c tt o p o g r a phy
( s e ef i g u r e3 . 2 0 ) . T h e d i a g r a mu s e sd i s t a n c et o i n d i c a t eh o w c l o s et h e v a r i o u sk i n d s
of language studyare from what ls hereseenas the centraland most importantform of
study,the studyof'languageas system'.It usesdistancein a figurative, yetfinely
language
' l a n g u a g e
a ss y s t e mt /h a nt h es t u d yo f
calibrated w,a y : ' p h o n e t i cfso' ,r e x a m p l ei s, c l o s etro
dialectsandregisters. And it uses relative size in the same way: the studyof language takes
up a much g r e a t e r
area t h a n a l l t h e o t h e r ' f i e l d o
s f s c h o l a r s h i p ' t o g e t hA esr i
. n m e dieval

O fig:.rf Electriccircuitdiagram(Hi11,1980)
i00 Conceptua I representati ons

logic and
mathematics

communications
engineering

graphlc
languagevarieties:
torn: grammar and
dialect

language
pathology

internalization aphasia etc

language language productionand


typology universals

I I ,;)
tt l ( psycho
t
i
linguist

t
--l---------1-' l
V ?
culture human
biology

O fiS f.ZO The placeof linguisticson the map ot knowledge(Halliday,lgTg)

m a p s /t h e c a r t o g r a p h e r / s ' h o mt e o w n ' i s b o t h e x a g g e r a t eidn s i z e a n d r e p r e s e n t e d


i n t e r n a l lm
y o r ea c c u r a t e layn dw i t h m o r ed e t a i tl h a nt h e s u r r o u n d i n q ' c o u n t r v s i d e , .

6 Dimensionaland quantitativetopography
Liketopographical visuals,chartsare drawnto scale.Thescale,however, is based,not on
the physicaldimensions of the participants,
but on the quantityor frequency of aggregates
of participantsthat aretakento be identical.pie charts,for instance(seee.g.figure3.ro,
dividea Carrier(the populationof Australia)into components, Possessive Attributesthat
are in fact aggregates of participantsanalysed as beingthe samein somerespect, and it
tellsus,notthat'Achievers'arefoundnextto, andto the southeast of,,Adapters,,but that
the numberof 'Achievers' standsto the numberof 'Adapters'as the sizeof the possessive
Attribute labelled 'Achievers' standsto the size of the Possessive Attribute labelled
'Adapters'.
Quantityis translatedinto relativesize- althoughit is,of course,alsopossible
to represent quantitywith quantity,as in figure3.21,a now perhapsratherold-fashioned
Conceptual representat i ons 101

WOMEN AT WORK
1890
ttti
l900 titti
r910
AAAAAAAA
1920
AAAAAAAA
I9 3 0
AAAAAAAAAAA
r940
AAA
AIAAATIITII
l9 5 0
AAAAAA AA
AA AAAA
AAAAA
Q figf.zf women atwork(ModleyandLowenstein,1952)

examplewhich usesOtto von Neurath's'picture language'Isotype(see e.g. Lupton,


1989).
Pie chartsand bar charts (seefigure 3.23) are one-dimensionali they showus one
(abstract)Carrier( in the caseof figure 3.16, the concreteCarrier'Australia' is a metonym
for the abstractCarrier'the populationof Australia')and its Possessive Attributes.Their
structureis quitesimilarto that of stackdiagrams(seefigure3.5). Theyare exhaustive,
c o m p o u n d eadn a l y t i c asl t r u c t u r e sT. h e d i f f e r e n c el i e s o n l y i n t h e i r p e c u l i a rk i n d o f
abstraction and peculiarkindof topographical accuracy, bothof whichare quantitative.

7 Spatio-temporalanalyticalstructures

Two-dimensional charts createa conjunctionbetweena sef of such (exhaustive, com-


pounded, quantitatively analyticalstructuresand a timeline,
abstractand topographical)
for the sal<e
of comparativeanalysisalongan orderedtimescale.In the caseof linegraphs
r02 ConceptuaI representati ons

this can result in the quasi-vectorial, quasi-narrative structuretypifiedby temperature


charts,profit charts,companygrowthcharts,etc. (seefigure3.22). In otherwords,the
s u b s t i t u t i oonf a l i n ef o r d i s c r e t e n t i t i e sc r e a t e s o m e t h i nlgi k ea d y n a m i cp r o c e s(sw i t h
m e a n i n gssu c ha s ' c h a n g e ' , ' g r o w ' , ' d e c r e aes tec' ., ) ,a n d b a c k g r o u n dosr e v e ne r a s e tsh e
a n a l y t i c aslt r u c t u r etsh a t u n d e r l i teh e g r a p hs, ot h a t t h e g r a p hn o l o n g e sr u g g e s t sh a t ' l n
1 9 B Bt h e C a r r i e r( " i n c i d e n c o ef AIDS") consisteo d f 5 0 0 , 0 0 0 P o s s e s s i vAet t r i b u t e s
("AI DS cases")and in 1990 the Carrierconsisted of 800,000 Possessive Attributesand
. . . (etc.)',but that'The Actor ("the epidemic")acts("spreadsrapidly,'),. To complicate
matters/one-dimensional structurescan be representedas though they were two-
d i m e n s i o n a ln, ds o p r o d u c ae s e n s e o f p r o g r e sosr d e c l i n ed,e p e n d i nognt h eo r d e ri n w h i c h
the Possessive Attributesare plotted alongthe horizontalaxis - an order not givenby
relationo s f c o n j o i n e d n eosrs c o m p o u n d n e sosr ,o f i n c l u s i o o n r e x c l u s i o nb,u t s o l e l yb y
quantity.Thebar chart in figure3.23 caneasilybe (mis)taken as suggesting, notthat more
peoplelive in ffatsthan in largehouses, but that people,as time progresses/ increasingly
livein ffats.
S o m e t h i nqgu i t es i m i l a rc a nh a p p e ni n l a n g u a g eH.o d g ea n d l ( r e s s( 1 9 7 8 ,1 9 9 3 )d r e w
attentionto the effectsof the transformationof nominalization, which turns clauses
(reportsof events)such as peoplelearnedinto nominals(namesof objects)such as
people'slearningwhichmay then becomeactors in newevents(the new learningspread.
Martin et al. described this processin historytextbooks. Theycommented that suchtrans-
f o r m a t i o n fsa v o u rt h e ' a n t h r o p o m o r p hm r f b i r t h ,g r o w t ha n d d e a t h ,( 1 9 8 8 :
i ce t a p h o o
1 5 7 ) a n da l l o wh i s t o r i a ntso a n a l y s e h i s t o r yi n s u c c e s s i vp e r i o d si n w h i c hs i m i l a rt h i n g s
go on. In otherwords,despitetheir apparentstatusas active,dynamicprocesses, these
structuresstill serveto establish stable,conceptual orders.

&iilstflis
th*"'1rutrl}rlln
$a rliJl# Wrrrtd
"qtxl$

l$ rig rZz c lobal epidemic(Baflet i n, 26 Junetgg})


ConceptuaI rep resentati ons 103

Leavrn9a space
34
f2 betweeneach baf
3A is optional
28
ct 26
2 t t

- 2 2
=
o
' 1 "8
u ! 4

= ^
6

2
0
Smal Semi- RoWterface Villa
houses houses detached units

Q fig\ZZ Typesof dwelling(Bindonand Williams,1988)

is possible,
but nol

6 of analyticalprocesses
risl.z+ Dynamization

This dynamizationof two-dimensional analyticalprocesses cannotoccur when the


arrangement is vertical(seefigure3.24).The Possessive Attributesthen remaindiscrete,
and the graphsas a wholesuggesta stableorderratherthan a dynamicprocess. This is
b o r n e o u t , f o r i n s t a n c e , b y t h e f a c t t h a t o n e c a n inno' t h ' f ei l sl p a c e ot hf e C a r r i e r .
It is our impression that therehasbeen,in Westerncultures,a shiftfrom a focuson the
v e r t i c atl o a f o c u so n t h e h o r i z o n t aal , c h a n g ef r o m a c o n c e r n w i t h ' W h a t i s t h e s t a t eo f
' W h e r ea r e w e ? ' t o ' W h e raer ew eg o i n g ? ' a n'dH a v ew e p r o g r e s s eodr a r ew e
a f fa i r s ? ' a n d
i n d e c l i n e ?T' h i si s b o r n eo u t a l s ob y t h e f a c t t h a t s c r i p t sw h i c hw e r et r a d i t i o n a l lwy r i t t e n
top-downare nowbeginning to bewrittenfrom left to right.
I n f i g u r e3 . 2 5 , w es u m m a r i zteh e d i s t i n c t i o ni sn t r o d u c eidn t h i s s e c t i o nN. o t et h a t t h e
superscript'm ' n dt h e s u p e r s c r i p t ' Tm' e a n s ' t h e nI'n. o t h e rw o r d s i,f t h e r ei s a
l ' e a n s ' i fa
t o p o g r a p h i ctai m , e nt h et o p o g r a p hm
l e l i n et h y u s tb eq u a n t i t a t i v e .
I04 ConceptuaI representati ons

-{
,-***r*,

Anarvticar | |
L
""tt""'-Lrnrr.u.,u."o

Q fiq f.ZS Analyticimagestructures

REALIZATIONS

Unstructured anaIyt i caI process An unordered setof participants('Possessive


Attributes')is interpreted as the setof parts
of a wholewhichitselfis not represented.
Temporalanalytical process A setof participants('Possessive Attributes,)
y n a ( h o r i z o n t aolr
i s o r d e r e dl i n e a r l o
vertical)timelineand interpreted as the setof
successive stagesof a temporallyunfolding
process.
Exhaustiveanalytical process A p a r t i c i p a n(t' C a r r i e r 'i)s d e p i c t e a
dsm a d e
up of a numberof parts('Possessive
Attributes')andthe structureis interpreted as
showina g l l t h e p a r t sf r o mw h i c ht h ew h o l ei s
m a d eu p .
Dimensional topographical TheCarrierandthe Possessive Attributesof
accuracy an analyticalprocessare drawnto scale.
Quanti tat i ve topographi caI Thesizeof the Possessive Attributesin an
accuracy analyticalprocess accurately represents the
numberor someotherquantitative attribute
of the Possessive Attributes.
Topologicalaccuracy TheCarrierandthe Possessive Attributesof
an analyticalprocess are not drawnto scale,
but the way theyare interconnected is drawn
accurately.
Abstraction Theparticipantsin an analyticalprocess may
be concrete.
I representati ons
Conceptua t05

S Y M B O L I CP R O C E S S E S

Symbolicprocesses are aboutwhata participantmeansor rs.Eithertherearetwo partici-


pants -the participantwhosemeaningor identityis established in the relation,theCarrier,
andthe participantwhichrepresents the meaningor identityitself, the SymbolicAttribute
- or there is only one participant,the Carrier,and in that casethe symbolicmeaningis
established in anotherway,to be described below.Theformertypeof process we will call
SymbolicAttributive; the latter, Symbolic Suggestive.
A r t h i s t o r i a nhsa v ec h a r t e dt h e f o r m a lp i c t o r i acl h a r a c t e r i s t iw
c sh i c hc a n r e a l i z et h e
S y m b o l i cA t t r i b u t er e l a t i o n( s e ee s p e c i a l lH
y e r m e r e n1,9 6 9 ) . S y m b o l i ca t t r i b u t e sa r e
o b j e c tw
s i t h o n eo r m o r eo f t h ef o l l o w i n g characteristics.

(1) Theyare madesalientin the representation in oneway or another;for instance, by


b e i n gp l a c e di n t h ef o r e g r o u ntdh,r o u g he x a g g e r a t e s idz et,h r o u g hb e i n ge s p e c i laylw e lI
l i t ,t h r o u g hb e i n gr e p r e s e n t ei nde s p e c i a l fl iyn ed e t a i lo r s h a r pf o c u so, r t h r o u g ht h e i r
conspicuou co s l o u ro r t o n e .
(2) Theyare pointedat by meansof a gesturewhichcannotbe interpreted as an action
otherthan the actionof 'pointingout the symbolicattributeto the viewer'- herewe
c a ni n c l u d e a l s ot h ea r r o w sw h i c hc a nc o n n e cvt i s u a rl e a l i z a t i o nosf p a r t i c i p a n twsi t h
verbalrealizations of the sameparticipant,or viceversa/as in figure2.4, for these
a l s oe s t a b l i sah r e l a t i o no f i d e n t i t yt h r o u g h' p o i n t i n g ' .
( 3 ) T h e yl o o ko u t o f p l a c ei n t h ew h o l ei,n s o m ew a y .
( 4 ) T h e ya r ec o n v e n t i o n aal lsys o c i a t ewdi t h s y m b o l ivca l u e s .

S u c hc o n v e n t i o nsayl m b o l w s e r ev e r yc o m m o ni n t h e M i d d l eA g e sa n dt h e R e n a i s s a n c e :
s e e ,f o r i n s t a n c ef i,g u r e3 . 2 7 , w h e r et h e a p p l e ,l o o k i n gs o m e w h aot u t o f p l a c ei n S t
J e r o m e ' ss t u d y ,s y m b o l i z etsh e F a l l , T e m p t a t i o n , 0 r i g i n aSl i n , a n d s o b r i n g st h e s e
i m m e d i a t etloy m i n df o r t h ev i e w e o r f t h e p a i n t i n gT. ot a k ea m o d e r ne x a m p l et h, es c i e n t i s t
d e p i c t e di n f i g u r e3 . 2 8 i s c l e a r l yn o t d o i n ga n y t h i n gw i t h , o r t o , t h e f u n g iw h i c h a r e
d i s p l a y eidn t h ef o r e g r o u nadn do f w h i c hh e i s h o l d i n go n ei n h i sr i g h th a n d H . i sp o s i t i o ni n
relationto the fungiseemscontrivedand posed.Thefungifunctionhereas the Attributes
that establish his identityas an experton fungi.
H u m a np a r t i c i p a n t isn S y m b o l i cA t t r i b u t i v ep r o c e s s euss u a l l yp o s ef o r t h e v i e w e r ,
ratherthan beingshownas involvedin someaction.This doesnot meanthat they are
necessarily portrayedfront-onand at eyelevel,or that they necessarily look at the viewer,

Symbolic
5tructures

@ figl.ZO Typesof symbolicimagestructure


106 ConceptuaI representat i ons

Q ligl.Zt St Jeromein his Stud! (Jan van Eyck,1434) (from Hermeren,1969)

eventhoughall of thesewill oftenbe the case.It meansthat theytake up a posturewhich


cannotbe interpreted as narrative:theyjust sit or standthere,for no reasonotherthanto
d i s p l a tyh e m s e l v et os t h ev i e w e r .
SymbolicSuggestive Processes haveonly one participant,the Carrier.Theycannotbe
interpreted as analytical,becausein this kind of imagedetailtendsto be de-emphasized
i n f a v o u ro f w h a tc o u l db ec a l l e d' m o o d 'o r ' a t m o s p h e r eT'h. i sc a nb e r e a l i z e idn a n u m b e r
of ways:thecoloursmay all blendtogether, into a hazyblue,for instance, or a soft golden
glow;the focusmay be soft;or the lightingmay be extreme,rendering the participantsas
o u t l i n e os r s i l h o u e t t eIst .i s t h i sw h i c hl e n d sS y m b o l i Sc u g g e s t i vpei c t u r e tsh e i rg e n e r i c i t y ,
their qualityof depictingnot a specificmomentbut a generalized essence. rhe way in
w h i c h t h eb l u r r i n go f d e t a i lo c c u r s t h e lne n d s s y m b o l i c v a l u et ht oe C a r r i e-ra s o f t g o l d e n
glow,for instance, wouldconferon the Carrierall the valuesassociated with softness, and
w i t h g o l d ,a s i n t h e B u s h e l lasd v e r t i s e m ernet p r o d u c eidn p l a t e2 . A s a r e s u l tS y m b o l i c
Suggestive processes representmeaningand identityas comingfrom within,as deriving
from qualitiesof the Carrierthemselves, whereasSymbolicAttributiveprocesses represent
meaningand identityas beingconferredto the Carrier.
E x p r e s s i o n ilsatn d s c a p e( se . 9 .N o l d e ' sT r o p i c a S l u n ,s e ep l a t e 1 ) a l s o d i m i n i s ht h e
detailof the representation, in favourof overallcoloureffectsevolvinga strongmood,and
imbuing t h e C a r r i e r( ' a u t u m ne v e n i n gw ' ) i t hs y m b o l im
c eanings.
Conceptual representations ' I07

!Ets, \.eiEre.B{i{'q"\ b**+'u;


&g .&eeg,g;,
*le}dru
{Fsx{e"';-a$
. tTi"'*li-*X
ffi
O f in l.zg Fun with fungi (s/dney M oming Henld,lS June 1992)

EMBEDDING

In language se, n t e n 6 ecsa n b e s i m p l e( c o n s i s t i nogf o n l yo n ec l a u s e / p r o c e so sr )c o m p l e x


(containing severalclauses, eachwith their ownprocess/ coordinated with or subordinated
to eachother).Pictures,too, can be simpleor complex.We havealreadydiscussed how
in figure2.1 the relationbetweenthe Aborigines and the fire constitutes a second, minor,
transactionalprocess/subordinated to the 'major' process,which is constitutedby
the relationbetweenthe British and the Aborigines; and how participantssuchas the
Britishor the landscape can themselves be readas analyticalstructures, with a Carrier
(e.g.the landscape) and Possessive Attributes(e.g.rocks and trees).which of these
s t r u c t u r easr em a J o ar n dw h i c ha r em i n o ri s ,i n v i s u a l sd,e t e r m i n ebdy t h e r e l a t i v se i z ea n d
c o n s p i c u o u s no e fstsh e e l e m e n t sT.h ec o v e ro f t h e b o o kf r o m w h i c hw e t o o k ' T h e B r i t i s h
o f t h i sk i n do f c o m p l e x i t(yf i T u r e 3 . 2 9 )W
u s e dg u n s ' p r o v i d ea sn e x a m p l e . e c a nr e c o g n l z e
in this picture:
four differentprocesses

( 1 ) A c l a s s i f i c a t i o nparlo c e sisn w h i c ht h e f i v ec h i l d r e na r e S u b o r d i n a t eo sf t h e c l a s so f
'youngAustralians"in what we havecalleda covert Taxonomy. They are shown
a g a i n sat n e u t r abl a c k g r o u nadn da r r a n g e idn a s y m m e t r i c a , a c i r c l ew h i c h
f al s h i o ni n
108 ConceptuaI representations

Q fig f.Zf Our Societ! antt Others(0akteyet at.,Lg}5)

is closed(and reiterated)by the globe.(we will discussthe meaningof circularity


m o r ef u l l y i n c h a p t e6r . )
( 2 ) A n u m b eor f a n a l y t i c apl r o c e s s ei nsw h i c he a c hc h i l di s C a r r i e irn r e l a t i o n to a number
o f ( p r o t o t y p i c a l , ' e s s e n t i aPlo' )s s e s s i A
v et t r i b u t e s( s k i nc o l o u rc, o l o u ra n d k i n d o f
hair,colourof eyes,itemsof clothing)- attributeswhich createvisualconceptsof
t h e i rd i f f e r e net t h n i c i t i e s .
3 ) A s y m b o l iact t r i b u t i v pe r o c e sisn w h i c ht h eg l o b ei s a c o n v e n t i o nsayl m b owl i t h s t r o n g
associations. It is placedin theforeground of the picture,andtwo of the childrentouch
it with a gesturewhichcannoteasilybe interpreted as an actionotherthan onethat
drawsthe viewer'sattentionto the globe.Thechildrenarethusshownto be part of the
worlda , microcosm o f t h ew o r l di n A u s t r a l i a .
G ) Transactional processes in whichthreeofthe childrenhaveputtheirarmsroundother
c h i l d r e nT.h eb o yo n t h e l e f tw o u l ds e e mt o b et h e m a j o rA c t o rh e r ew , h i l et h et w o g i r l s
ConceptuaI representati ons r09

are Goalwith respectto his action.Thetwo girls,who havetheir arms roundeach


way.Theyarewhatwe havecalled
other,on the otherhand,relatein a morereciprocal
Interactors.

F r o m t h i s m u l t i p l ea n a l y s i ws e c a n s e et h a t t h e p i c t u r e( e v e nw h e nw e d i s r e g a r tdh e
i n t e r a c t i vaen d c o m p o s i t i o n satl r u c t u r ew s h i c ha r e a l s op r e s e n ta, n d w h i c hw i l l b e d i s -
c u s s e idn t h e n e x t h r e ec h a p t e r sf )o r m sa o w e r f u m
p l , u l t i d i m e n s i o ns at rlu c t u r e .
Turningto the relationbetween these processes, we mightsaythat because of its sheer
s i z ea n dc o n s p i c u o u s ntehsesc i r c u l a ra r r a n g e m e n o tf t h e c h i l d r e ni s d o m i n a n w t , h i l et h e
analyticalprocesses are embedded in it: five childrenare co-classified, and eachof them
can be further analysedin terms of Possessive Attributes.In other words,the major
message is that thesechildrenbelongto the samecategory, despitethe difference in gender
and ethnicity;the minor messageis that they are, nevertheless, different.As far as the
t r a n s a c t i o na nl ds y m b o l ipc r o c e s s e a sr ec o n c e r n e d , w weo u l da r g u et h a t t h ew h i t el i n e so n
the boy'stracksuitform stronganddominantvectors.
P e r h a ptsh e ' t r a n s a c t i o n aplr' o c e sosf w h i c ht h i sb o yi s A c t o ra n dt h es y m b o l ipc r o c e s s
of whichhe is Carrierweighas heavilyin the scaleasthe classificational process/ andalso
form majorprocesses. Thetransactional processes of whichthe girls in the foreground are
b o t hA c t o ra n dG o a l a, n dt h e s y m b o l ipc r o c e sosf w h i c ht h e g i r l o n t h e r i g h ti s t h e C a r r i e r ,
however, are considerably lessconspicuous and might be interpreted as minor processes.
In otherwords,as Actorsof the gestureof solidarity, andas Carriersof the symbolicvalue
o f ' r e p r e s e n t i nt h
g e w o r l d ' ,t h e ' e t h n i c 'g i r l s h a v e ,i n t h i s r e p r e s e n t a t i oan m , uchless
significantroleto playthanthe whiteAustralianboyin histracksuit.

C O N C E P T U ASLT R U C T U R EISN L A N G U A G E

Thereare somepointsof contactbetweenthe way conceptual structuresare realizedin


l a n g u a gaen di n i m a g e sT.h ec o m p a r i s owno u l dh a v et o b e m a d ew i t ht h e k i n d so f l i n g u i s t i c
s a l l i d a yc a l l s ' r e l a t i o n a l ' a n d ' e x i s t e n t i a l ' p r o c e(ssseeesH a l l i d a y1, 9 8 5 :
s t r u c t u r eH
1l-2ff.).Thesehaveat leastthis in commonwith conceptual imagesthat theyrepresent the
world in termsof moreor lesspermanent statesof affairsor generaltruths,ratherthan in
termsof actionsor mentalprocesses. Hallidayrecognizestwo maincategories of relational
process, the Attributiveand the Identifyingprocess.Themeaningof an Attributiveprocess
clausecan be schematically describedas 'a is an attributeof x'. The attribute'a' is
thensimplycalledAttribute,andthe participantwhoseattributeit is, isthe Carrier-we
have borrowedtheseterms in our analysisof images.In Somepeopleare racist,for
instance, somepeopleis Carrier,are is the Relational(Attributive)process/ andracistis
the Attribute.Attributive processescan be Intensive- that is, they can be about what a
Carrier is,as in the examplejust given;they can be Circumstantial -that is,theycan be
a b o u t ' w h e r e ' o r ' w h e n ' o r ' w hw a ti t h ' a C a r r i e irs ( e . 9 .
i n T h e i rh o m ew a si n H o C h iM i n h
City, theirhomeis Carrier,razas an Attributive[Circumstantial]Relationalprocess, and in
Ho Chi Minh City an Attribute); and they can be Possessive - that is, they can be about
110 Concept uaI representati ons

whata Carrierhas(e.9.in I didn'towna thingin the world,I is Carrier,owntheAttributive


IPossessive] Relational process, anda thingin the world,the Attribute).
Themeaning of an ldentifyingRelationalprocessclause canbeschematically described
a s ' a i s t h e i d e n t i t yo f x ' . T h ei d e n t i t y ' a 'i s t h e nt e r m e dt h e V a l u ea n dt h e p a r t i c i p a n t ' x '
whoseidentity it is, is called the Token.In Rev Peter Nyanginguis the Minister of the
Uniting Church at Ernabella, Rev Peter Nyanginguis Token, rs is the Identifying
Relationalprocess/and the Minister of the Uniting Churchat Ernabellais the Value.The
Value,then,tendsto be a'status' or 'function'or 'meaning,whichservesto identifythe
Token,and the Tokenis the nameor somedescriptionof the holderof the status,or of
t h e o c c u p a not f t h e f u n c t i o no, r o f t h e s i g nw h i c hh a st h e m e a n i n gP. i c t u r ec a p t i o n a sre
oftenidentifying clauses, with a reference to the pictureas Tokenandthe meaningof the
Pictureas Value(for instance,This is oysterfarmingin Palm Islana.ldentifyingand
Attributiveclauses can be distinguished from eachotherby meansof the reversibility test:
in Identifyingclausesthe orderof the participantscan be reversedG.g. TheMinisterof
the UnitingChurchat Ernabellais RevPeter Nyanginguis just as acceptableas RevPeter
Nyanginguis the Minister of the Uniting Churchat Ernabelld, whereasin Attributive
clausesthis is not the case(Racistare somepeopleis much more unusualthan Some
peopleare racis).
E x i s t e n t i acll a u s e sf i, n a l l y s, i m p l ys t a t et h a t ' s o m e t h i n ge x i s t s ,T. h e yh a v eo n l y o n e
participant,the Existent- that is,the participantwhoseexistence the structureaffirms.
Existentsmayeitherbe EventsG.g.in It's terribleweatheAor EntitiesG.g.in Therewas
a public librarfl. The presenceof a 'dummy subject'(there or referentless i/) is the
p r i n c i p ai ld e n t i f y i ncgh a r a c t e r i s toi cf e x i s t e n t i ac l a u s e(ss e eH a l l i d a y1,9 8 5 :1 3 0 ) .
VisualClassificational and Analyticalstructurescouldthereforebe saidto be akin to,
respectively, Intensiveand Possessive Attributiveclauses, and SymbolicAttributivestruc-
turescouldbe seenas akin to Identifyingclauses, andthereis perhapsalsosomeaffinity
betweenSymbolicSuggestive structuresand Existentialclauses.But the differences are
greaterthan the similaritiesand,especially in the areaof Classificational and Analytical
images, the visualsemiotichasa rangeof structuraldeviceswhichhaveno equivalentin
language: the difference between visualconceptualization and linguisticconceptualization
is evidentlyquite large.All the more importantto havea vocabulary for expressing what
can be doneand is donewith eachin concretetextsthat combinethe two semiotics, texts
suchas 'The 0verland'(figure3.30),a schoolprojectby an eight-year-old boy.Looking
first at the words,the structureof the verbaltext on the left pageis Possessive Attribute
throughout.Thereis a Carrier('The Overland'), and thereare six Possessive Attributes
('fivefirst classcarriages','three secondclasscarriages','twodiningcar,s,,,twoengines,
and'one clubcar','onemotor rail cart'). Theprocessitselfis elided.
The picture,too, is concerned with the relationof parts to a whole.It has all the
hallmarksof an exhaustive, structuredanalyticalprocess: the pictureis front-onand eye
level,there is no Setting,and the differentPossessive Attributes(signs,windows,wheels,
s u s p e n s i oann d r a i l s )a r e s h o w nc l e a r l yb, u t w i t h o u tu n n e c e s s adr ye t a i l :i t i s r e l a t i v e l y
'abstract'.Therelationbetween pictureandtext is not oneof illustration. Theoicturedoes
not duplicatethe text, it doesnot representvisuallywhat has alreadybeenrepresented
ConceptuaI representati ons 111

Q rig s.lo'The overtand'

y .o r i s t h e r ea r e l a t i o no f ' a n c h o r a g e( 'B a r t h e s1, 9 7 7 ) i n w h i c ht h e t e x t


l i n g u i s t i c a l lN
elaborates the informationgivenin the picturewithout providingnew information. It is
true that both text and pictureare about part-wholerelationsbut this doesnot mean
theyduplicateeachother,because in the text the Carrieris the train as a whole,whereasin
thepicturetheCarrie sor n e o / t h e c a r r i a g oe fst h e t r a i n . T h e c h i l d h a s t a k e n a t o p i c , t h e
'0verlandExpress', andtreatedit verballyandvisuallyin sucha waythat eachpart of the
text supplements the otherpart.
Let us analysethe secondpage.Thewordsare as follows:

I t g o e sf r o mA D E L A I D
t oEM e l b o u r nrea i l A u s t r a l i a N
n a t i o n arlo o t .

Herewe havefirst of all a non-transactional action,with an Actor ('it') and an Action


( ' g o e s ' )a, s w e l l a s t w o C i r c u m s t a n c o e fs p l a c e( ' f r o m A d e l a i d ea, n d ' t o M e l b o u r n e , ) ,
Therefollowsan identification of the route,with the Tokenand the Processelided('rail
AustralianNationalroot'):the childobjectifies the action'going,,turns it into a thing,a
' r o u t e ' ,b y m e a n so f a n i d e n t i f y i n g
r e l a t i o n apl r o c e sisn w h i c ht h e ' r o o t ' i s V a l u eF
. inally
the route is describedin more detail,througha visualanalyticalprocessin which the
r e l e v a nst e c t i o no f A u s t r a l i ai s t h e c a r r i e ra, n d A d e l a i d eM, e l b o u r naen d t h e ' r o u t e ' t h e
r72 Conceptu a I rep resen tat i ons

Possessive Attributes,in a processwhich usesmany of the structuraldevicesof the


analyticalvisualto showprecisely howthe Possessive Attributesare spatially related,how
they'fit together'.
' M y A d v e n t u r e ' ( f i g u3r e. 3 1 )i s a n o t h esr c h o opl r o j e c t , w r i t t ebny a c h i l df r o mt h es a m e
schooland yearas the authorof'The 0verland'.Most of the verbalprocesses are trans-
actionaland non-transactional actionsin whichthe narratoris Actor: he walks,findsa
c a v e /f i n d ss o m en a i l s ,a n d s o o n . H e i s n o t d e s c r i b i nsgo m e t h i n b g u t t e l l i n ga s t o r y ,
narratina g p a r t i c u l aer v e n t :

N o t l o n ga g o I w e n tf o r a j o u r n e yd o w nt o t h e b e a c hI. w a l k e da l l a l o n gt h e b e a c h
until I founda cave.That cavewas big.Thentherewas somewood but I couldn,t
m a k ea r a f t w i t h o u tn a i l sJ. u s tt h e nI f o u n ds o m en a i l si n t h e c a v e I. s a i dt o m y s e l f
'Somebodymusthavebeenhere'. I got piece
So a of woodand startedmakingthe
raft. I useda pieceof woodfor a. . . .

S t i l l ,t h e r ei s a l s oa c o n c e p t u ae l e m e n it n. t h i s p a r t o f t h e s t o r yt h ew r i t e ri s p r e o c c u p i e d
with the materialsneeded for buildinga raft. Thereis a hiddenattributiveprocess, some-
t h i n gl i k e ' t h em a t e r i a lfso r a r a f t a r ew o o da n dn a i l s 'b, u t i t i s t r a n s f o r m eidn t ot h e s t o r y
o f t h ef i n d i n go f t h e s em a t e r i a l (sw h i c he m e r g em y s t e r i o u silnya b i gc a v e )A . s i n t h es t o r y

,MyAdventure'
@ rigr.3l
ConceptuaI representations 1r3

that opensthe bool<of Genesis, with conceptualizing


which is alsoa narrativeconcerned
the elementsthat makeup a whole,the world,the writer alternatesactionaland trans-
actionalprocesses('l foundsomenails')with moreconceptual processes
existential ('then
t h e r ew a ss o m ew o o d ' ) .
Theverbaltext stopsabruptly,in the middleof the Circumstance of Purpose, and it is
continued, not on the followingpage,but by the picture,whichshowswhatthesematerials
wereusedfor: the partsof a raft. It is an Unstructured Analyticalprocess, with the sail,
and the raft itself (alreadycompletewith mast and rudder),as Possessive Attributes.
Theseare,in the child'sconceptualization, the partsfrom whicha raft is made.However,
t h e r ei s a l s oa s t r o n gn a r r a t i v e l e m e nitn t h i s p i c t u r ei:t h a sa S e t t i n g( t h es h o r e l i n teh, e
cave) and it doesnot use the frontal,eye-levelperspective typical of most analytical
i m a g e s( a n da l r e a d ys u c c e s s f u lul ys e db y t h e a u t h o ro f ' T h e O v e r l a n d b ' )u t a h i g ha n d
o b l i q u ea n g l eI.n b o t ht h e p i c t u r ea n dt h e w o r d st,h ew r i t e rb l e n d tsh e c o n c e p t u a ln dt h e
narrativeor, rafher,narrativizes the conceptual. In the verbaltext he doesthis through
c o n j u n c t i o n( 'st h e n ' , ' j u st th e n ' e
, t c . )t,h r o u g ht h e u s eo f t h e p a s t e n s ea n dt h r o u g ht h e u s e
of transactionaland non-transactional Action processes with the narratoras a (first-
person)Actor.In the picturehe doesit throughthe use of a Settingand a narrative
p e r s p e c t i vFei.n a l l yt ,h e w o r d sa n dt h e p i c t u r ea g a i nc o m p l e m e neta c ho t h e rT. h ep i c t u r e
doesnot il lustratethe storybut continues it.
It is clear,we hope,that childrenactivelyexperimentwith the representational
resources of word and image,and with the ways in which they can be combined. Their
d r a w i n gas r e n o tj u s t i l l u s t r a t i o nosf a v e r b a tl e x t ,n o tj u s t ' c r e a t i v e m b e l l i s h m e nt h
t 'e
;y
a r e p a r t o f a ' m u l t i m o d a l l y ' c o n c e i vt e dx t ,a s e m i o t i ci n t e r p l a iyn w h i c he a c hm o d et,h e
verbalandthe visual,is givena definedandequalroleto play.
4 R e p r e s e n t a t i oa nn d i n t e r a c t i o n :
d e s i g n i n tgh e p o s i t i o no f t h e v i e w e r

In the previous chapterwe discussed visualresources for the representation of interactions


a n d c o n c e p t u ar e s e t w e e tnh e p e o p l ep, l a c e sa n d t h i n g sd e p i c t e di n i m a g e sB. u t
l l a t i o nb
v i s u acl o m m u n i c a t i oanl s oh a sr e s o u r c ef os r c o n s t i t u t i nagn dm a i n t a i n i nagn o t h e kr i n do f
interaction, the interaction between the producerandthe viewerof the image.Anotherway
o f s a y i n gt h i s i s t h a t i m a g e s( a n do t h e rk i n d so f v i s u a l )i n v o l v e
t w o k i n d so f p a r t i c i p a n t s ,
represented participants(the people,theplacesandthingsdepictedin images)andinter-
active participanfs(the peoplewho communicatewith each other through images,the
producersand viewersof images),and three kinds of relations:(1) relationsbetween
represented participants;(2) relationsbetweeninteractiveand represented participants
(the interactiveparticipants'attitudestowardsthe represented participants);and (3)
relationsbetweeninteractive participants(thethingsinteractive participantsdo to or for
eachotherthroughimages).
Interactiveparticipantsare thereforereal peoplewho produceand make senseof
imagesin thecontextof socialinstitutions which,to differentdegrees andin differentways,
r e g u l a t ew h a t m a y b e ' s a i d ' w i t h i m a g e sh, o w i t s h o u l db e s a i d ,a n d h o w i t s h o u l db e
interpreted. In somecasesthe interactionis direct and immediate.Producerand viewer
know eachother and are involvedin face-to-faceinteraction, as whenwe make photo-
graphsof eachotherto keepin walletsor pin on pinboards, or draw mapsto giveeach
otherdirections, or diagramsto explainideasto eachother.But in manycasesthereis no
immediateand direct involvement. The produceris absentfor the viewer.and the viewer
is absentfor the producer. Thinkof photographs in magazines. Who is the producer?The
photographer who took the shot?Theassistant who processed and printedit? Theagency
who selectedand distributedit? The picture editorwho choseit? The layoutartist who
c r o p p e di t a n d d e t e r m i n eidt s s i z ea n d p o s i t i o no n t h e p a g e ?M o s tv i e w e r sw i l l n o t o n l y
nevermeetall thesecontributors to the productionprocess faceto face,but alsohaveonly
ahazya , n dp e r h a pds i s t o r t e d a n dg l a m o r i z e di d, e ao f t h e p r o d u c t i opnr o c e s s ebse h i n dt h e
image.All they haveis the pictureitself,as it appearsin the magazine. And producers,
similarly, can neverreallyknowtheir vastand absentaudiences, and must,instead, create
a mentalimageof 'the' viewersand 'the' way viewersmakesenseof their pictures.In
everydayface-to-face communication it is easyenoughto distinguish interactive partici-
pantsfrom represented participants: thereis alwaysan image-producer anda viewer(who,
depending on the situation,mayswaproleswith the producer, addto the scribbled ffoorplan
or diagram,for instance), andthentherearethe represented participants (for instance, the
peopleon the quicksketchof the dinnertablearrangement, or the landmarks on the hand-
drawnmap),andthesemay,of course/includethe producerand/orthe viewerthemselves.
Producerand viewerare physicallypresent.The participantsthey representneednot be.
But when there is a disjunctionbetweenthe contextof productionand the contextof
reception,the produceris not physically present, andthevieweris alonewith the imageand
Rep resentati on and i nteract i on 115

cannotreciprocate - an illuminating exception is the caseof the 'defacement' of billboard


advertisements, when graffiti artists 'respond'to the initial 'turn' or statementof the
image.
S o m e t h i nsgi m i l a ro c c u r si n w r i t i n g .W r i t e r s t, o o ,a r e n o t u s u a l l yp h y s i c a l lpyr e s e n t
whentheir wordsare read,and must addresstheir readersin the guiseof represented
participants, evenwhenthey write in the first person.Readers, too, are alonewith the
writtenword,and cannotusuallybecomewriters in turn. Literarytheorists(e.9.Booth,
'real'
1 9 6 1 ; C h a t m a n!,9 7 8 ) h a v ea d d r e s s etdh i s p r o b l e mb y d i s t i n g u i s h i nbge t w e e n
a n d ' i m p l i e d ' a u t h oar sn,db e t w e e n ' r e a l ' a n d ' i m p l ireeda'd e r sT.h e ' i m p l i eadu t h o r i' s a
d i s e m b o d i evdo i c e o , r e v e n ' a s e t o f i m p l i c i tn o r m sr a t h e rt h a n a s p e a k eor r a v o i c e '
( R i m m o n - l ( e n a n , t 9 8 3 : 8 7 ) : ' hoer,b e t t e ri,t h a sn o v o i c en, o d i r e c tm e a n so f c o m m u n i -
cating,but instructsus silently, throughthe designof the whole,with al I the voices,by all
' i m p l i e dr e a d e r ' ,
t h e m e a n si t h a s c h o s e nt o l e t u s l e a r n '( C h a t m a nI,9 7 8 : 1 4 8 ) . T h e
' p r e f e r r erde a d i n g p o s i t i o ne' ,t c . s, i m i l a r l yi s, ' a n i m a g eo f a c e r t a i nc o m p e t e n cber o u g htto
the text and a structuringof suchcompetence within the text' (Rimmon-l(enan, I9B3'.
1 1 8 ) :t h e t e x t s e l e c t a s ' m o d e l r e a d e r ' t h r o u giht s ' c h o i c eo f a s p e c i f i cl i n g u i s t icco d e a,
c e r t a i nl i t e r a r ys t y l e ' a n db y p r e s u p p o s i n gs'pae c i f iecn c y c l o p e dciocm p e t e n c e ' ot hnep a r t
of the reader(Eco,I97g:7). Thiswe can know.0f this we haveevidence in the text itself.
Realauthorsand real readerswe cannotultimatelyknow.This bracketingout of real
authorsand real readerscarriesthe risk of forgettingthat texts,literaryand artistictexts
as muchas massmediatexts,are producedin the contextof real socialinstitutions, in
o r d e rt o p l a ya v e r yr e a l r o l e i n s o c i a l i f e- i n o r d e rt o d o c e r t a i nt h i n g st o o r f o r t h e i r
readers, and in orderto communicate attitudestowardsaspectsof sociallife andtowards
peoplewho participatein them,whetherauthorsand readersareconsciously awareof this
or not. Producers, if theywant to seetheir work disseminated, mustwork within moreor
lessrigidlydefinedconventions, and adhereto the moreor Iessrigidlydefinedvaluesand
b e l i e fo
s f t h es o c i ailn s t i t u t i ow n i t h i nw h i c ht h e i rw o r ki s p r o d u c eadn dc i r c u l a t e dR. e a d e r s
will at least recognizethesecommunicative intentionsand thesevaluesand attitudes
for what they are, evenif they do not ultimatelyacceptthem as their own valuesand
beliefs.Theycan 'recognizethe substance of what is meantwhile refusingthe speaker's
i n t e r p r e t a t i oanns da s s e s s m e n(tS s 'c a n n e l1l ,9 9 4 :1 1 ) .
Howeverimportantand real this disjunctionbetweenthe contextof productionand
the contextof reception, the two do haveelementsin common:the imageitself,and a
k n o w l e d goef t h e c o m m u n i c a t i vr ees o u r c et hs a t a l l o wi t s a r t i c u l a t i o a n n du n d e r s t a n d i n g ,
a knowledge of the waysocialinteractions andsocialrelationscanbeencoded in images. It
is oftensaidthatthe knowledge of the producerandthe knowledge of the viewerdifferin a
f u n d a m e n t ar el s p e c t h : e f o r m e ri s a c t i v ea, l l o w i n gt h e ' s e n d i n g ' aw s e l la s t h e ' r e c e i v i n g '
' m e s s a g e sP' .r o d u c e r s
o f ' m e s s a g e st h' ;e l a t t e ri s p a s s i v ea ,l l o w i n go n l yt h e ' r e c e i v i n g ' o f
areableto'write'aswellas'read',viewersareableonlyto'read'.Uptoapointthisistrue,
at leastin the sensethat the productionof imagesis still a specialized activity,so that
producers 'write' moreffuentlyand eloquently, and morefrequently, than viewers.But we
hopeour attemptsto makethat knowledge explicitwill showthat the interactive meanings
are visuallyencodedin waysthat rest on competencies sharedby producers and viewers.
116 Representation and i nteracti on

T h ea r t i c u l a t i o ann du n d e r s t a n d i n ogf sociam l e a n i n gisn i m a g e sd e r i v e fsr o m t h e v i s u a l


articulationof socialmeanings in face-to-face interaction, the spatialpositionsallocated
to differentkindsof socialactorsin interaction(whethertheyare seatedor standing, side
by side or facingeach other frontally,etc.). In this sensethe interactivedimensionof
i m a g e si s t h e ' w r i t i n g ' o fw h a t i s u s u a l l yc a l l e d' n o n - v e r b ac lo m m u n i c a t i o a
n ' l,a n g u a g e '
sharedby producers andviewersalike.
Thedisjunction betweenthe contextof productionandthe contextof receptionhasyet
anothereffect: it causessocial relationsto be representedrather than enacted.Because
the producersare absentfrom the placewherethe actualcommunicative transactionis
c o m p l e t e fdr,o mt h e l o c u so f r e c e p t i o n t h, e yc a n n o st a y ' l ' o t h e rt h a nt h r o u g ha s u b s t i t u t e
'l'. Evenwhenthe viewerreceives an imageof the 'real author' or a contributorto the
production -
process the presenter in a television programme, the painterin a self-portrait,
t h e o w n e ro f t h e c o m p a n y( o r t h e w o r k e ri n t h e c e n t u r i e s - odl di s t i l l e r yi)n a n a d v e r t i s e -
ment - that imageis only an image,a doubleof the 'real author',a representation,
d e t a c h efdr o m h i so r h e ra c t u a lb o d yA. n dt h e ' r e a la u t h o r sm ' a ya l s os p e a ki n t h eg u i s eo f
someone else,of a 'character', aswhen,insteadof the ownerof a company, it is UncleSam,
or a larger-than-life walkingandtalkingteddybear,who addresses us in an advertisement.
This dimensionof representation is anotherone which has beenstudiedextensively in
literarytheory (e.9.Genette,7972). The relationbetweenproducerand viewer,too, is
represented ratherthan enacted.In face-to-face communication we must respondto a
friendlysmilewith a friendlysmile,to an arrogantstarewith a deferentialloweringof the
e y e sa, n ds u c ho b l i g a t i o ncsa n n o et a s i l yb e a v o i d e w d i t h o u ta p p e a r i nigm p o l i t eu, n f r i e n d l y
or impudent. Whenimagesconfrontus with friendlysmilesor arrogantstares,we are not
obligedto respond, eventhoughwe do recognize howwe areaddressed. Therelationis only
represented. We are imaginarilyratherthan reallyput in the positionof the friend,the
customer, the lay personwho must deferto the expert.And whetheror not we identify
w i t h t h a t p o s i t i o nw i l l d e p e n do t h e rf a c t o r s- o n o u r r e a l r e l a t i o nt o t h e p r o d u c eor r
the institutionhe or sherepresents, and on our real relationto the otherswho form part
of the contextof reception. All the same,whetheror not we identifywith the way we are
addressed, we do understand how we are addressed, because we do understand the way
imagesrepresent socialinteractions and socialrelations. It is the business of this chapter
t o t r y a n dm a k et h o s eu n d e r s t a n d i negxsp l i c i t .

T H E I M A G EA C T A N D T H E G A Z E

In the previouschapterswe showedtwo picturesof an Antarcticexplorer, takenfrom the


Australianprimary-school socialstudiestextbook)ur Societyand )thers (}akley et al.,
1 9 8 5 ) . F i g u r e3 . 1 0 w a s a p h o t o g r a p ihn w h i c ht h e A u s t r a l i a nA n t a r c t i ce x p l o r e rS i r
D o u g l aM s a w s o nl o o k e d i r e c t l ya t t h ev i e w e rT.h es c h e m a t iacn d' g e n e r a l i z e d ' e x p l oi rne r
figure 2.4, on the other hand,did not look at the viewer.The two imagesare in fact
positioned sideby side,the photoon the left page,the drawingon the right.Together, they
combine t w o d i f f e r e nct o m m u n i c a t i fvuen c t i o n sT.h ep h o t os e e k sa b o v ea l l t o b r i n qa b o u t
Representati on and i nteract io n t17

an imaginaryrelationbetweenthe represented explorerand the childrenfor whom the


book is written,a relationperhapsof admirationfor, and identification with, a national
h e r oA . n dt h i s m e a n sa l s ot h a t t h e i m a g e - p r o d u c(et hr ei n s t i t u t i o on f e d u c a t i o n p aul blish-
i n g )a d d r e s s et hse c h i l d r e ni n t h e v o i c eo f t h e n a t i o n ahl e r oa n d m a k e st h a t n a t i o n ahl e r o
a n ' e d u c a t i o n a l ' v o i cTeh.ed r a w i n go, n t h e o t h e rh a n d s, e e k sf ,i r s t o f a l l ,t o b e r e a da s a
pieceof objective, factualinformation, and in this way aimsto set into motionthe actual
process of learning.
There is, then, a fundamentaldifferencebetweenpicturesfrom which represented
participantslook directlyat the viewer'seyes,and picturesin whichthis is not the case.
Whenrepresented participantslool<at the viewer, vectors, formedby participants'eyelines,
c o n n e ctth e p a r t i c i p a n tws i t h t h e v i e w e rC. o n t a c ti s e s t a b l i s h eedv, e ni f i t i s o n l yo n a n
imaginarylevel.In additiontheremaybe a furthervector,formedby a gesturein the same
d i r e c t i o na,s i n f i g u r e4 . 1 .
Thisvisualconfiguration hastwo relatedfunctions.In the first placeit createsa visual
form of direct address.It acl<nowledges the viewersexplicitly,addressing them with a
v i s u a'ly o u ' .I n t h es e c o n d p l a c ei t c o n s t i t u t easn ' i m a g ea c t ' .T h ep r o d u c eur s e st h e i m a g e

Q fiq e.f Recruitmentp0ster(AtfredLeete,I9l4) (ImperialWar Museum)


118 . Representation and interaction

to do something to the viewer. It is for this reasonthat we havecalledthis kind of imagea


' d e m a n d f' ,o l l o w i n gH a l l i d a y( 1 9 8 5 ) :t h e p a r t i c i p a n t 'gsa z e( a n dt h e g e s t u r ei f, p r e s e n t )
demandssomethingfrom the viewer,demandsthat the viewerenter into somekind of
imaginaryrelationwith him or her.Exactlywhat kindof relationis thensignifiedby other
means,for instanceby the facial expression of the represented participants.Theymay
smile,in whichcasethe vieweris askedto enterinto a relation of socialaffinitywith them;
theymaystareat the viewerwith colddisdain,in whichcasethe vieweris askedto relateto
them,perhaps, as an inferiorrelatesto a superior; theymay seductively poutat the viewer,
in whichcasethe vieweris askedto desirethem.Thesameappliesto gestures. A handcan
pointat the viewer,in a visual'Hey,you there,I meanyou',or invitethe viewerto come
closer, or holdthe viewerat baywith a defensive gesture, as if to say,'Stayawayfrom me'.
In eachcasethe imagewantssomething from the viewers- wantsthemto do something
(comecloser, stayat a distance)or to form a pseudo-social bondof a particularkindwith
the represented participant. And in doingthis,imagesdefineto someextentwho the viewer
is (e.9.male,inferiorto the represented participant,etc.),and in that way excludeother
viewers.
I n t h e h i s t o r yo f a r t ,t h i sl o o kw a sa s i g n i f i c a ni nt n o v a t i oAn l. t h o u g hi n I t a l i a np a i n t i n g
s m a l lf i g u r e sa m o n gt h e b y s t a n d e o r sf t h e C r u c i f i x i oann do t h e rb i b l i c asl c e n ecsa nb es e e n
to look at the viewerfrom the fourteenthcenturyonwards,the 'demand'picturecomes
i n t o i t s o w n i n t h e f i f t e e n t hc e n t u r yA. c c o r d i n tgo P a n o f s k(y1 9 5 3 : 1 9 0 ) ,i t o r i g i n a t eidn
self-portraits, and Jan van Eyckwas the first to useit in Man in a Red Turban(1433),
whichis regardedby mostart historiansas a self-portrait.

In 1.433Jan van Eyck made one of the great discoveries in portraiture.In the
portraitof a 'M an in a Redfurban', completed in October21 of that year,the glance
of the sitteris turnedout of the pictureandsharplyfocusedon the beholder with an
air of skepticismintensifiedby the expression of the thin mouth with its slightly
compressed corners.For the first time the sitter seeksto establishdirect contact
with the spectator. . . . We feelobserved andscrutinizedby a wakefulintelligence.
( P a n o f s k1
y ,9 5 3 : 7 9 8 )

0 t h e r st r a c ei t b a c kf u r t h e rA. c c o r d i ntgo B e l t i n g( 1 9 9 0 :5 7 ) , ' t h es u g g e s t i oonf r e c i p r o c i t y


between the viewerandthe persondepictedin the image'hada devotional purpose. By the
thirteenth century, m o n k s
i n t h e i rc e l l s ' h a db e f o r et h e i re y e si m a g e o
sf the V i r g i na n dh e r
crucifiedson,sothat whilereading, praying andsleeping, theycouldlookuponthemandbe
l o o k e du p o n w i t ht h ee y e so f c o m p a s s i o n ' ( o iut ra l i c s ) .
R e p r e s e n tpeadr t i c i p a n w t sh o l o o ka t t h ev i e w ear r eu s u a l l h y u m a n( o r a n i m a l )b, u t n o t
always:the headlights of a car canbe drawnas eyeslookingat the viewer, for instance, and
on the screenof oneautomaticbankteller,a creaturewhosecombinedheadand bodyhas
t h e b o x - l i k es h a p eo f a m a c h i n es,m i l e sa t t h e v i e w e rh, o l d i n go u t h i s h a n di n a n i n v i t i n g
gesture, thus'demanding' a friendlyrelationbetween the machineand its user(figure4.2).
The point is, whethertheyare humanor not,by beingrepresented as lookingat the viewer,
theyare represented as human,anthropomorphized to somedegree.
Representat i on and interact i on I19

O riq a.z ATMscreen

Otherpicturesaddressus indirectly.Herethe vieweris not object,but subjectof the


look,and the represented participantis the objectof the viewer'sdispassionate scrutiny.
N o c o n t a c it s m a d eT . h ev i e w e r ' rso l ei s t h a t o f a n i n v i s i b loen l o o k eAr .l l i m a g e w s h i c hd o
n o t c o n t a i nh u m a no r q u a s i - h u m apna r t i c i p a n tlso o k i n gd i r e c t l ya t t h e v i e w e ra r e o f t h i s
k i n d . F o r t h i s r e a s o n w e h a v e , a g a i n f o lHl oaw l l i nd ga y ( 1 9 8 5 ) , c a l l e d t h i s k i inmdaogf e a n
' o f f e r ' - i t ' o f f e r s ' t h e r e p r e s e n t epda r t i c i p a n ttso t h e v i e w e ra s i t e m so f i n f o r m a t i o n ,
objecto s f c o n t e m p l a t i oinm, p e r s o n a lal ys,t h o u g ht h e yw e r es p e c i m e ni nsa d i s p l a cy a s e .
It is alwaysinteresting to studywhichkindsof represented participantsare,in a given
context,depictedas demanding an imaginarysocialresponse of somel<indfrom the viewer,
andwhichare not.In OurSocietyand0thers(Oal<ley et al.,).985),theAustralianprimary-
schooltextbookfrom whichwe drewmanyof our keyexamplesin the first versionof this
b o o k ,i m m i g r a nfta m i l i e s m i l ea t t h e v i e w e rH. o w e v etrh, e h u m a np a r t i c i p a n tisn p i c t u r e s
f r o m t h e s ei m m i g r a n t s ' c o u n t r ioef so r i g i nd o n o t l o o ka t t h e v i e w e rn,o t e v e ni n c l o s e - u p
portraits,as,for instance, in the portraitof an Italiangrandmother who stayedbehind.In
the chapteron Aborigines, by contrast,hardlyanyof the Aboriginalparticipantslookat the
viewer. TheAboriginalpoetOodgeroo Noonuccal, referredto in the bookas'l(ath Walker',
a n dd e p i c t e d i n c l o s e - u ipn t h e l a s t i l l u s t r a t i oonf t h a t c h a p t e ri s, t h e o n l ye x c e p t i o(ns e e
f i g u r e4 . 3 b e l o w )H . e re x p r e s s i ohne, r m a k e - u ph,e rh a i r s t y l e a n dd r e s sh a r d l yd i s t i n g u i s h
herfrom non-Aboriginal womenof herage.At mosther skin is somewhatdarker,but even
t h a t i s n o t v e r y p r o n o u n c ei d n t h e b l a c k - a n d - w h ist he o t .O t h e rA b o r i g i n apl e o p l ei n t h e
chapterare much more cleariydepictedas'other', and even if they do, occasionally,
look directlyat the viewer, they do so from a longdistance, whichgreatlydiminishes the
i m p a c to f t h e i r l o o k ,o r a r e f i g u r e si n t h e b a c l < g r o u nl od o, k i n gb l a n k l ya n d m o r eo r l e s s
120 Representat i on and i nteract io n

@ fig l.f OodgerooNoonuccal(0akleyetal.,1985)

a c c i d e n t a lilny t h e d i r e c t i o n , t h i sp r i m a r y - s c h otoelx t -
o f t h e c a m e r a . A b o r i g i npael o p l ei n
book,aredepictedas objectsof contemplation, not assubjects for the pupilto enterintoan
imaginarysocialrelationwith.Immigrants, by contrast,at leastoncetheyare in Australia,
are portrayedas peoplewith whom the pupilsshouldengagemore directly,and in a
friendlyway,as equals.
The choicebetween'offer' and 'demand',which must be madewheneverpeopleare
depicted,is not only usedto suggestdifferentrelationswith different'others',to make
viewersengagewith someand remaindetachedfrom others;it can also characterize
pictorialgenres.In somecontexts- for instance, televisionnewsreading and the posed
m a g a z i npeh o t o g r a p h - t h e ' d e m a n d '
p i c t u r e
is p r e f e r r e d : of
t h e s ec o n t e x trse q u i r ea s e n s e
connection between the viewersandthe authorityfigures,celebrities and role modelsthey
depict.In othercontexts- for example, featurefilm and televisiondramaand scientific
illustration -the'offer' is preferred: herea realor imaginarybarrieris erectedbetween the
represented participantsand the viewers,a senseof disengagement, in whichthe viewer
musthavethe illusionthat the represented participants do not knowtheyare beinglooked
at, andin whichthe represented participants mustpretendthattheyare not beingwatched.
And what in onecontextis accepted convention may in anotherbe a startlingmistakeor
a n i n n o v a t i veex p e r i m e nFt .i l mt h e o r i s t (se . 9 .A l l e n ,1 9 7 7 ; W o l l e n t l g 9 2 )h a v eh a i l e dt h e
look at the cameraas a daring,Brechtian,'self-reflexive'-style figure,but in television
n e w s r e a d i tnhge l o o ka t t h ec a m e r ai s c o m m o n p l a caen d ,w ew o u l dt h i n k ,n o te x a c t l y ' s e l f -
reflexive'- at least for the oresenters: an interviewee who looks at the camerain a
televisionnewsprogrammebreaksthe rulesin an unacceptable way. Not everyonemay
Rep resent at i on and i nteract i on 121

addressthe viewerdirectly.some may only be lookedat, othersmay themselves be the


b e a r e r so f t h e l o o k .T h e r ei s a n i s s u eo f c o m m u n i c a t i vpeo w e ro r ' e n t i t l e m e n t(' S a c k s ,
1992) involvedin this,not only in pictures,but also in everyday face-to-face communica-
tion,for instancein interactions betweenmenanowomen:

As he answers the girl'slaststatementhe beginstalkingand reaches the pointwhere


normallyhe would look away,but insteadhe is still staringat her.This makesher
uncomfortable, because sheis forcedeitherto lock eyeswith him,or to look away
f r o m h i mw h i l eh e i s t a l k i n q I. f h e c o n t i n u et so t a l k a n ds t a r ew h i l es h ed e f l e c ths e r
eyes,it puts her into the 'shy'category, whichsheresents.If shebo/dlylockseyes
w i t h h i m ,h e h a sf o r c e dh e ri n t oa ' l o v e r , sg a z e , , w h i cshh ea l s or e s e n t s .
( M o r r i s1, 9 7 7 i 7 6 )

Diagrams, mapsandchartsare mostoftenfoundin contextsthatoffera kindof knowledge


which,in Westernculture,hastraditionallybeenvaluedhighly- objective, dispassionate
l<nowledge, ostensibly f reeof emotiveinvolvement andsubjectivity. Hencethe'demand'has
beenrare in thesevisualgenres.But therewerecontextsin whichthe two formsof address
were combined.Schooltextbooksof the kind we usedas data whenwe wrote the first
editionof this book,for instance,constructeda progression from 'demand,to 'offer,
pictures,and this not only in the courseof a chapter,as in the chapteron Antarctic
exploration b ,u t a l s oi n t h e c o u r s eo f a w h o l eb o o ko r s e r i e so f b o o k sa n d ,i n d e e di ,n t h e
c o u r s eo f e d u c a t i oans a w h o l e- i l l u s t r a t i o nt hs a t s e r v e dt o i n v o l v es t u d e n tesm o t i v e liyn
the subjectmatterthengraduallydroppedout as higherlevelsofeducationwerereached.
I n s e n i o hr i g h - s c h otoelx t b o o kw s ef o u n d' d e m a n d ' p i c t u r e a st m o s ti n t h ec a r t o o nw s hich,
in almostapologeticfashion,soughtto alleviatethe seriousness of the text from time to
t i m e ,a s i n a c a r t o o ni n a g e o g r a p ht ye x t b o o k( B i n d o na n d w i l l i a m s ,r g B B )w h e r ea g i r l
lookeddespondently at the viewer,with the words 'what doeshypothesis mean?,in a
dialogueballoonemanalingfrom her mouth.In the contextof education, the'demand,
p i c t u r ep l a y e da n a m b i v a l e nr ot l e . 0 nt h e o n eh a n d i,t w a sn o t a h i g h l yv a l u e df o r m ,b u t a
f o r m d e e m e ds u i t a b l eo n l y f o r b e g i n n e r a s , f o r m o n e g r e w o u t o f a s o n e c l i m b e dt h e
educational ladder;on the otherhand,it playedan indispensable role in educational strat-
egy: objectiveknowledgehad to be built, apparenily,upon a foundationof emotive
involvement, of identification with celebratorymythologies, for instance. Thisfoundation
wasthen,gradually, repressed, for if it was not repressed, the l<nowledge built on it could
n o t b e s e e nt o b e o b j e c t i v e . O u t s itdhee s p h e r eo f e d u c a t i o nt h, e v a l u eo f t h e , d e m a n d ,
picturedepended on the assumed educational levelof the reader.When,for instance, the
m a s sm e d i a( o r a u t o m a t itce l l e rm a c h i n e sb)e g a nt o u s e' d e m a n dp' i c t u r e st h, o s ee d u c a t e d
i n t h e l i n g u i s t iac n dv i s u a lg e n r e so f o b j e c t i v ke n o w l e d gaen d i m p e r s o n a ld d r e sw s ould
h a v ef e l t p a t r o n i z e d , ' a d d r e s bs e ldo wt h e i r c l a s s 'T. h o s en o t s o e d u c a t e (do r t h o s ew h o
contested the valueof suchan education) wouldhavefelt that communication hadbecome
moreeffective(andmorefun) thanwasthe casein the eraof more formalandimpersonal
p u b l i cc o m m u n i c a t i oAns. w e a l r e a d yd i s c u s s ei d n t h e p r e v i o ucsh a p t e r st h, i s s i t u a t i o ni s
n o wc h a n g i nagn d ,w i t ht h eg r a d u adl i s a p p e a r a nocfet h es e m i o t idc i s t i n c t i o tnh, ec l a s sa n d
Renresentati on and i nteract i on

age distinctions it supported(andthe differentvaluesand attitudesthat wereassociated


w i t h t h e s ed i s t i n c t i o n sa )l s ob e g a nt o e r o d e .
I t i s p o s s i b lteo r e l a t et h e m e a n i n gcso n v e y ebdy ' d e m a n d s ' a n d ' o f f e r s 'tthoe l i n g u i s t i c
s y s t e mo f p e r s o nA. s w e h a v es e e n , ' d e m a npdi'c t u r e a s d d r e stsh ev i e w e dr i r e c t l yr,e a l i z i n g
'l'. 'l'
a visual'you'.But this is not matchedby a visual The is absentin picturesQr,rather,
objectified, hidingbehinda he/she/they. The 'demand'picturethereforeremindsmore of
the languageOf,for instance, advertisements and instruCtions, Where'yoLl'S'abOundbut
'I'5' are rare,than,say,of the language 'l's'and'you's'are likely
of personalletters,where
t o b e e q u a l l yc o m m o n . ' R e apl r o d u c e r s ' c a n n roet f e rt o t h e m s e l v edsi r e c t l yT. h e ym u s t
s p e a ki m p e r s o n a l lays, t r a d i t i o n a l liyn b u r e a u c r a t iacn d s c i e n t i f i lca n g u a g ew,h e r e ' I ' s '
were,and in manycasesstill arelalsorepressed. Thepublic,on the otherhand,is addressed
'demands'derives
directly.And yet,as we haveseen/the distinctiOn between'offers'and
h i s t o r i c al yl f r o m a t t e m p t so f R e n a i s s a npc a e i n t e r tso f i n d w a y so f s a y i n g ' l ' i n t h e s e l f -
portraits which expressed their new-foundself-confidence and status of independent
artistsratherthan humblecraftsmen'
But the concepts of'offer' and 'demand'can alsobe relatedto anotherkeyconceptin
linguistics, that of the'speechact'.As mentioned, we havetakenthe termsf rom Halliday's
d e s c r i p t i oonf f o u r b a s i cs p e e c ah c t s( o r ' s p e e c hf u n c t i o n s ' ah s e c a l l st h e mi n h i s I n t r o -
ductionto FunctionalGrammar,1985).Eachof thesespeechacts,saysHalliday,is part
( a l t e r n a t i v es)o c i a l
o f a n i n t e r a c t i o n da yl a d a, n d h a si t s ' e x p e c t e d ' a nidt s ' d i s c r e t i o n a r y '
Thusspeechactscan (1) 'offer information', that is,form a statement, in which
response.
casethe response soughtis'agreement',althoughthe statementmay of coursealso be
contradicted; they can (2) 'offer goods-and-services' (e.g. Wouldyou like a drink), in
whichcasethe expected response is'acceptance', althoughthe offermayalsobe rejected;
t h e yc a n ( 3 ) ' d e m a n di n f o r m a t i o nt'h, a t i s ,f o r m a q u e s t i o ni n, w h i c hc a s et h e e x p e c t e d
response is an answer, althoughthe listenermayalsodisclaimthe question, for instanceby
saying1 don'tknowor I can'ttellyou that; andG),they can'demandgoods-and-services',
that is, constltutesomekind of command,in whichcasethe expectedresponse is for the
listenerto undertake what he or shehasbeenaskedto do,althoughlisteners mayof course
alsorefuseto do so.
'mood',that is, by syntactic
Thesespeechactsare realizedby the linguisticsystemof
permutations,permutations of the orderof the subjectandthe finiteelementof the verbal
'offer of
group(i.e.theelementof the verbalgroupthat expresses tenseandmodality).The
information'(statement),for instance, mood,in whichthefinite
is realizedbythe indicative
elementfollowsthe subject,as in this sentence from Vogue"

Women(subject)cannot(finite) live by diamondsalone.

' D e m a n c f i ni ngf o r m a t i o n ' , t h e q u e s t i o n , i s r e a l i z e d b y t h e i n t e r r o g a t i v e m o o d , w h i c h h a s t h


' p o l a rq u e s t i o n( t' h ek i n do f q u e s t i otno w h i c h
s u b j e cfto l l o w i n tgh ef i n i t ei n t h ec a s eo f t h e
'yes' 'no'), 'WH-question' a WH-sub jectfollowed
onecanjust answer or and,in the caseof a
by the finite,or the finiteelementfollowedby the subject:
Representati on and i nteract i on 123

Can(finite) women(subject)live by diamondsalone?


Who (wn-subjecil could (finite) live by diamondsalone?

What could(finile) women(subject)live by?

e o o d /t h e ' c o m m a n d ,h, a s n o s u b j e c a
T h e i m p e r a t i vm t t a l l a n d ,w h e nt h e p o l a r i t yi s
positive, no finiteeither:

Don't (finite) live by diamondsalone

Live by diamondsalone

Thespeechact of'offeringgoods-and-services', finally,is not realizedby a permutation of


subjectand finite,but by variousidioms(e.g.Hereyou ard,by questions, in conjunction
with specificmentalprocessverbsG.g. Do you wanta drink) or by commands(Havea
drinb and,indeed, in variousotherways.
Thereare many subtypesof thesefour basicl<indsof speechact. Theyare realized
t h r o u g hs p e c i f icco m b i n a t i o nosf a d d i t i o n al il n g u i s t ifce a t u r e sA.' p r e d i c t i o nf' o
, rexample,
i s a n ' o f f e ro f i n f o r m a t i o n ' w i tf hu t u r et e n s ea n de i t h e rs e c o n do- r t h i r d - p e r s osnu b j e c to,r
first-personsubjectwitlr a'non-volitional'verb(e.g. Youwill live by diamondsalone,or I
w i l l d i ey o u n g ) A . ' p r o m i s e ' i sa n ' o f f e r o f i n f o r m a t i o n ' w i tfhu t u r et e n s ea, f i r s t - p e r s o n
subjectand a volitionalverb G.g. I will buyyou diamondd.lt wouldtake us too far to
d i s c u stsh e s ei n d e t a i l T . h ep o i n tt o r e m e m b ei rs t h a t i n l a n g u a gteh e r ea r e a f e w ' c o r e '
typesanda verylargenumberof furthertypeswhichare constructed out of the coretypes.
T h es a m ei s t r u e i n t h e c a s eo f i m a g e sA. v i s u a 'l i n v i t a t i o n , i a s 'demandp , i c t u r ew i t h a
b e c k o n i nhga n da n d a s m i l i n ge x p r e s s i o an ;v i s u a l' s u m m o n sa' ,' d e m a n d ' p i c t u r ew i t h
a beckoninghand and an unsmilingexpression,'a visual 'warning',a 'demand'picture
with a raisedforefingeranda sternexpression; andso on.
Despite t h e s eb r o a ds i m i l a r i t i e si t, w o u l ds e e mt h a t ' i m a g ea c t s ,d o n o t w o r k i n t h e
sameway as speechacts.Whenimages'effer',theyprimarilyoffer information. 0f course,
an image,sayan advertisingimage,may showsomeone offeringsomething to the viewer,
andthis offer may in fact be a real offer,whichcan be obtainedby writingto an address
specifiedin the advertisement. But if there is suchan 'offer of goods-and-services' in
images,it musttake the form of an 'offer of information'.It must be represented.Itcannol
beenacteddirectly.
when images'demand',they demand,one could say,the 'goods-and-services' that
r e a l i z ea p a r t i c u l a sr o c i a lr e l a t i o n . 0 fc o u r s ea, n i m a g ec o u l ds h o wa g e s t u r eo f p u z z l e -
m e n t ,a ' s i l e n t ' q u e s t i o nb,u t t h e e x a m p l ei s s o m e w h acto n t r i v e a d n d w o u l dn e e dv e r b a l
reinforcement, or reinforcement by a conventional visualsign,for instance, a question
mark.Thereis no imageact for everyspeech act. But this neednot be so forever. Although
l a n g u a gaen d i m a g ed o h a v et h e i rs p e c i f iac f f o r d a n c e s , w hcaatnb e , s a i da, n d ' d o n ew , ith
i m a g e s( a n dw i t h l a n g u a g ed)o e sn o t o n l yd e p e n do n t h e i n t r i n s i ca n d u n i v e r s aclh a r a c -
t e r i s t i c so f t h e s em o d e so f c o m m u n i c a t i obnu, t a l s oo n h i s t o r i c a l layn dc u l t u r a l l sy p e c i f i c
rlq Representation and interaction

social needs.It is quite possibleto extendthe semanticreachand use of imagesinto


d o m a i nw s h i c hf o r m e r l yw e r et h e e x c l u s i vper o v i n c o
ef l a n g u a g e a ,s i s a l r e a d yd o n eo, n a
s h e r ep e o p l ea r eu n l i k e ltyo h a v ea n yg i v e nl a n g u a gien c o m m o n( f o r
s m a l ls c a l ei ,n p l a c e w
example, international airports).

S I Z E O F F R A M EA N D S O C I A LD I S T A N C E

Thereis a seconddimension to the interactive meanings of images,relatedto the'sizeof


frame',to the choicebetweenclose-up, mediumshot and long shot,and so on. Just as
image-producers, in depictinghumanor quasi-human participants, mustchooseto make
them look at the vieweror not,so they mustalso,and at the sametime,chooseto depict
themas closeto or far awayfrom the viewer- andthis appliesto the depictionof objects
also.And,lil<ethe choicebetweenthe 'offer' andthe 'demand',the choiceof distancecan
suggestdifferentrelationsbetweenrepresented participantsand viewers.In handbool<s
a b o u tf i l m a n dt e l e v i s i opnr o d u c t i o ns ,i z eo f f r a m ei s i n v a r i a b ldye f i n e di n r e l a t i o nt o t h e
h u m a nb o d y E . v e nt h o u g hd i s t a n c ies ,s t r i c t l ys p e a k i n g a ,c o n t i n u u mt h, e' l a n g u a goef f i l m
and television' has imposeda set of distinctcut-offpointson this continuum,in the same
way as languages imposecut-offpointson the continuumof vowelswe can produce. Thus
t h e c l o s es h o t( o r ' c l o s e - u p 's)h o w sh e a da n ds h o u l d e rosf t h e s u b j e c ta, n dt h e v e r yc l o s e
s h o t( ' e x t r e m e c l o s e - u p ' , ' b icgl o s e - u p 'a) n y t h i n gl e s st h a nt h a t .T h e m e d i u mc l o s es h o t
cuts off the subjectapproximately at the waist,the mediumshot approximately at the
k n e e sT. h e m e d i u ml o n gs h o ts h o w st h e f u l l f i g u r e .I n t h e l o n gs h o tt h e h u m a nf i g u r e
o c c u p i easb o u th a l ft h e h e i g h ot f t h ef r a m ea, n dt h ev e r yl o n gs h o ti s a n y t h i n 'gw i d e r ' t h a n
that. Stylisticvariantsare possible, but theyare alwaysseenand talkedabout intermsof
this system, as whenfilm andtelevision peopletalk of 'tight closeshots'or'tight framing',
o r a b o u t h ea m o u n o t f ' h e a d r o o mi n ' a p i c t u r e( i . e .s p a c eb e t w e etnh et o p o f t h e h e a da n d
t h e u p p e fr r a m el i n e ) .
In everydayinteraction, socialrelationsdeterminethe distance(literallyand figura-
tively)we keepfrom one another.EdwardHall (e.9.1966: 170-20) hasshownthat we
c a r r yw i t h u s a s e t o f i n v i s i b l b e o u n d a r i ebse y o n dw h i c hw e a l l o wo n l yc e r t a i nk i n d so f
peopleto come.Thelocationof theseinvisibleboundaries is determined by configurations
of sensorypotentialities - by whetheror not a certaindistanceallowsusto smellor touch
the otherperson/for instance, and by how muchof the otherpersonwe can seewith our
p e r i p h e r a( sl i x t y - d e g r evei )s i o n . ' C l o spee r s o n adli s t a n c ei s' t h ed i s t a n caet w h i c h ' o n ec a n
holdor graspthe otherperson'and thereforealsothe distancebetween peoplewho havean
intimaterelationwith eachother.Non-intimates cannotcomethiscloseand,if theydo so,it
will be experienced as an act of aggression.'Far personaldistance'is the distancethat
'extendsfrom a pointthat is just outsideeasytouching
distanceby one personto a point
wheretwo peoplecan touchfingersif they bothextendtheir arms',the distanceat which
'subjectsof personalinterestsand involvements are discussed'.'Close socialdistance'
b e g i n sj u s t o u t s i d et h i s r a n g ea n d i s t h e d i s t a n c e at which'impersonb au l s i n e sosc c u r s ' .
'Far sociad l i s t a n c e ' i s ' t h ed i s t a n c teo w h i c hp e o p l em o v ew h e ns o m e b o dsya y s" S t a n d
Representation and interaction L25

a w a ys o I c a n l o o ka t y o u " ' - ' b u s i n e s sa n ds o c i a il n t e r a c t i ocno n d u c t eadt t h i sd i s t a n c e


h a sa m o r ef o r m a la n d i m p e r s o n ac lh a r a c t etrh a n i n t h e c l o s ep h a s e ' . ' P u b l idci s t a n c e ' ,
finally,is anythingfurther than that,'the distancebetweenpeoplewho are and are to
r e m a i ns t r a n g e r sT' .h e s ej u d g e m e n tasp p l y o , f c o u r s ew, i t h i na p a r t i c u l a rc u l t u r e a, n d
H a l l c i t e sm a n ye x a m p l eos f t h e m i s u n d e r s t a n d i nwghsi c hc a n a r i s ef r o m i n t e r c u l t u r a l
differences in the interpretation of distance.
With thesedifferences correspond differentfieldsof vision.At intimatedistance, says
Hall (1964),we seethe face or headonly.At closepersonaldistancewe take in the head
andthe shoulders. At far personaldistancewe seethe otherpersonfrom the waist up.At
closesocialdistancewe seethe wholefigure.At far socialdistance we seethe wholefigure
'with spacearoundit'.
And at publicdistancewe can seethe torsoof at leastfour or five
p e o p l eI .t i s c l e a rt h a tt h e s ef i e l d so f v i s i o nc o r r e s p o ncdl o s e ltyo t h et r a d i t i o n adl e f i n i t i o n s
of sizeof frame in film and television;in otherwords,that the visualsystemof size of
frame derivesfrom the'proxemics', as Hall calls it, of everydayface-to-face interaction.
Hall is aware of this and in fact acl<nowledges the inffuenceof the work of Grosser,
a p o r t r a i tp a i n t e ro,n h i s i d e a sA. c c o r d i n tgo G r o s s e(rq u o t e di n H a l l ,I 9 6 6 i 7 I - Z ) , a t a
d i s t a n c eo f m o r et h a n 1 3 f e e t ( 4 m ) , p e o p l ea r e s e e n ' a sh a v i n gl i t t l e c o n n e c t i ow nith
o u r s e l v e sa'n, dh e n c e ' t h e p a i n t e cr a nl o o ka t h i sm o d e al s i f h ew e r ea t r e ei n a l a n d s c a p e
, nt h eo t h e rh a n d i,st h e' p o r t r a i t
o r a n a p p l ei n a s t i l l l i f e ' .F o u rt o e i g h tf e e t( r . 2 5 - 2 . 5 m ) o
distance':

t h e p a i n t e ri s n e a re n o u g hs o t h a t h i s e y e sh a v en o t r o u b l ei n u n d e r s t a n d i n t hge
sitter'ssolid forms,yet he is far enoughaway so that the foreshortening of the
f o r m sp r e s e n tnso r e a lp r o b l e mH. e r ea t t h e n o r m a d l i s t a n c oef s o c i a il n t i m a c ya n d
easyconversation, the sitter'ssoul beginsto appear.. . . Nearerthan three feet
[ 9 0 c m Jw , i t h i nt o u c h i n g
distance t h, es o u li s f a r t o o m u c hi n e v i d e n cf eo r a n ys o r to f
disinterested observation.

T h ed i s t a n c epse o p l ek e e pt,h e n ,d e p e n o - w h e t h etrh i s i s t h e m o r e


d n t h e i rs o c i a rl e l a t i o n
p e r m a n e nkt i n d o f s o c i a lr e l a t i o no n w h i c h H a l l m a i n l yc o n c e n t r a t e( st h e d i s t i n c t i o n
betweenintimates,friends,acquaintances, strangers, etc.) or the kind of socialrelation
that lastsfor the durationof a socialinteraction andis determined by the context(someone
in the audience of a speech givenby an acquaintance or relativewouldnevertheless stayat
p u b l i cd i s t a n c et h, e d i s t a n c o e f t h e ' s t r a n g e r ' )B. u t t h e s ed i s t a n c easl s o a, n da t t h e s a m e
time,determinehow muchof the otherpersonis in our field of vision- just as doesthe
f r a m i n go f a p e r s o ni n a p o r t r a i to r f i l m s h o t .
L i k et h e ' d e m a n dp' i c t u r et,h e c l o s e - ucpa m et o t h e f o r e i n t h e R e n a i s s a n cRei .n g b o m
( I 9 6 5 : 4 8 ) a r g u e st h a t i t h a si t s o r i g i ni n d e v o t i o n apli c t u r e sw, h e r ei t s e r v e dt o p r o v i d e
'the
" n e a r - n e s ss" o d e a r t o t h e G o d - s e e k i ndge v o u t ,I.n I t a l i a n a n d D u t c hp a i n t i n g s
o f t h e e a r l ys i x t e e n t che n t u r yi t a c q u i r e d a ' d r a m a t i c ' f u n c t i o na,l l o w i n g' t h e s u b t l e sot f
e m o t i o n arle l a t i o n s h i w p si t h a m i n i m u mo f d r a m a t i cs c e n e r y( ,p .4 B ) .
Thepeoplewe seein imagesarefor the mostpart strangers. It is true that we seesome
o f t h e m ( p o l i t i c i a n fsi ,l m a n dt e l e v i s i osnt a r s s, p o r t sh e r o e se,t c . )a g o o dd e a lm o r et h a n
L26 Representation and i nteract i on

others,but this kind ol lamiliaritydoesnot of itselfdetermine whethertheywill be shown


in closeshot or mediumshot or long shot.The relationbetweenthe humanparticipants
represented in imagesand the vieweris onceagain an imaginaryrelation.Peopleare
portrayedas thoughtheyare friends,or as thoughtheyare strangers. Imagesallow us to
i m a g i n a r i cl yo m ea s c l o s et o p u b l i cf i g u r e sa s i f t h e yw e r eo u r f r i e n d sa n dn e i g h b o u -r so r
t o l o o ka t p e o p l el i k eo u r s e l v eass s t r a n g e r s , ' o t h e rIsn' t. h e p r i m a r y - s c h osool c i asl t u d i e s
textbookfrom whichwe havequotedseveralexamples,three Aboriginalboysare shownin
longshot,occupying onlyabouta quarterof the heightof the 'portrait'formatframe.The
c a p t i o nr e a d s , ' T h e s ep e o p l el i v e a t R e d f e r na, s u b u r bo f S y d n e y . ' T h e ya r e s h o w n
impersonally, as strangerswith whomwe do not needto becomeacquaintances, as'trees
in a landscape'. Althoughtheydo lookat the viewer, theydo sof rom sucha distancethat it
barelyaffectsus. Indeed,they are so small that we can hardlydistinguish their facial
features.'Theirsoul doesnot yet beginto appear',to useGrosser's words.The caption,
significantg l yi ,v e st h e mn o n a m e ;i n f a c t ,w h e r et h e m o r ef r i e n d l y ' b o y s ' c o u hl da v eb e e n ,
t h eq u i t ef o r m a l' p e o p l e ' h abse e nu s e d .
The portrait of the Aboriginap l o e t O o d g e r o oN o o n u c c a(l f i g u r e 4 . 3 ) , a l r e a d y
mentioned in the previoussectionof this chapter,is a tight closeshot.Sheis depictedin a
p e r s o n awl a y .I f t h i s w a s a l l w e c o u l ds e eo f h e r i n r e a l i t yw , e w o u l db e c l o s ee n o u g ht o
touch her.As mentioned, the sectionin which the photo occursconcludes a chapteron
A b o r i g i n ei sn w h i c hn o o t h e rA b o r i g i nsem i l e sa t t h e v i e w e ri n t h i sw a y . 0 n eo f h e rp o e m s
i s q u o t e d : ' D a r ka n d w h i t eu p o nc o m m o ng r o u n d lI n c l u b a n d o f f i c ea n d s o c i a lr o u n d !
Y o u r st h e f e e lo f a f r i e n d l yl a n d l T h eg r i p o f t h e h a n d '( O a k l e ye t a l . , 1 9 8 5 : 1 6 4 ) .B u t
N o o n u c c a lm ' s e s s a gies n o t b o r n eo u t b y t h e w a y ' d a r k a n dw h i t e ' a r ep o r t r a y e di n t h e
chaoter.
Patternsof distancecan becomeconventionalin visual genres.In current affairs
television,for example,'voices'of differentstatus are habituallyframed differently:
t h e c a m e r a ' m o v eisn f o r b i g g e rc l o s e - u posf s u b j e c t w s h o a r e r e v e a l i ntgh e i r f e e l i n g s ,
whereasthe set-upfor the "expert" is usuallythe sameas that for the interviewer - the
breastpocketshot'. Both kindsof 'statused participants'tendto be 'nominated' (their
n a m e sa p p e a ro n t h e s c r e e ni n s u p e r i m p o s ec d a p t i o n s a) n d ' h a v et h e i r c o n t r i b u t i o n s
f r a m e da n d s u m m e du p ' ( B r u n s d o n
a n d M o r l e yl,9 7 B i 6 5 ) . l n o t h e rw o r d s d , i s t a n c ei s
usedto signifyrespectfor authoritiesof variouskinds,on televisionas in face-to-face
interaction.
I n d i a g r a m tsh e h u m a nf i g u r ei s a l m o s ta l w a y ss h o w ni n m e d i u ml o n go r l o n gs h o t-
objectively,'as if he werea tree in a landscape'. The picturesin figure4.4 illustrateda
front-pagenewspaper story about a murdercasein Sydney. The diagramsshowexactly
what happened, from an objectifyingand impersonaldistance(and from a high angle).
The close-upphotosaccompanytestimoniesby former patientsof the victim, but are
represented as also'friends'of we readersof the SydneyMorning Herald,andthereforeas
peoplewhoserelationwith the victim we shouldidentifywith. As in the chapteron
Antarcticexploration, the personalandthe impersonal, the emotiveandthe detached, are
combined.
So far we havediscussed socialdistancein relationto human-represented participants,
Representation and interaction . I2-l

}II5 PATIENTS

@ fig l.+ The murder of Dr Chang(SydneyMotning Heratd,sJuly799L)

b u t u n l i k et h e s y s t e mo f ' o f f e r ' a n d' d e m a n dt' h


, e s y s t e mo f s o c i adl i s t a n c cea na p p l ya l s o
to the representation of objectsand of the environment. As sizeof frame is traditionally
definedin terms of specificsectionsof the humanbody,beginning studentsof film and
television are oftenat a lossas to whichtermsto usefor describing shotsof objectsand
landscapes. Thescaleof sevensizesof frameseems too fine-grained. Thereare no clear-cut
equivalents for the shoulder, the waist,the knees.And objectscome in many different
shapesand sizes.We wouldnevertheless suggestthat at leastthreesignificantdistances
can be distinguished, andthat thereare correspondences between thesedistances and our
everydayexperience of objectsand the environment,' in otherwords,that size of frame
c a na l s os u g g e ssto c i arl e l a t i o nbse t w e etnh ev i e w e a r n do b j e c t sb,u i l d i n gasn dl a n d s c a p e s .
At closedistance, we wouldsuggest, the objectis shownas if the vieweris engaged with
it as if he or she is usingthe machine,readingthe book or the map,preparingor eating
t h e f o o d .U n l e s st h e o b j e c ti s v e r ys m a l l ,i t i s s h o w no n l y i n p a r t ,a n d o f t e nt h e p i c t u r e
includes the user'shand,or a tool - for instance, a knifescrapingthe soft margarinein an
a d v e r t i s e m e nFti.l m a n d t e l e v i s i o n ' c u t a w a y s ' ( ' o v e r s h o u l doef rosb' )j e c t si,n w h i c ht h e
o b j e c t s h o w na r e i n t e g r a t ei dn t oa n a c t i o nt h r o u g ht h ee d i t i n gu, s e t h i sd i s t a n c e . Amt i d d l e
distance, the objectis shownin full, but withoutmuchspacearoundit. It is represented as
r2a Rep resentation and i nteracti on

w i t h i nt h e v i e w e r ' sr e a c h b , u t n o t a s a c t u a l l yu s e d T . h i st y p e o f p i c t u r ei s c o m m o ni n
advertising: the advertised productis shownin full, but from a fairly closerange,and a
steepangle,as if the viewerstandsjust in front of thetableon whichit is displayed. At long
distancethereis an invisiblebarrierbetween the viewerandthe object.Theobjectis there
f o r o u r c o n t e m p l a t i oonn l y ,o u t o f r e a c h a, s i f o n d i s p l a yi n a s h o pw i n d o wo r m u s e u m
exhibit.The screenshot of the EuropeanPlayStationwebsite,in figure 4.5, usesboth
middledistanceand closedistance, significantly puttingthe closeshoton the right,as the
' N e w ' ( s e ec h a p t e6r ) .
Thesamekind of distinctions can be madewith respectto representations of buildings
and landscapes. We can seea buildingfrom the distanceof someone aboutto enterit, in
w h i c hc a s ew e w i l l n o t s e et h e w h o l eo f t h e b u i l d i n ga,s i s ( a g a i n )o f t e nt h e c a s ei n f i l m
shotsin whichthe buildingis relatedto someaction.Wecanalsoseeit from the distanceof
s o m e o nw e h o j u s t i d e n t i f i e idt a s h i so r h e rd e s t i n a t i o an n, d i s s u r v e y i nigt f o r a m o m e n t ,
b e f o r em o v i n gt o w a r d si t . I n t h a tc a s et h ef r a m ew i l l i n c l u d e o n l yt h eb u i l d i n a
g n dl e a v eo u t
the surrounding environment.0r we canseeit, soto speak,from behindthe gatesthat keep
the publicat a respectfuldistancefrom the palace/or the fortress,or the nuclearreactor,
a n d i n t h a t c a s et h e r e p r e s e n t a t iw oni l l i n c l u d ea l s ot h e s p a c ea r o u n dt h e b u i l d i n gL. a n d -
scapes/ too, can be seenfrom within;from a kind of middledistance, with a foreground
objectsuggesting, perhaps, that the vieweris imaginarilylocatedwithin the landscape,
but stoppingfor a moment,as if to take stockof what is ahead;or from a longdistance,
f r o m t h e a i r ,p e r h a p so,r f r o m a ' l o o k o u t ' p o s i t i o na,p l a c en o t i t s e l fi n t h e l a n d s c a pbeu t
affordingan overviewof it, as,for instance, in manyof the photographic illustrationsin
geography textbooks.

@ fig a.S Playstationwebsite(http://eu.playstation.c0m/europe-select.jhtml)


Representati on and interacti on r29

W e w i l l e n d w i t h s o m eb r i e f c o m m e n t so n t h e v e r y d i f f e r e n w t a y i n w h i c hs o c i a l
d i s t a n c ies r e a l i z e d i n t h e E n g l i s hl a n g u a gm e a i n l yt h r o u g hp e r m u t a t i o ni ns t h e f o r m a l i t y
of style(seeJoos,i-967).Intimatelanguage is a kindof personallanguage, spokenperhaps
o n l yb yt h e m e m b e rosf a c o u p l eo r f a m i l yo, r b y a g r o u po f s c h o oflr i e n d sT. h es p e a l < eor fs
s u c ha ' l a n g u a g eo f i n t i m a t e s ' o f t e n h a v es p e c i anl a m e sf o r e a c ho t h e r n , a m e sw h i c h
o u t s i d e rdso n o t g e tt o u s eA . n dt h e l a n g u a giet s e l fi s m i n i m al yl a r t i c u l a t e da: h a l f - w o r di s
enoughto understand eachother.Facialexpressions, eyecontact,intonation, voicequality,
etc.carrymostof the meaning, and peoplewho are in an intimaterelationwith eachother
becomefinelyattunedto the readingof meanings conveyed in thlsway.'Personal/ language
i s c a s u a lw, i t h a g o o dd e a lo f c o l l o q u i a l i samn ds l a n g N . o n - v e r b ae lx p r e s s i osnt i l l c a r r i e s
' S o c i a ll a n g u a g e ' ,
m u c ho f t h e m e a n i n gb u t n o t s o m u c ht h a t ' h a l f a w o r d i s e n o u g h ' .
t h o u g hs t i l l c o l l o q u i aal ,l r e a d yb e g i ntso i n t r o d u cae h i n to f f o r m a l i t yA. n d t h e r ei s ,i n t h i s
k i n do f s i t u a t i o nl ,e s ss h a r i n go f i n f o r m a t i oann d a s s u m p t i o nTsh. e l a n g u a gnee e d st o b e
m o r e a r t i c u l a t em , o r e v e r b a l l ye x p l i c i ts, o t h a t n o n - v e r b aelx p r e s s i oins n o l o n g e ra s
i m p o r t a nat s i n i n t i m a t ea n d p e r s o n aslt y l e .P u b l i cl a n g u a g fei ,n a l l yi,s t h e l a n g u a gues e d
i n m o r eo r l e s sf o r m a la d d r e s sH. e r el a n g u a g b e e c o m em s o n o l o g i cl i:s t e n e rns o l o n g e r
p a r t i c i p a ta e s t h e yd o i n t h e o t h e rs t y l e so f s p e e c hS. p e e c hi s n o l o n g e ri m p r o v i s e bd u, t
t h o u g h to u t i n a d v a n c ep,e r h a p es v e nf u l l y o r p a r t i a l l yw r i t t e no u t .I n t o n a t i o n a n do t h e r
formsof non-verbal expression becomeas formal,as muchsubjected to controlas syntax
a n d w o r d u s a g e .S p e e c hm u s t b e f u l l y e x p l i c i t ,m e a n i n g sf u l l y a r t i c u l a t e dv e r b a l l y .
Colloquialism a rseo u t o f p l a c ea n da m o r ef o r m a lv o c a b u l a rm y u s tb ee m p l o y e dW. r i t e r s
can of course use these styles to address us as friends or even intimates, evenwhenwe are
not, j u s t a s p i c t u r e sc a n g i v eu s c l o s e - u posf p e o p l e
w h o ,i n r e a l i t ya, r e a n dw i l l r e m a i n
strangers t o u s - t h i n k o f t h e c o l l o q u i acl h
, u m m y
i n f o r m a l i twy i t h w h i c hw e a r ea d d r e s s e d
in manyadvertisements.

A N D T H E S U B J E C T I V EI M A G E
PERSPECTIVE

Thereis yet anotherway in whichimagesbringaboutrelationsbetweenrepresented par-


ticipants a n d t h e v i e w e r p: e r s p e c t i v Pe .
r o d u c i n ag n i m a g e i n v o l v e n
s o t o n l y t h e c h oice
between 'offer' and 'demand' and the selection of a certain size of frame, but also, and at
t h es a m et i m e ,t h e s e l e c t i oon f a n a n g l e a , ' p o i n to f v i e w ' a
, n d t h i s i m p l i e t
s h e p o s s i b i l i t
oyf
expressing subjective attitudes towards represented participants, human or otherwise. By
saying'subjective attitudes',we do not meanthattheseattitudesare alwaysindividualand
unique.We will see that they are oftensociallydetermined attitudes.But theyare always
encoded a s t h o u g ht h e y w e r e s u b j e c t i v ien ,d i v i d u aa ln d u n i q u eT. h es y s t e mo f p e r s p e c t i v e
w h i c hr e a l i z e s ' a t t i t u d e ' w a s d e v e l o p e i n
d t h e R e n a i s s a n a
c ep,e r i o di n w h i c hi n d i v i d u a l i t y
b e
a n ds u b j e c t i v i t y c a m i
e m p o r t a n st o c i a v
l a l u e sa,n d i t d e v e l o p epdr e c i s e tl yo a l l o wi m a g e s
to become informed by subjective points of view. Paradoxically, while thesewere the
meanings e n c o d e dp ,e r s p e c t i vr e s t s o n a n i m p e r s o n ag l e
, o m e t r if co u n d a t i o na ,c o n s t r u c -
t i o nw h i c hi s a q u a s i - m e c h a n iwc a ly o f ' r e c o r d i n g ' i m a g e
o s
f r e a l i t yS. o c i a l l d
y etermined
and presented as 'studiesof nature',faithful
viewpoints could, in this way, be naturalized,
130 Representationandinteraction

c o p i e so f e m p i r i c arl e a l i t y . 0 n l yr e c e n t l yh a s i t b e c o m ep o s s i b l a e g a i nt o s e et h a t p e r -
s p e c t i v ei s a l s o ' a d a r i n ga b s t r a c t i o n('H a u s e r1, 9 6 2 : 6 9 ) , a n dt o d i s c u s si t s s e m i o t i c
effects,for instance, in film theory(e.g.Comolli,I97I).
Pre-Renaissance forms,frescoeson the wall of a churchnave,for example, or mosaics
in the domedroof of a church,did not haveperspective to positionthe viewer.Viewersof
suchworkswerepositioned, not by the internalstructureof the work,but by the structure
of its environments, both the immediateenvironment of the church,its proximityto the
altar,for instance, and the wider socialenvironments. In otherwords,the syntaxof the
objectdepended for its completion, its closure, not on a particularrelationwith the viewer
but on a particularrelationwith its surroundings, andthe pointof viewwasthe positionthe
v i e w ea r c t u a l l yt o o k u p i n r e l a t i o nt o t h e i m a g e : ' T hwe o r l di n t h e p i c t u r ew a se x p e r i e n c e d
a s a d i r e c tc o n t i n u a t i oonf t h e o b s e r v e r o ' sw ns p a c e(' A r n h e i m , ! 9 7 4 : 2 7 4 .A s a r e s u l t ,
the viewerhad a certainfreedomin relationto the object,a degreeof what, today,we
wouldcall 'interactiveuse'of the text,albeitin the contextof a highlyconstrained social
order.Fromthe Renaissance onwards, visualcomposition becamedominated by the system
of perspective, with its single,centralizedviewpoint.The work becamean autonomous
object,detachedfrom its surroundings, movable,producedfor an impersonalmarket,
ratherthan for specificlocations. A frame beganto separatethe represented world from
the physicalspacein whichthe imagewasviewed:at the time perspective was developed,
picturesbeganto bef ramedprecisely to createthis division, to markoff the imagefrom its
environment, and turn it into a kind of 'windowon the world,.At the sametime,images
b e c a m em o r ed e p e n d e notn t h e v i e w e rf o r t h e i r c o m p l e t i o nt h, e i r c l o s u r ea, n d v i e w e r s
becamemore distancedfrom the concretesocialorder in whichthe world had formerly
beenembedded: they now hadto learnto internalize the socialorder.Thisyieldecl greater
freedomwith respectto the immediate, concretesocialcontext,but diminished freedomin
r e l a t i o nt o t h e w o r k .A p a r a l l ecl a n b e m a d ew i t h t h e d e v e l o p m e nwt sh i c h ,m o r eo r l e s s
s i m u l t a n e o u st loyo, k p l a c ei n m u s i c( s e es h e p h e r d , r 9 7 7 ) . l nm e d i e v aml o d e sb, a s e da s
theywereon the pentatonic, any noteof the scalecouldstandin onlyintervallicrelationto
a n yo t h e rn o t e .H e n c ea n yn o t ec o u l dp r o v i d e a s e n s eo f r e s o l u t i o no,f c l o s u r eI .n t h e n e w
diatonicmusica strict hierarchywas established betweenthe fundamentals, so that any
melody, whateverthe harmonicprogressions it traversed, hadto return,ultimately, to the
samepredetermined note,the'tonic',in the keyof whichthe piecewasscripted.Thenotes
in musicthus relateto the keycentrein the samefixedway in whichviewersrelateto the
perspectival centreof the visualwork.
Thereare,then,sincethe Renaissance, two kindsof imagesin westerncultures:sub-
jectiveand objectiveimages, imageswith kentral) perspective (andhencewith a'built-in,
p o i n to f v i e w )a n d i m a g e sw i t h o u t( c e n t r a l p ) e r s p e c t i v(ea n dh e n c ew i t h o u ta ' b u i l t - i n ,
point of view).In subjectiveimagesthe viewercan seewhat there is to seeonly from a
particularpointof view.In objectiveimages, the imagerevealseverything thereis to know
(or that the imageproducedhasjudgedto be so) aboutthe represented participants, even
if,to do so,it is necessary to violatethe lawsof naturalistic depictionor indeed, the lawsof
nature.Thehistoryof art hasmanystrikingexamples of this- for example, the sculptures
o f w i n g e db u l l sa n d l i o n sw h i c hf f a n k e d t h e d o o r so f A s s y r i a tne m p l e sf :r o mt h e s i d et h e s e
Rep resentati on and i nteract io n I31

had four movinglegs,and from the front two stationarylegs,five altogether, so as to


provide,from everyside,a view from which no essentialparts were missing.Modern
technicaldrawingsmay still showwhat we knowaboutthe participantsthey represent,
what is objectively there,ratherthan what we wouldseeif we were lool<ing at them in
reality,ratherthan what is subjectively there.If we were,in reality,to seethe front of the
cubein figure4.6 the way we knowit 'objectively' is (a square), we wouldnot at the same
t i m eb ea b l et o s e et h et o p a n dt h es i d e I. t i s a n i m p o s s i bpl ei c t u r e( o r a p o s s i b lpei c t u r eo f
a highlyirregularhexahedron, ratherthan a cube)from the pointof viewof what we can
see in reality.Yet in many contexts(for instance, assemblyinstructionsfor a pieceof
f u r n i t u r e )a n ' o b j e c t i v ep' i c t u r el i k et h i s i s e n t i r e l ya c c e p t a b l e
0 .b j e c t i v ei m a g e st ,h e n ,
disregardthe viewer.Theysay,as it were,'l am this way,regardless of who or whereor
w h e ny o ua r e . '
By contrast, the pointof viewof the subjective, perspectival imagehasbeenselectedfor
the viewer.As a resultthere is a kind of symmetrybetweenthe way the image-producer
relatesto the represented participants, andthe waythe viewermust,willy-nilly, alsoreiate
t o t h e m .T h e p o i n to f v i e wi s i m p o s e d n o t o n l yo n t h e r e p r e s e n t epda r t i c i p a n t b s ,u t a l s o
on the viewer, andthe viewer's'subjectivity'is thereforesubjective in the originalsenseof
t h e w o r d ,t h e s e n s eo f ' b e i n gs u b j e c t etdo s o m e t h i nogr s o m e o n eI'n. a s h o r te s s a yo n
C h i n e saer t , B e r t o l tB r e c h h t a sc o m m e n t eodn t h i s :

As we know,the Chinese do not usethe art of perspective. Theydo not liketo see
e v e r y t h i nfgr o m a s i n g l ep o i n to f v i e w .C h i n e s ceo m p o s i t i otnh u s l a c k st h e c o m -
p u l s i o nt o w h i c h w e h a v e b e c o m ea l t o g e t h ear c c u s t o m e .d. . a n d r e j e c t st h e
subjugation of the observer.
( B r e c h t\,9 6 7 : 2 7 8 - 9 )

n a p e r i o di n w h i c h
T h es y s t e mo f p e r s p e c t i vi sef u n d a m e n t a lnl ya t u r a l i s t i cI t. d e v e l o p ei d
the worldof naturewas no longerseenas manifesting a divine order (which wasalso/and
at the sametime,a socialorder),but as an autonomous and ultimately meaningless order

Q fig l.o Frontal-isometric


cube
132 Representation and i nteracti on

w h o s el a w sa l s og o v e r n etdh e c o n d u c o t f p e o p l eI.t w a s e x p l i c i t l yg r o u n d e d in the new


scientificspirit,legitimized by the authorityof scientificobservation andthe physicallaws
of nature.Thenewmusic,similarly, wasconstructed as congruent, not with a (divineand)
s o c i aol r d e rb, u t w i t h t h e p h y s i c al la w so f s o u n d .
In the late nineteenth century,after centuriesof hegemony, both systemscame into
c r i s i si,n t h e h i g ha r t s ( C u b i s mt w , e l v e - t o nme u s i c )a sw e l la s i n t h e p o p u l a ar r t s .F i l m ,f o r
e x a m p l es,t i l lu s e sp e r s p e c t i viaml a g e sb,u t ,i n a n e a r - C u b ifsat s h i o np,r o v i d em s u l t i p l ea n d
c o n s t a n t lsyh i f t i n gv i e w p o i n tisn i t s e d i t i n gM. o d e r nt e l e v i s i o ne ,s p e c i a l li yn p r o g r a m m e s
not basedon the modelof film, suchas newsprogrammes, has gonea stepfurther,and
c h a l l e n g epse r s p e c t i va el s ow i t h i nt h e i m a g eA. n e w s r e a d m e ra yh a v eb, e h i n dh i mo r h e ro, n
t h e l e f t a v e r b atl e x t ,a n do n t h e r i g h ta c h r o m a - k e y e mdo v i n gp i c t u r eo n t h e w a l l ( a w a l l
whichis in fact a kindof two-dimensional screenon whichto projecta'layout',andin f ront
of whichto positionthe newsreader). Modernmagazine andwebsitelayoutsform another
c a t e g o ro y f v i s u awl o r k sw h i c ha r e n o l o n g e b r a s e ds o l e l yo n t h e c o m p o s i t i o nparl i n c i p l e s
of perspective. 0f course,theystill containmanyperspectival imagesbut thesehavebeen
subordinated to a structurethat can no longerbe saidto be perspectival. Two examples
m a yi l l u s t r a t teh i s .
T h ep i c t u r eo n t h e F o r dM o n d e ow e b s i t e( f i g u r e4 . 7 ) i s n a t u r a l i s t i W c . h a tw e o b s e r v e
herecouldalsobe observed in reality.Therecouldbea car positioned in thisway,in front of
t h i s p a r t i c u l acr o u p l ea n dt h i s p a r t i c u l a br u i l d i n gA. s a r e s u l to f t h e a n g l ea n dt h e s o c i a l
d i s t a n c (ea l o w - a n g l e ' l o nsgh o t 'w , i t ht h ec a r i n t h ef o r e g r o u n dv)i,e w e ras r et h e nm a d et o
r e l a t et o t h e r e p r e s e n t epda r t i c i p a n ti sn a c e r t a i nw a y .T h e ya r em a d et o ' l o o k u pt o ' t h e m ,
andtheyare madeto seethemas if theynoticethe car andthe stylishcouplefrom across
the street,with envy.In the pictureon the FordFiestapage(figure4.8),on the otherhand,
the viewer,ratherthan beingpositionedin the naturalworld,is confrontedwith a world

[]]e*i#'l*tltrN"l**:S.
lans*c{rr*{rillr}s}st p*.l""r"*"w[i6'ffi-
ffi !il

hle$*-l.xlk Msmd*rail
T}}* Mwd*a h$$ I
f&r|1, d{e.tldattre d6r{Ss
lsp.?iratin! s** t*vd;$
s? {ra*qpsr*ship sd
k{hsol*q?. e *!*
fa*il1 of reglx*r *Sei
sltrlr*dlrS
pd$oeas# &ed t!61
s{ticrs*art clf isatcl* }rs
C4t doirrlesr. Aid
v'tUblte f+-ffinFree;$4
treq*$l*ene P?*tq$io^
Sf!18a" t|:r f$d
l4eed*6 i5 oee *{ tiie
6tf<!t pl*ei t$ be.

@ fig a.Z NewJookFordMondeo(www.ford.co.uMe/mondeo)


and interaction . I33
Representation

qffi ir;r luvrr,6g


{..m*u'gr**t t,a'st
i:r'{.pt$*+itr6q}'r.} neaixr|.oc***rffi
'i...' P al,*, ..1-.e3{* ;:...+:
xali_. t'

Q fig n.A FordFiestaRockSolid(http://www.f0rd.co,uk)

which openly presentsitself as a semioticconstruct,mixing perspectivaland non-


perspectival elements in sucha wayasto givethe appearance of a continuumof formsfrom
t h e r e p r e s e n t a t i o nt a o l t h e s i g n i f i c a t i o n awl h
, i l e t h e v i s u a la s a w h o l e r e m a i n sn o n -
n a t u r a l i s t i ct h: e c a r ,o n t h i s p a g ec, a n n o tb e s a i dt o b e ' b e h i n d ' t h eL a s V e g a sr o a ds i g n
o r ' b e h i n d ' t h ew o r d' r o c ks o l i d 'i n t h e w a y t h a t t h e c o u p l ea n dt h e b u i l d i n gi n f i g u r e4 . 7
c a n b e s a i dt o b e b e h i n dt h e F o r dM o n d e o . ' l nf r o n t o f ' a n d ' b e h i n dl'o s et h e i r i d e a t i o n a l
d i m e n s i oa nn , d b e c o m tee x t u a lp r i n c i p l eosn l y T . h et w o p a g e st h u se x e m p l i fay s h i f tf r o m
the dominance of natureto the dominance of signification, andfrom the dominance of the
perceptual to the dominance of the conceptual - in a way very similarto that whichwe
observed in chapterl whenwe comparedBaby'sFirst Book(figure1.1) to Dicl<Bruna's
0n My Walk and 0n TheFarm (figureI.2).

I N V O L V E M E N TA N D T H E H O R I Z O N T A A
L NGLE

W h e nw e p r o l o n gt h e c o n v e r g i npga r a l l e lfso r m e db y t h ew a l l so f t h e h o u s e isn f i g u r e4 . 9 ,


they cometogetherin two vanishingpoints.Both pointsare locatedoutsidethe vertical
I34 . Representation and interaction

@ rig e.r Aborigines(oakleyet a1.,L985)

b o u n d a r i eosf t h e i m a g ea/ s s h o w ni n f i g u r e4 . 1 0 . T h e s ev a n i s h i n p g o i n t sa l l o w u s t o
reconstructwhat we can seeevenwithout the aid of geometricalprojection:the scene
hasbeenphotographed from an obliqueangle.The photographer has not situatedhimself
or herselfin front of the Aborigines, but hasphotographed themfrom the side.
Figure 4.10 shows how the positionfrom which the photo was taken can be
reconstructed by droppinglinesfrom the vanishing pointsin sucha way that they meetto
f o r ma 9 0 oa n g l eo n t h e l i n ed r a w nt h r o u g ht h ec l o s e sct o r n e ro f t h e c o t t a g e sF.i g u r e4 . 1 1
showsthe scenefrom above.Theline(ab) represents the frontalplaneof the subjectof the
photograph: the lineformedby the front of the cottages, which,as it happens, is alsothe
l i n ea l o n gw h i c ht h e A b o r i g i n easr e l i n e du p .T h e l i n e( c d ) r e p r e s e n t hs e f r o n t a lp l a n e
of the photographer (and henceof the viewer).Had thesetwo linesbeenparallelto one
another, the horizontalanglewouldhavebeenfrontal- in otherwords,the photographer
would havebeenpositionedin front of the Aboriginesand their cottages,facingthem.
Instead,the two linesdiverge:the angle is oblique.The photographer has not aligned
himselfor herselfwith the subject,not facedthe Aborigines, but viewedthem 'from the
s i d ei n
l es'.
Horizontalangle,then,is a functionof the relationbetweenthe frontal planeof the
image-producer andthe frontal planeof the represented participants. Thetwo can either
be parallel,alignedwith oneanother, or form an angle,divergefrom oneanother.
Representati on and i nteract i on t35

@ pointsof'Aborigines'(figure4.9)
fiq e.fO Schematicdrawing:vanishing

(zDr r^
a
*"-9oot>5 - !>46>Cn-(--e-I

@ fiq +.ff Schematicdrawing:topyiew0f'Aborigines'(figure4.9)

T h ei m a g ec a nh a v ee i t h e ra f r o n t a lo r a n o b l i q u ep o i n to f v i e w I. t s h o u l db e n o t e dt h a t
T.h e r ea r ed e g r e eosf o b l i q u e n e sasn,dw e w i l l ,i n
t h i si s n o ts t r i c t l ya n e i t h e r / odr i s t i n c t i o n
f a c t , s p e a ko f a f r o n t a la n g l es o l o n ga s t h e v a n i s h i n pg o i n t ( s )s t i l l f a l l ( s ) w i t h i nt h e
v e r t i c abl o u n d a r i eosf t h e i m a g e( t h e ym a yf a l l o u t s i d teh e h o r i z o n t abl o u n d a r i e s ) .
136 . Reoresentation and interaction

F i g u r e4 . I 2 h a s a f r o n t a la n g l e A . s s h o w ni n f i g u r e4 . I 3 , t h e r e i s o n l y o n e m a j o r
v a n i s h i npgo i n t a, n d i t l i e si n s i d et h e v e r t i c a bl o u n d a r i eosf t h e i m a g eF. i g u r e4 . 1 4 s h o w s
h o w t h ef r o n t a lp l a n eo f t h ep h o t o g r a p h(el ri n ea b )a n dt h ef r o n t a lp l a n eo f t h e r e p r e s e n t e d
participants(line cd) run parallel- that is, if one only considersone set of represented
participants, the teachers, the blackboardand the readingchart.Thefrontal planeof the
A b o r i g i n acl h i l d r e n( l i n ee f ) m a k e sa n a n g l eo f n i n e t yd e g r e ew s i t h t h e f r o n t a lp l a n eo f
the teachersand with the frontal planeof the photographer. TheAboriginalchildrenhave
b e e np h o t o g r a p h e f rdo m a v e r yo b l i q u e angle.
The differencebetweenthe obliqueand the frontal angle is the differencebetween
detachment and involvement. The horizontalangleencodes whetherthe image-producer
( a n dh e n c ew, i l l y - n i l l yt h, e v i e w e r )i s ' i n v o l v e d ' w i t h
t h e r e p r e s e n t epda r t i c i p a n tosr n o t .
Thefrontal anglesays,as it were,'What you seehereis part of our world,something we
are involved with.' Theobliqueanglesays,'Whatyouseehereis nol part of our world;it is
theirworld,somethingweare not involved with.' Theproducers of thesetwo photographs
have,perhapsunconsciously, alignedthemselves with the whiteteachersandtheir teaching
tools,but not with the Aborigines. The teachersare shownas 'part of our world',the
Aboriginesas 'other'. And as viewerswe haveno choicebut to seetheserepresented

@ fig l.fe Aboriginalchildfenat scho0l(Oakleyetal.,1985)


Representation and interaction L37

= lI
-. I I
--_\

\d ffi
@ point 0f'Aboriginalchildrenat school'(figure4.12)
fig e.ff Schematicdrawing:vanishing

- z - - - @ - - -: -
g
'--Jr
M0 l
iolloi
plEl
l:l l^ I

\ i
\ t f

&
() fiO C,f+ Schematicdrawing:topviewof'Aboriginalchildrenat school'(figure4.12)

participantsas they havebeendepicted. We are addressed as viewersfor whom'involve-


ment' takestheseparticularvalues.In reality,they might not - we might be Aboriginal
viewers,for example.It is onethingfor the viewerto be limitedby what the photograph
shows(and to understandwhat this means/for exampleexclusion,in the case of an
A b o r i g i n avli e w e r )i;t i s a n o t h e trh i n gt o a c t u a l l yi d e n t i f yw i t h t h e v i e w p o i net n c o d e d
in
the photo.We can acceptor reject,but eitherway we first needto understand what is
meant.
The primary-school socialstudiestextbook)ur Societyand Othersprovidesa further
r38 Representation and i nteracti on

i l l u s t r a t i o nA. s h o to f t h e N e wS o u t hW a l e sP a r l i a m e nHt o u s ei n S y d n e yi s f r o n t a l a , nd
t a k e nf r o ma l o wa n g l eA. s h o to f t h ec h u r c hi st a k e nf r o ma n o b l i q u e a n ds o m e w h ahti g h e r
a n g l eT. h ef o r m e ri l l u s t r a t eas s e c t i o na b o u tS y d n e yi n t h e c h a p t e r ' W h aits a C i t y ? ' t; h e
l a t t e ra, s e c t i o n a b o u ta M a o r if a m i l yi n t h ec h a p t eor n i m m i g r a n t sR.e l i g i oins d e p i c t e a ds
s o m e t h i nw g h i c h ,i n t h e c o n t e xot f p r i m a r y - s c h osool c i a sl t u d i e sd,o e sn o t b e l o n gt o ' o u r
s o c i e t y ' -t h e b o o kc o n t a i n s t a t e m e n tlsi k e , ' T h eB r i t i s hb e l i e v eidn o n e G o d '( n o t et h e
p a s t e n s e a) n dq u e s t i o nl si k e , ' D oy o ut h i n ka c h u r c ho r a c e m e t e riys l i k ea s a c r e ds i t e ? I' t
fostersa detached, outsider's attitudetowardsthe Christianreligion.
I n t h ed e p i c t i o o n f h u m a n s( a n da n i m a l s ) , ' i n v o l v e m eanntd' ' d e t a c h m e ncta' n i n t e r a c t
w i t h ' d e m a n d ' a n d ' o f f e r ' icno m p l e w x a y sT . h eb o d yo f a r e p r e s e n t epda r t i c i p a nm t a yb e
a n g l e d a w a y f r o m t h e p l a ntehoefv i e w e r , w h i l e h i s o r h e r haenaddl o r g a zm eaybeturned
towardsit (seee.g.figure4.24 below)- or vice versa.The resultis a doublemessage:
'althoughI am not part of your world,I nevertheless makecontactwith you,from my
o w n ,d i f f e r e nw t o r l d ' ;o r ' a l t h o u g ht h i s p e r s o n
i s p a r t o f o u r w o r l d ,s 0 m e o nlei l < ey o ua n d
me,we nevertheless offer his or her imageto you as an objectfor dispassionate reflec-
t i o n . ' T h el a t t e ri s t h ec a s ef,o r e x a m p l ei n, a n i l l u s t r a t i ofnr o m a D u t c hj u n i o rh i g h - s c h o o l
g e o g r a p htye x t b o o k( B o l se / a l . , l , 9 9 6 i 2 1 ) . I n a s e c t i o ne n t i t l e d' D e D e r d eW e r e l di n
onzesfraat'('TheThirdWorldin our Street'),two picturesare shownsideby side.0n the
left we seethreeolderwomen,their headscarves an emblemof their statusas immigrants.
T h e ya r e p h o t o g r a p h e f rdo m a n o b l i q u e a n g l eh, e n c ea s ' n o tp a r t o f o u r w o r l d ' a n di n l o n g
shot,henceas 'others','strangers'. 0n the right we see,left in the foreground, a blonde
g i r l ,c l e a r l ym e a n t o b e t a k e na s D u t c hw , h o h a sh i sa r m a r o u n dh e r .
, i t h a b l a c kf r i e n d w
The angleis a gooddeal morefrontalthan that of the shotof the threewomen,and the
s h o ti s a c l o s e - u ps:h ei s s h o w na s l i k e' u s ' ,D u t c hh i g h - s c h o sotl u d e n t sa,n d f r o m ' c l o s e
p e r s o n a l ' d i s t a n cBeu. t s h ed o e sn o t m a k ec o n t a c w t i t h t h e v i e w e r sS. h ed o e sn o t i n v i t e
the viewersto identifywith her,and with her relationship to a black man. Instead,the
vieweris invitedto contemplate her relationship detachedly, to ponderthe fact that some
p e o p l el i k e ' u s ' h a v er e l a t i o n s h i pwsi t h b l a c kp e o p l eb, u t n o t , i t i s i m p l i c i t l ys u g g e s t e d ,
'we'viewero s u r s e l v eS s .h ei s a p h e n o m e n ot on b e o b s e r v e dn ,o t a p e r s o na d d r e s s i nt h ge
viewer.
E q u a l l yc o m p l e xa n d a m b i v a l e nits t h e b a c k v i e w . O n eo f t h e a u t h o r sa, t a g e 2 1 ,
photographed his parentsin a snow-covered park,just outsideBrussels (figure4.15) and,
perhapsmoreimportantly, it wasthis picture hechoseto pin on the pinboard of hisstudent
roomin Amsterdam, ratherthanoneof the other,morefrontalpictureshe hadtakenon the
sameday.At the time,his feelingsfor his parentswerecomplex.Deepattachmentmixed
w i t h o n l y h a l f - u n d e r s t o odde s i r et o d i s t a n c eh i m s e l f r o m t h e w o r l d i n w h i c h h e w a s
b r o u g hu t p . P e r h a ptsh e p i c t u r ec r y s t a l l i z et dh e s ec o n f u s eedm o t i o nfso r h i m . 0 nt h e o n e
h a n d i, t s h o w e h d i sp a r e n t tsu r n i n gt h e i rb a c ko n h i m ,w a l k i n ga w a yf r o m h i m ( a r e v e r s a l ,
of course,of the actualsituation);on the other hand,it showedthis gestureof'turning
o n e ' sb a c k ' ,i n a s e n s e , ' f r o n t a l l iyn' ,a m a x i m a l l y ' c o n f r o n t i n g ' w B a yu.t t o e x p o s eone's
b a c kt o s o m e o n ies a l s ot o m a k eo n e s e lvf u l n e r a b l a en , dt h i s i m p l i e sa m e a s u r oef t r u s t ,
despitethe abandonment whichthe gesturealso signifies.Perhapsthe picturereminded
him of a passage from a Dutchnovelhe likedat the time:
and interaction . I39
Representation

i,ri...,'ff:i

@ fiq e.fl Photographof author'sparents,1968

Through m u c hI l o v et h a t m a n ' ,h et h i n k s ,
t h ew i n d o wh es e e st h e mw a l ka w a y . ' H o w
andhow impossible he hasmadeit for meto express that. . . . His motherhaslinked
armswith him.With hesitantstepsshewalksbesidehim on the frozenpavement. He
keepslookingat themuntiltheyturn the corner,nearthe tall featheredpoplars.
(Wolker1 s .9 6 5 : 6 I )

H o w i s ' i n v o l v e m e nrte' a l i z e di n l a n g u a g eP ? e r h a ptsh e s y s t e mo f p o s s e s s i vper o n o u n s


c o m e sc l o s e stto r e a l i z i n gt h e k i n d so f m e a n i n gw s e h a v ed i s c u s s ehde r e .B u t t h e t w o
systems, the visualsystemof horizontalangleand the linguisticsystemof possessive
pronouns/ differ in manyways.Involvement, as we haveseen,is alwaysplural,amatterof
'mine'and'his/herlits'; a matterof distinguishing between what belongs to 'us' andwhatto
'them'.And,whilein language onecannoteasilyhavedegrees of 'ourness' and'theirness',
in imagessuchgradationis an intrinsicpart of the systemof involvement. Finally,thereis
n o ' y o u r s 'i n t h e s y s t e mo f h o r i z o n t aal n g l eT. h ev i s u a 'l y o u - r e l a t i o n
i s' ,a s w e h a v es e e n ,
realizedby the systemof'offer'and 'demand'.Perspective puts a barrierbetweenthe
140 Representat i on and i nteract i on

viewerand the represented participants,


evenin the caseof a frontal angle:the viewer
looksat the represented participantsand has an attitudetowardsthem, but doesnot
i m a g i n a r ie
l yn g a g w
e i t ht h e m .

P O W E RA N D V E R T I C A LA N G L E

Textbooks of film appreciation neverfail to mentioncameraheightas an imporuanr means


o f e x p r e s s i oi n c i n e m a t o g r a p A h yh. i g ha n g l ei,t i s s a i d m
, a k e st h e s u b j e clto o ks m a l la n d
i n s i g n i f i c a natl,o wa n g l em a k e si t l o o ki m p o s i nagn da w e s o m e : ' L oawn g l e sg e n e r a l lgyi v e
a n i m p r e s s i oonf s u p e r i o r i tey x, a l t a t i o a n n dt r i u m p h. . . h i g ha n g l e st e n dt o d i m i n i s h the
i n d i v i d u a l , tfof a t t e nh i m m o r a l l yb y r e d u c i nhgi mt o g r o u n dl e v e l , t or e n d e h r i ma sc a u g h t
i n a n i n s u r m o u n t a bdl e t e r m i n i s m (M ' a r t i n ,1 9 6 8 : 3 7 - B ) .B u t t h i s l e a v e tsh e v i e w e ro u t
of the picture.We would rather say it in a somewhatdifferentway: if a represented
participantis seenfrom a highangle,then the relationbetween the interactive participants
(the producerof the image,and hencealsothe viewer)and the represented participants
is depictedas one in which the interactiveparticipanthas poweroverthe represented
participant- the represented participantis seenfrom the point of view of power.If the
represented participantis seenfrom a low angle,thenthe relationbetween the interactive
and represented participantsis depictedas one in whichthe represented participanthas
poweroverthe interactive participant. If, finally,the pictureis at eyelevel,thenthe pointof
viewis oneof equalityandthereis no powerdifferenceinvolved.
Thisis,again,a matterof degree. A represented participantcantowerhighaboveus or
l o o kd o w no n u s e v e rs o s l i g h t l yI .n m a n yo f t h e i l l u s t r a t i o ni sn s c h o otle x t b o o kw s e look
d o w nr a t h e rs t e e p loy n p e o p l -e w o r k e r si n t h eh a l l ;c h i l d r e n i n a s c h o oyl a r d .I n s u c hb o o k s
the socialworld liesat the feetof the viewer, soto speak:knowledge is power. Themodelsin
magazine advertisements andfeatures, andnewsworthy peopleandcelebrities in magazine
articles,on the otherhand,generallylook downon the viewer:thesemodelsare depicted
as exercising symbolicpoweroverus.As shownin figure4.5, productsadvertisedin the
advertisements maybe photographed bothfrom a lowangle,as havingsymbolicpowerover
us,and from a high angle,as beingwithin reachand at the commandof the viewer.The
photograph reproduced in figure4.16 showsa guardin the 'deathrow'sectionof a prison
in Texas.The angle is low,to make him look powerful.But what makesthis picture
extraordinary is that not the guard,but the horseis closestto the viewer, andthat it is not
the guard,but the horse,whoseeverymovementis commandedby this guard,who is
l o o k i n ga t t h ev i e w e rw. h a t c a nt h i sh o r s e' d e m a n d ' f r o m u s ?T h e r e i nl i e st h e m y s t e r ya n d
the forceof this picture.Empathywith a fate of beingsubjugated to the powerrepresented
by the guard?0r with a fate of suffering?
How is powerrealizedin language? Herewe need,again,to rememberthe difference
betweenface-to-facecommunication and mediatedcommunication. In the classroom.
for example,powerwill manifestitself first of all in the relationbetweenteacherand
p u p i l .T h i s ,a s c a t e P o y n t o n h a ss h o w n( 1 9 8 5 :c h . 6 ) , i s i n t h e m a i nr e a l i z e tdh r o u g ht h e
difference between the linguisticformsthat may be usedby the teachers andthe linguistic
Representation and i nteract io n

@ rig e,fO Prisonguard(DannyLyon,1969)


r42 Representati on and i nteracti on

formsthat maybe usedby the pupils;in otherwords,througha lackof reciprocity between


the choicesavailableto eachparty in the interaction. The teachermay usefirst names
in addressing the pupils;the pupilsmay not usefirst namesin addressing teachers.The
teachermay useimperatives to 'demandgoods-and-services'; the pupilswouldhaveto use
politeforms,for instance, questions.This lack of reciprocityhas its effecton everylevel
of language: phonology, grammart vocabulary, discourse,and on ideational,interpersonal,
as well as textualmeanings. If there is, in face-to-facecommunication, any questionof
powerrelationsbetweenrepresented participants andthe pupils,thenthis resultsfrom the
powerrelationbetween the teacherandthe pupils.
To someextentthis is the casein writingalso,and not just becausein writing- as in
mediatedcommunication generally- the absenceof the writer causes/ from the start, a
fundamentallack of reciprocity(you cannottalk backto the writer), but also because
the writer andthe readerare oftenunequalin a numberof otherways.Thereadermay be
addressed directly,by meansof the second-person pronounyou, whilethe writer hides
behindimpersonalforms. Mental processes may be attributedto the readerwhile the
writer's mentalprocesses are neverreferredto. Imperatives may be used,as modulated
processes predicatedof the reader(you can,you should,you need,etc.),while suchforms
are not usedof the writer.Hereare someexamples of textsin whichpoweris encodedin
this way- the first from a Revlonadvertisement, the secondlrom OurSocietyand 0thers
(Oaklee y t al.,I9B5):

Wrinkles.Theydon'tstart whereyouthinktheydo.Theystart underneath


yourskin.
That'swhy Anti-AgingDailyMoisturizergoesbeyondmeresurfacetreatment.

Whenyoustudyplacesandpeopleyouneedto havea way of keeping the information


youcollect.One wayof doingthis isto takenotesfrom the bookswhichyouread.You
t r i t ed o w na l l t h et h i n g sy o ur e a da s t h i sw o u l dm e a nw r i t i n go u t t h ew h o l e
c a n n ow
book.Notesare a shortway of recording the mostimportantinformation.

In the first text the writer doesnot directlyreferto himselfor herself,but writesas an
impersonal authority,in termsof relationalprocesses ('Theystart underneath yourskin').
The readeris referredto directly ('you'), and the writer not only knowswhat the reader
thinks('whereyouthink they do'), but alsothat the reader'sthoughtsare misguided: the
authorityof the writer is firmly basedon the reader'signorance. In the secondtext,too,
the writer doesnot directlyreferto himselfor herself,but writes impersonally, in terms
of relationalprocesses ('Notesare a short way of recordingthe most importantinfor-
e er e a d e irsa d d r e s s eddi r e c t l y( ' y o u ' )T. h ep r o c e s s w
m a t i o n ' ) , w h itl h e si t hw h i c ht h er e a d e r
is associated are modulatedin variousways('you need','yor.r cannot').In both casesthe
lackof reciprocity whichrealizespoweris encodedin the text itself.
B u t t h i so m n i s c i e nk tn o w l e d goef t h e r e a d e r ' m s i n d t, h i sd i r e c tp o s t u l a t i oonf w h a tt h e
r e a d e rn e e d s( m u s td o , s h o u l dt h i n k ,w i l l f e e l ,a n d s o o n ) a n d t h i s l a c k o f r e c i p r o c i t y
between the writerandthe readeror the speaker andhearer, cannotbe realizedin the same
way in images. In images,the powerof an image-producer must,as it were,be transferred
Representati on and i nleract i on t43

on to oneor morerepresented participants - the powerof the advertiser on the model,the


powerof the producerof the textbool< on the ordinarypeoplerepresented in the textbook.
The nearestequivalent in speechwouldbe the useof evaluative adjectives. We might,for
example,transcode a picturein whichwe lookdownon factoryworkersor refugees as'the
humbleworkerl or'the downtroddenrefugees'.lnthe issueof the Australian Women's
Weeklyfrom which we tool<many of our originalexamples, this kind of transcoding
o c c u r sa n u m b e ro f t i m e s .T h e m a g a z i n ceo n t a i n sp h o t o g r a p hosf a b e j e w e l l eQ dueen
Elizabeth, andthe actor MichaelDouglas - bothtakenfrom a low angle.0nthe coverthe
r e l e v a nat r t i c l e sa r e a n n o u n c ebdy t h e f o l l o w i n gl i n e s : ' D n z z l r r u- eT h e Q u e e n 'jse w e l s ' ;
'FASctNATIN - GM i c h a e lD o u g l a s ' " F a t aA l t t r a c t i e n " ' . B u t t h e r e r e m a i n sa v e r y b i g
difference. What in the imageis an attitudetowardsthe represented participants becomes
in language a characteristic of the represented participants: it objectifies the attitude.

NARRATIVIZATIONOF THE SUBJECTIVEIMAGE

In manycasesthereis no immediately apparentmotivationfor pointof view(andfor size


of frame).Theanglemay be highand frontal,and so conveypoweroverand involvement
with the represented participants,but the precisenatureof the relationof powerand
i n v o l v e m e instn o tg i v e nT. h u sa h i g h - a n g p l ei c t u r eo f w o r k e r si n a f a c t o r yc o u l db es a i dt o
betakenf romtheviewpointof a supervisor in an elevated office,with a windowoverlooking
the factory,but this remainsa metaphor. We do not seethe office in the picture.Other
p o s s i b i l i t i emsi g h ta l s os e r v et o m a k ec o n c r e t teh e r e l a t i o no f p o w e ra n d i n v o l v e m e nI nt .
othercasesthe (imaginary)viewerintrudesin the pictureto a greateror lesserdegree.
In an advertising campaign that ranat thetimewe worl<ed on thefirst versionof this book,
t h i s w a s d o n eb y i n c l u d i n tgh e h a n d so f t h e i m a g i n a r vy i e w e ri n t h e f o r e g r o u nodf t h e
p i c t u r eT. h e s ec o u l dt h e nb e m a l eo r f e m a l ea, n dg r o o m e di n d i f f e r e nw t a y s- t h e yc o u l d
weardrivinggloves, expensive rings,andso on.In figure4.).7theycreatedthe viewpointof
a couple.
I n f i l m s t h e s e q u e n c i nogf i m a g e sc a n f u l f i l t h i s f u n c t i o nT . he shotof the factory,
showingthe workersfrom a highangle,canbe preceded by a low-angle shotof the elevated
office,with a supervisor behindthe windowlookingdownat the workers.In suchcasesthe
text narrativizes the pointof viewand imposesa fictionalviewerbetween the represented
andthe interactive participants. But evenwhentheir originsare not shown,viewpoints can
a l w a y sb e r e l a t e dt o c o n c r e t e s i t u a t i o n s . 0 ncea n ,a n d p e r h a p s h o u l da, l w a y sa s k , ' W h o
couldseethis sceneinthisway?','Wherewouldonehaveto beto seethissceneinthisway,
andwhat sort of personwouldonehaveto beto occupythat space?'

O B J E C T I V EI M A G E S

Scientificand technicalpictures,suchas diagrams,mapsand charts,usuallyencodean


objectiveattitude.This tendsto be done in one of two ways:by a directlyfrontal or
t44 Representati on and i nteract i on

'rWhYdon'twe oXlfkata
takl nnine?" SMt*g td*d'
cil :iti;r:1.,\ !!l.til::ts:l.ii'
r !i r ;; :r j :,i r : ri: : x j. ljl !..1. ; i'; i. i! r jl i 1!;1.

(ivewldea,Novembel1987)
fiq +.f Z Sterlinqadvertisement
Q

p e r p e n d i c u ltaorp - d o w na n g l e S . u c ha n g l e sd o s u g g e svt i e w e rp o s i t i o n sb,u t s p e c i aal n d


privilegedones,which neutralizethe distortionsthat usuallycome with perspective,
because they neutralizeperspective itself.To illustratethis with a simpleexample, whena
cubeis drawn perspectivally its sides are not of equal length, and the degree of distortion
d e p e n dosn t h e a n g l eo, n t h e e n c o d evdi e w e rp o s i t i o nT. h ec u b ed o e sn o t l o o k ' a sw e k n o w
i t i s ' ,w i t h a l l i t s s i d e so f e q u a l e n g t hb, u t ' a s w e s e ei t ' , f r o m a p a r t i c u l a p
r o s i t i o nB. u t
fromdirectly i n f r o n t t h e t h i r d d i m e n s i odni s a p p e a rasn, d t h e c u b ea p p e a r fsl a t ,w i t h a l l
its sidesof equallength.Fromabove,exactlythe sameeffectoccurs.Perspective and its
attitudinizinq effect have been neutralized:

@ rig +.fe Cuheseenfrom an angle,frontallyandfrom above


o

.s
=

q
"[,Ove"n*f]lp'
'ullflavour
o[ a trulvxtisfring
granulatcd
coflce...lovca cupof
Bushclls
iVlastcr
Roast Co{Icc.

@ ftate Z Bushellsadyertisemenl(Woman'sWeekt!,1987)
Plalet Joshua Smith (William Dobell,1943)(Art callery of New South Wales)
@ etat. I Patick White(LouisKahan,1963)(Art Galleryot NewSouthWales)
gl
t.
t
I
f aI
3
a .rl

* ,
s*,,
J ': a

E
!
re
:w
:

Q etatez PalgraYe
colourscheme
w

Q C t a t eI Colourfulthoughls (transparency)
Representation and interaction - 145

Frontalandtop-downangles,however, are not objectivein entirelythe sameway.The


f r o n t a l a n g l e i s t h e a n g l e o f m a x i m u mi n v o l v e m e nItt. i s o r i e n t e dt o w a r d sa c t i o n .
The picturesof the Antarcticexplorer(figure2.4 couldbe transcoded as 'Theseare the
c l o t h e ys o us h o u l dw e a ra n dt h i si st h ew a yy o us h o u l dw e a rt h e mi f y o uw a n tt o e x p l o r teh e
A n t a r c t i c . ' T h fer o n t a la n g l ei s t h e a n g l eo f ' t h i s i s h o w i t w o r k s ' , ' t h i si s h o wy o uu s ei t ' ,
'this is how you do it'. Thetop-down
angle,on the otherhand,is the angleof maximum
power.It is orientatedtowards'theoretical',objectiveknowledge. It contemplates the
world from a god-likepointof view,puts it at yourfeet,ratherthan within reachof your
hands.Abstractdiagramscan sometimes be readin bothways.A communication model,
for instance(e.9.figure 2.2), can be read as a map ('top-down',a schema,a'theory
o f c o m m u n i c a t i o n ' : ' t hi issw h a t c o m m u n i c a t i ol ono k sl i k e ,f r o m t h e p o i n to f v i e wo f a
d i s i n t e r e s t eodb s e r v e r ' o
) ,r a s a f r o n t a lv i e w ,a b l u e p r i n ta, ' p r a c t i c a l m a n u a lo f c o m -
m u n i c a t i o n( ''t h i si s w h a ty o ud o w h e ny o uc o m m u n i c a t e- ' )a n dt h i s i s p e r h a posn eo f t h e
sources of its socialpower.
A third objective v i e w p o i n t ,h e c r o s s - s e c t i oann, d t h e , X - r a y , v i e w ,s h o u l da l s o b e
considered: its objectivityderivesfrom the fact that it doesnot stopat appearances, but
probesbeyondthe surface,to deeper, more hiddenlevels.In Westerncultureit is almost
e x c l u s i v eul ys e di n d i a g r a m sa,l t h o u g h o n ec a ns o m e t i m easl s oo b s e r veex p e r i m e nwt si t h i t
i n c h i l d r e n 'dsr a w i n g s .
Nof all diagrams/ mapsandcharts,however, arecompletely objective. Theverticalangle
o ft h e G u l W f a r m a p i n f i g u r4e. 7 9 , i sh i g h , b u t n o t c o m p l e t e l y t o p - d o w n , a n d i t s h o r i z o n t a l

tranrport planor drop


!t'dairy
' csttcr' botnbsofi
lslandln oreoaradonfsr

US plana dcrtroy
28 anks,26 othCr
wtdder thr.i
lItiugry Dlrcar
and tiici rrnmrniricn
{urer
lEri
"*dchi

@ fig l.ff cuff War map (s/dr e! Morning Hetatd,22Januaryt99I)


r46 Representati on and i nteract i on

u st o l o o ka t t h et h e a t r eo f w a r f r o m t h e s i d e l i n eisn, a r e l a t i v e l y
a n g l ei s o b l i q u ec,a u s i n g
d e t a c h ew w e f i n ds i m i l a ra n g l e s( s e ef i g u r e
d a y .I n b o o k sa b o u ts c i e n cfeo r y o u n gc h i l d r e n
5 . 1 1 ,f o r e x a m p l e )t ,h e i r o b l i q u e n e spse r h a p s u g g e s t i nt gh a t t h e y a r e n o t ( o r n o t y e t
quite)meantas 'howto do it' pictures.
Elements of perspective mayalsobeaddedto graphsandcross-sections, to givea sense
of reality,of physicalexistence, to abstract,two-dimensional visuals.Havingfirst been
abstractedfrom the concrete, three-dimensional world of people,placesand things,they
are now restoredto it, but in a transformed way,as neryhuman-made l<inds of thingsand
places.Thuswe can see- for instance, in lavishlyproducedannualcompanyreports-
t h r e e - d i m e n s i obnaarl g r a p h sl ,o o k i n gl i k es k y s c r a p eor sr m o n o l i t h sa,g a i n sat b a c k g r o u n d
o f c l e a na n d s m o o t hh i l l si n f f a t ,p r i m a r yc o l o u rI.n f i g u r e4 . 2 0 g r a p h sb e c o m ea s e t t i n g
for action:tourists movethrough the abstractlyrepresented, but nevertheless three-
d i m e n s i o n awl o, r l d o f t h e i n t e r n a t i o n taol u r i s tb u s i n e sjsu,s t a s m a y a l s ob e t h e c a s ei n
t e l e v i s i onne w sg r a p h i c s , w h ear e f u r t h e rs e n s eo f r e a l i t ym a yb e g i v e nt o s u c hp i c t u r e sb y
m e a n so f a n i m a t i o n .
The additionof perspective adds nothingto the representational meaningof these
d i a g r a m sm, a p sa n dc h a r t sb; u t i t d o e sa d da t t i t u d i n am l e a n i n g Isn. a l l t h e s ee x a m p l et sh e
angleis high,explicitlyattitudinalizing the objectivestanceof the god-liketop-downview,
andoftennarrativizing it as the viewfrom the satellite, that moderntool of the production
o f v i s u a kl n o w l e d gaen ds y m b ool f i n f o r m a t i o n p aol w e rT. h eh o r i z o n t aal n g l eo, n t h e o t h e r
handm , a yv a r y :w i t h t h e ' i n c r e a s ien t o u r i s m ' w ea r e d i r e c t l yi n v o l v e dt h; e e v e n t so f t h e
G u l f W a r ( f i g u r e4 . 1 9 ) ,o n t h e o t h e rh a n d w , e w a t c h ' f r o mt h e s i d e l i n e sa' s, b y s t a n d e r s .
This processof attitudinalization happens, not so much in the contextswherethis new

oun ?ounttlt sttLLAR ?Al(Gs (lrr


AND\ryHO'LLgEBRiltGtNG
tT tN* ym"

@ fig e.ZO An increasein tourism (Sydne! Moming Heratd,23 January]-ggll


ntati on and interacti on
Represe r47

visual knowledgeis producedand this new informationalpowerexercised, but in the


contextsin which it is disseminated in popularized form,and celebrated: hereconceptual
a n d s c h e m a t i icm a g e sa r e d r e s s e d u p i n t h e c l o t h e so f v i s u a lr e a l i t ya, n d l i t e r a l l ya n d
f ig u r a t i v e l y ' a n i m a t e d ' .
To conclude t h i s s e c t i o nw e a d d s o m en o t e so n d i f f e r e n tl,e s s ' s u b j e c t i v e ' k i n dosf
perspective. If, in centralperspective, the kindof perspective we havebeendiscussing sofar,
something is seenfrom the front and at eyelevel,the sides,top and bottomwill be hidden
f r o m v i e w A c u b ew o u l da p p e a a r s i n d r a w i n g1 o f f i g u r e4 . 2 I . I f t h e s a m ec u b ei s s e e n
f r o m a n o b l i q u e a n g l e , o nt he eo sf i d e s w i l l c o m e i n t o v i e w , b u t t h e o t hreerm w ai l il n h i d d e n .
I f t h e a n g l ei s h i g h s, ot h a t w e l o o kd o w no n t h e c u b et,h e t o p w i l l a l s oc o m ei n t ov i e wa, s
i n d r a w i n g4 o f f i g u r e4 . 2 I . B u t i n t h i sc a s et h e f r o n tw i l l n o l o n g e b r e a s q u a r eI .t w i l l b e
distorted. T h e h o r i z o n t a pl a r a l l e l s
i n an i m a g e i n c e n t r a p
l e r s p e c t i vceo n v e r gteo w a r d s
o n eo r m o r e v a n i s h i npg o i n t -
s a n d s o do t h e v e r t i c a p
l a r a l l e l s
a ,l t h ough t h i s i s o f t e nl e s s
obvious, as vertical distances are not so great, and as vertical distortion is often'corrected'
i n d r a w i n gas n dp a i n t i n g s .
D r a w i n g2 i n f i g u r e4 . 2 I t o n t h e o t h e r h a n d ,i s a n e x a m p l eo f ' f r o n t a l - i s o m e t r i c '
perspective. Herethe front of the cubeis not distorted, yetwe canseethe sideandthe top.
A n dt h e h o r i z o n t ap l a r a l l e lds o n o t c o n v e r gteo w a r d s
a v a n i s h i npgo i n t .F r o n t a l - i s o m e t r i c
perspective is basedon the 'objective'dimensions of the represented participants,on
what we know thesedimensions to be, rather than on how they appear to us. For this
reasonfrontal-isometric perspective is usedin technicaldrawings, whereit is important
to be able to measurethe dimensions of the represented objectsfrom the drawing.In
frontal-isometric perspective, then,there is not, as yet, a choice betweeninvolvement and
d e t a c h m e nItt. i s t h e a n a l o g yi n v i s u a l t e r m s o f t h e ' i m p e r s o n a l i t y ' c h a r a c t e r i sotfi c
scientific l a n g u a g e .
T h ep e r s p e c t i vues e di n d r a w i n g3 o f f i 9 u r e4 . 2 7 i s c a l l e da n g u l a r - i s o m e tpr iecr s p e c -
tive. Herethe front is distorted,the squareno longerrepresented as a square.But the
horizontaland vertical parallels do not converge. There is no end to spacein this kind
of perspective - it stretcheson indefinitely. Angular-isometric perspective was used,
for example,in eighteenth-century Japanese woodcuts - Japanese artists of this period
alwayschosean oblique point of view, as well as a relatively high angle. They looked at the
world without a senseof involvement, from a detached point of view, from a meditative
distance.

rfl
(3) cube in angular-isonetricpersp!ctive;
fig +.Zf (1) Cubeseenfrom the front; (2) cube in frontal-isometricperspective;
Q
(4) cubeseenfrom an angleIn centralperspective
148 . Representation and interaction

EtsBEE
Q Spanishnativity(from Arnheim,1974)
fig +.ZZ Detailfiom a fourt!enth-century

T h i sb r i e fs u r v e yd o e sn o t e x h a u stth e p o s s i b i l i t i eIsn. m e d i e v aalr t ' i n v e r t e dp e r s p e c -


tive'wassometimes used(seefigure4.22).Thisallowsbothsidesof an objectto be seen
and causes the perspectival vectorsto divergeratherthan converge. It can oftenbe found
i n c h i l d r e n 'dsr a w i n g s( y o u n gc h i l d r e na l s ot e n dt o d r a wt h e w o r l da s t h e yk n o wi t t o b e ,
ratherthan as they see it) and, in recenttimes,has beentaken up by painterssuchas
Picasso and Braque, who lookedfor moreobjective waysof representing theworld,regard-
ing the simpleviewpointof centralperspective as one-sided and restrictive, and viewing
r e a l i t ya s m u l t i f a c e t e da , c o m p l e xw h o l eo f o f t e n i n c o m p a t i b laen d m u t u a l l yc l a s h i n g
viewpoints. In this way,as Arnheimnotes(1974: I32),'Ihey makethe contradictions of
w h i c hM a r x i s t s o e a kv i s u a l ' .

A SUMMARY

F i g u r e4 . 2 3 s u m m a r i z et h
s e m a i nk i n d so f i n t e r a c t i vm
e e a n i nw
g e h a v ed i s c u s s ei n
d this
chapter.It shouldbe remembered that theseare'simultaneous systems/(as indicatedby
thecurlybrackets):anyimagemusteitherbea'demand'oran'offer'andselectacerta

REALIZATIONS

Demand gazeat the viewer


0ffer absence of gazeat the viewer
Intimate/personal c l o s es h o t
Social m e d i u ms h o t
Impersonal l o n gs h o t
Involvement frontalangle
Detachment o b l i q u ea n g l e
Viewerpower h i g ha n g l e
Equality eye-leve a ln g l e
Rep resented p art i cipant p ower l o wa n g l e
Representation and interaction . I49

T Demand
Contact ___________>l
L 0""'

Intimate/personal

Social distance Social

Impersonal
Intefactive
, i -

meanrngs Involvement
T
L Detachment

subjectivity viewerpower
f- I f
-+f Equaritv
| | - Representation
Auitude----rl power
L
I
I |.- Action orientatlon
L objectivitv ---1

Knowredge orientation

@ fig C.Zf Interactivemeaningsin images

sizeof frame and selectacertainattitude.In the nextfew sectionswe will discuss


some
examplesat greaterlength,to showhow the systemsof 'contact,,'socialdistance,
'attitude'interact and
to create more complexand subtle relationsbetweenreoresented
and interactiveparticipants.

TWOPORTRAITS
A N D T W O C H I L D R E N ' SD R A W I N G S

Rembrandt's famousSelf-portraitwith Saskiadatesfrom 1634.JohnBerger(1972: 111)


calls it'an advertisement of the sitter,sgoodfortune,prestigeand wealth,and,he adds,
'like all such
advertisements it is heartless'. Yet,from the pointof viewof the interactive
meanings we havediscussed in this chapteri the paintingis perhapsa little morecomptex
t h a n B e r g e r 'rse m a r k s u g g e s t . 0tnh e o n eh a n d i,t i s a ' d e m a n dp, i c t u r e- R e m b r a n d t
and
Saskiasmileat the viewer,Rembrandt perhapsa little moreeffusively and invitinglythan
Saskia:heevenraiseshisglassin a gesturedirectedat the viewer.0n the otherhand,hehas
s h o w nh i m s e l fa n d S a s k i af r o m b e h i n da, n d f r o m w h a t H a l l w o u l dc a l l ' c l o s es o c i a l ,
distance, with Saskiaa little furtherawayfrom the viewerthan Rembrandt - her headis
c o n s i d e r a bsl ym a l l e rt h a n R e m b r a n d te' sv e nt h o u g hs h ei s s i t t i n go n h i s l a p a n d s h o u l d
therefore, strictlyspeaking, becloserto the viewerthan Rembrandt (theangleat whichher
headis turnedio acknowledge the vieweralsoseemsunnaturaD.IsRembrandt distancing
himself(and Saskiaevenmore)from the viewer,excludingthe viewerfrom rnvolvement
and intimacywith his new-found(and Saskia'salreadyestablished) socialstatus,thus
150 Representation and interaction

contradicting the invitation?Perhaps - but the portrait is alsoa self-portrait.Rembrandt,


t h e m i l l e r ' ss o n ,n o w m a r r i e di n t o a w e a l t h ya n d r e s p e c t a bfl ae m i l ya n d l i v i n gi n g r a n d
style,alsodistances himselffrom his newself (andto someextentfrom Saskia),as if he
cannotfeel fully involvedand intimatewith his new environment. As a self-portraitthe
picturemay be self-congratulatory and smug,'heartless', but it also betraysa degreeof
a l i e n a t i o np,o s i t i o n i ntgh e r e p r e s e n t eRde m b r a n di n t a c o m p l e xa n dc o n t r a d i c t o rsyo c i a l
classposition,betweenthe world of his origins,which is also the point of view of the
p i c t u r ea, n dt h e w o r l do f S a s k i ai n t ow h i c hh e h a sm o v e dT. h i s w , e t h i n k ,m a k e si t a l i t t l e
lesssmug,anda little moretouchingthan Bergergaveit creditfor.
F i g u r e4 . 2 5 s h o w sa l a t e rs e l f - p o r t r a ipt ,a i n t e di n 1 6 6 1 .B y t h i st i m e ,S a s k i ah a sd i e d ,
and Rembrandt hasgonebankrupt.He now liveswith hisformerhousekeeper, Hendrickje,
i n a m o r ed o w n m a r k ente i g h b o u r h o oa dn ,di n m u c hr e d u c ecdi r c u m s t a n c eI ns t. h i sp o r t r a i t

Q fig +Zt Setf-poftrait with Sasftia(Rembrandt,1634)(Pinakotek,Dresden)


Representati on and i nteract i on 151

Q fiq+.ZS Self-portnit(Rembrandt,166I)(Kunsthist0rischesMuseum,Vienna)

he is ableto comeface-to-face with himself, to confronthimself(andthe viewer)squarely


and i n t i m a t e l w i t h h i m s e l f : ' H ei s a n o l d m a n n o w .A l l h a sg o n ee, x c e p at s e n s eo f t h e
y
q u e s t i oonf e x i s t e n coef,e x i s t e n caes a q u e s t i o n( 'B e r g e rI 9 , 72:112).
Thepictureon the coverof 'lVlyAdventure'(figure4.26),thestoryby an eight-year-old
boy whichwe havealreadyfeaturedin the previouschapter,constitutes a'demand':the
l i t t l eb o y i s l o o k i n ga t u s ,a n ds m i l i n gH . e s e e k so u r r e c o g n i t i o n
H.ew a n t st o b e a c k n o w -
l e d g e d . 0tnh eo t h e rh a n d , t h e , n dt h e b o yi s s h o w nf r o ma g r e a t
a n g l ei s o b l i q u ea,n dh i g h a
d i s t a n c eN. o t o n l y d o e st h e w r i t e r o f t h i s s t o r ys h o wh i m s e l if n t h e r o l e o f b e i n gs h i p -
w r e c k e dh,e a l s os h o w sh i m s e laf s ' o t h e r '( t h eo b l i q u ea n g l e )a, s s o m e o noev e rw h o mt h e
v i e w e rh a sp o w e r( t h eh i g ha n g l e )a n da s s o c i a l l yd i s t a n ta, ' s t r a n g e r ' ( t h el o n gs h o t ) .I n
I52 . Representation and interaction

otherwords,he usesthe interactive resources of the subjective image(quiteprecociously,


w e f e e l )t o s h o wh i m s e laf s s m a l l ,i n s i g n i f i c a a nn t d a l i e n a t e dy ,e t d e m a n d i nrge c o g n i t i o n
from the viewer.At the sametime the act of drawinghimselflikethis affordshim.as the
producerof the image,somepoweroverthat imageof himself,an outletfor hisfeelings. In
supportof this interpretation it can be notedthat the boy doesnot exactlyplay a heroic
role in the story.After creatingthe raft, and just as the raft 'startedto be goodfun',
everythinggoeswrong for him: he loseshis moneyand neverfinds it again,the raft
collapses and is lost irretrievably, andthe herohasto walk all the way home,wet andcold.
It is an unhappyendingfor a herounableto controlthe unpleasant eventsthat happento
him.
F i g u r e4 . 2 7 i st h ef r o n tc o v e or f a ' s t o r y 'o n s a i l i n gb o a t sb y a c h i l df r o mt h es a m ec l a s s
as the authorof 'My Adventure'. Its subjectis similar:peopleon a boat.But the systems
o f i m a g ea c t ' , ' s o c i adl i s t a n c e ' a n' d
attitude'tako e n v e r yd i f f e r e nvt a l u e sT. h ec h a r a c t e r s
do not look at us;the pictureis an 'offer'.Theangleis frontal and eyelevel,and the two
f i g u r e si n t h e b o a t a r e n e i t h e rp a r t i c u l a r l yd i s t a n t n
, o r p a r t i c u l a r l cy l o s e T
. h e r ei s n o
setting,no texture,no colour,no lightandshade.Thesailingboatis drawnwith geometrical
accuracy.But for the two figures- simplydrawn,and more or lessidentical,exceptfor
t h e i r s i z e( a f a t h e ra n d s o n ? )- t h i s c o u l db e a t e c h n i c adl r a w i n qA. s s u c hi t s u i t st h e

O Fiq 4,26 Coyerillustrationof My Adventure'


Representati on and i nteracti on r53

($ figl.Zl Coverillustrationol'sailing Boats'

o b j e c t i v eg,e n e r i ct i t l e ,' S a i l i n gB o a t s 'j,u s t a s t h e c o v e ri l l u s t r a t i o o


nf 'My Adventure'
suitsthat story'ssubjective, specifictitle. In mostof the illustrationsinsidethe essay, no
h u m a nf i g u r e sa r e s e e na/ s t h o u g ht h e c h i l da l r e a d yu n d e r s t a n dt hs a t t h e ' l e a r n i n go' f
t e c h n i c aml a t t e r ss h o u l db e o r e c e d ebdv a ' h u m a ne l e m e n t ' t oa t t r a c tn o n - i n i t i a t et os t h e
subject.
Clearly, chlldrenactivelyexperiment bothwith the interactive resources of language and
with the interactive resources of visualcommunication. Theyare activesign-makers. And
the differentways in which thesetwo childrenrepresentboatsshowtwo very different
subjectivities at work.
5 Modality:
g o d e l so f r e a l i t y
d e s i g n i nm

M O D A L I T YA N D A S O C I A LT H E O R YO F T H E R E A L

0 n e o f t h e c r u c i a li s s u e isn c o m m u n i c a t i oi snt h e q u e s t i o n o f t h e r e l i a b i l i t oy f m e s s a g e s .
Is what we seeor heartrue,factual,real,or is it a lie,a fiction,something outsidereality?
To someextentthe form of the message itself suggests an answer.We routinelyattach
morecredibilityto somekindsof messages than to others.Thecredibilityof newspapers,
f o r i n s t a n c er e, s t so n t h e ' k n o w l e d g e ' t h a th o t o g r a p hdso n o t l i e a n dt h a t ' r e p o r t s ' a r e
p
m o r er e l i a b l teh a n' s t o r i e s t' ,h o u g hs i n c ew e w r o t et h ef i r s t e d i t i o no f t h i sb o o kt h e r i s eo f
P h o t o s h oapn d ' s p i n ' h a vbee g u nt o u n d e r m i nbeo t ht h e s et y p e so f k n o w l e d g e .
M o r eg e n e r a l layn, dw i t h p a r t i c u l arre l e v a n ct o e t h ev i s u a lw, e r e g a r do u rs e n s e of sight
as more r e l i a b l e
t h a no u r s e n s eof hearing,'l s a w it w i t h m y o w n eyes' a s m o r ereliable
evidence 'l
than heardit with my ownears'.
Unfortunately, we also knowthat,whilethe cameramay not lie - or not much,at any
rate -those who useit andits imagescananddo.Thequestions of truth and realityremain
insecure, subjectto doubtand uncertainty and, even more significantly, to contestation and
Yet,
struggle. as members of a society, we haveto be ableto make decisions on the basisof
the information we receive, produceandexchange. And in so far as we are prepared to act,
we haveto trust someof the information we receive, anddo so, to quite some extent, on the
basisof modality in
markers the message itself,on the basis of textual cues for what can be
as
regarded credibleand what shouldbe treated with circumspection. These modality
markershavebeenestablished by the groupswithinwhichwe interactas relativelyreliable
guidesto the truth or factualityof messages, and they havedeveloped out of the central
values, beliefs andsocial needs of that group.
I n t h i sc h a p t ew r e w i l l d i s c u stsh e s em o d a l i t yc u e sA . s t h r o u g h o ut th e b o o k ,w e t a k e
them to be motivated signs - signs which have arisen out of the interestof socialgroups
who interact withinthe structuresof power that definesociallife,andalsointeractacross
the systems produced by various groups within a society.As we havediscussed in the
Introduction, the relationbetweenthe signifiersand signifieds motivated of signs is, in
principle,one of transparency. Sign-makers choosewhat they regardas apt, plausible
meansfor expressing the meanings theywishto express. We are thereforefocusingon the
rangeof signsfrom whichsuchchoicescan be made- someof themspecialized modality
markers,otherspart of a muchwider and more generalrangeof meansof expressing
meaningsof truth and falsehood, fact and fiction,certaintyand doubt,credibilityand
unreliability.
A s o c i asl e m i o t itch e o r yo f t r u t hc a n n oct l a i mt o e s t a b l i sthh ea b s o l u tter u t h o r u n t r u t h
of representations. It can onlyshowwhethera given'proposition'(visual,verbalor other-
wise) is represented as true or not. Fromthe pointof viewof socialsemiotics, truth is a
c o n s t r u cot f s e m i o s i sa,n d a s s u c ht h e t r u t h o f a p a r t i c u l a sr o c i a g l r o u pa r i s e sf r o m t h e
Modality 155

valuesand beliefsof that group.As longas the message formsan apt expression of these
beliefs,communication proceeds in an unremarkable, 'felicitous'fashion.Thisdoesmean,
however, that our theoryof modalityhasto accountfor a complexsituation:peoplenot
only communicate and affirm as true the valuesand beliefsof their group.They also
communicate and accorddeqreesof truth or untruthto the valuesand beliefsof other
gr0ups.
T h et e r m ' m o d a l i t y ' c o m ef rso m l i n g u i s t i casn d r e f e r st o t h e t r u t h v a l u eo r c r e d i b i l i t y
o f ( l i n g u i s t i c a lrl ey a l i z e ds)t a t e m e n tasb o u tt h e w o r l d .T h eg r a m m a or f m o d a l i t yf o c u s e s
on suchmodalitymarkersas the auxiliaryverbswhichaccordspecificdegrees of modality
to statements,verbslike may, will andmust Gf. the differencebetweenHe may comeand
He will comd andtheir relatedadjectivesG.g. possible,probable,certain) and adverbs
( s e e H a l l i d a y1, 9 8 5 : B 5 - 9 ) . B u t m o d a l i t yi s n o t o n l y c o n v e y e tdh r o u g ht h e s ef a i r l y
clear-cutlinguisticsystems(seeKressand Hodge,t978,7993'. 127). Tal<e this example,
from )ur Societyand 0thers(}akley et a1.,1985).Clearly,it not onlycontainsstatements
s u c ha s ' A b o r i g i n apl e o p l eh a dn o r e l i g i o n ' a n d ' t hw e h o l el a n dw a sa c a t h e d r a lb' ,u t a l s o
indications of the truth valueof thesestatements:

G o v e r n oPr h i l l i pt,h e s e t t l e r sa,n dt h e c o n v i c tcso u l df i n d n o c h u r c h eosr c a t h e d r a l s


o r w o r k so f a r t l i k et h o s eo f B r l t a i n .P e r h a ptsh i s m a d et h e mt h i n kt h a t A b o r i g i n a l
p e o p l eh a dn o r e l i g i o nI n
. f a c t , t h eA b o r i g i n ehsa dv e r yc o m p l i c a t erde l i g i o ubse l i e f s .
Thesehad beenpasseddownfrom one generatlon to the nextthroughDreamtime
storiesfor thousands of years.For the Aborigines the wholelandwas a cathedral.
Their art was joinedto their religion.Much of their art had beenkept safelyfor
thousands of years.
(Oaklee y t a l . , I 9 B 5 :I 4 2 )

T h es t a t e m e n t ' A b o r i g i npael o p l eh a d n o r e l i g i o n ' i sg i v e nl o w m o d a l i t yT. h ew r i t e r sd i s -


tancethemselves from it by attributingit to 'GovernorPhillip,the settlers,and the con-
victs'by formulatingit as a subjective idea('thismadethemthinkthat . . .') and by using
the pasttense(after all, what was true in the past neednot be true in the present).The
w r i t e r so' w ns t a t e m e n t(se . 9 . ' t h eA b o r i g i n a lhsa dv e r yc o m p l i c a t erde l i g i o ubse l i e f s 'a) r e
not qualifiedin this way,'they are formulatedas objectivefacts ('In fact, the Aborigines
had . . .') and not attributed(it is curious,however, that they are nol extendedto the
p r e s e ntti m e l ) .T h e s t a t e m e n ttsh a t e m b o d yt h e ' b e l i e f s ' o ft h e A b o r i g i n e sf i,n a l l ya, r e
givenlowermodality.Theyare explicitlyattributedto the Aboriginesand thereforenot
s u b s c r i b et do b y t h e w r i t e r s ,a n d t h e y a r e q u a l i f i e db y t e r m sl i k e ' s t o r y ' , ' d r e a m ' a n d
'belief' - termswhich,in Westernculture,signifylow modalityand are contrasted with
high-modality termssuchas 'reality','fact' and'truth'.
The exampleshowsthat modalityis 'interpersonal'ratherthan 'ideational'.It does
not expressabsolutetruths or falsehoods; it producessharedtruths aligningreadersor
listenerswith somestatements and distancingthem from others.It servesto createan
imaginary'we'. It says,as it were,theseare the things'we'considerIrue,andthesearethe
t h i n g s ' w e ' d i s t a n coeu r s e l v ef rso m , f o ri n s t a n c e : ' w e ' h a vneo r e l i g i o nb,u t t h e A b o r i g i n e s
r56 Modality

d o ,a n da l t h o u g thh i s r e l i g i o n i s t r u ef o r ' t h e m ' ,i t i s n o tt r u ef o r ' u s ' .N e v e r t h e l e a s s ,( f o r


' u s ' ) a r t a n d r e l i g i o na r e n o t ' j o i n e d ' , ' w e ' c a na p p r e c i a tAe b o r i g i n arle l i g i o na s ' a r t ' , a s
b e a u t i f u'ls t o r i e s ' a n d ' d r e a m s ' ( a r ti,n W e s t e r nc u l t u r e h, a s l o w e rm o d a l i t yt h a n ,f o r
instance, science- hencethe greaterlicencegivento artists).We call the 'we'the text
attemptsto produce'imaginary'because manyof the childrenwho are madeto readthe
bookmay in fact 'havereligion'.However, we realizethat the socialgroupings discursively
institutedin thisway may be veryreal andmayhaveveryrealeffectson children'slives.
The conceptof modalityis equallyessentialin accountsof visual communication.
Visualscan representpeople/placesand thingsas thoughthey are real,as thoughthey
actuallyexist in this way,or as thoughthey do not - as thoughthey are imaginings,
fantasies, caricatures, etc.And, heretoo, modalityjudgements are social,dependent on
what is considered real (or true,or sacred)in the socialgroupfor whichthe representation
i s p r i m a r i l yi n t e n d e d
C.o n s i d e r , fionrs t a n c e , t h e ' s p eceicr h
c u i t ' d i a g r afm
r o md e S a u s s u r e ' s
famousCoursein GeneralLinguistics (I974ll9I6J), shownoncemorein figure5.1 (see
a l s of i g u r e2 . 1 8 ) .I t d e p i c t st w o h u m a n s , ' A ' a n d ' B ' ,a n d a p r o c e s sc/i r c u l a ra n d c o n -
t i n u o u sd, escribed a s t h e ' u n l o c k i n go f s o u n d - i m a g ei ns t h e b r a i n ' ,f o l l o w e db y t h e
'transmittingof an impulsecorresponding to the imageto the organsusedin producing
sounds',followedby the 'travellingof the soundwavesfrom the mouth of A to the ear
o f B ' ( 1 9 7 4 t 1 9 1 6 1 : 1 1 - 1 2 ) .I n a n o t h e vr e r s i o n( f i g u r e5 . 2 ) ,d e S a u s s u rsec h e m a t i z e s
the diagramevenfurther,makingit lookalmostlikean electricalcircuit.
The photographin figure5.3 aiso represents the speechprocess, or ratherpart of it,
s i n c ew e s e eo n l y ' A ' s p e a k i n ga,n d o n l y ' B " u n l o c k i n gs o u n d - i m a g iensh i sb r a i n ' .I t i s a
scenefrom RobertAldridge'smovieTheBig lhife 0955), starringRodSteigerandJack
Palance.
Thethreerepresentations of the speechprocess differ in a numberof ways.First while
the photograph restrictsitselfto representing what wouldnormallybe visibleto the naked
eye,the diagramsdo not: they makevisiblewhat is normallyinvisible(mentalprocesses,
' s o u n d - i m a g ienst h e b r a i n ' )a n dt h e yd o s h o ww h a t c a n n o r m a l l yo n l yb e h e a r d( ' s o u n d
waves').To do so theytake recourse to abstractgraphicelements(dottedand continuous
lines,arrows)and to language. Second, whilethe photograph presents us with a moment

-\-I_ .-j-:
- - - - - -
- / - - - z

A B

pisf.f Speechcircuit(de
Saussure,t974n9l6l)
O
Modality 157

phonation

@ fig S.Z Schematized


speechcircuit(de Saussure,
1974[1916])

O fiS S.: RodSteigerandJack patancein fhe Big Knife (Atdridqe,


1955)

f r o z e ni n t i m e ,t h ed i a g r a mdse p i c at p r o c e stsh a tt a k e sa c e r t a i na m o u n o t f t i m et o u n f o l d :
o n eu t t e r a n c o e f ' A ' a s w e r ra s o n e u t t e r a n c o e f ' B ' , a t t h e v e r y r e a s tT. h i r d ,w h i r et h e
p h o t o g r a pdhe p i c t s ' A ' a n d ' B 'i n g r e a td e t a i ls, h o w i n g
s t r a n d so f h a i r , w r i n k r egsl i,m m e r s
of light in Steiger'sdark glasses, the diagramsreducethe two to schematicprofiles,or
evencircles,minimalgeometricshapes,abstractelements.And, whiie
the photograph
showsdepth,modellingcausedby the prayof rightandshade,and
a setting,a background,
the diagramsomit all of these.Theyare abstractand schematic wherethe photographis
158 Modality

concreteanddetailed;conventionalized andcodedwherethe photograph presents itselfas


a naturalistic, unmediated, uncodedrepresentation of reality.
Doesthis meanthat diagramsare less'real' than photographs, and hencelower in
modality,and that photographyis more true than diagrammaticrepresentation? Not
necessarily. Tothe viewersfor whomde Saussure's diagramsare intended, theymay in fact
be more real than the photograph, in the sensethat they reveala truth which represents
moreadequately what the speechprocessis reallylike.
Realityis in the eyeof the beholder;or rather,what is regardedas real dependson
how realityis definedby a particularsocialgroup.Fromthe pointof viewof naturalism
reality is definedon the basisof how muchcorrespondence there is betweenthe visual
representation of an objectandwhatwe normallyseeof that objectwith the nakedeye(or,
in practice,on the capacityof 35mm photography to resolvedetailand rendertonal or
c o l o u rd i f f e r e n t i a t i oinm: a g e si ,n c l u d i n p
g h o t o g r a p hcsa, n b e e x p e r i e n c eads ' h y p e r - r e a l ' ,
a s s h o w i n g ' t o om u c hd e t a i l '',t o o m u c hd e p t h ' , ' t o om u c hc o l o u r ' t o b e t r u e ) .S c i e n t i f i c
realism,on the otherhand,definesrealityon the basisof whatthingsare likegenerically or
regularly.It regardssurfacedetailand individualdifferenceas ephemeral, and doesnot
stopat whatcanbe observed with the nakedeye.It probesbeyondthe visualappearance of
things.In otherwords,realitymay be in the eyeof the beholder, but the eye has had a
culturaltraining,and is locatedin a socialsettingand a history;for instance,in the
communityof linguists, or of semioticians in de Saussure's day,a communitywhichsaw
r e a l i t yi n t h a t f o r m ,i n t e r m so f a b s t r a c t i o nasn d d e e p e r e g u l a r i t i eA s .' r e a l i s m ' i s p r o -
ducedby a particulargroup/as an effect of the complexof practiceswhich defineand
constitutethat group.In that sense, a particularkind of realismis itselfa motivatedsign,
in whichthe values,beliefsand interests of that groupfind their expression.
As the examplessuggest,definitionsof reality are also boundup with technologies
of representation and reproduction. The relativelyrecentchangefrom the dominance of
b l a c ka n d w h i t et o t h e d o m i n a n coef c o l o u ri n m a n yd o m a i n so f v i s u a lc o m m u n i c a t i o n
showshowquicklythesehistoriescandevelop, and howcloselytheyare relatedto techno-
logicalchange.For us, now as commonsenseviewers,everydaymembersof societyat
large,the definingtechnologyis perhapsstill that of 35mm colour photography, as we
suggested above.But the shift to digitalphotography is alreadycreatinga newstandard
for naturalism w,h i c hs t i l la i m sa t e v e rh i g h e r e s o l u t i o n ,a t u r a l i s t icco l o u rr e n d i t i o na,n d
so on,but hasin fact madea decrease in resolutionandcontrastto becomeacceptable as
t h e n o r mi n m a n yd o m a i n s .
Eachrealismhasits naturalism - that is,a realismis a definitionof what countsas real
- a set of criteriafor the real,and it will find its expression in the 'right', the best,the
(most) 'natural' form of representing that kind of reality,be it a photograph, digitalor
o t h e r w i s eo ,r a d i a g r a mT. h i si s n o t t o s a yt h a t a l l r e a l i s mas r e e q u a lA . l t h o u g hd i f f e r e n t
realismsexistsideby sidein our society, the dominantstandardby whichwe judgevisual
r e a l i s ma, n dh e n c ev i s u a m l o d a l i t yr ,e m a i n fso r t h e m o m e n tn, a t u r a l i s m a sc o n v e n t i o n a l l y
u n d e r s t o o d , ' p h o t o r e a l i sI nmo' .t h e rw o r d s t, h e d o m i n a nct r i t e r i o nf o r w h a t i s r e a l a n d
what is not is basedon the appearance of things,on how muchcorrespondence there is
betweenwhat we can'normally'seeof an object in a concreteand specificsetting,and
Modality I59

what we canseeof it in a visualrepresentation - again,atleastintheory,forin effectit is


b a s e do n c u r r e n t l yd o m i n a nct o n v e n t i o nasn d t e c h n o l o g i e s
o f v i s u a lr e p r e s e n t a t i oW ne.
judgean imagerealwhen,for instance, its coloursare approximately as saturatedasthose
i nt h es t a n d a r dt h, e m o s tw i d e r yu s e dp h o t o g r a p htiecc h n o r o g y .
w h e nc o r o u b r e c o m em sore
saturated/we judge it exaggerated,'more than real',excessive. When it is lesssatur-
atedwe judge it'less than real,,'ethereal,, for instancgor 'ghostly,.And the samecan
be saidabout otheraspectsof representation, the renditionof detail,the representation
of depth,and so on. Pictureswhichhavethe perspective, the degreeof detail,the kind of
c o l o u rr e n d i t i o ne,t c .o f t h e s t a n d a r d t e c h n o r o goyf c o l o u rp h o t o g r a p hhya v et h e h i g h e s t
modality,and are seenas 'naturalistic'.As detail,sharpness,
colour,eic. are reducedor
amplified, as the perspective flattensor deepens, so modaritydecreases.
Like manyotheradvertisements, the advertisement in plate 2 is a compositetext. It
showsa pictureof the product(the jar of instantcoffee),
with a verbarcaption,and a
p i c t u r ew h i c hv i s u a l i z et sh e p r e a s u rteh e p r o d u c t
w i i l a f f o r d .T h i sp i c t u r es, h o w i n gt w o
loverssharingan intimatemoment,usessoft focusand soft
colours,tendingtowardsthe
samegolden-brown hue,andso deliberately loweringmodality,representing ,whatusing
the
productwill be like'as fantasyor promise,as'what
'what is'. might be,, rather than as reality, as
The pictureof the productitself,however, is in sharperfocusand usesmore
s a t u r a t ea d n d d i f f e r e n t i a t ecdo l o u r st:h e p r o d u c ti s g i v e n
h i g h e rm o d a l i t yh, i g h e r e a l i t y
value,thanthe promiseof blissattachedto it, andthe advertisement
as a wholetherefore
accordsvaryingdegrees of'credibility'tothe differentrepresentations it contains,just like
t h e t e x t o n ' A b o r i g i n arle l i g i o n T ' . h e l o w e rm o d a l i t yo f t h e p h o t oo f t h e r o m a n t r cs c e n e ,
however, is not a matterof the sceneitselfbeingimprobable or fantastic(althoughthaf
t o o ,o f t e nh a p p e nisn a d v e r t i s i npgh o t o g r a p h sP)r. o b a b laesw e l l
a si m p r o b a b e t ev e n t m
s ay
havehighor low modality. Just as onecan sayTherecertainlyare ghtosts (highmodality)
and I believeghostsmay exist (ow modalitfl, so one can
also showrealisticand not-so-
realisticdepictions of ghosts.
what is the difference between theseusesof colour?we wouldput it thisway:the more
that is taken away,abstractedfrom the coloursof the representation,
tne more c0lour
i s r e d u c e dt h , e l o w e rt h e m o d a l i t yT. h e r ei s a c o n t i n u u mw h i c h
r u n sf r o m f u l l c o l o u r
saturationto the absence of colour,blackandwhite,in whichonlythe brightness valuesof
t h e c o l o u r st ,h e i r ' d a r k n e s s ' o r ' l i g h t n e sr es m
' , a i n sT. h e r ei s a l s oa c o n t i n u u m
w h i c hr u n s
f r o m f u l l c o l o u rd i f f e r e n t i a t i ot o
n a ' r e d u c e dp a l e t t e ' a n de v e n t u a l lm y onochromF e .o r
e x a m p l ee,i g h t e e n t h - c e n tluarnyd s c a ppea i n t i n g( e . g C . l a u d eL o r r a i n w ) a so f t e nr e s t r i c t e d
to variousshadesof brownfor the foregroundand to desaturated,
silverybluesfor the
d i s t a n c eT'h i si s n o tt h eo n l yw a yi n w h i c ha b s t r a c t i of nr o m' n a t u r a l i s t i c ,
c o l o u ri s p o s s i b l e .
The colour of many objectsis not even.pare skin,for instance,
may vary in redness,
may havethe blueveinsshowingthrough,and so on, and such
differences may eitherbe
rendered or abstracted from.In otherwords,colourmaybe idealized to a greateror lesser
d e g r e-e a s c a l ew h i c hr u n sf r o mn a t u r a l i s t ipch o t o g r a p hv yi a
t h ec h o i c eo f d i f f e r e nvt a l u e s
of a colourfor the representation of lightand shade, to the ffat,unmodulated colourused
by childrenin their drawings,or, for example, in the work of painterssuchas Matisse.
Matissewas not a chird,of coursgwhenhe producedthe paintings
we now admire.His
160. Modality

unmodulatedcolours expressed a differentview of what counts as real, as do the


u n m o d u l a t ecdo l o u r si n c h i l d r e n 'dsr a w i n g-s w e w i l l c o m m e not n t h i s i n m o r ed e t a i l a t e r .
Fromthe pointof viewof naturalism, however, modalityis decreased in suchimages. The
continuum f r o m m o d u l a t etdo f f a tc o l o u ri s a t t h es a m et i m ea c o n t i n u u m f r o m h i g ht o l o w
modality.And in both casesthe rule applies:the greaterthe abstraction(away from
saturation, differentiation and modulation), the lowerthe modality.
It shouldbe stressed that what we are talkingabout is not abstractionfrom what we
actuallysee,from 'the real world'. The literatureof other agesand culturesatteststo
the fact that peoplehavemarvelledat the 'lifelikeness' of workswhich,by our standards,
are f ar from 'naturalistic'. What we are talkingaboutat this pointis abstractionrelative
to the standards of contemporary naturalisticrepresentation.

M O D A L I T YM A R K E R S

S of a r w e h a v ed i s c u s s et hder o l eo f c o l o u ra sa m a r k e o
r f n a t u r a l i s t im
c o d a l i t vi n. t e r m so f
t h r e es c a l e s :

(I) Coloursaturation,ascalerunningfromfullcoloursaturationtotheabsenceofcol
that is,to blackandwhite.
(2) Colourdifferentiatio4a scalerunningfrom a maximallydiversified
rangeof colours
to monochrome.
(3) Colourmodulation,ascalerunningfromfullymodulatedcolour,with,forexample
useof manydifferentshadesof red,to plain,unmodulatedcolour.

A t o n e e n d o f t h e s es c a l e st h e p a r t i c u l a rd i m e n s i o on f c o l o u r i s m a x i m al yl r e d u c e d .
A t t h e o t h e re n di t i s m o s tf u l l y a r t i c u l a t e du,s e dt o i t s m a x i m u mp o t e n t i a lE. a c hp o i n to f
the scalehasa certainmodalityvaluein termsof the naturalisticstandard.However, the
p o i n to f h i g h e sm t o d a l i t yd o e sn o t c o i n c i d w e i t h e i t h e re x t r e m eo f t h e s c a l e n: a t u r a l i s t i c
modalityincreases as articulationincreases, but at a certainpoint it reachesits highest
valueand thereafterit decreases again.Naturalisticmodalityscalescouldthereforebe
r e p r e s e n t eads i n t h ef o l l o w i n g
example:

E i a c ka n d w h i t e lvlaximumcolour saturation

Lowest modalrty H i g h e s tm o d a l i t y Low(er) modality

Q fig s.+ Modality scale for colour saturation

We will now discussthe other key markersof visualmodalityon which we already


t o u c h e di n o u r d i s c u s s i oonf f i q u r e5 . 3 .
Modality r61

(4 Contextualization,
a scalerunningfrom the absence
of background
to the mostfullv
articulatedanddetailedbackground.

W i t h i nt h e n a t u r a l i s t icco d i n go r i e n t a t i o tnh, ea b s e n coef s e t t i n gl o w e r sm o d a l i t yB. y b e i n g


'decontextualized',
s h o w ni n a v o i d ,r e p r e s e n t epda r t i c i p a n tbse c o m eg e n e r i ca, ' t y p i c a l
example',ratherthan particular,and connected with a particularlocationand a specific
momentin time. The scaleof 'contextualization' runs from'full contextualization,,
' p l a i n ,u n m o d u l a t e d to
b a c k g r o u n d ' . O nset e pa w a yf r o m ' f u l l c o n t e x t u a l i z a t i owne, f i n d
settingswhichare out of focusto a greateror lesserdegree, or which losedetailthrough
overexposure/ resultingin a kind of etherealbrightness, or underexposure, resultingin
muddydarkness, or throughthe lossof visualdetailin the depiction.Furtherdecontextu-
alizationcan be achievedthroughellipsis:a few 'props,sufficeto suggesta setting,
o r a s m a l l ,i r r e g u l a r lsyh a p e dp a t c ho f g r e e nu n d e rt h e f e e t o f a f i g u r ew i t h a f e w l i n e s
suggesting grassindicates the setting,whilethe rest of the paperis left blanl<. 0r perhaps
the backgroundmay merelyshowan irregularpatternof light and shade,or a field of
unmodulated colour,or black,or white.
A g a i n ,t h e m o s tf u l l y a r t i c u l a t e db a c k g r o u nddo e sn o t h a v et h e h i g h e snt a t u r a l i s t i c
m o d a l i t yT. h e l i m i t a t i o n si m p o s e db y t h e r e s o l u t i o no f s t a n d a r d3 5 m m p h o t o g r a p h i c
emulsions and by the depthof field of standardlenseshaveaccustomed us to imagesin
w h i c ht h e b a c k g r o u nids l e s sa r t i c u l a t e d t h a n t h e f o r e g r o u n dW. h e nt h e b a c k g r o u nids
sharperand motedefinedthanthis,a somewhatartificial,'more than real,impression will
r e s u l -t a s ,f o r i n s t a n cien o l d e rH o l l y w o om d o v i e s h o ti n a s t u d i ow i t h b a c kp r o j e c t i o (na
c l o s e - uo p f a n a c t o ri n a c a r ,i n f r o n t o f a r e a rw i n d o wb e h i n dw h i c hw e s e et h e r e c e d i n g
l a n d s c a pi e n s h a r pf o c u s )o, r i n m u c hS u r r e a l i spt a i n t i n gs,u c ha s i n t h ew o r k o f S a l v a d o r
Dali.

(5) Representation, a scalerunningfrom maximumabstraction


to maximumrepresenta-
t i o n o f p i c t o r i adl e t a i l .

An imagemay showeverydetail of the represented participants: the individualstrands


of hair,the poresin the skin,the creasesin the clothes,the individualleavesof the tree,
and so on, or it may abstractfrom detailto a greateror lesserdegree.Again,there is a
p o i n tb e y o n d w h i c ha f u r t h e ri n c r e a soef d e t a i lb e c o m e s ' h y p e r - r eaanl ,d h e n c el o w e ri n
m o d a l i t yf r o m t h e p o i n t o f v i e w o f ' p h o t o g r a p h i n c,
a t u r a l i s mS. i m i l a r l yi,n d i s c u s s i n g
decontextualization, above, we havepointedout that reducedrepresentation of detailmay
form one of the waysin whichthe modalityof backgrounds, of what is .distant,,is lower
t h a nt h e m o d a l i t yo f t h ef o r e g r o u n(dt h e r ei s a p a r a l l ehl e r ew i t h t h e l o w e rm o d a l i t yo f t h e
p a s tt e n s ei n l a n g u a g e ) .
In photography it is not only sharpness of focus,but alsoexposure which can reduce
detail.In artwork a varietyof techniques couldbe rankedon a scale from maximumto
m i n i m u md e t a i lT' e x t u r ce a nb e c o m set y l i z e dr ,e n d e r ebdy l i n e sw h i c ht r a c et h ef o l d si n
the
clothes,for example, and theselinesmay be manyand fine,as in detailedengravings, or
few andcoarse/as in quickand readystylesof drawing.In medicaldrawings, for example,
r62 Modality

texturemay becomeentirelyconventional: dotsto indicatethe textureof onelayerof skin,


short,curvedlinesto indicatethetextureof another. Texturecanalsobeomittedaltogether
- the participantis thenrepresented merelyby the linesthat traceits contour.Beyondthis,
the contourmay be simplifiedto differentdegrees: a headmay becomea circle,the eyes
two dots,the moutha short,straightline.Diagramsand geometricalart take abstraction
evenfurtherand reducethe shapeof thingsto a smallvocabulary of abstractforms,as in
s f M o n d r i a no, r i n f i g u r e5 . 2 , d e S a u s s u r es' sc h e m a t i z e' sdp e e c hc i r c u i t ,
t h e p a i n t i n go
diagram.

6) Depth,a scalerunningfrom the absence


of depthto maximallydeepperspective.

By the criteriaof standardnaturalism, centralperspectivehashighestmodality,followed


by angular-isometric perspective, followedby frontal-isometricperspective,followedby
depthcreatedby overlapping only.Again,perspective canbecome'more thanreal,,aswhen
strongconvergence of verticallinesis shown,or a'fish-eye'perspective
is used.

(7) Illumination,a scalerunningfrom the fullestrepresentation


of the play of light and
shadeto its absence.

Naturalistic depictions represent participants astheyareaffectedby a particularsourceof


i l l u m i n a t i o nL.e s sn a t u r a l i s t iicm a g e so,n t h e o t h e rh a n d m , a ya b s t r a cftr o m i l l u m i n a t i o n ,
and showshadowsonly in so far as they are requiredto modelthe volume,especially of
r o u n do b j e c t sT. h e yh a v e ' s h a d i n g ' r a t h e t hr a n s h a d o w . 0 rt h e y u s es h a d i n gt o i n d i c a t e
recedingareasand highlightsto indicateprotrudingareas,often in wayswhich haveno
e x p l a n a t i oi n t e r m so f t h e l o g i co f i l l u m i n a t i o T
n .h i sc a nb ed o n et o d i f f e r e ndt e g r e e w
s :i t h
a fully modulateddarkeningof the areasof shadow;with just two degreesof brightness,
o n ef o r t h e ' l i t ' a r e a sa n d o n ef o r t h o s ei n s h a d o ww; i t h m o r eo r l e s sd e n s eh a t c h i n g or
dottingof theshaded areas;andsoon.At theextremeendof thescale,Iightandshadeareab-
stractedfrom altogether, andlineratherthanshadingis usedto indicatereceding contours.

(B) Brightness,a scale runningfrom a maximum number of different degreesof


to just two degrees:
brightness blackandwhite,or dark greyand lightergrey,or two
brightness
valuesof the samecolour.

Brightnessvaluescan also contrastto a greateror lesserdegree:in one picturethe


difference between the darkestandthe lightestareamaybeverygreat(deepblacks,bright
whites),in anotherthe differencemay be minimal,so that a misty,hazyeffect is created.
Thepaintingsof Rembrandt are interesting from this pointof view,in part because hisuse
o f i l l u m i n a t i oann d b r i g h t n e si s s o o f t e ni n v o k e d asa paradigm e x a m p l eo f n a t u r a l i s m ,
and in part becauseof the way his subtledivergences from naturalismoften acquire
i d e a t i o n af ul n c t i o n si n, a b r o a d l ya l l e g o r i c asle n s e .
T h e a b i l i t yo f p h o t o g r a p ht yo r e n d e rb l a c ka n d w h i t e i s l i m i t e da, s i s i t s a b i l i t yt o
differentiatebrightness values.Again,a contrastrangeand a rangeof brightness values
Modality .163

whichexceeds this ability may be experienced as \morethan real,and henceas beinqof


lowermodality.
I t f o l l o w sf r o m o u r d i s c u s s i ot nh a t m o d a l i t yi s r e a l i z e db y a c o m p l e xi n t e r p l a yo f
visualcues.The sameimagemay be 'abstract'in terms of one or severalmarkersand
'naturalistic'in
termsof others.Impressionist paintings, for example, oftenhavea narrow
brightness range,andabstractfrom light and shadow, but they havea highiynaturalistic
approachto colour.Yet,from this diversityof cuesan overaliassessment of modalitvis
derivedby the viewer.
From all this it might seemthat the realizationof modalityin imagesis muchmore
complexandfinelygradedthanthe realizationof modalityin language. yet language, too,
allowscomplexcombinations of differentmodalitycues.Take,for instance, the sentence
I absolutelydon't think he could possiblyhave done if. Is this 'low,, 'middle, or ,high,
modality?How does one 'compute'thesevarious modalitycues into one 'degreeof
credibility'?Frequently thereareevencontradictions: 1f isprobablydefinitelytruethat . . .
And in language, too,the valueof modalitycuesdepends on context.In academic writing,
for example, qualifications suchas It may wellbe the case. . .or It is quite possible that. . .
(both low modality, strictlyspeaking)servein fact to increase the credibilityof the text,
as indicatorsof the carewith whichthe writer,sjudgements weremade,and henceof the
r e l i a b i l i toyf t h e s ej u d g e m e n t s .

C O D I N GO R I E N T A T I O N

So far we havedescribed the valueof modalitymarkersin termsof the naturalistic criteria


for'what countsas real'.We havehypothesized that the abilityof moderncotourphotgg-
r a p h yt o r e n d e rd e t a i l b, r i g h t n e scso, l o u re, t c .c o n s t i t u t ef so r o u r c u l t u r et o d a ya k i n do f
s t a n d a r df o r v i s u a lm o d a l i t yw. h e n t h i s s t a n d a r di s e x c e e d e ad n, i m a g eb e c o m e s ' m o r e
than real'- an effectwhich can be achievednot only in art (and is oftenthe favoured
m o d a l i t yi n S u r r e a l i s mb) u, t a l s ob y m e a n so f t h es p e c i at le c h n i q u em s ,a t e r i a las n de q u i p -
mentof studiophotography. A certainstandardof photographic naturalism, dependent on
the stateof photographic technology and on currentphotographic practices, henceever
evoiving,has becomethe yardstickfor what is perceived as ,real, in images,evenwhen
t h e s ei m a g e sa r e n o t p h o t o g r a p h U s .n d e r p i n n i nt hg i s i s t h e b e l i e fi n t h e o b j e c t i v i t o
yf
p h o t o g r a p h ivci s i o n ,a b e l i e fi n p h o t o g r a p hays c a p a b l eo f c a p t u r i n gr e a l i t ya s i t i s ,
u n a d u l t e r a t ebdy h u m a ni n t e r p r e t a t i o n B.e h i n dt h i s , i n t u r n , i s t h e p r i m a c yw h i c h i s
accordedto visualperception in our culturegenerally. Seeinghas,in our culture,become
s y n o n y m o uwsi t h u n d e r s t a n d i nwge. ' l o o k ' a t a p r o b l e mw. e ' s e e ,t h e p o i n t .w e a d o p ta
'viewpoint'. 'focus'
we on an issue,we 'seethingsin perspective,. Theworld,aswe seeit,
(ratherthan 'as we knowit', andcertainlynot ,aswe hearit,
or'as we feel it,) hasbecome
the measure for what is 'real,and 'true,.
S ov i s u am l o d a l i t yr e s t so n c u l t u r a l l a
y n dh i s t o r i c a l ldye t e r m i n esdt a n d a r dosf w h a t i s
real and what is not, and not on the objectivecorrespondence of the visualimageto a
realitydefinedin somewaysindependently of it. At the momenthologramsare probably
r64 Modality

s t i l l s e e nb y m o s t p e o p l ea s ' m o r e t h a n r e a l ' .I n t h e i m a g e sw e a r e m o s t u s e dt o , t h e
absence of thethird dimension, theffatness of the picture,doesnot functionas an indicator
of low modality, just asthe absence of perspective in cultureswhoseart doesnot employit
doesnot functionas an indicatorof low modalityfor membersof thosecultures.
As we havealreadydiscussed in relationto de Saussure's'speech circuif',however, even
w i t h i no u r o w nc u l t u r et h e s a m es t a n d a r dfso r w h a t i s ' r e a l ' a n dw h a t i s n o t d o n o t a p p l y
in every context.In technologicalcontexts,a different conceptof reality underlies
v i s u am l o d a l i t ya, c o n c e p t w ceo u l dc a l| ' G a l i l e a rne a l i t y 'I.n t h ee a r l ys e v e n t e e nct e
hn t u r y ,
Galileo wrote:

I do not find myselfabsolutelycompelledto apprehendtobjectsJas necessarily


accompanied by suchconditions as that they mustbe white or red,bitter or sweet,
s o n o r o uos r s i l e n t s, m e l l i n gs w e e t l yo r d i s a g r e e a b l y . . . .tl h i n k t h a t t h e s et a s t e s ,
smells,colours,etc.with regardto the objectin which they appearto resideare
nothingmore than mere names.. . . I do not believethat thereexistsanythingin
externalbodiesfor excitingtastes,smells,sounds,etc.,exceptsize,shape,quantity
a n dm o t i o n .
( q u o t e di n M u m f o r dt,9 3 6 i 4 8 )

H e r e ' r e a l ' m e a n s ' w h acta n b e k n o w nb y m e a n so f t h e m e t h o d so f s c i e n c et' h ; a t i s ,b y


m e a n so f c o u n t i n gw, e i g h i n a g n d m e a s u r i n gB.y t h i s s t a n d a r do f w h a t i s r e a l ,a t e c h n i c a l
linedrawing,withoutcolouror texture,withoutliqhtor shade, andwithoutperspective, can
h a v eh i g h e rm o d a l i t yt h a na p h o t o g r a p h E.v e r y d acyo m m o ns e n s en a t u r a l i s m a n dr e a l i s m
n o l o n g e rm e r g eh e r e .T h e r e a l i s m( a n d h e n c et h e ' n a t u r a l i s m ' o) f s c i e n t i f i c - t e c h n i c a l
i m a g e si s o f a d i f f e r e n kt i n d ,b a s e di,n t h e e n d ,o n t h e q u e s t i o n s , C awne u s ei t ? , , , C a n
we measurethe real dimensions from it?', 'Can we find out from it how to set up the
experiment?', and so on. Whateverdoesnot contributeto this purposemerelyaddsa
d i m e n s i oonf i l l u s i o n i s m ' ttoh e p i c t u r ea, n dd i l u t e s' G a l i l e a n realism'with c o m m o ns e n s e
' n a t u r a l i s mT' .h e l a t t e ro, f c o u r s e ,
i s s o m e t i m edso n ef o r t h e p u r p o s e of communicating
s c i e n t i f i ci d e a so r t e c h n o l o g i c a co l m p l e x i t i et o s a p u b l i co f n o n - i n i t i a t eIsn. h i s b o o k
writing Biology(1990) GregMyerscompares the reportingof the'same'research find-
ingsin specialist and popularjournalssuchas ScientificAmerican,andvisualrepresenta-
t i o n si n t h e l a t t e rt e n dt o b e l a v i s hf ,u l l - c o l o uar n d' h y p e r - r e a w l ' ,h i l ei n t h e f o r m e rs p a r s e
linedrawingsare the only form of visualimage.Furthermore, we haveto be awarethat
thereare competingtheoriesof realityin today'sscience, despitethe fact that for many
p r a c t i c apl u r p o s eG s a l i l e a rne a l i t yr e m a i n o s f o v e r r i d i nigm p o r t a n c e A.l t e r n a t i vteh e o r i e s
mightleadto differentstandards for highand low modality.
I n o t h e rc o n t e x ttsh e ' h y p e r - r e a l ' d oneost h a v et h ed e c r e a s emdo d a l i t yi t h a si n ' p h o t o -
graphic'naturalism M .a g a z i n p e h o t o so f f o o da r e o n ee x a m p l eA. d i f f e r e npt r i n c i p l e for
what countsas real operates here,the converse of Galileanreality:the morea picturecan
c r e a t ea n i l l u s i o no f t o u c ha n d t a s t ea n d s m e l l t, h e h i g h e ri t s m o d a l i t yI.n s u c hi m a g e s
e v e r y t h i nigs d o n et o a p p e atlo ' s e n s o r y ' q u a l i t i erse:a l i t yh e r ei s c o n s t i t u t epdr e c i s e lbyy
t h o s es e n s a t i o nwsh i c hG a l i l e o brandea d s i l l u s i o n st e: x t u r ec,o l o u r i ' f e e Il 't. i s h e r et h a tt h e
Modality L65

affective v a l u e so f c o l o u r sc o m ei n t ot h e i ro w n ,f o r e x a m p l eT. h ee m o t i v ev a l u eo f c o l o u r
is sometimes seenas a generalcharacteristic of colour.But in scientific-technological
c o n t e x t sc ,o l o u rm a yb ec o n v e n t i o n(aml o r eo r l e s sa r b i t r a r y ' c o l o ucro d e s ' t of a c i l i t a t teh e
reading o f c o m p l ed x i a g r a m sa) ,n di n n a t u r a l i s m c o l o u r sa r et h e r e ' b e c a u st hee ya r et h e r e
in reality'.Fromthe pointof viewof the 'sensory'definitionof reality,on the otherhand,
coloursare thereto be experienced sensuallyand emotively- it is for this reasonthat
peopleenjoythe highlysaturated andunmodulated coloursof,say,Matisse, or that children
enjoythe highlysaturated andunmodulated coloursof their plastictoys.Withinnaturalism
t h e s ec o l o u r sa r e ' l e s st h a n r e a l ' ,b u t w i t h i na r e a l i s mt h a t t a k e ss u b j e c t i veem o t i o nas n d
sensations as the criterionfor what is realandtrue,theyhavethe highestmodality.
Thereis,finally,a third area in whichthe standardof 'photographic, naturalismdoes
not apply,the areaof'abstractreaiism'- bothin science(e.9.the'speechcircuit'diagram
i n f i 9 u r e s5 . 1 a n d5 . 2 ) a n di n a b s t r a cat r t . Hi g h e re d u c a t i oinn o u rs o c i e t iys ,t o q u i t es o m e
extent,an educationin detachment, abstractionand decontextualization (and against
naturalism), andthis resultsin an attitudewhichdoesnot equatethe appearance of things
with reality,but looksfor a deepertruth 'behindappearances'. Justas academically trained
personsmay accordgreatertruth to abstractexpositorywriting than to storiesabout
c o n c r e t ei ,n d i v i d u ael v e n t sa n d p e o p l es, o t h e y m a y a l s o p l a c eh i g h e rv a l u eo n v i s u a l
representations whichreduceeventsandpeopleto the'typical,,andextractfrom themthe
' e s s e n t i aqlu a l i t i e s ' .
Whileour ideashereare drawnto a largeextentfrom the theoreticalwork of Jurgen
Habermas(especially his Theoryof communicative Action,1984), and to someextent
f r o m t h a t o f B o u r d i e u( 1 9 8 6 ) ,w e w i l l u s eB e r n s t e i nt' es r m ' c o d i n go r i e n t a t i o n( ,1 9 8 1 )
for thesedifferentrealityprinciples. Codingorientations are setsof abstractprinciples
whichinformthe way in whichtextsare codedby specificsocialgroups,or withinspecific
i n s t i t u t i o n ac lo n t e x t sW. e d i s t i n g u i st h ef o l l o w i n g :

( 1 ) Technologicalcoding orientations,which have, as their dominant principle,the


'effectiveness' of the visual representation as a 'blueprint'.whenever colour,for
example,is useless for the scientificor technological purposeof the image,it has,in
this context,low modality.
( 2 ) Sensory codingorientations,which are usedin contextsin whichthe pleasure principle
is allowedto be the dominant:certain kinds of art, advertising,fashion,food
photography, interiordecoration, and so on. Herecolouris a sourceof pleasureand
affectivemeanings, and consequently it conveys highmodality:vibrantreds,soothing
b l u e sa, n ds o o n- a w h o l ep s y c h o l o goyf c o l o u rh a se v o l v etdo s u p p o rtth i s .
( 3 ) Abstractcodingorientations,which are usedby sociocultural elites- in'high'art, in
academicand scientificcontexts, and so on. In suchcontextsmodalityis higherthe
morean imagereduces the individual to the general, andthe concreteto its essential
qualities.Theabilityto produceand/orreadtextsgroundedin this codingorientation
o ,f b e i n ga n ' e d u c a t epde r s o n ' o ar ' s e r i o u a
i s a m a r ko f s o c i adl i s t i n c t i o n s rtist'.
G ) Thecommonsensenaturalisticcodingorientation,whichremains,for the time being,
t h e d o m i n a not n ei n o u r s o c i e t yI t. i s t h e o n ec o d i n go r i e n t a t i oanl l m e m b e ros f t h e
r66 M odality

c u l t u r es h a r ew h e nt h e ya r e b e i n ga d d r e s s eads ' m e m b e ros f o u r c u l t u r e 'r, e g a r d l e s s


of how much educationor scientific-technological training they have received.
Individuaw l si t h s p e c i ael d u c a t i oonr g r o u pa l l e g i a n cm e a yd r a wo n n o n - n a t u r a l i s t i c
codingorientations in certaincontexts, but theyare likelyto revertto the naturalistic
codingorientationwhenthey are 'just beingthemselves'. Theymay,for example,use
t h e a b s t r a c ct o d i n go r i e n t a t i ow n h e nv i s i t i n ga g a l l e r ya, n d t h e n a t u r a l i s t icco d i n g
o r i e n t a t i own h e nw a t c h i n g t e l e v i s i oonr r e a d i n ga m a g a z i n eF.o r t h o s ew i t h o u ts u c h
education, however, abstractand technological imageswill neverhavehighmodality
a n d a l w a y sr e m a i n ' u n r e a l T ' . o d a yh, o w e v e nr ,a t u r a l i s mi s c o m i n gi n t o c r i s i s a
, sa
resulo t f n e ww a y so f t h i n k i n ga n dn e wi m a g et e c h n o l o g i eI n s .t h i sc o n t e xtth e r o l eo f
s o m eo r a l l o f t h en o n - n a t u r a l i sct iocd i n go r i e n t a t i o ni ssl i k e l yt o b e c o m oef i n c r e a s i n g
importance.

T h e d i a g r a mi n f i g u r e 5 . 5 s h o w sh o w t h e s a m ec o l o u rc o n t i n u u mr,u n n i n gf r o m ' n o
abstraction'to'full abstraction'(abstraction alwaysbeinga matterof degree)can have
differentmodalityvalues,accordingto the four codingorientations. It is drawnherefor
coloursaturation, but it couldalsohavebeendrawnfor anyof the othermodalitymarkers
we discussed in the previous section.

M O D A L I T YI N M O D E R NA R T

The issueof modalitybecomesparticularlycomplexin modernart, becauseit has,to a


largeextent,beenthe projectof modernart to redefine'reality',and to do so in contra-
d i s t i n c t i otno p h o t o g r a p hni ca t u r a l i s mI n. t h i ss e c t i o nw e w i l l a t t e m p t o d i s c u sas f e w o f
t h e i s s u e sb,e g i n n i nwgi t h s o m eA u s t r a l i aenx a m p l e s .

F u lI c o l o u r ( Somewhatlessthan B l a c ka n d w h i t e
saturation f u l l c o l o u rs a t u r a t i o n )

Scientific/techno
Iogical L0wEST +-___________ > HIGHEST
MODALITY MODALITY

(a0ove
Abstract +---------+ LowEST +----_____+HIGHEST
modality) MODALITY IVIODALITY

(below
Naturalistic maxtmum H I G H E S T + _ _ - _ _ _ _ _ _ +L 0 w E S T
+---------+
mooailty) MODALITY MODALITY

Sensory H r G H E I I+ - _ _ - _ _ _ _ - _ _ _ LowESr
IVIODALITY MODALITY

@ fig S.S Modalityyaluesof coloursaturationin lour codingoiientations


M odality I67

T h e p i c t u r ei n p l a t e3 s h o w sw i l l i a m D o b e l l ' sp o r t r a i to f J o s h u as m i t h ,a p a i n t i n g
w h i c hw o n t h e a n n u a lA u s t r a l i a nA r c h i b a l dP r i z ec o m p e t i t i ofno r p o r t r a i tp a i n t i n gi n
1 9 4 3 . I tw a st h ef i r s t m o d e r np a i n t i n tgo d o s o .A l l p r e v i o uws i n n e r sh a db e e nc o n v e n t i o n a l
'academic'portraitists,
s t a y i n gw e l l w i t h i nt h e b o u n d so f n a t u r a l i s t idc e p i c t i o nA. f t e r
Dobellhad beenawardedthe prize,a numberof conservative painterstook the trustees
of the prizeto court for givingit to a paintingwhich,they argued,was not eligible,as it
was not a portrait but a caricature.The prosecutor, GarfieldBarwick,interrogated
Dobela l b o u te v e r yd e t a i lo f t h e p a i n t i n ga, s k i n gh i m w h e t h e rh e h a d f a i t h f u l l yr e p r e -
sentedthe ears,the neck,the arms,and so on. The painter,in exasperation, answered:
'Yes,within the limits
of arf.' From a naturalisticpoint of view,the paintingdoesof
c o u r s eh a v ec o m p a r a t i v elloy w m o d a l i t yb, o t h i n t h e d i r e c t i o no f ' l e s st h a n r e a l ,a n d i n
the directionof 'more than real'. Colourdifferentiationis greatlyreduced, to a palette
of orange,yellow and brown. The representation of detail, on the other hand, is
a m p l i f i e de,x a g g e r a t e d , ' m ot hr e a nr e a l ' .F r o ma n a t u r a l i s t ipco i n to f v i e wt,h e p r o s e c u t o r
was right. But he applieda criterionwhichwas no longervalid in the contextof modern
art, a way of matchingmodalityvaluesto the scalesof colourdifferentiation, represen-
tation and so on which,in the historyof modernart, had beensuccessfully contested
decadesearlier.In modernart, the truth of paintingno longerlies in beingfaithful to
a p p e a r a n c ebsu/ t i n b e i n gf a i t h f u lt o s o m e t h i n e g l s e- f o r e x a m p l et ,o s o m em o d e r n
abstracttruth, in the caseof this rather 'expressionist, painting,to .the spirit of the
man', and'the essence of what he lookslike',as Dobellhimselfformulatedit durinqthe
t r ia l .
Attemptsto alter definitionsof reality are always likely to producescandal,and
t h e r e f o r ree s i s t e dw. h e t h e ri n t h et r i a l so f D . H .L a w r e n c e n, so v eol r D o b e l l ,psa i n t i n g , t h e
issues arefar largerand morefar-reaching thanaesthetics or artisticconvention. Changes
in the definitionof reality haveprofoundcultural and socialeffects,and this helpsto
explainextremereactiotrs suchas the proscription of'entarteteKunst,or the burninqof
books.
D e s p i t teh e t r i a l ,f u t u r ew i n n e r so f t h e A r c h i b a l dP r i z ew o u l d w , i t h o u te x c e p t i o nb,e
modernartistsratherthan conventional portraitpainters.Thepictureshownin plate4, a
portraitof the writer Patricl< White by Louisl(ahan,is one of them.It showsa different
m o d a l i t yc o n f i g u r a t i oan ,d i f f e r e nst e t o f a b s t r a c t i o nasn d a m p l i f i c a t i o nUs n . l i k eD o b e l l ,
l(ahandoesnot deviatefrom tlrenaturalisticrepresentation ofcletail;ifone disregards the
strangelyunseeing eyes,White'sfeaturesare rendered with naturalisticfaithfulness. But
textureis amplified:the figureof White seemsto havebeencarvedout of somepgrous,
chalkyrock,and the renditionof its surfaceis so detailedthat one can almostfeel its
c o l d ,w e t t o u c h- a ' s e n s o r yo' r i e n t a t i o nb,u t i n t h e d i r e c t i o no f d i s p l e a s u r e a t h e rt h a n
p l e a s u r ec .o l o u ro, n t h e o t h e rh a n d i,s g r e a t l yr e d u c e di n: w h a t i s a t o n c ea p u no n w h i t e , s
nameand a symbolicgesture,White is drainedof all colour.ThusKahandepictshim as
c o l d ,h a r d ,a l m o s tr e p u l s i vteo t h e t o u c h ,a n d t h e e x p r e s s i oonf t h i s ' t r u t h , h a s t a k e n
precedence over the faithful renderingof outward appearances; indeed,has become
p o s s i b loen l yb y m e a n so f t h e s e ' d e v i a t i o n s ' f r o t hme n a t u r a l i s t isct a n d a r dT. h ev i s u aol u n
remindsus that modalityis alwaysrelatedto the values,meaningsand beliefsof a
168 M odality

particulargroup/in this casean 'ordering'of the figureof PatrickWhitewithinthe system


of Australianhighculture,andof Australiansocietygenerally.
0 u r s e c o n de x a m p l ep e r t a i n st o t h e g e o m e t r i ca b s t r a c t i o n i somf t h e 1 9 2 0 s .W h e n
Europeanpainters/after studyingvisiblerealityfor centuries, beganto conceiveof it as
madeup of abstract,geometrical elements(circles,cones/squares, triangles),realitywas
redefined as a configuration of basicelements, just as hadalreadyhappened, for instance,
in physics.Within this new definitionof reality,paintersat first still soughtto produce
recognizable representations, as shownin figure5.6. Mondrian,who triedto painttreesin
thisway,complained that it wasdifficultto represent treesas arrangements of rectangular
shapes.But soon these painterswent a step further and abandonedthe attempt to
reconcile the visiblesurfaceappearance of thingswith their geometricinnerstructure(see
figure5.7). Fromhereitwas onlyonestepto,forinstance, GerritRietveld's ColourProject
for the SchroderResidence (t923-4).
Rietveld's work (figure5.8) is no longera reduced, abstractrepresentation of reality,
but a designfor a new reality,yet to be constructed. 0f course/blueprintsand planshad
existedalongside visualrepresentation longbeforethe 1920s,but in separate domains.In
the twentiethcentury,however, they becameintertwined.Art becameintertwinedwith
design,just as sciencehad alreadybecomeintertwinedwith technology. The boundaries
betweenrepresenting reality and constructingreality becameblurred.And when real
thingswereproduced from designs suchas these,the processes of abstraction couldcome
f u l l c i r c l ea n dy i e l d' n a t u r a l i s t i ci m
' a g ea
s g a i n( f i g u r e5 . 9 ) .
In the work of Ryman,a contemporary Americanartist,abstractionis perhaps takento
its limit. Many of his paintingsare,at leastat first glance,white surfaces.Everythingis
reduced, everything abstracted. Thereis no colour,no line,no background. And, in terms
of our earlierchapters, there is neitherrepresentation of actionor of socialconstructs,
nor yet any indicationof textuality,of composition.This truly is the degreezero of
representation.

@ fi9S.O Cad-playerc(TheovanD0esburg,lg16-17)(DoesburgArchive,TheHague)
o risl'z compositiong(ahstractversionofcad-ptayers)(TheovanDoesburg,rgl6-u)(fromJaff6,1967)

@ rig s'a cobur Prciect fot theschrfuderResidence(Gerrit


Rietyeld,lgz3.4) (catal0gue 81, stedetijk Museum,Amsterdam)
I70 Modality

, t',1" ' '


-:l'::li:j::t:!

Q figS,f PhotographoftheSchrdderresidence('RietYeldHouse')(fromBrown'1958)

But if reductionandabstraction serveto revealotherwise hidden,innertruths,the same


m i g h tb es a i da b o u tR y m a n ' s p a i n t i n g s
i n. d e e d l i
, k e s o m a n yo t h e rn o n - n a t u r a l i sat irct i s t s ,
h e s e e sh i s w o r k a s r e a l i s t i ca, n d c a l l s h i s p a i n t i n g s ' r e a l i s tpi ca i n t i n g st'h: e y a s p i r et o
presentthe reality and the truth of the process of representation and the processof
p e r c e p t i oann, dt h e r e b yp e r h a pas l s oo f t h e s o c i a a
l n d c u l t u r a wl o r l d '
Thereis,however, another, muchlessabstractfeatureof Ryman'spainting:hisconcern
with texture.His work showsa constantpreoccupation with the materialsof representa-
tion, and with the materialityof the processes of representation. Someof the paintings
leavea patch of the canvasuncovered and only thinly cover the rest.Othersdisplaya
varietyof brushstrokes or, by contrast,completely de-emphasize the way in which the
p a i n t i s a p p l i e d r, e s u l t i n gi n t o t a l l y f f a t , p r e s u m a b lsy p r a y e ds u r f a c e sA. g a i n o t h e r s
emphasize lheframe,or the meansby which frames are attached to walls, or the flatness
of the painting,by rotating it through ninety degrees, and so foregrounding its two-
dimensionality.
In otherwords,thereis a strongrepresentational concernin thesepaintings, but it is a
concernwith representing the process of representation. Does thls suggest low modality,
giventhe enormousdistancefrom everyday naturalism? 0r does it suggest the highest
modality,in whichthe negationof representation forms an ultimate truth, or in which the
highestmodalityis accorded to the representation which does not represent but simply is?
Modality

As before,our answeris one which refersto the social.Whethera representation is


judqedcredibleor not is not necessarily amatterof absolutetruth.What onesocialgroup
considers crediblemay not be considered credibleby another. Thisis why we seemodality
as interactive/ ratherthan ideational,as social,ratherthan as a matter of some inde-
p e n d e n t lgyi v e nv a l u eM . o d a l i t yb o t hr e a l i z easn dp r o d u c esso c i aal f f i n i t y , t h r o u gahl i g n i n g
the viewer(or reader, or listener)with certainformsof representation, namely thosewith
w h i c ht h e a r t i s t ( o r s p e a k e o r ,r w r i t e r ) a l i g n sh i m s e l fo r h e r s e l fa, n d n o t w i t h o t h e r s .
M o d a l i t yr e a l i z ews h a t ' w e 'c o n s i d et r u eo r u n t r u er,e a lo r n o t r e a l .I n t h i sl i e ss o m eo f t h e
powerof art. To the extentthat peopleare drawninto this,we,,new
values,new modes
o f t h i n k i n ga n dp e r c e i v i ncga ne s t a b l i sthh e m s e l v eAsn. dw h e ne n o u g hp e o p l e a r ed r a w ni n ,
t h e o r g a n so f p o p u l a r f i z i n gc)u l t u r gs u c ha s a d v e r t i s i n w g ,i l l q u i c k l ym o v ei n t o a m p l i f y
the newforms,and movethem intothe mainstream of culture.

MODALITYCONFIGURATIONS

T h ee x a m p l eisn t h e p r e v i o usse c t i o n s h o w t h a t t h em o d a l i t yv a l u e si n a r l c a n0 ec o m p t e x .
A paintingcan reducenaturalismin the way it treatscolour,amplifyit in the way it treats
texture,andyet represent its subjectin a naturalisticway,as in plate4. It can be abstract
in respectof onemodalitymarker, naturalisticin respectof anotherandsensoryIn respect
of yet another, andthis allowsa multiplicityof possible modalityconfigurations, andhence
a m u l t i p l i c i t yo f w a y si n w h i c ha r t i s t sc a n r e l a t et o t h e r e a l i t yt h e y a r e d e p i c t i n ga n d
' d e f i n er' e a l i t y g e n e r a l .
in I n m a n yo t h e rk i n d so f i m a g e st o , o , ' m o d a l i t ym a r k e r sd, o n o t
moveen bloc in a particulardirectionacrossthe scales,say from the abstractto
the
sensory/but behavein relativelyindependent ways.Most glossymagazinefood photo-
g r a p h sf,o r i n s t a n c ea,r e h i g h l ys e n s o r yi n t h e i r d e p i c t i o no f
t h e f o o d .T h e c o l o u r sa r e
intense.The textureof the food is shownin sharp detail. Lightingenhances the fresh
d r o p l e t os f w a t e ro n a b u n c ho f g r a p e so,r t h ev i s c o s i toyf a s a u c eo,r t h eg l a z i n go f t h eh a m
and the cherriesin a pie. But the surrounding objectstend to havelowermodality.The
weaveof the tableclothon whichthe food is displayed, for instance, may be onlyjust be
visibleand often the settingis absentaltogether, with the food shownagainsta black
background. In otherwords,suchpicturesare not onlysensory, theyare alsoabstract.The
'sensorily'depicted
food is takenout of its context,idealizedand essentialized. And this
showsthat eachof the modalitychoicesin sucha modalityconfigurationis expressive of
s p e c i f im
c e a n i n gw s ,h i c ht h e nc o m et o g e t h eirn t h ew h o l e .
F r o m o u r i n v e n t o r yo f m o d a l i t ym a r l < e r w s e c o u l d c o n s t r u c t ' m o d a l i t yp r i n t s ,
( b o r r o w i ntgh e m e t a p h oor f ' v o i c ep r i n t , , ' D N Ap r i n t , ,
e t c . )t o c h a r a c t e r i zt h
ee m o d a l i t y
configurations, and showwhichmodalitymarkersare reduced, made'lessthan real,,and
w h i c ha r e a m p l i f i e dm, a d e ' m o r et h a n r e a l ' - a n dt h i s e i t h e r i nr e l a t i o nt o a n a n c h o r i n g
p o i n to f c o m m o ns e n s e c o d a l i t y( a so n em i g h td o f o r a n a u d i e n coef ' l a y ,
h i g hn a t u r a l i s t im
p e o p l ea t a n a r t e x h i b i t i o n , t a l <ti h
negr e p r e s e n t a t i o nf uanl c t i o no f a r t a s a c o m m o ns e n s e
point of departure)or in relationto an anchoringpoint situated[n some
other realism.
Figure5.10 is an attemptto showwhat we havein mind.
t72 Modality

N A T UR A L I S M
Colourmodulation
Colourdifferentiation
Background representation
Detailrepresentation
Tonality
etc.
(abstraction)
.r+ (exaggeration)

@ fiq S.fO Modalitycontiguration

Such modalityconfigurations would describewhat, in a specificgenreor a specific


work/is regarded as real/asadequate to reality.And it wouldalsodemonstrate that images
are polyphonic/ weavingtogetherchoices from differentsignifying systems, differentrepre-
sentational modes/into onetexture.In this view,a term suchas'painting'is an artificial
constructwhich bringstogetherand treats as a homogeneous unit what is in realitya
complexconfiguration of differentvoices,differentrepresentational modes.(ln the same
way it can be said that 'grammar' is an artifice of theory,describingwidelydifferent
representational modes- phonicsubstance, intonation,lexis,syntax,etc.) And it is of
coursefrom herethat the interesting questions can be asked.Are there,or couldtherebe,
socialand historicalexplanations for thesemodalityconfigurations?
Here is an example,from a sciencetextbookfor the upperyearsof primaryschool,
p r o d u c e di n A u s t r a l i a( f i g u r e5 . 1 1 ) . T h i s i s a s c i e n t i f i c - t e c h n i pc iacl t u r ef o r c h i l d r e n .
As such it forms a compromisebetweenthe naturalisticand the technological coding
o r i e n t a t i o np,e r h a p bs e c a u sae' p u r e ' t e c h n o l o g i cpailc t u r ew o u l dh a v eb e e nr e g a r d e b dy
t h e w r i t e ra s b e y o n tdh e u n d e r s t a n d i o n fgy o u n gc h i l d r e n0.n t h e o n eh a n d i, t i s a d r a w i n g
and not a photographand it lacks a Setting;on the other hand,it usesperspective
( a n g u l a r - i s o m e t rci co)l,o u r( i d e a l i z e fdf ,a tc o l o u r )a, n di t s h o w sa t l e a s st o m e t h i nogf l i g h t
and shade(thoughin a rathersimpleand,in part, inconsistent way),and of texture(the
grainof the wood,the textureof the headof the nail,the creasesin the pieceof cloth).The
producerof this imageperhapsoperateswith the assumption that childrenare familiar
with the naturalisticcodingorientation(that is,'wherethey comefrom') and haveto be
i n d u c t e idn t ot h et e c h n o l o g i ccaol d i n go r i e n t a t i o(nt h a ti s ,t h ep r o g r e s s i oi nnt od i s c i p l i n a r y
k n o w l e d g eT)h. ei m a g ec a p t u r etsh i st r a n s i t i o n aplh a s e .
Diagrams,maps and charts for lay readersmay be 'naturalized'in similar ways.
Newspaper diagramsand maps/for instance, may be drawnin perspective (seethe Gulf
W a r m a p i n f i g u r e4 . 1 9 ) .l v l a g a z i n e msa ya d dc o l o u ra n d p i c t o r i a l i zpei ec h a r t sI.n c o m -
panybrochures or annualreports,the barsof bar graphsmay becomethree-dimensional
a n d r i s e ,l i k ef e a t u r e l e sssk y s c r a p e r s , f r oam c l e a nl a n d s c a poef u n d u l a t i nhgi l l si n s t r o n g ,
f f a t c o l o u rT . h i s s h o w st h a t m o d a l i t yi s a s y s t e mo f s o c i a ld e i x i sw h i c h ' a d d r e s s e s ' a
particularkind of viewer,or a particularsocial/cultural group,and providesthroughits
systemof modalitymarkersan imageof the cultural,conceptual andcognitivepositionof
the addressee. At the sametime it showsthe transitionacrossand betweensuchgroups,
Modality 173

C) (Jennings,1986)
fig s.ff Compass

and in doingso demonstrates the socialaspectof modality.Most crucially,it showshow


modalityis motivated, in the closematchingof modalityandthe modaladdress(location)
of specific (and assumed) aspectsof the viewer'ssubjectivity.
F i g u r e5 . 1 2 s h o w sa d r a w i n gb y N e w t o ni,l l u s t r a t i ntgh e s e t - u pf o r o n eo f h i s c o l o u r
e x p e r i m e n tF s h o w i n gt h e s e t - u pf o r a n
s .i g u r e5 . 1 3 i s a m o d e r ns c i e n t i f i ci l l u s t r a t i o n
experiment by Strattonwhichcausedhimto seehimself stretched out in spaceas indicated
in the drawing.To moderneyes,Newton'sdrawinghas not yet advanced very far in the
ml o d a l i t yh: e u s e s
d i r e c t i o no f h i g ht e c h n o l o g i c a ( i n v e r t e dp) e r s p e c t i v aen, d s h o w st h e
Setting.The moderndrawing,by contrast,leavesout the setting and simplifies the forms,
concentrating on the relationbetweenthem, ratherthan on the representation of the
e x p e r i m e n taenr dt h e m i r r o r s .
A s H a l l i d a yh a ss h o w n( H a l l i d a ya n d M a r t i n ,I 9 9 3 i 5 4 - 6 $ , N e w t o n 'w s r i t i n gd i d n o t
yet havethe objective,impersonalstanceand the lexical density of modern scientific
w r i t i n g .A t t h e s a m et i m e h e m a d es o m ed e c i s i v m e oves t o w a r d d
s e v e l o p i n t h
g e gram-
maticalresources that wouldbecomecharacteristic of scientific writing. Clearly the same
can be saidof his scientificdrawings. And that shows that, however great the differences
betweenthe verbal and the visual grammar,they derivefrom similar concernsand
orientations.
@ fig l.fZ Drawingby Newton(BodteianLibrary)

Q fiq S.ff Drawingof Stratton'sexperiment(Gregory,1970)


6 T h e m e a n i n go f c o m p o s i t i o n

A N D T H E M U L T I M O D A LT E X T
COMPOSITION

In previouschapterswe haveconsidered the way imagesrepresent the relationsbetween


t h e p e o p l ep, l a c e sa n dt h i n g st h e yd e p i c ta, n dt h e c o m p l e xs e t o f r e l a t i o ntsh a t c a ne x i s t
between i m a g e sa n dt h e i rv i e w e r sA. n y g i v e ni m a g ec o n t a i n a s n u m b eo r f s u c hr e p r e s e n -
tationalandinteractive relations. In figure6.I,an imagefrom Bergman'sThrougha Glass
D a r k l y( 1 9 6 1 ) ,w e s e el ( a r i n( H a r r i e tA n d e r s s o nw) ,h o s u f f e r sf r o m a n i n c u r a b lm e ental
disease, and heryoungerbrotherlVinus( Lars Passgard). Fromthe pointof viewof repre-
sentation t h, e s h o tc o n t a i nw s h a tw e h a v ec a l l e da ' n o n - t r a n s a c t i rveea c t i o n('l ( a r i nl o o k s
o u t o f t h ef r a m ea , t s o m e t h i ntgh e v i e w e cr a n n o st e e )a n da ' t r a n s a c t i vree a c t i o n('M i n u s
looksup at hissister).Thesechoicesrelateto the themesof the dramaticaction:l(arinhas
v i s i o n ss,e e st h i n g so t h e rp e o p l ec a n n o ts e e ;M i n u si s c a u g h ti n t h e h e r ea n d n o w o f h i s
p r o b l e m a t irce l a t i o n w s i t h t h e o t h e rc h a r a c t e risn t h e f i l m . F r o mt h e p o i n to f v i e wo f
i n t e r a c t i vm e e a n i n gt ,h e v i e w e ri s p o s i t i o n ecdl o s e rt o l ( a r i n ( ' m e d i u ms h o t ' )t h a n t o
M i n u s( ' 1 o n g s h o t ' ) ;a n d ,w h i l eM l n u si s s e e nf r o mb e h i n dl ,( a r i nf a c e st h ev i e w efrr o n t a l l y .

,w'wk,

@ fhroughaGtassDarkt!(Belgman,1960)
figO.f HarrietAnderssonandLarsPassgardin
176 The meaning of composition

Clearly, the vieweris meantto be mostcentrallyinvolved with l(arin,andwith her mental


lurmoil.
Thesepatternsdo not exhaustthe relationssetup by the image.Thereis a third element:
the compositionof the whole,the way in which the representational and interactive
elements are madeto relateto eachother,the way they are integratedinto a meaningful
wholeM . i n u sf,o r i n s t a n c ei s, p l a c e do n t h e l e f t ,a n d l ( a r i no n t h e r i g h t .I f t h i sw e r et u r n e d
around,the representational and interactivemeaningswould not be affected.l(arin,s
reactionwould still be 'non-transactive' and Minus, reaction'transactive,, and l(arin
w o u l ds t i l lb e i n m e d i u ms h o t ,M i n u ss t i l l i n l o n gs h o t .B u tt h e m e a n i n o g f t h ew h o l ew o u l d
no longerbe the same.In otherwords,the placement of the elements(of the participants
and of the syntagms that connectthemto eachotherandto the viewer)endowsthemwith
specificinformationvaluesrelativeto eachother.We will discussthe valueof ,left, and
' r i g h t 'i n t h e n e x ts e c t i o n .
In addition,l(arinis the mostsalient,themosteye-catching elementin the composition,
notjust because sheis placedin the foreground andbecause sheformsthe largest,simplest
elementin the picture,but also because sheis in sharperfocusand receives the greatest
a m o u no t f lightT . h r o u g h o umt u c ho f t h ef i l m l ( a r i ni s d r e s s eidn l i g h tc o l o u r sa n dm a d et o
bathein light,in an almostsupernatural fashion,this in contrastto the othercharacters.
For thesereasons sheis alsothe mostsalientelementin the shotswhereoneof the other
characters, for exampfeherhusband, is placedin the foreground. Herwhiteclothesandthe
lighton her palefacedrawattentionto her,evenwhensheis placedin the background. To
g e n e r a l i zpei,c t o r i ael l e m e n tcsa n r e c e i v set r o n g eor r w e a k e r ' s t r e s s ' t h aont h e re l e m e n t s
i n t h e i ri m m e d i a tvei c i n i t ya,n ds o b e c o m e m 0 r eo r l e s si m p o r t a n t ' i t e mosf i n f o r m a t i o ni n,
t h ew h o l e .
A verticallineformedby the left edgeof the door of the shed,and continuedby the
d i v i d i n gl i n eb e t w e ean p a r t i c u l a r llyi g h ta n da d a r k e rb o a r do n t h e r o o fo f t h e s h e dr, u n s
t h r o u g ht h e m i d d l eo f t h e p i c t u r ed, i v i d i n gi t i n t o t w o s e c t i o n sl i,t e r a l l ya n d f i g u r a t i v e l y
'drawinga line'between
the spaceof l(arin,who can'look intothe beyond,, andthe space
o f M i n u sw , h o c a n n o tT. h ew o r l do f l ( a r i ni s t h u ss e p a r a t efdr o m t h e w o r l do f M i n u s i,n
t h i s p i c t o r i acl o m p o s i t i oans i n t h e d r a m a t i ca c t i o no f t h e f i l m a s a w h o l ew , h e r eM i n u s ,
d e s i r ef o r c o n t a c a t n dc o m m u n i own i t h h i ss i s t e r e m a i n su n f u l f i l l e dT.h e r ei s y e t a n o t h e r
d e m a r c a t i olni n e i n t h e p i c t u r e t: h e h o r i z o nw, h i c hd i v i d e st h e p i c t u r ei n t o t h e z o n eo f
'heaven'andthe zone 'earth'.ln
of his discussion of ritian's Noli Me Tangere,Arnhejm
(1982: lr2-r3) describeshow the staff of christ forms a 'visual boundary,between
c h r i s t ,w h o i s a l r e a d y ' r e m o v ef rdo m e a r t h l ye x i s t e n c ea' n , d M a g d a l e nw, h o i s n o t ;a n d
h o w ' t h el o w e rr e g i o ni s s e p a r a t ebdyt h e h o r i z o nf r o mt h e u p p e r e g i o no f f r e es p i r i t u a l i t y ,
i n w h i c ht h e t r e ea n dt h e b u i l d i n gosn t h e h i l l r e a c hh e a v e n w a r dI n' . f i g u r e6 . 1 ,s i m i l a r l y ,
l(arin straddlesthe two zones,half still of the earth,half alreadyin the realm of 'free
s p i r i t u a l i t yw' ,h i l eM i n u si s ' h e l dd o w nb y t h e h o r i z o ni n t ot h e r e g i o no f t h e e a r t h , .M o r e
generally, composition also involvesframing(or its absence), throughdeviceswhichcon-
n e c to r d i s c o n n eec lte m e n tosf t h ec o m p o s i t i osno, p r o p o s i ntgh a t w e s e et h e ma sj o i n e do r
as separatein someway,where,without framing,we wouldseethem as continuous and
c o m p l e m e n t a rt h y :e r ew o u l db e n o v i s u a'ld i r e c t i v eo,f t h i sk i n d .
The meaning of composition L77

Composition, and interactive


then,relatesthe representational of the image
meanings
lo eachotherthroughthreeinterrelatedsystems:

( 1 ) Inforntationvalue.The placement of elements (participants andsyntagms that relate


themto eachotherand to the viewer)endowsthem with the specificinformational
valuesattachedto the various'zones'of the image:left and right,top and bottom,
c e n t r ea n dm a r g i n .
( 2 ) Salience.The elements(participantsas well as representational and interactive
syntagms) are madeto attractthe viewer'sattentionto differentdegrees, as realized
by suchfactorsas placement in the foreground or background, relativesize,contrasts
i n t o n a lv a l u e( o r c o l o u r )d, i f f e r e n c ei ns s h a r p n e sest,c .
3 ) Framing.The presenceor absenceof framingdevices(realizedby elementswhich
createdividinglines,or by actualframelines)disconnects or connects elements of the
i m a g es, i g n i f y i ntgh a t t h e yb e l o n go r d o n o t b e l o n gt o g e t h eirn s o m es e n s e .

These t h r e ep r i n c i p l eosf c o m p o s i t i oanp p l yn o tj u s tt o s i n g l ep i c t u r e sa,s i n t h ee x a m p lw ee


h a v ej u s t d i s c u s s etdh;e ya p p l ya l s ot o c o m p o s i tvei s u a l sv,i s u a l sw h i c hc o m b i n e t e x ta n d
i m a g ea n d ,p e r h a p so,t h e rg r a p h i ce l e m e n t b s ,e i t o n a p a g eo r o n a t e l e v i s i oonr c o m p u t e r
screen. In the analysisof composite or multimodaltexts (andanytext whosemeanings are
realizedthroughmorethan onesemioticcodeis multimodal), the questionariseswhether
the productsof the variousmodesshouldbe analysed separately or in an integrated way;
w h e t h etrh e m e a n i n gosf t h e w h o l es h o u l db e t r e a t e da s t h e s u mo f t h e m e a n i n gosf t h e
parts, or whetherthe parts shouldbe lookedupon as interactingwith and affecting
oneanother. It is the latterpathwe will pursuein this chapter. In considering, for example,
'illustration'of
t h e p i c t u r eo f t h et r a i n( f i g u r e3 . 3 0 )w e d o n o s e e kt o s e et h e p i c t u r ea sa n
the verbaltext, therebytreatingthe verbaltext as prior and more important,nor treat
visualandverbaltext as entlrelydiscreteetements. We seekto be ableto lookat the whole
pageas an integrated text.0ur insistence on drawingcomparisons betweenlanguage and
v i s u a cl o m m u n i c a t i osnt e m sf r o m t h i s o b j e c t i v eW. e s e e kt o b r e a kd o w nt h e d i s c i p l i n a r y
boundaries between the studyof language andthe studyof images, andwe seek/as much
a s p o s s i b l teo, u s ec o m p a t i b llea n g u a g a en, dc o m p a t i b ltee r m i n o l o gt yo s p e a la<b o u tb o t h ,
for in actual communication the two, and indeedmany others,cometogetherto form
integratedtexts.
In our viewthe integrationof differentsemioticmodesis the work of an overarching
c o d ew h o s er u l e sa n d m e a n i n gps r o v i d et h e m u l t i m o d atle x t w i t h t h e l o g i co f i t s i n t e -
g r a t i o nT. h e r ea r et w o s u c hi n t e g r a t i ocno d e st :h e m o d eo f s p a t i acl o m p o s i t i ow n i,t h w h i c h
w e w i l l b e c o n c e r n eidn t h i sc h a p t e ra; n d r h y t h m , t h em o d eo f t e m p o r acl o m p o s i t i oT nh .e
former operatesin texts ln whlch all elementsare spatiallyco-present - for example,
paintings, streetscapes, magazine pages.Thelatteroperates in textswhichunfoldovertime
- f o r e x a m p l es,p e e c hm, u s i cd, a n c e( s e ev a n L e e u w e n1,9 9 9 ) .S o m et y p e so f m u l t i m o d a l
t e x t u t i l i z eb o t h ,f o r e x a m p l ef i l m a n d t e l e v i s i o na,l t h o u g hr h y t h mw i l l u s u a l l yb e t h e
d o m i n a nitn t e g r a t i vper i n c i p l e i n t h e s ec a s e s .
I t f o l l o w st h a t t h e p r i n c i p l eosf l n f o r m a t i ovna l u es, a l i e n caen df r a m i n ga p p l yn, o t o n l y
L78 Themeaning of composition

to pictures,but also,for example, to layouts.Plate2, an advertisement for Bushells instant


coffee,containstwo photographs and a smallamountof verbaltext.The largerphotois a
pictorialrepresentation of the 'prornise'ofthe product,and it is placedin the top section.
Thephotoof the productis smaller, and placedbelowthe largerphotograph, togetherwith
the text. Reversing this wouldproducean entirelydifferenteffect,and probablyresultin a
ratheranomalouslayout.Just what informationvaluesthis arrangement accordsto the
two sectionsof the pagewill be discussed below.As far as salienceis concerned, we can
notethat this pageis not dividedintotwo equalhalves. Thetop sectionis the mostsalient,
n o t o n l yb e c a u soef i t s s i z eb u t a l s ob e c a u steh e s a l i e n coef t h e w o m a nw , hoispositioned
o n t h e r i g h ta n dc a t c h e m s o s to f t h eg o l d e ng l o wo f t h e l i g h t T . h u st h e a d v e r t i s e m egnitv e s
greaterstressto the promiseof the productthan to the productitself,or the verbal
information. Finally,a sharplinecreatesa boundarybetween the photoandthe verbaltext,
dividingthe pageintotwo separate sections,two spaces/ reserved for two differentkindsof
meaning - onefor the promiseof the product,enhanced intimacybetweenlovers;the other
for the productitself.Just as thereis a dividinglinebetweenheavenand earth in Titian's
Noli Me Tangere, and in the stillfrom Bergman'sThrougha GlassDarkly,so thereis, in
this advertisement too,a dividinglinebetween the world of 'what mightbe',the happiness
the productmight bring,and the world of 'what is', the productitself - and, just as
in the two earlierexamples, this product,the jar of instantcoffee,straddlesthe two
domainsof meaning, forminga bridgebetweenthem.The homepageof Sony'swebsite
(http://www.sony.com) has a similarstructure.The top part showsthe pleasurederived
f r o mu s i n gt h ec o m p a n y 'psr o d u c t sa,n dw e l c o m et sh e u s e rt o t h e' w o r l do f S o n y 'w , h i l et h e
bottompart showsa rangeof actualproductsand allowsthe userto click on the pages
from wherethe productscan be ordered.
E a r l yp r i n t e dp a g e ss t i l l t r e a t e dt e x t a s ' v i s u a lm a t e r i a l 'W . a l t e r0 n 9 ( 1 9 8 2 : 1 1 9 f f . )
describes how sixteenth-century title pagesbroke up wordswithout regardfor syllable
boundaries, and useddifferenttypesizesin a way that was not relatedto the relative
importanceof words,but servedto createpleasingvisualpatterns.However, the printed
p a g es o o nd e v e l o p ei d n t ot h e ' d e n s e lpyr i n t e dp a g e 'i n w h i c hr e a d i n gi s l i n e a ra n dt e x t u a l
i n t e g r a t i oanc h i e v ebdy l i n g u i s t im c e a n s( c o n j u n c t i o ncso,h e s i vt ei e s ,e t c . ) .I n b o o k so f t h i s
kind it seemsthat the pagehasceasedto be a significant textualunit.The pageshownin
p l a t e2 , o n t h eo t h e rh a n d i,s a s e m i o t i u c n i t ,s t r u c t u r e dn,o t l i n g u i s t i c a lbl yu,t b y p r i n c i p l e s
o f v i s u a lc o m p o s i t i o nI n. s u c ha p a g ev e r b a lt e x t b e c o m e jsu s t o n e o f t h e e l e m e n t s
i n t e g r a t ebdy i n f o r m a t i ovna l u es, a l i e n caen df r a m i n ga, n dr e a d i n gi s n o t n e c e s s a r il li yn e a 4
w h o l l yo r i n p a r t ,b u t m a yg o f r o mc e n t r et o m a r g i no, r i n c i r c u l a fra s h i o no,r v e r t i c a l l y , e t c .
And this is the case,not only in contemporary magazines and websites, but also in many
othercontexts - for instance, in modernschooltextbooks, as we will showlater(e.g.figure
6.6).
It shouldbe noted,of course,that the layoutof the denselyprintedpageis still visual,
still carriesan overallculturalsignificancg as an imageof progress. The denselywritten
pagesof otherculturaltraditionsare laid out differently- as,for examplein the Talmud,
whichhasthe oldesttext,the M ishna,in the centre,the Gemarawrittenaroundit; and later,
m e d i e v acl o m m e n t a r i easg a i n ,a r o u n dt h e G e m a r ai,n c o n c e n t r i lca y e r sI.n s u c hc a s e s ,
The meaning of com?osition ' u9

however, everypageis still readthe sameway.In the caseof magazine pagesandthe pages
of moderncomputerscreens, eachsuccessive pagemay havea differentreadingpath.
Thisdevelopment beyondthe densely printedpagebeganin the late nineteenth-century
masspress,in a contextin whichthe rulingclass,itselfstronglycommittedto the densely
p r i n t e dp a g e a, t t e m p t e tdo m a i n t a i ni t s h e g e m o nbyy t a k i n gc o n t r o lo f p o p u l a rc u l t u r e ,
commercializing it, andsoturningthe mediao/the peopleintothe mediaforthepeople(see
W i l l i a m s1 9 7 7 : 2 9 5 ) .T h e i ro w nc o m p a r a b lm e e d l a- ' h i g h ' l i t e r a t u r a
e n dt h e h u m a n i t i e s
g e n e r a l l-y b e c a m e v e nm o r ef i r m l yf o u n d e d o n t h es i n g l es e m i o t i o c f w r i t i n g .L a y o uw
t as
not encouraged here,becauseit undermined the powerof the denselyprintedpageas/
literally,the realizationof the most literaryand literatesemioticmode.Thegenresof the
d e n s e lpyr i n t e dp a g et,h e n ,m a n i f e stth e c u l t u r a cl a p i t a l( ' h i g h ' c u l t u r af o l r m s )c o n t r o l l e d
b y t h e i n t e l l e c t u aaln da r t i s t i cw i n go f t h e m i d d l ec l a s st,o u s eB o u r d i e u t' se r m s( 1 9 8 6 ) .
Y e t i t i s t h i s s a m es o c i a g l r o u pw h i c hh a sb e e ni n s t r u m e n t ai nl s p r e a d i ntgh e n e wv i s u a l
literacyto thosewho werenot,or not yet,to be initiatedinto the formsof literacywhich
c o n s t i t u t e idt s o w n m a r k o f d i s t i n c t i o n( t h e ' m a s s e so' ,r c h i l d r e n )a, n d t o e m b r a c ei t ,
for examplein 'high'cultureavant-garde manifestations/ as an expresslon of their oppos-
itionalrole withinthe middleclassas a whole.As so often in the twentiethcentury,they
turnedout, in the end,to havebeensawingoff the branchon whichtheyweresitting.The
distinction b e t w e e n ' h i g h ' a n d ' l o w ' f o r m si s n o w e v e r y w h e rien c r i s i s ,a n d n e w w a y s
o f m a i n t a i n i ncgu l t u r a h l e g e m o nayr e r e q u i r e df o , r i n s t a n cteh e d e v e l o p m eonft d i f f e r e n t
and differentlyvaluedways of talking about forms which,themselves, are no longer
d i f f e r e n t i a t ei ndt h e o l dw a y ( t h e ' d i s c o u r s e s 'doif f e r e n t ' a u d i e n c eB s 'u) t.t h em o s th i g h l y
v a l u e dw a y so f t a l k i n g( a n ds e m i o t i ci s o n eo f t h e m )r e m a i nt h e m s e l v eb so u n dt o m e t h o d s
that cannotadequately describethe newforms.If we are to understand the way in which
vital text-producing institutionslike the media,educationand children'sliteraturemake
senseof the worldand participatein the development of newformsof socialstratification,
a t h e o r yo f l a n g u a gies n o l o n g e sr u f f i c i e natn d m u s tb e c o m p l e m e n t eb dy t h e o r i e w s hich
can make the principlesof the new visual literacyexplicit,and describe, for instance,
the roleof layoutin the process of socialsemiosis that takesplaceon the pagesof the texts
produced by theseinstitutions - as we will try to do in this chapter.

G I V E NA N D N E W :T H E I N F O R M A T I O NV A L U E O F L E F T A N D R I G H T

Manyof the double-page spreadsin the Australianwomen'smagazines we usedas oneof


our datasetswhenwe wrotethe first versionof this chapterusethe layoutshownin figure
6 . 2 . T h e i r i g h tp a g e sa r ed o m i n a t ebdy l a r g ea n ds a l i e npt h o t o g r a p hf rso mw h i c ht h eg a z e
of one or more womenengagesthe gazeof the viewer(what, in chapter4, we called
'demand'pictures).Thesepagesshowwomen in specificand sometimescontradictory
roles,with whichthe readersof the magazine are invitedto form a positiveidentification: a
m o t h e ra; f o r m e r' s o a p i es t a r ' t u r n e dh o u s e w i faen d h a p p yi n t h a t r o l e ;w o r k i n gw o m e n
c a p a b l eo f c o p i n gw i t h ' t o u g h ' , ' m a s c u l i njeo'b s .T h e i r l e f t p a g e sc o n t a i nm o s t l yv e r b a l
text,with graphically salientphotographs on the right.Thespreadshownin figure6.2 hasa
The meaning of composition

{i.

l:::l

fD fiS e.Z Gold-diggers(Iustra lian Women'sWeekry,Noyember1987)

photograph on the left also,but this photois smallerand,in contrastto the photoon the
r i g h tp a g ei,t i s a ' f f y o n t h ew a l l ' p h o t o g r a pw
h ,h i c hd o e sn o ta c k n o w l e d gt hee p r e s e n coef
the photographer, nor thereforethat of the viewer.It is what, in chapter4, we calledan
'offer' picture.0n suchpagesthere often is a senseof complementarity or continuous
movement from left to right,as in figure6.2,wherethe photograph on the left is tiltedto
form a vectorthat leadsthe eyesto the photograph on the right,andwherethe colourgold,
with its obviousconnections to thethemeof the story,is usedas anotherintegrating device:
it occursin the photograph as the colourof the helmetsand of the liquidbeingpoured,and
is usedalsoas the background againstwhichthe verbaltext is printed.
0n suchpagesthe rightseemsto be the sideof the keyinformation, of what the reader
mustpay particularattentionto, of the 'message'-whetherit is the invitationto identify
w i t h a r o l e m o d e lh i g h l yv a l u e di n t h e c u l t u r eo f t h e m a g a z i n e o r s o m e t h i negl s e ;f o r
example, an instanceof what is to be learnedin a textbook.It followsthat the left is the
sideof the'alreadygiven',something the readeris assumed to knowalready, as part of the
culture,or at leastas part of the cultureof the magazine. In figure6.2, goldminingis
Given,and the fact that womencan engagein it, and that you,the reader, shouldidentify
w i t h s u c h' t o u g h ' w o m e ni s/ N e wt,h e m e s s a gteh,e ' i s s u e ' .
Lookingat what is placedon the left and what is placedon the right in other kinds
of visualshasconfirmedthis generalization: whenpicturesor layoutsmakesignificantuse
The meaning of composition 181

of the horizontalaxis,positioning someof theirelements left,andother,differentonesright


o f t h ec e n t r e( w h i c hd o e sn o t ,o f c o u r s eh,a p p e ni n e v e r yc o m p o s i t i o nt h) /ee l e m e n tpsl a c e d
on the left are presented as Given,the elements placedon the rightas New.For something
to be Givenmeansthat it is presented as something the vieweralreadyknows,as a familiar
andagreed-upon pointof departurefor the message. For something to be Newmeansthat
it is presented as something which is not yet known,or perhapsnot yet agreeduponby
the viewer,henceas something to which the viewermust pay specialattention.Broadly
speaking, the meaningof the Newistherefore'problematic','contestable','the information
', is
" a t i s s u e " w h i l et h e G i v e ni s p r e s e n t eadsc o m m o n s e n s i csaell,f - e v i d e nTth. i ss t r u c t u r e
ideologicalin the sensethat it may not correspond to what is the caseeither for the
produceror for the consumerof the imageor layout.The importantpoint is that the
informationis presented as thoughit had that statusor valuefor the reader,and that
readershaveto readit withinthat structure, evenif that valuationmaythenbe rejectedby
a oarticularreader.
A s i m i l a rs t r u c t u r e x i s t si n s p o k e nE n g l i s h( s e eH a l l i d a yI ,9 B 5 : 2 7 4 f f . ) .A s i n v i s u a l
c o m m u n i c a t i ot h ne
, structure p lh r a s ei s, n o ta c o n s t i t u e n t
o f a ' t o n eg r o u p 'a, n i n t o n a t i o n a
structure, with strongframingbetween elements, but a gradual,wave-likemovement from
left to right (or, rather,from 'before'to 'after', sincein languagewe are dealingwith
temporallyintegrated texts),and it is realizedby intonation. Intonationcreatestwo peaks
o f s a l i e n cw e i t h i ne a c ht t o n eg r o u p ' - o n ea t t h e b e g i n n i nogf t h e g r o u pa, n d a n o t h e trh, e
majorone(the'tonic',inHalliday'sterminology),astheculm t hienNae t iw
o n, aotft h e e n d .
J u s ta s i n f i g u r e6 . 2 w e h a v eo n ep e a ko f s a l i e n coen t h e l e f t ,i n t h e b o l dh e a d l i naen dt h e
red bar whichseparates it from the articleitself,and anotheron the right,in the photoof
the two women/so we wouldhaveonepeakof salience on the syllablegold andanotheron
the syllablewo- of womenin:

c o l D - d i g g i ncga nn o wb e d o n eb y w o m e n

And just as the imageof the two womenis the New in figure 6.2, so the word women
wouldbethe New,the keypointof the message. in the clauseabove.In otherwords,thereis
a closesimilaritybetweensequentialinformationstructurein languageand horizontal
structurein visualcomposition, and this atteststo the existence of deeper, moreabstract
codingorientatlons which find thelr expression differentlyin differentsemioticmodes.
y h e r et h e h o r i z o n t adl i m e n s i o n
S u c hc o d i n go r i e n t a t i o nasr e c u l t u r a l l ys p e c i f i cc,e r t a i n l w
i s c o n c e r n e Idn. c u l t u r e sw h i c hw r i t ef r o m r i g h tt o l e f t ,t h e G i v e ni s o n t h e r i g h ta n dt h e
N e wo nt h e l e f t ,a ss h o w ni n f i g u r e6 . 3 , w h e r teh e E n g l i s a h n dt h eA r a b i cl a n g u a gvee r s i o n s
o f S o n y ' sM i d d l eE a s tw e b s i t e a r ec o m p a r e d .
So far we havetaken a compositetext as our example,but the Given-Newrelation
appliesalsowithin an image.Figure6.4 showsa fourteenth-century reliefdepictingthe
creationof Eve.God is the Given,agreedoriginand departurepoint of all that exists.
'Woman',on the otherhand,is Newand,in the contextof the Genesis story,problematic,
t h e t e m p t r e sws h o l e a d sA d a m i n t o s i n .M i c h e l a n g e loon, t h e o t h e rh a n d ,i n h i s f a m o u s
paintingThe Creationof Adam on the ceilingof the SistineChapel,placedGod on the
The meaning of composition

Q riq e.r Englishand Arabiclanguageyersionsot sony'swebsite(http://www.sony-middleeast.com)


Themeaning of composition 183

@ fig e.a TheCreationof Eve(LorenzoMaitani,lourteenth


century)(from Hughes,1969)

r i g h t ,i n l < e e p i nwgi t h t h e n e r yh u m a n i s t iscp i r i t o f t h e R e n a i s s a n ci ne .t h i s p e r i o dG o d


suddenlybecameNew,and problematic. Generations of philosophers wereto attemptto
redefineHim in wayscommensurate with the newscience, andto try to proveHis existence
b y t h e u s eo f l o g i cI.n t h i sp i c t u r et h e m o v e m e ni st n o l o n g e frr o m G o dt o ' M a n ' , b u t f r o m
' M a n ' t o G o d . ' M a n ' r e a c h eosu t ,a s p i r i n g
t o d i v i n es t a t u sa, n da l m o s ta c h i e v i nigt - b u t
n o tq u i t e .
I n m a g a z i nlea y o u t s u c ha st h e o n es h o w ni n f i g u r e6 . 2 ,t h es p a c eo f t h e G i v e ni s f i l l e d
by verbaltext, and the spaceof the New,or at leasta largepart of it, by one or more
i m a g e sB. u t t h i s i s n o t a l w a y st h e c a s eA . d o u b l e - p a gaed v e r t i s e m efnotr M e r c e d e s - B e n z
showed,on the Ieft, a Mercedesphotographed objectively(rather than, for example,
f r o m t h e d r i v e r ' sp o i n to f v i e w ) a
, n dw i t h t h e w e l l - k n o w M n e r c e d eesm b l e mi n t h e c e n t r e
o f t h e c o m p o s i t i o nT.h e r i g h t p a g ec o n t a i n e d o n l y v e r b a lt e x t ,w i t h a h e a d l i n esaying,
'Mercedes-Benz agreeswith its competitors. Youshoulddrivetheir carsbeforeyoudrivea
Mercedes-Benz.' In otherwords,the advertisement treatedthe Mercedes as an already-
k n o w n , ' G i v e n ' s y m boof ls t a t u sa, n dt h e m e s s a gteh a t ' y o u ,t o o ,m i g h to w na l v l e r c e d e s '
as the New.More generally, if the left containsa pictureand the right is verbaltext,the
pictureis presented as Given,as a well-established pointof departurefor the text,andthe
text containsthe New.If the left pagehas text and the right pagea picture,the text
The meaning of composition

containsthe Given,and the picturethe New.Theexamplepointsto the socialeffectsand


usesof this structure.what is taken for grantedby one socialgroup is not taken for
grantedby another.We mightexpectto find,therefore, systematic differences in the dis-
positions of materialin layoutacrossdifferentmagazines - for instance, according to their
readershi o.
The conceptsof Givenand New can be appliedalso to the designof diagrams.In
Shannonand Weaver's(1949) communication model(figure2.2) it mightseemthat the
horizontalorder of the elementsis motivatedrepresentationally: the processof'sending
information', for instance, musttake placebeforethe informationcanbe received. But the
left doesnot alwayssignify'before',nor doesthe arrowof time alwayspointto the right.A
diagramfrom a 1990 issueof Time Magazinewhichwe were not allowedto reproduce
hereshowed, on the right,a stickfigurewhoseverylargeheadwasa piechart representing
the composition of the workforcein the year2000 (i.e.ten yearsintothe futureat thetime
of publication). Anotherpie chart, on the left, was superimposed on a massiveoffice
buildingand represented the presentcomposition of the work force.An arrowshowedthat
the stick figurewas walkingtowardsthe door of the massiveofficebuilding,i.e.that a
changein the composition of the work forcewas graduallycomingcloserto the present,
but it was not movingtowardsthe right, becausethe currentcompositionof the work
force had to be treatedas Givenand the future additions(morewomen,minoritiesand
i m m i g r a n t sa)s N e wa n d p r o b l e m a t i cT.h i ss h o w sh o wt h e G i v e n - N e w s t r u c t u r ec a n b e
i d e o l o g i c aelv e ni n d i a g r a m sI .f t h e h o r i z o n t aol r d e ro f t h e c o m m u n i c a t i omno d e lw e r e
r e a r r a n g ei d
na similaw r a y( s e ef i g u r e6 . 5 ) ,i t w o u l dn o l o n g e dr e p i c ct o m m u n i c a t i o f rno m
t h e p o i n to f v i e wo f t h e ' s e n d e r 'w, i t h t h e ' r e c e i v e r ' a sN e w ,a n d p r o b l e m a t i(cW i l l t h e
message'hitthe target'? Wifl it havethe intendedeffect?).Instead,the readerwould
b e c o m teh eo r i g i na n dd e p a r t u r oe f t h ec o m m u n i c a t i op nr o c e s a s /n dt h e ' s e n d e r( ' a u t h o r ' )
problematic, as hasindeedhappened, for instance, in literaryreception theory.
Given-Newstructurescan also be found in film and television. Mediainterviews, for
examplgoften placethe intervieweron the left of the interviewee (from the viewer's
pointof view).Thusinterviewers are presented as peoplewith whoseviewsand assump-
tionsviewerswill identifyandare alreadyf amiliar,indeed, asthe peoplewho askquestions
on behalfof the viewers. The interviewees, on the otherhand,present'New,information -
and are situatedon the right (see Bell and van Leeuwen,r994i 160-4). The relation
betweenGivenand New may be emphasized by horizontalcameramovements ('pans').
In a currentaffairs item from an ABC 7.30 Report(March I9B7), the childrenfrom a
M u s l i mC o m m u n i tSy c h o ow l e r ei n i t i a l l ys h o w na s ' e t h n i c ' , ' d i f f e r e n t ' f r o' ums , ,v i e w e r-s

@ fig e.S Reversed


communication
model
Themeaning of comPosition r85

there was much emphaslson their non-Western dress,and there was Arabic music in
the background. But it wasthe pointof the programme to establish that theywere,despite
' , l a y f u ls, p o n t a n e o ucsr /e a t i v ee/t c .T h i sw a s
c h i l d r e np
t h i s , ' j u s tl i k eo r d i n a r yA u s t r a l i a n
realized,among other things,by varioushorizontalcamerarnovements: a ShotWhich
pannedfrom childrenin non-Western clothesto the teacher, a youngwomanin a Western
d r e s s , t y i nag b o w i n t h e h a i r o f a l i t t l eg i r l ;a s h o t w h i c hp a n n e da l o n ga c l a s s r o o m wall
f r o m a n A r a b i cs i g n t o a p i c t u r eo f a c l o w n ,e t c . I n o t h e rw o r d s , ' d i f f e r e n c ee't,h n i c
prejudice, wastreatedas Given;the fact that at leastthesechildrenshouldbe accepted as
'like us'was treatedas NeWand formedthe message the programmewas trying to get
acroSs.
In ongoingtexts,eachNewcan,in turn, becomeGivenfor the next New.The opening
p a g e so f t h e c h a p t e r ' l ns e a r c ho f a s t r a w ' f r o mt h e D u t c hj u n i o rh i g h - s c h o g oe l ography
textbook Werk aan de Wereld (Bols ef al.,i1986) have, on the far left, a single column of
text, o c c u p y i n ag b o u ta f i f t h o f t h e p a g e ,
w h i c h h a s a ' l a n d s c a p e ' f o r m a f
t a, v o u r i n ga
horizontally o r i e n t e l
d a y o u tT. h et e x tc o n t a i n as s s e r t i o s
n su c ha s ' M a n y p e o p l ei n t h e T h i rd
W o r l dh a v e n o t h i n g ' a n dq u e s t i o n ssu c h a s ' W h a t p r o s p e c tds o a l l t h e s ep e o p l e
h a v e ?'
G i v e ni s , i n t h e f i r s t p l a c e t ,
h e T h i r dW o r l da s a p r o b l e mT . h e r e m a i n d eo rf t h e l e f tp a g e h a s
photo a man sleeplng on the street, covered by a blanket (thereis no
a large colour of
i n d i c a t i ow n h e r et h i s p h o t oi s t a k e n ) T . h i sm o r ee m o t i v ew a y o f p r e s e n t i nt g he'problem'
therefore functions as New in relation to the text. The right page features a single photo-
g r a p hs, h o w i n g a l a r g ec r o w do f p e o p l es e a r c h i nag r u b b i s hd u m p a, r m e dw i t h c a n eb a g s
a n d b a s k e t sI n. r e l a t i o nt o t h i s p h o t ot,h e i m a g eo f t h e h o m e l e sm s a n( a n i m a g en o wa l s o
f a m i l i a ri n E u r o p e b ) e c o m eG s l v e nw, h i l et h e p h o t oi t s e l fw , i t h i t s ( i n N o r t h e r nE u r o p e )
lessfamiliar image of shocking mass poverty, is presented as New.Togetherwith the
introductory text, the two images constitute the Given of the chapter as a whole.Thuseach
new item of information, once received, becomes, in turn, Given for the informationwhich
follows,as shown in figure 6.6. This pattern of the New becoming Given is characteristic of
both in speechand in writing.
a1so,
language

G i v e nI New1

( l n t h e T h i r d W o r l d p e o p l eh a v en o t h i n g ( t h e h o m e l e s si n d i v i d u a l ,
What can be done?) instantiatingand dramatizing
the given)

G i v e n3

O fis O.OCumulative structure


Given-New
186 The meaning of composition

I D E A L A N D R E A L :T H E I N F O R M A T I O NV A L U E O F T O PA N D B O T T O M

Like manyother magazineadvertisements and marketingorientedwebsites(seeMyers,


1994: r39), the Bushellsadvertisement (plate D andthe Sonywebsiteare structured
along the verticalaxis. In suchtexts the uppersectionvisualizesthe \prsmi5se1 th.
product',the statusof glamourit can bestowon its users,or the sensoryfulfilment
it
c a n b r i n g .T h e l o w e rs e c t i o nv i s u a l i z et sh e p r o d u c ti t s e l f p, r o v i d i n m
g o r eo r l e s sf a c t u a l
informationaboutit, andtellingthe readersor userswhereit can be obtained, or howthey
c a n r e q u e sm t o r ei n f o r m a t i oanb o u ti t , o r o r d e ri t . T h e r ei s u s u a l l yl e s sc o n n e c t i o nl e, s s
ongoingmovement, between thetwo partsof the composition than inhorizontally oriented
compositions. Instead,there is a senseof contrast,of oppositionbetweenthe two. The
u p p e rs e c t i o nt e n d st o m a k es o m ek i n do f e m o t i v ea p p e aal n dt o s h o wu s ' w h a tm i g h tb e , ;
t h e l o w e rs e c t i o tne n d st o b e m o r ei n f o r m a t i vaen dp r a c t i c a sl ,h o w i n u g s ' w h a ti s , .A s h a r p
d i v i d i n gl i n em a ys e p a r a tteh et w o ,a l t h o u g ha,t a l e s sc o n s p i c u o ul esv e l , t h e rm e a ya l s ob e
connective elements. In plate2 this is createdby the way the jar of coffeeformsabridge
between the upperandthe lowersectionof the ad,whilein the Sonywebsiteit is createdby
t h ec o l o u rs c h e mw e h i c hu n i t e st h e p a g ea sa w h o l e i:n b o t ht h et o p a n dt h e b o t t o mp a r t o f
t h e p a g et h e d o m i n a nct o l o u r sa r es h a d eos f b e i g ew, i t h s o m eb l u ea n db l u e - g r eeyl e m e n t s
added(thejacketof the girl,the picturesin the bottompart of the page)as welI as some
red elements(e.g.the girl's lips and the words ,what,s new, in the top half, and the
headings of the four sectionsin the bottomhalf). Overall,however, the opposition between
t o p a n db o t t o mi s s t r o n g l ey m p h a s i z ewdi,t h p r o d u c tps l a c e df i r m l yi n t h er e a l mo f t h er e a l ,
as a solidfoundation for the edificeof promise, andwith the top sectionasthe realmof the
c o n s u m e rs' su p p o s eads p i r a t i o nasn dd e s i r e s .
In othercontexts, the opposition between top and bottomtakeson somewhatdifferent
v a l u e si .n a f a i r l yc o n s e r v a t i vbeu t ( i n t h e e a r l y1 9 9 0 s )s t i l lw i d e l yu s e dD u t c hg e o g r a p h y
t e x t b o o k( D r a g te f a l . , 1 9 8 6 ) t, h e u p p e rh a l f o f t h ef i r s t p a g eo f a c h a p t eor n ,a g a i n , . T h e
T h i r dW o r l d ' ,i s f u l l y v e r b a lp, r e s e n t i ngge n e r a l i z eads s e r t i o nasn d d e f i n i t i o nssu c ha s , A
l a r g ep a r t o f t h e w o r l dh a sa l o w d e v e l o p m e n t ' a n d ' T h eusned e r d e v e l o pceodu n t r i ew se
c a l l p o o rc o u n t r i eosr d e v e l o p i ncgo u n t r i e sT' .h i sp r o v i d eas m o r en e u t r aal n dl e s se m o t i v e
( b u t n o t l e s si d e o l o g i c a kl )i n d o f i d e a l i z a t i o na , r e p r e s e n t a t i oonf t h e w o r l d w h i c h i s
divestedof contradictions, exceptions and nuances. The lowerhalf of the page is given
overto a map of the world which usescolour-coding to dividethe world into regions
accordingto the averageincomeof the inhabitants, thus providingspecificand detailed
evidence to supportthe assertionsin the top half. Directionsfor action- for instance,
coupons for orderinga productin advertisements, or assignments or questions in textbooks
- alsotendto befoundon the lowerhalf of the page,
usually at the bottomriqht(hencealso
New).
The informationvalueof top and bottom,then,can perhapsbe summarized alongthe
f o l l o w i n gl i n e sI.f , i n a v i s u a cl o m p o s i t i osno, m eo f t h e c o n s t i t u e netl e m e n tasr e p l a c e di n
the upperpart,and otherdifferentelementsin the lowerpart of the picturespaceor the
page,thenwhat hasbeenplacedon the top is presented as the Ideal,and what hasbeen
placedat the bottomis put forwardas the Real.For something to be idealmeansthat it is
The meaning of comPosition r87

presented as the idealizedor generalized essence of the information,hencealso as its,


ostensibly, most sallentpart. The Realis then opposedto this in that it presentsmore
'down-to-earth' information(e.9.photographs as
specificinformation(e.g.details),more
or mapsor charts)/or more practicalinformation (e'g' practical
documentary evidence,
c o n s e q u e n cdeisr e , c t i o nfso r a c t i o n ) .
As is alreadyevidentfrom the examples givenso far,the opposition betweenIdealand
Realcanalsostructuretext-imagerelations. If the upperpart of a page is occupied by the
text andthe lowerpart by oneor morepictures mapsor chartsor diagrams), text(or the
r ot l e ( w h i c h h, o w e v eirs,
y ,e l e a dr o l e ,a n dt h e p i c t u r e sa s u b s e r v i e n
p l a y s i,d e o l o g i c a ltl h
importantin its ownway,as specification, evidence, practicalconsequence/ and so on)' If
the rolesare reversed/ so that oneor morepicturesoccupythe top section/thenthe ldeal,
t h e i d e o l o g i c a lfloyr e g r o u n d epda r t o f t h e m e s s a g ies,c o m m u n i c a t ev d i s u a l l ya,n dt h et e x t
servesto elaborateon it.
A s w i t h t h e G i v e na n d N e Wt h e l d e a l - R e asl t r u c t u r ec a n b e u s e di n t h e c o m p o s i t i o n
bothof singleimagesand of composite textssuchas layouts.Figure6.7, reproduced from
one of the Dutchgeography textbool<s we havediscussed (Bols e/ al.,1986), includes a
-
p h o t ow h i c hm a yh a v eb e e nt a k e ni n I n d i a i t so r i g i ni s n o tm e n t i o n ebdu, t t o t h e l e f to f t h e
s e ea m a po f i n d i a .A y o u n gm o t h e rc,a r r y i n ga b a b , o c c u p i e s /y
p i c t u r ea, s i t s G i v e n , w e b
h e r s e ltfh, et o p s e c t i o no f t h ev e r t i c a l l cy o m p o s epdh o t oa, s a ' T h i r dW o r l d 'M a d o n n w a ith

*"w*t
ffi*'$*i*'-
#it:kffi;
1
l:J.r'S,-;:.-.

ffi.
ffiffit:

ef ar.,1986)
figO.Z overpopulation(Bols
@
I88 Themeaning of composition

c h i l d .T h e b o t t o ms e c t i o ns h o w sa g r o u po f w o m e na n d c h i l d r e ns,i t t i n go n t h e g r o u n d ,
tightly packedtogether.The young mother looks at this group,a worried expression
crossingherface.In thiswaythe pictureas a wholeexpresses a contradiction between the
deep-rooted Idealof motherhood and the Realof overpopulation. Immediately belowthe
p h o t ow e f i n da c o l l a g eo f n e w s p a p ehre a d l i n e(s' l n d i as t r u g g l easg a i n s t
'Unemployment overpopulation,,
nightmarein India') as Real(the newspaper as sourceof ,hardfacts,,of
evidence) with respectto the moresymbolic,idealized and emotiverepresentation of the
p r o b l e mi n t h e p i c t u r e .
I d e a l a n d R e a lc a n a l s o p l a y a r o l e i n d i a g r a m sI.t i s s t r i k i n g f, o r i n s t a n c et h, a t
diagramsbasedon a verticaltimelinesometimes idealizethe present, sometimes the past.
Thealready-mentioned Dutchgeography textbookWerkaande Wereld(Bolset al.,1986)
featuresa diagramwhichrepresents the decrease of livingspaceper headof the popula-
tion,by meansof a verticalarrangement of what look likechessboards of differentsizes.
0n these'chessboards' stand cartoonfigures.0n top we see a gentlemanfrom 1900,
completewith top hat,on a large('6285 m2')'chessboard'. At the bottom,on the smallest
'chessboard',
w e s e ea ' p u n k ' c h a r a c t efrr o m 1 9 g 0 . H e r e ,a s i n m a n ya d v e r t i s e m e n t s ,
the past,the 'goodold days',is presented as Ideal.The other Dutchgeography textbook
we mentioned(Dragt ef al., 1986) featuresa 'geologicalcalendar,in whichthe present
('development of vertebrates', completewith a smalldrawingof a nakedwoman)becomes
t h e I d e a l t, h ec u l m i n a t i oonf p r o g r e sasn de v o l u t i o n .
M a n yv i s u a l sc o m b i n eh o r i z o n t aal n d v e r t i c a sl t r u c t u r i n gI n. f i g u r e6 . g ( a s i n f i g u r e
6 ' 4 ) G o di s G i v e na, n dA d a ma n d E v ea r e N e w .B u t t h e i rf a l l f r o m g r a c eh a si n t r o d u c e d
a ( New)opposition, between the Idealof Paradise, of the Gardenof Eden,andthe Realof
deathand decay- and the two are visuallyseparatedby the riverwhich surrounds
the
G a r d e no f E d e n .
T h ec o m m u n i c a t i omno d e li n f i g u r e6 . 9 a l s oc o m b i n ehs o r i z o n t aal n d v e r t i c a sl t r u c -
t u r i n g ,i n a n i n t r i c a t ep i e c eo f v i s u atlh i n k i n ga b o u t h e i m p o s s i b i l iot yf k n o w i n gr e a l i t y ' a s
it is',objectively. Givenis the 'event',as it exists'out there,,separate from our perception
of it. New,andthereforeproblematic, is our perception of the eventand,at the lowerlevel,
t h e w a y w e c o m m u n i c a toeu r p e r c e p t i o nt hsr o u g hl a n g u a g eI d. e a li s t h e ' e m p i r i c a l , ,
the
w o r l d ' a si t i s ' ,a n do u r p e r c e p t i oonf i t , u n m e d i a t ebdy c o m m u n i c a t i ocnu,l t u r el,a n g u a g e
(whichare positioned in the lowersection).Realare our interpretations of thesepercep-
t i o n s ,a s m e d i a t e d b y c o m m u n i c a t i oCn l.e a r l yt ,h i s d i a g r a mc o u l dh a v eb e e nv e r t i c a lo, r
h o r i z o n t aB l . u t i t i s n o t .C o m m u n i c a t i o i snp o s i t i o n e bd e l o w t h e , e v e natn, d i t s p e r c e p t i o n .
T h e ' e m p i r i c a l ' w o r ladn d ' p u r e ' o b s e r v a t i o anr e I d e a l .B u t t h i s I d e a li s a l s od e p i c t e di n
isolationfrom our'statements'about i! and our perceptions of thesestatements. This is
whatthe lowersectionof the diagram,the Real,tellsus.Perception is secondhand, filtered
throughcultureand language, which,as the double-headed arrowsindicate,feed back
intoour perception of nature,and henceinto natureitself.Thediagramtellsusthat reality
doesexist,but that our perception of it can only be 'subjective, selective, variableand
u n p r e d i c t a b l(eM' c Q u a ial n dW i n d a h l1, 9 9 3 :2 5 ) .
F i g u r e6 ' 1 0 i s a n o t h e ro n e o f o u r o r i g i n a le x a m p l e fsr o m l a t e t 9 8 0 s A u s t r a l i a n
women'smagazines, but it remainsa good exampleof the combinationof horizontal
The meaning of composition ' I89

O fis t.A Gott Shows Deathto Adamand Eye (French,titteenlh-century miniature from ms' ol De civitate ,eD (from Hughes'
r969)

'marriagewas madein
and verticalstructuring.Ideal is the moment,onemightsay,that
HeavenM ' . o d a l i t yi s ' d i s t a n t ' r, e p r e s e n t i nt hge ' n o t n o w ' ,t h e ' o u t o f t i m e ' .T h eb o t t o m
s e c t i o nb,y c o n t r a s tr,e p r e s e ntthsew o r l do f i s ' , ' n o w ' , ' i no u rt i m e ' .
Givenis the royalcouple,presented as the quintessential couple,the well-established
symbolof family values.New are Sydney'sGwenand Ray l(inkade,an instance,an
e x a m p l eo f t h e s ev a l u e sH . e n c ew h a t i s G i v e ni s t h e p r e - e m i n e n c( hei s t o r i c a l lsyo, c i a l l y ,
s e m i o t i c a l l yo)f t h e r o y a lc o u p l ea st h e p a r a d i g m example o f t h e m a r r i e dc o u p l eW . h a ti s
-
Newis oneinstanceof the paradigm wheremanyothersof a setof acceptable instances
w o u l dh a v es e r v e d equallw y e l l .T h i sd i s t i n c t i oins p e r h a pssh a r p e si nt t h e b o t t o mp a r t .T h e
two picturesin thetop part become, at the safedistanceof forty years,almostidentical, an
eouationbetween eoualterms.
In one,and a very real sense, the placeof the Newseemsmerelyperfunctory: it is the
placeof the replicationof the paradigm,of the reproduction of the existing classifications
o f t h e c u l t u r e , t h e p l a c e w h e r e t h e u n d e r l y i n g vt hael uceusl toufr e a r e r e a f f i r m e d .NTehwe
190 The meaning of composition

meansandcontrol
(or communicating)
dim!nsion

lQ fiq O.e Gerbner,scommunicationmodel (Watsonand Hilt, t9g0: 7i)

instantiatesand 'naturalizes'these values.But that very fact also makesthe position


problematic/ for it is at the sametime the placeof the affirmationof what is,the placeof
the reproduction of socialmeanings, andthe placewherethe contestation of paradigmatic
valuescan take place,the placethereforeof the constantproductionof socialmeanings
(e.9o . f n e wd e f i n i t i o nosf ' w o m e n ' w
s ork'in
f i g u r e6 . 2 ) , e v e n
w h e nt h a t p r o d u c t i osne e m s
to be merereproduction and henceconservative in its effects.Couldthere,for instance,
havebeena Vietnamese or Lebanese or Aboriginalcouplein this positionOn I9B7),notto
m e n t i o na g a yc o u p l e ?T h i sc o n t e s t a t i oonv e r , e s t a b l i s h e d , , , G i v a
e lnu,e sm a y h a p p e ni n
one0r two ways:a readerwho is not Anglo-Australian will eitheridentifywith the syntagm
of Anglo-Australianness,'assimilate', in otherwords;or will refusethe syntagmas having
no relevance or valueto him or her.In the lattercasetherewill be pressure on this placein
t h es y n t a g ma,n dt h i s i n t u r nw i l l r e s u l ti n p r e s s u roen t h e p a r a d i g ma sa w h o l e .
Thereis anotheraspectto this; while the syntagmdeclaresitself as unquestionably
established, its appearance pointsat the sametime to a problemwith the paradigm, to the
needpreciselyfor a testing and ke)affirmationof its legitimacy.Readfrom the right to the
left,the syntagmdeclaresthat it is the willingness of readersto read it as a relationof
identity(withina hyponymic structure)whichgiveslegitimacy to the royalcouple.Royalty
is the established, the Given.What has to be reaffirmedanew is that subjectsare still
preparedto enter into this paradigmaticrelation.A monarchytrying to establishitself,
on the other hand,might needto utilize a structurewherethe powerof the peopleis
represented as Given,and the identityof the monarchis to be established - that is.the
r o y a lc o u p l ew o u l da p p e a or n t h e r i g h t .
Themeaning of composition 191

'r/{;rgugry
${}
, loesft&rX!,1t{ti

I Prihoe* eiirrbeit srd


I Klip Marntb'ln
I (*ltd$idg4e{hand'
lsm qd.e* HadE&
sk
ttrifl.#{tys#,y'lf
rJ
'f
h*i'r* f relr dilr'.r$if
srrkj*. hut tl* lorg a*l rt*ngtlr ir
rlif,if nrfila$es d* !l!: lafia

rbt! n6lh4.hrod aMn! @e.


r h d ) o ! . !n g l D . dr t u n r d
Pd.ce Phllo. ond Srdq'3 Gsn
6d FryrtudcSdo d dt
,oddln! sn.lvaBry - rd lolt
iod !o(t .n 40 ts6 oltupll|*

@ fiq e.fO Royal couple(Australian Women,sWeekty,Nouember


1987)

Thusthis syntagmrevealsa numberof socialfacts:what is regardedas established


and Given;what the culturalclassification systemis with respectto a certainfeature,'and
whetherthe systemis progressive or reactionary. It is aboveall a syntagmwhichdoesnot
permitdeviance; or,rather, oncean item is in the syntagm,it hasto be readas beingin the
paradigm. Whereit doespermitdeviance is on the part of the reader,who can refuseto be
p a r t o f t h ec o m m u n i tdye f i n e d
b y t h i sp a r a d i g m .
In the Westernvisualsemiotic,then,the syntagmaticis the realm of the processof
semiosis, and the top-bottomstructurethe resultand recordof semiosis, the realm of
192 The meaning of composition

o r d e rt,h e p a r a d i g mt h , e m i m e t i cr e p r e s e n t a t i o n f c u l t u r e( H o d g ea n d l < r e s s1,9 g B ) .T o


maintainand unsettletop-bottomstructures, onehasto work on the left-rightstructures.
That this systemgoesback a long way in westernart can be seenin genressuchas
fifteenth-century Flemishdiptychs, which,for instance, may havethe Virginandthe child
as Given,and a donoror Saintas New,as in the diptychby the masterof Brugesin the
c o u r t a u l dG a l l e r ya, n d p o l y p t y c hfsr o m t h e s a m ep e r i o dw , h i c h m a y p a r a l l e la R e a l
( e a r t h l ya) n dI d e a l( h e a v e n l yv )e r s i o o n f t h es a m et h e m ei n t h e l o w e ra n du p p e rp a r to f t h e
panels, as in Bosch'sLastJudgement,where the lowerpart of oneof the left panelsshows
A d a ma n d E v eb e i n gd r i v e nf r o mt h e G a r d e no f E d e na n dt h e u p p e rp a r t t h e e x p u l s i oonf
t h e R e b eAl n g e l sf r o m h e a v e n .
As we havesaid in the Introduction, we are largelyconcerned with the description of
the visualsemioticof Westerncultures.Cultureswhich have long-established reading
directionsof a differentkind (rightto left,or top to bottom)are likelyto attachdifferent
valuesto thesepositions, as shownin figure6.3.In otherwords,readingdirectionsmay
b e t h e m a t e r i a il n s t a n t i a t i o nosf d e e p l ye m b e d d ecdu l t u r a vl a l u es y s t e m sD. i r e c t i o n a l i t y
as such,however, is a semioticresourcein all cultures.All cultureswork with margin
andcentre,left and right,top andbottom,evenif theydo not alI accordthe samemeanings
a n d v a l u e st o t h e s es p a t i a ld i m e n s i o nA s .n d t h e w a y t h e y u s et h e m i n t h e i r s i g n i f y i n g
s y s t e mw s i l l h a v er e l a t i o n so f h o m o l o g w y i t h o t h e rc u l t u r a ls y s t e m sw, h e t h e r e l i g i o u s ,
p h i l o s o p h i coarl p r a c t i c a l .
Wewillendthissectionwithonefurtherexam t hpel u
e soef s oGf i v e n a n d N e w , t h e w a y
in which Rembrandt usedGivenand Newfor the expression of affectiveaspectsof mean-
i n g ,a n d t h i s e s p e c i a l liyn r e l a t i o nt o t h e s o u r c ea n d d i r e c t i o no f l i g h t a n d t h e e f f e c t
producedby that. In many,perhapsthe majority,of Rembrandt,s paintings,whetherin
landscapessuch as Landscapewith a stone Bridge or in portraits such as A young
Womanin Bed or Double Portrait of the MennonitePreacherCorneliusClaeszAnslo and
his WifeAeltie GerritsdrSchouten(figure6.71),the light sourceis outsidethe left frame
o f t h e p i c t u r ea n d i l l u m i n a t em s a i n l yt h e l e f t p a r t , l e a v i n gt h e r i g h t o f t h e p a i n t i n gi n
greateror lesserdarkness. Iconographically speaking the metaphoric rangeof lightis wide
- l i g h tc a ns i g n i f y ' t h ed i v i n e ' , ' i l l u m i n a t i o n ' , ' h o p e ' ,
e t c .I n t h e s ep a i n t i n glsi g h t w
, hatever
its meaning,is in the areaof the Given,the takenfor granted,thenow/present.,Light, is
G i v e n , ' d a r k n e sNse' w .
Theheightof the unseenlightsourcealsovaries:in YoungWomanit is on or just below
the centre;in DoublePortrait it comesfrom a positionabovethe halfwaymark,perhaps
two-thirdsof the way up;and in Landscape it comesfrom somewhere highup,nearthe top
cornea r n di n t h e n e a rm i d d l ed i s t a n c eT.h a ti s ,l i g h tm a yb e i n t h ea r e ao f t h e R e a cl o m i n g
f r o m a ' m u n d a n e ' s o u r coer i t m a y b e ' d i v i n e ,I.n o t h e rp a i n t i n gtsh e l i g h tc o m e sf r o m
withinthe painting;for instance, in TheHolyFamilyon the Flight to Egypt,where it forms
t h e ( d i v i n e l)i g h t ' i nt h e w o r l d ' ( t h e r ei s a l s oa s e c o n df a , i n t l i g h tc o m i n gf r o m o u t s i d ei n
,
the sky above).In Belshazzar's Feast,by a most unusualcontrast,thelight source(the
g l o w i n gs c r i p ta n n o u n c i nt hg e d o o mo f t h e k i n g )i s s i t u a t e di n t h et o p - r i g hqt u a d r a n-t
the
spaceof the New and the'Ideal?divine'.The variationin the sourceand directionality
of lightthus hasa complexset of meanings. It can contrastthe secular/mundane andthe
The meaning of composition 193

O fis O.ff Douhle Portmit of the Mennonite Preachet Cornelius Claesz Anslo and his Wile Aeltje Gerritsdr Schouten
(Rembrandt,
1641)(StaatlicheMuseen,Preussischer
Kulturbesitz,
Gemaldegalerie,cat.no.828L)

divine/ideal; lightas Givenandtakenfor granted, and lightas Newandastonishing: andall


thesein variablecombinations. In DoublePortrait,for example,the light comesfrom
outsidethe depictedworld,is situatedin the areaof the Given(the areawherethe scrip-
turesare depicted), andcomesfrom just abovethe midwaypointbetweenIdealand Real,
s ot h a t i t c o u l db e i n t e r p r e t eads ' d i v i n e y' ,e tc l o s et o t h e R e a l0. n eo v e r w h e l m i n
egf f e c ti s
the brightness of the areaof the Given,andthe total darkness of the areaof the New(the
future?)to whichthe two figureshave,in any case,turnedtheir backs.Are we entitledto
readfrom this autobiographical, affectivemeanings - perhapsa deep,pervasive pessimism
about both the future,the New,and the present,the Real,whichthen contrastswith a
feelingof securityaboutwhat was,a faith in a divinelightfrom the pastcertainty, which
entailsthat we must turn our back on the New,on the future? If so,theseaffective,
personalmeanings are surelyas significant as socialandculturalmeanings and,of course/
relatedto them.
L94 The meaning of composition

T H E I N F O R M A T I O NV A L U E O F C E N T R EA N D M A R G I N

V i s u acl o m p o s i t i omn a ya l s ob es t r u c t u r eadl o n gt h ed i m e n s i o nosf c e n t r ea n dm a r g i nT. h e


mosttypicalmanifestations of this can be foundin children'sdrawingsor,for example, in
B y z a n t i naer t . A s A r n h e i m( 1 9 8 2 : 7 3 )n o t e s ,

I n t h e B y z a n t i nceh u r c h etsh ed o m i n a nitm a g eo f t h ed i v i n er u l e rh o l d st h ec e n t r eo {


the apse.In portrait paintings,a pope or emperoris often presentedin central
p o s i t i o nM
. o r eg e n e r a l l w
y ,h e nt h e p o r t r a i to f a m a ns h o w sh i m i n t h e m i d d l eo f a
framedareatwe seehim detachedfrom the vicissitudes of his life's history,alone
with his own beingand his own thoughts.A senseof permanence goeswith the
centralposition.

F i g u r e6 . I 2 i s a n e x a m p l -e a B u d d h i spt a i n t i n gi n w h i c ht h e c e n t r afli g u r ei s s u r r o u n d e d
b y a c i r c l eo f s u b o r d i n a t eAsr.n h e i mi n f a c t m a k e st h e c e n t r et h e c r u c i a e l l e m e not f h i s
theoryof composition, conceiving of the visualobjectsin a composition as'somanycosmic
bodiea s ttracting a n dr e p e l l i nogn ea n o t h e irn s p a c eQ ' 9B2:207).
I n c o n t e m p o r a rW y e s t e r nv i s u a l i z a t i ocne n t r a lc o m p o s i t i oins r e l a t i v e l u
yncommon,
thoughheretoo there may be changesin train. Most compositions polarizeelementsas

O fiq O.fZ Burldhistpainting (Arnheim,1982)


Themeaningof composition 195

G i v e na n d N e wa n d / o rI d e a la n d R e a l B . u tw h e no n eo f u sw a st e a c h i n o g n a m e d i ad e s i g n
coursein Singapore, he foundthat centralcomposition playedan importantrole in the
imaginationof youngAsian designers. Perhapsit is the greateremphasison hierarchy,
h a r m o n ya n d c o n t i n u i t yi n C o n f u c i a nt h i n l < i n gt h a t m a k e sc e n t r i n ga f u n d a m e n t a l
o r g a n i z a t i o nparli n c i p l e
i n t h ev i s u asl e m i o t i o
c f t h e i rc u l t u r eM. u c ho f t h ew o r kp r o d u c e d
by these studentshad strong dominantcentres,surroundedor ffankedby relatively
u n p o l a r i z emda r g i n ael l e m e n t s .
While many Anglo-Western tabloid newspapers tend to adhereto a basicleft-right
structurein the layoutof their front pages,othersplacethe mainstoriesand photographs
in the top section.Thefront pagesof the business sectionsof the SydneyMorning Herald,
however, for a time invariablyusedcentral composition/ featuringa large photo (or,
frequently, drawing)in the centreof the page:for instance, Asianstudentsenteringthe
n e o - G o t h iQ c u a d r a n g loef t h e U n i v e r s i toyf S y d n e yw h e nt h e p a g ef e a t u r e da r t i c l e so n
education as a moneyearnerfor the country'seconomy; a cartoon-lil<e drawingof two men
playingMonopoly(basedon Van Doesburg's Card-players-seefigure 5.6), when cor-
poratetakeovers dominated the news;and so on.Suchpicturesprovideda symbolickernel
for the issues of the day,anda centrefor the elements arrangedaroundthem- newsstories
at the top and to the left,as,still,the ldeal and the Givenof the newspaper, evenif now
somewhatmarginalized; advertisements as the Real;anda columnof expertcommentary
as New,henceasthe elementto whichreadersshouldpayparticularattention.Figure6.13

Q fig e.ff Goingon holiday(from Prosser,2000:


ll7)
196 The meaning of composition

s h o w sa d i a g r a mf r o m a t o u r i s ms t u d i e tse x t b o o ki n w h i c h' g o i n go n h o l i d a yi's t h e c o r e


issue, and in whicha rangeof reasons for goingon holidayis arrangedaroundthis Centre,
withoutanysenseof polarization.
To generalize, then,if a visualcomposition makessignificantuseof the Centre,placing
oneelementin the middleand the otherelementsaroundit. we will referto the central
elementas Centreandto the elements aroundit as Margins.Forsomething to bepresented
as Centremeansthat it is presented as the nucleusof the informationto which all the
otherelements are in somesensesubservient. The Marginsare theseancillary,dependent
elements. In manycasesthe Marginsare identicalor at leastverysimilarto eachother,so
that thereis no senseof a divisionbetweenGivenand Newand/orIdealand Realelements
amongthem.In othercases- for instance, the newspaper pageswediscussed above- Centre
a n d M a r g i nc o m b i n w e i t h G i v e na n d N e wa n d / o rI d e a la n d R e a l .
N o t a l l M a r g i n sa r e e q u a l l ym a r g i n a lC . i r c u l a rs t r u c t u r ecsa n c r e a t ea g r a d u a a l nd
gradeddistinctionbetween Centreand Margin,asfor instancein thecommunication model
by Anderschet al. in figure6.I4, wherethe process of 'structuring'is moremarginalthan
t h e p r o c e sosf ' e v a l u a t i n gI 'n. t h i s m o d e ln, a t u r e( t h e ' e n v i r o n m e n its' )C e n t r eo, r i g i na n d
p r i m em o v e o r f c o m m u n i c a t i oCno. m p a r etdo t h ed o m i n a npt o s i t i o o n f n a t u r ec, o m m u n i c a -
tion is a marginalphenomenon, just as, in the medievalmapsof citieswe discussed in
chapter3, the citiesthemselves wereplacedin the centreand depictedwith topographical

-i't'i\

\**)W,'*"91

@ figO.f+ Anderscheta/.'scommunicationmodel(fronWatsonandHill,lgS0:14)
The meaning of composition 197

accuracw , h i l et h e s u r r o u n d i ncgo u n t r y s i dwea s r e p r e s e n t eodn a s m a l l e sr c a l ea, n dw i t h


lessaccuracy. Verbalcommentarles do not necessarily try to 'translate'suchmeanings.
W a t s o n a nH d ill ( 1 9 8 0 : 7 6 ) , f o r
e x a m p l e , s a y t h a t i n t h i s m ot hdee'lm e s s a g e ' i s ' i n t e r a c t -
ing with factorsin the environment'. Yet,the modelitselfrepresents the relationbetween
c o m m u n i c a t i oa n dt h e ' e n v i r o n m e nnt o ' / ta s i n t e r a c t i o b
nu, ta sa o n e - w apy r o c e s a s /' n o n -
transactivereaction',accordingto our terminologyin chapter2 (thereis an arrow only
f r o mt h e ' e n v i r o n m e n t 'tthoec o m m u n i c a t i p ve rocesse t hsa t s u r r o u n idt ) .A n d' i n t e r a c t i n g '
s u g g e s tgsr e a t eer q u a l i t yb e t w e etnh e ' m e s s a g e ' a nt hde ' f a c t o r si n t h e e n v i r o n m e n t ' t h a n
d o e st h e c e n t r e d c o m p o s i t i oonf t h e m o d e l .
A s w e h a v es e e n G , iven-New a n d I d e a l - R e acl a n c o m b i n ew i t h C e n t r ea n d M a r g i n .
Dividing v i s u a ls p a c ea c c o r d i n tgo t h e s ed i m e n s i o nr se s u l t si n t h e f i g u r eo f t h e C r o s sa,
fundamentalspatialsymbolin Westernculture(seefigure6.15).Just how marginalthe
marginsare will dependon the sizeand,moregenerally, on the salience of the Centre.But
evenwhenthe Centreis empty,it continues to existin absentia, as the invisible(denied)
p i v o t a r o u n dw h i c he v e r y t h i n eg l s et u r n s ,t h e p l a c eo f t h e ' d i v i n er u l e r ' .T h e r e l a t i v e
infrequency of centredcompositions in contemporaryWesternrepresentation perhaps
signifies that,in thewordsof the poet,'thecentredoesnot hold'anylongerin manysectors
of contemporary society.
O n ec o m m o nm o d eo f c o m b i n i nG g i v e na n dN e ww l t h C e n t r ea n dM a r g i ni s t h et r i p t y c h .
In manymedievaltriptychsthere is no senseof Givenand New.The Centreshowsa key
r e l i g i o utsh e m es, u c ha s t h e C r u c i f i x i oonr t h e V i r g i na n d C h i l da, n dt h e s i d ep a n e l s h o w
S a i n t so r d o n o r s k, n e e l i n g d o w n i n a d m i r a t i o nT. h e c o m p o s i t i oins s y m m e t r i c ar la t h e r
t h a n p o l a r i z e da,l t h o u g ht h e l e f t w a sr e g a r d e ad s a s l i g h t l yl e s sh o n o r i f i cp o s i t i o nI .n t h e
sixteenthcenturyaltarpieces becomemore narrative,showing, for instance, the birth of
Christor the roadto Golgothain the left panel,the Crucifixion on the centrepanel,andthe
R e s u r r e c t i o n t h e r i g h tp a n e lT. h i sc o u l di n v o l v e s o m ep o l a r i z a t i o a n l,b e i ts u b o r d i n a t e d
to the temporalorder,with the left as the 'bad side'(e.9.the transgression of Adam),the

Q rig O.fs The dimensions


otvisualspace
198 Themeaning of composition

r i g h ta s t h e ' g o o ds i d e '( e . 9 t. h e R e s u r r e c t i oann) dt h e m i d d l ep a n e rl e p r e s e n t i nCgh r i s t ' s


roleas Mediatorand Saviour(e.9.the Crucifixion). Bosch'sLastJudgement(e.ndalsohis
EarthlyDelightl invertsthis,showingon the left the Gardenof Edenand on the right a
c a t a c l y s mvi ci s i o no f H e l li n w h i c ht h e r ei s n o p l a c ef o r t h e ' a s c e not f t h e b l e s s e d ' .
Thetriptychsin modernmagazines andnewspaper layoutsare generally polarized, with
a ' G i v e n 'l e f t ,a ' N e w ' r i g h t ,a n da c e n t r ew h i c hb r i d g etsh et w o a n da c t sa s ' M e d i a t o r 'I.n
August2004,thetop bannerof Nokia'swebsiteshowedon the left an imageof a fashion-
a b l ew o m a na n d o n t h e r i g h ta N o k i ai m a g i n gp h o n eT. h et e x t i n t h e C e n t r ec o n n e c t e d
the two. In fact it consisted of two alternatingtexts.First we read'lnspiringly. Welcome
to LondonFashionWeek',thenthis first text madeway for 'lnspiringlydifferent.Thenew
officialimagingphoneof LondonFashionWeek'.Theconceptof fashionis Given,andthe
N o k i ai m a g i n g p h o n ea s a f a s h i o na c c e s s o rNye w .
T r i p t y c h cs a n a l s o b e u s e dt o s t r u c t u r ed i a g r a m sI .e d e m ae t a l . ( 1 9 9 4 : 2 1 7 ) s h o w s
h o w t h e l e f t c o l u m no f t h e o r g a n i z a t i o n cahl a r t o f a l o c a lc o u n c i l i s t st h a t c o u n c i l ' s
'CorporateServices',so making the 'administrativeand financial backboneof the
organization'Given,while the council's'Development and Environmental Services', the
department which'connects all otherdepartments', is described in the centralcolumn,as
the Mediator.In a lectureon socialcognitionattendedby the authors,the lecturerused
t h e b l a c k b o a r(dc o n v e n t i o nballa c k b o a r dasl s oh a v ea t r i p t y c hs t r u c t u r e lt)o l i s t ,o n t h e
l e f t p a n e l a, n u m b e ro f k e y i s s u e si n l i n g u i s t i c(st h i sw a s G i v e nb e c a u sm e o s to f t h o s e
p r e s e nw t e r el i n g u i s t a s n ds t u d e n t o , t h e r i g h t p a n e la n u m b e r
s f a p p l i e dl i n g u i s t i c so) n
o f i s s u e si n s o c i o l o g(yt h i sw a s N e w ,a s t h e l i n g u i s tw s e r em e e t i n gt o d i s c u stsh e s o c i a l
r e l e v a n coef d i s c o u r saen a l y s i sa) ,n do n t h e c e n t r a l ' p a n e l ' aonu t l i n eo f h i so w nt h e o r yo f
socialcognition, whichhe presented as the necessary link between the two fields,andas an
issuethat shouldbethe centralconcernof thosepresentat the meeting.
Verticaltriptychsare also commonin websites. The triptych from the Universityof
0xfordwebsite(figure6.16) canbe interpreted as a simpleMargin-Centre-Margin struc-
t u r e ,t h o u g ht h e r ei s s o m ep o l a r i z a t i oi n t h a tt h et o p i m a g ei s a ' l o n gs h o t 'a n dt h eb o t t o m
i m a g ea c l o s eu p . 0 v e r a l h l , o w e v etrh, e s t u d e nits C e n t r eh e r ew , h i l ei m a g e os f h i s t o r ya n d
t r a d i t i o ns u r r o u nadn ds u p p o rht e rT . h et r i p t y c hi n f i g u r e6 . 1 7c o m e sf r o ma G e r m a jnu n i o r
high-schoo p lo l i t i c st e x t b o o k( N i t z s c h k e1,9 9 0 ) .A s t h e I d e a l ,w e s e e( i n c o l o u r )i m m i -
grants(Ausltinder,'foreigners') in high-status professions. As the Real,we see'foreigners'
in low-status professions. This Realis dividedinto a Givenanda New,with a colourphoto
as Givenand a black-and-white photoas New,as though,in the 1990s,the low statusof
i m m i g r a n tssh o u l db e l o o k e da t i n a m o r es o b e rl i g h t ,a n d n o l o n g e ra s ' G i v e n ' a si t o n c e
was.In the Centrewe see,againin blackand white,a singleimmigrantworkercleaninga
train. The accompanying text encourages studentsto explorewhat wouldhappenif 'one
day all foreignworkershad to leave'.What, it asks,would be the consequences for the
buildingindustry, the childrenof theworkers, the ownersof hostels, theworkersthemselves,
t h e m a n a g e rosf h o s p i t a lasn dc l e a n i nfgi r m s ?I n o t h e rw o r d s , t h itsr i p t y c h( i t s e l tf h e N e w
o n t h e d o u b l ep a g ei n w h i c hi t a p p e a r st)e l l s u s t h a t f o r e i g nw o r k e r ss h o u l dp, e r h a p s ,
i d e a l l yb e a b l et o m o v ei n t o h i g h - s t a t upso s i t i o n sb,u t i n r e a l i t ya r e n e e d e d to do'our'
menialjobs. The central image is an attempt to overcome, or at least mitigate,this
The meaning of composition 199

@ fig O.fO Verticaltriptych from the Universityof 0xford website(www.oxford.ac.uk)

c o n t r a d i c t i o nI t. s h o w sa w o r k e rw h o , l i k e t h e h i g h - s t a t uism m i g r a n t isn t h e l d e a l ,i s
depicted a s a n i n d i v i d u aal ,n d a s i n v o l v e idn ' c l e a n ' w o r k b , u t w h o ,a l s ol i k et h e w o r k e r s
shownin the Real,hasa low-status job * and is shownin the sober, documentary modality
of blacl<-and-white realism.
The structureof the triptych,then,can be eithera simp[eand symmetricalMargin-
C e n t r e - M a r g isnt r u c t u r eo r a p o l a r i z esdt r u c t u r ei n w h i c ht h e C e n t r ea c t sa s a M e d i a t o r
b e t w e eG n i v e na n d N e wo r b e t w e eInd e a la n d R e a l( s e ef i g u r e6 . 1 8 ) .
In this and the precedingsectionof this chapter,we havenot drawn any parallels
w i t h l a n g u a g eT.h o u g hs p o l < eEn n g l i s hh a s i t s o w n G i v e n - N e w s t r u c t u r et ,h i s i s n o t t h e
casewith the ldeal*Realand the Centre-Marginstructures.This is not to say that
the meanings thesestructuresexpresscannot,in someform, be expressed in language,
but rather that they are more readily and frequentlyexpressedvisually,and that
language u ,n l i k ev i s u a cl o m m u n i c a t i ohna, sn o t d e v e l o p e d ' g r a m m a t i c a l ' f otromesx p r e s s
t h e m .A s w e h a v ee m p h a s i z et hdr o u g h o ut th i s b o o k s, o m e t i m el sa n g u a gaen dv i s u acl o m -
municationexpressthe same kind of semanticrelations,albeit in very differentways,
but there are also many types of semanticrelationwhich are more often and more
easilyexpressed visually,just as thereare otherswhich are more often and moreeasily
e x p r e s s el idn g u i s t i c a lw l yi,t h e p i s t e m o l o g i ccaoln s e q u e n coef st h e k i n dw e d i s c u s s ei ndt h e
i n t r o d u c t i oann dc h a p t e1r .
200 The meaning of composition

{ wt"ui"l" xinder ausliin-


dircher [itern lind in eufer
Sc"r,le?
Fragt sie,seit wann ihre Elternin
Deutschland5ind,was sie arbea-
ten warum siehierhdrkamen,ob
sie bleibenwdhn und warum.

Ste$tim Atlas i:st woher ausl{n-


discheArteiter komrnen,rioher
Fliidrdinge.taBt euch enihlert
wie es dort aussieht"Lest in E d-
kundebiichemoder Reiseliihrern
rach. Achtet dabei besonder auf
die Arbeits- und Lebenssit'ratisn.

If n grr*a,iut eurer Sdlule


gibt es Zahleoiiber ausliindis<ire
fthtiler

Vorschlligezum Cesprach:
. Cibt es Problememit ausliin-
dischenSchiilerinnenund Schii-
lern{
o Welche Ldsungsmdglidrkeiten
seht ihr?
o W.s denkenausliindische
Schiler tiber die Eundesrepublik?
o Was denkendeutscheSchiiler
iiber Auslinder?

/l stettt euclr vor, da8 an


einsm Tag!alle ausliindisclcn
Arbeiter die Fuodesrepublikver-
lassenmii8ten. Eildet Gruppen
und iiberlegt was e.B.
o Bauunternehmer,
o Kinderder Arbeiter.
o caststiittenbesitzer,
o die Arbeiter selbst
o Leiter von l*artl<enhiius!m,
o Leitervon Reinigungsfrnenfiir
ProHerneUithn, trn miitlten ...
Tragt die lrgebdsse zusanrnen
und besprerl* sle.

f) fiq e.fZ Verticaltriptychfrom a Germanschooltextbook(Nitzschke,


1990)
The meaning of compositton 207

Given Mediatof

@ fiq e.fe Horizontaland verticaltriptychs

SALIENCE

Thefundamental functionof integration codessuchas composition is textual.Integration


codesserveto producetext,to placethe meaningful elements intothewhole,andto provide
coherence and orderingamongthem.So far we havediscussed how composition deter-
m i n e s' w h e r et h i n g sc a n g o ' a n d h o w t h e p o s i t i o n i nogf t h e e l e m e n t isn a c o m p o s i t i o n
endowstheseelementswith differentinformationvaluesin relationto other elements.
But the composition of a pictureor a pagealso involvesdifferentdegreesof salienceto
its elements.Regardless of wherethey are placed,saliencecan createa hierarchyof
importanceamongthe elements,selectingsome as more important,more worthy of
attentionthan others.The Givenmay be more salientthan the New,for instance, or the
N e wm o r es a l i e ntth a nt h e G i v e no, r b o t hm a yb e e q u a l l ys a l i e n tA. n dt h e s a m ea p p l i e tso
I d e a la n d R e a a l n dt o C e n t r ea n d M a r g i n .
Thesamephenomenon occursin temporallyintegrated texts.Rhythmalwaysinvolves
cycleswhichconsistof an alternationbetweensuccessive sensations of salience(stressed
s y l l a b i e sa,c c e n t e nd o t e se, t c . )a n d n o n - s a l i e n c e( u n s t r e s s e d
s y l l a b l e su,n a c c e n t endo t e s )
andthesecyclesrepeatthemselves with the time intervalsthat are perceived as equaleven
when,measured objectively,they are not.Theperception of salience, in speechas in music,
resultsfrom a complexinterplaybetween a numberof auditoryfactors:the durationof the
s t r o n ga n dw e a ke l e m e n tosf t h ec y c l e( ' l o n g ' - ' s h o r t ' ) , t hpei t c ho f t h es t r o n ga n dt h ew e a k
e l e m e n t(s' h i g h ' - ' l o w 't)h e i r l o u d n e s(s' l o u d ' - ' s o f t ' )a,n d i n s p e e c a h l s ot h e v o w e cl o l o u r
(vowelsmay be fully pronounced, for instancethe first'e' in element, or pronounced as a
' s c h w a 'l,i k et h es e c o n d ' e ' i ne l e m e not ,r t h es e c o n d ' a ' i na l a b a s t e rI)n. d e e da n y t h i ntgh a t
can createan auditorycontrastbetweensuccessive soundscan serveto realizesalience.
And evenwhenobjectivecluesfor salienceare absent,the first elementof eachcyclecan
b e p e r c e i v eads ' s t r o n g e rp' :e r c e p t i oi n
m p o s ersh y t h mw , a v e so f s a l i e n caen dn o n - s a l i e n c e
o n s o u n d( a n do n m o v e m e net )v e nw h e n s, t r i c t l ys p e a k i n g t h, e r ei s n o n e .
202 The meaning of composition

W h e nc o m p o s i t i oins t h e i n t e g r a t i om n o d e s, a l i e n c e i s j u d g e do n t h e b a s i so f v i s u a l
c l u e sT. h ev i e w e r o s f s p a t i acl o m p o s i t i o nasr e i n t u i t i v e layb l et o j u d g et h e ' w e i g h to' f t h e
v a r i o u es l e m e n tosf a c o m p o s i t i oann, dt h eg r e a t etrh ew e i g h o t f a n e l e m e n t , t hger e a t eirt s
s a l i e n c eT. h i s s a l i e n c ea,g a i n ,i s n o t o b j e c t i v e lm y e a s u r a b l be u, t r e s u l t sf r o m c o m p l e x
interaction, a complextrading-offrelationship betweena numberof factors:size,sharp-
nessof focus,tonal contrast(areasof hightonal contrast- for instance, bordersbetween
blackandwhite- havehighsalience), colourcontrasts(for instance, the contrastbetween
s t r o n g l sy a t u r a t eadn d' s o f t ' c o l o u r so/r t h e c o n t r a sbt e t w e e rne da n db l u e ) p , l a c e m e ni nt
t h ev i s u afl i e l d ( e l e m e n t s
n o t o n l yb e c o m e ' h e a v i e r 'tahse ya r em o v e dt o w a r d st h et o p ,b u t
alsoappear'heavier'the furthertheyare movedtowardsthe left,dueto an asymmetryin
t h ev i s u afli e l d )p, e r s p e c t i v( feo r e g r o u nodb j e c t a s r em o r es a l i e n t t h abna c k g r o u nodb j e c t s ,
a n de l e m e n t sh a t o v e r l a p o t h e re l e m e n tasr em o r es a l i e ntth a nt h ee l e m e n t sh e yo v e r l a p ) ,
and also quite specificculturalfactors,suchas the appearance of a humanfigure or a
potentculturalsymbol.And,just as rhythmcreatesa hierarchyof importanceamongthe
elements of temporallyintegrated texts,sovisualweightcreatesa hierarchy of importance
amongthe elements of spatiallyintegrated texts,causingsometo draw moreattentionto
t h e m s e l v et hsa no t h e r s .
B e i n ga b l et o j u d g et h e v i s u a w l eigho t f t h e e l e m e n tosf a c o m p o s i t i oi n s b e i n ga b l et o
j u d g eh o wt h e y ' b a l a n c eT' .h ew e i g h t h e yp u t i n t ot h e s c a l e d s e r i v efsr o m o n eo r m o r eo f
the factorsjust mentioned. Takentogether, the elementscreatea balancingcentre,the
p o i n t o, n em i g h ts a yf,r o mw h i c h ,i f o n ec o n c e i v eodf t h ee l e m e n tass p a r t o f a m o b i l et,h i s
m o b i l ew o u l dh a v et o b es u s p e n d eRd e. g a r d l e o s sf w h e t h etrh i sp o i n ti s i n t h ea c t u a cl e n t r e
of the composition or off-centre,it often becomes the spaceof the centralmessage, and
this atteststo the 'powerof the centre'(Arnheim,1982) to whichwe havealludedalready,
a powerwhichexertsitselfevenif the Centreis an emptyspacearoundwhichthe text is
organized - cf. Barthes'remarksaboutthe 'emptyheartof fokyo' (I970:44).
Perspective produces centresof its own,and by doingso contributes to the hierarchiza-
t i o n o f t h ee l e m e n tisn c o m p o s i t i o nAss. a r e s u l vt i e w e r m s a yr e l a t et o c o m p o s i t i o ni nst w o
ways:perspectivally, in whichcasethe composition is ostensibly basedon the viewer'sper-
spective/position; or non-perspectivally, in whichcasethe composition is not basedon the
viewer'sposition/perspective. In the formercasethe viewers, face-to-face with the infinite
recessof perspective, becomethemselves the centreof the composition, thus takingthe
p l a c eo f ,f o r e x a m p l et h, e d e i t i e si n B y z a n t i noer B u d d h i spt a i n t i n g sI n. t h e l a t t e rc a s et h e
representation is codedfrom an internalpointof view,as is borneout by the fact that what
is left and what is right is judgedfrom the point of viewof the represented participants
ratherthan from the pointof viewof the viewer.Uspensky (1975: 33-9) hasdocumented
t h i sw i t h r e s p e ct to i c o n - p a i n t i nH . c i t e st r a d i t i o n agl u i d e fso r i c o n - p a i n t ewr sh i c hs t a t e ,
ge
f o r i n s t a n c e , ' 0 tnh e r i g h t ,o r g o o ds i d e ,i s M o u n tS i n a i ,o n t h e l e f t ,o r b a d s i d e ,M o u n t
Lebanon', andthenshowshow,from the viewer'spointof view,MountSinaiis on the right
and Mount Lebanonon the left. He addsthat this is a generalfeatureof pre-Renaissance
art, andalsoof primitivecartographic drawing.
In the theoryof art, compositjon is oftentalkedabout in aestheticand formal terms
( ' b a l a n c e ' , ' h a r m o neyt'c, . ) .I n t h e p r a c t i c eo f n e w s p a p earn d m a g a z i n lea y o u ti t i s m o r e
The meaning of composition

o f t e nd i s c u s s ei d n p r a g m a t i ct e r m s( d o e si t ' g r a b t h e r e a d e r sa/ t t e n t i o n ' ? )I n


. our view
thesetwo aspectsare inextricably intertwinedwith the semioticfunctionof composition.
As we haveseen,in many magazineadvertisements (e.g.plate 2) the top section,the
'promiseof the product',is the mostsalientelementdueto its size.Thissuggests, not just
that suchadvertisements attemptto makereadersnoticethe attractivepicturefirst, so
as to 'hook'them,but alsothat Idealand Realare ranl<ed in importanceand opposed to
eachotherin this way.Composition is notjust a matterof formalaesthetics andof feeling,
o r o f p u i l i n gt h e r e a d e r (sa l t h o u g iht i st h a t a sw e l l ) ;i t a l s om a r s h a lm
s eaningfu el ements
i n t o c o h e r e nt te x t sa n d i t d o e st h i s i n w a y sw h i c ht h e m s e l v ef os l l o wt h e r e q u i r e m e not sf
mode-specific structuresandthemselves producemeaning.
Rhythmand balancealsoform the mostbodilyaspectsof texts,the interfacebetween
o u r p h y s i c aal n d s e m i o t i cs e l v e sW . i t h o u tr h y t h ma n d b a l a n c ep,h y s i c acl o o r d i n a t i oinn
t i m e a n d s p a c ei s i m p o s s i b lT eh. e yf o r m a n i n d i s p e n s a bml ea t r i xf o r t h e p r o d u c t i oann d
receptionof messages and are vital in humaninteraction.Moreover, it is to quite some
degreefrom the senseof rhythmandthe senseof compositional balancethat our aesthetic
pleasurein textsandour affectiverelationsto textsare derived.

FRAMING

T h et h i r d k e ye l e m e nitn c o m p o s i t i oi n s f r a m i n gI.n t e m p o r a l l iyn t e g r a t etde x t sf r a m i n gi s ,


a g a i n ,b r o u g h ta b o u tb y r h y t h m .F r o mt i m e t o t i m e t h e o n g o i n ge q u a l - t i m ecdy c l e so f
rhythmare momentarily interrupted by a pause, a rallentando, a changeof gait,andthese
j u n c t u r e sm a r k o f f d i s t i n c tu n i t s d
, i s c o n n e cstt r e t c h eosf s p e e c h o r m u s i co r m o v e m e n t
from each other to a greateror lesserdegree.Where such juncturesare absent,the
e l e m e n t as r e c o n n e c t eidn a c o n t i n u o ufsf o w .I n s p a t i a l l yi n t e g r a t ecdo m p o s i t i o ni ts i s
no different.The elementsor groupsof elementsare either disconnected, markedoff
from each other,or connected, joinedtogether.And visualframing,too, is a matter of
d e g r e ee:l e m e n tosf t h ec o m p o s i t i om n a yb es t r o n g l o y r w e a k l yf r a m e d .
Thestrongerthe framingof an element, the moreit is presented as a separateunit of
i n f o r m a t i o nC. o n t e xtth e n c o l o u r si n t h e m o r e p r e c i s en a t u r eo f t h i s ' s e p a r a t i o nT' .h e
membersof a group,for instance, may be shownin a groupportrait (as in groupphotos
o f s c h o ocl l a s s eosr e m p l o y e eosf a c o m p a n yo) r i n a c o l l a g eo f i n d i v i d u aplh o t o sm , arked
off by frame linesand/oremptyspacebetweenthem (as with photosof the managers
of a companyin a companybrochure).The absenceof framingstressesgroupidentity,
i t s p r e s e n csei g n i f i e si n d i v i d u a l i tayn d d i f f e r e n t i a t i o Inn. f i g u r e6 . 1 , f r a m i n ga c q u i r e s
dramaticsignificance. The left postof the doorandthe dividinglinebetween the lightand
dark boardson the roof createa frame line which,literal