Multimodality A social semiotic approach to contemporary communication Gunther KressMultimodality ‘Gunther Kress’s Multimodality is the definitive statement of a social semiotic approach to communication in all its modes and their admixtures from the man who Virtually invented the modern form of this field’ James Paul Gee, Arizona State University, USA “This book represents a paradigm shift beyond linguistics and literacy studies as we have known them. Yet, for all its path-breaking intellectual innovation and conceptual profuncity, it contains a d'sarminaly lear view af meaning-making, one in which human agency is at the centre.’ Mary Kalantzis, University of llinois at Urbana-Champaign, USA The twenty-first century Is awash with evermore mixed and remixed images, writing, layout, sound, gesture, speech and 3D objects. Multimadality looks beyond language and ‘examines these multiple modes of communication and meaning-making ‘Multimodality: A Social Seriotic Approach to Contemporary Communication represents a long:awaited and much anticipated adcition tothe study of multimadality from the scholar who pioneered and continues to play a decisive role in shaping the field. Written in an accessible manner and illustrated with a wealth of photos and illustrations to clearly demonstrate the points made, Multimodalty deliberately sets out to locate communication Inthe everyday, covering topics and issues not usually discussed in books of this kind, from traffic signs to mobile phones. In this book, Gunther Kress presents a contemporary, distinctive and widely applicable ‘approach to communication. He provides the framework necessary for understanding the attempt to bring all modes of meaning-making together under one unified theoretical roof. This exploration of an increasingly vital area of language and communication studies will be of interest to advanced undergraduate and postgraduate students in the fields of English language and applied linguistics, media and communication studies and education, Gunther Kress is Professor of Semiotics and Education at the Institute of Education, versity of London. His numerous titles include Reading Images (co-author with Thee W) Leouwen, 2nd edn, 2006), Literacy in the New Media Age (2003), Early Spelling (1999), Before Writing (1997) and Learning to Write (1993); all published by Routledge.Multimodality A social semiotic approach to contemporary communication Gunther Kress 4) Routledge RoteFirst published 2010 by Routledge 2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN, Simultaneously published in the USA and Canada by Routledge 270 Madison Ave, New York, NY 10016 Routledge san imprint ofthe Taylor & Francis Group, an informa business ©2010 unter Kress ‘Typeset in Bell Gothic by Keystroke, Tettenhall, Wolverhampton Printed and bound in Great Britain by Td International Ltd, Padstow, Cornwall All rights reserved. No part ofthis book may be reprinted or reproduced ‘or utilized in any form or by any electronic, mechanical, or other means, now known or hereafter ievented, including photocepying and recording, or in any information storage oF retrieval system, without permission Jin writing from the publishers. British Library Cataloguing in Pubication Data A catalogue record for this book is available fram the British Library Libcary of Congress Cataloging-in-Publication Data Kross, Gunther &. Multimodalty : a social semiotic approach to contemporary communication” Gunther Kress. pcm, Includes bibliographical references and index 1. Semioties. 2. Communication. I. Title, P99.K64 2009 302.2-¢22 ISBN10: 0-415-32060-7 (hbk) ISBNLO: 0-415-32061-5 (pbid) ISBN10: 0-203-97003-9 (ebk) ISBN 13: 978-0-415-32060-3 (hbk? ISBN 13: 978-0-415-32061-0 (pbk) ISBN 13: 978-.0-203-97003-4 (bk) Meinen Eltern Elly Johanna Charlotte Kress, 1912-1988 Johann Michael Kress, 1911-1987 Zwei Leben, gegen eine grausame ZeitContents Chapter 1 fi CiApter J List of ilustrations Preface Acknowledgements Where meaning is the issue Multimodalty: simple, really 1 From semiotic system to semiotic resource 5 Cultural difference and communication: the ‘reach’ of the theory and the ‘reach’ of modes & The politics of naming 12 A satellite view of language 15 ‘The social environment of contemporary communication ‘An ethical approach to communication 18 Assumptions 19 Environments for communication: social frames and communicational possibilities 19 Power, authority and authorship 21 Social and theoretical consequences: ruling metaphors of participation, design, and production 22 Personal choices: existential insecurity or agency through particivation and connection 23 Communication and meaning: fluidity, provisionality, instability 23 A prospective theory of communication: rhetoric, design, production 26 From language and grarnmrar to semiotic resources 27 Mobitity and portability 28 A word on ‘pace’ 29 The nex! for apt metaphors 30 13Contents > Giliel Part! 746 _/Shapter 3_/communication: shaping the domain of meaning 32 C Chapter 4 Labo ¢ Chapter 5 Chapter 6 Chapter 7 Communication as semiotic work: a sketch of a theory 32 ‘Reading’ and the reader’s design of meaning 37 Provisianality in communication: rhetoric and design, newly configured 43 Environments of communication: ahistorical view 46 Refashioning social and semiotic domains: rhetoric and design 49 A | of pyinpsatiy Ls Yptlrdlong! 54 From 4 linguistic to a *ultimdAal social-sefnidtic theory of 4 ~ meaning ané communication 54 Linguistics pragmasiewaga a speak semiotic aporaach V to represetatio ss A Horses for courses: at theories seful Frarflg“60 The motivated sian 62 The everyday, the banal and the motivated sion 65 Interest and the partiality of reoresentation 70 Mimesis, signs and embodied experience 7 y ” é ‘Materiality and affordance: the social making of mode 79 The ‘reach’ of modes 83 What is a mode? 84 Is layout a mode? 88 ‘Mode, meaning, text: ‘fixing’ and ‘framing’ 93 Mode as technology of transcription 96 Mode Meaning as resource: ‘naming’ in a multimodal social-semiotic theory 103 Naming aptly 103 New frames, new names. 105 Making signs: resources, processes and agency 107 Processes and effects: making and remaking meaning 120 Design and arrangements: making meat 19 material 132 Design in contemporary conditions of text-making 132 Design: an essential (refocusing 133 Wiat is design? A homely example 135 Desiga in social-semiotic environments 1 Changes in design: a brief look at recent history 14) Chapter 8 Chapter 10 Contents Arrangements: making meanings material 145 What else is framed? 153 Discourse: ontological and epistemological frarning 157 Multimodal orchestrations and ensembles of meaning 159 The world arranged by me; the world arranged for me 159 The world arranged by me, the world arranged for me: ‘orchestrating ensembles, staging of movement, motion, “pace” 162 Aesthetics, style and ethics in multimadal ensembles 169 Applying the theory: learning and evaluatio knowledge identity and Learning and identity in a communicational frame 174 Reading as design 175 Semiosis, meaning and learning 178 Recognition, metrics and principles of assessment 182 ‘The social semiotics of convergent mobile devices: new forms Cf composition and the transformation of habitus (Elizabetta Adami and Gunther Kress) The social frame for the semiotic analysis 184 Tne affordances of Smartphones: a social-semiotic account 185 Affordances of the hardware 186 “Shape’: designers" intentions and social implications 187, The representational affordances 187 The functionalities 189 Mobile Web access and usability: changes in social habitus 190 Irnplications 193 Gains and losses: some open questions 195 References Index a7? 1e4 198 208Illustrations Colour plates (between pages 78 and 79) Colour plate 1 The operating theatre Colour plate 2 CBBC homepage 2005 Colour plate 3. Salt and pepper, SAS: ‘health’ Colour plate 4 Salt and pepper, Delta Airlines: ‘social practices’ Colour plate 5 Salt and pepper, Lufthansa: “racition’ Colour plate 6 Salt and pepper, Austrian Airlines: “light and cheeky’ Colour plate 7 Bottle: as ‘toiletry item’ Colour plate 8 Bottle: as ‘ug’ Colour plate 9 Lil Snail 1 Colour plate 10. Lit Snail 2 Colour plate 11 Hedgehog, Em Colour plate 12 “My Birthday’: affect Colour plate 13 Lego car Colour plate 14 Outside the room: ‘Armenian wedding’ Colour plate 15. Inside the room: ‘Armenian wedding’ Figures 1.1 Morrisons supermarket 1.2 Waitrose supermarket 1.3 Temporary parking arrangements, Salzburg, UEFA 2008 1.4 Cell with nucleus 3.1 Saussure‘s diagram of communication 4.28 Map of a museum exhibition ‘London before London’: ‘the prehistoric cam’ 4.2 Map of a museum exhibition ‘London before London’: ‘Heathrow! 3.3 1935 Science: digestion 4.4 2002 Science: digestion 4 “This isa car’ 4.2 Football: “Arsenal vs. Chelsea? 4.9 Child's drawing of the visit to the British Museumli Ilustrations Ba 52a 5.2b 5.26 53 54 55 56 57 58 59 baa o.1b 62 63 64 65 b.ba 6.6b 7 72 73 74 aa a2 a3 a4 85 gaa 9ab 10.1a 10.18 10.2 2002 Science: digestion The digestive system: columns reversed The digestive system: rearranged Blood-circulation: concept map Alphabetic writing Character-based writing Western timeline The toner of my memories Modular layout: ‘dependence’ independence’ Bento box Senate House, University of London Entrance hall, Senate House Passport photo Posed shot EU passport stamps, ‘entered’ —»> and ‘departed’ < Giorgia Michael: classification 1 chorne) Michael: classification 2 (school) Potato peelers: new and traditional Poetry Archive website: General homepage Poetry Archive website: Teachers’ homepage Poetry Archive website: Children's homepage Blood circulation: circle Blood circulation: arrows and loop What English is for: English as popular culture What English is for: English as National Curriculum CBBC homepage 2005 Frogspawn 1 Frogspawn 2 ‘Smartphone closed Smartphone open Aesthetics of interactivity 85 90 a OL 95 8 98 100 100 101 101 cert n2 5 15, 8 us 126 127 135 140 140 ai 163 163 167 167 17 180 180 186 186 189 Preface This book has been a long time in the writing. I tell myself that maybe it will be the better for that. All this time a lot of work has been going on under the label of ‘iwitimodality’. While I have not documented any of that here, the traces of it are evident everywhere in the book. Work in multimodality comes from quite different disciplinary) perspectives to that taken here and one part of the task at present is to see how to make use of the connections. Carey Jewitt’s Handbook of Multimodal ‘Analysis (2009) gives a very good sense of the range and diversity of that work; and. of the interconnections. My aim here is to acknowledge the many people who, over a considerable time, hhave helped me in al sorts of ways in shaping the ideas: the smallest reconition of a (great gift of conversation, of challenge and argument, of a need to explain and defend ‘and to add; in many places, over many years. And, most important of all maybe, to have a sense of interest, support, engagement from and with like minds. I have been lucky in colleagues and friends whese ideas 1 admire and whose judgements I trust; ho took ideas about which I felt not a bit confident as ideas just like others, with a simple °Yes, 507’; or, more often, getting straight into a discussion, just carrying ‘on with the conversation, For many years | have wanted to mention Bob Hodge in that respect. We became olleagues at the University of East Anglia in the early 1970s. We taught courses Jointly; with him it felt safe being ignorant. Ideas that | had never had the courage to ay out aloud even to myself became the ordinary stuff of our talk, of disagreement ‘and agreement, of discussion. That gave me the confidence to say things; to try to publish ther even. In that long conversation, with David Aers and with Tony Trew too, we developed what has remained for me most important about meaning. | co-taught and co-wrote with the three of them (Aers, Hodge, Kress, 1982; Fowler, Hodge, Kress, Trew, 1979; Hodge and Kress, 1979). Roger Fowler, by then already tablished author, was generously supportive in our efforts to get this work published. Out of teaching a joint course on Philosophy and Linguistics over several years, Tony Trew and I wrote 2 book which we called The Chomskian Caunter- revolution in Linguistics; a book about Chomsky wresting Linguistics back from the jocially oriented path set out by his teacher Zellig Harris; turning transformations Irom the sociallyextually/discursively motivated operations they were in Harris's, writing and theory, into psychological operations. It was not a notion to catch the fnvood of the mid-1970s; and so the book was never published,v Preface Ten years on, in Sydney, I met Mary Kalantzis and Bill Cope: enormously generous personally; as profoundly serious in their intellectual positions as In their political views; utterly remarkable as activists. There too, Jim Martin and I joined forces to ‘support each other's thinking around genre, shelving theoretical differences in favour of comradeship, support and the shared political-pedagogical aims of equitable access inschools. And in Sydney Theo van Leeuwen and I started on a project trying to figure ‘out something akin to a ‘grammar’ of the visual. That project lasted a good ten years; with great enjoyment and benefit for me. Moving to London in 1991 meant moving to the —for me — new ‘site’ of Education Three research projects conducted jointly with Jon Ogborn, then Professor of Science Education, over the next nine years, here at the Institute, supplied a wonderful proving ‘ground for developing social semiotics; as well as an intensive part-time education in matters scientific with a wonderful teacher. In the last of these, ‘The rhetorics of the science classroom, Jon decisively ‘scaffolded’ ane of my more stumbling serniotic, attempts by throwing the word ‘affordance’ into the conversation. Carey Jewitt had come as the Research Officer on that project; she stayed on for the next one, “Enlish in urban classrooms’; and we have worked together since: enjoyable, illuminating, challenging, now as much as on that first project. She has been a source of inspiration and inestimable support. In London | met Brian Street. We decided early on that complementarity might be more productive than contestation; and maybe more ‘enjoyable; and so it has proved for me and I hope for him. ‘Ata conference in Toulouse, in 1990, I had met Ben Bachmair. Since then we have talked and worked on different projects around media and media education, talked and wrote about culture and communication. For me, his friendship has been truly delightful and an insightful means of reconnecting with things German; while hi (predominantly German) sociologicaUhistorical ‘take’ on education has been eye- ‘pening in thinking about issues of pedagogy. Christoph Wull’s more anthropological interest in ritual, in performance and his work on mimesis in particular, pushed me to try to became clearer about my own interests in meaning and learning, especially in relation to "the body’. Ingrid Gogolin’s coinage of the term ‘mono-lingual habitus! ~ in her work with immigrants in Germany ~ made me see Bourdieu’s concept con- cretely. In New London, as it turned out, | met Jim Gee. With his usual nonchalance, hhe made clear for me the connections of pedagogy, curriculum and “ast capitalism’, ‘while shooting some baskets with Phillip, the youngest member of our group, in the parking lot at the back of the hotel; for me, with the same ‘white light’ effect of Jon Ocborn’s mention of ‘affordance’. Staffan Selander had invited me to be a Visiting Professor in his Didaktik Design ‘group at the Laerarhoegskolan in Stockholm, between 2004 and 2006. His distinct sociological approach ~ broad, generously encompassing, precise and with a strangely real sense of the geographical proximity of Russian psychology ~ has been a constant stimulation and a challenge to staying alert. The time in Stockholn) gave me a chance to be still; to think and talk in an atmosphere of a taken for yranted intellectual an personal generosity. Among the scholars in that group, | Lenetited greatly trom Preface ‘conversations with Eva Insulander and Fredrik Lindstrand and their work. [also came lo know Roger Saljé, whose ways of thinking ~ with so much affinity to a social semiotic perspective and yet so productively distinct — have proved greatly enriching for me In London I met Denise Newfield and Pippa Stein. Both had been working at the University of the Witwatersrand during the years of apartheid and at the same time they had worked with teachers and children in schools in Soweto. Through their ‘experiences and in their reflection on that experience they had much more to teach me In their doctoral research than I could offer them in return. Their hopes for those students and teachers and thelr unbounded optimism for them in a future South AAirica showed me how entirely linked inspired academic work and political action are. Pinpa Stein's death since, is for me, as for so many people in mary places, a vast loss. ‘Some two years ago Roger Kneebone approached me with a view to doing some Joint research on the operating theatre seen as a pedagogic site. That has opened a ‘whole nev world of challenge, as well as truly enjoyable conversation. My colleagues at the Institute of Education continue to inspire and support me, with coffee and conversation. { want to thank Di Mavers for her enthusiasm, energy and sharp insights into meaning-making. Jeff Bezemer brought his ethnographic take land ways of working, a great addition to the social-semiotic stew. Anton Franks has heen an always insightful companion, Caroline Pelletier, Hara Sidiropoulou and Sophia Diamantepoulou have given me much material for thought from thelr research ‘and their own reflections; they have — I hope wittingly ~ been willing to listen and respond to my ideas. Jan Blommaert brought his galvanizing energy and ideas, so close yet with an importantly different take, uring his much too brief period at the Institute of Education, Norbert Pachler has been an enormously patient guide to some Of the pedagogical and wider social affordances of contemporary technologies; and in that, has helped me get some real sense of its profound implications. have learned with all the doctoral researchers with whom I have worked. It is Invidious to single out individuals; yet some need to be mentioned. Paul Mercer's exploration of forms of representation and social meanings in TV programmes ins for me an exemplary detailed study in semiosis, as well as an important re- jssment of the category of realism. Lesley Lancaster's meticulously detailed study ff semiosis of a two-year-old; Shirley Palframan’s study of semiosis in the work of early teenage students in classrooms, all have far-reaching implications theoretic~ ally and practically for learning and assessment and pedagogic practices generally. Inv the work that she calls ‘re-genreing’, Fiona English has opened new horizons on teaching and learning and representation. Elisabetta Adami came for a year and with hhor [ was able to think again about issues long debated in Linquistics and seemingly led ~ Grice’s conversational maxims for instance, which get pretty well stood on their head in the triple context of multimadal representation, ‘digital media’ and a social-semiotic theory of communication. Sean McGovern’s work on the semiotics of Japanese culture has far-reaching implications; and it has been very insightful for me. J wish to thank him for allowing me to use some of his data; Ihave borrowed from himwi Preface the notion of ‘modular composition’ and want to make acknowledgement of that right here, Marie-Agnes Beetschen and more latterly, Rebecca Elliott, have given me an ‘opportunity, over a considerable time, to learn, in a very full sense, about the issues of representation in the practical world — in a commercial context with its pressures and demands. That has been an invaluable source of insight and of confirmation too, for me. I wish to thank them for their wisdom, insight and support. Margit Boeck gave me comments that were incisive and proved crucial at just the right moment; what coherence and order the book may exhibit is due to her clear advice. [want to thank Lilly Ramp, who kindly took some photas of a sign by the side of a ‘ousy road in Salzburg. Rachel Kress gave me permission to use some of her images and drawings (colour plates 9, 10, 14 and 15), as have Michael Kress and Emily ress, and for that | wish to thank them. Eve Bearne kindly allowed me to use some of her data, which has been very insightful for me; it appears as Figure 4.3. The book draws on three research projects, funded by the UK's Economic and Social Research Council: ‘Gains and Losses: Changes in Representation, Knowledge and Pedagogy in Learning Resources’ (2007-2009); “English in Urban Classrooms’ (2001-200); and "The Rhetorics of the Science Classrooms’ (1998-2000). It aso draws on a project funded by the Swedish National Science Foundation, conducted in Stockholm and London "The Museum, the Exhibition and the Visitors’, directed by Professor Staffan Selander of Stockholm University and co-directed in London by myself. The deep benefits of such Support will, | hope, be evident in the pages of the book. Anyone who engages in this sort of work knows about the people who are essential yet 100 often remain unsung. No academic can work well without the help of those who smooth the paths of administration and who create a climate conducive to feeling ‘supported’. In that respect | would like to mention Sarah Gelcich and Manjit Benning for their help; differently, and each essential. It remains to thank the more than patient editors: Louisa Semlyen, over many years, and latterly Sam Vale Noya, whose job was not made one whit easier by my slowness; and without whose patience and increasingly firm pushing this book might still not be done. GRK London, October 2009 Acknowledgements ‘The publishers would like to acknowledge the copyright holders for permission to reprint the following materi 1935 Science: digestion from General Science, Part I1T, Fairbrother, F., Nightingale, E, and Wyeth, Ful. (G. Bell & Sons, 1935). 2002 Science: digestion from Salters GCSE Science: Year 11 Student Book (Heinemann Educational Publishers, 2002). Austrian Airlines for the reproduction of their salt and pepper packaging. The Poetry Archive for Screenshots from their website at www.poetryarchive.org, British Broadcasting Corporation for the screenshot from their 2005 website at worm bbe.co.uk/ebe. Every effort has been made to contact copyright holders. If any have been inadver- lently overlooked the publishers will be pleased to make the necessary arrangements ‘at the first opportunity1 Where meaning is the issue Multimodality: simple, really On my way to work the bus gets held up before a large intersection, even quite early In the morning. Sitting on the top deck, my eye is drawn to a sign, high up on the wall opposite; it shows how to get into the car park of a supermarket. It is nat a com- plicated sian by any means, nothing unusual about it really. But I have puzzled about It: how does it work? Above all, how does it work here? It is about 150 metres before this complicated intersection. Drivers have to keep their eye on the traffic; there’s fo time for leisurely perusal. OF course, my academic interest in the sign les in its Joint use of image and writing. And so, one morning, when the bus is held up in just the right spot, I take a photo on my mobile phone, as one does (Figure 1.1). If writing alone had been used, would this sign wark? I don’t think it could: there 4s to0 little time ta take it in. A litte later in the day, if shoppers tried to read the sign, the intersection would clog up. With writing alone, the message would, quite simply, be 100 complex. Using three modes in the one sign ~ writing and image and colour as ‘well ~ has real benefits. Each mode does a specific thing: image shows what takes too Jong to read, and writing names what would be difficult to show. Colour is used 10 highlight specitic aspects of the overall message. Without that division of semiotic labour, the sign, quite simply, would not work, Writing names and image shows, while colour frames and highlights; each to maximum effect and benefit. If writing by itself would not work, could the sign work with image alone? Well, Just possibly, maybe. Writing and image and colour lend themselves to doing different Kinds of semiotic work; each has its distinct potentials for meaning ~ and, in this case, linge may just have the edge over writing. And that, in a nutshell ~ and, in a way, as simple as that ~ is the argument for taking ‘multimodality’ as the normal state of human communication. Except that, just across the road, on the other side, there is another supermarket. It too has a sign, on its side, just as high uo; it shows its customers how to get into itscar park. Figure 1.2 isa photo of this other sign. The sign is different: not different Ii the modes used but in how the modes are used. Colour is different, tines are differently drawn; the sign has a distinctly different aesthetic. Multimodality can tell Us Whiat modes are used; it cannot tell us about this difference in sty; it has no means {to toll us what that difference might ean. What is the difference in colour about or UMuttimedatity 0058-9 Leys 1] LJ L. Carpark OFF HERTSLET ROAD a) oe) | Figure 1.1 Morrisons supermarket the difference in the drawing style? What identity is each of the two signs projecting? What are the supermarkets ‘saying’ about thernselves? What are they telling their ‘customers about themselves? Are these differences accidental, arbitrary? Would the style of one sign serve equally well for the other supermarket? To answer questions of that kind we need a theory that deals with meaning in all its appearances, in all social occasions and in all cultural sites. That theory is Social Semiotics Of course, there are many theories of what communication ought to be and how it might work. And then there is power. So just because something might nat really work writing by itself, in this ease —it does not mean that it wor't be used. There ar2 many n more important than ‘will it work?’. There is tradition, for instance. Writing has traditionally been used to do certain communicational things ~ regulations id Instructions being just two. Then there is officaldom. Bureaucracy assures that as long as something has been announced in writing it has been communicated and the rest will look after itself; or else it can he left to the law, where the excuse ‘but nobody could have read it in that time" doesa’t count. Here (Figure 1.3) isan example ofthat latter approach, It comes from ane of the cities whicly played host to games in ne European Soccer Championships of 2008. The sign ann temporary ehanay Where meaning isthe issue Fiqure 1.2 Waitrose supermarket {o parking regulations for the duration of the championships. A rough translation might be: Dear drivers! During UEFA 2008 the official times for bus lanes will be changed as follows. From 7 June to 29 June 2008, the normally applicable times for the bus lanes 1e Griessgasse, Rudolfskai, Imbergerstrasse, Giselakal and Schwarzstrasse will be extended until 2 a.m. of the following day. On days when games are being played, that is, 10.06, 14.06 and 18.06 2008, the ‘no-stopping rule in the bus lane in Griessgasse will come into force earlier, from 10am The instructions are not unfriendly ia tone; though they are complex, official and, above all, impossible to read in the brief time before the lights ahead turn again. But then, there are times ~ perhaps many times ~ when communication isn't really the Issue, and power is. That isa crucial point to bear in mind in thinking, theorizing and Writing about meaning, communication and soctal mattMultimodatity tober ge Figure 2.3 Temporary varkiny arraiyements, Salzburg, UEFA 2008 Where meaning is the issue ‘Simple paints often have profound consequences: and so it is here: consequences for learning, for knowing and shaping information and knowledge, for attending to and communicating about the world and our place in it. Developing ways of thinking about This a once simple_and complex phenomenon’ ~ that Ts, setting OUCa-sacial- semiotic theory of multimadality —is what th's book is about. From semiotic system to semiotic resource If that is the issve, then one ought to ask ‘What has produced the explosive interest In the issue of multimodality over the last decade or so?’ and, most obviously, ‘Why fnow?’. Image has been a part of human cultures longer than script ~ though the difference Between the two is not at al clear-cut. Image has been the subject of much interest, academic or otherwise, aver millennia. Gesture is a presence in all cultures, fen if in quite different ways, As ‘sign language” it has been elaborated and articu (ated intora-fanly Tunetioning representational resource. Whether as TeSTUre OF as signing it has been much stuaied (British Deaf Association, 1992). Academic ‘disciplines have their interest in particular modes: Psychology in gesture; AFt History in image, as has Mathematics, if differentiy so; music is studied in conservatories {te world over. One difference is that whereas before these were the subject of interest t areas of academic work, now there is an attempt to bring all means 1 making meaning together under one theoretical roof, as part of a single field in a unified account, a unifying theory. ‘A further reason, quite simply and yet most powerful of all, this: the world of ‘onynunication has changed and is changing still; and the reasons for that lie in a vast \web of intertwined social, economic, cultural and technological changes, ‘Any attempt at a satisfactory answer to the questions “Why?! and ‘Why now?’ has to uo in the direction of the Factors, everywmere connected, which have been and still fare sweeping the world. One shorthand term wich points to a collection of these is globalization’ ~ which, for me, refers not only to financial globalization but to Conditions which make it possible for characteristics of one place to be present {ani active in another — whether economic or cultural or technological. The Forces ‘of neoconservative ideology have sponsored and amplified these conditions, though they have been only partially causal. The factors at issue ~ let us say, forms of Management, or ideologies around schooling — always impact in one locality and there they enicounter locally present factors. There isa struggle of local forms and traditions With the features from outside, in which both are transformed, transformed in ways dependent on arrangements and dispositions of power in a locality. The effects of this vastly diverse and complex phenomenon have led in very many places to the corrosion, fraying, dissolution, desteuction and abandonment of older relations, forms, structures, ‘givens’. Globalization is not ane ‘thing’ it is differ ly constituted i different places, as are its effects and impacts, interacting with thw vastly varied cultural, social, economic and political conditions af any ane specific locality, Yet the deep effects are constant yind recognizable everywhere, They have 56 ‘Muttimodality brought a move from a relative stability of the social world over maybe the last two centuries (as in Western Europe) to an often radical instability over the last three decades or so. Stemming fromm that ~ and generated by it ~ are far-reaching changes in the domain of meaning: in representation and in semiotic production’; in dissemi nation and distribution of messages and meanings; in mediation and communication Al have changed profoundly. “The semiotic effects are recognizable in many domains and at various levels: at the level of media and the dissemination of messages — most markedly inthe shift feom the baok and the page to the screen; at the level of semiotic production in the shift {rom the older technologies of print to caital, electronic means; and, in repre- sentation inte shift fom the dominance ofthe mode of writing to the move of image, as wall as others. The effects are felt everywhere, in theory no less than in the practicalities of day-to-day living, Academic interest in the characteristics ofthis new communicational world, the world of the screen and of multimodality, has been relatively belated, stumbling after the horse which had left the stable some while ago. Belated or not, there isa need to catch up and get back inthe saddle. The effects of globalization are clear. From (relative) permanence and stability there hasbeen aimarked Sh to orovisionality ang instability. Curentiy fashionable metaphors are always revealing in that respect. For instance, while lam writing book, the metaphar of mobility has areat curreney, as in °mobile technologies’ or “mabe learning’. Like others, I se such metaphors; they are attempts to name and capture something ofthe essence ef the alterations, transformations, remakings of social arrangements and practices. Desig, isa term Ihave been usig since the early 1990s; for mei indicates a shift away both from the earlier social and semiotic goal of competence as from the somewhat later critiave Design accords recognition to the work of individuals in their social lives and builds that into the theory. in my use of the term, designs about a theory of communication and meaning, based — atleast potentially —on equitabe participation inthe shaping of the social and semiotic world. Design, by contrast with competence, foregrounds a move away from anchoring communication in convention as social regulation Design fecuses on an individual's realization of ther interest in their world. The move away from critique has, for me, aaierent focus and motivation: away from engaging with the past actions of ethers and their effects. Competence leaves arrangements unchallenged. Critique is oriented backward and towards superior power, concerned with the present effects of the past actions of others. Design is prospective, looking forward, Design focuses on my interests now in relation tothe likely Futur effects of my actions. The understanding which inheres in campetence was essential to carry cut evtique, jut a the understanding developed threuigh critique ic ecental in the practices of desian. Desian draws on both these, carves their insights forward anc deepens them, focused in a socia-semiati theory af muiltimodaiity. ‘As anther example, take the shift fom the ferry rammar tothe use ofthe tern semiotic resources. It indicates a shift from meanings travkitionally attached to Was fixed and highly constrained regularity. Occasionally I use te te Where meaning isthe issue 7 \garar’ to contest its meaning with my sens of ‘relative regularity ofa semiotic ester’ Tous the boc T eer to dro atierion tte trscrionsand ths Bi ecives ct ste mecaghorat ne arene #0, etree selene ante possible effects on what I want to achieve, While it may Sern pedantic t dos, I fpen! time here ad there drawing out such differences perspectives and effects In eof metaphors rn oats have oor Wc ‘lois ue ta understand acount Sor se word f cmmunisation asi aw. Nor therefore do we Rave an adequate set of tes tries ura: ue'ree to-do. a thr merce we tend to use he lorms that we have inherited from the former era of relative stability, in particular {rom theories and descriptions of language, where, over a long time, a complex set of ols has been developed. Bu using tools that had served well te Ye horse-drawn tarrages Becomes a prob in mending contemporary cars. Many ofthe older terms Beta epnces wii nat 2 coder aru see fo" Pen puri a BI rwrtineer comma aod obs calla n traducing ene nee te Many trequenly ised terms need careful re-examination; represetaion' one of those, The points: adequate theoretical tools ae needed to deal lr with the presant peste polkical ar itu esiatlon fed teresa Conon ft Fae tics ot rececary rea! vesnures il bey vetrn Iria» iers we haves ten kn to ery nme In Wester oles ‘thdugh there is not much agreement as to what it might mean. There is a certain feel- i= held with diferent depress GT conviction Tn different domains ~ that grammar fabout rules, conventians. certainty: phenomena that are fixed, settled. When business leaders, politicians, media pundits, bemoan that the young leave school and are Boheme thy cone e-n grapes) serena, ey lok hack some ‘thirty oF forty years and seem to discern, through a sepia-tinged haze, that in their day the teaching of grammar ha produced just such iealconitions. They rot thatthe young do not use the rules which they are certain that they had followed way back then; and which they think they use still, in their present practice. Adherence to linguistic convention is equated with adherence to soclal convention and consequently With social ‘stability’ Alas, the present is deeply unstable; no amount of nostalgia {ay Ghange that. The need is for a Tabour force that can meet the semiotic demands ‘ot conditions now. But in this, ‘grammar’ in its older sense of ‘a stable system of rules’ {s,m obstacte to necessary action. Representational and communicational practices are constantly altered, modified, _y~ f(s all of culture, in line with and as an effect of social changes. That inevitably Jakes the gramunar of gramunar bonks a veeaed ofthe past sacial practices of parti lar groups, in speech as in writing: useful maybe then, but neither used nor useful for those who wish it were atherwise. If | use the term Ygrammar’ it How, not eve Igaves me with the need to contest its implied meanings of ‘fixed rules’, ‘stable onvention’, and so on, oF to use a term which is free of such histories of meaning. At fhe moment 1 generally choose the latter youte, using the term ‘resource’, as in8 Multimodality ‘resources for representation’. Resources are constantly remade; never wifull arbitrarily, anarchically but precisely, inline al What (need, in FeSponse ‘to some ‘demand, some ‘prompt’ now — whether in conversation, in-waiting i silent engage- fent with some framed aspect of the world, or in inner debate Semitic resources are socially made and therefore carry the discernible regula ties of social occasions, events and hence a certain stability; they are never fixed, let alone rigidly fixed. No degree of power can act against the socially transformative force of interaction. This isnot the point (nor the purpose of this book) to talk about the benefits or disbeneits of stability, or the need for stable ‘ules in order to attempt ~ even if never to guarantee — relative security of communication. of Cultural difference and communication: thé ‘reach’ of the thi and thé ‘reach’ of modes Most rSadeF will take it as given that a society, its cultures and the representation ‘of their meanings, form a tightly integrated whole, at a certain level of generality at least. If that is so, then differences between societies and cultures means differences in-eppresentation and meaning. That is close to-@ commenplace, We know that lan- ‘quages differ and that those differences are entirely linked with differences of histories and cultures. What isthe case for ‘language’ isso for all representation ~ for modes as for discourses as for genres and in all communication: patently so with music, with image, with clothing, food, and so on. Chinese opera isnot the opera of Mozart cr Puccini. For me, as someone from “the West’, it takes much facused semiotic Gworbto understand its meanings, its aesthetics; roughly the same amount of work, imagine, hat itakes for someone wh has grown up with Chinese classical musie to make sence of ‘Western’ music The more pronounced resources of represent erences the greater are the the practices of thelr use. This means that in izing and writing about commurigatian can tale wth somé-canTdence about Tres that | know reasanably well Where my knowledge becomes veGue or ‘imore General, | can only talk vaguely and generally. T have a Gearee of confidence, ar ‘inwardness’, with several anglophone cultures, and different degrees of ‘inwardness’ with some ofthe cultures of mainland Europe. I have a glancing acquaintance wit sore cultures in Asia and Latin America, where I Fave acen a casual visto. I make ne strong claims about those cultures and none about those of Africa: it would be foolish fr me to do sc. ‘So what isthe reach’ ~ the applicability ~ ofthe theory I put forward here? Hove eel is it, how far and where and in what ways dies or an t app? Is # confines to Western Europe alone? What claims to understanding or insight can I make for this theory outside ‘the West, broadly? I hav, ary case, a problem with the notion of ‘universals’~ sueh as Yuniversls of language, oF “niversals of communication, $0 am nol tempted much in that direction. The universe ofcullures and of cultural Ailvereace on ouF small planet i to vast for such generalzan, Where meaning isthe issue There tend o be contradictory views on how to deal with this. As far as “anguage’ 's concerned, we know, on the one hand, that languages differ’ the way they name the word ~ in ‘words asin syntactic and grammatical forms; we know that lexical fields are close mappings of social practices. To give a banal example, Enalish does ot have the word“Weltschmerz’ and German does not have the word ‘literacy’. Each ‘ny strugate to find ways of bringing the other's meanings inte their culture; in the {ose ofthese two examples, with lite success so far. Culture is to complex to toer- ‘te difcut transplants readily On the other hand, we sor of assume that “anguage fuquls lanauage’s that is, if thare is @ novel in Russian it ought to be possible to \tanslate it into English. We expect significant overlap, even if nat a one-to-one ‘correspondence. In many cases the incommensurability matters. In a debate, Germans fo talk about “Wissen” Cknowing that’ and "Koennen’ Ckrowving how’): when the Wiseussion moves to translating ‘knowledge’ from English to German (or Swedish) itis not stratghtforward to know which of these to choose. Or, another example ii Enalsh there are the two words ‘teaching’ and ‘learning’, verbs with an implied Aivectionality of authority, in which teachers’ have ‘knowledge’ and ‘learners’ have (0r used to be assumed to have) the duty to acquire this knowledge. Vast theoretical A) practical edifices and industries can and have been erected on such distne- flons. Yet in many languages the same lexical root is used for this social domain i German (as in Swedish for instance, “leen’ and ‘ernen’ are morshological infec Nions ‘alternations’ ~ ofthe one lexemic stem ~ ‘learn’ in English, That makes it likely that German and Swedish societies might develop quite ctferent theories around ‘Viication’ to English society; and articulate them in their languages and their instititions. In each cas it isthe ‘accident of exis” ~ not of course an accent but the ‘pression of a history of non-accidental social differences ~ which leads to such Onotonical witd goose chases Two theoretical approaches have characterized this debate. One is focused on ‘ule; the other is focused onthe hurman brain. Warn Chorsky's universalism isthe Wot known recent exemplar ofthe latter, through his posited link between the organ- leaton ofthe bra ain mina ering and that of Tnavane— assupti Aout the Yinateness of linguistic competence (Chomsky, T9BET This asserts that ‘the organization (the ‘nature’) of the human brain is shared by all humans; the ‘deep’ Srganization of language derives from ‘dee, ‘innate’ mental organization, so that Irrespective of superficial differences, at a basic level all human languages share the same organization. From this, Chomsky derives a noble political project (to which J subscribe), namely that this profound and essential equality of humans ought to have lis reflection in a polities of real equality. From this I infer that the reach of .Chomskian theory in respect to my icsue here is general; Ty HAM Hyman; and the social and cultural really does not enter. My view on that is that there are some highly general semiotic principles, which are fommon to all human (as well as to most mammalian and some other a ‘eonymunication. Consequently, these are present and evident in all human societies and hele cultures. The most significant of theses that humans make signs in which form °10 s co— y ‘Multimodatity G6 meaning siard in a ‘motivated! relation. These signs are made with very many ferent means, in very many different moses. They are the expression of the interest of socially formed ndvguas who, wih ese sans, realize gh UIWATGexpresHon _{o= thelr meanings, using culturally available serniatic resources, which have been shaped by the practices of member: their cultures. Instead of a shared innate linguistic competence | assume shared secial, semiotic, communicational principles and dispositions ~ which includes the linguistic as one instance. These ‘principles and dispositions are articulated in communities in the ceaseless processes ‘of social (interaction. Hence the principles and dispositions take particular form, as ‘the result of the specific social concerns of a community. Given this stance, I assume that ‘translations’ across modes within a culture are both possible and hugely difficult; from image te Speech ~ the evocation’ rather than & &escription — of a painting in a conversation about av exhibition; or from image to writing; or from oral poetry to poetry in written form. I assume that translations across cultures, whether in the same mode (from writing to writing ~ from Russian ‘novel to that novel in English; from gesture to gesture ~ from the ‘French’ shoulder shrug_of indifference to an English version) or across different modes are also possible, though always achieved with enormously difficult selection; at a considerable level of generality; and inevitably with significant changes in meaning. “Stated ike tis, it implies that in its most general features, in is outline and general direction, the theory applies to all cultures. In the specificities of cultures there are ‘offen vast differences inthe articulation of these principles, which lead to difficulty in the transfer of the theory from culture to culture. Readers need to make their assessment whether, where, in what ways or to what extent my comments, based on ‘observations in some cultures and speculations based on these, are possible to ‘translate’ from one to anther of the cultures with which they themselves have “inwarcness'. This features as a topic in both Chapters 5 and 6. » To give an example of principles shared by all cultures: | have mentioned the three ‘most important: (1) that sic ‘motivated conjunctions of form and meaning; that conjunction is based on (2) the interest of He sign-maker; using (3) culturally -aeaabte resources. These are principles of sgn-making: There are Wen, aka differen Tevel, resources for making signs. Here too there are commonalities of @ genera kind. As one instance, for meaning-making to be possible, human cultures need and do provide means for franiing aspects of the Word te which an individual needs or wishes tOattend. A culture will therefore arovide Ws distinct semiatic resources for frais (complexes of) signs: what sorts of things are framed, how they are framed, what kinds of frames there are, and so on, and these will vary from culture to culture Expressing this arly ne meal wilt Fag These are instances of general principles; and of means ard processes of meanicg ‘making in any culture and in any mode. They provide a starting point in the analysis of meaning in any one culture. I do not call these “universals', though ¥ regard them as shared by all human cultures, as well as by many other species, These shared principles are based on experiences coramon to hivimans in social groups in their Where meaning isthe issue 11 ‘ngagerent with a world both vastly different at one level and yet presenting common hllenges Iba take ts Tine of argument we can acknowledge commratites at a~ vory general Tevaland yet be able to Focus or the specifieties of cultural difference. The theory can then provide a shared frame at its most generat Tevel and respond to <> Specie needs atthe level of ary one culture and ts That represents one theoretical tae on te issue hare is then the social yd cultural aspect of that notion, namely that of the reach’ of modes. Here ‘question is: ‘What areas are “covered” by a mode in a Specific culture?’ iffer Wily, "Bo all modes cover the same terrain?. In a multimodal aporoach@ is modes father than say,Tanguages’ = which are compared. Medes are the result oF asoclal— ai Historia saning of aera chaser by a socety for representation: there is broason to assume that the mode of gesture in CultureT covers the sare “area’ or 4. Wi same concerns, oF 1 used for the same purgoses and meani of pestore-ww-CuTture-2-— quite apart from the Clexieal and synta ‘mentioned just above, For instance, gesture may be used to deal with meanings around ‘attitude in two (ullures; yet what is vaddresse2", made Tito an area of common interest within the | personal stance to some event oF utterance. English culture does nat Fav ‘a equivalent to the French quick, limp-wristed Shaking of the drooping hand — usec yy women more than men — to convey ‘disapproving astonishment’. Quite to the trary, is likely that aspects of the cultural domain cavered by a mede in one iy ot covered by tat mode in anther culture. An area sah a ples ply. be dealt with extensive in-one-culture and by Afoxemics, by gesture by ficial expression in another eather Se ae ee Fhwaning, there is no reason to assume that the ‘modal division of labour’ will be the: ‘sane across societies. That is a radically different position to one which held when— the assumption was that Yanguage’ did all significant cultural semiotic work. howe differences have been addressed in the study of intercultural communication, ‘Yor here to0 an approach focusing on modes might be highly useful. An area in which his is real issue is that oF translation. Until recently that has focused on language ‘lone. In the subtitling of films for instance this becornes significant. But what about @lvanslation of a movement, an action, a gesture that is entirely understood in one soelety and either entirely misunderstood or not understood in any way in another? We simply can no longer assume that the each of modes is the same across different societies and their cultures. Modes occupy different ‘terrains’ from one society to jolly. We have to begin lodking at the Feld of meaning as a whole and see how” ‘noaning is handled modably across the range of modes in different societies.Multimodality The politics of naming Somewhat related to this is the difficulty of ‘naming’, It has two distinct, maybe unrelated, aspects: one is political, the other theoretical. The politcal aspect is how to name the political/social/eultural world about which Tam writing It isthe normal, ‘unremarkable, ordinary world, for me. As an insider, it characteristics are dificult for me to see; however much I attempt not to do so, I use my ‘insiderness’ as a yard: stick in judging other parts of the world around me. Should I use names such as the First World’, ‘the developed world’, “the post-industrialized world’, “the West" in talking about the world in which I have lived and worked; and about ‘the East’, 'the ‘Third! oF Fourth’ world in talking about parts ofthe world unfamiliar to me, different socially, economically, with profound differences in history and culture? Such ques tions are usefully discussed in the writings of Immanvel Wallerstein (2001). Apart from the difficulty of drawing meaningful boundaries for any of these constantly rruitiplying and overlapping terms, each brings with itspecific meanings, more often than not potentially harmful in their application, meanings which I would prefer not to import into my text From a semiotic perspective, all cultures, all semiotic ‘worlds’ are rich, if differ- ‘ently so. Each of these domains ~ the ‘First World’ say, or ‘Western Europe’, the nglophone world ~ is highly diverse within itself, culturally and seriaticaly, Seme parts of ‘the First World? (whatever that might be) are deeply diferent to ‘the West; and from each other. As ane small semiotic example, some societies in ‘the First World make use of character-based scripts while others use alphabetic ones. This one difference alone brings deep differences of ontology, of cultural disposition, cof dispositions towards representation, to media systems; it has effects on knowledge and identity. = = ‘Absolutely related too is te issue of globalization’. I take it that this term names something real ~ for instance, the fact that in very many places around the world, economic, political, social and cultural values and practices are subject to forces which come from ‘outside’ the domain regarded as the immediately framed “local anc have telling effects within i. Within that frame these ‘external’ factors interact with the meanings, values and practices of the ‘local site" differently therefore in diffrent places. Semiotically speaking, ths leads to constant change, transformation, ‘blending’. Constant blending is by no mears a new phenomenon, even though the: now fading) fashionability of the term ‘hybridity’ ad seemed to suggest otherwise Blending is as old as humankind itself, as archaeologists demonstrate even in parts of the globe still remote naw to the ‘first world’, Or as we can see everywhere around Us, even now, at any time. On a recent train journey in Germany ~ from the northern Rhineland to Munich ~ three couples in thei late forties or early fifties boarded the train in Frankfurt, about 9.30 in the morning. I had caught the train in Essen, about two hours tothe north. Listening to the group's German dialect, I could not tll f they ‘were from a Hessian dialect area, or from a Franconia oF a Swabian one (my overt dialect is middle Franconian). They were in high spirit rly indeed ~ inviting Were meaning isthe issue ‘ve to join in a morning coffee-cum-picnic which they had unpacked and spread on {wo of the tables, with garlands across the windows; and so I asked where they were rom. The answer was Germsheim: a town just north of Heidelberg, a region which Just about straddles all three dialect areas. Clearly, people in this area — as in very ‘many places (Rampton, 1995; Sebba, 1993) ~ are engaged in constantly assimilating, {ransforming features of the dialects around them and integrating them into a fonstant'y changing yet constantly coherent resource. What is new is the intensity of this phenomenon and the effect in its present manifestation, its ‘pace’, aided by current means of transport of economic, social and cultural commodities. Electronic forms oF communication can now make aspects “Lary Specific ‘where’ into features encountered everywhere, with an unspoken and Luegent requirement for it to be made sense of ‘there’. What is not new however, is the set of nares, the lexicon, which we have to name {his differently paced social worid. That is a problem, which makes the issue of theory 3 ail of naming within theory urgent. There is a need for new names. Eerdinand de ‘Aoussure suggested, atthe very beginning ofthe twentieth century, that Linguisties — Wi its theory and terminology — as the then’ most advanced study of aay semiotic ‘Gilem, es” for the semiotic feasts to ‘ine in the nov century. This seemed a reasonable ope, given that that expectation Was expressed in the high era of abstraction in the Social Sciences: Linguistics an Jomiotis included. In certain areas of the study of language in particular, concepts ‘elvas Slanguage, “angue', ‘parole’, ‘stood in’ for the tendency towards achieving ‘grasp of the particular via very high degrees of abstraction. By entire contrast, the the making of signs row, in this environment for this occasion. fn its focus on the Wilerial it also focuses on the bodilyness of those who make and remake signs in fonstant semiotic (inter action, ‘a. move away from high abstraction to. Wi snecific, the material: from the mentalist to the bod For that, itis essential to develop apt labels for a theory of representation and (ommunication for the whole domain of multimodal meaning as well as for the level ‘of specitic modes. The theory and the labels will need to attend to the materiality, th specificity and the histories of (social) work of any one mode. The specific asaects lf mode will need to be assimilated ta superordinate terms which capture what is Aerviotically general to all modes in, that society and to modes even across cultures; ven though what is general is always articulated distinctively in a specific mode. Itdevotea chapter to this issue of categories ofthe theory and of naming, Two terms (igure prominently in what 1 have written so far and do throughout the book: society id cultuce. the distinction between them is not easy fo make, hare Ts wetaclear line {i draw, Nor is there a generally accepted serse in which the two terms are used (Williams, 1985); for many writers the two are Gear synonymous, The distinction Uhl [ make has two features: sociely emphasizes human action in social groups (that Ji, groups organized in a recognizable fashion around goals, purposes, organizations, shared practices, values and meanings+” Social groups and actions are always S14 Multimodatity ies by (differences in) power. Cults asizes the effects, the products social actions and social work, whether physical and material or abstract and, for By powex(difference), “want of a better term, conceptuz re the social is marked by powe(d 2 Thecalra is marked by valu evaluation sll te effect oF Ca power. Culture, in_my_use, in_of socially made values; tools; meanings; knowledge; resources ofall kinds; society isthe field of human (inter/action in aroups, always; / oC work: ices of the use and ef Tuse the term work, wishing to stra sacial orientation and aims. Work involves 2 worker, tools and that wich is worked on, Work praduces change, in the worker, in the tools and in that which is worked on. Being socal and purposive, change produces and embodies meanings. [n_working,-the worker has experienced_and learned ||) something; the worker has changed. The tools have (been) changed by their use; anc what has been worked on has changed, Each ofthese chances has produced meaning, C) sew meaning The effect of these changes is to produce cultural resources. For me, ‘therefore, culture-ts-the name for the resources which have been made-produced, remade, ‘Yransformed’, as the result of social work. Everything that is socially made land remade becomes part F CUNTUraI Fescurces, imbued with the meanings of the work of those who have made and remade the resources. Cultural resources, being meaningful, are semiotic resources, It isthe social’ which generates ‘the cultural and, in that, ‘the semiotic’. Where meaning isthe issue A satellite view of language ping outside the long tradition of seeing “language” as a full means of making 19, seeing it instead as one means among others, one can gain a ‘satellite view" ff language (Kress et al,, 1996). That metaphor recalls ‘our’ fist views-of the Earth \Wvough photog-aphs from a satelite ~ that is, from outside the Earth, beyond Its ‘mosphere. That view gave ‘us’ on Earth a starilinaly different perspective; for Instance showing with frightening clarity the boundedness, the limits of our planet: his ls what we have, there is no more. From that time on, for many of us, it became {/oubling to pour old oil down the kitchen sink, because we could see, actually, that | wasn't simply going to disappear, that it would emerge somewhere else on this Hiownded Earth, with unknowable consequences. The satelite view showed us what We had knovn and had been able to ignore, in a way: that aur planet, our Earth, was (we small part of a much bigger whole. ‘A multimodal social-semiotic theory has a somewhat similar effect. It shows the Init st= al ne ere oo EIT ich. ewe Eg — a idee we id nown about the destiny of the oil Language’ Tsv"ta big enoug [etupraete Tor all the semiotic Se et se we uid pour Thto Tt. But a ‘wall foaiball or snooker, tennis or goTf, I can see that the conceptual world of the (outball, snooker oF tennis player as of the golfer Isa Tleld where semiotic wark js Constant enzagenent in (social) work as defined above, with socially made cultural- Midst docidedly not done via speech — lt alone writing = the occasional shouted ‘here’, semiotic resources and their constant remark fay actions, shapes my own Taner ‘ol or “Tessa"on an Enalish football pitch notwithstanding. With snooker, the e, transformed by ine, This describes the effects of communication-and'of ning: attention to an aspect of the social world; engagement with it in trans: formative action; which constantly remakes my inner resources; and in that, changes ‘my potential for future action in and on the world. Differently seen, it describes my construction of knowledge for myself — always in social environments; with culturally available resources imbued with the meanings of those who have shaped and reshapes! them in their social environments; responding to the needs of their times. In bot! of these processes — a differently focused perspective on very much the same phe nomenon ~ I am constantly remaking myself and refashioning myself in terms of my subjectivity (Seen from a social and political perspective) and fashioning ani constantly refashioning my identity ~ the process viewed from the perspective of my biography (Endress et a., 2005); a biography which shares much and yet is never Identical with the biography of any other member of my or any other social group. ‘Throughout the book, two other terms turn up frequently: ‘sign’ and ‘concept While the book does nat deal with ‘concept, the term names muuch the same territory as does ‘sign’ — which, together with made, is atthe centre of attention. The term si belongs to Semiotics ~ the conjunction, imvsible and indivisible, of form and meaning wile concept belongs to PsychoToay-and, deriving from there, to Pedagoay, wher it names something alled 'knawledge’ in an entity called ‘nnd | plible precision oF planning of geometrical strategies for specific shots and for the ‘ilonvied shape of the arrangement of the balls four moves ahead, is a damain outside [am tempted to say ‘beyond’ language’. That is before we get to the practices of ‘uallwmaticians (to which snooker or football players of course belong). The reach ‘al spoech oF writing simply does not extend there. The semiotic/conceptual work doné WW these fields, as in countless other human social tasks, pleasures and professions, ‘elles on modes remote from speech and writing, There are domains beyond the reach oF language, where it fs insufficient, where semiotic-conceptual work has to be and is ‘Hone by means of other modes. {W this as in other ways, a social-semiotic approach to multimodal work makes a ilforence, The task is to establish, with as much precision as we might, what Tese iilfererices are, in specific cases and circumstances. What new kinds of questions anorye and are made possible; how do persistent, older questions get recast, in ways jpotelbly that lead (o more plausible answers; and who might benefit in what ways from {iw different answers, In my area of work, the insights and benefits will accrue in all Ue areas of (heorias off learning, meaning, evaluation, assessment, subjectivity and entity JW this itis important to be aware of the twin focus of the social-semiotic theory and A multimodality, The issue of ‘access for instance, can benefit from the insight that Huinans ray have different orientations to modes and ensembles of modes. ~ maybe Wil specific preferences for the tempora-er the spatial, for image or speech, for the 1516 ] Multimodality fap the communicational encounter. This demands _rhetorical approach to communication, The hitherto sufficient (sos of competent performance and possibly oF critique cannot meet the nen demands. Inwardness with ‘grooved convention had, in the past, been sufficient for -ompetent communication By contras) fhe rheiprlas maker of a message now makes ‘an assessment of all asgectef the com: fonal situation; of her or his interest ‘BE The characteristics of the audience; the semiotic requirements of the issue ot Stake and the resources available for making an apt representation; together wit © Tstablishing the best means for its dissemination, ~ Tn a model of communication for full and equitable participation in the new com ‘municational worid, the rhetor’s interests need to be fully acknowledged, The rhetor in turn is aware of the resources needed to give material shape to these interests the world; she/he understands the audience and its characteristics; and understand what the matter to be communicated demands. This is the basis for desfgns to sha these representations ready for their production and dissemination. A rhetoric: approach draws on the resources both of competence and af critique and utilizes the in the process of design. Given the presence of modal choice in ropresentation ii a multimodal world of communication and a social world where choice is demande and the instability of the environment of communication, a rhetorical essential. Design meets the interests of the rhetor (most usually the same person) in {ull awareness of the communicational potentials of the resources which are available The socia environment of contemporary communication 27 lerprowse drawn In the environment and needed for the implementation of the rhetor’s interests. DeSz ives shape to the interests of both rhetor and designer in thelr world. Production isthe implementation of design with the resources available in the world ich the communication takes place. In production, meaning is made material and bbocomes subject to review, comment, engagement and transformation. Production happens both in making of messages and in their remaking Ta participant's engage Went with and transformation of a message. Production has semiotic (form-as- fontent), conceptual (content-as-concepts) and affective (semiosis-as-expressive, Always reflecting interest and personal investment") features; all three always at th sine time. ‘A rhetorical approach is based on the agency of maker and remaker of messages. |\ has direct implications for knowledge production. Knowledge is made and given shave in representation, according to the potentials of modal affordances; the process ‘of representation is identica! to the shaping of knowledge. Makers of representations V We shapers of knowledge Mi this conception, knowledge production is entirely part of social-semiotic Wocesses and organization of participation. That is, knowledge is always produced, falhor than acquired. Acquisition isa relatively non-agentive conception of the relation 6f individual, learning and knowledge. The new, participative sites of aopearance and tes of dissemination of messages and knowledge ~ YouTube, FaceBook, but also ‘hin logitimated sites such as those created by graffito artists ~ are an integral part Wf the new media landscape. Sites are associated with specific characteristics of iMtvibutions of power and agency in communication. Sites of appearance and sites Wf dissemination are a part of the process of knowledge production; they come with {jinciples of evaluation as part of the logonomic rules of ‘engagement//reading’/ ‘yaluation in communication (Hodge and Kress, 1988). Theories of communication ave to describe this world and attempt to articulate and theorize its practices both in Wins of continuities and changes. Given the changing environments for meaning and knowledge-making — multimodal seprowentation, participative production and sites of appearance’ — knowledge-making {uke new routes in content, form and social engagement, These routes shape what i aged with, processes of engagement and as a result sociaVindividual dispositions. Tite is a need for careful considerations of designs for meaning and knowledge- Making: the shaping of routes and enviconments of meaning-making and production Ji Knowledae and, in this, the shaping of ‘inner’ semiotic resources. The sites, the ‘yencosses, the desians all shape ‘concepts’ and, in that, they shape what dispositions Hegre habituated as subjectivities and as identity. From language and grammar to semiotic resources Ji. the monomadalyy conceived work, in other words, in a world regarded as operating Willone-kind-of resource In a sprcific domain, reflection on the potentials of that lage! warraill that there was; and “language” was Fasdurce could not arise8 Multimodality regarded as a means fully capable of dealing with all human (rational) meaning. In a rmultimodally conceived communicational world, two questions arise: one is about the ‘aptness’ of the means for representation; the other is about the complexes of mades designed for achieving complex representational and communicational requirements and tasks. Instances of commonly used modes are speach: still image; ‘moving image: writin sic; 3D modols; action; colour. Each offers specific botentials and is therefore in principle particularly suited for specific representational’ severe oneves Ipcomuniaton Gone gads) Const ats, in_ modal eg cen ere Dau he et ta Ce ee A j Design isthe process whereby the meanings of a designer (a teacher, a public speaker q but also, much more humbly and in a sense more significantly, garticioants in everyday interactions) become messages. torical) analyses, on aims salons ofa rica al hey are the implarented ooo te STRAT Fe Peg ewe at ten imsenened rows He eSenTeT Design resis on the possibility of choice ~ “this could have been chosen rather thar that. That permits the description of style as the effect of a series of choices mae ir the design of a message. Choice is always circumscri wer in different ways, foreta soci, clr fower ans se ee ice: Siyesare sect 7 alae Ese ead To a social ranking expressed ae ester iudgeiments, Hence socialy, aestetics can be seen as the politics oT SIV. Gestion around eth, such as: "Who benes?-OF Wh Sffers damage Or Fur", How ar social environments likely to be changed by this?” lead to judgements of social benefit. In this approach, ethics is the politics of value and (evaluation, It is not fanciful to see a progression from social fragmentation and the dislodging of individuals from social bonds as forming part ofa sequence of aclons Gahere G an on and the GisvougTng ofan indefdval from socal v nds, forms, struct ism — individuation — social ‘social consequences and costs of that dynarifc are visible mow in many ways and i many places. In a periad when social pathologies are increasing — aided by th: dominance of the market and its values ~ this is a major consideration, Mobility and portability Currently, mobility is a hot topic. In my view, mobility is in part both an expression and an effect of larger-scale social moves towards instability and provisionality ‘Mabitity tends to be discussed in terms of the rently available tech Tologies: physic ‘mobile as hand-held ot various Kinds; and lending communizatenal molly thous fast_increasing rapge of Features of the (former) mobile phone. This is the subject of discussion in Chaptor 10. The social environment of contemporary communicatian 29 The Interest in mobility is a response to aspects of current conditions. Jncertain domains ~ institutional education, for instance ~ mobility is seen as a paniacea to a Pile of pcblems. Two stand out effectivenes and motivation OL uinan resources’ (ie, teachers) promises tosohearange Pesto cat acai! | (oblems of The-School (conceived in traditional terms); ag methation ie sams to Iylrrpatr aay from the alienation from schoo! felt by many = predaminantly male” | i physicality of the school, its confining time structures and its general lack ff attractiveness to many students and to provide instead the physical, social and | ‘iotlonal mobility as freedom ofthe individual. Both come together in promising to inal the attractions of the market thr unded access and individual choice fi oxpressed in concepts such as ‘the personalized curriculum’. ‘What is suggested Tor tre school can serve as a metaphor for wider social domains Ii \seisential to subject tis metaphor and the claims of mobility to searching scrutiny. | are profound questions about what social effects ‘mobility’ might have: Who or imobile2, What is portable? And in any case, in what ways are portability Wily a social, evitural, pedagosic, affective, Cognitive and generally hurnan ‘vod? What conditions for sociality does mobility propose anc what kinds of socality ‘wl hese be? Is i the case that society ether can or ought to ‘develop’, change, at {iw sane pace as technological devices or in the directions indicated inthe facilities fhe ‘new’ technologies? A world of constant accessibility and availability is also a 1 constant surseilanse and in that word. what opportunity i tere for doin Tor being idle’, fr reflection? What vision Tor society is entalled? The person- Tel curriculum is at the same time the loss of one major force Tor social cohesion, {Wit of knowledge, information, values shared by all members of a group. Are total Hiumwetlvity and accessibility a social good or are they markers on the road to (ivnatarrsotr pao? Arete a ter asta fatal Fagrerttion Aljhlipiegration or are they covert means for dealing with the absence of refabTe Saal bonds? ‘The Combined effects of these factors of communication suggest the need for aids’ as an absolute requiremert: in relation to text-making, to reading, iveornment and discrimination, to reflection on metaphors and their effects. us i they suggest the need for communication to-be founded on principles which A word on ‘pace’ Hivloglea’, Individual, social, economic, political and technological developments 90h) have their own, distinet ‘pace’. The current fashion is for valuing ‘speed, the AuAHVPtION being that slowness is not just boring, Dut wasteTUT DI Fit Tie Tat ynesTTon th coVMMMERTTON as Teather soctal practices TATqM MAE TOR30 Multimodality Nat tumae cr vs Wa bum cede greet ssn onditions?’, ‘Under what conditions is slowness of pace essential?’. The pace oF tech- eal cane cannot possaly be mitrored by sll ietltions, ver though tha ‘Seems intended in calls to accommodate to every innovation as a means of furthering ~gitncies he ceasles restrucurrgsaForganizalions ar ope sush splat “thereqarement of individuals constant to adapt The more urgent question Isto reflect what the relation between technalosica instutloal, social and human pace shouldbe. Society cannot hope te mimic the pace ier in evetytchnolagieal nevetionrotsoul Hater ada, ta bly cil ica, urine a Pras ie ake pede The need for apt metaphors Wis essential iE o K herent in the theory being constructed am manana around Taming’, which T develop at various points in the book. yf “fll sassaemetactrs Alsons or aay ely rade, So maghors=i aizabvays ney made bl eee ecco FRITS EROMECGE aval. Slane ar rowedge are tools dealing wih prob ithe Sgemaker’s Tfe-world ol Sis ame caus anh ‘purposes, so metaphors too are ride Tor specific audiences and purposes, That make= it @ssentTal to Be aware of what and who the audrences are and vit the purpases are The fact that signs-as-metaphors and metaphors-as-knowiedge are tools makes i essential to exercise the highest care in the fashioning of tools. A sigoetpnor rade fr aay ainsi ts pos fie, aga ishing understanding. As one example, take the metaphor of ‘multimedia’. It seems: a SEE well Tor quick SEE WAITTOF queK communication. But what is or are multimedia? What does th: signifier multimedia refer to? What does it name? What does the metaphor project AAs far as I can see the metaphor of multimedia has much the same relation to thy Present communicational landscape as the metaphor of horseless carriage has t the age of the car. It is a naming of new givens in terms of old frames, thoroughly lodged in a previous environment where the media of print, of sound, of photography, and so on were all discrete. Each of these dealt with specific modes: radia with soun’ as music and speech and soundtrack; print with writing and image. Film was, in that context, already a border category as it brought together a large variety of modes with their then still distinct traditions of production. with present givens; to So-to.use Lhe term sauitimedia is to confuse past practice confuse the cultural teclnologies of dissemination —raMio, newspapet,-TW,etc.— with ase the culture chr Oloples ot dissemination “rad £; ntoprcbetern Iie communicaenal ensemble might be signTicant for Ts eanerirainee, =) V7 Jn Fameg —> fe ae conta and ently atte to otaMUaTy fica ces ‘uleitial promats tis his interest as trainee surgeon that turns any one of these — or ©? Jif nto a pronpETarRim- TCs his decision. Once turned ito a promot, his eet fares De rae a el ny asec ears fran tat res sofa . a se ae hans ta epee Vora Te aaa UTS Sonia 1. Wor changes tings: the too, the worker and that fl srian ae so ae orSSUTII-GEOTa ness tobe a least equally which is worked on, Semiotic work is no exceptions It is work it the dormall AMlontive to prompts of acl ical athe than a pcos king, Freq the saa ua canes proceed yaa seve work rear Mean maiein MMW" ihe ae oie te he Gad ot a eT com ris for SMe ertion Westy toheTie rar rater han ie eceean ir et retances Assume that we take the situation in Colour plate 1 as @ normal condition of com ff Soi ‘munication. Colour plate 1 shows an operating theatre; an operation is in its ear! iatonniles ta ant in this situation applies to the other participants; Woroutly, depending on their position, role, perspective Ti the complex ensemble — Communication as semiotic work: a sketch of a theory stage. A ‘scrub nurse’ is in the foreground. Behind her, to the right, is the ‘lead surgeon’; opposite him is a ‘trainee-surgeon’ ~ a qualified medical doctor training Sent ot thelr postion. Crucially, communication’ to become a surgeon. Behind them, separated by a screen, is the anaesthetist; fa Siperoting theatre ig raltimad: a gaze, a touch, e'spoken comment, a gesture, back on the right stands an Operating Theatre Technician, Representatives of fou Sh otin see ee ee distinct professions are present; each with specific traditions and practices, way: Piju pvt alma acts apron ToWEpTion of communication showin Colour plate 2 is a very different one. of talking and doing things. Their tasks are closely interrelated and integrated. The yp iy thing, eve mode fs not an issue. It may be that language as speech is assumed occasion i first and foremost a clinical situation; an instance of (communication in) 44 Jy-th modo of communication; or, more likely, the focus Is on a message as professional practice: a patient is here to be made better. It is also a pedagogi “edntont’ quite absteactly and generally, rather than on the material and semiotic form situation, an environment of (teaching and) learning: a trainee surgeon is here to yf iw inemage become a fully trained surgeon. Wf the Saussurean schema, yo interlocutors ave linked in a dyadic structure, One Communication is multimedal: by speech at times, as spoken comment, as instruc lig a message: the diagram andthe theory both suggost that itoriginatostrom / sore ea re Sas aL Teaching out for 31 WIN literlocutor’> NERC; There Its shaped into speech, seemingly; it is uttered; ion rac les communication js. response a rome a azz j Qght produce a spoken comment; that Teads to an action; [poking at the screen by and is met ey sn-antcument being passed, Communication has happened 1 space aaa acused on some aspect of the communication; 3 \, Fas ten thao a mess = that message asa prom {QUIET SF Timself. That prompt has been inter ‘a new Inward Framing —> Interpretation is ceaseless; it involves all tv participants here, at all Wines, though aifferently in each case.” “THETAFGer social event here can be framed in at least two ways ~ from a clinic and from a pedagogic perspective. IF we frame the event pedagoaically, as one 0! teaching and learning, the senior surgeon and the trainee move into focus. Questions flyupe 4.2. Saussure’s diagram of commnamreation om)34 Multimodatity the other participant receives this (spoken) message; and in that interlocutor’s ‘head It becomes the basis oF a respanse “Versions of this model had been active in twentieth-century conceptions communication; the sll haunt conceptions ef communication = er If nthe ser negation of many adaptations and alterations ofthat model. Glaus the domina conception of communication in the latter part of the ce on the Sende: eS Receiver scheina of Shannon and Weaver (1948), a model derive: fom lere the origi ive cause of communication lie with the sender, who ‘encodes’ a message in a code shared by sender and receiver, ser along a channel, to be ‘decoded’ by the receiver. This version recei in critique in Roland Barthes! (1968) article ‘The death of the author’, which insisted o ‘fiz dominant role of the reader in communicgbign. ——— “Tid Lemme te cosas ah 0) is s0cral interaction and i ound meaning, oriented to the processes! ‘igingad orale meaning teeugh eatin a ogee Seresetaan Sgnmakers a ther agincy a8 sol actor are pI TET dard with them the social envronrentsn wich they make sian. Ce ish or making meaning on modes and ther aforances Te ‘air condion and nears for RSanPaT AG meaTing= The mela and Use ace PHtheory Or communication MEETS TO WEA Wi te semiotic work done in relation to 21 thes ard withthe meanings which est. Questions ofthe Ked “Who des wht for whom are etal ys acs. racist rere, hs teonsfarmation of ther sell from the porspecive meaning. tn that proces the social” ar wiles and forme, 2 OO Dractces~is constantly articulated in (materia) sero frm: the Sot Callovated e-rgistered with semoticfcultaral resources. The process of (usualy Iminute)'ecaliration’ ensues that gaps between social structures and processes the one hand and semiotic accounts of them on the otter donot open to ready too far; and thatthe semiotic resources remain capable of representing St phenomena, This ceaseless recalibration acadues a serse of Tab asenss pf noticeable gaps hes sense to members of the: Poa That things are as they ‘nave always been ~AY_the-same Time, social interaction via semiotic means produces more thev ‘documentations, records, more than ralTications’ so to spea, oT Socal harge: ‘produces Tew meanings. Even the most ordinary social encounter is never entire predic, new in some way, however slcht, so that the ‘accomnis dations’ produced in any encounter are alwavs rew in some way. They project social | possibilities and potentials which clffer, even if slightly, {rom what there had bee before the encounter. As a consequence, the semiotic wark of interaction is alwg \ socially productive, projecting and proposing possibilities of social and semiotic forms, entities and oraces ch reorient, refocus, and ‘90 beyond’, by extending » el GI dr oom eit Communication: shaping the domain of meaning {onmunication can only be understoog it as an always complex interaction whodded in contradictory, contested] fragmentarysocial environments: whether liveen groups or between individuals, coming together front sacTaT Tocatfons which aways distine. in some PESpEETS Tr the interaction, Te SOCTaT divergences) rovide the generative dynamic of com- on. Inthe process cifferences are reshapadfransTormed Th temporary social HTotic accommedations. These in turn are the semiotically and socially {hctive force of communication: they project what is socially problema into a Hille ace’ and produce temporary recordings of the social and the semiotic state Witfars; in transforming it, they shape it diferent Iiother words, the huely-burly of social ife isthe generative force whch constantly ‘He Yehapes a society’s semiotic resources and in doing so documents and ratifies new Aielal givens. Communication and the resources made in that process, have the ‘rictristics that they have because both bear the imprint of their social environ ‘inte. The central point for the theory is: she soctal ee motor for communicational ijalic chance: for the constant remaking oF éuTtural/semfat resources; and forthe Minis tence ist that ‘the social has priority’ is to say, above all, that the forms, the snl thé contents of communication are social MOT, they are socially ve mms hed, that communicatiow TH social environments, arrangements Wil practices; that communication is itself a form of social action, of social work; and {et communication is always a response by one participant to a prompt by other AHtM pants in social events, Some element of ‘the social’ prompts me into shaping a Ngil-complen as a message — which is my response to a priar prompt. Individuals act HW Aoinmunication, prompted by and in a social enviranment, with social-cultural Foyourees. Hinicipants are embedded in networks of social relations with athers who make Hieanlrgs by making signs. Signs and sign-complexes — messages — have ‘shape’, ‘Wiuelure and content, representing the interest of the sign-makers. Some of the signs/ ‘Heyes ore taken up by participants in an interaction as prompts. As I have ‘Wiyested, the two central assumptions are that communication Is the response to a WeHP ad that communication happens only when there is interpretation’ (see Kress Yl Pais Leeuwen, 2001), Communication depends an the transformative/interpretative Lb av want iti an interaction with a message made by another ~ in Medici cs teroett ons is tninesesar oF eormenieson ber angen Diner commutation Biearuerce for is treory of communiation and We practice that oven | essage with the intention to be a prompt, by another participant, A message, ranges the attention of a participant in the interaction Aliention derives in the first instance from the participant's interest — where inter fis the momentary Nocusing’ of a social history, a sense of who Lam in thi $4418! iKuation now, as we as a clear sense of the social enviorenent in which the 35%6 0 a ‘Multimodatity prompt occurred, ‘Refuse negation both of a bath ofa soc aula. Tterpretation is central in communication and so, therefore, i the interpreter. A intefretalion ea rcporce to aptlur marae Ths careers, the shape of tat ccemngE CORRES Hr TT on wilt heir pettin happens, At HEE EET tnercTore always mb oF enor oF he "Ground as We prompt framed b the interpreter, with the resources brought by the interpreter. An interpretation the result of a series of teansformations in which aspects of the prompt and aspee of the resources brought by the Tnferpreter are shaped, jointly, into a pew senior: eit. — ~The theory needs to be able to serve as the basis for description and analysis c{ all instances of communication. In suggesting that the environment of the operatin: theatre could be seen as ‘normal’, [ am aware that reading the pages of a book cr engaging with a website are different kinds of activities, in some sense. However, or theory should, nevertheless, be able to deal with all instances of communication. “TETATGHT be Useful to say — simplifying massIvely — that theories of communication ve veered, broadly, between dyadic, interactive models, such as Saussure's, either more sociologically or more psycholocically oriented; and dyadic, unidirectionat models such as the mode! of Shannon and Weaver (1948). Both imply social rela ns: Unidirectional models tend be more authoritatively oriented and interactiv models less so. The power relations implicitly coded in ‘authoritative models’ requiv: thes “ to recover — to decode — the meaning encoded by the authoritatig Both the models mentioned earlier were conceived of as monomadal ~ one depen! ing on the mode of speech, the other on a modeticode) based on the materic! affordances of electricity, miodulated into ‘code’ by relevant technologies. Or, it might be better to say that the issue of made did not arise: it wasn’t ‘present’. That has remained 0 right into the present. As mentioned, both models have been challenge! in different ways — for instance by the Gournalistically based) model of ‘uses and ratification’ (Blumlee and Katz, 1974; Palmgreen et al, 1985; Dervin et al., 198) /—n which users and their needs are in focus; and fundamentally by the explicitly sem \ otic theory of Roland Barthes in his ‘The death of the author’. From that perspectiv my sketch here is an attempt to give a social-semictic articulation to the critiq offered by Barthes in 1968. To restate: in the sketch put forward here, three assumptions are fundamental ‘ommunication happens as a re ympt: communication has happened wh Tras been an Interpretation: communicatian is alvays multimodal Betas Interpretation is cortral, so therefore isthe interpreter; without interpretation there {ig_conmunicati t vompt_whith Coristitute the ground on 2 yes Staae on Sean ike Thay comorantor arate wistin staes. by the interest of MTINTTal maker of the sign-camples. the thetor ——ET—e = Communication: shaping the domain of meaning 37 ion of disseminating the sion-complen as to be taken ii-o1amn. In_stage two, the iferest and attention of an interpreter is in focus it Joni to selection of whats criteria Tor the Tterpreter Tr the Tita messace and to Nag often ape oT eat re The seTectod aspects of the nilal message as a prompt mh is, sub Seen, nerrete Ts mea made Th hat nereretavion can become Te Basis i aker terete sequence ofa prior promo and is Infergretatin») new sia | Mle Cased on We makers inferest and a sree of the auciences characteristics) einlenced as prompt =» te itera sent Ins does not make the semiotic work of the rhetor and the audience’s/interpreter’s mt Tot the same kind of work, with the sa Vaud chara hesame results. After all, ‘setting out the around’ ‘ory different in all kinds of ways to ‘making Selections’ from that ground and Jiderpreting aspects of it. The rhetor and the interpreter both perform semiotic wark; Wl Is liferent work with eiferent eects, Oden structures, in genres, is ol 1 gion- complex. com, a | odie ‘Heading’ and the reader’s design of meaning Jip model of communication rebalances-power and attention, with equal emphasis WW the fiteroreter of a message-promot and the initial maker of the message, the Mawr tose afr of eon eT Ot Hwatie, in the ‘reading’ of a quite traditional text; in the engagement with a website; 1 iting an exhibition; or in other environments. To show this, I will discuss two witint examples: one, Colour plate 2, is a website — the homepage of Children’s BBC; {Nie other, Figures 3.2a and 3.2b, comes from a ‘visitor study’ in a museum (Selander, 80008) Insulander, 2008; Diamontopoulou and Keess, forthcoming). Heading is communication. In mary approaches — especially in pedagooic ay f ents — ‘reading’ has been thought of quite analogously to the Saussurean mode| Figure 3.1, and maybe even more so to the $ -> Mi» R schema. ‘Older’ pages ‘wilting embodied notions of autharity and authorship quite compatible with those ugilaley the author assembled and organized knowledge on behalf of readers and Myplayed that as a Gwe ordered) text on pages of print. Readers engaged with that Yulas knowledge in the-oedorset-ont-b the author: an order OF TS, OP Warts On The of arrangements of words as syntactic elements; of genres; oftexts-as-qenres, Hipage; of chopters. That had been the embodiment of a unidirectional, herarchical, Hib roltionship in which power rested with he author38 ‘Multimadality Contemporary pages, whether of information books such as those produce’ for instance, by Dorling & Kindersley or of websites such as the Children’s BB« homepage, Colour plate 2, are designed on the bass of a quite diferent social relation of author, reader and meaning-making. Above all the texV/age is shaped generical with the assumption that engagement takes place on terms of the (child) reader’. interest: an assumption based on 2 very different social relation to that of th “traditional text and its genres. Unlike the traditional page, designed with a given arderfarrangoment for the reader engagement, this site—a ‘homepage’, which has visitors’ rather than readers ~is give an ordering by the readers” interests through thelr (ordering-as-Jcesian, The readers’ Interests determine how they engage with this page. In doing so they design not so mu a coherence for this siteMpage’ as an ordering which represents their interest. T sequence in which the elements are ‘read’ and ordered by them accords to choices whic! veflect their interests. In effect, the readers’ interests, reflected in the manner of thei fengagement, provides for them the design for this page: readers redesign the pe (see Kress, 2003). This notion of ‘reading’ accords with the theory of communication ‘outlined: material is presented; and readersiviewers shape their ordering of that material. Formerly, an author, asthe initial maker of the text/ message, had proviced that ordering; now that is work shaped by the reader's interest. Clearly, this chome)page fs designed. In its design it embodies and assumes the necessity ofthe reader's semiotic work. To an older reader the page may well give an impression of incoherence; for its intended reacer it provides what this reader expec to:do (atleast some of) their own semiotic work of design. With this site the question of modes is unavoidable, Ata first look itis not at all evident that writing and lexically conveyed content do provide the main path of engagement for the reader/vsitor; or whether image does; or colour; nor whether placement of the main semiatic elemen —for instance, in terms of a left-right and top-bottom order of the traditional paye determines an order of engagementreading for the readerNisitor. In the page, 2 i is encountered on the internet, the cartoon characters along the band at the top of the page move. Movement as mode provides a further point of choice; again there may be generational factors at work in shaping preferences for engagement. It is 2l:0 important to observe that a formerly profound divide between the ‘programme isn and the programme itself has become blurred. Whereas differences of genre hil usually been done within one mode, by a specific selection and ordering of semio' ‘entities ~ syntactic, lexical and textual ~ here the generic differentiation is done iy means of modes: the mode of moving image makes the programme isting itself into performance. In some ways this is akin to a crossing of the boundaries of social semiotic ‘domaine’ ‘work’ versus 'pleasute!, for instance. But the generic lifermce marked modally is more profound. In considering the homepage, the question of ‘yeading’ and generation became central one. If it is the case that members of a. ‘younger’ generation (quite likely by now to be separated by two generations from generations more ‘traditionally oriented) read according to a design of the reader even though on the Communication: shaping the domain of meaning ‘yeni of someone else’s message ~ it is highly unlikely that they will not carry this Jiwhitus into their engagement with any semiotic domain or entity in their world. These ‘lyorgent practices and expectations about reading by readers of one generation Fnwelirs) ~ and another (students) is, I assume, ane of the major problems about “Feaiing’ in schools. It might be objected that reading for pleasure is one thing and What readino/engagement which takes place in institutional sites — of work, of Jfesion, of school ~ is another. For one thing, the issue of power is overtiy and ‘{istently present in institutional sites. Some readers will respond in alignment with {iver vw others not; with generation playing a major role. Once develope, a habitus ‘Wf Feading is likely to be applied in all instances of communication, though the ‘Willigent factors of particular environments and the specific characteristics of ‘HeiiiOr will, as always, play their role. My fast example here ~ Figures 3.2a and 3.2b ~ comes from a research-project on ‘Viilor studies in museums (funded by the Swedish National Science Foundation). It WH) fonducted in Stockholm and in London (at the Museum of National Antiquities ‘W Mockhoim; and at the Museum of London, in London). In Stockholm the abject of WE Feweareh was an exhibition on Swedish prehistory; in London two exhibitions were ‘We liited ~ ‘London before London’ and ‘Roman London’. The research project aimed. WW ive Insioht into how visitors make meanings for themselves of these exhibitions; il 0 a lesser extent, what sense they did make. Unlice schools, museums tend not WH Phireike power over their visitors in their engagement with an exhibition; even HON they might wish to be able to so. That makes conditions of learning different Wilpon vay. Ji Heth Stockholm and London, visitors were invited to participate as ‘couples’: as WAipnront and grandchild; as friends; as married couples; and so on. Those who Hee the invitation were given a camera to take photos of objects or displays which rr, ‘Their interest. They were videoed as they made their way through the exhibition. {He Gonclusion of their visit they were asked to ‘draw a map’ representing their WHEE Of the exhibition; they were also interviewed briefly. Each of these four ‘takes’ ‘4.1 means of obtaining material that could serve as data to gain insiht into “Hf eragement and serve as evidence of ‘learning’, Two maps, from the London yuh drawn by a member of each pair, Figure 3.2a and Figure 3.2b, are shown titow Mi Wahibition is designed; its designer(s) have specific aims: to show objects, TW, reconstructions; to tell stories of the prehistory of a nation, and, in that, to ‘WHieve social, cultural and maybe political purposes. While these aims tend not to be ely sated, in interviews with curators or the curatorial teams it was clear Hl Ulweke exhibitions were the result of much discussion, framed by policies of the pate, (ommuinicationally and semiotically speaking, an exhibition is a complex sign, ‘Wialined 10 function as a message. It is meant as a prompt for the visitors’ engage- WHEN. Pedagopically speaking, ay exhibition presents a curriculum for the visitor/ Jeni (0 that context, the 1 Way here are indications of the visitor's interest 3940 Muttimodelity Figure 3.24 ‘Map of a museum exhibition ‘Londo before Londen’: “the prehistoric camp’ Figure 3.2b ‘Map of a museu exhibition ‘Londo before London’ ‘Heathrow’ which shaped their attention; which in turn framed particular aspects of the overall (design-)message. Semiotically and communicationally speaking, the maps arc response to this prompt. Pedagogically zpeaking, the maps are signs of learn. ‘The ‘maps’ are of interest from any perspective. They do not, of course, provid full account either of what the exhibition offers, nor of all of the meaning made by either of the map-makers ~ one an 18-year-old wornan, one an 11-year-old boy ~n their visit, The photographs they took offered other material for data, as did the king video and the final interviews, Neverthelo tra the maps provide one teas on Communication: shaping the domain of meaning Meir engagement and transformationtransduction of aspects of the exhibition in each ‘90 They show a clear difference in interest; a consequent cifference in attention and Janing; nd distinctly different interpretations ofthe same large overall message. The W exhibition was turned into distinctly different prompts by the interests of each of We two map-makers. Most immediately they show a starkly different sense of what a ‘map’ is, what it ‘hit mean, what it does or can be, based on different resources brought; inthis case ‘uikaplions of what might count 2s a map, and of what is to be mapped. In neither ie Is the notion of the map a conventional spatial one; the exhibition had been ‘Aiyajwed as.a large ‘room: with relatively indistinct sections. Some questions posed, W Show map-makers, seeminaly were: "What was the central topic to engage my slleniion?’, ‘What objects cid 1 find most interesting?’ The maps are a record of Wwplures of the exhibition, not of the layout ofthe exhibition; a record of the objects ‘il tableaux that seemed most salient and interesting. In the case of Figure 3.26, Wie notion of map is ‘conceptual/atfective: one question posed, seemingly, was: * ‘Whit for me, was (the most) significant cbject/entity of this exhibition, along with ‘Hie others and how shall I arrange them spatially to give a sense of ther relative ‘iypifiance?’. For the first_map-maker, that which incited her attention was {faved along vith some of the entities which prompted that; for the second map- Heke, the map vas set out as a visual display, a spatially ordered arrangement, in ‘Wii lle most significant object has the mast salient position. Fran the same displayimessage the two visitors had each fashioned their own 1 Jesign and a distinct interpretation. A number of questions jostle for attention ‘Wwe interest has been dominant: the curator's or the visitor's?’ “Has the SiyAlor suecveded more in one case anc less so in the other?", ‘What interest and what Heehuries of the visitors are evident in these mags, ‘Has one of the two visitor ‘ples failed in their ‘reading’/engagement, or have both2’. These are questions ‘Heal aad urgent for a curator; they provide one motivation for engaging in ‘visitor Wille allure’ or ‘success’ are probably not concepts the curator uses in relation Wi tle visitors; though ‘effective communication’ might be. They are central and Will uestions within this theory of communication, irrespective of the site: Wier In. museum; a school; a site for public information; a site of professional aller wich as surgery; or in an ordinary conversation. What conclusions can be drawn from these examples for this theory of commun- ‘Allon? Taking the two maps as.an instance, one response would be to say that Fewpo ings made from the exhibition ~ are unpredictable, individualistic’, “Wihieeivo’, anarchic even. Or we might say that the most significant criterion isthe Valles Interest. Both ace unsatisfactory accounts: tho First amounts to an admission Wal hore is Ho theory that ean account for differences in engagement and reading Wie second ionores the role of the maker of the initial sign-complex/message in AHowwina the message/promyt. That, alter all, has ‘set the around” Fromm which the Seeminaly ‘anarchic’ or Ssubjectve" responses are produced, a fact which is demon- eat bots maps s of the exhibition, such as the model ) a2 ea ‘Muttimodality airplane at a neolithic camp. While the response to this complex message/promp! may not be predictable, it is not anarchic. 1f we stay with the perspective of th interpreterfrecipient, there seems ta be a sequence where the recipient's existing iereseersta bora vequence where:the:ren|plentis cxlsting interest shapes attention, which produces ‘engagement leading to selection of elements from the message, leading to a AW cm is He iran ofthese eleven, ch fads their re a new ner sgn f ! nf Men Aso ‘The interpreter’s interest produces attention; attention shapes the form of the engagement this leads to sles bring made; te selections are frame there the subsauet trarsfomationard transition ofthe elements nthe ame; ar, that, the inwardly made) sions produced. The sequence reshapes (aspects) of th nial message, the around, laa prompt interest isthe motive tore: isthe bas {or atientian tothe “qroundconsitued bythe extn, fr engagement with al ‘ground’; it shapes selection, transformation and transduction; and interest become evident nthe new sg, the a. The maps are subjective inte ee of being bast onthe maker's ners: yeti exiting the prints of the makers” Inerest Clear relation tothe grourlprompt, the maps ate principled. They are not anarei This Ureory es welier te sipnifcanceof the semiotic work ofthe make of he tial mesons, the reer, nor rat of te Interpreter: the work of cesion ht fashions the ‘grou’ on which interpretation ake place is one essential element the t-par structure of he proces communication. Forte aragst as rh as cverydey communication, some aspects ofthe dein of te Inia ressape rust Trobe Sent camel nf ak ese Oe eas Wore inerprlaion, Th semiotic work of transformation and rasan whieh under interpretation is te ste of the production ofthe new’. It Is work whch fad semiotic enlies whichare always new, nmovative, rete nat because ofthe pei ef the patcponts in te tteractin but because ofthe very characteristic forms these nracions, nuh one cancption of he worl the ermal express interes of one parteipant~ ime bythe diferent terest fe ieterioctor. 1 he Alek out ofthat clferncey wih pow and ws paying ena necicrl new conception is arrived at. --Fealled that Yan accommodation’ to indicate | both its provisionality and i ence cuof different interests, power and affec| Practically, the curator INirto ask what consequences follow fram this concept of communicatian far her or him or for the museum in strategies of design. There ve specific questions: ‘What path had the curator constructed for the visitors?, “Why «iN! how was that path followed or not followed?’. The curator might well want (0 understand the principles underlying the differences in the path taken: in attention The concept of interest provides the overarching reason. As i 1s in interpretation, Communication: shaping the domain of meaning 43 Jnppened, one of the map-makers was an 18-year-old student from Germany on a self- ‘uHyanized 'study-trip’; her interest is expressed in the detail of attention to the objects, ‘Wi lior map; the other map-maker was an 11-year-old boy, a highly reluctant visitor. Hu in citferent ways—each time precisely and specifically ~ that is the case with each el tie maps a Ue, oem seta tee ‘Frovisionality in communicé ‘and design, newly fontigured qe like any semiotic entity — a website, the operating theatre, a book, a i's drawing ~ is the result of design. Design is shaped by the prior analysis provided Ihe rhetor through questions such as: \Wllatis the environment of communication2 Telations of power ace at issue?’ Who are the participants in communica oo i val these, the rh in-ifferent ways, with ciferent ions of purpose, power and affect ~ the lkaly Yesponses-ot-aTmagined re Tree-dinerstonal objects more Salant, Tore “Slane”, are le than writin caption o than longer written accountfexplanations?”, "Are ed scenes mg 3 than tse dimensional tba? What fect es [7 have in creating root and affect? “1s the distance at whiphrvitors are able With objects —for instance, whether they are sented en object, © er?”. The es a glass panel or a rail — a significant Wok ot affect “Addressed in all aspects of the éxhibition: affect modulates jie ota “Agfest oblate te seen Filed, I ae Cc (oransoclra sivana totem oa REairensional) image: though elements are drawn in to the ‘mag’ from other Mat oF ‘map’. The big skull at the top left of the map is located at the very beginning Wile exhibition. Selection by the map-maker has changed a large room with many “wher image where few objects are drawn into an entirely new coherent ee oT Heripby-transtorcation ~ changes in ordertnerand ConTTgUTatIONS oF efements within the change from meaning expressed in one mode to44 Multimodality Curators might see themselves first and foremost as communicators and their response to these maps might be shaped by 2 wish for better, more ‘effective’ co munication. Just Uist cneatng theatre, te museum more anc more becom aateatih is Soubly famed 3s a etveatie Governments See museums as environments for social education. In both framings — a site of entertainment or of learning — curators are bound to be interested characteristics oftheir audience which have an effect on both. In the present unstable social and communicational environment, toe category ol abl ishing the characteristics of communicatici as 10 be treated — conditions of communication: ‘What are my purposes, "What do I wish ‘communicate?’, ‘What are the characteristies of my audience?', ‘What are the best sources to do this, ven the characteristics of my audience?”, What relations 0 fower abtain between myself and my audience?’ ‘What resources for communication ‘are available”, “How are these resources best arranged to represent what is tole communicated?’ and the larger framing question: ‘Am I attempting to educate, entertain, or both?’. These are essential questions forthe rhetor, in any environme:i at al times. Inthe past, convention had provided ‘routines’ which obviated questior now they are starkly present and need to be addressed, newly each time. Rhetoric needs now to be seen in the two-phase structure of communication, witheut which it cannot deal with contemporary social givens. Communication is joint ay ttcipracal wack The sign-complex which is sketched by the rhetor on the basis 0’ preceding analysis is elaborated in detail by the designer and is then given mate” al form by a producer. The sign-complex is presented to the audience as a (comolex) ‘message. Ideally, members of the audience shape the message as a prompt and ear! transforms that prompt into the new inner sign inthe light of interest and the semio\i: (and wider social, cultural, aesthetic and ethical) resources which each brings ence To cr as eee ng ene Pe attend to and endaae with and interpret Ba prompt for he 1 Crnmurtcaton rest on both phase ital work ofthe rhetor and The subsea. engagement and interpretative work of the auience, seen as interpreters. To cal part o agement is to use a relatively ‘neutral’ naming; it @ESErt« ‘fended by she-tetor= It doas assume the attention to the rhetor’s message. Whi! aitention and engagement Tead to section and the reraniiaa ab ubal has been sete] ae Communication: shaping the domain of meaning 45 Ln leersny Crane off an, Food Comme SL Lich Jeg ‘ahr, whether of the initial maker of the sign-message oF the interpreter of the Wael. Or we might use the label chetor for the maker of the message that initiates ‘Heeauence and becomes the around for the interpretation; and use the label interpret WW the sign-maker who interprets the prompt. That brings a distinction between ) ‘Willing the ground’ and ‘shaping and interpreting the prompt’, both seen as the ‘Hub iny oF signs. | adopt that naming ~rhetor and interpreter ~to mark the distinction Weipeen setting the ground and interpreting the prompt. Each of the two has a distinct Wid! function and effect. How each of them is carried out is a separate matter, Ives sway, decisively, from a conception of successful communication measured TW ietins oF the ‘closeness’ of the interpreter’s sign to the sign of the rhetor. Social present twice inthis framework: theough the interest of the rhetor, FENG Sonse-OF the social characterstiesG fn Wo the rhetor; and through the soctal location and interests of interpreters. Ine distinct vantage points oftheir social stories and present socTaT post ons, Hill bing their sense of the demands of the immediate social environment, ‘ieMentational and communicational requirements of the phenomenon ~ the ever 1 thject = to be communicated and interpreted. Each brings their culturalsemiotia\ ‘Peerevs and values. iis conception of communication gives appropriate recognition to the semiotic rh of curator and visitor; to teacher and student; to writer and reader; to the person ‘WH oloads a video to a social site and to those who respond to that video. It provides 4 Wines (rare of rhetorical and communicational work, modified according to the use. , writing was the dominant made, carrying all the information Mijught tore essentiat or central to the curriculum; image had the function of Mlustyatina’ In 2002, writing is one of two rades with seBriing equa semTate st TAA no longer has the function of ‘illustrating’. Image shows those curricular Wailala yuich-ava.test_— most aptly — represented in image. Writing deals with Aestion ws well, though not the physiology of digestion as before, but the bio. shieaniitry of digestion, a748. Multimodality Anil Seiten» Sorin Mana 18. Dent at stn 9 Taare tte ee ee sat sp iag ted dace ee ot SRS Shay bee pen Figure 3.3 1935 Science: digestion In 1935, writing was seen as a sufficient means for carrying the meaning ‘hl had to be carried; image was very much a supplement (Barthes, 1966). By 2007) writing and image are treated by the designers of the textbook 2s offering spec i affordances; and these are utilized in relation to present ‘curricular need” and \ly assumed characteristics — as likes, dislikes, preferences — of the audience. The dominant view, that writing carried all that was crucial and significant, me: that no real attention could be paid to what image contributed. Figure 3.3 shows {hi Image did in fact provide information not present in writing; but that was not in foc ‘The dominance of the mode of writing provided categories both for representa’ and categories for "recognition’: what was not highlighted in the lexis, the syntax 0 the grammar of ‘language’ did not emerge into visibility. So the fact that the ina ‘made specific what the diameter of the cesophagus is compared to the size of the chi cavity, was not visible’ as a significant contribution by the image. Communication: shaping the domain of meaning Piyire 4.4 2002 Science: digestion Jashioning social and semiotic domains: ign eerste: and communication are distinct social practices. Representation on my faterest in my engagement with the world and on my wish to aive ‘erin realization to my meanings about that world. Communication focuses on my ‘ened to make that representation available to others, in my interaction with ‘The dual frame of rhetoric and design permits both: rhetoric as the polities Aonimunication and design as the translation of rhetorical intent into semiotic yiwortation. Rhetoric is oriented to the social and political dimensions of n;_ design is orient ~The retor has a political purpose: to bring about an alignment between her or his cn With its ideological position and the position of the audience with their Jogieal position. The designer has a semiotic purpose: to shape the message, using Wie available representational resources, for the best possible alignment between 1 Wirposes of the rhetor and the semiotic resources of the audience: mediating the Feauves of what is to be communicated with the resources and characteristies of / Hw mitionce. Benin as esigne hase sino intaess wile thle tasks ifr. The thetor environment for She oF he need pication-as, 49 \ /| Multimodality Communication: shaping the domain of meaning S1 ssuch-that the audience wll engage with it and, ideally, assent ‘0 swoment:Chird, ny interest is shaped by mn experiences overtime Te That UNS tsk Te cevnr asses wha nic apreeiaenal” | oT arts: sn er eure fod roy meres OT resources are valle, wif aul uacerstandig ofthe. celars needs and ain Init whet SARC TO RTENG OTST social ervrcrnent RTE ARETE STE {uch a way that the rhetor’s intevests, needs and requirements, are met and make th on to this phenomenon or object. The question tn representation Is What are the best possible match with the interests of the audience, in ynment where thy A the a Tor siving matertal form, material realization to my meanings’. FESOUFES Tar doing so are usually Inadeouate. Thetis 2 @rfoipeack. Ac seca (al we representation envrarments Change, eo Ue designs oF Te message Ed TS Change. Tha se moter | EQpmmreRTOayby contra, to ut the meanings to vbich Lam aving mater hich dies snag charge ine wth social change STG Cs text) into a THEPRTTOT wEers in my enviroment: to make * ox @verviay cammunicatiop the two tasks and roles come together in ori ingnings known To my a5 Tmierest has shifted fom person, so that rei? Is also WaT fe aptess oF ataral Tor ta reaizt Ay means egos pone Tar andere are-ParCe pars Th he That: ime with oi process of communications the socral relations with the audience = STEEL ESSE TE TOT TETTTETTaMI_ oF mode with the phenoMenor Hisehanger What Is rip relaTOM Te TrGse with won’ I am communicating?”, be represented and communicated. And designing, too, a communicational enviror oy members of my more Immediate soctal group Or are they more distant”, ‘ent congental to The audTence’s interest and for that which is to be communicates for instance, designing ‘everyday realism’ for a younger audience in school or au 2 greater degree of abstraction for a scientific one; or with aesthetic considerati uppermost for particular audiences. Always too, designing matters of affect, whether lighting in a room at home, in an exhibition space, or for a film; designing the messarx q also for the apt medium of dissemination and designing the spaces of display within Holl representation and communication are social processes, but differently so. that. Fjrwentation is focused on me, shaped by my social histories, by my present social 7 The exhibition at the Museum of London, “London before London’, shows amon) JiWi, iy my focus to give material form through socially available resources to many other objects, dioramas, tableaux, etc. — neolithic tools found in the Thame iw element in the enviconment.. Communication is focused on sacial Cinter-Jaction ‘They are cisplayed in glass vitrines, lit in a bright bluish light: much as they might ov | J go¢lal relation of me with others, as my action with or for someone else in a in an art gallery or in a jeweller’s shop even. In the exhibition on Swedish prehistory | elf social environment, with specific relations of power. Interest remains central, at the Museum of National Antiquities in Stockholm, the lighting employed for tiv | Wal Wy focus, its direction and attention shifts: from ‘me and my focus on aptly rooms dealing with the equivalent period Is ‘low’, with greenish and reddish/brow | ¥ajjaisenting some entity or phenomenon to my satisfaction’, to ‘me in inter-action tones. The one exhibition seems to want us to see neolithic artefacts as objects of | Wil) others in my social environment and my focus on success in engaging and beauty; and to think about the people who made them in that ight’. The lighting of | jMawiadiny others’. Representation is oriented to self; communication is oriented to the aher exibition encourages us to se the periods diy wsbe fr us, ort UMM, Representation takes place ina social environment; communication con. age’ tous, even though it also displays artefacts of great delicacy and beauty in srl! | iis social environment. SiansCas Trays Shap , alass cases integrated into the tableaux of the rooms. Wwleyeats: by my interest in aptly realizing my meaning and my interest in aptly ‘of rhetoric and design are neither exceptional nor rare. They are part of | uiyeying it to an Other. the everyday, mundane, banal, unremarkable business oF Conmmcafion as rich Wwlununication is a quintessentially social activity. It may be that as humans we ‘aLTimes part of ‘heightened’ occasions of interaction Servant OF Tetoric | gapulefined by our need to communicate as much as by our abilities to do 50. Certainly, ~"or to put it differently: the political and social interests of the PHetor are the | Willwn@ is an effect, a result, of communication and not possible without it; in turn, o the semiotic arrangements of the desire” | wjiyiinication is framed and shaped by culture and changes culture in the process ‘The perspective of fepresentatiy, asks: ‘What is it that TWish To say, write, g&tae; | WE wyjiwnnication. Germanic languages have words Such as: mittefen in German “exoress”,at this poif?™"What is my interest at this point in giving material sare | Kill wily Yellen = to share) —Yo have and share something in common with ya, and form to my meaning?’. Representation is focused on myself and_my inter We Hake you and me the same in respect to some knowledge or feeling’; Mitteflung in _Cegnnounicatioy is focused on iny interest in tS relation to others, With representation | WePFAN ‘sharing sornething about myself with you in respect to this message’; and there is(irst) gomething to whis alization, making some [NMAMAlende in Swedish ~ letting you participate in a part of me’, as crude translations. rnaganing fangible in he take" on wliat I wish to represent ari Filey te s0 basic and common a trait for higher-order mammals as to qualify as out of my interest interest directs my attention to somelhinu Ihab now engages me, a yitlonal ofthe species, human as well as other‘Moltimodality Communication: shaping the domain of meaning 53 Being social, the conditions for representation and communication chance wit changing social conditions; at the same time, representation and communicatio Hiclng (what are now non-existent internalizations of) boundaries of ‘public’ and constantly change social conditions, though each differently so. The social condition ts the notion of Your space’ has broken dawn with the breakdown of a sense of set the ground, they lay out the arena, so to speak, for representation and commun’ 4 4uiic’, The British state responds by criminalizing such actions (through the ever Cation. Communication constantly (reconstructs this social ground, the soc! MMMM frequent issuing by courts of “ASBOs" — Art Socal Behaviour Orders). An ces ee a earch, Choire Nia issued with’ an ASBO is constrained not to engage in certain actions, thatitatways potentially changes distributions of power. Potentially at least it mak: ing for instance; or to stay away from certain spaces. In other European societies ‘Toratmestin Folically- proleratc. Representation constanly_comakes, th Yall be treated asa sotal misdemeanour and dealt within that fashion From resources for making meanings and, in t@ remade resources, shapes those who ‘We could attempt to characterize social conditions, construct social taxonomies remake them. That_is the eff resentation 1 The COTBaM ser" 7 ‘{juulate their likely social and political effects, and from that assess conditions. effects on power and identity, communication and TepreSE Hore to be kept clean. In the UK, public disapproval no longer works as a means jwinication Iiyportant question is whether there isa need to distinguish between presentation ‘presentation. Is every sign that we make a re-presentation in some way of (a) ign(s)? Is every text a remaking of prior texts? At one level the answer has ‘yes. The model of communication that I am proposing here assumes that nication begins as the response to a prompt. There is the possibility of seeing hig either as constantly transformative, as a constant remaking —the position SF nopt; or we assume that sign-making, at times, proceeds from a fresh start’. ‘Hille one case we acknowiedge that our utterances work on the ‘ground’ that has ach vifferenty. mde interest, active in the world, Tt constantly changes the resoUrCes and the potentials For-configuring how the world Ts construed. Tn. CHangTNg’ the TESUUPCES or SEM THE WOTTE, Te CHANGES WT at any One moment to be seiticd knowledge. That makes representation ontologically and epistemological prose" atic. Communication reshapes, (reconstructs the social environment; it change ‘the potentials Tor acon, OF agency, OF Mose wto-are-pa re process ol | faajepablished by the form and the content ofthe prompt we have received. By doing ‘ssmaunication. That makes communication politically p gis ane Sa We we ourselves as always connected to, integrated into, the (prior) actions of ‘omalexes are shaped by both and hence they are always problematic for knowledr and poner. /-__Theeffects of communication ar both moe overt and more key toe subi policing than is the case with representation. *Policng’ may range from an unnctic adherence to conventions which have become ‘naturalized’, ‘made mine 0 a strovs \ enforcing af them, so that I hesitate to incur the penalties of contravention; to outro prohibition on engaging with oF producing specific messages (censorship), wt penalties for transgression. It follows that members of a community adhere to or challenge conventions for quite diferent reasons. ‘Narmai times’ might te those when conventions are kro are ‘second nature, by nd laroe regarded as essential and generally adhered to, « | amatter of course plo vert porremg Ts needed )Where they are not adherd lo, penalties may take the form of disapproval in ess normal times, penalties take sev" , Teumiwhere power & used overty to force acherence, A banal example might |x “ittering’. ACBottom, “littering” is a matter of the boundaries of public’ and privat of ‘purity’ and ‘dirt’; while the private space is kept clean, the individual fh no responsibilty about the public space (Dauglac, VORA). Fuery act of ciscardiny 9 sweet wrapper by dropping it on the ground s in that sense an act of marking ad communicating boundaries of public and private. There are, even now, socteties 10 | Europe where, by and large, these boundaries work such that itera’ fs not significant issve. Iti held in check by public disapproval and even more by interna ized boudaries of ‘public’ and ‘private’ *p a Your collective space’ an | liv the other case we assert that we right set the ground newly, in our own 1, in our own right; using, of course, existing cultural resources of all kinds, ting them to set our agenda newly. The concept of representation places us in ‘Heaweless chain of semiosis, which we reshape in our signs and texts, but reshape iv the limitations set by that chain. The concept of presentation suggests that atleast at times ~ able to step out of that flow and may wish to start with W agenda, ip a point of ideological, social and personal choice: to see ourselves immersed ‘ceaveless flow of sernfosis ~ both inner and social ~ and yet wish to frame an ment with the world and its presentation in such a way that our agenda, our #1), uF interests newly set the ground for an ensuing chain. Here ideological shapes the theoretical frame. =A social-semiotic theory of multimadality 35 Wuking of signs are made in social interactia the semiotic 4 Asocial-semiotic theory Uhre the form ofthe sian i on Fi ; ietiag hl to eld tna ese te fe of multimodality [ii te nurs oe caer fhe nearing It) Figure 4.1 below, a circle is used ~ by a three-year-old boy — as the signifier to Bes tre crewing, several crcles-er-wheel serve oo apt siglo express We signified ‘car’. Figure 4. 2 is a drawing by-a six-year-old boy of two footballers ‘squaring up’, to From lingutstie toa multimodal sacta:semiotic theary Wy the versclar Be positon ofthe vo frs an gt ster terres = of meaning and communication Hynified of ‘confrontation’ or “opposition. The drawing is &sfan-complex made with We vival node Toregroundes and carrying major fu Like al signs and Multimodality names both a field of work and a domain to be theorized. Anyoxi| jp complexes\ is a metaphor, newly madéTin €Social-semiiotioXeke on represen- \working inuttimodally needs to be clear what theoretical frame they are using; «| Yaion and cornmonicationgall signs are metaptogs, always newly mage. Init ‘realist! ‘ake that position expint. SgsTar-semiotic Theory is interested aT i ation this connects with aspects oF the eurrently popular theory of metaphor oT forms. Meaning arises in social environments and ih social interactions. That mish wd Johnson (1982; also Laka TTS Tir soctal basis However, the social Tito UR Sourze, TE OFT TIEPATOT OF WEARTAG. TE theory he lols departs from the cogntivist approach of Lakoff and Johrson. “the sociat is generative of meaning, of semiotic processes and forms, hence the theory pb ond. Is a social-semiatic one. dahon. The core unit of semiotics is the sign, a fusion of form and meaning. Signs ex's ral nego Ui odcerecd ence oto taeee net Gy rani oa sgnaorp, aes tr heel cen Tei) ‘he active making of signs in social (interactions — SITS APE TAGE reer than 0 V “i fice. on sien ane rae War sla ose ie OF severe Temare wed — ‘inalshessociasemitc theory from other forms of senToTRs. rrsoetseri | CN coun of ening, aids, wide social Wve socaly shaped, cates | social environments, using socially made, culturally available resources, are agen and generative in sign-making and communication Figure 4.2. “This is a car’ There are several, elativaly distinc. al Sena rivet ] the writings & aViewed from a relatively abst | s \eFGh they end to cfer in relation to one issue: whether they base themselves on Y —chinguistiyor the Semitic perspective of Halliday’s theory. Wie there 15 2Oree? i BUEN Th broad soettne ane THaeh detail among provorens of either of the \v possibilities, the differently placed emphases do have significant effects. The thei] a , lyf used ee acops the seme perspective of Halliday theory, Rater tan ae to construct a compromise view, from here on in | use the label ‘Social Semitic refer tothe aporoach set out in Hodae and Kress, 1988; Kress 1993, 19973, 200) 7 Kress and van Leeuwen, 1996/2006; Kress and van Leeuwen, 2001; and in respects, van Leeuwen, 2005). It rest on several fundamental assumptions: sigs. ; aluays newly aden soll latracln, sons are olvat at aca fla Oe PA | (of meaning and form: the motivated relation of a form and a meaning is based of ai Figure 4.2 Football npis2s Out ofthe intorest of makers oF signs; the forrm/igniiers which are used ie iy Arsenal vs, Chelsea’6 Muitimodatity A social-semiotic theory of multimodality 57 jorrelates with these social features’ or ‘the occurrence of this form points this social environment’). Correlations show links(but provide no inguistics, pragmatics and a social-semiotic approach to representation Before elaborating this approach, I briefly want to ‘locate’ it among a few disciplinary relatives to provide sore orientation Given that thinking about meaning has in ray Nays been shaped by a linguistic understanding of Tanguage, it might help clear the | decks ~ soto speak ~ of some longstanding notions around meaning and commu cation to sketch some differences in focus between a linguist, a pragmatic and » social semiotic “ake on representation and communication. The orip of the Tinguistic take’ on communication rests an the long history of th study of language _in European intellectual traditions. Like so much of ‘European soci iba ass ron tLe Tn this caSe fradittons going back io ‘the grammarians of Sanskrit (e.9. the granmariap Panini, between 500 anc iC. STEMS We aramid y valaoe Gitano Green to Rome ia ab scholarship to medieval Europe ~ and much transformed along the way. In tie Sanskrit grammatical tradition, concern for the ‘purty’ of transmission of sacred texts fed toa nee fs angus a & necessary guaantor for that security of eta in ian In many banal but social sia Ways sare Tie that alftude remains, adapted to contemporary givens, Now, reliance on tie stability of languageis-seen as essential in relation to significant public tex! canonical or ot Jnty apout the re i it Sil a the guarantor of rationality and knowledge, of all that was and is seen as qui eSentiatly hartan, Nor is the notion of purity far from current public” as witness the outrage caused by changes to ‘conventions’; of which panics about th moral and-social effects of ‘texting’ are a current instance. If language guaranteed meaning and rationality, then theories of language cov also be relied on! to provide tools to explain what needed to be understood abou central issues in representation; as indeed about much else of value in culture aiv! society. In that way language and understandings about language have long sinc ‘come to provide ‘naturalized’ access to theories of meaning and communication. To sketch some differences of linguistic, pragmatic and semiotic approaches, | will use an example from my first years of teaching, in the early 1970s, at a ‘new University in the UK. It was not uncommon ~ things have become ‘tighter’ since — (or students to ask for ‘an extension’ to complete some work. The approach was usualy something like: fovin of this oft repeated request puzzled me. It seemed that power and power- nee were the key to understanding what was going on and would lead to ining variations in language use. My question at that time was: how can 1 ii) th social meaning of ths simple yet complex request "I wanted to ask, could suiextonsion far my essay?" | had other, more specific questions: ‘How is it tha Tnnwe (the morpherne Ged), whether in ‘wanted oF in (can + ed) = could) can sign of (relations of power?’. A grammatically/syntactically founded approacityan readily provide a description of clauses and their relations in ace; oF of the "sequence of tenses’. It can indicate what elements il rules of ‘grammar’ are being used: the use of the past-tense morphemes; Wee, The categories of 2 syntacticlgrammatical accowit, €-9-chawse, subject sentence, or adjective, tense-marker, noun, can readily be used to descri "Ws of that utterance Wher This approach is used, the focus is on tl H/o OF form and of formal relations. ontrast, erate Cy scinliaaulé) approach asks: ‘Under what con- Wh, When anc =, are these forms used and what tend to be likely effects of That provides descriptions of the environments of use and ofthe conitions N such forms tend to appear. It describes the socialinguistic roles of Nts) their status and relations of power; Figen see etc sinh as face, repair, politeness. To some een Te Tocus ison correlation: iste {orn Correlates with that Social context. The orientation is to use, to ‘ul environments of use and 4a same extant to-offe joach asks: ‘Whose interest and agency is at work here in the iilig?', ‘What meaning ts being made here?', *Hovr is meaning being [With what resources, in what social environment?” and ‘What are the mean- Mite of the resources that have been used?’, ‘How can past tense be an lof of power’, “How is it that a form that signals distance in time can signal Hi islonco, a “distance” produced by difference in social power?’. There is an Wun 10 the intorests of the sign-maker Cdistancing’, making her/himself 10 the environment in which meaning is made (the relations of power which AN the sonctiors which exist in not acknowledging power); to meaning and to jl/eultural resources wich are available for the realization/materialization tivated sign, bol the social-semiotic aspects. It does not deal with the “Iwanted to ask, could I have an extension for my essay?” Ae that time I was teaching linguistics ~ undoraraduate courses in (transformation!) syntax, semantics, sociolinguistics, psycholinguistis, stylists. I was trying to unc stand regularit was not comfortable’ with the ‘correlation Aha mode One response might be: ‘wel, the only mode here is speech, approaches la William Labov (1966, 19727 then dominant in sociolinguistic yulstie account is sufficient, nothing else to say.” A Jinguistic account, owe To me they did not offer a sufficient explanatTon of what it should be possible to know Mga tla it worwnyac 4 what Linguistics recognizes as siniticant about) iis about other ‘and say about such utterances (e.g, Yin this environment, this linguistic oF phone wy ciftax pikes ees58 Multimodality sneallin ree stele the moo peach itaeentd-atven Core eton of Spee ras 7 EAT us wheter few spoken ath 8 rising ora talon intonation at Tr “BEE am pcan oth souls eter eter ofthese yes i saa smile, acheeky grin, a serious or emt er supenstat? ilTirasesier ok at rf “gorse eer? de rsh cve ra i TE TOUTTOF ai they stay wedged ir TFE WAT-ope door? | wanseroton of ies Gut anyother canola aspects of mann 4 ie Calegory-of the mode of speech-forces attention toll signs 1): ? ‘snake lanolin” ard the cotgord ‘which that supplies. The semiatic category of mode attends to the potentias | making rearing of sound-as-sveech differently to the linguistic category of spe««l The other point here is that if a linguistic account of this interaction is see suiicient, necessarily it cannot give attention to other modes witch are in play: ca facial ex tial p9sitioning. ~ Each of the signs at Eh positioning inthe haf-open dor rmuthas the shape and the extent of the intonation contours. The ensemble of sic a a whole makes meaning: the distancing past tense coheres with the positioniny the half-open door; as does the tentativeness ofthe rsing intonation, The indivisis si semble can of course be used to contrast, to contradict, to ci tension: the distancing oF the past tense can Be ‘BeFSOM wali © iahein tothe office; or with the marked fal in intonation which turns the gran) matical form of interrogative into th ategory of statement. \ its shape, is ‘ready’ inrors’ Ors sak — the ‘meaiiny Tiere, a (Grammatical) indicator oF Temporal ds. ‘Whe past leMSETHOTpTIeMe -ed) is used as the 19 social distance, distance of power. It is possible to call that an Ganic relations ein time ‘is ke ther ference between the tree approaches les in what I cll he eo eeepc fe prac, Aah i description of naulstie forts and tel relations. As soclotngTtes, Tis ese” AE area at the absitton of conditions an coretan. redma spp ‘Recs gallon in meaning inked wo variation BeoreeT=y "Necessarily, the linguistic approach has nothing to say about other modes in makin meaning; though te principles of pragmatics do rot exclude an eres In extend its reach to other modes (Morris, 1970). In some sense both linguistics and pray in terms such as Yextra-tinguistic f matics recognize the presence of other modes A social-semiotic theory of multimodality “WM inauistic’, ‘non-verbal’ or in different kinds of acknowledgement to features of iNet’. That, however, is a recognition of the phenomenon in the same moment as ‘stant dismissal; a backhanded theoretical compliment: | notice you and you're THE Higniticant enough for me to bother. 1} course, as a generalization this is too sweeping. There are and have been clear PH wlons (iictmsiev, 1961; Halliday, 1978). Yet in a ‘disciplinary common-sense’ If such a thing exists ~ it is not too far off the mark. Often this position is ‘S/eiypanied by an assumption that, in any case, images are organized like language, HWilnoss terms such as ‘visual literacy’, ‘visual language’. End of story. Tike the example in Figure 4.2. In a social-semiotic multimadal account of JN, all signs in all modes ace meaningful mntation: to modes, mez Tigements. It can, for instance, ig.about meaning relations and Ur Tistartation in image through the | avrangemeadof visual elements; it can elucidate a syntax of this visual ilalion ~ the meaning-potential of spatial orientation of the players standing Joce, the use of colour as an ideationat-resuree; to identi sams; of that is, Te use of distance as a mean ; avon sect tyealized Wiplonce at which the players stand); their Eis epreson dour Tally to fective size and the prominence of the stua® or their boots. Should we want TMF We dichotomizing terms, we could say that affect and cognition are equally HNultaneously present; though much more significantly, social-semiotic theory sin Uslorhaleng thet dicho, via ths xorple france. Hi Is a multimodal text; the modes in use are ing, image, number, colour (and pression), Social Semiotics is abe to say something about the function of each Modes in this multimodal text; about the relation ofthese modesto each ather; ht the main entities in this text, Hi sunnmarize: linguistics provides a descrotion of forms, of their accurrence and of Jelllons between them. Pragmatics ~ and many forms of sociolinguistics —tells us sa! circumstances, about participants and the environments of use and likely {6 Socal semiotics and the multimodal dimension of the theory, tell us about Hai) ageocy; about meaning\-making); about processes Of sian-making in social HNonts; about the resources for making meaning and their respective potentials illers i the making of signs-as-metaphors, about the meaning potentials of WAenotic for. The theory can describe and analyse al signs in all modes as AAA Mlle Interrotation in any one toxt. = Wr Felavion to the example of the spoken request, Sal Semiotics tap say: power Ht retative social postion it can be expressed ab a distance MAAN Ti T0 sia al distance of powor. The degre of Wenn etrar terre rilated by @ manced and fayered use of apt pipiens Hor brtance °t had wanted to ask I wondered, could | possibly ask if 5930 Multimodality A social-semiotic theory of multimodality 61 you'd mind, if...) to make ever more complex/wvanced signs of relations of power ‘hilting the frame only slightly, the choice of a theory from Distancs is less power ys ideologically founded theorieSTs Tiself Bar herp tra) speaker both to acknowledge the distance in power bety the ald Hitloning is unavotdable; positfoning isthe result of chotce Fram-amiong a range of Fessee and ¢ means ~ being distant in time {rather than in space) — of temgor,) iilties; that choice is Socially meaningful ~ itis ideological. That is the-case with distancing from the request; in Tam telling you something about a pas! han with other forms of semiotics. It is nat to lapse into wish, about something that 1 had wanted to do then, whereas actually, now, Ta Pea io longer in What positon.’ That is, the request can be expressed and STWTTANEG) ‘Semrdented in this (weakened) form; it is modalized.\Being modalizeg (Halliday / 1967, 1964; Kress, 1976), Hodge and Kress, 1979\ TORS TT Fecipient tly ‘opoortunity to adopt o al possible readings. fn finguTsUIcs This s usualy treated as an instance of @fybiauly. In the sketch oF communication developed Chapter 3, the utterance/ressaJE provides a ground which can readily be framed @ prompt in either direction, according to the interpreter’s design. In the case of Figure 4.2, Social Semiotics can say which mode is foregrouncd which mode carries major informational weight (functional foad); which made |x what function in the overall textual entity (writing as caption, image as major ai! number as minor carrier of meaning). In the mode of image it can describe tiv arrangement of the entities of the image and their meanings ~ of ‘opposition’, of ‘confrontation’, of ‘challenge’, and so on. sod to describe multimodally constituted texts. The descriptive, analytical and logical apparatus of the prior theory is brought along and leads, necessarily, toa Woscription of the domain to which it is mow applied but for which it had not been oped. [The choice of a) theory carries, however implicitly, potent messages about what ake to be conceptions of culture, the relations between ‘representation’ and culture te fundamental characteristics of culture itself. A full theory of meaning needs a Hotion of culture and of meaning. |modal Social Semiotics th Ueniess 5 PTET TTS a re media of communication, From arsectve of eins ry el ih aan al mesa the relations bet ss. Here there ar€ question such as: ‘What Kinds of actual eke are common to all mades and to al te tyes of relations Horses for course: pt theories, useful framing Theories, of course, are just that, theories: multiple, competing, aften internally contradictory. Theories specify their domain, more or less explicitly and precisely they provide categories to describe and analyse the phenomena which they constrict Language, more maybe than other social phenomena, suggests itself ‘naturally’ 2% 9 ‘metaphor of the social or the political ~ or of their negation; and linguistic theor i act as potent metafoy ‘Among linguists, / a trenchant opponent of political oppression it the twentieth century, Is TE UMewWho has been mast insistent on separating languay) Tar aTfordances, Ws Wstories of social shaping ard the val origins/provenance of elements of that mode. As an instance, the mode of hn English has the semiotic category of intonation. Even though the modes Soundtrack oF music also use the material of sound — with its affordance of pitch from social and politic = er iS his political writing has becor lution ~ these modes do not have the category of intonation; nor of course do modes; UTTveTEliy’ the common characteristics and capacities of human brains every and at all times. This (seemingly) asocial theory isa potent metaphor of his poi position. A similar case can be made for the linguistic theory of Michael Halliday though i ite dire +s assum out the Generative pow! the social” explain thea jal organization, human action in scl ‘environments and the meaning potential of linguistic forms and processes. ll ‘rnetical terms, the three perspectives are difficult to keep apart; yet, for certain. Hiptive and analytical purposes it is useful and at times necessary to do so. 7 ‘Most Amainstrea fiLseMullimedal Soca! Semiotics deals with IE miwaning and ppear-as Bn integrated whol, a sigr>y\s Signs are always newly made according n ngustic theories separate meaning and form — syntax be analVlic edge was used in Critical Tes Ukress and Hodge, 1979; Fowler ct vi. | (AWW interests of sigr-makers in specific social enviranments, there is neither need 1979; Hodge and Kress, 1993) ag the tool for social critique: the social agent Ye place for a theory of use, that is, for Pragmatics. Use is, simply, a normal part capacity to exercise ‘choice’ from a complex system of /inguistic meanimaypotential iy Wh he making of every sign. A theory of use is reckundant in-an-appreaety- which has: Me COMEXT OT a SPAT sOTTAT evra Auman te sign maka ts centre the sian, + all, is made in and for theMultimodality Conditions ofits use, Signs are made in a specific environment according to the sign ‘ihaker’s need at the moment of sign-making, shaped by the interest of the make ‘of the sign in that environment. The environments and circumstances of ‘use’ are. therefore, always an absolutly integral part of (the making of) the sign: they are a the centre ofthe concerns ofthe theory. The signs made are as precise as it i pessib to make them to realize the sign-maker's me re are other theoretical consequences, ILsgns are precise, that leaves th Geegy e catnatnt ont itis reveled as an-alempE TD patch up problem at the core of a problematic theory; a theory, that is, which had no plausible Account of meaning-mating, Siar arguments apply to enterprises suchas Smet: and Stylistics. Cleary attests important to focus on meaning; and ste is important component of a social-semiotic multimodal theory. Yet the need or justi, cation for (sub-Iaisciplines such as stylists, semantics has gone: style isan outeor of Simrescive-chaiceso jn-the-agking OF Sig's ar slgn-complenes i i ‘Similarly, Yhe,goncer oF Sociolingatsticg vanish when all occasions of sigr-making are embedded in and A social-semiotic theory of multimodality " fielice classified signs according to the, {rind which they {0 that which they represent in the/world: an iconic sign ‘resembles! what it “Wifesents ~ the circle as weet; an indexfcal sign ‘points to’ (as in ‘detxis’) an object, Nt ~ Peirce’s example was that of snoke as indexing fire; 2 symbolic sian stands conventionally agreed relation between a form and the WH Crossed cresce bols for a humanitarian organization (Colapietro |, 1966). In his theory, the function of the sign does not — except in the case of wile signs ~ stand in an arbitrary relation to the world, but is shaped in and by Hite rotations. 2d}, was on relations of the ‘ouler4yorld to-aa in the sign., relation dued-getween a phenomenon in the o Din that inner mental world the sianifiee cFasTan, the sigi is expressed in an external, outwardly ‘oF otherwise materially TaamibTe form.,in Saussure’s sign, signifiarand Wve linked in a relation which is arbitrary and bound byxEomvention> Shaped by social environments ‘The motivated sign The sign is the central concept of semiotics. In the sign, meaning and form are fuse! in one entity. In 2 Social Semiotic theory, signs are made—not used by a sign-maker who brings meaning into an apt conjunction with a form, a selection/choice shapes! by the sign-maker’s interest. In the process 0 dealing with the social world ‘The two main strands of Semiotics which have dominated in ‘the West” are based on ‘the work of one or both of two major figures: Charles Sanders Peirce (1857-1913), an American philosopher; and Ferdinand de Saussure, a Swiss linguist (1839-1914), both teaching and writing int 1e nineteenth century ard into the very sort part ott treneh Coe Patan mod) offers three crucial emphases: 9 classification of signs based_on the TeTatToM-of signs to “the world’; the process of iosis; and the category of the ing ts = In Peirce’s account of signs (musty the process of semiosis ~ ‘recipient«/ inierpreters of a sion make thelr sense of the sign they ‘receive’: they form interpretant of that sign for themselves. The faterpretant is the meaning of that si for the recipient. ‘Readers’ are agentive and transformative in their semiotic engase ‘ment with signs. Once formed, the interoretant becomes (asthe new objec in Peirc terms) the point ef departure for a new sign, leading to an ongoing, constan'y transformative chain of semiosis. In this process the 'veader'Pvecipient’ as interpreter 4s the maker of new signs and, in that, is the remaker of the semiotic material of the culture. It is a process of a ceaseless remaking of meaning, of interpretants newly with a prior sin, ] formed in the transformative engageme . irariness- and convention pcint to social pawez, though in distinctly Invent ways: @rarine ae dcation ofa socal power which is suficin {0 tie any Torm to-aty meaning; and égrventioD the effect of social nower i a socTal TOrEe which acts to keep signs stable, a stabilizing force for the Monty whe 5-10 i. In this account, the actions of individuals cannot ‘signs or the relation oF the sian to the system of signs (Hodge and Kress, ean approach assumes change based on the actions of individuals i :[ulination of thacinterpretantyyavich Bee ‘Object of the next stant The iuiyian accolar rites oor TaTvranAT action as a possible means Tor chang (yaleiils stable held a pracechy-the Torce of Collective social paw alized Jotin( social account of meaning based on the significance of the agency of Ula, is entireTy-at odds with a conception of an arbitrar il, established and held in place by conventipn. ‘tsanple used by Saussure to make his point was a lexical one, the French wo Meee re ee ifier/sound shape (tri:] is linked with that same signified, as Is Lbauml in German? milation between sound shape and meaning is an arbitrary one. That seems Hible enough, J. castle say, ther we could use any small object, a button maybe, to replace/ | This seems to confirm both the arbitrary relation between form and meaning (thé Aeetion of Ue chess piece) and the strength of convention, agreed on by both players {We the duration of the game. 63 >) Wiligenstein (1935) extended the scope of this reasoning, stressing the force Ot SHeeilon by introducing the example of a (chess) game. Assume, he suggested, that ‘we were setting Up a game of chess, we discovered that a piace was missing, a. yy |64 Multimodatity A social-semiotic theory of multimodality tease ck ates tty ta we woud wea Dk ben sad for the black piece even though the notion of arbitrariness suggests that the choice of | awhite button would do just as well. At this point we have made two decisions that avr not arbitrary: the decision to use a button to stand for a chess piece and the choice vf at bate. te omer een tecnico ur anae auton forte acest ay sn nema end eso Src sre tance or toe ietane ne mf paca | Svan: yh prnr rl bv aged on se sew crs arias oderewoten erhearatona hesene Had wei oe Bathe ole vaso daca tad atts of eres we wale erg ton St ner Dice egress seo tor agin fe ape Iie I tthe Hck en ewe pres were mtn path e wel ardor sabe ero bats De me mu are a Sct lay rnd te ming we ew let a versa It seems clear that ny rate the principle of arbitrariness won't wor What seems lke wirat convent are socially motivated, aistneey a Choosing “bTacK" in the rst example meant select ‘Colour’ as the criterial aspect of the object to be represented; choosing ‘size’ in Ii second case meant taking "value’C-as-size) as criterial. Tobe effective, whether in game of chess or more so in the social game of language — as Wittgenstein put i both selections need the joint agreement of the players. But the decision also needs \0 ‘work effectively for playing the game: using white buttons for black pieces and smal buttons foe-mare valuable pieces would work against the necessary “transparency i: We can push his example further. If the black pawn was missing and we had son Inlues the symbotie sin, Theoretically, to allow for itl bs of arbitrariness” or there, allowing maybe forthe edd erbitrary sign in some environments s Bod istake, The effect so totally unckermine the power and uschaess of fy ofthe motivated sign ~ wheter Inthe direction ofthe pssbltyof refutation 'the theory itself or in the use of the notion as a heuristic device to hypothesize about Hig, Assuring the positon that al tgs are metivated conunctions of form ireatina fin orale ta eign of he Red Cros Te REO CORT beg 9 oer) the colours of the Swiss flag) might pethave been yafrelated to Swiss neutrality, aie ofthe sive’ oF being an ‘urirwole, reutral party ‘an Sparta tation of assistance’ et to menor the nately ofthe fener afte ization. Nor Is it implausible aiven the religous mearing of res, that in some lies the Red Crescent was chosen asthe apt signifier or ther. tn that attr case ‘Tp the ‘cross’ as religious symbol rather than as reminder of the Swiss flag which fred criteria. T have not checked, but it would be revealing to find what versions of I raed Greseet tore ar,and wh Wrench coe thingie of rath: i ightforwardly in ordinary everyday interaction or in forms of research. If the ype’ of the signifier aptly suggests the ‘shape’ of the signified (e.g. the circle and Giotvatjod each ooint to soci ‘Mee, it allows an analyst ~ whether in everyday interaction or in research ~ to icin Se maga = ay jwothiesize about the features which the maker of the sign regarded as criterial about and tramspare Tations of form and meaning in the sign. The apt Fea ‘abject which she or he represented. Positing that relation between ‘sign’ and ‘oF material form and cultural meaning isan expression OF the Sonsmaker’s interest i (s crucial; it opens the possibility of a path to understanding what in the g ‘ms meme ey eee ee 7 wmenon or object to be represented was treated as criterial by the maker ofthe rzation” of Their meaning and tha, in-uuc, is needed crucially in communica Al the moment of representation. That can lead to an understanding ofthe sign- ers position in their world at the moment of the mal H hypothesis is of fundamental importance in all commur ‘Historically as critique or as design, w of the sign, Such tion — whether framed ‘The everyday, the banal and the motivated sign “Diyneure's Aarstaken assumption th of signifier and sianified is an Vultee vasa all tory, a product ane realization of he social conditions Wis ting. Here ate three objects. Kis arbitrariness takes no account citer. of Social Senate, i | want > be wners08 by preference [Use the resorcisthat ose Taig the signs WHET reed to make. [[Tan-nol fariiar wih Urose resource Take sia i TU Tarn avery saapene te ean TORSO 655 Multimodality the patent facts of the histories (of change) of semiotic resources (see in this respec raymond WilTams’s (1985) Key ‘ior of the facts oF contemporary sign-makine prasiee nary soe See ts ona soles or SURES SETA ia characterises and OS Ah sper se anmete CRE Genies agency To those who make meaning in making Sigs: Twisting To buttPess one ‘ilar of he cca ary tre FeO RETNE PONTO pero ecOTECT Oe Tord Source o at poner valve agree Trova We at v MOTWTaUATs acting as Sigi-MaKers Out of socially shaped interes with socially made resources in sock is. To deal with te Tast SE Tlrst: observation of semiosis at any level show: constant change rather than rigid conformity as repetition or ‘copying’ — whatever the modes which are involved. Every example in this book provides an instance of this, ‘The second objection rests an a confusion on Saussure's part: the signifier of “ree ness! is not a sequence of sounds, that is, not Lazbr], but an existing lexical-item-as signifier, ‘tree’, used in its potential for becoming a new sign. The meaning-potenta Of the signifier ‘tree’ is the sum of all the instances in which I have encountered tiv sign tree as a signifier: that enables me to make a prediction about its aptness a a signifier for the new sign that J want to make now. The signified TREE and the sic nifier ‘tree’ are elements at the same level and of the same kind: not as in Saussure's assumption where one is a semantic entity and the other is a phonetic one, one a entity of meaning and the other an entity of sound. ‘Assume | have bought a pot plant, a ficus. | know nothing about the plant’s natural habitat or the dimensions that it reaches in its natural habitat (in Australia it is 0 rainforest tree which grows to huge expanse and height): my plant is about 50 ¢m i height. Should I call what I have bought 2 ‘bush’ or a ‘shrub or maybe somethings else? Or is it best described as a ‘tree’? At the point of searching for a name, th signifier that 1 am looking for is not a string of sounds but a abel’, a name, that [have encountered before, sufficiently often to know that its meaning-potential ~ 0 woody, perennial plant, of significant size with a strong, usually single stem — best captures what this thing that I need to name is. My question is not ‘will [shrub] sou! best for this plant, or maybe Cbush?” Rather, my question is: ‘Where does this pia “belong’”? *Does it belong with flowers, bushes, shrubs, trees?’ and: ‘What lexical entity will serve as an apt signifier for this plant in terms of its present characteristi or in terms of its future grovith?’. Someone from Norway or from the Black Fores! used to trees as towering pines, might, on a first visit to the National Parks nea Sydney, (Kuring-aai Chase to the north; the Royal National Park to the south) be unwilling to dignify the twisted, gnarled, wonderfully shaped vegetation on the sans stone plateau with the name ‘tree’. The ‘settlers’ who encountered the Mallee (re of the South Australian inland used the ward ‘scrub’ to name the expanses of these hauntingly beautiful, low-growing eucalypts. That gave legitimacy to uprooting thousands of square miles of that forest and sow the land to wheat ~ at east until the A social-semiotic theory of multimodality in topsoil had blown away. The use of the signifier ‘ree’ fs not automatic; itis Klally motivated and individually enacted as apt for me, on that occasion. ‘My problem with naming the pot plant is not about the sound that would seem right this plant, but about the lexical and social ‘place’ where it would fit. Had it been, former, then, yes, bub or sub, slib even, might all have served equally asa name; though questions Gf euphony aight have been a consideration, itself “Wojarmatier of an arbitrarily made dSTS79%. In other words, insian-making there js Mircloapyiacen sented ane soni: both are fom the same ler. Soa pes dato Wal instances of sion-n makina, Tor” 25s, 1988). jal, the everyday and unreTTarCaDTe El ‘@xamples here are chosen with that in mind. Colour plate 3 shaws two Tmages 0 ts of salt and pepper, the kind you used to get (and still do, in some cases) on iy ray, travelling in economy class on an aiine. These things have a practical use: snake the meal on your tray more palatable, if you like salt or pepper. They ar also Wand they interest me as signs —of salt oF pepper. On a different airline the sait pepper sachets are diferent (Colour plate 4); and ona third and Fourth they are wnt yet again (Colour plates 5 and 6). This variety requires an explanation; and thmonstrates to my satisfaction that the sign-makers have chosen ~ lferently fch case —what isto be taken as criterial about the object to be represented: how Jopresent ‘salt’ and ‘pepper’. There is a vast range — infinite maybe - of possibilities Aepresenting ‘salt-ness’ and ‘pepper-ress’. Each of the choices provides insight the sign-maker’s intrest. In Colour plate 3, the colours on the sachets ~ a kind fange red for the iodized salt and dark brownist-orey for the pepper ~ point to A\est and motivation: the colour of iodine andthe colour of pepper ‘As with the chess pieces, in each case a criterial feature of the thing to be repre ted chosen, In Colour plate 3, colour is chosen to distinguish the condiments. For passenger dealing with lunch oF dinoer on their tiny tray — with the light not ail 1 Q000 in the middle seat of the central row ~ the most important thing is easy Heognition. Most users might agree that the greyish-brown is an apt choice as a lier for pepper and would not work well as an apt signifier for salt. And wile it ibe objected that salt is rarely orangey-red, the colour does work both in inguishing it from the pepper and in signifying that this salt is ‘iodized’. with of me, I am unlikely to go wrong in my selection of salt or pepper, even ht not manage to read the word “iodized! on the sachet. A feature ofthe et toe represented is selected as criteria and becomes an apt signifier in these iti circumstances. Di nother avtine (Colour plate 4), a different criterion ie selected asa signifier for “Hie samme object ~ salt and pepper. There are two words, as labels; the message is Wejwated’ in a way; maybe because, asin the other case, words may nat be reliable ya! under the circumstances. Here colour is not criteria for distinguishing salt Mil perwer Instead, there isa visual reference to a culturally specific practice of 1 arn using salt and pepper in do nts — in salt and esti¢ oF public envir he best site to anchor thean 67 mer us +2 Multimodality pepper shakers. We can ask whether colour is still important: there are bound to be Colours that would not be apt as signifiers for either salt or pepper. If so, it woule indicate that colour is significant here too, though implicitly. This too is a motivated sign, even though the signifier chosen is from an entirely different domain to that 0 colour, namely a reference to social use. That signifier too is apt for the signified “The tivo signs show a different ‘angle’. In Peirce’s schema, the Delta sachets mish be regarded as an indexical sign, pointing to an earlier sign, an actual shaker and its use. in Multimodal Social Semiatis this is seen as a matter of provenance, a referenc: booth to a cultural practice and to the object ‘salt and pepper shaker’. The question posed and answered here is “Where does this signifier come from?’ (Kress and vai) Leeuwen, 2001). The signs of the sachets with iodized salt belong to the Peircear category of iconic sign ~ of “looking like or being lke’. Its the sigr-maker’s decision as to what is to be taken as criterial ~ with any sign. The different ‘angle’ indicate ferent positioning: we are in the realm, however mildly, of ideology. There may be other criteria. The deep navy blue colour of the salt and pepper sachets in Colour plate 4 loaks to me — | am not a frequent user of the airline ~ like the ‘corporate blue’ of the airline's livery and logo. In other words, the sachet of salt and pepper may be being used to signify a ‘corporate meaning’, ‘Delta-ness’ —a fee good factor to sprinkle on and digest with your food. Provenance ~ the questio’ "Where does this come from?’ puts us inthe difficult waters of difference of cuttur practices: in some cultures you get salt from the shaker with the one hole and in other from the one with many holes; and vice-versa with pepper. That does not change th principle of motivation: it points to different social histories, valuations an! consequent differences in the use of these condiments in the cullnary and gastronori histories of different cultures. In Colour plate 5, the sign refers to specific social practices: the uses of sait an pepper in the social histories of cooking — the practice of grinding the ‘raw’ materia ina marble mortar; a cultural reference leaning on tradition, on a specific aesthetics and evoking notions of authenticity ‘The relation of signifier to signified is not an arbitrary one in any of these example Tt Is doubtful whether any one of these signs will become generally accepted 0! signifiers for salt or pepper. The airlines’ intentions, if anything, are to keep the distinctive; and maybe to allow themselves to engage in alittle mild playfulness wi) their ‘guests’. Nevertheless, the theory ought to be able to tell us under what cov ditions this might or might not happen. What it does show is that the relation ol ‘gnifier and signified in the sign has a social and therefore ‘political’ and ideologic component. A ‘playful relation’ with someone nowadays often called a ‘custom fr a ‘client’ i to make a sacial and idenlogical paint about that relation and about a wider ideological frame. In my last of these examples, Colour plate 6, ‘ideology’ in the sense of the suggestion or projection of a particular ‘world’ seems to me quite clearly present. Hevs the colour-scheme works together with the faint 's p ic y’ printed acrass the bottow fof one side of the two joined sachets ~ spicy meting that adds fun to lie A social-semiotic theory of multimodality {s no longer just about salt and pepper, but about a characteristic which might je not just an individual but a community, about a whole attitude to fife, about ‘Austrian and Viennese. Here discourse ~in the sense of a positon in and to life, oping of a way of being in the world, of ways of being and knowing ~ is most ly evident, Equally with genre, as an expression of the social re Ui and its ‘guests’ and the ‘address’ of the reader/viewer. ‘uci point is the unnoticed, near invisible social and ideological effects of signs of the everyday, the sions of ordinary life, of the unremarkable and banal, in Wiscourse and genre and with them ideofoay are potently at work — nearly ibly ~ as or more effective than in heightened, clearly visible and therefore ble instances Koncept ofthe motivated signin no way places restrictions on sign-makers; the Jp ns open or as restricted as the sign-maker's intrest, which shapes the sian an 1 which isan effect and a realization ofthe histories in social environments of ‘ignemaker. The examples show this clearly: one speaks of health (the iodized J) another of social practices; the third of culinary traditions, of the aesthetics ing and of kinds of authenticity; and one speaks of lifestyle, of pleasure and All Awse are present, differently weighted, differently valued. The social, its ies, knowledges, its forms of social relations, its discourses and genres, are here, {in these unremarkable, everyday, banal objects-as-igrs. the examples show the social embeddedness of sions and sign-making practices: ies ofthe value and valuation of pepper inthe histories of European cooking; of {uses of salt; of their preparation in households; of the recovery of tradition; Just being “cool. At an Italian restaurant near my place of work — one of a ul ‘chain’ ~ there are small, shallow, white saucer/plates on the tables, filled arse flaky sea-salt. They invite me to imagine myself as an Italian peasant, up salt with my fingers, crushing it and sprinkling it on my bread after dunking Ae saucer with olive oll. 1 am reminded of the one-piece heavy glass salt and I containers of my childhood, with two shallow ‘bowls’ on either side, in the othe table for everyone to take their pinch of salt or pepper, or ‘dip’ their cr bread. These objects show that signifies and signs carry, in their make-up, hice of long histories of practices. The meanings of these practices are present ‘ignifiers asa potential for meaning and are carried forward’ in constantly lornved fashion into new signs, remade in the light of the resources that (re)- #9 sions bring with them. In signs, sian-makers mediate their own social } their present social position, their sense of their social environment in the ‘of communication; and this becomes tangible in the reshaping of the cultural 196 used in representation and communication. The makers of signs ‘stamp! ML social conditions into the signs they make and make these signs into the Jor of social histories. CS70 Vv Muitimedality A social-semiotic theory of multimodality vinaking however remains constant. Partiality of interest shapes the signified at oment of the making of the sign. AT Te vary next moment the sign-maker’s Interest and the partiality of representation The next example shows the processes at the core of sign-making with great clar'ty A three-year-old, sitting on hs father’s lap (mine, as it happened), draws a series o! Circles, seven to be exact. At the end he says: ‘This isa car' (see Figure 4.1, above) How is, or could this be ‘a car’? While drawing, he had said ‘Here's a wheel Here’s another wheel... That's a funny wheel... This isa car.’ For him the criter feature of car was its ‘wheel-ness'; it had (many) wheels. Two steps are involved! ‘the making of this sign. At the first step, ‘wheels’, the signified, are represen by circles, as apt signifiers. At the second step, the signified ‘car’ Is represented by \w ‘apt signifier of “(arrangement of) seven circles’. To represent wheels by circles rests as with the replacement ofthe chess pieces (‘a black button is like a black piece’) — 0 the principle of analogy: ihe moment of the making of the sign, representation is always partial; yet it Always Mull’, ‘complete’. It is partial in relation to the abject or phenomenon full in relation to the sign-maker's interest at the moment of making ln. That isthe case with this ‘car’ as much as withthe representation of any car ¥ avhertisement. Interest produces attention. Attention frames the world to be quslales THETEST A selects'what is to be represented as the Lbasis of the relation of analogy. The outcome of that prOCESSTS a metaphor. Step one : aphors, always TERY made, resting on, maigrlallzing and Giplaying analogy —> ‘circles are rourd; whee's are round; circles are lke wheels; Stays TenIT made, resting on, matealzing and Spaying Step twor jnlation =the mearing that 1 wish to realize, to make material ~is not analogy —» a car has many wheels; many wheels are like a car’, “inne race ator tach errant or Seheeanwa Gar. Thetorpleesioy car rade ferent probate waldng two slg cool, a dub proses of analy, eli in two metaphors rl are le hea ar ny hess ea car's anally a oe meta (the complex ua si ea car Tse how or why wheels could be the citer feature for cr, we have oa thepoint of view, eral phyilonely, pachoogally, eaturally sent the tree eared. Trwe lage en lack ate faye aa 82 YW Ga it Oneal Sis uPeh, epee sk TERRE AC coreegeanalel wart Eovelude thet this sign'maker’s poston inthe worl, Iteraly,phystealy, But = psreialy,atfecively, ight welled im ose ‘ear In tht way. Hs tet a ‘out of his (physical, affective, cultural, social) posit in the world at that monien!, ny THE Dro ore as Aierey Tose RpTeEaton ToS On Ty Mt) communication focuses on the assumed interest ofthe TIDE OFTE ig, ‘ign. That leads to the demand for transparency in communication. fildr an example, wellenwn to mast of us We are In-a plane, jst before Hof, The cabin staff are going through the safety instructions, and are about to ie ereroency ets. Sxtcen ote rane afte plan, alg he fpf the cockpit, that with her bark to the passengers, an atenant Bors o ve spokevanncurcement: "vo exis ate ony ave el ore rah” alses cr ie arm ard gestures nthe drection of the lefhond ext she rales Berman geaures the rection of ie eah-tand ex er gestures are fis the raising of arm ard gesturing with er hand reap sinters let arm Hl exit, right arm to right exit. Spoken and gestural signs are in agreement. Both Berar what sto be represmcd, as announcer cots wi “bey ‘foar, one left, one right’, the attendant has turned around and now faces the yigers. On ‘one left” she raises her right arm and gestures with her right hand ‘the exit which, for the passengers, is on the left-hand side; on ‘one to the right’ as tr et ar ae ester with er fe han to the exit othe right of Mibsanges She docs the sare to "end four es over te wings tw Kf oo Ih anata Ue wave othe hands asi lapping’ ings, Now Mba contrat orc hr rte tee aod conor tr are area ew Tat TaTerest at the moment of sign-making arises out of the position in the world; it shapes attention; that frames a part of the world and acts 0 a principle for the selection of apt signitiers. Clearly, the chilo’s interest I: tyere is more to a car than wheels, even for # three-year-old. Theoretically we have a choice: we can treat this as an instance f TE Tepresentation and dismiss it; or we ean take it as a contral feature al representation in all instances. That is the route | have taken: all representation spoken announcement: they are no longer rapresentatianally apt: ‘ore left” is always partial, What the sign-maker takes as criteria! determines what she or lv | lalied with her right arm and hand and Yone right’ with her left arm and hand. In wl repre bo tha ety. An dl cho ae mare shaped by history ‘worth, in this environment the demands of communication, the overriding need, ‘experiences in various sacial envirSimments; and adults have greater awareness of uni) WW adopt the passengers’ viewpoint, which makes her (complex) signs internally lure, The princi olf itfudictory. The interests of the pwssengers have overridden her interests; or, ‘access to the resources for representation available in ti ¢ n +72 Multimodeality A social-semiotic theory of multimodality 73 better, she has factored the interests of her audience into the complex sign that shiv hhas made. The signs are communicationaily apt. The greater demands of the com ‘municational partner override the representational needs of the sign-maker for th aptness of realization of her meanings. 00, class, gender, profession, lifestyle, have their specific and distinct semiotic Ee, differently arranged and valued, That reinforces the point I have just mad iy the need to shape my signin relation to the person or group for whom I have i the sign and on the effect which that is Vkely to have. As societies and as ‘This may seem too obviously necessary to be useful or persuasive as an examp lions of communication become ever more diverse, that is an increasingly lent factor, an area of work which Intercultural Communication has dealt with for the theoretical point here: yet in many situations contradiction is exactly whal hhappens; often it is the normal condition oF sign-making. The sign 1s Contraqiexon, ite some time (Scollon and Scollon, 2000). from Ter perspectNe as much as WONT the Perspective of (the luckily few) literal e-be expressed relationally, as position and direction, charac minded passengers. It is also the case that most passengers would not notice ti | Power is constantly contested, with v Instance ih Subversive genres such as allegory, joke, aware of the degrees of superior contradiction. Most passengers (like mo3t of us at most Times) assume that the wor'd werted to limit such expression (Bakhtin, 1986). Even i Tistances or periods of sign-making is ordered by our interests, or at least on this occasion; so no I pawer difference ~ or maybe particularly then ~ such challenges exist. contradiction is noted. On this occasion it is the passengers who count; and so their interest prevail ‘urrent fundamental challenges to and shifts in social power in nearly all lis of Western’ sites have had and are having semiotic effects. Equally, there od Peoreicaly i means tat af Sin-making has 1 be Founded on a careful assess " of the social environment and the relations of power 1 ‘eavironment. That musi ieee te pons ferro Fe Uegone oration of sign potentials of semiotic power as means of challenge to social power. Power and making is that others’ sign-making is ordered to fit the interests oF the powerful jets were the core of Critical Linguistics (Kress and Hodge, 1979; Fowler rer ste erties AMAL Ir raSparery FAO Te GreTion of opal 1979; Wodge ard KresS-T98B7 Hodge ar Kress, 1993), parts of which ater into the larger and theoretically more diffuse project of Critical Discours asses as ‘politeness’ (in anglophone cultures) is evidence of 1! skewing effects of power in communication, The more powerful the maker of the sion, tess, 1984/89; Kress, 1991; Fairclough, 1989). It remains one of the ‘Tecriore she or he can ignore te requirements cf ransparency = hati, attention io I inerests in communication, That includes (Foucault's notion of power from ye te cw ena a ls, THE UTS Tv to dhe Se (Foucault, 1982) or power exerisodimarshaled(seemingh) barony \ vfork that makes up for the neglect of the privilege¢———__—— this affects the neteBor of postion; n each istance-thase who exert power nf up, even if only fr a moment of ra istace” oF ‘All signs a ‘these tno perspectives and interests: mine in relation toy representation and interests; and yours in relation to commun"eaffon and to the nev! wailing action, Refusal to acknowledge the demands of power is for factoring Tm your Titerest and the requirements of power. While power introduc Jit.as refusal to act is to exercise power. “cpacity into the world of signs, it does not disturb the principles of the motivate! ‘we all familiar with examples Ir Tanguage. In their landmark article ‘The Sign. A common assumption in debates around communication is that ‘good communi (9 of power and solidarity’, Brown and Gilman (1966) drew attention to a cation’ is “transparent communication’, that Is, communication where there are 110 ‘nl semiotic change in seventeenth-century England, when the second person ‘obstacles to an understanding between the person or institution who shapes tly; pronoun you (the vous of French, the Sie of German) changed from being message and the individual or group who are its intended audience. The idea ol foun of power and became the pronoun of address for everyone. Such social *aptness’ works in that direction: the maker of the sign attemy i the material ‘Winlotic change produces a far-reaching unsettling of social arrangements; sTgnifier which is best fitted to ‘realize’ the (meaning of the) signified, However, (nt lv te expression of power and of power itself but it does not — and did not in raker’s Tnterest cannot ever be known To the reciprent. But ifthe recipient can safely ‘(hi ~ abolish it. New forms for expressing power needed to be and were found. assume that the relation of aptness between form and meaning will hold, then sl I | can address my boss as you and by his first name (students in higher or he can form a useful hypothesis about the signified and the interest of the maker of on in the UK tend to address their teachers by their first names) but I do not the sign an the basis of the ‘shape’ of the signifier. The recipient may not know abo W ny boss with an imperative ~ e.g. “Shut that window, Bil’; nor, usually, would any limitations in the availability of signifier-recources ta the maker of 1 tole te ‘An example is my knowledge of French: anyone reading something written by 10 1 Ie case For speech isthe case for all modes. Speech provides means fo in French would be helped by knowing about the limitations of my resources in tlt TsocTal other In interaction, but so do gesture, gaze, proximity and } language. TH nat Uifcult to test out How, with each ol these modes can Indicate A usual situation is eke Lo power oF my amelioration of power in cornplex signs. Averting my 4a: groups? — co HAW talking here about ‘my’ social group ~ these signifiers and Signs are highly creed 77 winication acrass boundar whe by virtue of factors such74 Multimodaity A social-semiotic theory of multimodality 75 ‘When 1 got to the museum it looked bigger than I thought. When I went in I took “lf ny coat and went into the men’s toilet and after [ ran upstairs I went into the ‘ft, Then I went to see the mummies and all those stuff. Then we went to our ‘lakroom so we can get our coat and then we went to Waterloo station on a tube WM! train to Clapham Junction and walked back to school and went home very ‘nny and I told my mum, sister and my brother. The end. sensitive to social difference) has a diferent effect to fixing someone with my and heldina it, Where modes are Involved which do not afford the ‘address’ of ao: ier but tic nevis Inet veneesept social rai, stn eX of foros and cf emooratian cans produced ina solomag te ete pre {high to tow; low to hah canbe varied Infinitely anced gradations, In wate the ening hows It de possi to make reasonable Inferences about the sate oft “British economy based onthe poston from which Ue an oT ENG bul ‘TS Shown. Distante-tar be mantpotaret ira shor ih Him or in a photo; as it ca tHephysieal avangements of enture na place of wark xin seating errengemen ata dinner table In all ese, power relator can be represanted to indete det ence or amelioration. Soft focus or its opposite in photographic image; choice of w: from a formal or a familiar register; the direct expression of a request or indirect yo the choice of medium — not text-message for those to whom T defer but a more Fors o tien recount and the drawing are both recollections of the visit, set down on the 19 day. The ‘story’ of the visit conforms both to the genre of recount and to the ie) logic of speech ~ a mode familiar and congenial to a six-year-old — here Kn its transcription into ‘writing’: a chronologically arranged sequence of ‘and events. The depiction of the day in the drawing is entirely diferent. It too Ws to the semiotic logic of its mode: the simultaneous presence of entities, 1 spatial relations within a framed space. The logic of speech seems to suggest, tion: ‘What were the salient events and actions and in what order did they I, The logic of image seems to suggest a different question: "What were the i eniitos in the visual recollection of the day and in what relation to each other stand?” leotard nay ara ae eating oa od ate snag poner ARE pos sition ih revtably ei assh nears prove SoS UN Ts environment point to The power of a participant, then her or his use of si of the diminution of power — in ‘politeness’ for instance ~ can be read as such anc ni as a sign of lesser power. This complex relation of signs (and ensembles of s yi ~~" of often quite different kind) Tg encapsulated In Te Totion of the logontomic syn) J (Hodge and Kress, 1988), itselt a complex of signs Wat GWeS readers DEE reading, ‘navigating’, of ‘placing” te Interrelations and valuations of signs in sa } complexes: means for reading comPadiction, tension, oppositfon and apparent “Mid Tacticlty, and so on. Th that context, contradiction oF One STG WET others W 1 SIgr-complex Ts in no way dysfunctional but supplies essential infarmation for 3 accurate reading of the sacial/communicative environment, which includes the nay of the sign. Makers of signs, no matter their age, live in a world shaped by the histories of th work of their societies; the results of that work are available to them as the resow of their culture, Inevitably, what has been and what is around” and available, |i shaped and does shape the interest and the attention of the maker of the sign. Ivo year-olds can pick up pens and make marks on paper because pen and paper <6 there. They can (still — these materials may be passing) shape objects with playdov yh make shapes with scissors and paper. Culturally provided material resources ‘we their effect in shaping and directing, inthe ‘channelling’ of interest A request by a primary school teacher to write a story about a visit toa muse suggests specific possibilities to the child writer, different to those of a request 10 ray an image of that same visit. The world projected in the written recount of the visit. ‘the world projected in the visual account (see Figure 4.3) are likely to be profount, different. Here is the written ‘story’ Piquer 4.4. Chile's drawing of the visit to the British Museum Hei thanks to Eve Bearne)76 Multimodality A social-semiotic theory of multimodality 77 ough the modes of image, writing, colour, but also in actual or imagined ‘inner’ Aiesis through touch and feel, scent and smell, in action — imagined or real. The feel the plastic container; its texture; the shape of the ‘bottle’; the action of pouring or ruse suggested by the shape; its imagined and actual fit into the hand; the scent the lid is undone; all engage more of our body in their materiality than sparser jllons of representation’ might usually suggest. Jn the engagement with any sign, the materiality of modes ~ where sign and mods stood broadly ~ Tera res ATT Signs, whether that I make in my actions, or remake in my inner transformative and/or trans- embodied, for maker and remaker alike. In 3 Way The question about the image seems to be answered as I was really impressed ly the facade ofthe museum, with those huge columns; and by the marmies which Ts: in the museum, The two mummies were a bit different from each other, but they Ww: equally interesting, and I drew them in front ofthe museum in which {saw them Uf this, in its brief Form, a reasonable account, we might infer that the tw moc at image and speech there as written transcription) organize representation di ently, that each poses specific Kinds of questions’. Putting it ore stronaly, we mish say that the request ta make a representation of the day's experiance in two mos Itads fo sign-complexes which order tre recaTectTon of The Way Tine with tv affordances Cand the logics) of Te Two TOTES WE MTG GOA ep FATT FEAT “whether a habituation to representing in one made or the other could come to orga iz and shape our engagement wit the world prior even to any request from outsi: 0 nee to make a representation aterestYs decisive because it Forms the basis of the choices of what I take criteria, the availability of modes, their materiality and Tet artorganees_IT entas ARE TTS TT ‘a can be or what fs more Hikely to becors real When fre ois oe ems rush vn than whe hear thro the auditory apparatus). The mades and the kinds oF ‘ade of ede in a society bring with them certain orientations, a certain ake” 0 OAR By and large, Tat Take” becornes Trwksole” (audible, intangible) — Tn ie arse of explicit awareness to those who do take” it. Cultural an social habit, to modes, genres and oractices shares hour we represen tn time That habituatio SFgage with the world which we then represent, OTH Ue broad Tange oF r Te individual's decision to make Choi To_use fhese modes rather Wan frase TVs environment Tor these reasons processes named as affect ard cognition coincide absolutely a5 ONF lly effect. In this way too, identity is embodied and becomes more than a merely pmenon, an ‘attitude’, or perform, 7 that children diic many of the physical attributes, practices, habits — facial expressions, gestures, ios of walking, other forms of bodligexsy- of thelr parents or of others close to 1) At another fever TOTTWITCH TOrCES us to reconsider, profoundly, the Miration of categories such as mind and body, of cognition and affect. From an analytic perspective of a social-semiotic theory of sign- and meaning Jing, the following might be said. [he sign which the sign-maker has made gives us of the worid that Isat issue; am indication of their expe TAT Tees WME WOT, Signs are shaped by That and give usa sense of the ia, the principles, the interest, which ted to that representation. In doing that five us an insight into the subjectivity ofthe sign-maker. Taking this stance forces me to add another element to my example of the chess aps, There 1 attempted to show the motivated relation between form and meaning. Ht iy alscussion there Lt ot constr the- materiality of the form ofthe sign- ful it simply: my characterization of playing chess was—as T were aisembodTed, ed mentaistic account. Chess can be (and at times is) played like that, as ital chesd’; it is then regarded 45 a particularly difficult achievement. In most. of playing, the pieces are there in fully three-cimensional extension and for Mimesis, signs and embodied experience nthe reception of a sign the materiality of modes interacts with the physicloy) of bodies. (When t-seeargesture i understand itn large part in an action of ‘silent Sar mineds (Wal, 2005¥-Leame to unerstand te meaning say the eT th Sqeay OT RTNOTUVERER or pace ENT DOA aT EARS REMC Ua gre MONET Nae Trae tee mganing. There are, as well, the meanings suggested by signs in Uhe accompany ood reason: they have a material wali they have an aesthetic effect; they erlronment, Stans made in gesture are culturally shaped as are all signs in any 0 tellectual activity, At that level it comes to matter y shares thep ly iF it happens on foo miany occasions ~ that the button-pawn lies flat on the J ary sians we encounter ae in tiree-dimesional form; or they are in time; ae BBB Ts awkword to pick up, and takes away even slightly from my pleasure in signs we make, for instance, via the senses of smell oF tou. Heve too the conc «l Ml Mlayng the game. My pleasure lessens, Something i different, missing, The material mimesis is crucial. We engage vith the objects represented in the images in Colout | MMs pee — carved or cast —has a tangibility and an aesthetic which hasan effect om plates 7 and & ~ when we encounter their i a supermarket oF at home ~ not only fe AMTarialy, a a body in The wor miboaied a rraerTatity that18 Muitimadality ‘adds to my pleasure of playing the game. I might much prefer to play with pieces wi a certain dimensionality, weight, texture, aesthetic and which for reason of tho qualities give me pleasure. That sensory, affective and aesthetic dimension is tooo ignored and treated as ancillary. I reality, tts tsSOTUBIy part of semiosTs Boor vr Colour plate 2 CBBC homepage 2005 (Pages 37Colour plate 3 Salt and pepper, SAS: ‘health’ (Page 67)Colour plate 11 Hedgehog, Em Colour plate 12 ‘My Birthday’: affect (Page 109) Colour plate 13 Lego car (Page 120) Colour plate 14 Outside the roam: ‘Armenian wedding (Page 149) (Page 109) lour plate 15 Inside the room: “Armenian wedding Page 14! 5 Mode Moteriality and affordance: the social making of mode fis ssi shaped and cutraly svn sri resource for making meaning Inaye, writing, layout, rusk, gesture, speech, moving Tage, soundérack and 3D Dbjgcts are examples of modes used in representation and communication. Phenomena iW objects which are the product of social work have meaning in their cultural ronments: furniture, clothing, food, ‘have’ meaning, due to their social making, The purposes b¢+thetrmmaking and the regularity of their use Tn social life. As their_ Bary srtion sot ta of ereetton and conmnniaton theres a avesion lier They should be considered as mode=— ever thougtr We RTIOW Tat Mey can Be W#RTo make meaning and to communTeate.— The introduction ofthe concepts of mde and multimodality produces a challenge to Vitherto settled notions of language. Ate al, if all modes are used to make meaning It powes the question whether some ofthese meanings are merely a kind of duplication Hf wwanings already made in, say, speech or writing ~ maybe for relatively marginal Feasons such as ‘illustration’ or for aesthetic reasons such as ‘ornamentation’ — or Wether they are “full” meanings, always quite distinct from other modes. If the latter Ih the case, then *language’ has to be seen in a new light: no longer as central and Homhinant, fully capable of expressing all meanings, but as one means amang others fir makina meaning, each of them specific. That amounts to a profound reorientation, Wis the route taken in social semiotic approaches to multimodal representation. Different modes offer different potentials for making meaning. These differing pileitials have a fundamental effect on the choice(s) of mode in specific instances of Fiimurication, Writing Gn English, as Tm mary atfer Tanguages) has words, clauses, PONENCES, Organized through grammar and syntax. It has graphic resources such as Hoi size, bolding, spacing, frames, colour. To frame its units, it has syntactic, textual Wil social-semiotic resources (e.0, sentence, paragraph, textual block, genre). In Witing, the trames use graphic resources such as punctuation marks, visual means ‘ und paragraphs and increasingly, ‘blocks’ of Writing, often in diferent colours, on surfaces such as pages or screens or others. rent cultures. Crucially, cultures may use Aforont script systems. That makes it problemiatc to speak of writing as such; instead iced 1a say. iting in his culture ot Lat, with an alphabetic or a character-based wernt, What Was spice hetwren words or Those resources have specific forms in dif jlies inthis respect to the mode of writing applies to all modes, +30. Multimodality The mode of sp and grammar with writing, Th ? materia SUD speech however, sound, is enticey diferent from the graphics! aT wetting Sous received via the physiology of hearing; the araphie SUF oF writs CT ae ysology oF sight. Sound offers reources such as (variation in) reray —TOMIMESS OF SOTTESS— AHTEh cam be Used to produce alternations oF stress! “ecunsresec elements, oF rhythm and accent, wMich produce the rhythmic orga: “eal of spech andthe acentuatln oT WOE PIED A PEN Varo” TWAT i" ‘OF We Trequency of oscillation of the vocal” chords) prow re TanGUaG ‘ever a Marder or Too) and lneanattor Ty arguages Such ae ERTS Sew vowel quality, lenath and ues Sega ees one-or Te ng of sw. Suits, English for instance, the contours of lntoration are used ta make and frame Titration AigisTTRSEMATETIaT entities are used to mark out given and new VaTormation ineiniotic entities, information units. (Halliday, 1967). If say It was last Saturday he came! with high intonation on ‘last’ and falling away (i.e. contrasting with ‘rol Saturday two weeks ago"), the informational meaning is different to “it was ls Saturday he came’ (ie. contrasting with ‘not last Sunday’, These units, held together in larger intonational frames, produce chunkings ol rmeaning similar in some vays to the (written) paragraph and yet also quite ditt in their potentials for meaning, Sound happens in time and that allows the voi'v to ‘sustain’ a sound, to ‘stretch’ it as a resource for meafing — as in the lengthen) ES Ee TT a ape come Fer SRM, FEES Writing uses graphic means — bolding, size, spacing — (0 ) elev selec eters wilh ary only sue rene SuTree re ! SHUT SBBECT. BOTEAT and Toudness” differ materially; yet insome cultures ty are Tnfegrated Tito and ‘belong’ to one more general semiotic category of intens! Bolding in writing and “loudness in speech are both signifiers of intensity and are 0 ‘means of realizing meanings (signifieds) of ‘emphasis’. Ina social-semiatic approach to mode, equal ernphasis is placed on the afforda of the material ‘stuf’ of the mode (sound, movement, light and tracings on surfac etc.) and on the work done in social life with that material over often very low) periods. The distinct material of sound (in the case of speech) and of graphic stuf (i the case of writing) is constantly shaped and reshaped in everyday social lives, in \\w ‘most banal as in the most extraordinary circumstances. Banality and exceptionally ales ensure that ale sala if is captre in that shaping, Social —and there rk. with the same! material, sound can ead very different mod: of course, to soundtrack as 77 Im Janquages; to so-called whistle languages. These rest on different use of the affor ances of Uie same material fealures: variations ia_eneray_(laudness/sofin Teresi itch (igh ptivow pic, ising and fli oie “changes in dura) TEMES Tong vowelsehor VOWETST pauses and silence, Social work performed ceaselessly by members of social groups with ty affordances of the material, together produce semiotic resources. That is, mod the product jointly of the potentials inherent in the material and of a culture Hip tts underiying-sesniotierfogic™ The Togies’ of space and tire are profoundly fet and offer dane - - rhe modes. esture or movly i space. In gesture uence in time through mevement oF arms and ‘Mode lection from the bundle of aspects of these potentials and the shaning over time by jvembers of a society of the features selected. Hence the resources of modes, say e oF Speeci, are both similar to and different from each other within one society: weds and demands of its members, yet different in that the materiality of different Modes offers different resources and potentials for social shaping. That difference ‘MW modal affordance also leads to differential use and hence to a further cause for wons: the different requirements of different societies and their members and the Terman) differences in the-custural-use-of say, vocal. Wh as English, Frency, ocalic quality, and so on, Be rareeetevateian a nan ma ts Wariner Conmoraltes. Speech havpes tne. In speech one sound, one rd, one clause follows another, so ei isane Tundamental organ- ina orrchie raTaeIgearng in tis ode. THs shore across ‘are Simultaneously present; the arrangement of TT That space in relation to each other is one fundamentally organizing ent elements Th that (usually Wamed) space igo The Togic OP SEqUENTE Th Time is provided by the succession Of frames of images, ptt otet pwn ormOREDTBEE Hersey {W alphabetically written languages, wrlting SOTMeWaT OF border category in Breer 1s satially islaved yeti lane on speech ins lose scans fo ged i ig pn sce a ners Bs ipl systems which use the Tine Lan which anne olsplayes THis spatial display H writing and of its elements on the line, its ‘linearity’, gives to the impression Hat it works like an image. However, the elements of an image can (usually be ‘reac! Wan order shaped by the interest of the ‘viewer’, while the reading of writing Is overnied by the ordering of syntax ard the directionality left to right or right to left ‘the lie, Writing & not, dominantly and finally, organized by the logic of space; fs readers we are bound both by the orderings of syntax and the directionality of the Hine. Culturally, both speect and writing share ~ even though insignificantly dtterent al snsequent different shaping. As a semiotic resource, image in one culture is i identical to image in another: aly related cultures and ‘languages! or of pitch 2 #2 i Muttimodality ways ~ the organizing features of syrtax and the resource of lexis. Superimposed 0» that are the stint different material features of sound and graphic “tut, al socially shapes Looked atin this way, there are significant ferences between speech and itn ‘al times maybe more significant than Te stmitarities—This Makes it surprising thet Peony a soTa-senotc perspective We use ofthe oe label obscures Westnet as modes wit related yet importantly distinc aFfordences. “The resources of the move of image differ from these of ether speech or writin Image does not “have’ words, nor sounds organized as phonology, nor the syntax and grammar of speech or writing, nor ary oftheir enitiesnits. There fs no poi searching forsylables, morahemes, words, sentences clauses or anyother languas based category in image. While spech i based onthe logic of tie, (stil) ima is based on the logic of space. It uses the affordarces ofthe surface of a (frames) spacer whether page or camvas, apiece of wall or the hack or front ofa T-shirt. in mage, meaning Is made by the postioning of elements in that space; but aso by Size, colour, ine and shape Image does nt ‘nave’ words; fuses “depictions. Wor Gan be ‘spoken’ or "write, images are splayed’ Image uses depition’, cons 0 various kinds ~ circles, squares, triangles for instance. Meaning relations are esta! lished by the spatial arrangement of entities in a framed space and the kinds of relation between te depicted entities Given the distinct afordances of different modes they can be use to do spec semiotic work. The uses of mode constantly reshapes arGant2s along te nsf he Socal Trements of those who make meanings; that ensures that mode i constantly changed tn the direction of social practices and requirements. Mod Shang vats cl hag, We ci et do et ati ner elaborated inthe enities of ammode, Ara COnSaqUENE, Te FOTeKials inherent materiality are never Tony Uses to become affordances of a mode in a particular cultore; nor ae all the affordancs which are available used for similar purpo across diferent cultures. My mention above of pitch may serve as the example. tove-languages pitch i used — among other things - fr lexical purposes: differen in itch withthe same syllabic Cor mult-sylabi) form protuces diferent words. In English pitchmoverent is used for arammatical purposes: for forming question ov statements Pitch is used for lexical effet in English too, but to a much smal extent: ry saying "es" to mean “Are you realy sure? or Maybe’ ora whole range ster meanings. Societies and ther cultures select ‘waterials’~ sound, clay, movement (of pats) of the body, surfaces, woad, stone which eer sel or ecessary for meaning 0" Inthat eure te be done. Selections irom the potentials for making meaning wich these materia offer, are constantly made in the socal shaping of maces. 1 communities of humais wh are speechimpaited the afordances ofthe body ~ th positioning end the movement of limbs of facial expression ~ are developed ito Mode lable, only a narrow selection of these affordances is used, in gesture, Different “sities have selected and continue to select diferenty, shaping different cutural/ iviotic resources of mode. The focus on materiality marks two decisive moves: one is the move away from shag language’, the Tguste system’, grammar” and a rove towards he sneciticity of a mode and patertals as Geveloped in social uses. The othe Wir makes Tope cation With the bodiliness mans: nat only in the physiology of sound af hearing, oF Sight and see also nthe fact that humans make "0% af taste and tasting, Bi an [itirough all these means are the tact Twat all These are Inked ane Take MEANS gether Br ITY OFFerS Me pOSSIIMy OF Seer MEATS modied — as in our bodies: a means of getting beyond separations of those other stractions, mind and body, of affect and cognition, ‘The ‘reach’ of modes Biroons> rage withthe world through these socially made and eutturally spec Greer and Grey Gipson ways that arse ut of Ter Uterets. Tis gives rise to the ‘Well enough understood problems of (spoken or written) translation: certain cultural pains may be well supoled wih syrtactic and lexical resources in oe cuture but Bry suplied a arother, or cse adorn ay BeeTrcy ia as Set prod 4 parte varia oF WepIHler” compared tothe world that might ‘We hamed or depicted, The semiotic ‘reach’ of modes — what is ‘Covered™ by Uitte or speech or wrltng or pestare=Teatways specie ana paral Wa clare Nah aifereny spear paral This partiality Ut raming-or-depetion-of labelling, is a feature of all modes and all cultures: there are always gaps. Aveas in the Bip sclal attention are wll suplied wit sent resoures; others lax su'or ott ai Becitics have modal preferences: this mode is used for these purposes, that other ‘Wide for those other purposes. Over long periods, ‘Western’ societies have preferred Writing to image for most areas af formal public communication. Hence there is a UWileential erst" of semiotic entities, of naming and offers of relations even Within one society. If a mode is preferred for a specific social domain it will become ‘Wore specialized in that respect. Not all cultural domains either can be or need to be ‘Wieribed and elaborated equally in each mode, The material affordances of modes By io: the en” of a gesture nity variable expres fc ture fords infty gragabe signifies. Gestures areimperanent once made, pete trace roman ot them Gesere ning WareTaes Be Sow Ra OS THREATS War the reach’ of modes varies from culture to culture. What may be se by speech in one carte may DE Tanaker by gesture in another; what Way be-weil a3 U € V J done fi Tagen one culture may be better done WT 3 Torms Tr another, and so on. We anol assume that teanslation from one niode to that (same) mode acrass cultares— ’4 b Y b Multimodality wil works In other words, an implicit assumption, namely that ‘languages and now rai BeT reat wl HE Sarne cultural domalrs = even Tes wel toe elt and better in another — 15 Tikely to be unfounded. It may be that what is handled’ by ‘the Tide of speech in this language may be handled by the made of gesture in that. what may be handled by writing in this language may be handled by image in tha! ter socey. That ference ike to exis inthe use of modes een amon ctrl froups in one society. In ather ods, the asumplion that what Is represented in eech in Culture A will also be represented by speech in Culture B (of course wil different lexis, syntax and genres) may be quite unfounded. — OO What is a mode? ‘The former certainties about language had acted as a barrier to posing a raft of ‘questions, such as ‘What other means for making meaning are there?’ and “What ori they ike; what can they be used for, what can they do?’. Turned around, that certairy could have led to unsettling questions: ‘What, actually, is language like?’. ‘A ffordanc+ — the question of potentials and limitations of a mode ~ applies to all modes, av! ‘language’ is no exception (Gibson, 1986). The idea of limitations in relation to la ‘guage, however, is nev. But that question is now firmly on the agenda in a multimodal approach to communication. ‘Language’, which had been seen as a full means of expression; as the foundation of rationality; sufficient for all that could be spoken avi! vrritten, though, felt and dreamt (Eco, 1979), is now seen as a partial means of doi) these: oo Considered from the perspective of multimodality, profound questions around language pose themselves where there were none before. The question of ‘reach’ is 0° of these; it has three interconnected aspects. The first is: “What is the social ani cultural domain that it covers or that it does not cover?’. The second is: "What can i) ‘mode ~ image, speech, gesture, writing ~ do in the cultural domain that it “cove and what can it not do?. The third is: "What semiotic features are in the made on which are not, and why?’. These are often lumped together in the question: ‘What are the potentials and what are the limitations of this mode?’. These questions ap»ly to all modes. Just to give a flavour of what is entailed: when a gesture has been made and ith been interpreted, its meaning for the interpreter is ‘there’ but no material tro: remains. So some offensive gesture can be made, have its effect and yet be unavailal for examination. Positively, n the Science classraom a series of gestures may proc a convincing rhetorical effect yet be unavailable for interrogation after. Consider I example below, including Figure 5.1. Firct, a broad transcription of the teacher's (lh and an equally broad indication of the teacher’s actions simultaneous with the tal “The teacher's talk (where // marks off intonation- information units): ‘We can think about it as a circle of blood like this // going round /P and al various points / say // the lungs are here f the small intestines here // and the cell Mode are here / and the kidneys up here / okay //so it’s going all the way around // and what it needs // is something to start punping it again / to give it a bit more ‘motion to go around /f okay /l” The teacher's actions: Points at head; traces finger around circle; returns hand to heart; draws on arrons; places opened hand at left of diaaram; places opened hand at bottom left ‘of diagram; places opened hand at bottom right of diagram; places opened hand at top right of diagram; draws arrows on circle; points at heart; bends elbow; ‘rms at side, makes bellows action; makes bellows action three times; puts lid on pen, the final of four lessons on blood circulation, the teacher had drawn the single large le (top, Figure 5.1) on the board before the students had come into the classroom. the class has settled, the teacher begins his account of blood circulation. He ls out that the single circle is an oversimplification, that there is in fact another tle, which he draws, to make a double circle, with the heart indicated at the itine of the two circles. He goes on to say 'We can think about it as a circle of l0od like this, going round, and at various points say, the lungs are here, the small stine here, the cells are here, the kidneys up here, okay so It’s going all the way und and what it needs is something to start pumping it again to give it a bit more lon to go around, okay?” As he says this he places his flat opened hand on the left the diagram where the first arrow is, just to the left of the juncture, then below |-and So up to the juncture and then around the small loop. The point he is making that the biood is pumped around to af! the organs, that it circulates from organ organ. « t Figure 5.1 Blood circulation B586 Multimodality Mode 87 ‘The diagram, together with the overlay ofthe teacher's gestures, which indiat side and which are outside a mode and why, ‘Why is facial expression not part of ‘specific points on the diagram, ‘show’ plausibly what is going on ~ the movement of ph?" Tt seems that everything that happens ‘inside’ the barrier of lips and nose Ios terion, ipmcoren af We Madam ortan ong. The pesto a Bi sre pele ant nose vases of Se etycaliatatal tf soon spa ‘the organs is indicated (and a little later repeated and marked with a pen), as is Uv ‘Of speech. The sarcastic cur! of the lips is not. The first criterion is indecisive on the mmoverent ofthe biood. THs maping mightzere asa topological indcain of whe MMEMaer, te second makes lear a cur of We ip sro sound the organs are~thougias with ary topology woul be best rt to Iagine tht th Bi contrac on par fm ceo ne omar tou social kidneys are roughly at the same height in the chest cavity as the lungs, not to mention Practices and histories; about what has been essential, important, satientrarseciely_ {location ofthe Smal neste Nevertheless, the soqena of gestures ha ache Ms YORTONS Tic goes nt, However, arsver the question about potentials tte eacher waned achieve: Ue tngiby, mimeticaly witnessed movement iW ntatiens To do a we would reed to undertake ful inventory cach mode the blood from organ to organ. The fact that in that process the position of the orga’ yom that perspective, with an understanding of the potentials of al! ather modes in sins cra eee a a essa ele, parte etree. save ategiod a snl etn of ome Tears atee8 “Ty thatthe elle’ are not an organin the serge of the ngs nor tat they have Moder of speech ard writing There are some que simple meas of starting thal ‘peu allan ith body both gesture and pech had suggested" ela Bd acer xamol of saya "ex to ean Are you relly sure? It shows oe fee. Te ephemeral character of Gesture Ih eeu ety pate for patch How coud tsa meaning be realize in wr? moment Tor spa Fe OBA TTET|y AUN pest might be the and withthe par of the hand hel ued ups in fatal eBTTTESOTT TA? Doles, and hes po, th Trea DORI Tor ange wh pesion it might be the puesed expression or the ralsed eyebrow withthe corners i comMTEMeNt made beyond this Wow" ir, but you gestured . . ” does nat have tle “W the mouth quizzically turned up. Such questions would begin to give an answer Spe OUT CAT RECA OT TOCF OTS Be pearls fuse node ~ of mec gery Ring facta expression tis mmol we ace qrstre aot ase” Be we diversity of punom a ‘of their respective limitations. / $Ssembled unde he term Yargunge’ Booms apparent, We ca eXpE WAT Two further means of answering the question ‘What is a mode?’ emphasize the [Brera Paso telah a soles rire. ee alin Social Semitcs onthe one raassia e oni or apceh' ak ‘What pce of carerercs MMC yc fans Be iecry te cer, Toput wily Cacia, what cous asad ea rater ith speech we can ask: ‘What dees pitch variation have in common with lexis Wa community and its social-representattona iced. What a community-dockes-t0 V ‘What connects levels of eneray — loudness and softness - with syntax? Or tonal vara , I the community of designers have a need to develop ¢ tion with morphology?‘, ‘What do rhythm and pace share with vocalic and consonan\.a! i potentials of font or of cofour into full means for representation, then font and ‘eatures?’. These are entirely disparate kinds of phenomena, clamped together u lu ill be mode in that community. Of course their decision to do this wil not be Ret Mie to that comunity alone: ayone who come nt contact with tei work the label speech. Writing has a no less disparate set of features and we can ah “Similarly: ‘What does font have in common with lexis, or orthograchy with syntax” “Will become affected by that decision. [n semiosis, as in culture more widely, ‘no man {F woman — is an island’. ining al these features under one label, of speech or of writing, shows one problen Gul collapsing speech and writing with their entirely diferent materfality into 0» Formally, what counts as mode is a matter of what a social-semiatic theory of Jf) eteseny thereby joining and blurring over the distinct logics of time and space, of joe requires a mode to be and to do. In this T adont the semiotic approach of ‘Giorno egcs fay ela reals is difficult fo See what principles of coherence might serve to unify all these fear fs events in the world — the jdeational function, to repre So I take speech and writing to be distinct modes. ini meanings about the social relatTons of those engaged Tn communication — In sae aproaches seston ob prt of rg comple a reniny I soa TT RTE Te apy orm ees, tal Scape sean resources, together with speech (or as part of language’) (McNeill, 1988). Hero jes which can project a complete (social) world, which can function as comalet the issue of the entirely diferent materiality rules that out as a pessibility in tiv Aporoach here. The affordances of the material stuff af mavements (largely of ha = prisinaly then, there is no straightforward answer to questions such as: ‘Is ‘though other pai Involved - the shrug of the shoulder, the pokii») Wi 7 cur ofthe tongue -iewitrely diferent Jona ve; is Layout a mode; is eotour anode. This ies inthe face of traditional Thats one sp toward arsverg the questi What a made. Aer al ‘ptions: here, what alonguage s, was Keown, feed, given treating speech and writing as modes is to accept thal modes consist of bundles «J To the question “Is fort a mode’ there is, on the one hand, a socially oriented answer (often deeply diverse) features. Questions whieh might ‘Ait a formally oriented one on the other, The former seeks to establish whether here cal y oriented ane on former seeks to ‘their environment ~ the textual SARE CHTTTCS WATCH cahere Internally and8 Multimodatity is. sources of font with relative regularity, consistenc ;.groun of neoole who use the resources of font with relative regularity, consisten afaulth shared assumptions about ts meaning potentials The second answer ) cencered te zac aT does Tine three theoreveaTnctons ,rSquTereis ofa theory of communiationT Rojhae sli HG far Bat gragp. Such ar approach is unlike present and stil active unde-standin around language for instance. This new approach seems improper, nearly, to those o ls who had been socialized in a period when an authority decided such issues, base) P “atc Fo sean st are such that it can become mode. Most of us are aware of the meaningfulnes of font in our everyday representational activities. I am typing this text using ‘Arial preferring it to ‘Times New Raman’. Newspapers tend to stay with specific font whether for their mastheads or for particular sections ofthe paper, a clear indicatio of the meaning that adheres to fort. Take colour as another instance (Kress and van Leeuwen, 2002). Most of * Conere the "us’ always refers to a socially and culturally specific group) have qui a strong sense of the meanings of colour; whieh isnot to say that we could easy articulate what such meanings are, The meaning of “he litle black dress’ is both tint it is little — with meanings about gender and the erotic ~ and that it is black ~ will) ‘meanings about the erotic and its Inks with power, Cultures where the phenomeno of ‘power-cressing’ exists, have their colour schemes, whether for men’s suits, shir\: and tes, oF for women’s skirts shirts and shoes. In ather words, meaning can be mai through the affordances of font as it can through colour. These meanings are cially pith te socially ore aproae, sare understandings and pacties ar focus. With the theoretically oriented question ‘What is a mode?” the focus ison communicational task we expect to be able to do with the semiotic resource: neti fort nor colour can be a mode if they do not meet the theoretical requiremet stated. An approach via such general function allows us to test 'eandidatesfor moi status; we can ask whether and how ary semiotic resource meets these criteria 9 instances of use. With that in mind [ as: Is layout a mode?” Is layout a mode? Coreider the flare fist shown i Chapter 3, which i sown again here'as Figure 5.2 It is froma textbook for 13-14 year olds, published in 2002. My tree cues tre: Can layout orn message-entiis we are inerally coherent and wiih cober tb alata Ch sual fete“ apa egress the socal relations Gf Those Sigaged in communication?” (he interpersonal Tomato ssi" avout represent meanings about the word of tales, a (fhe Weational function). SS ‘Fhe-question ‘Can-there be versions of this “semiotic entity’” (the “page”, tl ernatly? deals with “double-page spread”) which are incalmrent internally and x Mode 89 ontral requirement of the textual function, coherence. This ‘semiotic entity’ appears Jn a school; for a class in the school-subject Science, in a science lesson; and in a sghool year for which it was designed; it has a place in a curricular sequence, and so fo), Clearly, the text coheres with its environment. Conversely, wwe can Imagine any umber of environments in and with which it would be incoherent. As far as inter oherence is concerned, a simple testis to ask: “Will changes in the “arrangement” of | th elements witich make up this “semiotic entity”, produce a different text?", ‘Will ich changes produce incoherence?’ or "Will such changes make no difference at all?’. i, no matter what change we make, the text remains coherent then there is nothing {p say as far as the textual function is concerned: it does not apply. lon to ideational and interpersonal meanings, the formal technique o ‘am concerned with, by subst s Feu rose and the mists dissolved The mists dissolved and the sun rose. The two clauses “the sun rose’ and ‘the mists Uisolved” remain the same; their oer has been changed by commutation of one with Mir other. As 2 consequence, the meaning of the two sentences lifes. To apply ths test we need: to know what the elements ofthe larger unt are. In my Hample just above, the elements are the two clauses the mists dissolved and the sun “Jose and the conjunction and, In Figure 5.2a, on the left-hand page the immediately Ahvious elements are the two columns ~ of writing and of image. I call that the ‘Wiohest’ level, Level 1. Within each of the columns there are further units. The column “Bf writing consists of three ‘blocks’, each with a heading, at Level 2. At one level ‘Wuriher down, Levet 3, the top-mast block consists of two segments and the bottom- ‘os block of three. Turning to the image-column, at Level 2, the column consists of {wo images, the one at the bottom much larger than the one atthe top. The top-most ‘Wage, Level 3, is relatively easily analysable into three elements: the left-most, the Ayotral and the right-most, tagether with the arrows which connect the lft-most “Alement tothe central element and which connect that in turn to the right-most. Inthe ‘wiginal, the lower image (Level 3) shows a division of a pale pink background and a “wreenish- blue foreground, e.. the digestive organs. This simple description allows both the identification and the rearrangement Hl entities atthe various levels. At Level 1, Ican switch the left-right ordering of two Alums, as in Figure 5.2b. 1 can commutate elements from Level 3 across the {wo columns, as in Figure 5.2c. Now the sun no longer causes photo-synthesis but ‘ines instea! on the nether regions ofthe digestive system. The offect is clear: when the two columns are inverted, the meaning is changed; though the page stays coherent. With other such changes neither the page-as-text nor | the {wo largest elements any longer cohere (Figure 5.2c). There was coherence * Welore; now there no tonger is, The ideational aspect ~ meanings about the world of states, actions and events Jinthe world ~ is clearly involved in the conwnutation of Figure 5.2. In a culture with0 Multimodality Figure 5.2a 2002 Science: digestion a left-to-right reading direction, left-most and right-most position have differen! ‘meaning-potential: ‘eft’ as the point from which I start, and ‘right! where I am mov iy to, give different information value to what Is placed there. So in Figure 5.2b, {lv physiological aspects of digestion are taken as ‘given’, as ‘established’, as ‘what already known’; while in Figure 5.2a it was the bio-chemical aspects which were presented as already known. Figure 5.2c however, is not a coherent layout. Take the example of the char arrangement of the Level 2 element, the image of sun-arrovr-plant-arrow-rab Here the original has a broadly (causal) meaning of ‘sun (unit 1) provides energy (\\ arrow, unit 2) to a plant (unit 3) which (in a transformed state, suggested by |! arrow (unit 4)) a rabbit (unit 5) ingests’. A rearranged version, say 4—5—3-2-1, |. nothing like that meaning. We can labour and give it a reading ‘a rabbit, identifies ty an arrow, has turnad its back on a plant; for some reason it seerns interested i ty ‘writing to its left, while the sun sets’. The rearrangement has led to a change, ari! \ considerable puzzlement if not incoherence for a possible reader, However, it might be said that this reordering affects elements of the mode of in. rather than settling the question about /ayout ~ the arrangement of elements on a» ‘of appearance as mado. Before answering this second question I turn to the third: ‘Does layout enable ws 1 aged in communication represent meanings about social relations between those Mode 91 Figwe2 ‘Multimodality In Figure 5.2b, the order of the tivo columns is inverted. In this Western’ exampl the taken-for-oranted reading direction Cw art fcom and where we 90 to!) | from left-to-cight. In the culture for hich bis page made ie EGR OE meaning of: left = information assumed to be ‘known am-as‘social: ‘ven’, and right = information regarded as ‘nen’ Taformation, as not known to th “airessee an not shared by Te socral group. The classification of information as either ‘already known to the audience’ or “new tothe audience’ has social and ontological consequences. In each case a partic lar status in relation to knawledge is implicitly ascribed tothe audience, and furthr communication, further social interaction proceeds on that basis. That distribut of ‘given’ and ‘new’ ascribes, whether correctly or not, specific social characteristic to the audience. That ascription organizes the communicational relation of mak and receiver of the message in a specific way and, in that, has effects an their soci ‘elation. That meets the theoretical requirement of the interpersonal function. a return then tothe question of ideational aspects of this mode, which may bet most vexed. The organization of material through layout produces and reali specific social and ontological arrangements in the word ofthis interaction, In socs semiotic approaches to multimodality itis an absolute assumption that modes hav different affordances; speech and writing, for instance, ‘name’; image ‘depict gesture ‘enacts’ and ‘indicates’, emphasizes and ‘sketches out’ themes and topic Layout does not name or depict; it does however ‘dispose’, organize and indical ‘aspects of The Soclal/ontological ‘status’ of representations, as “known” and "give a5 part of my group OF NOE; epistemolagicaly, as ‘knowing or rot'; and ontological in indicating the ‘social status of knowledge’ The disposition of elements in a framed space ~ a page, a screen ~ does not ‘nan as words do and it does rot “depict” as (elements in) images do. It does hawev dispose information in semiotic space; it positions semiotic elements and thei reli tions; it ‘orients! viewers/readers to classifications of knowledge, to categories su! as ‘centrality’ or ‘marginality’, ‘given’ or ‘new’, ‘prior’ and ‘later’, ‘real’ and ‘idea! These are states of affairs in the socially made world, aspects of the ideational/ experiential function; different to, yet as significant in their ways as ‘namine ‘relating’, ‘depicting’. It layout the st 19de, it has consequences for how we thik about the scope of the ideation: in comparison with the alToroa " ‘other monies. Tewhere, thinking about meaning and communication Tei Terr chpe by a previoalyvoaselaned nso arguage’ sal cee of op senator. We cannot afford tote Seeman based hina to erstratn hi whe set mou CE SCATOE TERY Te RES a UT Sea {akin Meanagser tO TS That rethinking, new thinking and, wih thal, ew nani Gooures eer As hae poled ow i procedog chap, wat fe nested ar categories at a level general and abstract en all the meanings ol contemporary social life in the multimodal communicational world gh to. encompa Mode 93 ‘A nwitimadal approach to representation offers a choice of modes. Depending on hve rhetorical requirements and the media involved, there are different possibilities: do ‘you wish to realize your meaning mainly as writing oF mainly as image, as moving [nage or a&-sssek? Te existence of sh coe revels that ‘esr ey (ist other than when it has become materialized, realized as mode or as a multi- Tevien niosis_ the making of meaning, i isa ‘prompt’ to make W Sess visible, and there is then an ‘utterance’, ‘Ward material sign-complex, always as a soars WIE TOMES punctaton of miosis: the ceaseless BrOCeSS OT WIE? FTaETIG f Halted for a moment It is ‘fixed’ and it is ‘framed’ (Kress, 2001). The example below, first discussed in fapler 4, Te an instance. A class of six-year-old ina school in South London are taken Uy heir teacher to the British Museum. On the day after ther visit she asks them to Ihre a ‘story’ and to draw a picture oftheir visit. The ‘story’ is reproduced again here. When I got to the museum it looked bigger than J thought. When I went in I tock ‘ff my coat and went into the men’s toilet and after 1 ran upstairs I went into the lift. Then I went to see the mummies and all those stuff. Then we went to our cloakroom so we can get our coat and then we went to Waterloo station on a tube ‘nd a train to Clapham Junction and walked back to school and went home very happy and [ told my mum, sister and my brother. The end. The topic of ‘story’ and drawing were the same: the vsit to the British Museum. Yt the mode of writing, leaning, at the age of six, heavily on the organization ofthe Mode o speech and tpespora gre FOUN wth its emo oe of emeora Seauence of actionfevents there in chronological order) leads this young man to an ‘mlieely different representation to that in the mode of image. The written recoumt ‘Wnplicitly poses the question: i ‘der did they occur?” The made tial Togie HPTCIy ASKS Te question: Suna wer nt objects for you in that day and what ordering do they have forme?! The relation of meaning and mode presents itself in three ways: First as a rhetorical Jhsue, a matter jointly of modal affordances and of rhetorical requirements. Which ode is apt and therefore ta be selected, given the rhetorical task to be achieved? fiocond, given the different orientations of modes and their different ‘takes’ on the World, it presents itself as an_antological and epistemoloaical issue; this will be fxplored in the next exg n the case of arrangements of several modes fimultaneously, in ‘multimodal ensembiys’, Wnresents (seit as We question of Invderammerte sofectoc and inwhat ‘arrangements, a matter discussed in Chapier 8, “Tira social-stmiotic approach to meaning-making, it is seen as both social and id social and ‘internal’, The social (inter-Daction in which v is outw1 Multimadatity ‘meaning Is constantly created, in a transformative process of Interactions with and response to the prompts of social others and of the culturally shaped enviranment; an there is constant ‘internal’ action, an (inner) response in constant engagement wt! the world. Most of this semiotic action never sees the light of day, so to speak. When it does, when this flow is arrested momentarily by social and individual need an design, the question is: "In what mode(s) should meaning be “fixed”?’. I am borrow ing the metaphor of ‘fixing’ from older forms of photography, where a chemica substance on a film was the means of ‘fixing’ light and thereby ‘fixing’ that which wa the subject of the photograph. Different kinds of chemical coating provided for and ‘departed’ < ca Figure 6.5 Giorgia Meaning as resource: ‘naming’ ‘and connected features: knowing that I am in a sermon (orientation) stil! Ieaves me in need of features so that I can navigate (defxis) in that (discursively organized) social space. In Figure 6.5, a drawing made by four-year-old Giorgia, orientation and deinis are tevident in a number of ways: as orientation, in the domain, that of the family; in a domain of affect and aesthetics; in the social domain of the child framed by the house, by hor parents, in her family, in between mum and dad; affective, spatial, social. Spatial deixis is present in avert markers of place and position ~ between the parents; iv her drawing herself as being auite small; of her admiration of her mother shown in the ‘colourfulness’ of the drawing of that figure. Deixis is a socially produced cultural-semiotic resource. Consequently the deictic resources of cultures differ widely and at times in ways that seem inexplicable. The Dyirbal are an Australian Aboriginal people living along the escarpment of the Great Dividing Range in far North Eastern Australia. The terrain is cut by steep ravines and deep valleys; it is covered by dense rainforest. Dyiroal, the language (Cixon, 1976), has an enormously finely articulated set of deictic particles; not just the English that, this, there, here, out forms which allow a pinpointing of the Kind ‘there, 200 yards upstream across the creek, at the base of that hillside over there’, or ‘up there, Inthe undergrowth on the side of that creek’. In the rugged terrain ~ the rainforested ‘escarpment ~ which is the home of the Dyirbal, that degree of precision is essential {or ordinary day-to-day social and spatial navigation. Linking Linking of entities — humans with humans, with places, objects; objects with objects; ‘Objects with processes; processes linked with processes ~ is a major resource for ‘making meaning. Much of semiosis is about linking of various kinds: linking by and through actions; by adjacency and proximity, temporal or spatial. Again, cultures Wiffer vastly in what their societies have regarded as essential, as central, as marginal or as simply not present, and not visible therefore. Modes differ in how their atfor- ances for realizing such relations have been developed, along the demands of the society in which the mode has been used. Human action and (inter-action in and on the world, involves relations between participants people, objects, phenomena; human, animate, inanimate — in processes ff action and of relation, of doing and being, spatially, temporally, socially. Participants fare involved in various ways ~ as ‘doers actors’, as ‘done to" sufferers’, as those who fare ‘beneficiaries’, or as those who are simply ‘involved’ in some circumstantial way. The range and kind of such relations varies fram culture to culture. Actions and relations are initiated by one or more of the participants; or actions and relations may be represented as simply happening, as simply “being the case’, in image, in movement land actions in space. Relations may affect one participant alone or many participants {sin games played in a schoo! playaround — or they may involve reciprocal actions between the participants. ue‘Multimodatity The possibilities of realization of such relations vary from mode to mode. Whatev: the mode and its affordances, its forms of linking and separating ~ of participant: objects, events ~ are always meaningful. Originating in one mode, say in play in th schoolyard, a meaning moves across to another mode and genre ~ in a spoken recour! of the game, for instance, in forms apt for that mode. It repays close attention to se what actions, relations and processes exist in any mode, and what relations can b» imagined or are implied in that mode: whether in the tableaux which young childee ‘constantly make on the floor of thelr bedrooms; in a comic strip; in textbaaks; in as Illustrated book for children; in a Lego construction (Kress, 19974. The three-dimensional features of Lego blocks and their constructions (Colow plate 13) afford meanings about balance and symmetry, something far less availab| in two-dimensional drawing and not available ~ or utterly differently — in writing 0 speech. These are the building blocks of arrangements of all kinds, in all modes, « actions and events in time — to precede, to follow; as actions, events and relations \ space ~ to be before or to be after — to invade, to dccupy, to inhabit; to be at, to com: from, ta be with, to 90 to, 10 lag behind as relations of action and Interaction; « relations of existence and being - to be, to become, to happen. There are actions whic\: more directly show, or demonstrate or name social links ~ to involve, to follow, \0 co-operate, to join; in actional and non-actional relations such as to be inferior, t0 below, to equal, to be above (Hodge and Kress, 1979).. ‘Many events, phenomena, objects, are linked, that is, they have been brovs) together, collected together, as the result of social decisions and judgements. Thess relations are handied in distinct ways in any society and its cultures and hence handles! distinctively semictically. Linking may be the most useful general term. The question socially and semiotically is ‘What forms of linking are available in a particular soc and cultural world?’. Are the links made through actions or through various forms 0! non-acticnal relations? Are they links of spatial or temporal contiguity and proximity Other terms name specific relations of linking: conjoining, adjacency, for instar« each of these being specific in particular ways. There is a judgement to be mah between the need for general, less specific terms and those needed to name partici ir relations. Processes and effects: making and remaking meaning ‘The processes described in this chapter relate to meanings as resource. The repros tation of meanings is the subject of the next chapter. In social-semitic theory ot) ‘processes of semiosis also need to be described. ‘Semiotic work’ names all proces which are part of the making of meaning ~ in the ongoing process of semio externally and visible, or internally and not immediately) visibte, ‘Meaning as resource: ‘naming’ Processes of semiosis, ‘The processes of serniosis are specific; and in this chapter several are identified and famed. The making of a text ~ as an externally realized semiotic entity ~ entails that the flow of semiosis be arrested for a moment. That constitutes 2 momentary puric- ation of semiosis in which processes of framing play their part. Preceding, though also concurrent with and in a lesser way following that moment there are rhetorical processes. In these, the rhetor analyses and assesses the environment of communication fand reflects on the resources needed to shape the message and the media to be used for maximally persuasive effects with the audience of the message. Using the term rhetoric esponds to the new social and technological givens and provides a necessary frame for considering communication at all points. Rhetorical processes Rhetorical processes underlie, precede and then become design processes. The rhetor's {ask is a political one, with political and communicational effects, namely to provoke ‘and produce the rearrangement of social relations by semiotic means. The designer's tusk by conteast is a semiotic one; to transform political intent into semiotic form In the present period it may be essential to focus on the contrasts between desian wl critique. While critique attempts to uncover ‘political purposes’ in the effects of past uses of power, of past rhetorical action as it is manifest in existing texts, design ‘lms at implementing rhetorical intentions, which will have their effects in the future. Critique is necessarily backward-oriented, focused on the past actions of (usually) powerful others; desfan is prospective, future-oriented, focused on putting the rhetorical conceptions of the rhetor/designer ~ mast often the same person ~ into the world. Before the processes of (material) framing can take place, the meanings to be framed need to be fixed modally — using fixed in the sense of the fixing of light on the prepared chemical surfaces of older forms of film. The fixing of meaning has several faspects: in choice of modes, of discourse, of genre ~ as an answer to such usually ntirely naturalized and therefore implicit questions ‘In what modes, in what iscourse(s), in what genres shall we present the meaning?’. The selection of modes fects choice of discourse and vice versa, as each of these does with genre, with its {vestions about the social relations which shape and appear in the text: as editorial Ina newspaper; as a contribution to a professional journal; as an entry on a website; 8 an uploaded video on YouTube; and so on. In other words, meaning is fixed three HHmes over ~ materially and ontologically/semiotically as mode; institutionally and (pistemologically as discourse; and socially in terms of apt social relations, as genre. qi2 Muttimodality Framing Framing of various kinds is essential in all decisions about punctuation of semiosis av) the subsequent fixing of meaning in modal, generic and discursive forrn: through mos: and its entities; through available types of genres, through available discourses; arn) Framing in forms of its dispiay onfin sites of appearance and media of aissernination ‘The process of making a text can be imagined as a sequence: starting with a promi and the interpreter‘s interpretation of that prompt. On that basis the rhetor shar: a message as a response, influenced and shaped by the assessment of the comm nicational environment and their interest in that erviranment. The rhetor’s assessmor and interest leads to design processes in which the modal implementation, a8 orchestration of modes, apt for the rhetar’s meaning, is settled and shaved, usin) available culturaVsemiotic resources. This Is a nev punctuation of semiosis; wit meaning fixed in modes and frames ~ modal, generic, discursive ~ apt to the socio! ‘environment. This constitutes ~ at the sare time ~ the processes of production, lead) 10 orchestrated multimodal ensembles as material text Semiotic work has social effects. Social as much as semiotic work is preceded by ye! other processes: of attending (attention) to an aspect of the sacialAemiotic wor'd, 1 this case the prompt, of choosing (choice) of processes and resources and of tl consequent apt selection of these in the shaping of a response. Process and change The origin of decisions to link and the principles for doing so ~ depending on the kin of inking involved — are judgements about similarity and difference, relatedne complementarity, functions which link actors and objects, objects and objects, accor ing to culturally specific criteria. As a consequence, Jinking entails and rests 0 Classification, more strongly or less so. In any one society and its cultures, classifi and the forms of arrangements available are strongly related. Classification Classification is a social and semiotic process carried out by semiotic means. Its eff is to stabilize the social world in particular ways; as such it has far-reachin co" sequences. Classification provides the legitimation of present and future actio" processes, judgements and valuations. Classification reaches from the most innocu.0\ ~ the ‘private’ holiday snap, the class photograph to the mare cuspish wed) photograph, to the anything but innocent official photograph of participants at 50 significant meeting (e.g. the once famous photos of the leadership of the former Soviet, Union arranged on the dais of the Kremlin Wall on the anniversary of the Octo Revolution). The processes of classification are active in all semiotic processes avid entities: whether in architecture, in newspaper layout, in darnestic design, relations. Its seemingly innocuous character helps to make its poll in family Meaning as resource: ‘namin’ effective. Given that the semiotic resources of a society are constantiy (re)fashioned In the social-semiotic work of interaction, through the resources of its signifiers, Classifications reflect the social organization which has produced them and which is constantly reaffirmed, remade and naturalized through thern. Meanings of causality as against a-causal meanings) are entailed, more strongly or less, through ‘action’ as well as through the ordering of events, states or processes and the participants involved. Ordering rests on and suggests an interpretation of the ‘priority’ among phenomena, where ‘prior’ can have any one of a number of meanings: prior in ‘significance’; prior ‘temporally’ or spatially; prior causally in relation to the Subsequent phenamenon. This can be realized in spatial relations within and between Images; as Yirst! in @ sequence or as last; in moving images and their temporal rela- tons. Classifications can be established using signfiers capable of suggesting ordering of any kind. Inthe two images of Colour plates 14 and 15, ‘prior’ and ‘later’ depend on a sequen- tial ordering, suggesting ‘entering’ and ‘leaving’, for instance. This isa different means ff achieving a ‘similar effect’ to the placement of the arrows in the passport stamps. Such meanings have distinct material and formal articulations in different modes. Some modes have a more elaborate set of terms in one semiotic area than do other modes. What Is expressed by spatial or temporal adjacency, or as position in the Sequence of a linear arrangement in image, may be expressed as a preposition in speech or in writing; what is expressed as the word ‘follow’ in speech or writing, can be expressed through temporal order in speech, dance, in gesture; or through spatial ‘ordering in other modes. Some terms and the meaning-potentials narmed by them, may hot exist at all in some modes; or, putting it in terms that | have used just now, some modes may not have the potentials to realize certain serniotic features. In Figure 4.2 (page 55), a drawing by a six-year-old football fan, linking is done through spatial proximity, Additional meaning is conveyed by the degree of proximity ‘and by the orientation of the players, facing each other. In London teenage speech the heading might be “You look’n at me?’ spoken with steeply rising intonation, aggres- sively challenging in tone of voice. The mode of writing adds information, though in a plain form. Many of the meanings discussed here can be realized in different modes. How they are depends on history and culture. In the passport stamps (Figure 6.4) the irection of the arrow — either pointing in to the littie rectangle or ut — ‘realizes’ What in writing would become words such as ‘entering’ or ‘departing’. Certainly, the stamp ‘means’ that, and yet it also ‘means’ differently and more: itis, literally, ‘spatial’, in a way the word ‘enter! is not. That is a reminder that ‘meaning’ exists in Ite specific realization, when it has heen fixed in a specific made and framed modally/ ‘ontological, generically/socially and discursively/ideotogically. It is not difficult to verify this for oneself by constructing ‘glosses’: ‘translations’ land transductions of images such as Figure 4.2 (Chelsea vs. Arsenal) ~ a visually! spatially realized form of a meaning that might be realized as ‘confrontation’ in speech or writing 1234 Muitimodality Translation Translation sa process in which meaning is moved. It is moved ‘across’, ‘transporte: — from mode to made; fram ane modal ensemble to another; from one mode in om culture to that ‘same’ mode in another culture —what has been regarded as translatio. from one “anguage’ to another. Meaning is moved from one genre to another; fro) fone discursive/ideological complex to another. Meanings connect across made ‘genres, objects. The processes and facts of classification as they appear in architectu are connected with classification as it appears in the organization of a family or of » business; they in turn are reflected in classifications apparent in photographs arin organization of a business, reflected as in the visual organogram of its manageme' structure Processes of "moving meaning’ and of relating meanings across modes need a nari The term transfatfon most readily suggests itself. Here itis used to name a major r articulation of meaning, of whatever kind: from one ‘semiotic domain to another from one semiotic category to another. This might be within one culture, for instav= froma text in one genre to a text in a different genre, staying in the same mode. O° i ‘may be a mave from a mode in one culture to the ‘same’ mode in another cult a novel can be translated from English to Russian, staying within the mode of writi Or it may involve shifts in mode, from writing to speech, sound track, music, a5 in (i translation of a short story to a radio play. [t may involve shifts in discourse; either within the same mode or with a change in mode. Here, translation is the term used | describe significant shifts in meaning: across genres, across modes, across cult and across any combination of these. If transition is used as the most general {21 to name changes in representation, it leaves the need for terms to name changes of more specific kind, The two terms used here are transduction and transformation, When the articulation of meaning involves a change of mode it necessarily entails a change i the entities of the mode. As mentioned before, the modes of writing and speech | words, the mode of image does not. The change of entity in the re-articulation |: 9 thoroughgoing pracess; for instance, it involves a change in ontological orientatio ‘The term transduction is used to signal this process of drawing/dragging’ means across from one mode to another. In transformation the process involves no change mode. As a consequence, there is no change of entities; the process operates on oti with the same set of entities. The process is not as thoroughgoing: for instance 10 ‘change in ontology is involved. Orderings of entities are changed, but staying wit) ‘the set of entities of the one rode. To restate: there are two kinds of moving meaning and/or altering meaning: on by moving across modes and changing entities (and usually logies) ~ transductoi the other, staying within a mode (and staying therefore with the same logic) but reordering the entities in a syntagm ~ transformation. Meaning as resource: ‘naming’ Transduction Transduction is seen as subordinate to ~ as one kind of ~ translation. It names the process of maving meaning-material from one mode to another ~ fram speech to image; from writing to film. As each mode has its specific materiality ~ sound, move- ment, graphic ‘stuff’, stone ~ and has a different history of social uses, it also has different entities. Speech, for instance, has words, image does not. That process entails a (usually total) re-articulation of meaning from the entities of one mode into the ‘entities of the new mode. ‘Asa (not so) simple example, imagine the transduction of the meaning of the image Jn Figure 6.5: not as a description, as in: ‘In this image J see three figures, one seem- ingly a child... .”, but as a full recasting of what that image means, though novr in a spoken story’ oF recount: “I think my mum is the most beautiful mumn in the world. J love it when she lets me wear my party dress, My dad Is very nice. He has a bad leg ‘and has to carry a stick.’ In the image there are no ‘names’ for any of its entities or their relations; nor for the attributes of entities ~ if we should even describe it like that; wards have to be found for these, but the selection of these words rests on our selection of specific aspects of the meaning of that image: others could have been selected by other ‘transductors’. Where in the image there is colour, size, distance, eight, and so on, to realize affect for instance, words now have to be found. There is ‘need to ask seriously how a meaning realized in one mode can be newly articulated ‘a5 meanings in the new mode(s); that is, how the meaning of the original can be (1e-Jarticulated in the ‘translating mode’. In gesture, for instance, movernent is used to ‘mean’ moverent — an iconic relation of meaning and form; in writing, movement Is conveyed by a name ~ ‘he walked siowly. ‘Many of the examples in this book involve transduction. The change from the diary ‘account of blood circulation to the concept map involves transduction. It is an absolutely common, constant, ordinary and profound process in everyday interactions: ‘our dreams are made of it. Here is one other example. Figures 6.6a and 6.6b exemplify number of points about transduction and about the process of synesthesia, on which transduction vests. Figure 6.6a is a page made up (by me) of six small square pieces of paper from a phone notepad. One summer weekend, the then five-year-old fad been drawing these Images, unbeknown to his parents, who were entertaining friends in the garden. Seeing hhim laying them out in pairs on the floor in the hallway of their house, I asked what the was doing, Of the first pair he said: ‘The boy is in life and the dog is in life, so they're in the correct order.’ OF the second pair he said: ‘The plane is in the air and the bomb isin the air, so they’re in the correct order.’ OF the third paie he said: ‘The patterns are in the correct order.” Some few weoks later, at the end of the school year, he brought home his eho! exercise books. On leafing through them, somewhat aimlessly, I discovered the page shown in Figure 6.6b, a page from a school exercise book, The date on the page reveals that the exercise had been done some four weeks before the images in Figure 6.69. rrMoltimodatity mE 487 E Deh HE IW LiPE, eRe in He Unteer chee r ine Foriny Bee 2 in EH dnc HE denne ti Mh fo EV in HE Cee onan” ee {WG Faas ME in HE Cmte ence K ¢ > Figure 6,6b is an exercise in classification — “inking like with like’, When that wa completed, time left in class had been spent, it seems, in colouring in the shapes: exercise in control of pens and a means of filling time enjoyable to the children. {question is about the relation between the two events, separated by several weeks, 0) clone in schoo! as ‘work’, the other out of school and done cut of the child's interes Unbidden. Classification connects the two, though in the interval the characteristics of that notion ceem to have changed significantly: ‘like’ is still connected with ‘lk though now the notion of ‘like’ has become more abstract, much more general ‘Semiosis, involving classification, had clearly continued, silently and invisibly, so | speak, At a particular point ~ the prompt is not known to me and I did not ask — inne semiosis became external; it was fixed, punctuated, framed. The conception of ‘like was ‘framect” and rodally “fixed! in the shape of the six images as well as in thei Figure 6.64 Michae\ classification 1 (hon ‘Meaning as resource: ‘naming’ nic HAS Draw a line to join the things which are the same. Figure 6.6b Michael: classification 2 (school) laying out as pairs on the floor. In Figure 6.6b, the process of linking “ike” with ‘like’ was done manually; the manner of halding the pen and of drawing the lines typical of A five-year-old is vi the lines, The act was physical/manual and semiotic/ onceptual at the same time: as indeed it had become in the later instance, now with a imuch more general sense of ‘ike’. Again there is the material stuff, the bits drawn ‘00 paper. That material act of laying out followed the material act of drawing and that followed tho inner concaption and design of the drawings, assume that language was not iavalved in this at al, at least not overtly, audibly. Language as speech appeared only when | asked Michael what he was doing and he ‘ieoded to formuate a response. Over the intervening weeks, semiosis in relation to this topic had obviously continued, though where before there had been the teacher's, es by a fine, demanding a material, a visual as well prompt to link ‘ike’ entitiesfim a7: Multimodatity as a conceptual/seriotic act, now the conceptual work of drawing entities which wer Mike’ at some remove in abstraction and ordering them in a layout that showed thei conceptual order, was done without speech or writing. Before, the "like" entities had been on the page of the exercise book; the task wa the linking of entities present and that was done manually/graphically. Now entities that were (going to be) ‘like’ had to be imagined, then drawn; at that point the linking had been done conceptually; the Tinking was then done in the process 0! laying the cards out in their matched order to show the classificatory schema. Layo! was the made used. These processes all involved transduction, as well as a ‘mov’ between ‘inner’ and ‘outer’ representation. My verbal prompt 'What are you dois thera?’ then produced a further transduction, from layout on the floor to a spoke, account of the principles of layout and classification. ‘This example has been enormously insightful for me: it exemplifies the ‘trac! between inner and outer representation; the evidence and persistence of semiosis on prompted: the prompt in school and the action five or so weeks later; the trav formative and transductive character of semiosis — whether from the relatively sim task of classification, prompted in the classroom, to the later, unbidden, much mor abstract version; or the transduction of imagined entities —> to drawn images —> to tn ‘ordering in layout an the hallway floor —» to the transduction, finally (as far as ti example is concerned) into speech. Given my present professional environment, my interest focuses on learning; © therefore on means of getting insightful evidence of learning; an means for thy recognition of learning; and of course, real questions about assessment of learn) ‘Any principles of assessment need to include the realization that wherever seriot ‘work has been done, meaning has been made, whatever the modes in which that fay pened, It is the meaning made, not the meaning expected, which should be the foc of interest in assessment, at the first step. From there one can elucidate the princip! in this case, of the child's semiotic work. From that position one can attempt \0 construct a path to the learning that is expected, based now on an understanding of li principles that this learner brings to the tas. My interest is also theoretical. If we were to assess only the child’s linguistic accouit ‘of his images, his spoken commentary, we would have hugely reduced eviden available for assessment: nothing like the richness of the images themselves. 1h images and the words are not just different, the images are much ‘fuller’, more precise, more specific as indications of just what the child considered as ‘like’. |i child's spoken account is hugely general compared to the specificity of the in ‘The verbal account is a reduction of what was available to be understood ancl assess inthe visual representation. Whether from the point of pedagogy or that of semiotics, the central issue is (hit of recognition: first, the recognition of learning in the semioti¢ ane conceptual wor! done as such; and, secand, the recognition of the work done in all modes, That nee a detailed description of the respective affordances of speci and drawing ‘Meaning as resource: ‘naming’ Transformation Transformation is a less far-reaching process than that of transduction. It describes processes of meaning change through re-ardering of the elements in a text or other semiotic object, within the same culture and in the same mode; oF across cultures in the same mode. In other words, the ‘translation’ of a novel from German to English would fall under the heading of transformation. The pracess is significant but as it Involves the same entities and there is no ontological change, it is much less far~ reaching, ‘The two images shown in Colour plates 9 and 10 stand in a sequence of 'earlier’ ‘and ‘later’ in the children’s story in which they occur. They have the same elements; however, the order of the elements has been altered: there has been a transformation; and the viewer's perspective has also altered. That is, there has been a transformation In the sequence of elements in the two images; and there has been a consequent transformation in the inscribed viewinglreading position. Each can be described; each Is meaningful, though clifferently. Transformations apply to all syntagms, texts, and semiotic objects: a change from ne genre to ancther — for instance from my tape-recorded account of a meeting to my telling about that meeting later; my writing of a report and then preparing an execu Wve version of it. Similarly, a change of discourse would be seen as a transformation: the made(s) stay, though different terms and relations appear in the transformed text Transformations are processes in which the elements remain while their ordering | an arrangement is changed. Hence a simple statement about transformations ‘might be: same mode, same entities, in different order = new semiotic entity (= different meaning). Transformations, like transduction, can apply to any semiotic object. The two processes, of transformation (re-ordering of elements — within the same mode, leading to change of semiotic object) and of transduction (moving meaning from one made to another, by re-articulating meaning in the entities of the ‘new! mode, leading to change of semiotic object) encompass the central aspects of semiotic, thange. Frequently both processes occur in the one text or semistic object. Semiotic hange always entails change in meaning; the changes in meaning entailed by these Iwo ‘operations’ are each significant and profoundly different in kind. Transformation ‘does not change the ontological constitution of the textlentity, the ontological location’ of the semiotic object; transduction does. 1) the rhetorical and design tasks so utterly characteristic of the contemporary social and communicational world, transduction and transformation are essential processes. Asa matter of course many websites now address multiple audiencess in a s0cial world deeply marked by profound difference and diversity it could not be different. This multiplicity right be to do with generation; profession; differences in euitu [Bourdicu’s term ~ due to education and social class; pleasure; Hot to mention more specific and equally significant differences: religion, cultural i ‘aonder; and so on, This means that there is a constant al capital ~ tow ices of various Kir 129) Muitimodality need for translation in the sense here indicated: each group may need to be adresse taking aecount of its interests and willingness or not to engage. As one such ~ entire innocuous ~ example, take the website www -poetryarchive.org. It has a homepage, set of pages for teachers, pages for students; and pages for children. Each ofthese | designed with clearly distinct styles and aesthetics in mind. Information ‘ransducted, transformed: translated, with distinct modal ensembles. This leaves a major problem of naming the process usually called ‘translation’ — transport of meaning from one language’ to another. It remains, of course, av instance of translation inthe sense used her; its specific form is transformation. T) ‘mode remains: from writing to writing: or from speech to speech. It isa change with a mace, but across cultures. As modes are shaped by the histories oftheir making 1 c socetios, modes differ from culture to culture and from society to societ ind of Mransformation’ takes on some aspects of a more profound change. Thi is especially so if the script systems as well as the cultures differ: the affordance of an alphabetic script, for instance, differ profoundly from those of a character-base! script Yet these ‘transformations across mode and culture’ are nat, in principe, difer= to those within one culture ~ though with the addition of cultural cifference, that the effects of social history inthe shaping of mode. It might be necessary therefor= 0 speak of intra-cultural and cross-cultural transformations and transductions. Characteristics of states and processes Meanings of the kind described here often have a characteristic manner of occurre ‘or appearance. A social process, action, state, may be permanently as it i; it may i In the process of becoming so; or it may be so intermittently. A social relation connection let's say) may be constant or it may be intermittent. The connection may b unstable In some other way. Social relations may change their characteristics they may undergo a change of state. To use an analogy: at some point, the gravy that I stir begins to thicken; or the sky outside darkens before an approaching starm. Tl meanings of characteristics of states, relations and processes are meanings to whl ‘we need to be attentive; and they require signifiers for their realization. Follow i) linguistic usage (it seems unproblematic here), | use the term aspect for these meat Ings: that is, a particular aspect of a state, relation, process is brought into for. These meanings are highly culture-specific. In human (interaction and relations, social and affective proximity and distan are constant, crucial meanings. What is signalled are relations of power and relation of zolidarity. The perspective from which power is viowod, as well as the direction ol power, need to be specified. Is the point of view one from lesser power or one {01 greater power? The more powerful participant may or may not want to insist on {hy overt (semiotic) recognition of their greater power, the ability to make that choice | itself a sign of power. The participant with lesser power has no such choice ~ unle she or he is prepared to risk giving offence. Meaning as resource: 'narning’ Distance — whether spatial or temporal ~ is a ready signifier of power; the more power a issue, te createrthe distance tat is felt to exist; the less the power involve, the less the distance that i felt. Following tracitional usage here (Brown and Gilman, 1966) I refer tothe two poles involved as those of power and solidarity. ‘With power, as with other social factors, there are meanings — and therefore apt signifiers — around magnitude and intensity, significance and importance. These are often signified by height, amplitude, length, weight, size, extension of various kinds. in fa given arrangement or syntagm, salience can be an indicator of social significanee ~ the soloists in a choral performance who stand at the front of the choir; the local tigritary who stands in Front of a group of officials For wham she or he speaks. This dignitary may have other indicators of importance at her or his disposal, such as the Salience bestowed by a large gold chain as the sign of office. 37 Design and arrangements: making meaning material Design in contemporary conditions of text-making Chapters 2 and 3 sketch a frame in which to think about communication, repr sentation and, importantly, semiotic production in contemporary environments. T! sketch characterizes the social environment as marked by instability and provi sionality; it stresses the role of the market in shaping a habitus of agency-as-choic even though it needs to be said immediately that for many members of society realities of agency and choice are relatively spurious. These conditions make possible and demand that individuals assume agency in the production of semiot entities of all kinds ~ texts, ‘arrangements’, practices, objects. They do so not le: in relation to the making of knowledge, of transforming information which they hav selected in accard with their interests and needs, into the tools they need in th ‘everyday social and communicational lives. In this sketch, the environment of meaning-making is radically different to 0 still active in common-sense imaginations of communication. Yet shifts in author’ hhave made individual interest central. These same changes have made formerly stab: resources unreliable: genres are present, ‘there’; yet they are fluid and insecu representation, understood now as multimodal, is no longer dependably caro! There Is choice. What genre to use; how to reshape it; what modes to use for wha purpose and for which audience; all are questions which ~ in principle at least ~ ned to be dealt with newly in any act of communication. That is, the environment in which processes of ‘realization of meaning’ ~ ‘makiryy meaning material’ ~ take place, has to be understood. The frame for action is 9 rhetorical one; the rhetor's interest and assessment of the environment of communi cation shapes the ground of communication; that and the interest of interpret of that ground, remade as a promt, are decisive in shaping meaning. Design is (i process of translating the rhetor's politically oriented assessment of the environriew! of communication into semiotically shaped material, In realizing the interests rhetor and interpreter, design has moved to centre stage. Design is prospective. It responds to demands which, in some way, are constany znew. Rather than being a campetent implementation of conventionally ven practice design is transformative, hence inevitably innovative. In contemporary semiotic production and communication ~ the two now always linked ~ agency in the interested jal and-sernbotically ~ of the processes of design meets with the affordances Design and arrangements: making meaning material resources available for production. Interest and agency now extend to the choice and the uses of the facilities of sites of appearance and the media af disseminatic Design: an essential (re)focusing The term design has quite recently come into widespread use ~ fashionabilty even ~ Inareas of the Humanities and the Social Sciences. As with any term that erupts into High visibility, one needs to ask about the reasons. Is there more than just fashion? Is it an indicator of corresponding changes in the larger social environment? ‘The earlier chapters give one answer. In thinking about. ‘semiotic production’ ~ ‘whether in relation to language or much more widely ~ it is possible to delineate a path, over the last seventy to eighty years, which begins with (adherence to) conven tion; solidly, unchallengeably established ~ other than in fenced-off areas such as the Creative arts, grudgingly too in advertising and more recently in popular culture In that environment, production was seen as composition in accordance with \well-understoad and accepted rules, The mid-1950s witnessed a turning in that path, Serious challenges began to be made to its direction, above all in popular music. From the 1960s on and moving into the mid-1980s, the path of adherence to canvention gave way, so much so that critique = being a ‘critical reader', for instance — has now in tum become a comfortably mainstream road, as semiotic motherhood and apple pie, Naw design is challenging to become the central term in semiotic work. To take two concepts — text and knowledge —as examples. Texts, as essay, as report ‘or ‘story’, were composed, guided by relatively well-understood and settled generic Conventions. From the late 1960s, these conventions began to weaken, the rules began to fray. It became an urgent academic enterprise to subject generic conventions 0 critique, to challenge the power that seemed entrenched in and supported by Conventions, seen to wark to the benefit of some and to the detriment of others. When Critique replaced convention, composition became problematic. The ‘linguistic turn’ (Rorty, 1967) marks that point, with its challenge of the ‘innocence’ of language and of processes of ‘composition’. Yet critique can work only in relation to stable struc tures and environments; its task is to bring these into crisis. Environments marked by instability, provisionality, fluidity do not lend themselves to critique; the political Aieeds of such environments demand the shaping force of design. In contemporary conditions, knowledge is made in many sites: in wikis, in blogs, but ‘also ~ without fuss or notice ~ in everyday conversation, in instances of unremarkable, tbanal interaction; in the compilation, for instance, of personally downloaded music Hibrarios.. Whon knowledge is made anywhere, by anyone, ‘knowledge’ ceases to be Yeanonical’: witness the increasingly bitter disputes around accounts of evolution. Canonical representations of knowiedge become unstable, whether as made (Should | use writing or image?) or as genre (Which is apt, the essay or the narrative or the Cartoon’). Writing, previously the canonical mode par excellence, is giving way to Imave 1334 Multimodeality The ‘school, as society's designated purveyor of hitherto canonical (Forms 0 representation of) knowledge, has an impossible task. When knowledge is made by anyone anywhere, what is, what can and should be the place of the school? The scho! thas the task of upholding canonical forms of knowledge and of representation witho. the support of clear direction from state or society. [n popular culture, the very s2n\ practices of production which are censored in the schoo! have acceptance and his) status: ‘sampling’ in music, or ‘mashing’. It is an extremely serious matter tha’ contemporary forms af semiotic production ~ forms of composition ~ are poor'y tnderstood or not at all, as witness the moral panics around ‘plagiarism’, ‘cutting av pasting’, and so on (see Aspetsberger, 2008). The state faces a cacophony of voices of a profoundly diverse society. The schoo! bound to be at a loss: ontalogically, epistemolocically, socially, aesthetically, ethical ideologically. Nor are universities irnmmune from these conditions and effects. Wikiped appears as a source of reference in student work, including at Ph.D. level; internet sit: jostle alongside the canonical media of book and journal in reference lists. The debate of the 1990s, around information versus krowledge, has abated son ‘what and settied into a new sense that ‘knowledge’ is always newly mace rather thr being communicated. What is communicated is ‘information’. Knowiedge is produc by individuals according to their interest and their need in their life worlds at tv moment of making (Boeck, 2004). Knowledge and meaning, as much as the texts 2 objects which are their material realizations, are seen as the outcomes of processes 0! design motivated by individual interest. Chapter 2 sets out some of the social and economic reasons for the emergence 0! design as the central term in semiotic production: instability of social environ (that is, the fragmentation, disappearance of stable, reliable, accepted! convention the strong insistence on and assumotion of agency by individuals — including child at ever younger ages: a result of the dominance of the market rather than the stat as the major sociaVpolitical force and, increasingly, the emergence of children as 0 segment of the ‘market’ with purchasing power and hence a social force; and this in turn supporting a change from notions of social responsibility to individual choice ‘What is the case in the world of commodities is no less the case in the world of semiotic actions, with the multiplicity of semiotic resources available forthe shapi/ realizations of meaning, including meanings of identi At the same time, the instability of social environments has destroyed clear ani acceptable models for action and behaviour. In the neo-liberal capitalist marke Ingividvals are assumed to take and have responsibility for their actions. The resun tion of agency by individuals — urged by the state and forced by the market ~ mes that the shaping of meaning and of identity, becomes a matter of individual desi ‘A multiplicity of resources provides the means for those who can actin the mart for their individual” shaping of identity, even thouah the means are those mod available to all by the market. Even those with lesser means or few, are subject to the requirement to assum responsibility for the shaping of meaning in their social enwironments, Th Design and arrangements: making meaning material ence of social conditions, changing semiotic means and affordances in production and dissemination come together to make design the usual, norma, taxen-for-granted, ‘the necessary and essential semiotic cisposition and practice. What is design? A homely example In my kitchen cutlery drawer there are two potato peelers, shown in Figure 7.1. The ‘one on the right I regard as traditional; that is, asa child I was asked to peel potatoes \with a peeler quite like that; and I do so still. The other remains, for me, ‘new’. The Nvraditional’ pecler assumes a particular relation between my right hand and the implement; it also assumes a particular relation between my left hand and the potato. It assumes that the potato fits ~ snugly, comfortably ~ into my hand curved around it leaving about a third of the potato free to be peeled. When I do, my hands touch, the thumb of ry right hand partly resting on the thumb of my left hand, acting as a kind of hinge; the fingers of my right hand, curved around the peeler partly rest Con/touch the rim of the hand holding the potato. I hold the potato firmly: I feel that [ have control in this process, which is comfortably integrated with my body, with the shape of my hands. The action is unforced, it saems natural, When the potato is too large, this ensemble becomes awkward, the more so the bigger the potato. Now the ‘traditional’ peeler is less suited for my hand, which can hho longer touch and rest on my left thumb; the ensemble is less prepared for the large ‘object to be peeled. Hands, potato, peeler still form an ensemble, but not harmoni- ously. Now I do notice that I have to hold the peeler at a certain angle to get it to ‘work; [have to give thought to the fact that if the angle s too steep and the pressure ‘of hand and peeler too heavy, the peeler will take too much of the ‘meat’ of the potato (or apple or carrot) with it; something for which my mother and more so my ‘grandmother would have severely reproached me. Figure 7.2 Potato peelers: how and traditional reMultimodality Usually, the design of this peeler seems to fit naturally into my world. In part, th hhas to do with the fact that my sense of myself with the peeler makes me choose without thinking about it potatoes that fit the ensemble. I don't natice that the pee configures an ensemble of body, object and process in a specific fashion, In fact, th» design of the peeler configures a precise arrangement of the world. That arrangeme is subject to social regulation — how thick the peel can or should be, itself a signific: ‘among other things, of Social conditions surrounding the ensemble and the proce ~ the need to be frugal, for instance. ‘When I change to my ‘modern’ peeler I do notice a difference: my relation with bot! tool and potato has changed. I have to hold the potato differently, not according t« the ‘natural’ ‘comfortable’ shape of my hand but a shape dictated by the form of tiv blade, its action of cutting, the shape of this peeler. In fact, the action is much mor fone of ‘slicing’ than ‘peeling’. 1 hold this peeler/slicer with thumb and finger my hands do not touch. In the process I experience a different relation between mysei! and the tool, between the tool and the potato. I am doing something to it, where befor I felt that Iwas doing something with it: I feel distanced. The slicer/peeler is mu mote effective with large potatoes than the traditional peeler, especially potato which de not fit comfortably or at all into my hand. Of course, the action of slicing may be more congenial to some people than that o! peeling; not everyone will feel the need for a more intimate relation with a potato. T'y peels tend to be thicker with the slicer; reason enough for my grandmother to hav ruled it out of her kitchen. I did do some ‘market research’, asking twa people their views. Both are a gener tion younger than I am. Both of then feel more comfortable with the ‘new’ peeler both foel that it gives them more control. They do agree that the peels are ruc!) thicker ~ but this does nat bother them. Where tools are invalved, socialization play its role, making the cultural and technological ‘natural’, At the beginning of design stands a task; the parameters of the task and 1h ‘assessment of the task come from some other source. Design starts with the designe’ imagining of the task; a knowledge of the resources available to make the tool [i will be used to perform the task; an understanding of the characteristics of the object to be worked on or with — carret, potato or apple; an understanding of the wider soc | conditions; and a knowledge of the worker/agent and her or his capacities ~ the fac\ for instance, that the hands of a five-year-old ciffer from those of an adult. Design projects and organizes the arrangement of an entire ensemble: of the worke' the object to be worked on or with; of a tool for action with or on the abject; inte grated in specific ways into the capacities and affordances of the human body; a boxy With a history of experience af ather, pricr physical/social processes; an encemil which is subject more or less, and in different ways, to social needs and requlatio of the pracess. The design configures relations of body an tool; of tool and object. It also configures and projects affect. The design of the potato peeler is always than the design of one abject or tool; it is the design of a complex ensemble, of av tervironment of social relations, of social practices and configurations, of purpose Design and arrangements: making meaning material goals, alms, tasks; and of affect. A design is the imagined projection of a complex, losely interrelated social array in witich the designed entity, object, process is used, has social effects, meanings; and produces affect. Identity is shaned in such complex. rays by aspects of design, even if in small and seemingly insignificant ways (Latour, 1988; Elias, 1994). ments Design in social-semiotic en In the semiotic damain, that Is, in the domain of meaning, all these features of design ‘are present and active. Society, founded on meaning and, in that, on the shaping of Identity, rests on design, Its requirements and principles, whether overtly acknow- ledged or implicitly practised. That Is the case in the design ofall social environments, Whether seen as environments of communication broadly or as environments of Jearning and teaching specifically. Design is at issue in the shaping of all environments ‘of communication and through that in the shaping of sacial relations everywhere. ‘An approach to communication and to the production of semiotic entities through design — whether as texts or as semiotic objects of any kind — presupposes familiarity With the affordances of all materials involved, of the characteristics of the social environments in which the designed ensembles will be active and of the facilities and affordances of the media involved. It may seem a large step from the potato peeler ~ traditional or new ~ to a website. ill, the point is to develop a frame for thinking about design and to test the use- fulness of the example once It Is generalized ta all instances and domains of design; and in particular in semiotic domains, where meaning is the issue. Here I apply the Dutline of this account to a comparison of two pages of one website, that of the Mayor ff Landon (wwe. {ondon.gov.uk If we start with the homepage of this site, ‘Mayor of London’ then on the day I visited there were seven "biocks' or ‘modules’, neatly arranged in a rectangular pattern. Six of the seven modules have the structure of an image taking up the top half of the module ‘und writing taking up the bottom half. The seventh madule, called “Highlights’ consists ‘of a menu, a vertically arranged list of ten items. This ‘screen’ clearly has a specific social/communicational function, namely to show a range of interesting and significant issues that encapsulate the day-to-day life of London. As a semiotic object it demands not that much semiotic work from its Visitor; though itis the visitor who has to order the screen for her or himself. No order Js (pre-) given to indicate a sequence for engaging with the screenipage: it is the Yisitor’s interest which decides. At this point { do not wish to engage in an analysis andl description of the sereen nor of the modules; though of course each module raises ‘questions of a semiotic/discursive/ideotogical kind. The other screen, ‘Mayor of London: About Us’ consists of a vertically arranged list. ‘of topics, either brietly described or as a url offering to lead the visitor further. This sv has no images, Nor bs it arranged from blocksimodules. It is equally clear that visitors: come hore with different and specific purposes: not casually wondering what 13738 Multimodality London might have to offer but with a clear notion of seeking something specific. \ ‘might ask about the vertically organized layout, rather than a more traditional horizo" tal one — and especially as there is only writing (except for the framing of the footer) “The task (to be) performed by the site is multiple: to provide information; to act ayehicle for public relations; to communicate, as political work; and ne doubt a rar of others ‘The parameters of the task are, presumably, set from some central (policy-)uni 1 don’t know if there is such a unit, but let’s call it ‘The Mayor’s Policy Unit’, Here the parameters are elaborated, spelled cut, by a Working Party’: what the too! is 0 achieve; how it is to be used to do its job. The design-team imagines the task: Who | the audience? What are the purposes? What aesthetic is required? The team nec to understand the resources which are available to make the tool. They neec!\: imagine the object to be worked on or with: the imagined audience. ‘The rhetor/designer needs an understanding of the wider social and political con ditions; in this case for instance, a relatively recent Lord Mayoral election, 11! political echoes persisting, And there is the need to understand the capabilities of tho \who will or might want to use the website or whom the party in power might wish 1 bbe users of the site ‘The text constituted by the website overall, as well as its individual pages, shou! answer these issues. The design task is to mave from general specifications to a tc! which will exemplify the features of the specification. A comparison of two of ts pages can make this more concretely visible. In organization the two pages differ significantly: the genre of the Homepavy projects a social relation of relative equality: informative; a quasi ‘democratic’ alec of citizens chatty told about activities of their elected leader, undertaken on ther behalf. The page is ‘open’; the choice about paints of entry is left to the interest of the visitor; the items are small, quickly taken in; they offer choices of informat’ packaged as small sub-textual units ~ modules ~ consisting of image (top) and writ (bottom); the distribution of word and image, together with the use of colour prod a ‘bright’ affect. Images are salient, colour is used as a design-feature. The page "About us’ contrasts markedly; its genre is strictly ‘informational’: 9 ifferent social relation to that of the homepage is projected, more formally fficien| In keeping with a social affect of efficiency, the page is sparing in the use modes: wastefullness here; let's keep to the business’ seems to be the motto. The layout of \/v ‘page differs: the items to choose from are presented asa list of links with brief aloss« organized as a colurmn of “lines! with ‘headings’ for different topic areas: no modliy units for this page; the aesthetic is less “inviting” than that of the homepage. There i ‘much more ‘stuff’ on the page; it demands effort and attention in engagement. Wits) is the dominant mode; there is sparse use of layout and of (one) colour. The visitor needs to do mast of the semiotic work, Genres are projections and realizations of social relations; a diferent social relat Is projected by each ‘page’. Each social relation and each genre assumes, implicitly usually, what serniotic work fs to be dane and by whom, In each case we can ask wht Design and arrangements: making meaning material semiotic work has been done for the visitor and how much; and what semiotic work land how much the visitor is expected to do. What resources are offered and required to make use of the twa pages? Other pages differ yet again; each is generically specific, cistinct. Design realizes “social relations; in that realization in text it also projects and constructs social felations. Each instance of the design of a text is the result of choices; each feature “chosen becomes a sign of (aspects of) the social relation. Each choice made realizes “an aspect of the imagined (and to be projected) social relation. Choice, in this asin all bnvironments, is shaped by power: the power to ascribe a social position to those who ‘vill engage with the fext. Each choice is a political act. The totality of choices realizes “style. Style is the politics of choice. ‘Style is always subject to the effects of the use of social power in assigning value. Aesthetics is the result of the application of power in the evaluation of style: aesthetics |s the politics of style. The two pages display a very different aesthetic; they project different social relations, which embody differing conceptions and relations of power. ‘Seen in this way, style and aesthetics are means to connect the social quite precisely 10 the semiotic. Figures 7.2, 7.3 and 7.4 show three screenshots from the Poetry Archive website (ww.poetryarchive.org). Here, some of the telling social differences ‘are profession ~e.g. teachers as against lay members of the public; generation (as the social construction of age). The choice of modes realizes social difference as semiotic, difference —a foregrounded use of writing compared to a mix of image and writing. Inthe case of the screen for children, meanings of generation are realized by layout, by the shape of the modules and above all by the colour palette: pastels, desaturated colours, a range of colours in the lower energy end of the spectrum: all to realize/ signify childlike fun, joy and sensitivity ‘Multimodal design refers to the use of different modes ~ image, writing, colour, layout ~ to present, to realize, at times to (re-Icontextualize social positions and Felations, as well as knowledge in specitic arrangements for a specific audience. At all points, design realizes and projects social organization and is affected by sacial and {echnological change. To say that a lot has changed in design over a given period is to say — and to ‘document it in semniotic/aesthetic form ~ that a lot has happened sacially over that time. This can be tracked and described in specific media and genres. In schoo! text- books, for instance, there are now many more images than there were, say, sixty or seventy years ago; and their function has expanded and changed. Social change is realized in changes in design. Diversity is one factor with effects on design. While social changes over time ~ ‘history’ — lead to changes in what may be perceived 8 tradition, eoovention, av to judgements oF Yheing too madern’, diversity may lead to divergence, to exclusion, to inequity in access, to fragmentation. Design projects ‘aspects of the social which may not be visible by other means and hence be bayond Convnon-sense awareness. Technological change, such as the introduction of the ‘compuiter screen or the internet ~ itself driven by and producing social change — opens up new possibilities for multimodal design. 13940 Multimodality Figure 7.2 Poetry Archiv website: Gen homepage Figure 7.3 Poetry Archiv website: Teachers! . howepage Design and arrangements: making meaning material Figure 7.4 Poetry Archive website: Children's homepage Changes in design: a brief look at recent history Textbooks, as one such medium, have undergone deep changes over the last century. Society has changed, curriculum and pedagogy have changed in line with social ‘hanges; consequently textbooks have changed, both in ‘look’ and in ‘content’. If we Compare a textbook for the subject “English” in secondary schools published in 1930, ‘with one published more recently, we can see — from the point of view of content — that, the subject now includes material on ‘popular’ culture and media. Pedagosically — fand for epistemological reasons ~ where before individuals engaged with the book Individually, now there are requirements to engage with materials working in groups ‘The tasks for engaging are set out, relatively explicitly. Books have changed; the role of the book has changed; the social relations of book, feader and institution of school have changed. We can see changes in the semiotic Work of design that has been done by design teams and that which is now expected {0 be done by ‘readers’. Now there are full colour images on almost every page, Hoth photographs and drawings; the overall text, consisting now of writiag and image, Is orsanized in layout of a page which is configured as a sito of display, a site of ‘appearance, a semiotiellayout unit that did not exist and could not have been imagined lithe 1930s. A page, then, was a page; a means of presenting parts of what was then the relevant unit of the chapter, a unit organized by content. Content determined organization: ach chapter was as long as content (mediated of course by author and publisher) wai2 ‘Multimodality ictated that it should be. The page had no function other than to ‘carry’ what lis been written and was now going to be printed. In contemporary textbooks, the pu: more usually, the double-page spread ~ determines what Isto be represented, ho Js to be presented and communicated. Now content is shaped by the contingenc Of the site of appearance. The page has become a semiotic unit, that is, a unit o| meaning, What has changed, in other words, it not just subject content; not just the tas! which are set uo for learners so as to engage with subject knowledge; what changed is how these pedagogic interests are graphically realized on paper. Or rath the ‘look’, the fayout, the arrangement of the site of appearance is a graohicivisv| realization of new social relations of the participants in educational environmen! Design, had of course always been invisibly present; now it has become a major Factor in the shaping of social relations and in their semiotic realization. The ordering, arrangement of materials using the space of a page or screen's done from the perspe: tive of an educator/rhetor, who has an eye equally on Yown interest’ (as pedagosur) (on students as ‘audience’, on the ‘phenomenon to be communicated’, on the ‘broaver social environment’ and on the ‘effect of the arrangement’. The arrangement, in ot ‘words, instantiates the social/educational purposes underlying the design of tiv materials. Designers use the resources for making meaning which are available and apt | serve the educational purposes. The category of ‘textbook designer’ includes writers well as illustrators, editors, typesetters and other professionals. Each professiorial ‘Group has specific resources and uses thelr potentials. Different resources ~ moc genres, discourses — do specific kinds of ontological, epistemological or pedagoui/ social work, Professionals now tend to operate as teams, as ‘an ensemble’; they bri) their distinct contributions to the overall design. Increasingly now teachers act rhetors/designers of digitally mediated materials, bringing thelr interests and age ry into this semiotic work (Jewitt, 2008). Two examples from Science textbooks (Figures 3.3 and 3.4, pages 48 and 49) c.) serve to illustrate briefly some aspects of the range of changes in design that law taken piace over the last seventy years or so. One is a Science textbook from 1935, th cother from 2002. In both books images are used to depict parts of the human boy with writing directly attached to ‘anchor’ (Barthes, 1977) the meaning of the ima But the placement of the images on the page differs. In the 1935 textbook, placernt seems to have followed the then active principle of ‘insert Figure X about fie That principle implies that writing is the central means of conveying meaning; ‘hl it is functionally dominant in carrying major ‘informational load’ of the text overall; that image is functionally subordinate; and that the sequential logic of writing supplies the principle which organizes the ordering of content material on the pay) and in the chapter. In the 2002 textbook, writing and image are placed in parallel columns. T)\ principle suggests that writing and image are on an equal footing, Now it is neither {lv logic of image nor that of writing which dominates, but the logle of layout ~ itself of Design and arrangements: making meaning material ‘course founded on and exhibiting logics of space. The status relation between image ‘and writing has shifted from an unequal to an equal position. Thetr functional tasks ave altered: from illustration to fully careying information. Ontologically this implies Ashift in the valuation of modes and knowledge: from an ontology in which knowledge constructed in writing dominates over knowledge constructed in image, ta an ontology ‘which the two bases for knowledge have equal standing In the 1935 example, image was ~ spatially ~ embedded in the sequenced ordering ‘of writing. In the example from 2002, image and writing are placed ~ spatially ~ in faralle|, Most immediately, this has directional implications: in the 1935 example, there is reference to the image at the beginning of the section on ‘The Alimentary @anal’ and a further reference later in the section. In between, the reader had to turn the page to view the image. Less immediately it has effects on how readers engage jth the text, encouraging specific forms of reading, of ‘engagement’. One suggests attend to this first and that after’. In other words, the 1935 layout does not facilitate ‘parallel reading’ of word and image in the way it does in the example from 2002 Hore, layout steers the reader into a *back-and-forth’ movement between word and Image. It is a mode of engagement which suggests ‘attend to these as equally sio- hificant; read them in mutual interaction’. In the 1935 example, layout steers the ‘reader into a first-then’ manner of reading. [n the 2002 example, the page on which these ‘chunks’ are laid out is itself part of a larger structure, that of the double-page spread. This contrasts with the 1930s textbook, where text-as-writing was ‘put! ‘on to pages withaut much attention to haw this played out spatially. What mattered was not where on the space of the page an image appeared, but where it was positioned Jina sequence of writing. The designed page or more frequently now, the double-page “spread (here itis both) is organized as a spatial/semiotic unit, linked to the temporal Soiniotic unit of ‘lesson’; the chapter in the old textbook was organized as a unit of Yeontent’. ‘As a multiplicity of modes ~ image, writing, speech and moving image — becomes Available through the facilities of the contemporary media, the potentials for design filter radically. Resources with particular affordances become available for specific lise; here for instance, lexis or depiction bring implications for generality or speci- ficity; syntactic resources have implications for variable arrangements of entities, as ‘well as bringing the means for the expression of the social relations of the maker of a message and its ‘reader’ ~ the relation of ‘command’ for instance. For the designer Df the learning materials the question becomes one not just of the ‘aptness’ of the fepresentational resources for the specific occasion but also and maybe more so, the question of the design of social relations. Thic change means that participation in semiotic production within a general ~ theoretical and practical ~ frame of design now best describes the characteristics of ommunication in most sites; though least so, still, in schools. That has profound ‘effects om knowledge production, Social change has led to an emphasis on the agentive Action of all participants in communication, even if differentially, and that in turn has led tot s in the making of knowledae. e potential agency of indivi 143‘4 Multimodality ‘These social and representational changes are everywhere evident in contemporary media. The participatory affordances of current media technologies blur form distinctions of production and consumption, of writing and reading, The simultaneous ¢global and local ‘reach’ of media challenges the boundaries of communities global a local, with severe effects on genres; it mixes contents both global and local; ub/qui! fof access to information, convergence of media and connectivity in the sphere | individual lives entails that occasions of and resources for knowledge production 2» creativity are not tied to particular sites and times, Multimodality, representation in many modes, allows and demands the choice of apt communicational resources ail situations. ‘The newer dispositions towards agency have deep effects on design processe Al aspects of the domain of meaning are drawn into the new social givens, wit far-reaching effects. In relation to the making of texts, for instance, questions o| authenticity and authorship have changed profoundly. In downloading, ‘mixiin) ‘cutting and pasting, 'saryplina’, recontextualization, questions such as Where did (i come from?’, ‘Who is the originaVoriginating author? seem not an issue. Much |i the use, in former times, of a ruined castle or a monastic building as a quarry, a sour « cof building materials ~ a large stone here to use as a lintel, another there as part 0! wall —existing texts are taken as ‘resources’ to be ‘mined’ for the making of new tex! ‘There is a need to understand the practices, epistemologies, aesthetics and ethics 0! contemporary forms of text design and compositional princioles. This would need to start with a clear sense of what are now to be regarded « ‘compositional elements’. It may well be that the principles of composition have changed very much; the entities that enter into composition have changed. Of cous in the high era of writing, the basic compositional elements were not letters but wor! and phrases. That is, composition did not start with the production of words, lets by letter. What letter-by-letter practice did do was to re-record, in their remaki instances of words; had there been another means of producing words ‘at one go’, to speak, the letter-by-letter route would not have been taken. Now, the contempor:y means of recording, documenting, referring, provided by digital media make ifferent compositional elements available (see Chapter 9). Instead of describiny a historic building, a landscape, a scene in a market, word by word, my mabile 1 vergent device allows me both to take a photogrash and to send it. T can down|o.! ‘segments of written text — longer or shorter from the internet directly and intear-v it into ‘my’ text which I send on, as ‘my’ composition, [At the moment these practices and units are discussed in terms of ninetee)t) century models; terms such as ‘plagiarism’ or ‘mere copying’ are readily ~ and sii) inappropriately — to hand: that is, the invocation of models from an era where conc tions of authorship were clear and legally buttressed in an era entirely differently constituted socially, culturally and technologically ~ though not yet legally In this situation, the notion of design and detailed! descriptions, analyses ond elaborations of principtes of design can give, instead, relevant means of describiny and analysing current practices of text-makiing and the principles underlying thes usually 0 practices by cert rationally defined, groups Design and arrangements: making meaning material What might be termed a ‘social common sense’ ~ in reality itself an ensemble of generational layers, gendered differences, distinctive educational background and hence cultural capital ~ is caught between traditional and contemporary conceptions of authority and agency in relation ta the production of knowledge, to the author- ing of texts, the authority/canonicity of knowledge and of semiotic forms much more generally. Political authority is contradictory: a demand for ‘the new/, for innovation ‘and creativity is countered by anxieties around loss of control. Learning has fong since left the confines of institutions such as school, university, college; and forms of pedagogy have to accommodate to ‘life-long’, ‘life-wide' learning, that is, learning at all times, by those who demand that their interests be taken with utmost seriousness, in all sites, in all phases of professional and personal life. In school, many young people see themselves as authors of the knovrledge they want and need, authors of the kinds of texts that meet their social, personal and affective needs ~ even though authored by processes wiiich bring them into conflict with authority which remains focused traditionally. In that, they came into conflict with the sharply differing, contradictory ‘conceptions and practices of the school. Conceptions of pedagogy held by the school 4s institution are at loagerheads with those held ~ however implicitly ~ by those in school. To resolve that stand-off productively, conceptions of pedagogy will need to be developed which accommodate the conflicting interests of generation, of power, ‘of politics and of a market-dominated economy ~ not to mention ethics. Clearly, the agency of learners nowy has to be taken seriously and placed at the centre of pedagogic. attention. Equally clearly, the insights, understandings, values, knowledges which are the result of centuries and millennia of social and cultural wark, cannot and should not suddenly be ditched. Arrangements: making meanings material Occasionally, in the ceaseless, ongoing meaning-making ~ whether in social inter factions or in the flux of “inner” semiosis ~ meanings firm up into visible, tangible, ‘audible, material entities, as signs, complexes of signs, texts, semiotic objects of Warious kinds: the fixing and framiing discussed in Chapter 5. That is the issue: ‘making meaning material’, in some way, making meanings real, in ‘realizations’, Meanings-as-resource become material ‘arrangements’ as texts and other semiotic objects. These become evident in sites of appearance with specific Ineans of dissemination. Fixing and framing entails a momentary interruption of the ‘ceaseless flow of semiosis, provoked by some event, action, process, brought about by a message taken as.a prompt for a response or by some Inner equivalent of that externally produced prompt. The flow of sernfosis is ‘punctuated’ in some way; and inner’, unmaterialized meaning is given ‘outer’ material form. It is fixed modally, as Speech, as image, as gesture, as gaze; most usually in ensembles of mades, depending 1 the assessment of the rhetor/designer who shapes the response. Arrangements of various kinds are in focus here; as are the resources available for producing them, If what is to be arranged’ was the focus of Chapter 6, now the focus Is.on the ‘means for making srrangements'. The arrangements ~ ensembles of the kinds 46 Multimodality ‘of meanings outlined in the previous chapter ~ are social in thelr origin and effects; ensembles every aspect of these meanings Is given shape in arrangements. Arrangements Arrangements are the semiotic face’ of social organization. That means that 1! social environments in which the arrangements are produced and which they in {ur ‘make material are profoundly significant. That was the point of my brief excursion into the ‘emergence’ realization of discourses in architecture. Meaning cannot | discussed without a sense of the shape ~ the organization ~ of the social environme: in which itis produced, whether as hierarchy or network; as more web-ike in sir ture; whether the metaphor invoked is that of the rhizome (Deleuze and Guatta 1973) or that of a matrix. These are ali possible forms of social organization ‘0 ‘which the semiotic theory needs means of expression, realization, materializing. He too belong social/cultural, ontological (mmeaning-) orientations to the world and production as an ontological-semiotic space: for instance, the social and semiot « significance of centre and margin, of vertical or horizontal ordering, of top vers bottom, of left versus right; or the variety of possibilities of representation in the real of actions, events, processes. The realizing features of the arrangements at issue are of many kinds, all at oll times related to the ideological and ontological organizations which shape cultu' in a particular society. Formal features ~ signifiers — such as lines and In versus squares and centrality, circular versus rectilinear shapes, direction(ality) 1) spatial and temporal orientations (Kress and van Leeuwen, 2006), Arrangeme! are syntagms, that is, orderings of signs or ~ to put it paradoxically ~ at times « designed seeming absence of order. All syntagms, at whatever level, of whatev' kind, express meanings of affect, of indications of factualityfacticity/fictiona|ity of kinds of realism, The question is: ‘What are the means for making these meanings as sions, syntagms, as texts, as arrangements?’ and ‘How are these means (to be) used making apt arrangements? By and large I will Focus on three of these means 0 making meanings material: on mode as the material stuff, the socially shape material means; on text ~ or equivalent semiotic entity ~ as the largest level unit communication; and on syntagms as arrangements of many kinds. There are in unresolved issues in relation to each of these. In some cases I suggest directions lov possible answers — for instance, ‘What is a mode?’ ~ but unresolved questiov abound: about the types, the functions, the names of ‘sub-textual’ units in the vari modes: units, for instance, that might correspond ta the ‘movement! in a sonata; 1 panel’ in a carved and painted triptych of an altar; to the ‘pages’ in a website the stanza’ in a poem; the ‘phrase’ or the ‘riff’ in a musical performance; and que tions about the units of which these in their turn are composed, mayhe equivalent the relation of paragraph and sentence in writin, Design and arrangements: making meaning material Texts Texts and their sub-textual entities are realized through various resources; many devices are necessary and available to give shape to text and sub-textual units as well as to the relations between them. My assumption in the theoretical sketch put forward here is that while each mode needs to meet the requirements of the three functions = the ideational, interpersonal and textual — different modes do not (need to) have the same types of units and entities; nor should we expect that to be the case. AS long 5 we do not take speech or writing as models, demanding that all modes conform to these —for instance, in having the same entities — we need to learn to feel comfort- ‘able with the sense that different modes have different kinds of unit. Image, as 1 have said, does not have words; nor does writing have vectors; and layout does ot have tense. It is that very difference which, among other things, gives modes their Wistinctive affordances. Given the range of modes which need to be encompassed in the naming, and given contemporary modes of composition, I will use the term module as a catch-all term {tb name the units which serve to make up texts and other semiotic entities. In this 1 Jean on suggestions made by Sean McGovern in his research about a fundamental feature of the semiotics of Japanese composition. Modular composition characterizes Computer games; in cutting and pasting it is a major principle both of the selection ‘and the composition of texts and other semiotic entities; as it is in sampling and downloading; in recording and documenting of various kinds, It is the notion of the (bric-Icollage as one of the now dominant forms of composition, in which the module Is central as the element of semiotic production ofall kinds. The contemporary notion of the module, of modular composition, even if not necessarily composition as (bri: Jeollage, can be readily applied as a compositional principle retrospectively, and of Course it had been, in “iraditional’ forms of composition: inthe text and the paragraoh, In the composition of pages in fayout, in photographs, but also in the ‘composition’ Of buildings: of doors and doorways, of fireplaces, and so on. What is to be ‘glued’ tnd how — and what not — is subject to different forms and degrees of regulation at ifferent times in different places. ‘The text (or the serniatic entity), the largest level entity, is recognized — from the maker's as much as from the viewer'shearer’s/reader’s perspective — by a sense ofits ‘completeness’ in meaning, in the social and communicational environment in which It is made, in which it oceurs and in which it is active: the sense that this unit is ‘complete by itself ~ in some way ~ that it ‘makes sense’ by itself, in its appropriate social environment. The sense of its completeness derives from the ‘completeness’ of the social event/activity and the environment in which the text Is produced and Where it ‘belongs’; that is, from the fact that the text is produced in and shaped by @ Social activity which has regularity and recognizability for members of a social group. Hence the recoanition of completeness is a factor of, arises out of and assumes Smembership’ of the group in which the social event has its place and where its ning is understood. 178 Multimodality A part of that sense of completeness, both of the social event and of the semiotic object/text rests on features of cohes‘on — a formal attribute — and of coherence ~ a attribute of meaning — internally within the text and of the text with the environment in which it has been produced and is used. A sense of ‘completeness’ is essential to the recognition of a semiotic entity of a ind, at every level. Within the text, the smaller, sub-textual units require their distinct ~ sense of completeness: each has to be experienced, bath from the maker and from the viewer’s/hearer'sfreader’s perspective, as a unit, as meaningful, at level which it cecupies within the text. If it is not, it will not be recognized and car not function as a constitutive/constituent part of a text. In other words, ‘completenes is relative to the level at which a unit functions. The text is special in that it functioy in the social environment, it is a part af social practices directly. In that sense, text’s environment is social rather than semiotic. The sense of completeness do not reed to be identical for maker and viewer/reader. Indeed, aiven the definition of communication as resting on interpretation, it could not be the same for the tv positions. To restate: a text is a multimodal semiotic entity, seen as ‘having completeness’ ‘those who engage with it. Its sense of completeness derives from a (shared) unde: standing of the social occasions in witich it was produced, in which it functions or which it alludes. The text has features of internal and external cohesion and, 2s «\\ integrated meaning-entity, of coherence. The form and the characteristics of sub-textual units, the modules, are derived fro their functions and uses within the text. Putting it this way means that the directiov of analysis and of description, of definition and constitution of both the fext and sub-textual units, s ‘top-down’. This is inevitable if the starting point, the point of origin of meaning, is the priority of the social. This contrasts with a ‘bottom-up’, 0 “building-block” approach, in which small units make bigger units and bigger wi make very big units. In a top-down approach, text and modules are seen as shape by the contingent circumstances of those who make the text in its social setting. /\\ account of making the text might be something like: "I have to write a report; I these four Issues 1 want to set out, so I better have four paragraphs, one for eacl 0 these. I need a paragraph to introduce it; I had best put a sentence at the becinn in that says what each paragraph is about; and then I suppose I had better pul summary paragraph at the end.” In this approach, the modules of a text ~ of writin of image, or multimodally constituted — or of other semiotic entities, are shaped by tl Interests of the maker of the text, designed by the purposes they meet in the text onl in the contingencies of a specific environment — whether that is a website, a pop sa) a computer game, a bleg. Signs are the minimal units of semiosis and as such the (minimal) units of moc Signs are units of meaning in which ‘something to be meant’ ~ a significd — \ combined with ‘something that can mean it’ —a signifier. Modules consist of sig ‘most commonly they consist of several signs or signcomplexes, Signs are waits of ‘meaning, which exist as sian-complexes infas modules. An example right be the w Design and arrangements: making meaning material of a colour (as one sign) with full saturation (as another sign); or a syllable (as one sign) said with a rising intonation (as another sign). This means that a siga functions as part of a sign-complex in a module; a sign which does not function in a module cannot ‘stand on its own’ ‘A text is semiotically and communicationally complete. That makes it distinct from other semiotic entities in its environment. text is frarmed. The framed text creates, provides and occupies a discrete ‘semiotic space’. Frames Frames and means of framing are essential to meaning-making in all modes. The frame marks spatial and/or temporal extension and limits of a text or other semiotic entity. My slogan "Without frame no meaning’ entails that we need to focus on frame, ‘on forms of framing and on that which is framed, at all times, equally. When painters or film directors want to shape what they wish to represent, they form a rectangle with thumb and fingers of both hands and look at the world through that makeshift frame. What is in the frame now appears separated from what is Outside the frame; what is inside the frame now forms (or should form) a unity in some way. At a general semiotic level the word ‘frame’ names the formal semiotic resources which separate one semiotic entity from its environment ‘pre-frame" or from ‘other semiotic entities. In this, the frame provides unity, relation and coherence to what is framed, for all elements inside the frarne, Without a frarne we cannot kniow what to put together with what, what to read in relation to what. If we do not know what entities there are, we cannot establish relations between them. We can- tiot know therefore where the boundaries to interpretation are: we cannot make meaning. Frames and means of framing are essential for all meaning-making, in all modes (Goffman, 1986; Bateson, 2000), ‘A frame defines the world to be engaged with; it excludes and it includes; and in doing that it shapes, presents the world according to the interest and the principles of those who frame. Dramatists and stage designers, painters and filmmakers, architects ‘and urban planners, interior decorators, photographers ~ amateur and professional = have long exploited the potentials of frames. In Photographic Journalism it is a ‘well-understood major device for ideological shaping (Hall, 1982). In Colour plate 14 there is a double frame: that of the photograph; and the frame within the photograph. The first of these gives us a clear sense of being in a space from which we can see trough a stronaly defined frame into another sace, framed by a doorway. The first frame gives us no real sense of the space in which we are located; though it does seem ‘much darker than the space into which we look — and of wich we ean see only a part. The room into which we might be invited to move has a person in it, and a row of lights hanging from the ceiling takes our eye straight to-another Fearne. In Colour plate 15 we are in the lighter room. The frame of the photograph again eops us from seeing everything that we might want to see. A piano is in the far left Corner, some easy chairs are lined up along the right hand (largely glassed and 140 Multimodatity urtained) wall; two people seem to be sitting on one of the settees. The emptiness 0 the room is the overwhelming expression; it seems that what is being framed emptiness; the pieces of furniture and the people placed in that vast empty spi merely reinforce the feeling of being Yost’ in that space. Modes which have a long history of social use have highly elaborated means (01 framing, Sentences are frames in the mode of writing; punctuation is a resour for framing in that mode. When children learn to write (Kress, 1982; 1994), on« Cf the difficulties they encounter — among many others ~is that they are familiar vl the framing devices of speech, but not with those of writing. In their early experi ‘menting with punctuation as framing, the questions they pose for themselves are bo! readily apparent and highly instructive. These tend to circle around two issu ‘How do the framing devices of speech relate to those of writing? and ‘What are {/\ potentials for framing in writing?’. The changes ~ or ‘corrections’ ~ they make (0 the shape of their sentences give an insight into the questions they pose. ‘The power of sentences as frames to shape meaning can be readily demonstrate the use of banal examples. Here are three such: George came into the room. Mary left. George came into the room; Mary left George came into the room and Mary leit. Placement of an element in the frame — here that of the sentence —has effects; so i! | Invert the order of the clauses in the examples, the meanings alter: Mary left. George came into the room. Mary left; George came into the room. Mary left and George came in to the room. In writing, there are frames for units such as texts; for paragraphs within tex! for sentences within paragraphs; and for ciauses and phrases within sentences. |) term for the (graphic) means far framing in writing Is punctuation. Commas, full si semicolons, colors, dashes are ‘punctuation marks’: each a resource for connects) ‘and separating entities, so that each can be brought, newly, into a relation with other ‘clauses with clauses, sentences with sentences, etc. All punctuation marks frame sw! in doing so contribute meaning. Some punctuation marks/frames also co1vy seciaVattitudinaliaffective meanings, such as the question mark *?' and the excl tion mark!’ as questioning, seeking confirmation, ordering, commanding. hes as frames for social relations. In writing, in Enlish, questions can start with so-callod SWh: forms’: What ... 2, Who. ...2, Where... 2, or How... ? Hera, frames hw been made into words, lexicalized, both to frame and project a social relation, Decisions about punctuation as framing are decisions about making meaning. Ot frames in writing, the paragraph as one such, have equally significant effects, enter Grass's autobiography, Varn hduten der Zwiebel (Peeling the Orion, 2007) lw describes, in a longish paragraph, how, as a thirteen-year-old, he Loved reading, tying Design and arrangements: making meaning material on his tummy. His mother would come and, using one of her hairclips, pin back the hairs that had fallen across his eyes. Grass concludes the paragraph with the sentence “Ich duldete das’ — difficult to translate, but (in that context) something like: “I was prepared to allow her to do that to me,’ (In the official English translation itis ‘I put up with it” somewhat flat and anticlimectic.) Grass could have placed this sentence on a separate line, making it into its own sentence-paragraph. The effect of realizing his meaning via the signifier of paragraph rather than of that condensed concluding sentence, would have been to make the comment — with Its feeling of an enveloping affection lovingly given and gladly received though, almost, as his due — into the sign of a momentous issue, given much different weight as the expression of a young man’s relation with his mother. The material means for framing differ from mode to mode. In the theatre there is, the framing of the stage, itself a historically shaped and changing form. Attempts to do without the frame of the stage show the power of this frame. The play to be performed itself has framings of acts and scenes. Amateur photographers carefully frame the scenes they shoot: the doorway, the overhanging branch; in cinemas there are frames ~ in some, still, a curtain is raised and lowered. ‘As each mode has its specific means for framing, a social-semiotic theory of muiti- modality is required to elaborate these means. In Figure 3.4 (page 49), the image of the digestive system uses colour as a resource for framing. A desaturated pinkish background is used to indicate the whole of the body, witich is not the focus of the "unit of work’; and the saturated blues are used for the organs, which are the focus. Colour is a quite different resource for framing to that of a spatial or a temporal gap, ‘an empty space or a pause; or of a graphic mark such as a comma, a dash); or the frame made with sound, of an intonation contour, which ‘clamps together’ the elements within it, Colour, as @ background ‘wash’, frames by establishing the "ground ‘on which things ‘are’, ‘belong’, happen’, ‘take place’. This is other than the framing which marks a boundary. On a road, a single unbroken line is a stronger framing than a broken line; often there is an unbroken line on one side and a broken line on the other: indicating that frames mark boundaries and ‘territory’ differently for (temporarily) different social groups. Inevitably, framing devices are more elaborated — more conventionalized, more subject to regulation ~ for modes winich have received social and semiotic attention ‘over long periods. With modes which are not so elaborated — and hence not subject to hhigh degrees of control — there is more room for individual innovation. { recall a (then tegional) BBC news-'reader’ wha, having read the evening news, invariably concluded the presentation with the tiniest of nods, eyes firmly pressed together and lips tightly pursed: a full full stop indeed. Some years later, when I saw him prosonting the news ‘on national TV, gone were the nod, shut eyes and tightly pursed lips, replaced by an ‘open smile and cheerful ‘Good night’: a makeover of modal resources; and of identity. Semiotic entities, texts included, have several kinds of framing. The “Thats all folks’ of old cartoon shows, the ‘FIN’ or ‘The End’ of early movies, are one kind. These are the material franvings of text as text, the fading of the picture at the end, 15)2 Multimodality for instance, in early movies; the rising and falling of the curtain in the theatre (stil! a feature of some cinemas in various parts of the world). Here too belong the variou: resources of layout; and of colour which indicate boundaries; white space around pri) ‘on the page; fences and hedges in suburbia; open spaces in front of buildings. Then there isthe framing of social relations, of genre: generic framing. Text open ings and closings, such as ‘Once upon a time, there lived ..." and... and they lived happily ever after’ belong here, as markers of textual beginning and ending: qua ‘entry into fairy tale’. We know how to navigate an interview, because we understand ti ‘generic framing; and we know when a conversation has begun to examination. Texts are framed discursively, as are all semiotic enti framings allow us to navigate the world of institutionally produced knowledge. The rolling of the credits at the end af a movie is an ambivalent framing; so (aesthetically or fan-oriented) moviegoers regard it as part of the filmic text, whi cthers are already on their way cut of the cinema. Ambiguity in framing points to av), is ameans of realizing social ambiguities of various kinds. Not having a fence betwes te neighbour's garden and yours in the suburban US (my experience relates to sor New England states) seems to mean, I take it, that ‘we don’t need such things — we’ pretty good neighbours here in this community’. Yet the lawn mown with geomet’ precision exactly up to the border of my/their land contradicts and negates the absen Of the fence and constitutes a framing which has to be constantly, newly, reaffirmes By comparison, the English saying that ‘Good fences make good neighbours’, takes » pretty pragmatic and for Enalish conventions fairly direct route. In the case of spatially organized texts, frames can be lines, Solid or broken; stron) or faint; straight or squiggly; a wash of colour ‘behind’ the elements superimposed 0 and framed by that wash; or, in the case of the more traditional page, a clear fram) of white space of varying extent around the text overall and around its parts. \ speech, one function of framing via the means of pitch(-contours) ~is to mark infor mation units and the distribution of information (via a perceptibly major pitcli movement) ~ as assumed to be shared and as not shared — and to ‘chain together information units into integrated, internally coherent, larger level units of inform tion (Brazil et af,, 1997). In the case of temporally organized texts, various visi! (the newsreader’s closed eyes, his tightly pursed lips, the slight nod) and/or aural cw (silence, a fall of intonation to the usual resting point of the voice for a particu’) speaker} a beat on a drum) exist to signal endings or separations of intra-textu elements from each other. Elements within the framed entity are treated — in reading, listening, viewing, in th joint reading and viewing of any multimodal text ~ as belonging to something that hhas ‘unity’, coherence. For instance, whatever definition of ‘sentence’ we might its framing implies that what is Yn’ the frame of the sentence belongs there and whi is not ‘in’ that frame does not. Similarly, whatever entity follows on from a semico! a colon, and s0 on. Framing devices are signifiers, resources for making signs and meaning potentials of quite specific kinds. 1 suet they hi re differen! amities framed by a colon Design and arrangements: making meaning material to those framed by a semicolon or those framed by a full stop. The meaning of punc~ tuation marks is a semiotic issue; it defines what kinds of intra-textual entities there can be. Punctuation marks are entirely conventional, yet the signs made in their use are specific, individually made with this culturally given resource. If I chose to frame a segment of my writing with a comma I have made a sign different to that had I framed it with a semicolon or full stop. In other words, punctuation marks have a ‘meaning potential and allow me to make my meaning ~ as do all framing devices. Frames hold together and they separatelsegment. The elements within the frame are held together and the viewer/reader/hearer is asked to engage with the entities in the frame as connected, as coherent, as having some kind of unity. A bricolage works because of its frame; take the frame away and it has become an unordered heap of things. At the same time frames separate what is in the frame from what is outside; the viewer/istenerfreader is asked to regard what is outside the frame as different from what is inside and if connected, as differently connected. What else is framed? Frames are specific to modes; they are also ~and that is close to a tautology — specific to culture. In Chapter 5, 1 mentioned some of the cultural/ontological implications of script systemis. When children learn to write characters, they do so on squared Paper; each character is produced as perfectly balanced in a square, The lettors of the alphabet are placed on a line. Square and line are frames, in an only mildly extended sense ~ that is, they indicate, mark out, delineate the space, the site and the manner of appearance. In a research project with a colleague (Kenner and Kress, 2003), we attempted to understand what sense young learners of different script systems made of the Uifferences in the scripts they were learning. One means whereby we tried ta ‘get at” their sense of this was to ask them to teach their peers (in the English school, during the week) the script system of their home culture On one such occasion, when one of the young Chinese learners was ‘teaching’ one of, his (English) peers how to construct a character, the young ‘teacher’ several times ‘ubbed off the attempt made by his peer on the black-board, Finally, with unconcealed frustration, he said Yhe’s doing it wrong’. When my colleague asked what he was doing wrong he said: ‘he's not putting it in the square’. My colleague, unable to see the square he was referring to, asked him which square he was talking about. With Increasing frustration he pointed at the board. Looking intently, my colleague cauld se that the blackboard had once been ‘squared, though ta ‘normal’ ways of looking, the squares had been all but entirely scraped off by countless cleanings uf the board. Neither she nor the non-Chinese young eer had seen the exceedingly faint traces Of formerly present squares. For this young ‘teacher’ on the other hand, they were clearly there; more ‘present’ than the “lines? on the blank inside of greetings cards ge, with the base of each letter placed precisely on the 15Multinodality The question: ‘Where do frames exist?” applies to the framing of all semioti entities and forms: In the ‘world’? In the ‘semiotic world’? In the semiotic wor!) that we have, each of us, created in our own heads? In the image of the frame provided by our cultures? It is there that frames exist, firmly lodged, with profour: ontological effects. Seeing the world as ‘framed’ by a line or by a square leads profound differences in conceptions about the world. The kinds of phenomena relevar | here are direction(ality) ~ arrangements of elements from left to right or from rig! 10 left; centre-margin arrangements; vertical-horizontal arrangements; line: arrangements, to contrast with circular, spiral arrangements or arrangements in he! form; oF combinations of these. All point to potential use as signifiers of soci! forms, structures, processes, as well as being involved in and emergent in organi2in) semiotic arrangements such as layout. They might appear jointly, for instance, in ls combination of linearity and (left-right) directionality. These form a resource ol powerfully plausible signifiers for social and/or epistemological dynamics with wide range of potentials, Frame, text, communication are inextricably interwoven. Without frame no tex! without framed entities no communication. Without syntagms no order; withow syntagms-as-arrangements not sufficient stability for communication. Without cor ‘munication no renewal of text or frames or meanings. To be in a world of meanii\) is to be in a world of frames, of framing, of syntagms, of arrangements and of tI constant remaking of all these in transformative representation. This is so yh the seating arrangement of guests at a wedding service in the church: two parties, lol and right of the aisle; then the seating arrangements at the reception; and tiv intensely scrutinized — and felt ~ meanings of these arrangements: at a wedding; \ the church; in a procession. Other arrangements of ‘the larger level social organizations (and their metapho articulations)’ are temporally and sequentially (and usually also spatially) organi rituals such as parades, church services, installations of the new ministers ol ‘government, state visits, openings of Parliament, graduation ceremonies are instarice bbut so are family breakfasts and mealtimes. Hour do these realize social arranger such as hierarchy, web, network, rhizome? Are parades, church services, state openit of Parliament, and so on, apt semiotic arrangements of such social meanings? Modes ‘Modes offer specific orientations to the world, and as such need to be mentioned he in the section on framing: temporally oriented, spatially oriented or a combination 0! both. Thece orientations produce distinet eultural arrangements and oricntatiov which differ from the more specific social arrangernents just discussed in being ‘non ‘general and abstract; that is, they are relatively though rot entirely indepenilen! of social arrangements; they are not immediately articulated in oF applicable to al social arrangements. Mode brings the ‘deep’ ontological orientations of a sociely av! zing/nate its cultures with it, In its guise as the means for reali Design and arrangements: making meaning material the answer to the ‘With what means?” question — it belongs here. Itis as though each ‘mode provides its specific lens on the world and with that lens the world seoms organized as specific arrangements in space, in time, or bath. These ‘lenses’ produce ‘complex picture in wich ontological and epistemological effects are colled in with ‘more broadly cultural ones. Mode, as the material stuff af signs, is central in giving ‘material form to meaning. Given the assumption that ‘form is meaning’ and that the relation of form and meaning is motivated, choice of mode is Foundational to meaning- making. Mode brings its logics, its entities, its syntagms and (social and) ser arrangements more generally. OF course, in other ways, that is the case with the choice ofall form, though differently and at different levels. Relations, processes and connections Relations, processes and connections among entities tend to have relatively stable ‘conventionalized forms in modes which have had a long period of use in a society. Best known, of course are linguistic names; for instance of clause types such as transitive ‘ intransitive, each of which implies larger level arrangements such as — looking at the syntagm syntactically ~ subject-verb-object (or complement); or looking at the syntagm in terms of meaning: actor-action-affected/nvolved. Arrangements of ‘entities can suagest classification; movement; action andfor interaction; state/being; ‘various spatial or temporal arrangements ~ whether spatial in image or temporal in music. Spatial or temporal ordering of entities and elements adds meaning to the symtagm. The social meanings of an aspectual kind, as I have called them in Chapter 6, need formal semiotic resources ~ stgnifiers ~ for their expression. social relation (as a Connection, let’s say) may be constant or intermittent. Using a line to represent that relation in the mode of image, might lead to the use of a continuous, unbroken line; Or it might be a broken, interrupted, dotted line. To indicate the strength of the Connection, we might use a thick line or conversely a thin(ner) line for an attenuated relation. For example: Their relationship was ‘pretty steady': as ——— Their relationship was ‘off and on!: as Their relationship was ‘really fraail Their relationship was ‘pretty up and down’ Their relationship was ‘solid The social conection may not be ‘straightforward’, that is, it might be unstable in some way and the line representing that relation might therefore he curvy, meandering, As with gravy, which at some point begins to change state, so social relations «an 15‘Multimodalty also undergo changes of state: these are culture-specific meanings to which the theory needs ta be attentive and to recognize through the provision of signifiers and names. ‘An action may be ongoing — continuous ~ is walking; it may be seen as completed ~ has walked, Here as elsewhere, modes have their specific means for representin such meanings — in the mode of speech as in writing in English, such meanings can bi expressed by specific marphemes, and the several meanings can be brought togeth to form more complex entities ~ inceptive, inchoative, continuous, completed: th sky darkens (inceptive + inchoative) ~ the sky is darkening (continuous + inchoative) = the sky has darkened (completed + inchoative); or, the gravy thickens Cinceptive inchoative) ~ the gravy is thickening (continuous + inchoative) ~ the gravy thickened (completed + inchoative); and so on. [am using linguistic examples to make the point relatively accessibly; the meanings — or meanings such as these though different ~ will occur in many mods. ‘gesture, pace and change of pace may be used; in image, colour and saturation cov be used as a signifier for such meanings: it might be less saturated on the left ‘the framed space and gradually change to greater saturation towards the right; or ‘might change in hue. In speech the speaker might speak increasinaly more /oudly faster, These resources are specific, socially and culturally: depending on what society needs or wishes to mean and to express. In speech or in writing, in anglophow cultures, meanings such as continuous, iterative, simultaneous, inceptive are relative\y ‘common in speech and writing. That means these meanings are ‘there’ in the cultw ‘and that makes it likely that these signifier resources exist in other modes in th culture. For instance, parallel ines; walking in step; rhythmic patterns in speech in poetry; in movement or in gesture; may all serve as signifiers of simuttaneity 0! states or actions or of congruence and coherence of views, actions, values. Visually the increasingly thicker shaft of an arrow can signal the (social) meaning of th Incentive character of a process, and so on. Depending on the made and its affordances, relations and connections may tv any number of forms. 1 may stand close to someone at a party as a means of mak ‘meaning; or turn away from someone. I may glance at someone or avert my glari an action performed with the mode of gaze. Metaphor met frame is one of the major categories of a social-serniotic theory. Mull \ modality and the notion of transduction add complexity and intensity to that fact Aptness and likeness are not the same: likeness is what it says: 'X is like Y thal \ eress focuses on what scems to be the criteria! feature of what is to be represent! and says that something else is like it. Aptness focuses on itness for purpose’: "this the best fit (the most apt) for this purpose here’ Likeness points to the relation of analogy and frame; in principle it provides a trai for two elements in an iconic relation, Analogy is te fourwlation of metaphor. Hen every sign, being formed on the basis of likeness and on the principle of analogy, Design and arrangements: making meaning material (formed as) a metaphor. Apiness speaks about the resources wiich are available in the ‘making of signs: ‘these resources are the nearest best fit”, The questions that need to be asked in relation to aptness is “Apt for whom?’, “Apt to what?', “Apt to whom?’. Analogy is shaped in a dual response: to features of the entity or phenomenon to be signified and to the demands of power with its effects in skewing the sign ~ the relation of signifier and signified ~ away from transparency and in its direction. Power acts, to ensure, where possible, that metaphor stays within boundaries set in certain ways inspecific environments. Power acts as a frame on the extent of transformations ~ for instance by setting limits on what is taken as *X-like’. Signs, and hence signifiars, come fromm specific social places and cultural sites. The red-and-white check tablecioth in a restaurant — in England as in other places in Europe — is a signifier of ‘Mediterranean-ness'. The oversize peppermill waved by the waiter over my plate is a signifier of “Italian-ness’. Many sians(signifiers bring, with the frame oftheir provenance, additional meanings into a specific mode ~ whether an instrument in music ~ the ukulele, the mandolin; or a French accent in speech, pronouncing maison with a nasal vowel; or a beret as part of my dress. What is signified in frames can be traced back to the interest of the maker of the sign. For the three-year-old, whee's proved criteria for representing cars; ust as some years later flames drawn shooting out from the wheels ofa different car signify power and speed, flames andi guns mounted on the car are by then both criterial as signifiers of ‘car’. Signs, inthis theory are at all times the result of a motivated connection of a signified — a meaning ~ and a signifier ~ a form which is already apt to mean that ‘meaning in a frame that provides the space for the link of analogy. Foremost among the terms integral to this theory is that of affordance, a term which points to the potentials and limitations of specific modes for the purposes of making signs in representations. Affordance rests, on the one hand, on the materiality of the stuff, which work in social environments has fashioned into a cultural and semiotic resource on the other hand. The materiality of the stuff of which modes are fashioned means that they have distinet meaning-orientations to the world, which I have called logics. Texts are always multimodal, so the rhetorical and design decisions lead to the making of ensernbles of modes; these are themselves assembled as orchestrations of modes, in which purposes and needs of the maker of sign-frames are brought together with ifordances in the best possible manner, from the rhetor’s and the designer's per- spective. Ensemble, in this context, names an emphasis on the madal multiplicity of the text, while orchestratfon names an emphasis on the aptness of the selection, the ‘mutual interdependence and the ‘semiotic harmony’ of such ensembles. Within such ‘orchestrations there are: elements and processes which link and reach across modes transmodal elements or processes; elements which link between modes ~ intermodal elements and processes; and those which (ink within modes ~ intramode elements and 15Multimodality processes, Such framing and linking devices are essential in overall multimodal textu! organization, | pointed to intramodal elements when I mentioned the function ar ‘meaning of punctuation marks earlier: question marks, dashes, colons, etc. These =!) have meaning-potentials, whether trans-, inter- or intra-modal. 8 Multimodal orchestrations and ensembles of meaning The world arranged by me; the world arranged for me ‘The meanings we make and encounter in all aspects of our daily lives are complex. Texts —spoken, gestured, written, dravin, mimed and any combination of these — are the means of making some of these complex meanings material. By means of texts we ‘can communicate if not ail then at least those meanings for which the modal resources f our cultures provide means of representation, Texts are made by me as ‘messages’ for others; and others make texts as "messages for me, so to speak. Most of the texts I make — as coherent complexes of signs — actually go entirely unnoticed by me and by others: they are meaningful all the same and still function as means of making my way in the world. Here I look at texts as, complexes of signs from two perspectives, that of arrangements and that of movernent. ‘Arrangements are made by me ~ as a modal ensemble made for myself or for others — br else someone has mace an arrangement as a modal ensemble for me. Arrangements ‘are made as ensembjes in a world of moverent: I, as maker of meaning move in the ‘world, literally, in different ways; and the world around me is in motion, in constant ‘movement; and more often than not, I move in a world in motion. Much ~ probably most ~ meaning made by me and by athers goes unnoticed or at least, is noticed but below the (conscious) semiatic radar, so to speak. When I cross street I need as much information as I can get about my immediate environment: the speed of an approaching car; its positioning on the road; maybe the blinking of an Indicator. I assess what else is happening on the road and what the implications are, for me and for others. 1 know that the Number 91 bus will not turn left here; I try to See where drivers are looking and what that might suggest; | try to make eye contact \with the driver of the car nearest; she or he night give me a fleeting signal ~a nod, a finger gesturing that it’s fine to cross the road: all this in fractions of a second. [ interpret all this stuff and so for me this is communication, even though most of It ic not intended as such by anyone, The blinking indicator is, yes; the positioning of the car on the road most likely is not. No court of law would accent my interpretation bf ‘what was communicated’ — but law and reality go their own ways. All of that stuff 5 regularity: if it did not, I could not use it asa basis for my decisions. Most of itis ot recognized as mode: not overtly seen as a regular, never mind as a ‘codified means of making meanina, Nevertheless, from all of this | make what become for me0 Multimodality Useful meanings, of different kinds. My judgement about the speed of the car tells mv accurately enough for me to act on, whether it i likely to stop; its position and ories: tation on the road gives me information whether it is intending to turn before it ce to me or to overtake a cyclist; or whatever. I take these as different kinds of infor tion ~ and interpret them as signs on the basis of quite different kinds of evidenc: lye contact; the finger suggesting 'OK, go on’; speed; the indicator; all come to me information through different material and via different senses. The stuff Lam ‘taki in’ isa diverse, complex set of materials, which I turn into the signs of a multimacia! ensemble of my own making, The ensemble and all the action that precedes and go with it isbased on my interest. My interest directs my attention, guides my framing o! this moment and that shapes the selections | make from this segment of this worl here and now, J don’t spend all of my day crossing roads —or sitting down for that matter. Anco course, [also hear what goes on around me; at times I even listen. Most mornin: conditions permitting, I go for awalk. | humour myself that I am walking ‘against clock’; and so 1 dor’t like stopping or slowing down when I cross a side street, to tur round and see ifa car is coming. I listen instead. If I can hear a car, | try to judge fro) the sound whether it is slowing down. If not, I cross the road without looking. \y attention and my framing of informational stuff, my consequent selection from th! stuff and my making of my ensemble of information is shaped by the contingencies 0! the immediate situation, If Lwas sight-impaired my disposition and orientation to the world and the infor tional stuff it provides for me to make meaning with would be profoundly differs! and so would, as a consequence, my attention, framing, selection, Now my miajvv resource would be the 'soundscape' of the street; the sounds in the spaces of the under ground station; the sounds of a park in a city. | would hear ‘perspective’: that motorbike sound seems further off than this bus sound; that car has passed me, 1) other one behind me is not slowing down; that one is approaching. I have at ti watched people who are sight-impaired negotiating what are complex environ for sighted people: I am thinking of an underground station in London which | w regularly. What for me, relying mainly on sight, isa cacophony of noise, is, assy for a sight-impaired person a rich environment of sound, as orderly perhaps oi! as regular as the flood of sight-impressions are for ine. My mare general point is th at any moment we occupy distinct positions and that (difference in) position ent. differences in attention, framing, selection. This applies to al of us, at each and every moment in each and every environment; always in specific ways. At times my attention is solicited; and the request is supported by varying degre fof power. The captain of the flight I am on tells me over the intercom that I shoul for my own sake, attend to the safety demonstration that the cabin staff are about to give. Now the framing is not mine; or at leas, it is not, if, unlike most of my fellow passengers, I choose to attend to the demonstration. There are spoken instruction gestures pointing out the location of the emergency exits, There is the actional performance: this is how the buckle is done up; this ts how you fasten the lifejacket Multimodal orchestrations and ensembles of meaning the oxygen mask is put on like this. However, the spoken ‘Place the mask over your nose and mouth and pass the strap over your head’ is much clearer for me.as a demon stration than it is as a spoken message alone, Demonstrating the use of the oxygen ‘mask, or ofthe lifejacket, works better for me than the spoken instructions, which do ‘not tell me which or where ‘the buckle here’ actually is. In the demonstration I can see I cannot hear it from the speech. Some airlines encourage passengers to ‘study’ the card with its written and visual insteuctions at the same time as the demonstration is, ‘going on. This too is a multimodal ensernbie; though this time itis clear that the material has been carefully arranged for me. Different meanings have been allocated to specific modes, in a particular sequencing. This ensemble has been orchestrated, much as the ‘orchestration of an otherwise simple tune. It has been ‘scored’ with care and precision. ‘The modes involved have been chosen with rhetorical intent for their affordances ‘and the orchestration has been designed with the characteristics of the specific environment ~ the difficulty of hearing the intercom announcements; the noise in the cabin; the somewhat stressful atmosphere; and the generally uninterested audience — all in mind We might reflect whether the demonstration also serves as a quasi ‘auxiliary channel’ for communication. On a plane where Portuguese is the first language used, the ‘demonstration pravides enough information for me to be sufficient by itself. We might conclude from that that the actional made of the demonstration is the foregrounded ‘mode for me and that for me it carries equal or even major communicational load. For a speaker of Portuguese, on that same plane, the sooken mode may well be the foregrourided made. My ‘position’ affects how I engage with a multimodal ensemble; it Is essential to be aware that that Is more likely than not to be the condition in most cases of communication; though our attention is rarely drawn to that. OF course, when I am crossing the road, I make my multimodal ensemble out of the ‘many resources available. Yet here too, in the plane just before take-off, this carefully orchestrated ensemble meets up with my interest (or lack of it) and itis still my interest which shapes my attention and guides my framing of this moment ~ even though the ‘ensemble had been carefully framed for me. In other words, presenting an orchestrated ensemble does not of itself guarantee or determine my attention, nor my framing and not therefore my interpretation, It does however provide the ‘ground’ on which ‘my Selection and interpretation take place. The ensemble makes certain resources available in a specific order; that is the material, the ground, from which I shape the prompt which is the basis of my interpretation. That is a fundamental point in communication: the difference between the outwardly directed making of the sign —> taxt —+ message by somaone for: me and my Inwardly directed making of a new sign in my engagement with that signifier ~> text > message. It is easy to lose sight of that crucial difference, lodged between necessary emphasis on the double process of sign-making: that of the ‘initial’ maker of the sign as a ground for me and that of my engagement with that sign prompt. Thw initial maker of the sign ~ the sign that becomes the prompt for my lo2 Multimodality response — ‘sets the ground’ on which my attention focuses (or not), and framin selection and my making of a sign/meaning for myself takes place. To repeat: that | ‘a fundamental point about communication and about theories of communication alike ‘The recognition of multimodality makes theorizing what is a complex process just tha: much more complex. The issues of orchestration and ensernbles are entirely related and yet distinct; thy former names the pracess of assembling/organizing/designing a plurality of sighs i: ifferent modes into a particular configuration to form a coherent arrangement; thy latter names the results of these processes of design and orchestration. Orchestratior scribes the processes of selecting/assembling/designing the semiotic ‘material which seem essential to meet the rhetor’s interests and which will be given shape ast semiotic entity of text as an ensemble, through the processes of desian. Orchestration land the resultant ensembles can be organized in space and they can be organizc:\ in time, in sequence, in process, in motion. The basic principles of design apply to bot! inds of ensembles, spatial and temporal; some principles apply separately av! ifferently to each of the two types of arrangement. ‘The world arranged by me, the world arranged for me: orchestrating ensembles, staging of movement, motion, ‘pace’ “The ensemble (the genre) of the safety demonstration on the plane is carefully orcie trated and it ‘unfolds’ in time. Different modes are foregrounded at different momen in the sequence and different modes carry specific information. By and large, mock ‘are used aptly: gesture to *point to’ specific aspects; speech to explain and recou action to demonstrate; and were we to look at the safety card there would be image to show specific situations and writing to provide complementary information for \ images in specific ways. The temporal unfolding is intended to reveal information a staged sequence for me. The next example shows a specific instance of an entirely usual example, ‘complex as my account from the plane. It is meant to push the exploration of ru! ‘modal orchestration and of multimodal ensembles a step further. The example com: from a series of science lessons on ‘blood circulation’ in a class for thirteen-year-ol in an inner London school (Kress et al,, 2001). The class has had three lessons 0) blood circulation and this is the fourth and last lesson on this topic. Before {\ students had come into the classroom, the teacher had drawn a large circle (with lines, to make a ‘tube’ with an inner and an outer ring) on the whiteboard (Figur: 8.1). At the top of the circle he had drawn a symbol to indicate the heart. As tl students enter the classroom — after a mid-morning break — this image is there 0” ‘board. During the lesson the teacher modifies the diagram several times; by drawn) arrows on it as he suggests the direction in which the blood is moving and whic! ‘organs the blood is ‘visiting’; by adding a second Joop on the top (Figure 8.2) (0 indicate that the structure of the circulatory system is i fact more complex than tly first drawing had indicated. His drawing on the diagram during the lesson allows hi ‘Multimadal orchestrations and ensembles of meanings Figure 8.1 Blood circulation: circle Figure 8.2 Blood circulation: arrows and loop to develop a more complex model from this simple one. However, he also ‘modified’ the diagram in other ways, which were no longer visible afterwards. He did this by ‘overlayina’ gestures on the diagram with his hand: moving his hand over the circle in the direction of the blood’s circulation; placing his hand lightly on the diagram at ‘various points to indicate the organs visited: "It goes to the lungs... to the small intest to the cells... to the heart.’ In this sequentially unfolding ensemble, at times speech is foregrounded, with the image as the focus of what is said. At first he uses gestures alone in relation to the diagram, to indicate the location of organs. He then picks up a marker pen and draws arrows inside the circle, at points which he had previously indicated in gesture. The‘ Multimodatity image is now permanently modified, A bit later in the lesson, the teacher bends dow and from below the desk he gots a 3D model of the upper part of a human torso; ti places this in front of him on the desk. This model is a three-

You might also like

• Shoe Dog: A Memoir by the Creator of Nike

  

  From Everand

  Shoe Dog: A Memoir by the Creator of Nike

  Phil Knight

  Rating: 4.5 out of 5 stars

  4.5/5 (537)
• Lesson3 5notes 1

  

  Document24 pages

  Lesson3 5notes 1

  grace_16

  No ratings yet
• The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good Life

  

  From Everand

  The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good Life

  Mark Manson

  Rating: 4 out of 5 stars

  4/5 (5795)
• • Magazines
  • Podcasts
  • Sheet music
• Final Project SFL

  

  Document4 pages

  Final Project SFL

  Septi Kayanti

  No ratings yet
• Hidden Figures: The American Dream and the Untold Story of the Black Women Mathematicians Who Helped Win the Space Race

  

  From Everand

  Hidden Figures: The American Dream and the Untold Story of the Black Women Mathematicians Who Helped Win the Space Race

  Margot Lee Shetterly

  Rating: 4 out of 5 stars

  4/5 (895)
• Paul McIlvenny Julia Zhukova

  

  Document3 pages

  Paul McIlvenny Julia Zhukova

  Juan carlos

  No ratings yet
• The Yellow House: A Memoir (2019 National Book Award Winner)

  

  From Everand

  The Yellow House: A Memoir (2019 National Book Award Winner)

  Sarah M. Broom

  Rating: 4 out of 5 stars

  4/5 (98)
• Michael Toolan - The Writer's Craft, The Culture's Technology (2005)

  

  Document293 pages

  Michael Toolan - The Writer's Craft, The Culture's Technology (2005)

  Anonymous uD8YEGV

  No ratings yet
• Grit: The Power of Passion and Perseverance

  

  From Everand

  Grit: The Power of Passion and Perseverance

  Angela Duckworth

  Rating: 4 out of 5 stars

  4/5 (588)
• New Perspectives On Narrative and Multimodality (Routledge Studies in Multimodality)

  

  Document243 pages

  New Perspectives On Narrative and Multimodality (Routledge Studies in Multimodality)

  Chioma Muonanu

  No ratings yet
• The Little Book of Hygge: Danish Secrets to Happy Living

  

  From Everand

  The Little Book of Hygge: Danish Secrets to Happy Living

  Meik Wiking

  Rating: 3.5 out of 5 stars

  3.5/5 (400)
• Comics in The English Classroom - A Guide To Teaching Comics Across English Studies

  

  Document20 pages

  Comics in The English Classroom - A Guide To Teaching Comics Across English Studies

  Xavier Richardson

  No ratings yet
• The Emperor of All Maladies: A Biography of Cancer

  

  From Everand

  The Emperor of All Maladies: A Biography of Cancer

  Siddhartha Mukherjee

  Rating: 4.5 out of 5 stars

  4.5/5 (271)
• RRL of Multimodal Texts (2 Studies)

  

  Document1 page

  RRL of Multimodal Texts (2 Studies)

  Sarah Jajimin Jul

  No ratings yet
• Never Split the Difference: Negotiating As If Your Life Depended On It

  

  From Everand

  Never Split the Difference: Negotiating As If Your Life Depended On It

  Chris Voss

  Rating: 4.5 out of 5 stars

  4.5/5 (838)
• Beliefs and Practices of Foreign Language Learning: A Visual Analysis

  

  Document25 pages

  Beliefs and Practices of Foreign Language Learning: A Visual Analysis

  Rocco78

  No ratings yet
• The World Is Flat 3.0: A Brief History of the Twenty-first Century

  

  From Everand

  The World Is Flat 3.0: A Brief History of the Twenty-first Century

  Thomas L. Friedman

  Rating: 3.5 out of 5 stars

  3.5/5 (2259)
• Language Acquisition Guide Updated

  

  Document83 pages

  Language Acquisition Guide Updated

  Mahesh Badgujar

  No ratings yet
• On Fire: The (Burning) Case for a Green New Deal

  

  From Everand

  On Fire: The (Burning) Case for a Green New Deal

  Naomi Klein

  Rating: 4 out of 5 stars

  4/5 (74)
• Linguistic Landscapes in The Philippines

  

  Document6 pages

  Linguistic Landscapes in The Philippines

  Amiel Demetrial

  No ratings yet
• Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future

  

  From Everand

  Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future

  Ashlee Vance

  Rating: 4.5 out of 5 stars

  4.5/5 (474)
• Purposive Communication Course Guide Term 1 AY 22-23

  

  Document11 pages

  Purposive Communication Course Guide Term 1 AY 22-23

  Chester Atacador

  No ratings yet
• A Heartbreaking Work Of Staggering Genius: A Memoir Based on a True Story

  

  From Everand

  A Heartbreaking Work Of Staggering Genius: A Memoir Based on a True Story

  Dave Eggers

  Rating: 3.5 out of 5 stars

  3.5/5 (231)
• 2nd Quarterly Assessment in English 10

  

  Document5 pages

  2nd Quarterly Assessment in English 10

  SaluibTanMel

  No ratings yet
• Team of Rivals: The Political Genius of Abraham Lincoln

  

  From Everand

  Team of Rivals: The Political Genius of Abraham Lincoln

  Doris Kearns Goodwin

  Rating: 4.5 out of 5 stars

  4.5/5 (234)
• My Dearest Dinosaur ES1 S1

  

  Document43 pages

  My Dearest Dinosaur ES1 S1

  S TANCRED

  No ratings yet
• Devil in the Grove: Thurgood Marshall, the Groveland Boys, and the Dawn of a New America

  

  From Everand

  Devil in the Grove: Thurgood Marshall, the Groveland Boys, and the Dawn of a New America

  Gilbert King

  Rating: 4.5 out of 5 stars

  4.5/5 (266)
• A Study of Critical Literacy Lau 2013

  

  Document32 pages

  A Study of Critical Literacy Lau 2013

  Victor Birkner

  No ratings yet
• The Hard Thing About Hard Things: Building a Business When There Are No Easy Answers

  

  From Everand

  The Hard Thing About Hard Things: Building a Business When There Are No Easy Answers

  Ben Horowitz

  Rating: 4.5 out of 5 stars

  4.5/5 (345)
• Seven Principles For Visual Culture Education

  

  Document6 pages

  Seven Principles For Visual Culture Education

  Carlos Castro

  No ratings yet
• Yes Please

  

  From Everand

  Yes Please

  Amy Poehler

  Rating: 4 out of 5 stars

  4/5 (1891)
• Unit Plan My Place

  

  Document19 pages

  Unit Plan My Place

  api-252762683

  100% (2)
• The Unwinding: An Inner History of the New America

  

  From Everand

  The Unwinding: An Inner History of the New America

  George Packer

  Rating: 4 out of 5 stars

  4/5 (45)
• GE 2 Complete SIM (Modified) 8-9-21

  

  Document312 pages

  GE 2 Complete SIM (Modified) 8-9-21

  louie jay aguado

  No ratings yet
• Rise of ISIS: A Threat We Can't Ignore

  

  From Everand

  Rise of ISIS: A Threat We Can't Ignore

  Jay Sekulow

  Rating: 3.5 out of 5 stars

  3.5/5 (137)
• Multimodal Discourse Analysis: Media, Modes and Technologies

  

  Document18 pages

  Multimodal Discourse Analysis: Media, Modes and Technologies

  Gabriella Marques

  No ratings yet
• Principles: Life and Work

  

  From Everand

  Principles: Life and Work

  Ray Dalio

  Rating: 4 out of 5 stars

  4/5 (599)
• Summary Discourse

  

  Document8 pages

  Summary Discourse

  Devi Selfiani Sabri

  No ratings yet
• Fear: Trump in the White House

  

  From Everand

  Fear: Trump in the White House

  Bob Woodward

  Rating: 3.5 out of 5 stars

  3.5/5 (738)
• Grade 1 Science Curriculum Guide

  

  Document66 pages

  Grade 1 Science Curriculum Guide

  Jerico

  No ratings yet
• Angela's Ashes: A Memoir

  

  From Everand

  Angela's Ashes: A Memoir

  Frank McCourt

  Rating: 4.5 out of 5 stars

  4.5/5 (440)
• A Multimodal Discourse Analysis of Selec PDF

  

  Document8 pages

  A Multimodal Discourse Analysis of Selec PDF

  Sebastian Jimenez

  No ratings yet
• Bad Feminist: Essays

  

  From Everand

  Bad Feminist: Essays

  Roxane Gay

  Rating: 4 out of 5 stars

  4/5 (1016)
• Study Guide by Jennifer Connolly: ISBN: 978-1-74295-536-0

  

  Document16 pages

  Study Guide by Jennifer Connolly: ISBN: 978-1-74295-536-0

  Richie Huang

  No ratings yet
• Steve Jobs

  

  From Everand

  Steve Jobs

  Walter Isaacson

  Rating: 4.5 out of 5 stars

  4.5/5 (806)
• Welcome To Advanced Communication Class

  

  Document10 pages

  Welcome To Advanced Communication Class

  John michael Del rosario

  No ratings yet
• The Glass Castle: A Memoir

  

  From Everand

  The Glass Castle: A Memoir

  Jeannette Walls

  Rating: 4.5 out of 5 stars

  4.5/5 (1713)
• 2021 - 3JA - IOP - Manfredini-Leardini - Critical - Urbanism - and - The - Digitally Augmented City

  

  Document13 pages

  2021 - 3JA - IOP - Manfredini-Leardini - Critical - Urbanism - and - The - Digitally Augmented City

  Manfredo Manfredini

  No ratings yet
• The Gifts of Imperfection: Let Go of Who You Think You're Supposed to Be and Embrace Who You Are

  

  From Everand

  The Gifts of Imperfection: Let Go of Who You Think You're Supposed to Be and Embrace Who You Are

  Brené Brown

  Rating: 4 out of 5 stars

  4/5 (1090)
• Identity in (Inter) Action Introducing Multimodal Interaction Analysis (Trends in Applied Linguistics) by Sigrid Norris

  

  Document317 pages

  Identity in (Inter) Action Introducing Multimodal Interaction Analysis (Trends in Applied Linguistics) by Sigrid Norris

  lavinia tiganila

  No ratings yet
• John Adams

  

  From Everand

  John Adams

  David McCullough

  Rating: 4.5 out of 5 stars

  4.5/5 (2409)
• Johnson Columbia 0054D 13353 PDF

  

  Document419 pages

  Johnson Columbia 0054D 13353 PDF

  Edward Brookings

  No ratings yet
• The Outsider: A Novel

  

  From Everand

  The Outsider: A Novel

  Stephen King

  Rating: 4 out of 5 stars

  4/5 (1839)
• PC Syllabus For Canvas Ver 2

  

  Document9 pages

  PC Syllabus For Canvas Ver 2

  Aine Cebedo

  No ratings yet
• The Light Between Oceans: A Novel

  

  From Everand

  The Light Between Oceans: A Novel

  M.L. Stedman

  Rating: 4.5 out of 5 stars

  4.5/5 (789)
• Etextbook 978 0393265293 Everyones An Author With Readings Second Edition

  

  Document61 pages

  Etextbook 978 0393265293 Everyones An Author With Readings Second Edition

  elizabeth.myers417

  100% (48)
• Manhattan Beach: A Novel

  

  From Everand

  Manhattan Beach: A Novel

  Jennifer Egan

  Rating: 3.5 out of 5 stars

  3.5/5 (792)
• Untitled

  

  Document567 pages

  Untitled

  Andrea English

  No ratings yet
• The Sympathizer: A Novel (Pulitzer Prize for Fiction)

  

  From Everand

  The Sympathizer: A Novel (Pulitzer Prize for Fiction)

  Viet Thanh Nguyen

  Rating: 4.5 out of 5 stars

  4.5/5 (121)
• Purpossive Module 2

  

  Document41 pages

  Purpossive Module 2

  cham

  No ratings yet
• The Woman in Cabin 10

  

  From Everand

  The Woman in Cabin 10

  Ruth Ware

  Rating: 3.5 out of 5 stars

  3.5/5 (2322)
• Brooklyn: A Novel

  

  From Everand

  Brooklyn: A Novel

  Colm Tóibín

  Rating: 3.5 out of 5 stars

  3.5/5 (1937)
• A Man Called Ove: A Novel

  

  From Everand

  A Man Called Ove: A Novel

  Fredrik Backman

  Rating: 4.5 out of 5 stars

  4.5/5 (4610)
• The Perks of Being a Wallflower

  

  From Everand

  The Perks of Being a Wallflower

  Stephen Chbosky

  Rating: 4.5 out of 5 stars

  4.5/5 (2104)
• Wolf Hall: A Novel

  

  From Everand

  Wolf Hall: A Novel

  Hilary Mantel

  Rating: 4 out of 5 stars

  4/5 (3811)
• Little Women

  

  From Everand

  Little Women

  Louisa May Alcott

  Rating: 4 out of 5 stars

  4/5 (104)
• Her Body and Other Parties: Stories

  

  From Everand

  Her Body and Other Parties: Stories

  Carmen Maria Machado

  Rating: 4 out of 5 stars

  4/5 (821)
• The Art of Racing in the Rain: A Novel

  

  From Everand

  The Art of Racing in the Rain: A Novel

  Garth Stein

  Rating: 4 out of 5 stars

  4/5 (4200)
• Sing, Unburied, Sing: A Novel

  

  From Everand

  Sing, Unburied, Sing: A Novel

  Jesmyn Ward

  Rating: 4 out of 5 stars

  4/5 (1103)
• The Constant Gardener: A Novel

  

  From Everand

  The Constant Gardener: A Novel

  John le Carré

  Rating: 3.5 out of 5 stars

  3.5/5 (104)
• A Tree Grows in Brooklyn

  

  From Everand

  A Tree Grows in Brooklyn

  Betty Smith

  Rating: 4.5 out of 5 stars

  4.5/5 (1929)