HANDBOOKS “ ROUTLEDGE ultimodal Analysis Second Edition Edited by Carey Jewitt_ The Routledge Handbook of Multimodal Analysis Mulkimodaligy is an amovative approach to representation, eatimunic.ition and interaction which looks be age 10 investigate the multicide of ways we commumeate: through images, sound and muse to gestures, body posture and the use of space The Rourledge Handbook of Multinaadal Aualyses, Socom Faition provid -arch (ool kit’ for mullimodal analysis. with dhirty-four chapters written by lesdbng nid Ta hen a comp) Figures in the fielel an a wide range of chearetical and methodolugical xsues. This new edi- slopments, ancl tion includes twelve new chapters on theoretical and metho fexts and interaction. mnultitnodal research on digitally mediated The Huudbook inclinies chapters on key themes within multimodality such ay tech nology, culture, notions of identity, social justice and power, anid macro issues such as Interacy policy. Taking a broad nok at multimedality, the contributors engage wath how 1 variety of other theoretical approaches have looked! at sultimodal communication acl representation, inchiding viswal stvshes, anthropology, conversation analysis, socio cultural theory, sociolinguisties, new literacy studies, amltinedal corpora studies, critica] discourse, semiotics and eye-crackineg. Decailed multimodal analysis cise studies are abo included, along with an extensive updated glossary of key terns, to support these new to multiniodality and to allow those heady engaged a muTémoddil resestel ts explore the fundamentals farther. ‘he Roatedge Hawdaok of Mati sot and post griduate studeties as well as researchers involved in th dil Analysis essential reading for une study of multimodal eonmaunication Carey Jewitt is Professor of Technology and Learning at University College London, Institute of Education,Related titles he Routledge Handbook of English Language Teaching ql Exdived by Gran The Routledge Handbook of Language Learning. and Technology Eudited by Fie Farr aid Liana Many The Roudedge Handbook of Language ane Identity Pudited by Sian Provce The Routledge Handbook of English for Acaclemic Purposes Edited by Ken Efyland sii Philip Shaw The Roudedge Handbook of Language and Digital Communication Fulited by Terera Split The Routledge Handbook of Literacy Studies Edited by Kaw Pak The Routledge Handbook of Interpreting Fulited by Renée foneeerses ‘The Routledge Handbook of Hispanic Applied Linguistics Fediseal by Manel Lacoste The Routledge Handbook of Educational Linguistics Eulited by: Mart low and Jobaina Ennser-Kananet The Routledge Handbook of Foren: c Linguistics Fslied by Malesliar Gowlehand and Aliso Jobson The Routledge Handbook of Corpus Linguistics alited by Anne VK ike McCarthy The Routledge Handbook of World Englishes Felted by Any Kirkpatrie The Routledge Handbook of Applied Linguistes E The Routledge Handbook of Discourse Analysis xl oy Jamtes Simpson Edited by Janes Pal Goe acl Michac] Handling The Routledge Hanelbook of Second Language Acquisition Fated by Sica Css anid Alison Mackey The Routledge Handbook of Langoage and Intercultural Communication The Rutledge Hi Eultea! by Gletnn F jandbook of Language Testing: leber and Poed Davidson ‘The Roudedge Handbook of Multitingualise Faitcd by Marilyn Maron Jones, Adrian Blaekleus The Routledge Handbook of Teanslation Studies Edued by Carmen Millin-Varela and Francesea Bastrong The Routledge Handbook of Language and Health Communication Falned by Heli T. Hamilton and Wen-ying Sylvia Chott ‘The Roudedge Handbook of Language and Professional Cormunication Fated by Stephen Bremer and Vijay BhatThe Routledge Handbook of Multimodal Analysis Second Edition Edited by Carey Jewitt RQ Routledge | ONDON AND NEW YORKFirst publisied ia paperback 2017 First published 2014 by Roueleelge 2 Pork Square, Milton Park, Abungdon, Oxon XH ancl by Rooutle 17 Pied Avene, New York, NY 10017 asi i. pr of the Tg £5 ia Crys an ina a 96.2024, 2077 welecon and editorial matter. € cchapeers. he eomu iautors e Jewnte:incivital The right of Carey Jewite o be xlonntied as she author of Us teitorial mnaerial ant of the auckers foe hein indvisial chhpters, hay been asorrecl m) asourbiace with sextions 77 and TH af the (Copynglit, Designs amd Paterts Act LSS. All riyes reserved, Ne purt af ths book may be meprinted of produced! or uniliscal sy any fora oF by any elvetzanie, eicch nica, ier inventgal insling anal recording, oF ip aly anfornacit stor photocopin out perHussiont a wring thom che publishers Wy be tradesnarks a Fiat noties Product ov coxponite nanies 1 registered Crademarks, and axe used only tir icleribfieatien anal fi Publication: Para Baile Labuny Catal A catalogue record to this book is available thon the Britidh Libsary in Pablvations Data ra Tchides bibhingraphical st L Modes (Ls 2. Communication, 3. Seaaories. 4, Sgnibolisin E Jewat, Casey. J1-Title: Elandbonk of rltinnodal analysis POO. MOP oN Dt + B2P2-de23 DUN MUL T667 uaasies} Haoxdboks, nama ete ISBN 9780-415. S1U7E-8 chisky ISBN: 978-1. 13 21519-8 ip) Typesct in Bem by Bonk Now Lud. LondonDedication To Gunther KressContents Lis of ilestnnions “ List of comers a Ackrawledgenient: NXW Intraduction: handbook rationale, scope snd structure 1 Part I: Theoretical and methodological tools for multimodal analysis 9 Introduction to Part [ ul 1 An introduction to wekimodality 18 jy fet 2. Different approaches to multimodality ay Carey Jewitt 3) Whar are multimodal data and transenption? 44 Roxie levi, Regine Hampel, Migaru Hla wud Lesley Laasten 4) Whatis mocle? 60 Gunther Kress 5 Parametric systems: the case of voice guilty 6 ‘Theo Vin Lermsen 6 Modal density and 2uodal configurations: snultimodal actiuns 86 Sigrid Novis Transformation, crunsduction and the transiodal nronient 100 Denise Nevfilt Part I: Further readin: WsCONTENTS Part I: Key themes for mul fon to Part U1 modality iL changes in the semiotic lanchcape: from caleulation te computation Kay LO'F 9 Technology and sites of disp Rodney HE Je 1a Mulsimediahty and mobile culture Kevin ML rider did Lalitha M. Vistadeasns 11 Mylimes ay Larne lity, wlevacity. and tase 12) Multisiosdality aid simige text intenieton Le Uynswonle aan Cris (izing dings the coustruction of meaning: Uarough 13 Power, social justice and nmlsmadd pedagogies Arlene Archer Part II: Multinaodality across different theoretical perspectives Introdvsetion to Part HL HH Mulsimodsbyy and lingnages «retrospective and prespeetye view Hon Scollon and Susie Hong Siallon 15° Mulumodality and shearies of the visual David Me 16 Multintiodality al New Litewtey Stadies Brie Set, Kare Ball ard frongion Rosevell Using multimocul corpora for enipirieal research: fal. Baseman IN Towards an analytics af mediation Fale: Choline 19 Semiotie prradignts ad multimedalicy Hani Stocked 20 Reception of multimoslaligy: applyin an tute inwdal research Jiurat Helsanons eyesteack nictlodolagy 21 Representations in pretiees: a socio-cultural approach to sulimedality Je teas, Jonas Lidenurs anal Reger Sid 7 119 123 139 152 165 176, 189 199 201 208 237 209CONTENTS: 22 Indefinite precision: artefacts «ned interaction in design ni Baud Ley? Christine Heal arad Karas Pets h 25 Andmopology and multimodahty: the conjugation of the senses 323 David Howes Pare IV: Multimodal case studies 335 Incoaluetion to Pare IV 337 24 Practical fancon and messing: 4 case study oF IKEA ables 342 Anders Bronk 25. ‘The use ef gesture in operations 354 Aff Beever 26 Gesture ani movement in tourist spaces 365 Adan Jacorsk and Crispin Mnafow ‘The kineikomie moc: towards a tnulomodal approact to movingeimage media a5 Andrew Barn 28 Multimodal analytics: sothwave and vinalization techniques for analyying, and interpreting multimodal aiea 386 Kay L O'Hdalloane, Wlaroea KA. 2 anal Sabine‘ 29 Colour: code, made, modality — the case of film snd video 397 “Them Ven Leviesven 30° Mosic and designed sound uy ‘Tove Hist 31 Space and commsenication in exhibations: unravelling the nexus. 49 Maree Stengtine 32, Image in dhe anultisnodal estsemble: cluldren’s drawing 431 Diane Mavers 33. Online newspapers: structure and Layout 440 Jota S. Knew 34 Whac next for mulimendalicy? 450 Carey Jensie Glossary 436 References ah Jee 513Illustrations Figures Bal Transcription of synchronous online intentetion ian avnbo conferencing environment 3.2 Transcription ofa section of an episode of naturally occurring imteracoon berween adult and child arownd the construction af a text 33. Transcription ofan advertiverient 3.4 Transcription of multiparty interaction 3.5. Drawings integrated into: an extensive transerspt of speech 3.6 Visualisation of 60-minute visleoconfereneing session, showing use of broadcasting tool fspeeciy, chat and other tools 44 Digestive system: orgiaul a double-page innage (pp. 90-91) 42 Digestive stem: columns of inage and writing, reversed 43° Digestive system: Ginteajcextual ancokterennce” 44 Blood-cireuation: diary 45 Blood-circulation: concept maps 5.1 System network for the expaewion of satin it mages 5.2 System network for vonce quality 6.1 Higher-level action of assembling DSL:simuleancous highe® level action of interacting seh the researcher 62 Sandra's foregeounded ani! mid-rounced bigher-Jevel actions constructed sinulsancously with the sume modes hut a differing modal density 63 Modhal configurations for ovo simultaneously performed higher-level actions TA Kim's artefiet, firs "page? 7.2 Translations inside the matchbones Sola The Method of Musions ant badhiae Sores 8th La Chamcivie 53 4 55 5b 57 66 68 6 7 72 82 83 95 96 ow. 107 131ILLUSTRATIONS, Kile Muchentatiselie Sifitiens: Baril Soi Muthemarscler Sifntien: Bar b Philosophiae Nirtwraliy Principia Mathewatiea Beethoven’ Filth Symphony coded in br: An example ofa ste of splay a noticeboard ina park i China wh the daily newspaper is hung every serning and passers by slop to peruse it ¥.2 A personal profile on the popular Hong Kong guy website Eridze.com 9.3 The disphy parts of the body posted o the websice Ye The “hearts toolbar of Vridae.corm 10.1 Piccure sent by Sophia of foc! holdings 4 snerophone and apparently loaning over to kiss a puppy Git was sittings on top of (band menibes) Bonijrs shoulder 12 The layer loaded with the worek “wish + sas” in white bubble leners 1.3 A readily available map of Naboo which could be sealed at different love, and directly pasted over Brian's enihoxticd (avatar) perspectnee of the world Ho4 A granspucne smmediate oF local perspective on the map 15° A dluagranimatic representation of Jory’ modal engagements wath the PSP and the subsequent artifacts that result 12.1 lumge or language as identities 2 Lile Cycle of v eiead 12.3 Making switehes 124A Greek water elock 17.1 Exanph of working with ANWIL on gescure TWA “ARewvity ruins everything’ Colgate Toothpaste, Prolm ¥8R. Santniges de Chile 21.1 Architectital critique as 2 communicate prictice 21.2 ‘The student image discussed 22.1 The initia sketch for che ship snedek room made en she paper phot with dhe “hub‘in the centre: other cases are shown as feint circles around the bub and dae case for the treasure model above it, labelled PREASURE? (ketch is mae a viewed Hon above) Fragment P images Stoves dhawings oF « case positioned just helow the phan (ade clevation) Fragient 2 amaages The extended drawing showing the ‘weasune model" Vhe computer-aided design: procicedl asa consequence of hese diseussions Human table intesicavity: system network: The Lack table The Bjorktulalon table The Grunle cable The Laver tal An angestheti¢ toant ring. a conversition A surgical ceam doing an operation Example |: consultant surgeon holds a titel up Exarnple 2: consultant surgeon a skes sweeping movement alosg the 140. 144 197 149 158 158 160 160 163 181 12 183 186 245 2m 305, 306,26,3 204 2d 27.2 28 301 3h ILLUSTRATIONS Exanople Sesurgical roggistear hole tissue up 350 Examples of tourist’ pointing behaviour 309 Exanaples of tourists’ posta Synibolically marking, chunbsing thie Tow _plucing and posing: sm Wand demtarcating space: {af che effort 0% Eb} rows of visitors sea fed ow the steps at the foot of the Tower — many of whom are also gazing up a it a7 Prosthetic etmboxbinents: f) mapping anid capeoring the’Tower by hang; (h} sinmchancously photographing: and pois 373 The metamatlal kineikonee sF8 Metamodal analysis af a scene fivay Olivier’ Fanta 379 Shor-revene-shor flow camera angle). speech and auusic 383 Seren shor of the analyvcal eranscryption wool, Tatcog 415 The entrance to the Stills Gallery =a elear Path Veiue eraectory swith song Prontinence. Ells Deeytos, Sidi, Pile of Leters, Still, Gallery, Syatney, 2008 470 Prominence continues to luse visitors, Ell Dreyfias, Ssuanty is Showing the GivenNew sealing path, Lhe Dreyfvs, Seana, Stills Gallery. Syne, 2008 1 Given-Now: the new provides a context for the phrases. Ell Dewy tis 1 Stills Eallory, Sydney Suuanbug. Stills Gallery, Sydney, 2608 Photographic diptyct. Ella Droytus, Silay, Prateet Myself, 2008 Rosied map lage 99 A node froma Michaels map fage 2 435 A node rons Dapikas map fage 8) 36 A node fons Rose’ map {age 9b A linked theme fom Rosie's sap {1 438 33.1 System network of mewsbite design elt} anal neswsbites (right font the main home page of the Cauardian, 17 August 2012 aus Plates 3.1 Malnmeydal transcription frame TA ‘Thebuwa cloth with snaps and poems (elotl, thread, paint, papper doll-like figures, photographs, safety pins) 20.1 Reader's interaction wich a newspaper spread 20.2 A newspaper spread is read for one sunmute by two readers 20.5 Mathematical probler-solving in vector calelus with illusteatwon 24 Reading ove versions of information graplacs; (a) separated graphics: (b) anteggrateel graphics 20.5 Example of color coding: keywords in the text and labels identifying, parts of the iThtstration sue of the same color 28.1 Time-scanped mnultimadal database 28.2 The distribution off che chisters over tine 28.3 Most Foquent cluster comtbinations-of- have 28.4 Neowork cagrans for textual metatimetion: fa) Jon Seustt: (b} Kevin Trenberth; fe} Myron FbellILLUSTRATIONS 33.1 The wai home p. the Cutie home page: the aa G Desa hone pagetand a story page (left-right) 33.2. Atmontage of menus font che tap of fe home pages frome che ruardiare, 1? August 2012 Tables 2.1 Acommparnon of historical influcnees snd the theo weal exuphanas of three mulkimedabty porypee 121 tne age in the suits caption text for thee water clack 16.1 Typology to deseribe different ese isualizang la sch takes on the inerseetion benween the New Literacy Studies and multimodality 28.1 Clusters for textunal metafiunction Major clustors for interpersonal nietalunction Major ¢ ing 4 sters for process types Isr 235 391 ww 393, 3MContributors Editor Carey Jewitt is 4 Reader sn Education and Technology at dhe Lo: Research Director of the [oE Ceri for Multimodal Research at the bastitate af Education University of London. Her research is involved! in developing: visaal and multimodal method and theory and investigates the relationships between representation, technologies and ped wrtnal Visual Comienication, wath Theo Vax » Knowleslge Lab and agogy: Carey is a founding evitor of the j Locuwen anal her books include: Ferlnoly, Literacy: Learnings A Multinadatiry Appr (2008, Routledge}: Fay Jeagues: Mut Scieme Classtognn: A Mu leagues, A Handhoot of Vital Analysis edited with Theo Van Leeuw 1 Urban Chasrooms (2003, Routledge) with Guather Kress and cole Peter Langs Rleetories of she amadal Literacy edited wath Gunter Kress (2008, 2001, Continuum) wich Gunther Kress and col (2001, seal A porous Contributors atthe University of Cape Towa, Souls wud Media. Arlene Archer is the director of the Writing, Comin Africa, She teaches in Applica! Language Stsdies, Hi Her research eniploys a social semiotic perspective to 1elook at key oncertis antl concepts offare“acach whinge and research. 1 alse and miultimental peclawogies to enable sevatent aceess to Higher Editcation. She has published ovial Dystaniis anal Brstish foam ef Bide + Education Studies, fue lite tie i aciesTapproach c0 jes on popular culeare in journals such as La 4 Euducations, Pieadt Commutation, Reaching in Higher Eaucation nls ie Lsducatin WFeshnology, John A. Bateman is Profesor of Applied Linguistics st Brostien Laverty: and leads hieulty research group on transinedbal textuality, He has heen researching issttes of reaming. makin, empleo 1 diverse senmnotic mouluisies since the mid-19YUs, Alter working on the automatic generation of mixed-mienle page based documents, vats in dhe carly 20008 co apply oth sate and dynantic niudtimodal arte linguistically inspired snethodtologses to facts (es narrative filmy) and to address the semiotic foundations of multimodality: Books ins the area nyILLUSTRATIONS 33.1 The main home page:the Ctene home page:the nr & Design home page: and a story page (lele-righth 33.2 Ammontage of menus from the top of live horn pages front she Guelian 17 Auguse 2012 Tables A comparison of historical aaflnences and the theoretical of duce mulsinoxdality perspectives 12.1 tmage visualizing language in the iain caption text For the water clock 1.1 ‘Typology te describe differcnt research takes art the antersection hetseer the New Lis Clusters for cescial sie acy Stunlies andl naileimostaligy untca0en Major clustess fir interperse niet Majar elustexs for proces Spoabong tine 393 304Contributors Editor Carey Jewitt wa Reader in Education and Tochnole y at che Landon Knowledge Lab and Research Dizector of the [oF Centre for heltsmadal Research at the Institute of Exueation, Unversity of London, Her research is involved in developing visual and multimodal method and theory and investigates the eelavion ships beoween representation agogy: Carey’ is a founding editor of the journal Motil Gowan, wah Theo Van Leeuwen andi her books incluele: Jerbnalvey, Bitenaey, Leaning: A Mutiniadadity cl pproarle Q8, Routledge): Fyglist ine Urban € ) with Gumther Kress and col- hnologies and pot roams (2005, Rented gues; Mudtinwdul Eatery edited with Gunter Kress (20013, Peter Lang); Rhetories of the Scone Classroom: A. Multineetal Approach (2001, Gontineumn} with Gunther Kress and col leagues, and A Haedbook of Vi ins Leeuwen Analysis echted with Theo Contributors Arlene Archer is the director of the Writing Centre at the University of Cape Town, South Applied Langue Stuckes, Higher Edacation Stuedes, Fslay and Media Her researeh employs a sueial sentiorie perspective to re-look at key concerns amd concepes ofan maculenine bteracies approach 1 ind research, [abo diss on popular culture tion. She lias published iil Communication. Teaching ia Higher Val and multimode pedagogies to chable student access to Higher Edu age anad Hida Dyas an journals sc as Lang Finglis i Journal of Edewational techunobgy John A. Bateman is Profesor of Apphed Linguisties ac Bronwen Universiey and leas faculey cateh group on transmmedial texte: He has heen researching issaes off meanting-anakings employing divetse semiotic modalities since the mnil-1 990s, Aker wouking on the aattomnatic generation of mixed-mode pa ding aarnaeive fil vsed docuntents, he began an the carly os 1 both static and dynanie snulisnodal arn Ud to sildress the semtiotic foundations of multimodality: Books int the area 2001s to apply stically inspired rmethadolog: facts fe. svCONTAIBUTORS inchacde: Mudtimodatiey wed Grass A Fnendition for the Spsseanatic Analysis off Mubiinedal Daeaients 2008), ysis: Hew Plas Mew (with Rarl-I enrich Schmid 2UT1} and the forthcoming textbook Text and fagescL Chival larenduetion tthe Visual Verbal Dirafe éuue 2014, invedal Fifa Jeff Bezemer isa Senior Research Fellow and Co-Director of ch Rese communication, learning and entity fiona mulinsadal perspective, mong ethnographic, semiotic and interactional theorws anid mrethods. He bus (co-) drected finnded re For Mehimodal © Insgtite of Edicatio, Universicy af London, Peis interested in reffaming rch 00k, ie patient books, addressing methods training pragrammes and fiance research projects on mnlilingualisnr in s uation ha textbooks, chizea] communiestion, surgieal education anid alin repre Fonuis. He has published widely ana range of different journals substantive igues (social anequuality, patient aly safery), issues in muiluisstodality gesture, visual digital ceca and methodologieal sues (uulcimockl crasenpion and vidleo-analyss). For mstare details please visit jeflhezemer.sondpress.coms Anders Bjéirkvall is Associate Professor att Seunilinay and Muleifia ordinaces a research project on ‘one-to-one’ computing and young people’ ninltimodal 1 Langriges at the Deparunent of walisat, Stockhtolen Universty. Andes currently c= Swedish angus Titexacies, His recent publications snelile ‘Multiniodality” in Fla i awoke af Prgymarics 12012), The muateriality of discourses and the semtiotics of materials: social perspective en the micanitiy potenbals af written costs ad foenaure’ in Srrrosiea (301 1 with Anna- Malin Karlson} and “Young children’s exploration of semiatic resources during uno ficial computer activities in che chassronsni” sn fern with Charlotte Enghlomn) analysis (Den visnella serene: Mls af Endy Chilatiivod Literacy 2010, salso pubhshed a Swedish handbook of nuulsinodal ders dul uals» Je, 2009}, intvashieanys a social semiotic approach (© cnultimodality to a Seandingvian aushenee. Andrew Barn is Profesor of Media Education at the Insature of Education, University of London. He hay researched and published work on inany aspects oF the mncctiaarts inelabing yourig people's production af digital media such av computer gunes,ligatal video and anna tion, He has rese.treh interests it Hae yl games. His recent research projects include studies of childen’s playyground games (vith the Brivsh Library}. of younys peoples videogames of Shakespeare (with the Globe Theacre} and of young people's machinians-mnakinsg (vith the British Fins lsstitace), His ntost recent backs ore Mak ta: Creative Production wed Digital Litenies (New Yorks Peto Laryg)s and Meili Lieniy si Schoise Pracve, Pneucrien, Progression (Loxton: Pal Chuapmast) ¢ Chouliaraki is Professor in the Deparment of Media and Communications, London School of Beononties, She is the author of Discourse in Late Modernity (with N. Fanrelough, BUD, 2000, The Spectavanhip of Suffering: Sage, 2000) and The Ime Spectatar, Solidarity i de Age of Post-hunuatarianis (Polity, 2012) as w as rm As of ous publacations i the Bi med elses, journalism, buansaniarsa cc mumunication, discourse theery an analysis Chris Cléirigh has been 4 Research Fellow at the University of New England and is currency a Researclt Fellow and Research Associate at the University of Sydney: His mic publication is: Thommen, EAA. € Co, Head Land Muti, P2008) “Gardenes ialCONTRIBUTORS linguistic study of relationships between covitenmental attitude Linguists sand the Hlaaas Srienes, Val. 2.3, Bqqnitioss Londen belioh and practices’, in Marissa K. 1. E isa Teaching Fellow at the Cenere for English Langsiage Comienication at the National University of Singapore. Her research interests include systensic finctiarral Jinguiscies. mulsienodal chscourse analysis and theis aplication sn analyzing the constitution, perpetuacion and evolution of ideologies and eultore as manifested an traditional anid emer gent forms of media communication, Ske has worked on inter-disciplinary piojects at the Multimodal Analysis Lab at the National University of Singapore savolving the application of mathetnatical mock rechaiiques to che analysis of multimodal daca and die develop ment of interactive and collaborative sotivare five mnulsieiodal analysis for research and edu cational purposes. Her publieasions reflees this synthesis of diggital and cadatienal iuantaniti purr P in various areas such as climiate «cienee news, istitmonal business news, corunite advertise ments and oeucation. Rosie Flewitt is. Senior Lecturer in Ear Edu include communication and literacy int homie and educational settings, with a fheus en edt Years and Primary Education at the Inseituce of aon, Lonlox, ancl a member of the Londen Knowledge Fab, Per rescarch interests nographic and mulcimodal methodologies. Recent research intebudes ant evaluation of Vivian Gussin, Paley’ Helicopter Techoique for storytelling and story acting in early education (Hsmée Fairbairn), dhe suahimodtl po University) andl Multimodal Literacies sn the Barly Years (ESRC) literacy skills ancl practsces that 3+ anct 4-year-old children develop as they eng diverse printed and witial of the iPad for ely literacy leareing (pen exploring tie range of we with Regine Hampel is 2 Senior | eeturer in Modern Langtiges al the Open University and Associate Dean tResearch and Schokarshap) su the Faculty tor Education a Studies. Her rese teaching, She is paraeulirly interested in the ud Language ich focuses en the impact of using digital media for language Tearing arid atfordances of new tehnologies. cask design, learner interaction andl collaboration, onlise ht include the book Calin: Conmmvaicattore ity Lan Syston: Aly Hateraational fowl of 1 cicies, nel teaclter trainings. Her publications sel Tea opie Faigguistirs guage dacansio Hing. She is co-editor of cational Tish Mitjain Hauek is a Sexior Lecturer in German and che Associate F ead of the Dey Languages ac the Open University UK, For over a decade she his heen invalved in invest gations of virtual spaces tor the learning of Languages and culsure.dn her current researet anid publications she explores the role of digital Liteaey in globally networked Jearning, where syochronows and asynehiengas online tools are used to bring together students from differ ent parts oF the world for the development of callaborsive project work and intercultural exchange. Her particular focus in this co co mmultintedal Xt is the intendepernesice commumcative competence and intercoltural communicative competence an online language} learning and coaching envinonmenss Christian Heath 3 Professor in the Department of Managetes London and is cowdinector of tie Work, hiteraction and Technology Research Centre, He hus hell posi= tions at the Universities of Nowinghani, Manchester, Konstanz, Trepte and Trondheint and Uke Xerox Research Laboricorivs. He specializes int fine-grained video-based stadies of socialCONTRIBUTORS Interetion a0 organizational settings and is currently undertaking esearel in areas that include su ios. Projects are fiandeal by the research couneals (ESRC. EPSRC, AHRC). the EU IST progyranimies and dhe NSE His publications include numerous journal artieles and ery, auctions, conimamd and control, and museums and g pt books including ios! recently fhe Dynamix of clusions Suciul futenetion wand the Sake wf Fane Ara ad Asnignes (Cambridge) and Videw ia Qa Reseaneh Analysing Seal butenuctiow die Everyday Lik {with Hindiansh and Lof'Sage). He is co-editor of the aronegraph se (Cambridge) js Learning it Domg Jana Holsanowa 1 Associate Prolissar in Cognitive Sere winonment Cognition, Commmunication, and Le oan Senior Researcher it Linnaeus ing’ ac Land University, Sweden, Her { images, cognition and visnal communication, Jana wes eye isual shinking (mental images) anel tteraction with 4 Cugitone 2008), Mitts aad Fate be arionns Meier 2010) and Methods ona nacional Communication Association and Chairman of the Swedish Braille Authority at the Swedish Ag fig, Cn he innveal Reseavel 2012, Special su ay Boraeess Lat of Visita of the Visual communication alivision. Ins | Sage). Jana is Chair eleet ency for Accessble Media David Howes is Professor of Anthropology at Goncondna Univer is he Director of the Centre for Sensory Studies, He Ue co-author of Hays af Sensing: Und erstanating she ng te (2003). and editor of The Hinettes af Sensory: experience Fe Chasen}, author of Sensizal Re noes fn Saviery (213, with Constan ass Bay Sovses far Culture anit [IVT Chess Cile works, He Anizon: pology. prychalogical andurapolog af the Somes (2005), anoayy other utied out fied research on the cule atid Papurs New Guinea, His other re as a) life af the seases in Argentina, rests revolve aound iedical anthr- ss-suyral forispructesce, Jonas Ivarsson 3s Professor in the Deparustent of Edveaners, Communication and Learning, and, Research Dean at the Faculty of Education, University of Gochenburg, He hasa background in copmatixe science anal conmuumnicanon stuslies, with « PhD inn echicanonal science. His general research interests concern the tole of representational keclinologies in the preseneition. asia sion and develo 1c, Ivarsson fas seudiedd work, ment of specialized knowledge and compe carried out in the 4 reas of sciesice ealucation. taciology. and archutesture Adam Jaworski is Professor at the School of English and Associate Dean (Research) at che resis incl nees of specel and svrating asital arts, He co-eds the book series Oxford Studicy iy Suculfuguisates (Ostont University: Press, with Nikolas Coupland} Facul of Arts, The Universty at Hong Kong, His research i discourses of robibty and globalization, nonverbal conmuinication and affoed Rodney H. Jones is Associate Head of the Departnent of English at City University of Hong Kong, For miore than twenty years be has been anvolved in the study: of discourse andl the various metheuls of discourse analysis 10 contexts av lives as sexual health, drug abuse, come pure meat role an the dev commivnication, professional writing and extreme sports. He has taken a major analysis, opment of the theory anel micthods of medkinted scoursCONTAIBUTORS 8. Knox iy Leetnrer im the Department of Linguistics, Macquarie University, Sydney, Jo Austrailia, His research interests include multimodality; media discour age medicare earch in journals such as Visa ton, and systemi¢ functional theory, He has published x Couimnaization, Social Serniotis, Déxourse wed Camauuieat inn and Distanie Fas Gunther Kress 1s Prolissor of Scrniatics and Editcation at the Inscinite of Education, His interests ane in meaning-mnkiny: uso aistssone to use any insights gained to develop an ape theory of learn znd commmnication ist contemporary exteironmicats, He has ancl ape lows to commnue work ind ‘of valuations and, at the same 4 ‘eloping social semiotie theory intics (LYS8); He The Grams of muldimadal communication. Some ot his books axe Social Rethinking te Paths to Literacy (1996): Reaaling (1996/2006) Maclin (201): Literacy ts rhe New Media elec 2003); Muitiaroditys Contemporary Communication 201M Heri of Graphic Desig lal Discourse: Phe Mader and Media of Coarcaiporary Consus Seraiitc Apyprowcte ao Lesley Lancaster i 4 Senior Lecturer at Manchester Metropolitan Univenuty. She current rches appked lingwistics a¢ undergraduate, postgraduate, and doctoral lew research interests are in how very young children understand symbole ferns and systems, in embodied meaning, and int the development of muliimodal systens of description alysis, She has recently been involved in the Communicating Maccers 3-3 (IFES! and Gumnatcization of Barly Mark Making (ESRC) projects. She hus contrabuted articles to the forernal of Barly Childhood Literacy, anal to 2 nunttver of publications including, Medteinnd Literaey and The Hanedback of Barly Chililwod Litera. Kevin M. Leander is Awociate Professor in Langu if Culture at Vanderbilt University, Leande’s special arcas of interest inch wgraphy: practice theo room interaction, aid mnleinodabty. Leander Bas puab= anc fi rhe ‘aching of Englisfi, 2thos, Reding <. With nel Practice, pub ries of identity, digital Titeracies, elas lished articles in Cognition and Jasmtion, Re Resear: Quarterly, Disco iatgaret Sheehy, he has co-edited! che book 5) me Processes, Journal af Literacy Research, and other ve Ished! by Peter Lang, Leander’ recent research projects include F any examination -actvees (avith Suncra Poxtzanesi andl Mariette de Han} spatial design asd learning envitoaments with Rogers Hall and Embodied Math, sult institutional praject on how enbodied! activities in social spaces at diferent seales timetion in the Learning of maths Jay Lemke iy Senior Research Scientist in the Laboricory of Comparative Human Cognition ALCHC) anil adjunct Fall Profesor of Communication ac the University of California, San bi aitiiated with the Ph.D. progearuy in Seience Edveation, L Langnage and Culture, and Profesor and Executse Olliver of the: 0. He wis foreneily Profesor in the School of Education at the University of Machigan, -arninig Technology. anal Liveracy h.1 Progrant an Urban Education ac the Gradnate Center of dhe City Universiey of New York, where he is Professor Finentus, Jay ts the author of alking Scienre: Langage, Leanging, aud Fibacs and ‘Teta Politi Discounse and Sock of discourse linguistics, multimedia semiotics, social theary, and science andl litertey edicae tion, Hoy current research facts 1s on the integration af feeling and ncaning, in multimedia activity syscoms and Design Research Dynamies as well as oF over 108 other scholarly pubheasons sn the fellsCONTRIBUTORS: Jonas Linderoth isan associate profissor at tie Departement af Labacatian, and Learning, Uni mnnuiication sity of Gothenburg, He has a background as artist and art teacher, Mest of iy research concerns games and gaming as eultral practices, Jonas is the author of several articles for journal sitel as Jauraal of g ming ard viata seonkts an ey ENGR CFiansations of j] He ivabo 4 contrisucor to the volume: Sports Viegas publshed by Routledge. Currently most of his work concerns neues about gante perception the Dial Games Reseaile ssa from un ecological perspective Paul Laff is Profissor of Organiz.tions and Technulogy at the Department of Management, King’s College London. His research involves the detailed analysis of work and in ction and dzvang upon video recordings of everyday hnnssit consict, With his colleagues in the Work, Interaction and Tecliilogy Research Centre, he Tas ondertaken studies in ad verse varsety of settings including contal rooms, news and broadcasting. healtheare, museums, gaillertes and science centres and within designs, architeceure a construction. Over the past ey as, Paul Lai hay beer particularly concerned wath the use oF apparently: mundane abjects, specitically paper documents, and how these support what are offen very complex: ssork practices, This r eh. arnt related stsdies, hay heett Feportedt in numeras articles in the Hickls oF CSCW, HICH, Requirements Engineering, Studies of work practices aad ubiqe tntous and mtobile systenis, Paul Lalas co-author with Chistian Heath of Technology itt Acca, published by Cannbridge University Press David Machin is Profesor of Media acul Communication Studies at Orcbry Unnversity, He this published cight books, including hetrodsation s Adulsinsodul Analysis (20007], Gobel Media Disconrse (2007), cheulysing Papirlar Music QU1G), Tie Langage uf Crinse and Deviewe 2012) and ‘Tie Lineage of Har Mormenty 201.31, a number of edited collections and aver sixty journal articles and book chapters maialy around Lopes relating to linguistic analysis of media and insti tional texts and also around systemiaue analyses of both visual design and sound as connmunication, He is Co-Rditor of the journal Suvi Seanetie, Diane Mavers 2 Visiting Research Associate at che Institste of Ldveation, Unisersity of London, Taking a social semiotic approach, her rescarch investigates. a va oy of ways im sn the everyday + tw pechgogic int whieh carly years and primary childben interpret and produce aneani chisstoor how what they draw and serice, anal say and do, re sronnd curricular engities, learning: materials ancl digital eechnol hhave investagated conmiumication around class visualizer (9 dh af video for recording Ute dotial of what goes an usd for tn 4. More rovent projects display technology’. ried! analysis has led to an crest in methods of transcribing the malimodality of social interaction. Dian academic inal of Barty Childhood Literary, Hoe book Childro’s Daag aad [iting Me Reanantalte i he Craemnastatle was published by Roueledge (New York} in 2011 journal articles include contribusions 10 dhe fi Denise Newfield is an Assaciate Profissor in English and Education at dhe University of the Witwatersrand, South Attica, She applies anultimedal social semiaties co her work int wacker education, tieratore, lnguage and litemey pedagogy and currscilas and popitlar culture. Hee research focuses on the teasformatye role multimodal pedagsizies can play an chsstooms, especially these of disadvantage, and during, dines of political transition. Her PhD study Wniversity of Lonskan} ts called “Tiansmodul semiosis in chasoons!, She has published in theCONTRIBUTORS Ids of English edeation, enual hteracy, multiliteracies and moltimedality: She co-edited a special edition of Fetish Snudies in fic, emticled “English Education in Attica’ 2006). whacls contains her invard- winning article. Mobilising Amongst her receist attieles sire “Multisnodslity, social justice and becoming, a really Soutle and medalising poctry ma Soweto classroony African democracy’ and 'Multimodalicy ancl children pat currently comeing nod.lity in South Ativan education ricipanan in classrooms’. She is book on mum Sigrid Norris Director ofthe Multimodal Research Centre in the School of Communication Studies at Auckland Unversity of Technology in New Zealand. Sigricl s che auchor of Multinadal fh i Interaction: | Meshedalogical Framework, che edivor of Multimortaity ant Puctice: nvestigarang Theory-in- Pre -Metod anid co-editor of Disease én Ar Disconse Anaiysis. She is ako the edstor of the journal Melsinsedal Come Mert dn dleeaston: braroduc sion Analysis and cialysing Miki handing Menge tions, Siggrid's anain research interests are the Gmtefaction analysis ane snultisuodal entity production ical ans methodological development of rmulcinodal Kay L. O° Hallor School of Edneation at Curia University in Perch, Western Australia, She was founding Director of the Multimodal Analysis Labs 1 Media Instiruce at the National py AAvsoetate Professer in thy and Dep University of Singapore, Ler areas of res oties, nologies and snathemuatical and selentific visualization techn y Director of the [aceractive & Dagzita rch inclade enulcinodal analysts, social seni mathematics discourse, and the development of interactive digital mecha wesh- s for mulcinvodal and socio-cultural analytics. She wees Principal fnvestigator far hurge research projects in thes areas in the Multimodal Analysis Lab at the National University of Singapore and las published widely: in chese fields. She is expanding this research at Cx tin Cniversiey Kate Pabl is 2 Reader in Literacies in Education at the University of Sheitield. She is the authos, with Jomtiler Rowsell, of clsifuca es: Livery Object Tells @ Story 2010} and Seite Literacy Sties tw Fhe Classroom, 2nd Baition ollaberatve ethnogea n partnership with eommiumty organyyations, artists, museums and cultaral Literary cate Faduestion: Uieberctarting oh (2012). She is particulaely intonesced pic research in commun nity contests with 4 focus on che cultural making ef sive engagement, She works int cos awit terest inn the co-production of research, She ehrects the EdD itt Li Langs ty of Sheffield and teaches on the MA in Workin ci uty Director of the Research Exe and collaborative research with local Communit macy and with, ange. promoting, ened eat the Unve mnmmunicies, anid is 1 Karola Pitseh received her PAID in Langgunsties from Biclefeld University, Germany (2006). She has held since pasitions ay postcoetozal researcher in EL-projects at Kings College London, UK (2005-20085) aed the Research Instituce for Cognition and Robotics, Bielefeld University (20008 2011), fa 2011, Karola ind Principal Investigator of projects within the € Technology’ and the CRC 673 ‘Ahgnmenc in Communication’. $ the research group ‘Interactional Lingnisues & Human-Robor ecatea Dilthey Fellow (Volkswagen Foundation} naire of Bs nee *Coggative Interiction 1¢e 2012 ,she has beaded reraction’ at Bielefeld University: Karol’ researel focuses on mulginodal communication and hnguistics i eve od settings. She investigates the pruetices by which coparticipancs coordinate andl orzanize their courses of actions and the finguistie and nyday, professional and technologically medGONTRIBUTORS bovied revourves lop Wy te do so, She also drwy on Conversion Analysis 0 inform — tn iutercisciphinary collabor: robot/agent intersetion ton the modelling and evaluation of huntan— Jennifer Rowsell i Professor and Canad Research Chair an Mubtiiteracies at Brack Univenity’ Faculty of Educatien where she alireets the Cent it al reseurelt and lor Mustaliteracies, She has co-written, weritton several books in the mouliliomens, Her cure of New Litenicy Studies imlamodiibryccol ly interests intelacle children’s aL and isnenersive worlds; ddopeing and applyange nmultnrodal epistemsologies with adolescents td teonagerssanl ecologseal anining: everyday hieeracy practices, She iy che anthor of Hing wit Motnclality: Faring ia a Digital cle ouitleslee work ant con miinities ¢ Roger Salj6 is profesor of psychology of ethuction at the Universay of Gothenburg, Previously he Tas been profesor of coniniusication research at the University of Linkaping, ate fre has held visiings prafissorships at Utrecht, University af California Sant Diego, Oslo, Gitwingen and otter universities. His research interests concern learning, wteriction and development in a soew-cultural perspective. Recently he has worked with nsues af how Jearning and developmentitre conbngent en the anes oF reprsesitatiomal {and other} tock nologies, He is also the director of The Lumacus Cenue for Research on Learning, Inceraction and Mediated Communication in Cantestporary Sovicty {Lins}, whiel is a contte of escetlence Handed by the Swedish Reseach Council: Recent publications inclde Literacy, digntal literacy and epistemie practices: The co-evolution of hybrid nti external memory systems’, Nenie Jornal of Digital Hitemey QOL 2) and 'Analysinge and pre venting school failure: Exploringe che role of multi-profesaoaahty re: pupil health veal Jour of Halacutfonsal Resean de (yeah E, Eljorne, 20133 and od with issues of ob rreironnrents of resource develope Ron Scollon was a linguistic anthwpologist whose professional work c world concern food in the workd system, daa opment stakeholders. sve antong, local commutes anc local secwrity resource de ment and way, zie Wong Seollon is 1 linguistic aneliropol st with degrees sn poyehology and bngeisties who has foc dan lo ing, the aaerosociological perspectives of political scientists and seenomists in die nnero-intetactian of humus boches with objects in the physical « iment ot various timescales iron Maree Stenglin is an Honorary Associate of the University of Sydney andl che University af Teelinology Sydney. Her research interests include nul umodahey, social semiotics and Titertey: Her doctoral chesis, Packaging Curiosities: Zowurds a Graannnar af three-dimensienal spive, developed & metalimetionally divessitied theary af 3D space. Inthe past 1D years she tas applied the space graetnatar to the analysis ofare and history eshibitions in aw range of cultural mscitutions as well as homes, restaurants and a winery. She is currently extending the theoretical toals of Bindi spaces for women. cand Bonding to che analysis of hospital birt Hartmut Stéek1 is fill professor af English and Applied Linguistics at Salzburg Univeraty Austrut, His main reseuch areas ane ans jotcs, text hnguisties/syhisties, praggnatics andmuliimiodal communicagon. A recurn with « pluralistic theory of the multimodal text and effective analytical methols for uncover ing als structures anid atyles, He iy parcicularly interested iv the linkage of language wid nage in modern media, typoxtaphy and an aesthetic appreciation of advertising, His latest edited noxtalitit (2011 Le is currcnely edising the handhook Spray ine snitimadales Koutext @014, de Gruytes) an the 21 volumies are: Bildlingwistile 2011, Brich Schmidt, Medieardworiew und Mul Herbert vou Halen} and HHrbuing — Keine Kenst!? (20012, Winer), Spc Brian Street is Professor Emeritus of Language in Exlucation at Kings Gollege, London University and Visiting Profesor of Education in the Graduate School of Education, University of Pennyyhania, He hay a commiment to knking edhnographie-style the cultural dimension of lan; und literacy with contemporary prictiee in eelueation Kken anthropological fiekt research ind been consultant to progerts an these Bekls 1 conntnes ef both the North and Sour Nepal. §. Attica, India, USA, UK}, Recent publications include: Sureet, B And Lette 2007 Routle aes ¢ aud Litewwy Resear, NaGonal Gonference on Research in Langu Sollege Cohumbia, 4194 in developnignt, Over the past 25 yours his his uae nA rey’ ae Adv Longon: ant Heath, S.B-and Servet. B. fet Resource Ho i Lehinaphyselppnmaites te Late ace tne Literacy. Teachers € Sabine Tan lectzes in the Department of English & hi rature at the Natorial scmnent University in Singapore. ch intersts mielude inulimodal discourse analysts. sock semiotics. and viswal cont nication, She is pardculely interested in the application of mulsidisciplinary perspectives University of Singapore ard the Singapore Institute of Many Her nese wich social semiotic theory so the analysis of institution, discourses involving wadisional such as buses. news micdiated ou the iacernet, corporate advertisements und other emergent aultimosal discourse ial Analysis Lab ac the Nasional disciphnary projects involving the cewelop in print and television, corporate web-p. hb scam at the Multi genres, As st member of the resea Uns yy oF Sisigupene she parccipated sn ante nie of interictive aml callahorative sofware for mulimextal analysis for research and odnestion purposes, Crispin Thurlow is Professor (Language and Communication) sn the School of Interinciphnary Arts & Sciences, Universigy of Washington, Bothell, LISA. His resea uunines how fan sume andl other semiotic modes work to praduee social dierence /anequatlicy it wo key domains of con His books include cmporiry hie: the new anedia anel global mobil Huon Disconse (Pal Dione (Oxford) rave}, wrigten with Achin Jaworski, ans the exited collection 1} Len Unsworth » Professor it Edncanos at Griffitt University, Brisbar ny Leu A Hache (Macmillan, 1993}, Rescuninyy ba Conant (Cantinanm, 2000}, Tar sing DOU) Fie Thomas, Alyson Simpson and Jenny Asha) tralia, His books include Bite ae fi Sido aed “iamienltaas (Open University re fon Childs anud Classroum Literacy Leaning (Routledge, 2006) (wich ronacies Aeros Press, Any clang Cikdve’s Literature with byformaation vad Gonmarntoation Tichnotegies (McGraw Hill Open University Press 20005), New Liveries Crealm Malin etal Semone mn, 20008} anel Recdine the Hy J Porgpecsives (Cantanuuen, 2008}, Muli sual Nuraatives (Equine, 203)