The International

JOURNAL

of

the ARTS
IN SOCIETY

Volume 4

Towards a Model of Intersemiotic Translation

Daniella Aguiar and Joao Queiroz

www.arts-journal.com

Towards a Model of Intersemiotic Translation

Daniella Aguiar, Federal University of Bahia, Bahia, Brazil Joao Queiroz, Federal University of Bahia, Bahia, Brazil
Abstract: The phenomenon of intersemiotic translation (IT) represents a special creative domain of language procedures and practices. Hypothetically it involves a radical change of habits of interpretation and new forms of sign manipulation. Despite its theoretical relevance, and in spite of the frequence in which it is practiced, the phenomenon remains virtually unexplored in terms of conceptual modeling, especially from a semiotic perspective. Indeed, the phenomenon of IT is difficult to characterize and compare with analogous phenomena (inter-linguistic translation). As it involves systems of rather distinct nature, its analysis creates additional difficulties in any theoretical approach compromised with the logic of semiotic processes. This work proposes an approach based on Charles S. Peirces model of sign process, to provide a preliminary conceptual framework to the phenomena, emphasizing hierarchical properties and aspects. One of the consequences of our approach is the importance ascribed to the materiality and dynamic involved in IT, prioritizing the semiotic properties of hierarchical relations between the source and the target signs. Keywords: Intersemiotic Translation, Semiosis, Intersemiosis, C.S. Peirce

Introduction

HE PHENOMENON OF intersemiotic translation represents a special creative domain of language procedures and practices. It involves a radical change of habits of interpretation and new forms of sign manipulation. The phenomenon was defined by Roman Jakobson (1971 [1959]: 261) as transmutation of signs an interpretation of verbal signs by means of signs of non verbal sign systems. Despite its theoretical relevance, and in spite of the frequence in which it is practiced, the phenomenon remains virtually unexplored in terms of conceptual modeling, especially from a semiotic perspective. For Gorle (2007: 347), creative sign transmutation involves the reconstruction of an artwork into a distinct semiotic system, creating a sophisticated collection of interconnected signs (see also Plaza, 1987). The modalities of translation proposed by Jakobson, according to Gorle (1994: 147-168, 1997: 240-244, 2005: 34-35), are related to the notion of translation in an extra-linguistic horizon. This leads us to a general acceptance of translations of texts of all kinds, taking away from the term its exclusive allusion to linguistic material (Petrilli, 2003). The processes are observed in several systems and include: literature and cinema (Herman Melville > John Huston; William Shakespeare > Orson Welles; Vladimir Nabokov > Stanley Kubrick), literature and comics (Herman Melville > Bill Sienkiewicz), poetry and dance (Thephile Gautier > Michel Fokine; Stphane Mallarm > Nijinsky). Many other examples could be mentioned in several sign systems as theater, sculpture, music, painting, video, and so on.

The International Journal of the Arts in Society Volume 4, 2009, http://www.arts-journal.com, ISSN 1833-1866
Common Ground, Daniella Aguiar, Joao Queiroz, All Rights Reserved, Permissions: cg-support@commongroundpublishing.com

THE INTERNATIONAL JOURNAL OF THE ARTS IN SOCIETY

The main methodological difficulty is related to the comparison between radically different semiotic systems. As we know, a translation is not committed only to semantics (Campos, 1972), to which meaningful dimensions of semiotic processes are usually attributed. It seems theoretically natural to describe an interlinguistic translation by establishing direct correlations between comparable semiotic layers of organization morphological-morphological, phonetic-phonetic, rhythmic-rhythmic (see Jakobson & Pomorska, 1985). However, an intersemiotic translation does not exhibit the same principle of corresponding layers (see Plaza, 1987). Here we propose an approach based on Charles S. Peirces model of sign process, to provide a preliminary conceptual framework to the phenomena emphasizing hierarchical properties and aspects.

Intersemiotic Translation Model

Our approach is based on two basic premisses: (i) translation is fundamentally a semiotic operation process (semiosis) (Hodgson, 2007; Petrilli, 2003; Stecconi, 1999; Plaza, 1987); (ii) semiosis is a multi-layered process (Queiroz & El-Hani, 2006, 2004).

Translation as Semiosis or the Action of Sign

Peirce defined semiosis as an irreducible triadic relation between a Sign, its Object and its Interpretant we will hereafter refer to this sign triad as S, O, and I (CP 2.171, CP 2.274).1 According to Peirce, any description of semiosis involves a relation constituted by three irreducibly connected terms, which are its minimal constitutive elements (MS 318:81; CP 2.242). In his words: A Sign may be defined as a Medium for the communication of a Form. [...] As a medium, the Sign is essentially in a triadic relation, to its Object which determines it, and to its Interpretant which it determines. [...] That which is communicated from the Object through the Sign to the Interpretant is a Form; that is to say, it is nothing like an existent, but is a power, is the fact that something would happen under certain conditions (MS 793:1-3; EP2, p. 544, n. 22). According to De Tienne (2003), Peirce is emphatic when he observes that form is nothing like a thing. It is something that is incorporated into the Object (EP 2.544, n. 22) as a habit, an action rule (CP 5.397, CP 2.643), a disposition (CP 5.495, CP 2.170), or a real potential (EP 2.388). It is important to observe that the form communicated from the Object to an Interpretant through the Sign is not the particular form (shape) of the Object, but a regularity, a habit, that allows the interpretative system to interpret it as an entity class, of processes or phenomena, to respond properly. There are some important consequences from Peirces conceptualization of sign process. This model of semiotic operation describes a phenomenon as essentially triadic, dynamic, interpreter-dependent, and materially extended (embodied) (see Queiroz & Merrell, 2009,
1

We shall follow the practice of citing from the Collected Papers of Charles Sanders Peirce (Peirce, 1931-35, 1958) by volume number and paragraph number, preceded by CP; the Essential Peirce by volume number and page number, preceded by EP. References to the microfilm edition of Peirce's papers (Harvard University) will be indicated by MS, followed by the manuscript number.

DANIELLA AGUIAR, JOAO QUEIROZ

2006). Emphatically, a semiotic process is NOT a bi-lateral relation, bi- univocal, between two entities.

Translation as a Multi-layered Process

Another presupposition is related to what we call layer of organization or description level: if semiotic processes are multi-layered (multi-level systems), as we have argued in other papers (see Queiroz & El-Hani, 2006, 2004), a translation is a relation between multi-layered processes, and an intersemiotic translation (IT) can be described as a multi-hierarchical process of relation between semi-independent layers of descriptions. The layers of organization do not act independently but they are autonomous in functional and descriptive terms. Basically they are coordinated in terms of mutual constraints (see Salthe, 1987). In this sense, although we can describe the scenic dance space, for instance, without reference to movement dynamic morphology, in fact they are mutually constraining each other in a dance choreography. IT operates on different description levels, selecting relevant aspects from the source and recreating them into the target (see Eco, 2007: 59). In this way, certain layers have their relevant properties selected and translated into new materials and processes. For example, from literature to dance, linguistic layers (rhythmic, prosodic, syntactic, or psychological ambience) are translated into dynamic of movement, organization of space, light design, costumes, scenography, etc. Notably, a mapping of correlations cannot be easily established between layers of different nature (different semiotic systems). If a translation from a literary work into a dance choreography results in very different materials and structures, how is it possible to compare semiotics source and target? In any case, possibilities of conceptual mapping (probably non-univocal) between different systems and levels should be provided.

Triadic Translation Model

An important consequence related to our premises, as was specified above, indicates that a translation is a triadic (S-O-I) relation, not a dyadic-bilateral one. According to Eco (2007: 17), to translate means to understand the internal system of a certain language, the structure of a given text in this language and to construct a double of the textual system which, submitted to a certain discreetness, could produce analogous effects on the reader, on the semantic and syntactic, as well at the stylistic, metrical, phono-symbolic levels, and also upon the passional effects to which the source text tends. Formulating this better; there are two competing model possibilities: The sign is the semiotic source (translated work). The object of the translated sign is the object of the semiotic-source and the interpretant (produced effect) is the semiotic target (translator sign). (Figure 1).

THE INTERNATIONAL JOURNAL OF THE ARTS IN SOCIETY

Figure 1: Triadic Relation in which the Sign is the Translated Work, the Object of the Sign is the Object of the Work, and the Interpretant is the Translator Sign The sign is the semiotic-target (translator sign). The object of the sign is the semiotic source (translated work) and the interpretant is the effect produced on the interpreter (interpretant). (Figure 2).

Figure 2: Triadic Relation in which the Sign is the Target, the Object of the Sign is the Translated Work, and the Interpretant is the Interpreter In an effort to a better explanation of model possibilities, we will exemplify them with the Spider-Mans comic-film translation. According to the first possibility, we could replace the sign-object-interpretant triad with the comic book - comic book object - film relation. In this case, the sign is the Spider-Man comic book; the object is the Spider-Man comic book object that, in a simplified explanation, should be the overcoming of a spider-man hybrid hero; and the interpretant, the effect, is a Spider-Man film. According to the second model, the sign-object-interpretant triad could be replaced with the film - comic book - effect on the

DANIELLA AGUIAR, JOAO QUEIROZ

audience. Hence, the sign is a Spider-Man film; the object is the Spider-Man comic book, and the interpretant is the effect of the film on the audience. According to the process described above, the form communicated from the object to the effect (interpretant), produced by means of the sign, is different in each version. How can these differences be helpful? In further works, we should speculate about how those alternatives provide insights about the phenomenon examined. At this point, we insert the hierarchical schema to the triadic process considering the first alternative model. According to this version, a (semiotic) relation of translation between multi-structured processes is established in terms of irreductible triadic relation. The interpretant (costume, rhythm, movement) is determined by the object (object of semiotic target), through the sign (history, pragma, synthax) (Figure 3):

Figure 3: Translation Model from Literature to Dance that Includes the Notion of Layer of Organization or Description Level The second version provides us another perspective of the phenomenon, with focus on the reader. Including the hierarchical relations schema to this version, we will have the interpretant (the effect on the interpreter) determined by the object (semiotic-source), through the sign (semiotic-target) (Figure 4).

THE INTERNATIONAL JOURNAL OF THE ARTS IN SOCIETY

Figure 4: Translation Model (II) from Literature to Dance that Includes the Notion of Layer of Organization or Description Level If the translation produces on the interpreter an analogous effect to the one produced by the semiotic-source, we must conclude that the translator is a sign of similarity (icon) of the semiotic-source. In another paper, we will explore in detail a typology that includes sub-divisions of icons. For Stjernfelt (2007), the icon involves a direct presentation of the qualities that belong to its object. As indicated by many authors, there is a strengthening of the concrete aspects of semiotic process, involving the materiality of sign. Briefly, when manipulated, the icon reveals the qualities of its object (see Farias & Queiroz, 2006). This property has a special relevance when we consider creative translation cases. Augusto & Haroldo de Campos have proposed different expressions to name a creative translation practice, attending to the materiality of the sign: transcreation (Campos 1972: 109; 1986: 7), creative transposition (Campos 1972: 110), and re-imagination (Campos 1972: 121).2
2

According to this practice, it could be possible to talk about critic-creative intersemiotic translation. In contrast to this, we could determine another modality as illustrative transposition. It tends to privilege the displacement of components of the translated work. In the case of transpositions the translated object is what the sign refers to, the sign object. Critical-creative translations use to involve re-creation cases of formal procedures, of formal structures, or of composition strategies identified as characteristics of a period, or style. On the other

DANIELLA AGUIAR, JOAO QUEIROZ

Discussion
IT represents a domain of new language processes because it tends to produce different habits of sign manipulation and interpretation. Beyond that, it involves a pragmatic view of the processes resulting from direct comparison of very different semiotic systems. Nevertheless, there are small amounts of theoretical works systematically produced about the phenomenon. They are mainly descriptive, they lack explicative models, and they are dissociated from results produced in the area of general semiotics and translation studies. Indeed, the phenomenon of IT is difficult to characterize and compare with analogous phenomena (e.g. inter-linguistic translation). As it involves systems of rather distinct nature, its analysis creates additional difficulties in any theoretical approach compromised with the logic of semiotic processes. An approach of this phenomenon can be empowered with speculations on interesting problems, from general contexts in which the translations are conceived, to historical affiliation of translations, to technico-scientific, aesthetic and philosophical ambiences, and so on. This is important. But, in these approaches, the most crucial aspect is missing: the logic involved in the translation process, the semiotic operations at play. One of the consequences of our approach is the importance ascribed to the materiality and dynamic involved in IT, prioritizing the semiotic properties of hierarchical relations between the source and the target signs. The partial results exhibited constitute a preliminary attempt toward modeling IT.

Acknowledgments
We thank Floyd Merrell and Lucia Naser for helpful comments during the preparation of this manuscript.

References
Bressane, Julio. Alguns. Rio de Janeiro: Imago, 1996. Campos, Haroldo. A arte no horizonte do provvel. So Paulo: Perspectiva, 1972. Campos, Augusto. e.e. cummings, 40 poemas. So Paulo: Editora Brasiliense, 1986. De Tienne, Andre. Learning qua semiosis. S.E.E.D. 3 (2003): 37-53. Dusi, Nicola, and Siri Nergaard (eds). Sulla traduzione intersemiotica - Versus 85/87 (2000). Eco, Umberto. Quase a Mesma Coisa . Rio de Janeiro: Record, 2007. Farias, Priscila and Joo Queiroz. Images, diagrams, and metaphors: hypoicons in the context of Peirces sixty-six fold classification of signs. Semiotica 162 (1/4) (2006): 287-308. Gorle, Dinda L. Semiotics and the Problem of Translation, With Special Reference to the Semiotics of Charles S. Peirce. Amsterdam/Atlanta: Rodopi, 1994. . Intercode translation: Words and music in opera. Target 9 (2) (1997): 235-270. . Singing on the breath of God. In Song and Significance: Virtues and Vices of Vocal Translation , ed. Dinda L. Gorle. Amsterdam/New York: Rodopi, 2005. . Bending back and breaking. Symploke 15 (1-2). (2007): 341-352. Hodgson, Robert. Semiotics and Bible translation. Semiotica 163 (1/4) (2007): 37-53. Jakobson, Roman. On linguistic aspects of translation. In The Translation Studies Reader , ed. Lawrence Venuti. London/New York: Routledge, 2000 (1959).
hand, structures refer more properly to the manipulated semiotic components. These modalities would be developed in further works.

THE INTERNATIONAL JOURNAL OF THE ARTS IN SOCIETY

Jakobson, Roman, and Kristina Pomorska. Dilogos . So Paulo: Cultrix, 1985. Peirce, Charles S. The Essential Peirce, Selected Philosophical Writings . (Vol. 1 ed. by N. Houser & C. Kloesel; Vol 2 ed. by the Peirce Edition Project). Bloomington and Indianapolis: Indiana University Press, EP1 1992, EP2 1998. (Quoted as EP). Peirce, Charles S. The Collected Papers of Charles Sanders Peirce . Electronic edition reproducing Vols. IVI [C. Hartshorne & P. Weiss (eds.), Cambridge: Harvard University Press, 19311935]; Vols. VIIVIII [A. W. Burks (ed.), same publisher, 1958]. Charlottesville: Intelex Corporation, 19311935. (Quoted as CP). Peirce, Charles S. Annotated Catalogue the Papers of Charles S. Peirce . (ed.) R.S. Robin. Massachusetts: The University of Massachusetts Press, 1967. (Quoted as MS). Petrilli, Susan. Translation and Semisis. In Translation Translation , ed. Susan Petrilli. Amsterdam/New York: Rodopi, 2003. Plaza, Julio. Traduo Intersemitica. So Paulo: Perspectiva, 1987. Queiroz, Joo, and Charbel El-Hani. Towards a multi-level approach to the emergence of semiosis. Technical Report DCA-FEEC 04 (07) (2004): 1-21. . Semiosis as an emergent process. Transaction of C.S.Peirce Society 42 (1) (2006): 78-116. Queiroz, Joo, and Floyd Merrell. Semiosis and pragmatism: toward a dynamic concept of meaning. Sign System Studies 34 (1) (2006): 37-66. Queiroz, Joo, and Floyd Merrell. On Peirces pragmatic notion of semiosis a contribution for the design of meaning machines. Minds & Machines 19 (2009): 129-143. Salthe, Stanley N. Evolving Hierarchical Systems, Their Structure and Representation . New York: Columbia University Press, 1985. Stecconi, Ubaldo. Peirces semiotics for translation. In Fidelity and Translation: Communicating the Bible in New Media , ed. Paul A. Soukup and Robert Hodgson. New York/Franklin: American Bible Society/Sheed and Ward, 1999. Stjernfelt, Frederik. Diagrammatology, An Investigation on the Borderlines of Phenomenology, Ontology and Semiotics . Dordrecht: Springer, 2007. Torop, Peeter. Translation as translating as culture. Sign System Studies 30 (2) (2002): 593-605. . Methodological remarks on the study of translation and translating. Semiotica 163 (1/4) (2007): 347-364.

About the Authors

Daniella Aguiar Daniella Aguiar obtained her bachelor degree in Dance from Anhembi Morumbi University, So Paulo, Brasil, in 2004; master degree in Dance from Graduate Studies Program in Dance, from Federal University of Bahia (UFBA), Brasil, in 2008. Her main research interests are in Dance Theory, Semiotics, Cognitive Science, and Cognitive Aesthetics. Her main research topics are (i) dance techniques, (ii) cognitive artifacts, (iii) intersemiosis and (iv) intersemiotic translation. Awards include Klauss Vianna Dance Prize by Funarte (National Art Foundation Brazil), 2007, for a theoretic research project about intersemiotic translation in dance; Yanka Rudzka Prize by Funceb (Bahias State Art Foundation), 2007, for dance choreography translation from Gertrude Steins prose. Joao Queiroz Joo Queiroz is a post-doc researcher at the Graduate Studies Program on History, Philosophy, and Science Teaching, Institute of Biology, Federal University of Bahia (UFBA/UEFS), Brazil, director of the Group for Research on Artificial Cognition (UEFS) and associate re-

DANIELLA AGUIAR, JOAO QUEIROZ

searcher at the Dept. of Computer Engineering and Industrial Automation, State University of Campinas (UNICAMP). In 2002 he earned a Ph.D. in Communication and Semiotics from the Catholic University of So Paulo. From 2003 to 2006 he conducted postdoctoral research at the School of Electrical and Computer Engineering (DCA-UNICAMP). His research interests are cognitive science, cognitive aesthetics, biosemiotics, and Peirces semiotics. His recent publications include the books Gene, Information, Semiosis, with C.El-Hani and C.Emmeche, Tartu University Press, in press; Semiotics and Intelligent Systems Development, with R.Gudwin, Idea Group, 2007; Advanced Issues in Cognitive Science and Semiotics, with P.Farias, Shaker Verlag, 2006; Semiose segundo Peirce (Semiosis according to Peirce), EDUC, 2004.

EDITORS Mary Kalantzis, University of Illinois at Urbana-Champaign, USA. Bill Cope, University of Illinois at Urbana-Champaign, USA.

EDITORIAL ADVISORY BOARD Robyn Archer, Performer and Director, Paddington, Australia. Mark Bauerlein, National Endowment for the Arts, Washington, D.C., USA. Tressa Berman, BorderZone Arts, Inc., San Francisco, USA; University of Technology, Sydney, Australia; San Francisco Art Institute, San Francisco, USA. Judy Chicago, Artist and Author, New Mexico, USA. Nina Czegledy, University of Toronto, Toronto, Canada; Concordia University, Montreal, Canada. James Early, Smithsonian Institution, Washington, D.C., USA. Mehdi Faridzadeh, International Society for Iranian Culture (ISIC), New York, USA, Tehran, Iran. Jennifer Herd, Queensland College of Art, Griffith University, Brisbane, Australia. Fred Ho, Composer and Writer, New York, USA. Andrew Jacubowicz, University of Technology, Sydney, Australia. Gerald McMaster, Curator, Art Gallery of Ontario, Toronto, Canada. Mario Minichiello, Birmingham Institute of Art and Design, Birmingham, UK. Fred Myers, New York University, New York, USA. Darcy Nicholas, Porirua City Council, Porirua, New Zealand. Daniela Reimann, Institute of Media in Education, University of Education, Freiburg, Germany; University of Art and Industrial Design, Linz, Austria. Arthur Sabatini, Arizona State University, Phoenix, USA. Cima Sedigh, Sacred Heart University, Fairfield, USA. Peter Sellars, World Arts and Culture, University of California, Los Angeles, USA. Ella Shohat, New York University, New York, USA. Judy Spokes, Arts Victoria, South Melbourne, Australia. Tonel (Antonio Eligio), Artist and Art Critic, Havana, Cuba. Marianne Wagner-Simon, World Art Organization, Berlin, Germany.

Please visit the Journal website at http://www.Arts-Journal.com for further information about the Journal or to subscribe.

THE UNIVERSITY PRESS JOURNALS

Creates a space for dialogue on innovative theories and practices in the arts, and their inter-relationships with society. ISSN: 1833-1866 http://www.Arts-Journal.com

Explores the past, present and future of books, publishing, libraries, information, literacy and learning in the information society. ISSN: 1447-9567 http://www.Book-Journal.com

Examines the meaning and purpose of design while also speaking in grounded ways about the task of design and the use of designed artefacts and processes. ISSN: 1833-1874 http://www.Design-Journal.com

Provides a forum for discussion and builds a body of knowledge on the forms and dynamics of difference and diversity. ISSN: 1447-9583 http://www.Diversity-Journal.com

Maps and interprets new trends and patterns in globalisation. ISSN 1835-4432 http://www.GlobalStudiesJournal.com

Discusses the role of the humanities in contemplating the future and the human, in an era otherwise dominated by scientific, technical and economic rationalisms. ISSN: 1447-9559 http://www.Humanities-Journal.com

Sets out to foster inquiry, invite dialogue and build a body of knowledge on the nature and future of learning. ISSN: 1447-9540 http://www.Learning-Journal.com

Creates a space for discussion of the nature and future of organisations, in all their forms and manifestations. ISSN: 1447-9575 http://www.Management-Journal.com

Addresses the key question: How can the institution of the museum become more inclusive? ISSN 1835-2014 http://www.Museum-Journal.com

Discusses disciplinary and interdisciplinary approaches to knowledge creation within and across the various social sciences and between the social, natural and applied sciences. ISSN: 1833-1882 http://www.Socialsciences-Journal.com

Draws from the various fields and perspectives through which we can address fundamental questions of sustainability. ISSN: 1832-2077 http://www.Sustainability-Journal.com

Focuses on a range of critically important themes in the various fields that address the complex and subtle relationships between technology, knowledge and society. ISSN: 1832-3669 http://www.Technology-Journal.com

Investigates the affordances for learning in the digital media, in school and throughout everyday life. ISSN 1835-2030 http://www.ULJournal.com

Explores the meaning and purpose of the academy in times of striking social transformation. ISSN 1835-2030 http://www.Universities-Journal.com

FOR SUBSCRIPTION INFORMATION, PLEASE CONTACT subscriptions@commonground.com.au