Hand gesture structure and functions Hand gesture structure and functions during conversation: conceptual problems and

developments.

Marino Bonaiuto*, Fridanna Maricchiolo*, Augusto Gnisci**

* Dipartimento di Psicologia dei Processi di Sviluppo e Socializzazione, Universit degli Studi di Roma La Sapienza ** Dipartimento di Psicologia, Seconda Universit degli Studi di Napoli

Abstract

Through a review of the research groups programme on hand gestures, some of the main conceptual issues (structure and function) and pertinent research developments will be highlighted. Priority goes to a universally shared category system to describe the phenomenon, taking into consideration traditional scientific principles (such as reliability); moreover, given the nature of the phenomena studied, multi-media methods for presenting the analytic tools used and the results of research should be created to improve use and diffusion of the taxonomy system. Another priority is the study of the functions of gesture. Research should not be limited, as already done in the literature, to qualitative studies of verbal and gesture co-occurrence and sequence: rather, it should aim at quantitative studies providing statistical evidence for such associations. In order to pursue these aims, cross-cultural analyses as well as the study of physiological correlates of both person and interlocutor are crucial.

1. Introduction

When people talk, they move their hands and their arms, to a certain degree. These movements seem to have some type of linkage with concurrent speech. The aim of the studies presented here is to try to understand such gesture-speech relation. This paper synthetically presents the results of some researches carried out in the last sixseven years and currently still in progress on hand gesture occurring during conversation in order to mark the importance of gesture study, to underline conceptual problems linked to this research field, and to outline future research lines aiming at improving scientific knowledge of hand gesture. Bonaiuto, Maricchiolo, Gnisci

Hand gesture structure and functions The starting point for a scientific study of any object is to make it recognizable and observable. Typically, in studying nonverbal communication, the definition of the structure of an examined phenomenon, and of the categories along which it can be articulated, is particularly crucial. Indeed, scientific literature on hand gesture offered different classifications of gestures (e.g., Ekman, Friesen, 1969; Kendon, 1983; McNeill, 1992), encompassing both different categories (for example, ideational, beats, cohesive, adaptors, etc.) and sub-categories. This situation is probably the result of different theoretical orientations, not always explicit, and of different epistemological approaches to scientific knowledge. Nevertheless, especially in behavioral studies and in observational methods, it is desirable to adopt and/or to adapt coding systems which are shared across the literature. A taxonomy as coding system based on the use of exhaustive and mutually exclusive categories is of fundamental importance to promote the advancement of scientific knowledge. Moreover, it is necessary to verify the reliability and the validity of any category system which should be used to code gesture (Bakeman, Gottman, 1997). In observational research, the most widely used approach to measure reliability is inter-observer agreement (Pedhazur, Pedhazur-Schmelkin, 1991) by Cohens K index (Cohen, 1960). Hand gestures, being among all nonverbal signs one of the most linked to speech in different ways, can vary across cultures; therefore another important step in gesture research is the description and the analysis of hand gestures in different cultures and in different languages. Traditionally, gestures were considered as cultural signals (De Jorio, 1832; Kendon, 1995), and their dissimilarities in different language have been studied (Efron, 1941; Goodwin, 2000; Morain, 1997, amongst others): there can be cultural differences in the use of hand gestures, both passing from a language to another one and from a culture to another one. Gesture analysis in different cultures and languages can improve the build of universal category system for gesture study. On the other hand it is important to explain specific functions of different categories of gestures. In speech, for example, some hand gestures can play an important role in conveying and communicating semantic information to the receiver (Alibali, Flevares, Goldin-Meadow, 1997; Beattie, Shovelton, 2000; Kelly, Church, 1998; Kendon, 1995); other gestures can be useful for the speaker, facilitating linguistic and syntactic production of the speech (Krauss, Chen, Chawla, 1996; Feyereisen, Havard, 1999; Rim, Shiaratura, 1991). In general in the literature gestures has been described as: a) representing some aspects of the content (objects, concepts, actions, cultural symbol, pointing entities); b) following discourse structure (relations, cohesion) and rhythm (stress, emphasis); Bonaiuto, Maricchiolo, Gnisci 2

Hand gesture structure and functions c) expressing mood or emotional states. More generally, gestures serve the function to exert social influence in a variety of contexts, such as group discussion, political oratory, public discourse, sales-customer interaction, etc. Only few researches investigated the role of gestures in persuasive speech occasioned in different situations (Argentin, Ghiglione, Dorna, 1990; Atkinson, 1984; Bull, 1986; Burgoon, Birk, Pfau, 1990; Leigh, Summers, 2002; Schultheiss, Brunstein, 2002; Whitehead, Smith, 2002): in general, some types of gesture, such as rhythmic, metaphoric or deictic gestures would make concurrent discourse more emphatic and therefore the speaker more persuasive; other types, such as adaptors, would create opposite effects. In interpersonal communication, speech is important to influence ones own interlocutors, and verbal signals make the speaker more influential and authoritative: for example, powerful and powerless speech were investigated as having opposite influential cues (Ng, Bradac, 1993). Moreover, verbal as well as nonverbal aspects are part of the same communicative process and both are important in making the speaker influential and authoritative (DePaulo, Friedman, 1998; Exline, 1972; Henley, 1977; Mehrabian, 1972;). However, to date no studies verified experimentally the effect of different gestures in coordination with speech on message and speakers evaluation and on persuasion.

1.1 Objectives With reference to such theoretical considerations the aims of the studies synthetically presented here are the following: 1) To describe the structure of hand gestures, that is the main different types and shapes of gestures: a) developing a hand gesture taxonomy to be used as a classification and category system for gesture coding, consisting of universally shared gesture categories; b) measuring and verifying its reliability through inter-observer agreement and generalisation across different cultures and languages; c) constructing multi-media tools useful for such observational method. 2) To individuate the functions of hand gestures. Some main relevant functions of communication could be described as follows: - rhetorical functions: such as rhetorical devices that are emphatic and powerful modalities to present discourse content (see, Atkinson, 1984; Edwards, Potter, 1992); - discursive functions: such as discursive markers, marking syntactic structure of discourse (see, amongst other, Fraser, 1999; Shiffrin, 1987); Bonaiuto, Maricchiolo, Gnisci 3

Hand gesture structure and functions - interactive functions: features of talk turns in conversation as marking interaction managing and social relationship with the interlocutor; - persuasive functions: convincing and influential communication. So specific objectives about the individuation of gesture functions are: a. verifying co-occurrence and sequence of gestures with discursive markers, or rhetorical devices, or types of questions; b. studying the role of hand gestures in persuasive communication in contexts of social influence (group discussion and political oratory), through correlational as well as experimental studies. This line of work has also the aim of taking into consideration further developments important for gesture research namely: a. cross-cultural studies to verify the universality of hand gesture taxonomy via the analysis of cultural differences and similarities; b. psycho-physiological studies to individuate physiological correlates of gesture in both agent and recipient of the communication.

2. Structure

Concerning the description of hand gesture structure, two studies are here illustrated. The first study regards: (a) synthesis of the different existing classifications in one taxonomy, in order to achieve a categorical system which is as exhaustive as possible and useful in observational research; (b) test of this classification within the context of group discussion in Italian, (c) measure of the reliability of the coding system based on the taxonomy through inter-observer agreement; and (d) realization of a digital coding manual in the form of a multi-media tool for using this hand gesture coding system within observational research. The second study concerns a systematic observation of hand gesture in a different cultural context (Burkina Faso) in order to test the universality of such taxonomy and therefore to show possible cultural differences.

2.1 A taxonomy of hand gesture Italian samples In the proposed taxonomy (Figure 1), synthesising previous classifications (e.g., Ekman, Friesen, 1969, McNeil, 1992), hand movements are considered as basically organised into two macro-categories: discourse linked gestures and discourse non-linked gestures. Therefore the

Bonaiuto, Maricchiolo, Gnisci

Hand gesture structure and functions proposed taxonomy is based on the fundamental classification criterion according to which gestures can, or cannot, be linked to the speech (McNeill 1985, 1992).

Figure 1. Classification of hand gestures (adapted from Bonaiuto, Gnisci, Maricchiolo, 2002)

The discourse linked gestures category includes rhythmic, cohesive and ideational gestures. Rhythmic gestures (termed beats by McNeill, 1992, and baton by Efron, 1941, and Ekman, Friesen, 1969) are related to speech but do not refer to the actual speech content; these are hand/finger movements moved along with the rhythmical pulsation of the speech, tending to provid the discourse with emphasis and rhythm by punctuating speech, i.e. by accentuating or emphasizing some parts of the utterance. Cohesive gestures are repetitive hand movements performed by the speaker in order to accompany the development of the discourse and to give it continuity and coherence(they are basically circular, forward-backward, or right-left hand-movements). These gestures do not refer to the discourse content but to the narrative structure: i.e., their shape is not representative of the speech content. They are further subdivided into a range of sub-categories according to the shape of the movement of one or both hands (e.g., Contento, 1999). Ideational gestures refer to the content of speech, including emblems and illustrative gestures (Ekman, Friesen, 1969): emblems are all symbolic gestures, i.e., those which can be directly translated into spoken language and thus are easily understandable by everybody within the same culture. Illustrative gestures includes iconic, metaphoric and deictic gestures (McNeill, 1992): iconic gestures are gestures reproducing the form of the object being spoken about which the gesture itself refers to; metaphoric gestures refer to abstract concepts; deictic gestures point out an object which is present either in the physical environment of the speaker or in the ideal environment represented by the content of the discourse. These gestures are characterized by the fact that they depict various linguistic reference points, belonging to the material world (objects) in the case of iconic gestures, to the abstract world (ideas) in the case of metaphoric gestures, or to elements of the circumstances in which the subjects express themselves in the case of deictic gestures. Non-linked to discourse gestures include so-called adaptor gestures (Ekman, Friesen, 1969), which are gestures of contact. They typically are assumed to have the purpose of satisfying psychological needs or of maintaining interpersonal contact and they are divided into heteroBonaiuto, Maricchiolo, Gnisci 5

Hand gesture structure and functions adaptor gestures (contact with what is external to the subject: object-adaptor and personadaptor) and self-adaptor gestures (hand in contact with parts of ones own body). Using this taxonomy as a coding system, hand gestures used by young female conversant in four- or eight-member group discussion (see Figures 2 and 3) were observed and coded to test the presence of each gesture category of the taxonomy and to test the general classification criterion (link or not of gesture to speech). As showed in Table I, the positive significant correlation (r = .76, p = .000) between Linked to discourse gestures and Talk turns reveals the relation between this type of gesture and speech, therefore the validity of the main classification criterion (presence or absence of link between gesture and speech). At the same time, the absence of significant correlation between linked to discourse gesture and non-linked to discourse gesture (r = .17, p = .32) and between the last and talk turns (r = .12, p = .50) respectively indicates the independence between these gesture macro-categories and the independence of the last from speech activity.

Figures 2-3. Pictures extracted by group discussions of our sample.

1 1. Linked to discourse gestures 2. Non-linked to discourse gestures 3. Talk turns N = 36: ** p < .001 .17 .76** .12

Table I. Correlation between gesture and talk turns in context of group discussion (Bonaiuto et al., 2002).

To test the reliability of coding system, hand gestures used by different kinds of conversants from different interactive contexts (TV political interviews, legal interrogations, courtroom examinations, dyadic conversations, group discussions) were coded by two independent observers. Interactions were transcribed through a transcription system similar to the classic conversational analysis transcription system integrated with alternate lines in which gestures were reported. The beginning and the end of each gesture were indicated with square parentheses, as for speech overlap. In this way co-occurrences between verbal and hand gestures were available (see

Bonaiuto, Maricchiolo, Gnisci

Hand gesture structure and functions Extract 1 as an example: B is the speaker and during the first line of verbal speech she made a metaphoric and an iconic gesture).
Extract 1. Transcription extract by an Italian dyadic conversation. 105 106 107 108 109 110 111 112 B: [cio, a parte che lui pi bravo][per se c uno stipendio fisso] [ metaphoric ][ iconic ]

[vuol dire che la giornata tu la devi lavo-cio devi stare in [ rhythmic

ufficio. Per qualsiasi cosa tu devi stare l] (.) [cio lui non che r ] [ cohesive

: bra:vo][un giorno][e poi si va a fare le vacanze una settimana.] ][ iconic ][ cohesive ]

Agreement percentage and inter-observers agreement through Cohens K index on each category and on each context are brought on. According to the results, coding system of hand gestures based on the taxonomy is reliable: 1. percentage of agreement on coding unit (gesture) is 91.6%; 2. Cohens K index on coding system is .82 3. all K indexes on each gesture category and on each examined context are higher than .75. According to Bakeman and Gottman (1997), Cohens K >.75 is set as the threshold to test coding reliability, therefore the above mentioned results are satisfying. After having verified the reliability of the hand gesture category system, a multi-media manual (presently in Italian language) has been realized as a tool aimed at describing the gesture taxonomy and at making it shared by other researchers. It is composed by a theoretical introduction, conceptual and operational definitions of each gesture category with examples (ideal, real, and problematic examples of each gesture categories) in video and in photo, a guide for the observation and coding of gesture during interactions, and bibliographical references.

2.2 Test of the taxonomy in another culture Burkina Faso sample To test the possible universality of the taxonomy, video recorded data were collected in Burkina Faso in order to observe and code hand gesture used by Burkinab. Nine dyadic conversation in Moor dialect (ethnic language) and in French (the official language of Burkina Faso) were video-recorded, transcribed, translated in to Italian and coded for hand gestures (Extract 2).

Bonaiuto, Maricchiolo, Gnisci

Hand gesture structure and functions

Extract 2. Transcription extract by a Burkinab dyadic conversation (More dialect). MOOR A [Sa n ya ne tond] Nanor kaane [self-adaptor ] tond tara ekol [maternel yeng bala] [ cohesive ] [(.)rn kitame ti ps n ka naana ye] [self-adaptor ] B [mm ekol a ye ta self-adaptor mme ekol Primeera yaa] self-adaptor foo [meng neeme ti neba meng] [rhythmic ] [neba waoga ka ye (..) [cohesive elevramb waogame] [Parandelevramba cohesive ] [ cohesive ITALIAN TRANSLATION A Per quanto riguarda noi qui a Nanoro A di scuola materna ne abbiamo una sola A (.)ci fa s che non del tutto facile B mm una sola scuola B anche la scuola primaria B tu stessa vedi che la gente B la gente troppo numerosa (..) B gli allievi sono tanti i genitori

All expected gesture categories were found in the Burkinab sample (Table II). Moreover a new typical form of gesture, very similar to rhythmic, was observed in Burkinab conversant: it looks like a single applause (hand palms beat each other one time, as showed in Figures 4-6). Compared with Italian conversants, Burkinabs use more cohesive and self-adaptor gestures and less metaphoric, deictic and object-adaptor gestures (Table III).

GESTURE Frequencies F% Language French Moore Total French Moore Total Cohesive 1004 982 1986 57,08 55,8 56,44 Rhythmic 143 112 255 8,13 6,36 7,25 Iconic 10 19 29 0,57 1,08 0,82 Metaphoric 81 79 160 4,6 4,49 4,55 Deictic 77 101 178 4,38 5,74 5,06 Emblem 24 33 57 1,36 1,87 1,62 Object-adaptor 97 67 164 5,51 3,81 4,66 Person-adaptor 8 8 16 0,46 0,45 0,45 Self-adaptor 379 394 773 21,55 22,39 21,97 Typical 79 77 156 4,49 4,37 4,43 1759 1760 3519 100 100 100 Total
Table II. Frequencies and percentages of each gesture category in each language in Burkinab sample.

Figures 4-6. Photo sequence of typical gestured observed in Burkinab conversant.

Bonaiuto, Maricchiolo, Gnisci

Hand gesture structure and functions

Nazionalit Italiane Gesti f Cohesive 709 Rhythmic 200 Iconic 16 Metaphoric 532 Deictic 678 Emblem 24 Object-adaptor 496 Person-adaptor 10 Self-adaptor 510 3175 Totale

f% z 22.3% -13.83 6.3% -1.41 0.5% -1.25 16.8% 10.69* 21.3% 12.87* 0.8% -2.45 15.6% 9.78* 0.3% -0.74 16.1% -4.53 100%

Burkinabe f f% z 1731 51.4% 13.43* 255 7.6% 1.37 29 0.9% 1.22 160 4.8% -10.39 178 5.3% -12.50 57 2.38% 1.92 164 4.8% -9.51 16 0.5% 0.72 773 23% 4.40* 3363 100%

Table III. Comparison between Italian and Burkinab conversant in the use of different gesture categories during dyadic conversations (frequencies, percentages, and standard residual).

3. Functions

With regard to the individuation of hand gesture functions during conversational interaction, the main results of two clusters of study are presented. The first cluster of studies has the aim of identifying the functions of each gesture category through its co-occurrence with some aspect of speech, such as discursive markers (Fraser, 1999), rhetorical devises (Atkinson, 1984), as well as through the sequence of gesture used in replying to different question types. The second cluster of studies were aimed at verifying the role of hand gestures in persuasive communication within social influence contexts through the analysis of the relation between gestures and power measures in group context, as well as through the analysis of gestures influence on the persuasive outcome within the context of an oratorical speech.

3.1 Discursive and interactional functions On the same material of the previous study (taxonomy test in Italian contexts) the amount of co-occurrence among gesture categories and some verbal acts was calculated in order to infer the functions of different hand gestures on the basis of their association to specific speech parts. With regard to the coding of verbal acts, different types of acts has been considered: discursive markers, rhetorical devices, question types. Discursive markers has been distinguished as fellow: verbal markers with interactive functions (all those concerning turn taking, or feedback, verbal or vocal signals of attention, etc.); social cohesion markers, aimed at establishing the relationship with the interlocutor and at marking Bonaiuto, Maricchiolo, Gnisci 9

Hand gesture structure and functions shared knowledge (e.g., my darling, as you known, etc.); markers with meta-textual functions that demarcate passages and/or different parts of discourse or relations between different parts of discourse (e.g., so, therefore, then), or stress focal points (e.g. just, exactly, but, even if), or with the function of repetition, correction, exemplification (e.g., that is, such as, on the contrary, or rather, but). Then, the amount of occurrences of different categories of gesture with the different markers were recorded. The same was made for rhetorical devices. They are communication strategies aimed at making the discourse more strong, powerful and persuasive. They were coded into some of the different types described by Atkinson (1984), Edwards and Potter (1992) and others: list, contrast, denial, metaphor, etc. The amount of occurrences of the different gestures and the different rhetorical devices were recorded. Question types, were coded only in the context of courtroom examinations, during the interrogatory according to their coerciveness (close/open questions, see Gnisci, Bonaiuto, 2003) and face-threatening (hostile and menacing questions, see Elliott, Bull, 1996). These types of questions are considered as inducing anxiety or strong tension in the interviewees. In order to find what category of gesture is used in replaying to stressful questions, sequential analyses were carried out on the first three gestures carried out during each answers opening. In general, results showed that cohesive gestures are significantly associated to markers with functions of demarcation and reformulation of the discourse structure (meta-textual markers). This result would confirm that function of cohesive gestures is to demarcate discourse structure. Rhythmic gestures are co-occurring with focalization markers stressing focal point of the discourse, but also with demarcation and with non-marked speech. Ideational gestures, illustrating the content of speech, are associated with non-marked speech. Adaptor gestures are significantly associated with pauses (confirming they are non-linked to discourse gestures), but unlike the hypothesis they are also associated with interaction and social cohesion markers. Probably they can have an interactive function of managing social relationship and conversation, or they can show psychological distress related to such practices.. With regard to rhetorical devices, illustrator (iconic, metaphoric, and deictic) gestures are positively associated with the presence of rhetorical devices in the speech: they illustrate the discourse content in a powerful way. On the contrary adaptor gestures are negatively associated with rhetorical devices. While, contrary to the expectations, rhythmic gestures have a negative association with rhetorical speech: this is probably due to the high amount of frequency of rhythmic gestures during non-rhetorical speech.

Bonaiuto, Maricchiolo, Gnisci

10

Hand gesture structure and functions Concerning the gestures used at the beginning of answering during courtroom examination, in synthesis, sequential analyses show that more coercive questions more probably induce replays beginning with adaptor gestures, while replays to less coercive questions more probably begin with cohesive or ideational gesture. Similarly, more threatening questions produce replays beginning with adaptor gestures therefore, both results shows that adaptor gestures are used more during speech when dealing with a more stressful rather than a less stressful conversational context. In synthesis, according to our results, cohesive gestures would have a function of demarcation of discourse structure; rhythmic gestures would have the function of stressing focal points of the discourse; ideational gestures would illustrate discourse contents, particularly when they are rhetorical packaged; adaptor gesture, used during speech, probably have an interactive function of relation and conversation management at the beginning of talk turn or they can signal distress in replaying to tough questions.

3.2 Persuasive and influential functions Three studies were carried out with the aim of understanding the role of hand gestures in influential and persuasive communication. The first study examines, in young women small discussion groups of different size (eightmember and four-member groups), which verbal and gestural features of communicative dominance, considered together, could predict perceived social influence. The aim of this study is to assess the relationship of particular categories of gestures and of conversational dominance (defined as the number of conversational turns held) with perceived social influence (by group members), in the context of small group discussions. Multilevel models analyses showed that in eight-member groups, verbal dominance appears to be a stable predictor of perceived social influence. As regards gestural behavior main effects, no gesture category shows a significant effect. However, participants' cohesive, ideational and heteroadaptor gestures interact significantly with verbal dominance in the prediction of influence so as to suggest that if the participants' verbal dominance score is lower, the relation between behavioral gestures and perceived influence was more substantial than if the verbal dominance score is higher. That is, when, in a more numerous small group (i.e., an eight- rather than four-member group), a member takes fewer turns, she is perceived as more influential if she uses more cohesive, ideational and/or hetero-adaptor gestures than if she uses fewer gestures of this kind. But, if one takes more turns, then there is no difference between using more or fewer gestures of this kind. In four-member groups, the results scenario is quite different. Asymmetric participation is much softer, and even if the pattern is similar to the previous sample, a main positive effect for Bonaiuto, Maricchiolo, Gnisci 11

Hand gesture structure and functions ideational gestures, besides the usual significant positive main effect for verbal dominance (only in the model which includes verbal dominance and ideational gestures), and an interaction effect of the two variables on perceived influence were found. Thus, according to this model, in smaller groups (four-member groups) one could be perceived as influential if speaks more, uses more ideational gestures or both. Unlike the results of other studies (Burgoon, et al., 1990; Carli, LaFleur, Loeber, 1995; Henley, 1977), which found a negative effect for self-adaptor gestures on influence or persuasion, this study shows that self-adaptor gestures have neither a main negative effect nor interaction effect with verbal dominance on perceived social influence. The second study had the aim to describe hand gesture used by political leaders during speech in TV interviews during the 2001 Italian electoral campaign official period. Comparing two leaders of opposite political groups, Francesco Rutelli (for center-left coalition) and Silvio Berlusconi (for center-right coalition), some difference emerged on the categories of hand gestures: in synthesis, non-parametric analyses (Chi-square and adjusted residual) show that Berlusconi used more cohesive and object-adaptor gestures than Rutelli, which used more ideational (particularly, emblems and metaphoric) and self-adaptor gestures. The two politicians are different for the preference in the use of some particular types of gesture amongst all used: Berlusconi preferred to use cohesive gestures, which contribute to provide the discourse with continuity, coherence, and structure (McNeill, 1992), as well as object-adaptor gestures, which are non-linked to the discourse, but are functional to satisfy speakers emotional needs (Ekman, Friesen, 1969), for example, allowing to relieve from tension state on surrounding objects (e.g., a pen, a paper, etc.). Rutelli showed a preference for the use of emblems, which are cultural symbols with a shared meaning; of metaphoric gestures, which have the function to illustrate and, in some way, to disambiguate discourse content (Argentin et al., 1990); and of selfadaptor gestures, i.e., non-linked to discourse functional to the management of emotional discomfort state (Ekman, Friesen, 1969). According to the literature (Argentin et al., 1990; Burgoon et al., 1990; Henley, 1977; Moore, Porter, 1988), Berlusconi amongst adaptor used less penalizing gestures (object-adaptor) than Rutelli (which amongst adaptor gestures performed self-adaptor), in terms of impact on the speaker credibility and influence. Hand gestures, particularly rhythmic, cohesive, deictic, and metaphoric gestures, are intensifying, emphasizing, stressing, and illustrating verbal communication. They have a particular and important value in public political speech, given their high visibility with respect to other

Bonaiuto, Maricchiolo, Gnisci

12

Hand gesture structure and functions nonverbal signals All the above mentioned studies, however, given their correlational nature, fail to demonstrate a causal link among gestures use and persuasion. Therefore, a necessary following step is an experimental study that, through a careful manipulation of speaker gestures is able to test their specific impact in terms of speaker credibility and persuasiveness. And, given the fact that gestures are closely connected to the speech, it should also assess the impact of verbal aspects of communication as well as of the verbal-gesture interaction, on such evaluations about the speaker. The third study is in fact an experiment aimed to test the role of different types of gesture on the perception by the receiver of persuasiveness and credibility of a speaker, as well as to test experimentally persuasive effectiveness of speech-gesture coordination. An experiment on undergraduates was carried out using ten video-messages in which gestures of speaker, who was a professional actress, and rhetorical power of verbal message were varied. The topic of messages and the argumentations in support of it, chosen on the basis of a preliminary study, were the same in all conditions. As regard to manipulations, there were five levels for gesture, that are: cohesive; ideational object-adaptor; self-adaptor; with the fifth level being a control condition (no gesture). There are two levels for speech: powerful and powerless speech (Ng, Bradac, 1993) that means presence or absence of rhetorical devices in the speech. Receivers involvement in the topic of the message was also manipulated: it was considered high or low involvement of receivers as previous researches on persuasion showed it to be relevant (e.g., Petty, Cacioppo, 1986). Measures were gathered though a questionnaire on two hundred male and female undergraduates. Measures regard evaluations of persuasiveness and reliability of message, evaluations of persuasiveness and expertise of the speaker and evaluation of effectiveness of her communicative style. Moreover receivers attitude towards the topic, their vote intention about it and their actual vote were measured. Confirming the literature and the hypotheses, ANOVA results show that using self-adaptor gestures has negative effects on both evaluation of speakers expertise and effectiveness. Expertise increases only in absence of gestures, while effectiveness increases with absence of gesture and with discourse-linked gestures (cohesive and ideational). There are two interaction-effects speech by gesture on speaker style effectiveness and on message persuasiveness. In both cases, the most positive results are obtained with a co-occurrence of cohesive and powerful speech, while more negative results are obtained with object-adaptor co-occurring with powerful speech. Only message persuasiveness can improve also with the use of object-adaptor gestures with powerless speech. On the whole, these results can be interpreted in terms of coherence among speech style and gesture Bonaiuto, Maricchiolo, Gnisci 13

Hand gesture structure and functions style (powerful speech with cohesive gestures and powerless speech with object-adaptor gestures). Finally there is a three ways interaction effect on message reliability: it basically shows that the above-mentioned speech by gesture interaction works with low involved subjects. This result can be interpreted in terms of peripheral route to persuasion. This means that low involved subject are more sensitive to peripheral signals as verbal and nonverbal forms. Contrary to the literature (Ng, Bradac, 1993), no main effects of speech resulted on the evaluations of message and of speaker. All results can be synthesized through the diagram in Figure 7.

Gesture
Speaker expertise

Gesture x Speech
Style effectiveness

Gesture x Speech x Involvement

Message reliability

Message persuasiveness

Attitude Vote intention

Vote
Figure 7. Final model of the role of the gesture-speech co-ordination and of gesture-speech-involvement interaction in the persuasive process.

As Figure 7 shows, gestures influence evaluations of speaker expertise and communicative style effectiveness; interaction between speech and gesture influences communicative style effectiveness and message persuasiveness; interaction among speech, gesture and receivers involvement influences evaluations of message reliability. In turn, regression analysis shows that the evaluations of communicative style effectiveness, and of message persuasiveness and reliability predict the attitude of the receivers towards the topic of the message; the attitude predicts the vote intention, which in turn predicts the actual vote. In general, this study proves that there are some gesture expressions that, in coherent coordination with rhetoric-discursive aspects of speech, can realize a communicative style evaluated as effective and persuasive by the receivers. Moreover, it shows that these evaluations of message and speaker by the receivers influence their attitude towards message topic and indirectly even their vote behaviour.

Bonaiuto, Maricchiolo, Gnisci

14

Hand gesture structure and functions

4. Conclusion and future developments In the studies presented, in synthesis, we have found a category system of hand gestures which can have satisfying reliability when used in different contexts of social interaction (having also digital manual in CD-ROM). Concerning hand gesture functions these studies are an empirical, statistical, and experimental demonstration of the pragmatic link between hand gesture and cooccurring verbal speech as well as of the essential role of gesture and gesture-speech interaction in persuasive and influential speech. The analysis of the co-occurrence among specific types of gestures and specific verbal phenomena helps to clarify gestures functions. Correlational studies confirming that there are gesture patterns more frequently associated, positively or negatively, to social influence. However it is only via experimental manipulation that gestures can be linked to persuasion. Moreover, measuring different constructs/dependent variables (speaker, message, attitude, behaviour) helps to articulate more specific effect of gesture and of gesture-speech cooccurrence. Many other important steps should be run in gesture study to improve the knowledge of this phenomenon even from other different points of view. In our opinion, the study of physiological correlates of both gesture agent and his/her interlocutor are crucial. In particular in the study of gesture decoding and perception, it is interesting to measure physiological reactions to different types of gesture. Our research project in progress aims to individuate gestures main effects on the following physiological reaction: 1) blinking as measure of attention; 2) heartbeat increase as measure of activation and arousal; 3) labial muscle contracture through electromyography as measure of positive attitude; 4) wrinkling muscle contracture even through electromyography as measure of negative attitude. These and other parameters (e.g., ocular movements) could help in understanding why some gestures are more effective in persuading (e.g., describing perception attention and attitudinal processes in the audience).

Bonaiuto, Maricchiolo, Gnisci

15

Hand gesture structure and functions References

Alibali, M. W., Flevares, L., Goldin-Meadow, S. (1997). Assessing knowledge conveyed in gesture: Do teachers have the upper hand? Journal of Educational Psychology, 89, 183193. Argentin, G., Ghiglione, R., Dorna, A. (1990). La gestualit et ses effets dans le discours politique. Psychologie Franaise, 35, 153-161. Atkinson, J. M. (1984). Our Masters Voice. London: Routledge. Bakeman, R., Gottman, J. M. (1997). Observing interaction. An introduction to Sequential Analysis. II ed. Cambridge: Cambridge University Press (I ed. 1986). Beattie, G., Shovelton, H. (2000). Iconic hand gestures and predistability of words in context in spontaneous speech. British Journal of Psychology, 91, 473-492. Bonaiuto, M., Gnisci, A., Maricchiolo, F. (2002). Proposta e verifica di una tassonomia per la codifica dei gesti delle mani in discussioni di piccolo gruppo. Giornale Italiano di Psicologia, 29, 777-807. Bull, P. E. (1986). The use of hand gesture in political speechers: Some case studies. Journal of Language and Social Psychology,5, 102-118. Burgoon, J. K., Birk, T., Pfau, M. (1990). Nonverbal behaviors, persuasion, and credibility. Human Communication Research, 17, 140-169. Carli, L. L., LaFleur, S. L., Loeber, C. C. (1995). Nonverbal behavior, gender, and influence. Journal of Personality and Social Psychology, 68, 1030-1041. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37- 46. De Jorio, A. (1832) La mimica degli antichi investigata nel gestire napoletano, Napoli: Arnaldo Forni Ed.Paura. De Paulo, B., Friedman, H. S. (1998). Nonverbal Communication. In D. T. Gilbert, S. T. Fiske, G. Lindzey (Eds), The Handbook of Social Psychology. Fourth Edition (vol. II, pp. 3-40). Boston, Mass: McGraw Hill. Edwards, D., Potter, J. (1992). Discursive psychology. Thousand Oaks: Sage Publications. Efron, D. (1941). Gesture and enviroment. New York: Kings Crown Press (trad.it. Gesto, razza e cultura. Milano: Bompiani, 1974). Ekman, P., Friesen, W. V. (1969). The repertoire of nonverbal behavior. Semiotica, 1, 49-98. Elliott, J., & Bull, P.E. (1996). A question of threat: face threats in questions posed during televised political interviews. Journal of Community and Applied Social Psychology, 6, 49-72.

Bonaiuto, Maricchiolo, Gnisci

16

Hand gesture structure and functions Exline, R. V. (1972). Visual interaction: The glances of power and preference. In J. R. Cole (Ed.), Nebraska Symposium on Motivation 1971. Lincoln: University of Nebraska Press. Feyereisen, P., Havard, I. (1999). Mental imagery and production of hand gestures while speaking in younger and older adults. Journal of nonverbal Behavior, 23, 153-171. Fraser, B., (1999). What are discourse markers?. Journal of Pragmatics 31, 931-952. Gnisci, A., & Bonaiuto, M. (2003). Grilling politicians. A study on politicians answers to questions comparing televised political interviews and legal examinations. Journal of Language and Social Psychology, 29, 4, 384-413. Goodwin, C. (2000). Action and Embodiment within Situated Human Interaction. Journal of Pragmatics, 32, 1489-1522. Henley, N. M. (1977). Body politics: Power, sex, and nonverbal behavior. Englewood Cliffs, NJ: Prentice-Hall. Kelly, S. D., Church, R. B. (1998). A comparison between childrens and adults ability to detect conceptual information conveyed through representational gestures. Child Development, 69, 8593. Kendon, A. (1983). Gesture and Speech: How they interact. In J. M. Wiemann, R. P. Harrison (Eds), Nonverbal Interaction (pp. 13-45). Beverly Hills, CA: Sage Publications. Kendon, A. (1995). Gestures as illocutionary and discourse structure markers in Southern Italian conversation. Journal of Pragmatics, 23, 247-279. Krauss, R. M., Chen, Y., Chawla, P. (1996). Nonverbal behavior and nonverbal communication: What do conversational hand gestures tell us? Advances in Experimental Social Psychology, 28, 389450. Leigh, T. W., Summers, J. O. (2002). An industrial buyers impressions of salespersons nonverbal cues. Journal of Personal Selling & Sales Management, 22, 41-53. McNeill, D. (1985). So you think gesture are nonverbal? Psychological Review, 92, 350-371. McNeill, D. (1992). Hand and Mind. Chicago: The University of Chicago Press. Mehrabian, A. (1972). Nonverbal communication, Chicago-New York: Aldine-Atheron. Moore, H. E., Porter, N. K. (1988). Leadership and nonverbal behaviors of Hispanic females across school equity environments. Psychology of Women Quarterly, 12, 147-163. Morain, G, (1997). Visual Literacy: Reading Signs and Designs in the Foreign Culture. In R. Paula (Ed), Pathways to culture: readings on teaching culture in the foreign language class (pp 363-375). Yarmouth, ME: Intercultural Press. Ng, S. H., Bradac, J. J. (1993). Power in language: Verbal communication and social influence. Newbury Park, CA: Sage Publications. Bonaiuto, Maricchiolo, Gnisci 17

Hand gesture structure and functions Pedhazur, E. J., Pedhazur-Schmelkin, L. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Erlbaum. Petty, R. E., Cacioppo, J. T. (1986) Communication and persuasion: Central and peripheral routes to attitude change, New York: Springer-Verlag. Rim, B., Schiaratura, P. (1991). In R. S. Feldman, B. Rim (Eds), Fundamentals of Nonverbal Behavior. Cambridge: Cambridge University Press; Paris: Editions de la Maison des sciences de lhomme. Schultheiss, O. C., Brunstein, J. C. (2002). Inhibited power motivation and persuasive communication: A lens model analysis. Journal of Personality, 70, 553-582. Whitehead, G., Smith, S. H. (2002). The use of hand gestures and smiles in the inaugural addresses of Presidents of the United States. Journal of Social Psychology, 142, 670-672.

Bonaiuto, Maricchiolo, Gnisci

18