See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/257387946

An introduction to multimodal communication

Article  in  Behavioral Ecology and Sociobiology · September 2013

DOI: 10.1007/s00265-013-1590-x

CITATIONS READS

150 10,157

2 authors, including:

Eileen A. Hebets
University of Nebraska at Lincoln
150 PUBLICATIONS   5,441 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Sexual Selection in Arachnids View project

Science Communication View project

All content following this page was uploaded by Eileen A. Hebets on 15 April 2014.

The user has requested enhancement of the downloaded file.

Behav Ecol Sociobiol (2013) 67:1381–1388
DOI 10.1007/s00265-013-1590-x
ORIGINAL PAPER
An introduction to multimodal communication
James P. Higham & Eileen A. Hebets
Received: 28 February 2013 / Revised: 26 June 2013 / Accepted: 27 June 2013 / Published online: 17 July 2013
# Springer-Verlag Berlin Heidelberg 2013
Abstract Though it has long been known that animal com-
munication is complex, recent years have seen growing
interest in understanding the extent to which animals give
multicomponent signals in multiple modalities, and how the
different types of information extracted by receivers are
interpreted and integrated in animal decision-making. This
interest has culminated in the production of the present
special issue on multimodal communication, which features
both theoretical and empirical studies from leading re-
searchers in the field. Reviews, comparative analyses, and
species-specific empirical studies include manuscripts on
taxa as diverse as spiders, primates, birds, lizards, frogs,
and humans. The present manuscript serves as both an intro-
duction to this special issue, as well as an introduction to
multimodal communication more generally. We discuss the
history of the study of complexity in animal communication,
issues relating to defining and classifying multimodal sig-
nals, and particular issues to consider with multimodal (as
opposed to multicomponent unimodal) communication. We
go on to discuss the current state of the field, and outline the
contributions contained within the issue. We finish by
discussing future avenues for research, in particular empha-
sizing that ‘multimodal’ is more than just ‘bimodal’, and that
more integrative frameworks are needed that incorporate
more elements of efficacy, such as receiver sensory ecology
and the environment.
Communicated by E. Fernandez-Juricic
This manuscript is part of the special issue Multimodal
Communication—Guest Editors: James P. Higham and Eileen A. Hebets
J. P. Higham (*)
Department of Anthropology, New York University,
25 Waverly Place, New York, NY 10003, USA
e-mail: jhigham@nyu.edu
E. A. Hebets
School of Biological Sciences, University of Nebraska,
Kearney, NE, USA
Keywords Multimodal communication . Animal signals .
Receiver sensory perception
Introduction
Animals signal to both con- and hetero-specifics using mul-
ticomponent signals in every sensory modality, exhibiting
visual signals, olfactory signals, auditory signals, and more.
The idea that animals communicate with multiple signal
elements in multiple modalities has been expressed for some
considerable time. Indeed, it was emphasized by Darwin
(1872) in The Expression of Emotions in Man and Animals,
and the point continued to be made by other researchers
during the transition from ethology into the more
hypothesis-driven behavioral ecology (e.g., Marler 1965;
Uetz and Stratton 1983). However, studies of multimodal
communication received a jolt in the 1990s, with high-
profile papers published in Nature on multimodal signaling
(Rowe and Guildford 1996) and in Science of the first
classification framework for categorizing multimodal signals
based upon receiver responses to isolated versus combined
components (Partan and Marler 1999a). The high-profile
publication of this framework helped spur renewed and
widened interest in multimodal signaling, and the years since
have seen growing realization among behavioral ecologists
that animal communication is frequently multimodal in na-
ture. This transition has culminated in the publication of this
special issue, which brings together a range of multi-
disciplinary scientists. These present contributions on the
description and classification of multimodal signals, game
theoretic models of multiple signaling, conceptual articles,
and whole organism multimodal signaling case studies and
comparative analyses. The diverse groups covered by the
contributions include studies of spiders, frogs, birds, pri-
mates, lizards, and humans.
The present paper serves as an introduction to the field of
multimodal communication, as well as to the present issue.
The origins of this issue lie in a symposium that took place at
1382
the Animal Behavior Society (ABS) meeting in Williamsburg,
Virginia, on the 25th–29th of July 2010. This was funded by
ABS, and we are extremely grateful to them for their commis-
sioning and support of the symposium. Behavioral Ecology
and Sociobiology very kindly offered to host this resulting
special issue—thankfully almost all speakers were able to
contribute, and we have taken the opportunity to invite other
names in the field to add additional contributions. The overall
aims of the issue are: (1) to serve as a defining reference for the
current state of research on multimodal communication; (2) to
stimulate more research in the field; and (3) to help promote
the further multidisciplinary collaboration necessary to
achieve breakthroughs in our understanding of complex com-
munication systems. More specific aims are: (1) to highlight
the importance of multimodal communication in animal sig-
naling; (2) to present and discuss new models of signal evo-
lution; (3) to present ways of studying multimodal systems,
and of teasing apart the effects of specific signals using
experimental approaches; (4) to investigate case studies of
signaling systems, and explore the ecological and social pres-
sures and constraints that influence their evolution; (5) to
explore the extent to which information from multiple-
signals is received, interpreted, and integrated into decision-
making; and (6) to develop suggestions for future areas of
research likely to prove fruitful.
What is a multimodal signal?
Before going further, we would like to address the question
“what is a multimodal signal?”, which in itself first requires a
definition of a signal. This is a myriad area, with almost as
many definitions as there are authors. We advocate a defini-
tion similar to that proposed by Ruxton and Schaefer (2011),
in which one individual stimulates the sensory systems of
another, exerting influence on that individual. We also con-
cur with many others in advocating a distinction between a
signal and cues (e.g., Maynard Smith and Harper 2003),
where signals represent traits that have been under selection
specifically for their communicative function, and where
cues are potentially informative traits that have not been
under such selection. Regarding the definition of a multi-
modal signal itself, there are two main ways of answering
this question. The first is to consider a signal on the basis of
its form—its physical properties in terms of the channel
through which it is sent. The alternative is to choose a
definition based on the receiver sensory systems used to
detect it. To us, this is a preferable definition, but is more
complex as it requires greater understanding of receiver
sensory systems. Such a definition may therefore sound
easier to apply than it is in practice. First, as one of us has
pointed out elsewhere (Hebets 2011), different types of an-
imals have evolved different mechanisms for the production
Behav Ecol Sociobiol (2013) 67:1381–1388
and detection of signals that may share physical properties. A
signal with a particular physical form may be perceived by
different sensory systems. In insects for example, the detec-
tion of chemicals on a substrate occurs with distinct sensilla
(contact chemoreceptors) when compared to the detection of
chemicals in the air (detected by olfactory or multiparous
chemoreceptors). The information is then processed inde-
pendently (see discussion in Hebets 2011). While this exam-
ple highlights signals with similar physical form (i.e., chem-
ical), the signals themselves are separated in space. Other
examples exist where an identical signal may stimulate mul-
tiple sensory systems in a single animal. For example, the
particle motions of underwater sound can be detected by ears
in both vertebrates and invertebrates, but many aquatic ani-
mals can also detect these stimuli using other systems (e.g.,
neuromasts, Dambly-Chaudière et al. 2003). In the event of a
signal where the production is in one modality but where
more than one sensory system is stimulated, should this
signal be categorized as a unimodal or a multimodal signal?
Would this categorization then differ with a different receiver
that perceived and processed the signal through one sensory
system only? Consistent with recent articles stressing the
important of receiver perception in animal communication
(e.g., Ruxton and Schaefer 2011) and with the conceptual
frameworks for classifying multimodal signals that are based
on receiver responses (Partan and Marler 1999a, 2005; Smith
and Evans 2013), our answers would be ‘multimodal’ and
‘yes’. The correct categorization of communicative signals
into different sensory modalities is highly relevant for un-
derstanding how both: (1) the signaling environment (e.g.,
modality specific transmission properties) as well as (2)
receiver sensory systems (e.g., reception and processing),
influence signal function and evolution. As such, our ap-
proach needs to incorporate information about both the sig-
naling environment and the receiver’s sensory system. While
an initial categorization based upon the physical properties
of signal form is an obvious place to start, this may change as
more information regarding the receiver’s sensory and pro-
cessing system becomes available, and we feel it is important
to acknowledge when such information is incomplete.
Next, we would like to highlight a distinction between
signals that are and are not intrinsically multimodal—“fixed”
versus “free”. When a frog croaks, its throat sack inflates and
this visual cue is inextricably tied to the production of the
sound. Such instances where components are obligately
produced together are referred to as “fixed” (Smith 1977;
Partan and Marler 2005). In some frogs, females are able to
use the visual cue to identify individuals calling loudly in
otherwise acoustically noisy environments (Taylor et al.
2008; Grafe et al. 2012; Preininger et al. 2013). Similarly,
as Ghazanfar points out in the present issue, movement of the
lips is an essential part of making many different sounds
from the primate mouth. Certainly human language is
Behav Ecol Sociobiol (2013) 67:1381–1388
inherently multimodal, and it is impossible to form the words
with which we communicate without concomitant move-
ment of the lips (McGurk and MacDonald 1976). There
appears to be an evolutionary origin to this well before lan-
guage evolved—rhesus macaques match specific rhesus ma-
caque vocalizations to the facial expression that accompanies
that sound (Ghazanfar and Logothetis 2003, Ghazanfar 2013).
Regardless of the obligate nature of the combination of com-
ponents, selection can subsequently act upon components of
“fixed” multimodal signals. For example, in many aquatic
crustaceans, chemical signals are disseminated by signalers
through the generation of water currents, such that they are
obligately tied to hydrodynamic cues. Numerous studies in
aquatic crustaceans suggest that receivers respond to water
currents alone and that selection may therefore act directly on
these hydrodynamic stimuli (making them signals in their own
right, reviewed in Hebets and Rundus 2011).
In contrast to “fixed” multimodal signals, there are a
number of other multimodal communication systems where
each signaling component is intrinsically unimodal, but in
which multiple signals in different modalities are given as
part of a multimodal display. Multimodal displays in which
signal components are not obligately tied and do not neces-
sarily always occur together have been referred to as “free”
(Marler 1961; Partan and Marler 2005). Excellent examples
of “free” multimodal displays can be found in male jumping
spiders in the Habronattus coecatus clade. Males in this
group display complex multimodal courtship incorporating
temporally coordinated combinations of motion displays and
vibratory songs—each of which consists of multiple ele-
ments (Elias et al. 2012). An analysis of high-speed video
in conjunction with laser vibrometry in the jumping spider
Habronattus dossenus was the first to document such tem-
poral coupling of visual and seismic courtship components
and signal ablation experiments confirmed that the temporal
coordination is not due to a common production mechanism
(Elias et al. 2003). A similar experimental approach docu-
mented the independent production mechanisms of the mul-
ticomponent seismic courtship display of the wolf spider
Schizocosa stridulans which is produced in conjunction with
an intermittent visual display (Elias et al. 2006). In the above
examples, knowledge of signal production mechanisms pro-
vides information on the facultative obligate nature of tem-
porally coupled signal components and confirms that each
component is independently produced and thus not inherent-
ly multimodal (i.e., not “fixed”) in the same way as the
expression of a frog’s croak or a human word. While this
distinction (“fixed” versus “free”) may be extremely impor-
tant in terms of understanding the evolutionary origin of
multimodal displays, it does not necessarily have major
significance for hypotheses aimed at understanding the cur-
rent function of multimodal displays. Nonetheless, this dis-
tinction remains important, as a focus on “free” multimodal
1383
displays could provide significant insights into the potential-
ly significant implications of simultaneous versus sequential
signal assessment (see also Uy and Safran 2013).
Finally, we highlight the distinction between multimodal
signaling and multimodal communication. These terms are
often treated as synonymous, but the latter term is wider in
scope as it incorporates cues, where there has not been past
selective pressure on the trait specifically for a signaling
function (Maynard Smith and Harper 2003; Ruxton and
Schaefer 2011). Though we have used the term multimodal
signaling commonly here, we certainly accept that cues may
be critical elements of many multimodal communication
systems—even in some of our incorporated examples.
Distinguishing signals from cues in multimodal communi-
cation systems is not trivial. In the above example of
a croaking frog, the signal contains elements in two
modalities—the acoustic call and the visual cue of the inflat-
ing and deflating throat sack. Though the visual element of
the signal is demonstrably salient to females (Taylor et al.
2008; Grafe et al. 2012; Preininger et al. 2013), this is not
sufficient evidence that this is a visual signal. The throat sack
did not presumably originally evolve for a communicative
function (unlike the vocalization that it accompanies) given
that it is a necessary component of acoustical signal produc-
tion. Correctly defining the visual element as a cue or a
signal seems very difficult, and would require determining
whether the visual element seems to be under separate se-
lection for increased expression relative to the amount of
noise produced. However, if that were the case, the visual
signal might no longer be useful as an index of call ampli-
tude, and so the signal would cease to be reliably informative
to receivers. As mentioned earlier, elements of a cue may
also be coopted over evolutionary time for a signaling func-
tion. For example, Tungara frogs Physalaemus pustulosus at
one population in Panama express natural variation in the
conspicuousness of a white stripe on their vocal sac,
suggesting a potential signaling function. However, no sup-
port for such a signaling function has yet been found as
tested females did not distinguish between robofrogs with
or without conspicuous white stripes (Taylor et al. 2011).
Conceptual frameworks for multimodal signals
The first conceptual framework for classifying multimodal
signals was proposed by Partan and Marler (1999a, 2005)
and involved a classification system whereby multimodal
signals were defined by receiver responses to isolated versus
combined components. This classification scheme built on
work by others, which classified multiple signals as redundant
or non-redundant depending on signal information content
(Johnstone 1996). Partan and Marler’s (1999a) extension of
such ideas further elaborated redundant signals (where the
1384
information content is the same) into ‘equivalent’ signals,
where the response intensity is unchanged, and ‘enhancement’
signals, where the response intensity is increased. Partan and
Marler (1999a) also split non-redundant signals (where the
information content differs) into ‘independent’—separate re-
sponses to each signal, ‘dominant’—one signal dominates the
other, ‘modulation’—one signal modulates the other, and
‘emergent’—an entirely new response. This framework has
proved extremely useful in many cases, but may not be
sufficient on its own to incorporate the variation seen in all
signaling systems. For example, consider the following find-
ing: receiver response to component ‘a’ → nothing; receiver
response to component ‘b’ → ‘↔ ’; receiver response to ‘a +
b’→ ‘□’ or ‘◊’. Examples of such patterns of receiver response
are not uncommon in wolf spiders, where receivers may only
attend to variation in one component (visual ornamentation) in
the presence of a second component (seismic courtship sig-
naling) (Hebets 2005; Wilgers and Hebets 2011; Stafstrom
and Hebets 2013). As Rowe (1999) points out, such inter-
signal interactions may not necessarily always fit easily into a
redundant/non-redundant framework (though see response by
Partan and Marler 1999b). Other frameworks have been pro-
posed that are not based on redundant/non-redundant distinc-
tions but that have greater focus on inter-signal interactions
and signal efficacy (e.g., Hebets and Papaj 2005). We feel that
some of the central issues that will have to be addressed by
any new frameworks include issues of signal efficacy and
receiver perception. These issues include how different re-
ceivers might respond differently to the same signal based
on variability in receiver sensory systems (Ronald et al. 2012)
and prior experience (e.g., Hebets and Vink 2007), as well as
how the same receiver may respond differently to the same
signal depending on differences in the environment where it is
detected, and the wider ecological context (Munoz and
Blumstein 2012).
Is multimodal communication special?
Multicomponent and multimodal communication are often
treated as similar and related phenomena. Multicomponent
and complex signaling can occur within one sensory modal-
ity, and frequently do, and when multiple components within
one sensory modality are received and/or processed differ-
ently these categories may indeed be equivalent. When we
imagine the varied shapes, patterns, and colors that make up
the striking plumage of a bird of paradise, or the face of the
male mandrill, these multicomponent aspects of the signal
can be studied both separately, and together, to determine the
extent to which they form the overall visual scene (e.g., the
mandrill’s face, Renoult et al. 2011). When doing so, we may
be able to use the same conceptual frameworks that were
developed for multimodal signals (e.g., Partan and Marler
Behav Ecol Sociobiol (2013) 67:1381–1388
1999a, 2005; Candolin 2003; Hebets and Papaj 2005) to
assess the effects of the second component on the receiver’s
response to the first. It is also worth pointing out that the
main theoretical approaches that have been used to address
the evolution of multimodal signals, such as game theory, are
by their nature unable to distinguish questions of multiple
channels from questions of multiple modalities. As Wilson
et al. point out (2013), though we can think of arguments
such as orthogonal noise between signaling channels, which
we biologically might assume to be much more likely across
modalities than within one modality, we can only model this
as orthogonal noise between multiple channels per se. It is
left to our understanding of what such constraints might
look like biologically to conclude that this is likely to lead
to multimodal signaling specifically over multicomponent
signaling.
Despite this, there are a number of reasons why multi-
modal signals should be considered differently to multicom-
ponent or complex signaling within the same modality. To
begin with, there are three closely related efficacy issues of
signal structure and permanence, environmental noise, and
sensory systems, all of which are more likely to differ be-
tween than within modalities. Firstly, signals expressed in
different modalities have very different transmission dis-
tances, and different permanence. An olfactory signal usual-
ly travels slowly through a forest, but mammalian olfactory
signals deposited on a branch may be present several days
later (Heymann 2006). In contrast vocal and visual signals
travel fast but are transient, and are very differently
constrained by environmental noise. This is a second, but
closely related point, which is that the noise experienced by a
signal in an environment is likely to be quite similar within
one modality but very different between them. A visual
signal may travel far in a perfect environment before it de-
grades, but may often be blocked by foliage, clouds, or other
physical barriers, which can include general cloudiness in
the air or water that interfere with signal transmission. Other
visual signals would suffer from the same obstacles.
However, a vocal signal may feature little disruption from
such physical barriers, though the presence of other acoustic
noise from the physical or social environment (an entirely
different set of environmental noise) will prevent transmis-
sion in this modality. Again relatedly, a third issue is that
differences in the sensory systems that are used to detect
different signals mean that each element in a multimodal
signal has a specific set of detection issues. This means that
even from close range within a clear environment different
modalities can enable different signals to be sent in both
private (sent to a specific receiver) and public (sent to nu-
merous receivers) channels. Given that many animals com-
municate within social environments of multiple conspecific
and heterospecific eavesdroppers, this latter point may be very
important. Multicomponent signals where all components
Behav Ecol Sociobiol (2013) 67:1381–1388
are detected by the same sensory system offer far less
scope for private and public messages to be given simul-
taneously (Wilson et al. 2013).
In addition to the above issues, because it is often phys-
ically easier for different types of underlying qualities to be
signaled through different modalities, multimodal signals
offer more potential for multiple quality signaling compared
to multicomponent signaling in the same modality. Variation
in physiology, such as in the production of specific steroid
hormones, may easily be converted into excreted chemical
signals, because those chemicals are naturally being pro-
duced and excreted anyway (e.g., in human pheromonal
communication, Kohl et al. 2001). Similarly, when a bird
collects carotenoids that have the natural effect of reddening
the tissue it builds (e.g., Price 2006), their carotenoid-
collecting ability (hence potentially their quality) seems like-
ly to be easier to signal honestly in a color signal than a vocal
signal. Such natural links between underlying state and sig-
nal form are common, and when multiple qualities of differ-
ent types are being signaled, multimodal rather than
unimodal multicomponent signaling may be much more
likely (see also Wilson et al. 2013). Finally, it may be that
there are cognitive reasons, such as effects on learning and
memory, which may make multimodal signals specifically
more memorable and therefore more likely to become fixed
into populations (e.g., humans, Lovelace et al. 2003;
jumping spiders, VanderSal and Hebets 2007; frogs, Akre
and Ryan 2010; and Rowe and Halpin 2013).
Current research on multimodal communication
and issue contents
There are a number of major foci to current research on
multimodal signaling, and these can be illustrated by the
contents of the present issue. Smith and Evans (2013) intro-
duce a new heuristic for assessing and presenting descrip-
tions of multimodal signals, building on the framework of
Partan and Marler (1999a, 2005). Following this, Wilson
et al. present a discussion of game theoretic approaches to
multimodal communication, using modeling approaches to
ask questions about why animals signal in multiple modali-
ties. This is a question also addressed by Rowe and Halpin,
who assess several hypotheses for why warning displays
may have evolved to be multimodal. Together with Ruxton
and Schaefer’s interesting commentary on Wilson et al.,
which includes ideas on how to test some of Wilson et al.’s
theoretical elaborations, these manuscripts collectively illus-
trative the need for further conceptual and theoretical de-
velopments in order to ensure that we are able to classify and
consider multimodal signals appropriately, while under-
standing the likely selective pressures that have made multi-
modal communication so common.
1385
Several contributions represent empirical species-specific
studies of multimodal communication, and illustrate the vari-
ety of methodologies and approaches that can be employed in
studies of multimodal signals (Preininger et al. 2013; Higham
et al. 2013; Uetz et al. 2013). These manuscripts incorporate
consideration of signal efficacy issues, by addressing ques-
tions about environmental noise (Preininger et al. 2013) or by
incorporating measures and models of receiver sensory per-
ception (Higham et al. 2013; Uetz et al. 2013). Several studies
go beyond the species-specific, to ask comparative questions
about the evolution of multimodal traits (Ghazanfar 2013;
Hebets et al. 2013; Rowe and Halpin 2013), the role of
different selective pressures on multimodal signal evolution
(Hebets et al. 2013; Ossip-Klein et al 2013.) and the role of
multimodal signals themselves in species evolution (Uy and
Safran 2013). Investigation of multimodal signals from com-
parative perspectives is a key emerging area of research, and is
likely to become more common as data for more species
becomes available.
Prospects for research on multimodal communication
One of the primary aims of the present issue is to stimulate
more studies, both empirical and theoretical, on multimodal
signaling systems. In the closing contribution to the present
issue, Partan (2013) does an outstanding job of setting an
agenda for the field. Though it is not therefore our aim to give
a comprehensive list of future research avenues here, there are
nonetheless a few areas we would also like to consider here.
First, as discussed above, an important point to consider is
that most conceptual frameworks posited for understanding
multimodal signal outcomes are based on how the receiver’s
response to one signal is influenced by another (see above),
and the majority of conceptual and empirical studies to date
have focused on bimodal signals. It is difficult to think of
many signals that are in themselves more than bimodal (such
as the croak of a frog and human speech outlined above).
However, there are many examples of displays that may be at
least trimodal. For example, the blue crab, Callinectes
sapidus, has a male courtship display that incorporates visu-
al, chemical, and hydrodynamic components into a trimodal
signal (Gleeson 1991; Bushmann 1999; Kamio et al. 2008).
Many communication systems exhibit multimodal complex-
ity that goes far beyond bimodality. For example male ba-
boons are highly aroused by the visual signal of female
sexual swellings (which themselves vary in multiple compo-
nents such as size, shape, symmetry, and color), which they
then touch (possibly to assess how inflated the swelling is),
before sniffing the vagina and its secretions, which they then
taste. If the male chooses to mate with the female, she is
likely to give a copulation call during coitus, which may
indicate further aspects of her status to this and other males.
1386
The male has, within a matter of seconds, been exposed to
signals and cues from the female that have stimulated his
senses of vision, touch, olfaction, taste, and audition. It will
be a challenge to incorporate such complexity into existing
theoretical frameworks, but one that must be overcome be-
fore we can consider the complexities of multimodal com-
munication fully.
For most communication systems we seem a long way off
having either the theoretical frameworks or the empirical
ability to measure all these signals and cues, and to be able
to conduct the types of experiments that would be necessary
to assess their function fully. In our baboon example, for
correct assessment of the effect that each signal or cue has
independently and combined, experiments would need to be
performed in which each separate element, as well as each
possible combination, is presented to male receivers. If we
were to consider that signal presentation order matters, this
gives a total of 325 different experiments that would need to
be undertaken [5!/(5–4)!+5!/(5–3)!+5!/(5–2)!+5!/(5–1)!],
with each set of a few experiments possibly being around
the appropriate workload for a full PhD thesis. Given the
constraints, the problem currently seems intractable.
However, this does not mean that we should not renew our
attempts to increase studies beyond bimodality where appro-
priate. We are likely to need new conceptual and methodo-
logical developments to begin this process.
A second important area for future attention is classifica-
tion. Current classification schemes tend to focus on signal
function, and there is a need to incorporate more elements of
signal efficacy into existing frameworks and theory. Many
signals have similar functions in very disparate species,
including mate choice, social status signaling, the avoidance
of predation, and so on. However, this commonality of
function does not explain differences in evolutionary signal
design. Instead, the bewildering diversity of signal form is
often related to efficacy, with the need to influence receivers
with very different sensory systems in different types of
environments. As stated above, the incorporation of issues
of sensory perception (including Higham et al. 2013; Uetz
et al. 2013) and environmental noise (e.g., Preininger et al.
2013) into empirical studies is becoming more common-
place. We encourage the development of communication
frameworks that incorporate more aspects of signal efficacy,
including of variability in receiver perception (Ronald et al.
2012), and of variability in the environment (Bro-Jorgensen
2010; Munoz and Blumstein 2012).
Third, we need more theoretical work on why an animal
should signal in multiple modalities. Wilson et al. (2013)
have laid out several areas that look highly promising for
future research in this regard, and have highlighted several
theoretical models in economics that appear to have great
potential for application to biological systems. Further elab-
oration of these and similar models will help to clarify the
Behav Ecol Sociobiol (2013) 67:1381–1388
adaptive benefits that have led animal communication to
become so complex.
Fourth, though the present special issue primarily focuses
on behavior, there is a large amount of work underway on the
neuroscience of sensory integration (see Stein et al. 2012 for
an overview). Increasingly, some of this work lies at the
interface of behavior and neuroscience. For example, re-
search on rhesus macaques has investigated how sensorially
ingrained the processing of different signals from acoustic
and visual modalities is within the brain, revealing that
sensory integration occurs very early in cognitive processing
(Ghazanfar and Schroeder 2006). Such studies are extremely
useful in indicating that both elements of the signal are
highly important, and that they are likely to have evolved
together, which has implications for our understanding of
how a multimodal signal may function (sequentially or si-
multaneously), and the likely selective pressures that have
acted upon them. The body of neuroscience work focusing
on signal integration in the brain needs to be considered more
in laboratory and field studies of behavior.
Fifth, the role that signals and related sexually selected
traits play in population level processes, and in creating and
maintaining species, remains largely unclear. Sexual selec-
tion and mate choice are thought to be key driving forces in
speciation (e.g., insects, Rodríguez and Cocroft 2006,
Rodríguez et al. 2006; indigobirds, Sorenson et al. 2003;
sticklebacks, Boughman 2001; cichlid fish, Maan and
Seehausen 2010), and communication is fundamental to all
aspects of mate choice (Andersson 1994). Uy and Safran’s
contribution to the present issue discusses these issues at
length, but much work remains to be done.
Finally, it is important to note that there are other issues to
consider when determining how receivers respond to animal
signals. These include audience/bystander effects, where
signals are given differently (Slocombe and Zuberbühler
2007; Slocombe et al. 2010) and responses altered (Semple
et al. 2009) depending on the surrounding social audience.
They also include the importance of both the general prior
experiences of the receiver (e.g., Hebets and Vink 2007), but
also specific experience (familiarity) between particular sig-
nalers and receivers, which can affect the response of re-
ceivers to the signal (e.g., Higham et al. 2011). Issues like
audience effects and signaler–receiver familiarity are not yet
well studied or understood for most systems, but are likely to
increase the complexity of communicative interactions even
further (see also discussion in Stevens 2013).
Conclusion
Recent years have seen a great increase in the number of
animal communication studies incorporating aspects of mul-
timodal signaling into their analyses. The present issue seeks
Behav Ecol Sociobiol (2013) 67:1381–1388
to stand as a reference to researchers interested in the area,
highlight recent advances made, and discuss the direction in
which studies of multimodal communication are heading.
The field has made great strides in its conceptual and empir-
ical assessments of animal signals, but as we continue to
incorporate increasing signal complexity and efficacy issues
related to the environment and receiver sensory perception
into our models, we will undoubtedly require new frameworks
and empirical approaches to appreciate animal signals fully.
There are exciting times ahead in animal communication.
Acknowledgments We are very grateful to Esteban Fernandez-Juricic
and two anonymous reviewers for comments on a previous draft of this
manuscript. We extend our special thanks to Theo Bakker, James
Traniello, and Saskia Hesse for their considerable help with the produc-
tion of this special issue.
References
Akre KL, Ryan MJ (2010) Complexity increases working memory for
mating signals. Curr Biol 20:502–505
Andersson M (1994) Sexual selection. Princeton University Press,
Princeton
Boughman JW (2001) Divergent sexual selection enhances reproduc-
tive isolation in sticklebacks. Nature 411:944–948
Bro-Jorgensen J (2010) Dynamics of multiple signalling systems: ani-
mal communication in a world in flux. Trends Ecol Evol 25:292–
300
Bushmann PJ (1999) Concurrent signals and behavioral plasticity in
blue crab (Callinectes sapiduss Rathburn) courtship. Biol Bull
197:63–71
Candolin U (2003) The use of multiple cues in mate choice. Biol Rev
78:575–595
Dambly-Chaudière C, Sapède D, Soubrian F, Decorde K, Gompel N,
Ghysen A (2003) The lateral line of zebrafish, a model system for
the analysis of morphogenesis and neural development in verte-
brates. Biol Cell 95:579–587
Darwin C (1872) The expression of the emotions in man and animals.
John Murray, London
Elias DO, Mason AC, Maddison WP, Hoy RR (2003) Seismic signals in
a courting male jumping spider (Araneae: Salticidae). J Exp Biol
206:4029–4039
Elias DO, Lee N, Hebets EA, Mason AC (2006) Seismic signal pro-
duction in a wolf spider: parallel versus serial multicomponent
signals. J Exp Biol 209:1074–1084
Elias DO, Maddison WP, Peckmezian C, Girard MB, Mason AC (2012)
Orchestrating the score: complex multimodal courtship in the
Coecatus-group of Habronattus (Araneae: Salticidae). Biol J Linn
Soc 105:522–547
Ghazanfar AA (2013) Multisensory vocal communication in primates
and the evolution of rhythmic speech. Behav Ecol Sociobiol.
doi:10.1007/s00265-013-1491-z
Ghazanfar AA, Logothetis NK (2003) Facial expressions linked to
monkey calls. Nature 423:937–938
Ghazanfar AA, Schroeder CE (2006) Is the neocortex essentially mul-
tisensory? Trends Cogn Sci 10:278–285
Gleeson RA (1991) Intrinsic factors mediating pheromone communi-
cation in the blue crab, Callinectes sapidus. In: Martin JW, Bauer
RT (eds) Crustacean sexual biology. Columbia University Press,
New York, pp 17–32
1387
Grafe TU, Preininger D, Sztatecsny M, Kasah R, Dehling JM, Proksch
S, Hödl W (2012) Multimodal communication in a noisy environ-
ment: a case study of the Bornean rock frog Staurois parvus. PLoS
One 7:e37965
Hebets EA (2005) Attention-altering signal interactions in the multi-
modal courtship display of the wolf spider Schizocosa uetzi. Behav
Ecol 16:75–82
Hebets EA (2011) Current status and future directions of research in
complex signaling. Curr Zool 57:i–v
Hebets EA, Papaj DR (2005) Complex signal function: developing a
testable framework for testable hypotheses. Behav Ecol Sociobiol
57:197–214
Hebets EA, Rundus AS (2011) Chemical communication in a multi-
modal context. In: Breithaupt T, Thiel M (eds) Chemical commu-
nication in crustaceans. Springer, New York, pp 335–354
Hebets EA, Vink CJ (2007) Experience leads to preference: experienced
females prefer brush-legged males in a population of syntopic wolf
spiders. Behav Ecol 18:1010–1020
Hebets EA, Vink CJ, Sullivan-Beckers L, Rosenthal MF (2013) The
dominance of seismic signaling and selection for signal complex-
ity in Schizocosa multimodal courtship displays. Behav Ecol and
Sociobiol. http://link.springer.com/article/10.1007/s00265-013-1519-4
Heymann EW (2006) The neglected sense—olfaction in primate be-
havior, ecology, and evolution. Am J Primatol 68:519–524
Higham JP, Hughes KD, Brent LJN, Dubuc C, Engelhardt A, Heistermann
M, Maestripieri D, Santos LR, Stevens M (2011) Familiarity affects
assessment of facial signals of female fertility by free-ranging male
rhesus macaques. Proc R Soc Lond B 278:3452–3458
Higham JP, Pfefferle D, Heistermann M, Maestripieri D, Stevens M
(2013) Signaling in multiple modalities in male rhesus macaques:
sex skin coloration and barks in relation to androgen levels, social
status, and mating behavior. Behav Ecol Sociobiol. doi:10.1007/
s00265-013-1521-x
Johnstone RA (1996) Multiple displays in animal communication: 'backup
signals' and 'multiple messages'. Philos T Roy Soc B 351:329–338
Kamio M, Reidenbach MA, Derby CD (2008) To paddle or not: context
dependent courtship display by male blue crabs, Callinectes
sapidus. J Exp Biol 211:1243–1248
Kohl JV, Atzmueller M, Fink B, Grammer K (2001) Human phero-
mones: integrating neuroendocrinology and ethology. Neuro
Endocrinol Lett 22:309–321
Lovelace CT, Stein BE, Wallace MT (2003) An irrelevant light en-
hances auditory detection in humans: a psychophysical analysis of
multisensory integration in stimulus detection. Cogn Brain Res
17:447–453
Maan ME, Seehausen O (2010) Mechanisms of species divergence
through visual adaptation and sexual selection – perspectives from
a cichlid model system. Curr Zool 56:285–299
Marler P (1961) The logical analysis of animal communication. J Theor
Biol 1:295–317
Marler P (1965) Communication in monkeys and apes. In: DeVore I
(ed) Primate behavior: field studies of monkeys and apes. Holt,
Rinehart and Winston, New York, pp 544–584
Maynard Smith J, Harper DGC (2003) Animal signals. Oxford Univer-
sity Press, Oxford
McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature
264:746–748
Munoz NE, Blumstein DT (2012) Multisensory perception in uncertain
environments. Behav Ecol 23:457–462
Ossip-Klein AG, Fuentes JA, Hews DK, Martins EP (2013) Information
content is more important than sensory system or physical distance
in guiding the long-term evolutionary relationships between signal-
ing modalities in Sceloporus lizards. Behav Ecol and Sociobiol.
http://link.springer.com/article/10.1007/s00265-013-1535-4
Partan SR, Marler P (1999a) Communication goes multimodal. Science
283:1272–1273
1388
Partan SR, Marler P (1999b) Response to Rowe. Science 284:744
Partan SR, Marler P (2005) Issues in the classification of multisensory
communication signals. Am Nat 166:231–245
Partan SR (2013) Ten unanswered questions in multimodal communi-
cation. Behav Ecol Sociobiol. doi:10.1007/s00265-013-1565-y
Preininger D, Boeckle M, Freudmann A, Starnberger I, Sztatecsny M,
Hödl W (2013) Multimodal signaling in the Small Torrent Frog
(Micrixalus saxicola) in a complex acoustic environment. Behav
Ecol Sociobiol. doi:10.1007/s00265-013-1489-6
Price TD (2006) Phenotypic plasticity, sexual selection and the evolu-
tion of colour patterns. J Exp Biol 209:2368–2376
Renoult JP, Schaefer HM, Sallé B, Charpantier MJE (2011) The evolu-
tion of the multicoloured face of mandrills: insights from the
perceptual space of colour vision. PLoS One 6:e29117
Rodríguez RL, Cocroft RB (2006) Divergence in female duetting
signals in the Enchenopa binotata species complex of treehoppers
(Hemiptera: Membracidae). Ethology 112:1231–1238
Rodríguez RL, Ramaswamy K, Cocroft RB (2006) Evidence that female
preferences have shaped male signal evolution in a clade of special-
ized plant-feeding insects. Proc R Soc Lond B 273:2585–2593
Ronald KL, Fernández-Juricic E, Lucas JR (2012) Taking the sensory
approach: how individual differences in sensory perception can
influence mate choice. Anim Behav 84:1283–1294
Rowe C (1999) One signal or two? Science 284:743–744
Rowe C, Guildford T (1996) Hidden colour aversions in domestic
chicks triggered by pyrazine odours of insect warning displays.
Nature 383:520–522
Rowe C, Halpin C (2013) Why are warning displays multimodal?
Behav Ecol Sociobiol. doi:10.1007/s00265-013-1515-8
Ruxton GD, Schaefer HM (2011) Resolving current disagreements and
ambiguities in the terminology of animal communication. J Evol
Biol 24:2574–2585
Semple S, Gerald M, Suggs D (2009) Bystanders affect the outcome of
mother-infant interactions in rhesus macaques. Proc R Soc Lond B
276:2257–2262
Slocombe KE, Zuberbühler K (2007) Chimpanzees modify recruitment
screams as a function of audience composition. Proc Natl Acad Sci
U S A 104:17228–17233
Slocombe KE, Kaller T, Turman L, Townsend SW, Papworth S,
Zuberbühler K (2010) Production of food-associated calls in wild
View publication stats
Behav Ecol Sociobiol (2013) 67:1381–1388
male chimpanzees is dependent on the composition of the audi-
ence. Behav Ecol Sociobiol 64:1959–1966
Smith WJ (1977) The behavior of communicating: an ethological
approach. Harvard University Press, Cambridge
Smith CL, Evans CS (2013) A new heuristic for capturing the com-
plexity of multimodal signals. Behav Ecol Sociobiol. doi:10.1007/
s00265-013-1490-0
Sorenson MD, Sefc KM, Payne RB (2003) Speciation by host switch in
brood parasitic indigobirds. Nature 424:928–931
Stafstrom JA, Hebets EA (2013) Female mate choice for multimodal
courtship and the importance of the signaling background for
selection on male ornamentation. Curr Zool 59:200–209
Stein BE (ed) (2012) The new handbook of multisensory integration.
Massachusetts Institute of Technology Press, Cambridge
Stevens M (2013) Sensory ecology, behavior and evolution. Cambridge
University Press, Cambridge
Taylor RC, Klein BA, Stein J, Ryan MJ (2008) Faux frogs: multi-modal
signalling and the value of robotics in animal behavior. Anim
Behav 76:1089–1097
Taylor RC, Klein BA, Stein J, Ryan MJ (2011) Multimodal signal
variation in space and time: how important is matching a signal
with its signaler? J Exp Biol 214:815–820
Uetz GW, Stratton GE (1983) Communication in spiders. Endeavour
7:13–18
Uetz GW, Roberts JA, Clark DL, Gibson JS, Gordon SD (2013)
Multimodal signals increase active space of communication by
wolf spiders in a complex litter environment. Behav Ecol
Sociobiol. doi:10.1007/s00265-013-1557-y
Uy JAC, Safran RJ (2013) Variation in the temporal and spatial use of
signals and its implications for multimodal communication. Behav
Ecol Sociobiol. doi:10.1007/s00265-013-1492-y
VanderSal ND, Hebets EA (2007) Cross-modal effects on learning: a
seismic stimulus improves color discrimination learning in a
jumping spider. J Exp Biol 210:3689–3695
Wilgers D, Hebets EA (2011) Complex courtship displays facilitate
male reproductive success and plasticity in signaling across vari-
able environments. Curr Zool 57:175–186
Wilson AJ, Dean M, Higham JP (2013) A game theoretic approach to
multimodal communication. Behav Ecol Sociobiol. doi:10.1007/
s00265-013-1589-3