Professional Documents
Culture Documents
Author manuscript
Autism Res. Author manuscript; available in PMC 2021 September 01.
Author Manuscript
Jeremy Veenstra-VanderWeele,
Department of Psychiatry, Columbia University, Center for Autism and the Developing Brain, New
Author Manuscript
York Presbyterian Hospital, and New York State Psychiatric Institute, New York, New York
Mark T. Wallace
Department of Psychiatry, Vanderbilt University, Nashville, Tennessee; Department of
Psychology, Vanderbilt University, Nashville, Tennessee; Department of Hearing and Speech
Sciences, Vanderbilt University, Nashville, Tennessee; Kennedy Center for Research on Human
Development, Vanderbilt University, Nashville, Tennessee
Abstract
Abnormal sensory responses are a DSM-5 symptom of autism spectrum disorder (ASD), and
research findings demonstrate altered sensory processing in ASD. Beyond difficulties with
processing information within single sensory domains, including both hypersensitivity and
Author Manuscript
hyposensitivity, difficulties in multisensory processing are becoming a core issue of focus in ASD.
These difficulties may be targeted by treatment approaches such as “sensory integration,” which is
frequently applied in autism treatment but not yet based on clear evidence. Recently,
psychophysical data have emerged to demonstrate multisensory deficits in some children with
ASD. Unlike deficits in social communication, which are best understood in humans, sensory and
multisensory changes offer a tractable marker of circuit dysfunction that is more easily translated
into animal model systems to probe the underlying neurobiological mechanisms. Paralleling
experimental paradigms that were previously applied in humans and larger mammals, we and
others have demonstrated that multisensory function can also be examined behaviorally in rodents.
Here, we review the sensory and multisensory difficulties commonly found in ASD, examining
laboratory findings that relate these findings across species. Next, we discuss the known
neurobiology of multisensory integration, drawing largely on experimental work in larger
Author Manuscript
mammals, and extensions of these paradigms into rodents. Finally, we describe emerging
investigations into multisensory processing in genetic mouse models related to autism risk. By
detailing findings from humans to mice, we highlight the advantage of multisensory paradigms
that can be easily translated across species, as well as the potential for rodent experimental
systems to reveal opportunities for novel treatments.
Lay Summary:
Address for correspondence and reprints: Justin K. Siemann, Department of Biological Sciences, Vanderbilt University, 8270 MRB III
BioSci Bldg, 465 21st Ave South, Nashville, TN 37235. justin.k.siemann@vanderbilt.edu.
Siemann et al. Page 2
Sensory and multisensory deficits are commonly found in ASD and may result in cascading
Author Manuscript
effects that impact social communication. By using similar experiments to those in humans, we
discuss how studies in animal models may allow an understanding of the brain mechanisms that
underlie difficulties in multisensory integration, with the ultimate goal of developing new
treatments.
Keywords
multisensory integration; autism spectrum disorder; mouse models; serotonin; visual processing;
auditory processing
Introduction
Autism spectrum disorder (ASD) represents a behaviorally defined cluster of complex
Author Manuscript
There is clear evidence that sensory processing across multiple modalities (sight, sound,
touch, smell, and taste) is impacted in ASD [Baum, Stevenson, & Wallace, 2015a;
Robertson & Baron-Cohen, 2017]. Studies have shown that individuals with ASD can be
Author Manuscript
of improving adaptive behaviors, along with interventions focused on activating the arousal
system with more passive inclusion of sensory information in the child’s normal routine,
which may ameliorate some of the observed atypical responses to sensory stimuli (i.e.
hyper- or hypo-sensitivity) [Ayres, 1972; Case-Smith, Weaver, & Fristad, 2015]. In addition
to the Ayres Sensory Integration intervention, studies have evaluated these sensory-based
therapies with mixed results, largely concluding there is little efficacy of these treatments or
at least more evidence is still needed [Case-Smith et al., 2015; Lang et al., 2012; Schaaf,
Dumont, Arbesman, & May-Benson, 2018; Watling & Hauer, 2015]. In order to apply novel
responsible for the atypical sensory responses consistently observed in ASD. Toward this
end, investigators have recently begun to harness studies on the neuronal, systems, and
behavioral levels to evaluate performance not only in response to stimuli within a single
sensory modality, but also when stimuli from multiple modalities are present [Baum et al.,
2015a].
Investigations in both humans and animals have shown that the combination of information
from multiple senses can result in behavioral enhancements, such as increased accuracies
and reduced reaction times, when compared to behavioral performances under unisensory
(i.e. visual, auditory, or tactile alone) conditions [Stevenson et al., 2014]. For example,
human psychophysical studies have shown that the added visual information of seeing a
speaker’s lip movements can facilitate speech intelligibility in a noisy environment [Sumby
& Pollack, 1954]. In addition to this work in the behavioral and perceptual realms, a great
deal of work has now characterized the neural circuits and processes that underlie
multisensory integration [Stein & Stanford, 2008]. Collectively, this work has shown that
multisensory inputs converge at many sites in the nervous system [Meredith, 2002; Wallace,
Author Manuscript
Meredith, & Stein, 1992; Wallace, Ramachandran, & Stein, 2004] and that this convergence
frequently results in dramatic changes in neuronal responses when stimuli from multiple
modalities are presented collectively [Meredith & Stein, 1983].
Cascio, Stone, & Wallace, 2011; Marco, Hinkley, Hill, & Nagarajan, 2011; Russo et al.,
2010; Stevenson, Segers, Ferber, Barense, & Wallace, 2014; Wallace & Stevenson, 2014].
Evidence of decreased multisensory gain or benefit has been reported utilizing both simple
and complex stimuli [Collignon et al., 2013; Mongillo et al., 2008; Stevenson et al., 2014;
Stevenson et al., 2014]. Further, these multisensory deficits have been correlated with
symptom severity [Brandwein et al., 2015; Yaguchi & Hidaka, 2018], and the development
of multisensory function and its neural underpinnings may be atypical in ASD as well
[Beker et al., 2018; Brandwein et al., 2012; Cuppini et al., 2017; Foxe et al., 2015;
Author Manuscript
One common method to measure multisensory integration is with the use of cross-modal
illusions. The McGurk effect is a well-known audiovisual speech illusion that has been
evaluated in children, adults, as well as in individuals with neurodevelopmental disorders
[Mcgurk & Macdonald, 1976; van Wassenhove, Grant, & Poeppel, 2007; Woynaroski et al.,
2013]. In this illusion, a speaker typically mouths the phoneme “ga” while viewers
simultaneously hear an auditory “ba.” Under these conflicting audiovisual conditions,
individuals tend to perceive an entirely new speech percept, typically reporting that the
speaker uttered the phoneme “da” or “tha.” Under these illusory conditions, it is believed
that subjects are integrating the visual “ga” with the auditory “ba” to create the novel percept
[Mcgurk & Macdonald, 1976]. Brain imaging studies have demonstrated that the superior
temporal sulcus (STS) is integral for the perception of the McGurk effect [Nath &
Author Manuscript
Beauchamp, 2012; Nath, Fava, & Beauchamp, 2011], and when this brain region is disrupted
by transcranial magnetic stimulation (TMS) in typically developing individuals, illusory
percepts decrease dramatically [Beauchamp, Nath, & Pasalar, 2010].
A variety of studies has also evaluated the McGurk effect in clinical populations, including
in those with ASD. Overall, studies have demonstrated that individuals with ASD tend to
perceive the McGurk effect less frequently in comparison to TD individuals [Irwin,
Tornatore, Brancazio, & Whalen, 2011; Mongillo et al., 2008; Stevenson, Siemann, et al.,
2013; Stevenson, Siemann, Schneider, et al., 2014; Taylor et al., 2010], with the majority of
the studies describing this as a decrease in the strength or magnitude of multisensory
integration. In addition, studies have evaluated and found differences in the perception of the
McGurk effect across development in individuals with ASD [Stevenson, Siemann, et al.,
Author Manuscript
2013; Taylor et al., 2010]. Also, studies, utilizing various psychophysical behavioral
measures, have determined that the development of multisensory integration might be
delayed in these individuals [Beker et al., 2018; Brandwein et al., 2012; Foxe et al., 2015;
Stevenson, Siemann, et al., 2013; Taylor et al., 2010], along with evidence of sex-dependent
differences in audiovisual speech development in both typically developed and ASD
populations [Ross, Del Bene, Molholm, Frey, & Foxe, 2015]. Interestingly, neuroimaging
studies have found both structural and functional differences in the STS of those with ASD,
and this brain region, which has been shown to be important for the McGurk illusion, has
also been established as a neural hub for multisensory integration and social processing
[Beauchamp, Argall, Bodurka, Duyn, & Martin, 2004; Boddaert et al., 2004; Calvert,
Campbell, & Brammer, 2000; Gervais et al., 2004; Redcay, 2008; Stevenson & James, 2009;
Zilbovicius et al., 2006].
Author Manuscript
One of the critical factors in the integration of information across the different senses is
time, in that only when these stimuli are sufficiently close together in time are they
perceptually integrated. Specifically, there appears to be a window of time within which
events specified by the different sensory modalities are perceptually “bound” [Hillock,
Powers, & Wallace, 2011]. This multisensory temporal binding window (TBW) makes a
great deal of ethological sense, since sensory stimuli presented in close temporal proximity
are highly likely to be associated with the same event. To measure the TBW, auditory and
visual stimuli are presented at various asynchronies relative to one another (including at
Author Manuscript
simultaneity), and participants are asked to respond if they perceived these sensory signals
as synchronous or not [Stevenson, Ghose, Fister, et al., 2014]. Typically developing
participants tend to report high proportions of simultaneity for stimuli separated by about
300 ms, depending on the complexity of stimuli [Stevenson & Wallace, 2013]. However, it
has been shown that the TBW can be dependent on participant task demands [Mégevand,
Molholm, Nayak, & Foxe, 2013], audiovisual processing may be due to attentional
switching [Shaw et al., 2020], and these capabilities may be developmentally different for
typically developed individuals compared to individuals with ASD [Crosse, Foxe, &
Molholm, 2019]. These points highlight the fact that task design, motivation, and attentional
components are all highly relevant when selecting and utilizing psychophysical paradigms to
investigate multisensory processing, especially in clinical populations such as ASD. Lastly,
the width of the TBW has been shown to relate back to the perceptual illusions described
Author Manuscript
above, specifically in that individuals who have wider TBWs tend to report the McGurk
illusion less frequently [Stevenson, Zemtsov, & Wallace, 2012]. Such a finding suggests an
important relationship between the TBW and the strength or magnitude of the multisensory
binding process.
Emerging evidence suggests that the multisensory temporal binding window may be
abnormally wide in individuals with ASD [Baum et al., 2015a; Baum, Stevenson, &
Wallace, 2015b; Foss-Feig et al., 2010; Kwakye et al., 2011; Stevenson, Siemann,
Woynaroski, et al., 2014; Wallace & Stevenson, 2014], resulting in degraded perceptual
representations. For example, Stevenson and colleagues measured the TBW in both TD
children and individuals with ASD across three levels of stimulus complexity: simple visual
flashes and auditory beeps, more complex tool stimuli (i.e. a hammer hitting a nail) and a
speaker uttering speech syllables. Temporal binding windows increased in width as the
Author Manuscript
stimulus complexity increased for both groups, with the widest windows being observed for
speech stimuli. Significant differences in the TBW widths were observed between the
groups under the most complex conditions (i.e. speech), where individuals with ASD
demonstrated wider TBWs compared to TD individuals. Interestingly, no significant
differences in TBWs were observed between the groups for simple flashes and beeps and
tool stimuli. This suggests that multisensory temporal processing may be typical in ASD for
more simplistic audiovisual stimuli, however may become atypical as stimulus complexity
increases. An alternative interpretation is that by utilizing a speaker uttering syllables, there
may be intrinsic differences between the groups as there is evidence of atypical processing
of faces in ASD [Dalton et al., 2005; Dawson, Webb, & McPartland, 2005]. This study also
measured the McGurk effect, and individuals with ASD perceived the McGurk illusion less
frequently compared to TD individuals. Importantly, it was found that for individuals with
Author Manuscript
ASD, a wider TBW, regardless of the stimulus complexity, resulted in less perception of the
McGurk effect. Therefore, this study hypothesized those differences in multisensory
temporal processing for even low-level sensory stimuli (i.e. visual flashes and auditory
beeps) in individuals with ASD could result in potential cascading effects into higher order
domains such as speech or communication [Stevenson, Siemann, Schneider, et al., 2014].
In addition to the McGurk effect, the sound-induced flash illusion has also been examined in
individuals with ASD. In this illusion, one visual flash is paired with multiple auditory
beeps, resulting in the frequent reporting of multiple visual flashes [Shams, Kamitani, &
Author Manuscript
Shimojo, 2000]. Using this illusion, it was observed that children with ASD had a temporal
binding window that was almost twice as wide compared to TBWs of typically developing
controls [Foss-Feig et al., 2010].
In sum, numerous findings, using a variety of approaches, have demonstrated the presence of
multisensory and temporal processing deficits in individuals with autism spectrum disorder
[Baum et al., 2015b; Brandwein et al., 2015; Collignon et al., 2013; Foss-Feig et al., 2010;
Foxe et al., 2015; Irwin et al., 2011; Kwakye et al., 2011; Smith & Bennetto, 2007;
Stevenson, Siemann, et al., 2013; Stevenson, Siemann, Schneider, et al., 2014; Stevenson,
Siemann, Woynaroski, et al., 2014; Woynaroski et al., 2013]. In addition, there is emerging
evidence that (multi)sensory function may play a critical role in the establishment and
maintenance of more complex functions such as social communication [Baum et al., 2015a;
Robertson & Baron-Cohen, 2017; Stevenson et al., 2018; Stevenson, Siemann, Schneider, et
al., 2014; Thye, Bednarz, Herringshaw, Sartin, & Kana, 2018]. With mounting evidence of
atypical multisensory integration in ASD on both the behavioral and neural levels, cross-
species studies in animal models may allow for investigations to understand the underlying
Author Manuscript
neural mechanisms of multisensory processing, with the goal to translate that understanding
into testable hypotheses—and potential treatments—in the human population.
circuits and multisensory behaviors in a variety of species ranging from mice to non-human
primates [Cappe, Murray, Barone, & Rouiller, 2010; Hirokawa, Bosch, Sakata, Sakurai, &
Yamamori, 2008; Olcese et al., 2013; Wallace, Meredith, & Stein, 1998].
One of the best-studied models for this work has been the cat [Meredith & Stein, 1983].
Work in this model served to establish three main principles of multisensory integration
(spatial, temporal, and inverse effectiveness) dependent upon the physical characteristics of
the paired stimuli [Meredith, Nemitz, & Stein, 1987; Meredith & Stein, 1986a, 1986b]. The
Author Manuscript
spatial and temporal principles specify that stimuli from different sensory modalities
presented in close spatial and/or temporal proximity result in greater neural responses and
behavioral benefits compared to those presented in a disparate manner [Meredith et al.,
1987; Meredith & Stein, 1986a]. In addition, the principle of inverse effectiveness states that
as the effectiveness (i.e. loudness, brightness) of the unisensory stimuli decreases, the
resultant behavioral multisensory gain or benefit increases when stimuli are combined across
sensory modalities [Meredith & Stein, 1986b].
Two of the traditional means to measure and evaluate the gains conferred by multisensory
combinations are the interactive index (ii) and the mean statistical contrast (msc) [Stein &
Stanford, 2008]. The interactive index compares multisensory responses to the largest or
most effective unisensory response [Meredith & Stein, 1983, 1986a, 1986b]. The interactive
index can be either positive or negative depending on whether the stimulus combination
Author Manuscript
Multisensory processing can be further evaluated by the mean statistical contrast. This
compares the responses to multisensory stimuli to the combined responses from the
individual sensory modalities [Perrault, Vaughan, Stein, & Wallace, 2003, 2005; Stanford,
Quessy, & Stein, 2005]. Three types of responses (i.e. subadditive, additive, or
superadditive) can be observed as a result of this evaluation.
In the cat, two of the best-studied structures from a multisensory perspective are the deep
layers of a subcortical structure, the superior colliculus (SC) [Meredith et al., 1987;
Meredith & Stein, 1986a, 1986b], and the anterior ectosylvian sulcal (AES) cortex [Wallace
Author Manuscript
et al., 1992; Wallace, Carriere, Perrault, Vaughan, & Stein, 2006]. Intriguingly, it is the
projections from the AES to the SC that gates the integration seen in SC neurons [Wallace,
Meredith, & Stein, 1993; Wallace & Stein, 1994]. Pharmacological [Wilkinson, Meredith, &
Stein, 1996] and deactivation studies [Jiang et al., 2001; Jiang, Jiang, & Stein, 2002; Wallace
& Stein, 1994] show that the AES is critical for this gating, in addition to being integral for
the behavioral enhancements observed under multisensory conditions.
Beyond studies in the cat; neural, circuit, and behavioral-based studies in the non-human
primate have been crucial in understanding the underlying mechanisms of multisensory
processing [Cappe et al., 2010; Cappe & Barone, 2005; Maier, Neuhoff, Logothetis, &
Ghazanfar, 2004; Rockland & Ojima, 2003]. Striking similarities have been shown in both
the development [Wallace, 2004], and neural response properties [Wallace & Stein, 2001] of
multisensory integration in the cat and monkey [Wallace & Stein, 1996]. As in the cat, work
Author Manuscript
in the non-human primate has shown the superior colliculus to be a major subcortical hub
for multisensory integration, and neurons adhere to the established principles of
multisensory integration [i.e. space, time, and effectiveness) (Wallace, Wilkinson, & Stein,
1996]. These similarities in development and neural response properties, between cat and
non-human primate models, suggest that multisensory integration may be highly conserved
across larger animal species [Table 1] [Stein & Stanford, 2008].
There are a variety of cortical regions in the non-human primate shown to receive
Author Manuscript
convergent input from multiple sensory modalities, and to demonstrate many of the classical
multisensory features [Ghazanfar & Schroeder, 2006; Kajikawa, Falchier, Musacchia,
Lakatos, & Schroeder, 2012]. These include the prefrontal cortex [Fuster et al., 2000;
Romanski et al., 2005; Romanski & Goldman-Rakic, 2002; Sugihara et al., 2006], parietal
cortex [Avillac et al., 2007; Cohen, 2009; Mullette-Gillman, Cohen, & Groh, 2005; Schlack
et al., 2005] and the medial superior temporal area [Bruce et al., 1981; Ghazanfar et al.,
2008; Gu et al., 2008]. Further, primary unisensory cortices such as visual and auditory
cortex have been shown to be modulated by multisensory stimulation, expanding greatly the
view of what constitutes a “multisensory” brain region [Foxe & Schroeder, 2005; Ghazanfar
et al., 2005; Lakatos et al., 2007; Wang et al., 2008]. In addition, non-human primate work
has identified both subcortical and cortical connections and circuits needed for multisensory
processing [Clavagnier, Falchier, & Kennedy, 2004; Falchier et al., 2010; Rockland &
Author Manuscript
Ojima, 2003; Smiley & Falchier, 2009]. Most relevant to this review are those studies that
then investigate the underlying neurophysiology and multisensory behavior in non-human
primates [Angelaki, Gu, & DeAngelis, 2009; Frens & Van Opstal, 1998; Gu et al., 2008;
Kayser et al., 2008; Miller et al., 2001], allowing for the mapping of neuron-behavior
relationships. Overall, these studies have demonstrated strikingly similar neuroanatomical
and functional connections between non-human primates and humans, see Table 1
[Ghazanfar & Schroeder, 2006; Kajikawa et al., 2012], and has provided critical insights into
the neural substrates needed for multisensory processing in an animal model that more
closely resembles the neural architecture found in humans.
While the original work that evaluated the underlying neurobiology and development of
multisensory processing was initially performed in animal models [Murray & Wallace,
2011; Stein et al., 2014; Stein & Stanford, 2008], numerous studies have identified the
Author Manuscript
key use of these animal models is the ability to back translate relevant clinical findings from
the human to ask questions focused on the mechanisms into how (multi)sensory systems
become atypical. Rodents may offer a translational bridge to evaluate the underlying
mechanisms for multisensory function in genetic models.
In most rodent multisensory behavioral studies, rats have been used to assess multisensory
function based on their ability to complete complex operant tasks [Gleiss & Kayser, 2012;
Sakata et al., 2004; Sheppard et al., 2013]. Based on the relative ease of stimulus
presentation, and the relevance to the human population, audiovisual stimuli have been most
frequently used to assess multisensory function behaviorally [Carandini & Churchland,
2013; Sakata et al., 2004], despite the fact that rodents tend to have fairly poor vision and
more specialized somatosensory and olfactory systems [Ihara, Yoshikawa, & Touhara, 2013;
Petersen, 2014]. While these studies have differed on the variety of stimuli, specific stimulus
features, and task, they have all successfully demonstrated multisensory processing in rats,
finding similar effects to those found in larger animal species (Table 1) [Carandini &
Churchland, 2013; Gleiss & Kayser, 2012; Hirokawa et al., 2008; Raposo et al., 2012;
Raposo et al., 2014; Sakata et al., 2004].
Author Manuscript
Beyond behavioral studies, work has identified neural structures important for multisensory
processing in rats [Barth et al., 1995; Brett-Green et al., 2003; Hirokawa et al., 2008; Menzel
& Barth, 2005; Sieben et al., 2013; Tees, 1999], including specific border regions between
primary sensory cortices (visual, auditory, somatosensory) that contain a large number of
multisensory neurons and demonstrate similar multisensory characteristics to those found in
larger animal models [Schormans & Allman, 2019; Schormans, Typlt, & Allman, 2017;
Wallace et al., 2004; Xu et al., 2014]. In addition, studies have demonstrated that seemingly
irrelevant unisensory stimuli (i.e. auditory) can modulate responses in primary sensory
cortices (i.e. visual) [Bieler et al., 2017; Maruyama & Komai, 2018], which could be a result
of connectivity between primary sensory cortices [Stehberg, Dang, & Frostig, 2014]. In
addition to primary unisensory cortices and their overlapping border regions, studies have
Author Manuscript
identified the cerebellum [Ishikawa, Shimuta, & Hausser, 2015], insula [Rodgers et al.,
2008], parietal cortex [Brett-Green et al., 2003; Lippert et al., 2013; Menzel & Barth, 2005],
prefrontal cortex [Lipton et al., 1999; Reid et al., 2014], perirhinal cortex [Jacklin, Cloke,
Potvin, Garrett, & Winters, 2016], and the lateral entorhinal cortex [Doan, Lagartos-Donate,
Nilssen, Ohara, & Witter, 2019], as hubs for multisensory integration in rats. In addition to
these cortical structures, studies in rats have also investigated the role of the superior
colliculus (SC) in multisensory function on both the neural and behavioral level [Gharaei et
al., 2018; Hirokawa et al., 2011; Lau et al., 2018; May, 2006; Sparks & Hartwich-Young,
1989]. Importantly, many of these subcortical and cortical findings are similar to those that
have been observed in larger animal models (Table 1) [May, 2006; Wallace & Stein, 1996].
While similarities in responses to multisensory stimuli have been observed across animal
species in similar neural structures, it is important to note that there are large inter-species
Author Manuscript
differences in the functional roles of these regions. It is important to recognize that while
there may be structural similarities between species (i.e. regions that were once thought to
respond exclusively to unisensory stimuli, yet can respond under multisensory conditions)
this does not necessarily mean that these structures function in exactly the same context as in
larger animals and humans. Rodent studies allow us to gain insight into the cell-type
specificity, genetic underpinnings, and offer an initial view into the brain structures that may
be implicated. These findings are necessary as they can then be applied to larger animal
models, which share more analogous structural and functional brain regions to humans, with
Author Manuscript
the ultimate goal that these insights would lead to more clinically relevant applications.
population [Wallace & Stevenson, 2014]. Interestingly, studies in the mouse have also shown
anatomical connections between primary auditory cortex (A1) and V2L, as well as primary
visual cortex (V1) and V2L [Charbonneau, Laramee, Boucher, Bronchti, & Boire, 2012;
Laramee, Kurotani, Rockland, Bronchti, & Boire, 2011], suggesting that V2L may be a
multisensory structure conserved across rodent species.
neuronal responses in primary sensory cortices [Cohen et al., 2011; Deneux et al., 2019;
Knöpfel et al., 2019; McClure Jr & Polack, 2019; Meijer et al., 2017; Morrill & Hasenstaub,
2018; Zhang, Kwon, Ben-Johny, O’Connor, & Issa, 2020];results that may be supported by
growing evidence that primary sensory cortices may share underlying connections with one
another [Masse, Ross, Bronchti, & Boire, 2017; Morrill & Hasenstaub, 2018]. As observed
in the rat (Table 1), studies have also shown that the parietal cortex [Kuroki et al., 2018;
Lyamzin & Benucci, 2019; Najafi et al., 2019; Song et al., 2017], orbitofrontal cortex
[Sharma & Bandyopadhyay, 2019], cerebellum [Chabrol, Arenz, Wiechert, Margrie, &
DiGregorio, 2015] are neuronal hubs for multisensory integration in the mouse. Elegant
work identifying subcortical (i.e. inferior colliculus - [Dillingham, Gay, Behrooz, &
Gabriele, 2017], superior colliculus - [Doykos et al., 2020; Drager & Hubel, 1975a; Drager
& Hubel, 1975b] to cortical connections [Lesicko, Hristova, Maigler, & Llano, 2016],
Author Manuscript
In addition to these physiological and circuit-based findings, our lab was the first to
demonstrate multisensory processing on the behavioral level in mice [Siemann et al., 2015].
We found that mice from different background strains (i.e. C57 and 129 S4/S6) demonstrate
visual conditions alone. Furthermore, this work in mice has begun to establish similarities in
the stimulus characteristics and principles that support these behavioral benefits [Siemann et
al., 2015]. Interestingly, a more recent study designed and utilized another behavioral
paradigm to demonstrate inverse effectiveness in mice as well, and with this task the authors
observed that stimulus detection is significantly improved under audiovisual conditions
[Meijer et al., 2018]. The establishment of these behavioral paradigms set the stage for work
structured to assess how molecular and circuitry manipulations impact multisensory
behavior.
To this point, most recent work is now attempting to connect the underlying neural
mechanisms with the associated multisensory behavior in the mouse model. It has been
demonstrated that neurons in the V2L, also known as area AL, are responsive to audiovisual
stimuli and critical for multisensory behavior [Meijer et al., 2020]. In this study, mice were
Author Manuscript
head fixed and performed a stimulus detection task in which auditory, visual, and combined
audiovisual stimuli were presented while simultaneously utilizing calcium imaging to record
from neurons in layer II/III in V2L or V1. Meijer et al., found that neural responses could be
elevated under audiovisual conditions in the V2L, and the neural population activity in this
region corresponded with behavioral performance under multisensory conditions. This work
is impactful as it identifies a novel neuronal hub, in the mouse, crucial for multisensory
behavior, and also demonstrates the similarities across rodent species (i.e. V2L is implicated
in both the rat and mouse), highlighting the possibility to utilize the mouse model in more
complex brain-behavior based studies that may be relevant for larger animal species.
Based on emerging results in wild type mice, there have been a few investigations of
multisensory processing in genetically modified mice (Table 2) [Gogolla et al., 2014; Wada
et al., 2019]. Gogolla and colleagues evaluated the neural mechanisms of multisensory
processing in four mouse lines, including the Shank3 knockout mouse that models Phelan-
McDermid syndrome, which is robustly associated with autism diagnosis. They also
evaluated the Mecp2 null mouse model of Rett syndrome, the Gad65 knockout mouse model
of GABA deficiency, and the BTBR T+tf/J inbred strain of mouse that demonstrate
behaviors which serve as proxies for the human clinical phenotype of autism. However, it is
important to note caveats such as the BTBR T+tf/J inbred mouse strain represents no known
idiopathic form of ASD [Meyza & Blanchard, 2017], Rett syndrome itself is no longer
considered within the autism diagnosis [APA, 2013], and the Shank3 mouse model is
typically homozygous whereas in the clinical population, individuals display heterozygous
Author Manuscript
The authors assessed multisensory processing, in anesthetized mice, in the insular cortex by
recording neural responses utilizing in vivo imaging of intrinsic flavoprotein fluorescence to
auditory (i.e. pure tones with varying frequencies and intensities to the contralateral ear) and
somatosensory stimuli (i.e. air puffs to the contralateral front paw) along with the combined
multisensory presentations. Multisensory responses were atypical and the magnitude of
multisensory enhancement was reduced in the four selected lines in comparison to wild type
animals. In addition, differences were observed in the size and densities of GAD65 puncta,
Author Manuscript
and atypical miniature inhibitory postsynaptic currents in pyramidal cells from layers 2/3 of
the insular cortex were found in response to auditory stimuli. Interestingly, the authors also
found that the maturation of responses to auditory stimuli was atypical in BTBR mice across
multiple developmental time periods. Based on these differences in inhibitory signaling and
altered developmental trajectories, the authors chronically administered diazepam, a
GABAA receptor positive allosteric modulator, between postnatal days 15–28. Multisensory
function was then assessed during adulthood, with neural responses normalizing under
multisensory conditions, compared to responses in control wild type mice, as well as a
normalization of the abnormal self-grooming behaviors seen in these models [Gogolla et al.,
2014]. In addition, the authors treated these mouse models with diazepam later in life, at
postnatal days 48–54, and found that diazepam did not normalize electrophysiological
responses to multisensory stimuli. These results suggest there are sensitive periods of
Author Manuscript
In addition, our lab demonstrated the first evidence of abnormal multisensory behavioral
responses in a genetically manipulated mouse (Table 2) [Siemann et al., 2017]. In this study,
we utilized a mouse with a point mutation in the serotonin transporter (SERT) in which a
glycine (Gly56) is substituted to an alanine (Ala56). In a study of multiplex ASD families,
the Ala56 allele was the most common of a group of rare amino acid coding variants in the
Author Manuscript
serotonin transporter [Sutcliffe et al., 2005]. The SERT Ala56 mouse shows elevated whole
blood serotonin levels, paralleling the hyperserotonemia found in 25% of children with
ASD, as well as abnormalities in social, communicative, and repetitive behaviors [Gabriele,
Sacco, & Persico, 2014; Muller, Anacker, & Veenstra-VanderWeele, 2016]. We found that
SERT Ala56 animals demonstrated a significant decrease in response accuracy to visual and
auditory targets; however, the most significant behavioral deficits were observed under
combined audiovisual conditions compared to wild type littermate controls. These
multisensory effects were larger compared to differences between groups for either auditory
or visual alone stimuli, however our results may also reflect poorer unisensory performance
in SERT Ala56 mice that is further compounded when auditory and visual stimuli are
combined. When examined in more detail, it was found that the behavioral gain normally
seen under multisensory conditions was eliminated in SERT Ala56 mice. Overall, these
Author Manuscript
The importance of these studies is reinforced by the fact that serotonin has been consistently
implicated in ASD [Cook & Leventhal, 1996; Mulder et al., 2004; Muller et al., 2016], has
been shown to be important for sensory cortical development [Cases et al., 1996; Gaspar,
Cases, & Maroteaux, 2003; Salichon et al., 2001], and can modulate responses to a variety
Author Manuscript
of sensory stimuli [Esaki et al., 2005; Hurley, Thompson, & Pollak, 2002; Waterhouse,
Azizi, Burne, & Woodward, 1990]. Most relevant to our work, there is evidence that
serotonin may play a role in cross-modal plasticity [Jitsuki et al., 2011]. Jitsuki and
colleagues demonstrated that under conditions of visual deprivation serotonin receptors are
critical for increased synaptic strength and more precise tuning of somatosensory responses
in the mouse barrel cortex. More recently, a study from Tang and colleagues demonstrated
that serotonin can modulate microcircuits in the dorsal cochlear nucleus to enhance
multisensory signaling by dampening auditory-related pathways [Tang & Trussell, 2017].
The loss of multisensory gain in the SERT Ala56 knock-in mouse further supports a role for
serotonin in multisensory processing, with the potential for future studies to assess the
ability of serotonin transporter blockade or other approaches to rescue this behavioral deficit
[Robson et al., 2018].
Author Manuscript
To this point, studies are already using very similar psychophysical experiments as observed
in humans to demonstrate multisensory temporal processing and temporal binding windows
(TBWs) in rats and have identified the neural structures needed for this multisensory
behavior [Schormans & Allman, 2018; Schormans, Scott, et al., 2017; Schormans, Typlt, &
Allman, 2017]. Therefore, can mouse studies be refined to evaluate multisensory temporal
processing, including temporal binding windows, and would these tasks implicate brain
regions (i.e. V2L) similar to those observed in the rat model? Interestingly, V2L has just
recently been identified in the mouse as a neural structure capable of responding to
audiovisual stimuli and critical for multisensory detection behavior [Meijer et al., 2020].
This would strongly suggest that V2L may have a critical role for multisensory temporal
Author Manuscript
processing in the mouse, highlighting the need for future studies to design novel behavioral
tasks to measure temporal binding windows in this animal model as well. As a result, a
number of important translational and clinically relevant studies could be evaluated. Prior
work has already begun to show atypical multisensory processing in mouse models
associated with ASD (Table 2) [Gogolla et al., 2014; Siemann et al., 2017; Wada et al.,
2019]. Would future investigations find similar multisensory temporal behavioral deficits in
genetic mouse models of ASD as are observed in the human clinical population?
In addition, mouse models allow for the screening and testing of specific pharmacological
Author Manuscript
point there are only two currently approved drugs for the treatment of ASD, yet these treat
co-occurring behavioral problems, rather than core symptoms [Ghanizadeh, Sahraeizadeh, &
Berk, 2014]. A variety of pharmacologic compounds have been found to rescue social or
repetitive behaviors observed in genetic or behavioral mouse models of ASD [Gogolla et al.,
2014; Silverman et al., 2012; Silverman et al., 2015];however, those compounds that have
been tested in translational clinical trials have not yet demonstrated comparable benefits in
humans with autism spectrum disorder [Ooi, Weng, Kossowsky, Gerger, & Sung, 2017;
Silverman & Crawley, 2014; Veenstra-VanderWeele et al., 2017].
Although the focus of this intervention research, both in humans and in model systems, has
focused on these core symptoms, we argue that it may be beneficial to test treatments or
interventions that target altered (multi)sensory function. Since multisensory integration is
Author Manuscript
critical for our perception of the world and is integral for the interpretation and
understanding of social communicative signals, it is possible that treatment that targets of
these sensory disturbances may have therapeutic effects that extend to core symptom
domains. Indeed, if multisensory integration is a key building block in the construction of
higher-order cognitive representations, it may serve as a tractable target for intervention that
would have benefits that have cascading impact across both the processing hierarchy as well
as in development [Baum et al., 2015a; Robertson & Baron-Cohen, 2017; Stevenson et al.,
2018; Stevenson, Siemann, Schneider, et al., 2014; Thye et al., 2018]. With this possibility, it
Author Manuscript
Currently, the hypothesis that atypical multisensory processing underlies core ASD
symptoms is plausible, yet it has only some evidence in humans, and much of that evidence
is indirect. For example, Stevenson et al. demonstrated that atypical multisensory temporal
processing in individuals with ASD resulted in poorer performance in perceiving a
multisensory illusion that utilizes speech related stimuli [Stevenson, Siemann, Schneider, et
al., 2014]. In addition, the brain region (STS) shown to be critical in the perception of this
illusion [McGurk effect) (Beauchamp et al., 2010; Nath et al., 2011; Nath & Beauchamp,
Author Manuscript
2012], is also a key hub for the processing of social communicative stimuli [Pelphrey,
Morris, & Mccarthy, 2004; Stevenson et al., 2011; Zilbovicius et al., 2006], and has been
found to be both structurally and functionally atypical in ASD [Boddaert et al., 2004;
Redcay, 2008]. In addition, studies have found that sensory disturbances correlate with
greater deficits in social communication, along with the presence of more repetitive
behaviors in individuals with ASD [Boyd et al., 2010; Boyd, McBee, Holtzclaw, Baranek, &
Bodfish, 2009; Feldman et al., 2019; Foss-Feig, Heacock, & Cascio, 2012; Hilton et al.,
2010; Kern et al., 2007; Schauder, Muller, Veenstra-VanderWeele, & Cascio, 2015;
Stevenson et al., 2017; Stevenson et al., 2018; Watson et al., 2011; Zhou et al., 2018].
A number of future investigations could evaluate these relationships. Studies have shown
that multisensory training paradigms with feedback can result in the narrowing of the TBW
Author Manuscript
and increased functional connectivity between the STS and regions of the auditory and
visual cortices in typically developed individuals [De Niear, Gupta, Baum, & Wallace, 2018;
De Niear, Koo, & Wallace, 2016; Powers 3rd et al., 2012; Powers, Hillock, & Wallace,
2009]. Work has just begun to investigate if these same paradigms result in improved neural
and behavioral multisensory function in individuals with ASD [Feldman et al., 2020], and
plan to assess if these effects result in lasting changes to other domains such as social
communication. There is evidence that seemingly irrelevant sensory stimuli can impact
primary sensory systems [Ghazanfar & Schroeder, 2006]. Future studies could also utilize
sensory-based therapies and/or perceptual training to investigate the underlying mechanisms
for which improving unisensory function can result in increased multisensory function or
temporal processing [Stevenson, Wilson, et al., 2013], and if this ultimately leads to lasting
and positive changes in higher order domains for both typically developed and clinical
Author Manuscript
Animal models could also be used to evaluate the relationship between multisensory
processing, social function, and repetitive behavior. Gogolla and colleagues demonstrated
that a pharmacological intervention in mouse models associated with ASD not only
normalized atypical multisensory neural responses, but it also reversed the presence of
repetitive behaviors found in these animal models [Gogolla et al., 2014]. They did not,
however, establish causality directly. Future studies should use optogenetics,
electrophysiology, and pharmacology to more directly investigate the known circuits and
Author Manuscript
neural populations that support multisensory function. For example, if relevant subcortical
(i.e. superior colliculus) and/or cortical brain regions (i.e. unisensory cortices, V2L, or
parietal cortex), critical for multisensory processing are selectively silenced either
pharmacologically, through cooling blockade or lesioning methods, and/or specific neuronal
populations (i.e. parvalbumin-positive interneurons) are reversibly silenced through
optogenetic techniques [Olcese et al., 2013], does this result in impaired social function or
the presence of repetitive/restricted behavior? The development of multisensory operant
tasks in mice [Meijer et al., 2018; Siemann et al., 2015] also raises the opportunity to
evaluate whether training in animal models can improve multisensory function, which may
parallel approaches being developed in humans that seek to capitalize on multisensory
training [De Niear et al., 2016; De Niear et al., 2018; Powers 3rd et al., 2012; Powers et al.,
2009].
Author Manuscript
Conclusion
Atypical multisensory integration may be a tractable phenotype that can be observed across
species and is becoming increasingly recognized as an important feature of ASD. In contrast
to social communicative behaviors, which are quite species-specific, or repetitive behaviors,
which take on very different forms in rodents than in humans, multisensory integration
appears to have preserved brain mechanisms and is readily tested through behavioral assays
that have been developed across species. With the coupling of these behavior paradigms and
genetic manipulations, it may be possible to investigate molecular, cellular, and systems-
level questions, and then attempt to translate these findings to the human clinical population.
This approach is further supported by emerging evidence that suggests that altered
(multi)sensory function has cascading effects that impact core symptoms, making this area
Author Manuscript
Acknowledgments
This work was supported by the Vanderbilt Kennedy Center HD1505 (MW), 5T32MH018921-24: Development of
Psychopathology: From Brain and Behavioral Science to Intervention (JKS), MH094604 (JVV), and MH096972
(JVV, MW).
Conflict of Interest
Consulting or Advisory Board: Roche, Novartis, SynapDx. Research funding: Roche, Novartis, SynapDx, Seaside
Therapeutics, Forest and editorial stipend: Springer, Wiley for JVV. The remaining authors report no financial
interests or potential conflicts of interest.
References
Author Manuscript
Allman BL, & Meredith MA (2007). Multisensory processing in “unimodal” neurons: cross-modal
subthreshold auditory effects in cat extrastriate visual cortex. Journal of Neurophysiology, 98(1),
545–549. 10.1152/jn.00173.2007 [PubMed: 17475717]
Angelaki DE, Gu Y, & DeAngelis GC (2009). Multisensory integration: Psychophysics,
neurophysiology, and computation. Current Opinion in Neurobiology, 19(4), 452–458. [PubMed:
19616425]
APA. (2013). Diagnostic and statistical manual of mental disorders DSM-5 development. Washington,
DC: American Psychiatric Association.
Avillac M, Hamed SB, & Duhamel J-R (2007). Multisensory integration in the ventral intraparietal
area of the macaque monkey. Journal of Neuroscience, 27(8), 1922–1932. [PubMed: 17314288]
Author Manuscript
Ayres AJ (1972). Sensory integration and learning disorders. Torrance, CA: Western Psychological
Services.
Barth DS, Goldberg N, Brett B, & Di S (1995). The spatiotemporal organization of auditory, visual,
and auditory-visual evoked potentials in rat cortex. Brain Research, 678(1), 177–190. [PubMed:
7620886]
Baum SH, Stevenson RA, & Wallace MT (2015a). Behavioral, perceptual, and neural alterations in
sensory and multisensory function in autism spectrum disorder. Progress in Neurobiology, 134,
140–160. [PubMed: 26455789]
Baum SH, Stevenson RA, & Wallace MT (2015b). Testing sensory and multisensory function in
children with autism spectrum disorder. JoVE (Journal of Visualized Experiments), 98, e52677–
e52677.
Beauchamp MS, Argall BD, Bodurka J, Duyn JH, & Martin A (2004). Unraveling multisensory
integration: Patchy organization within human STS multisensory cortex. Nature Neuroscience,
7(11), 1190–1192. [PubMed: 15475952]
Author Manuscript
Beauchamp MS, Nath AR, & Pasalar S (2010). fMRI-guided transcranial magnetic stimulation reveals
that the superior temporal sulcus is a cortical locus of the McGurk effect. The Journal of
Neuroscience, 30(7), 2414–2417. [PubMed: 20164324]
Beker S, Foxe JJ, & Molholm S (2018). Ripe for solution: Del-ayed development of multisensory
processing in autism and its remediation. Neuroscience and Biobehavioral Reviews, 84, 182–192.
[PubMed: 29162518]
Bieler M, Sieben K, Schildt S, Röder B, & Hanganu-Opatz IL (2017). Visual-tactile processing in
primary somatosensory cortex emerges before cross-modal experience. Synapse, 71(6), e21958.
Boddaert N, Chabane N, Gervais H, Good C, Bourgeois M, Plumet M, … Samson Y (2004). Superior
temporal sulcus anatomical abnormalities in childhood autism: A voxel-based morphometry MRI
study. Neuroimage, 23(1), 364–369. [PubMed: 15325384]
Borges-Merjane C, & Trussell LO (2015). ON and OFF unipolar brush cells transform multisensory
inputs to the auditory system. Neuron, 85(5), 1029–1042. [PubMed: 25741727]
Boyd BA, Baranek GT, Sideris J, Poe MD, Watson LR, Patten E, & Miller H (2010). Sensory features
Author Manuscript
and repetitive behaviors in children with autism and developmental delays. Autism Research, 3(2),
78–87. [PubMed: 20437603]
Boyd BA, McBee M, Holtzclaw T, Baranek GT, & Bodfish JW (2009). Relationships among repetitive
behaveiors, sensory features, and executive functions in high functioning autism. Research in
Autism Spectrum Disorders, 3(4), 959–966. [PubMed: 21475640]
Brandwein AB, Foxe JJ, Butler JS, Frey H-P, Bates JC, Shulman LH, & Molholm S (2015).
Neurophysiological indices of atypical auditory processing and multisensory integration are
associated with symptom severity in autism. Journal of Autism and Developmental Disorders,
45(1), 230–244. [PubMed: 25245785]
Brandwein AB, Foxe JJ, Butler JS, Russo NN, Altschuler TS, Gomes H, & Molholm S (2012). The
development of multisensory integration in high-functioning autism: High-density electrical
mapping and psychophysical measures reveal impairments in the processing of audiovisual inputs.
Cerebral Cortex, 23(6), 1329–1341. [PubMed: 22628458]
Brett-Green B, Fifkova E, Larue DT, Winer JA, & Barth DS (2003). A multisensory zone in rat
parietotemporal cortex: Intra- and extra-cellular physiology and thalamocortical connections.
Author Manuscript
Burnett LR, Stein BE, Chaponis D, & Wallace MT (2004). Superior colliculus lesions preferentially
disrupt multisensory orientation. Neuroscience, 124(3), 535–547. 10.1016/
Author Manuscript
Case-Smith J, Weaver LL, & Fristad MA (2015). A systematic review of sensory processing
interventions for children with autism spectrum disorders. Autism, 19(2), 133–148. [PubMed:
24477447]
CDC. (2019). Autism Spectrum Disorder (ASD): Data & Statistics—Prevalence.
Chabrol FP, Arenz A, Wiechert MT, Margrie TW, & DiGregorio DA (2015). Synaptic diversity enables
temporal coding of coincident multisensory inputs in single neurons. Nature Neuroscience, 18(5),
718–727. [PubMed: 25821914]
Charbonneau V, Laramee ME, Boucher V, Bronchti G, & Boire D (2012). Cortical and subcortical
projections to primary visual cortex in anophthalmic, enucleated and sighted mice. European
Journal of Neuroscience, 36(7), 2949–2963. 10.1111/j.1460-9568.2012.08215.x [PubMed:
22780435]
Clavagnier S, Falchier A, & Kennedy H (2004). Long-distance feedback projections to area V1:
Implications for multisensory integration, spatial awareness, and visual consciousness. Cognitive,
Affective, and Behavioral Neuroscience, 4(2), 117–126.
Author Manuscript
Cohen L, Rothschild G, & Mizrahi A (2011). Multisensory integration of natural odors and sounds in
the auditory cortex. Neuron, 72(2), 357–369. 10.1016/j.neuron.2011.08.019 [PubMed: 22017993]
Cohen YE (2009). Multimodal activity in the parietal cortex. Hearing Research, 258(1–2), 100–105.
[PubMed: 19450431]
Collignon O, Charbonneau G, Peters F, Nassim M, Lassonde M, Lepore F, … Bertone A (2013).
Reduced multisensory facilitation in persons with autism. Cortex, 49(6), 1704–1710. [PubMed:
22818902]
Colonius H, & Diederich A (2004). Multisensory interaction in saccadic reaction time: A time-
window-of-integration model. Journal of Cognitive Neuroscience, 16(6), 1000–1009.
Author Manuscript
Dove D, Warren Z, McPheeters ML, Taylor JL, Sathe NA, & Veenstra-VanderWeele J (2012).
Medications for adolescents and young adults with autism spectrum disorders: A systematic
review. Pediatrics, 130(4), 717–726. [PubMed: 23008452]
Doykos TK, Gilmer JI, Person AL, & Felsen G (2020). Monosynaptic inputs to specific cell types of
the intermediate and deep layers of the superior colliculus. Journal of Comparative Neurology,
528, 2254–2268. [PubMed: 32080842]
Drager U, & Hubel DH (1975a). Responses to visual stimulation and relationship between visual,
auditory, and somatosensory inputs in mouse superior colliculus. Journal of Neurophysiology,
38(3), 690–713. [PubMed: 1127462]
Drager UC, & Hubel DH (1975b). Physiology of visual cells in mouse superior colliculus and
correlation with somatosensory and auditory input. Nature, 253, 203–204. [PubMed: 1110771]
Driver J, & Noesselt T (2008). Multisensory interplay reveals crossmodal influences on ‘sensory-
specific’ brain regions, neural responses, and judgments. Neuron, 57(1), 11–23. 10.1016/
j.neuron.2007.12.0132427054 [PubMed: 18184561]
Author Manuscript
Esaki T, Cook M, Shimoji K, Murphy DL, Sokoloff L, & Holmes A (2005). Developmental disruption
of serotonin transporter function impairs cerebral responses to whisker stimulation in mice.
Proceedings of the National Academy of Sciences of the United States of America, 102(15), 5582–
5587. [PubMed: 15809439]
Falchier A, Schroeder CE, Hackett TA, Lakatos P, Nascimento-Silva S, Ulbert I, … Smiley JF (2010).
Projection from visual areas V2 and prostriata to caudal auditory cortex in the monkey. Cerebral
Cortex, 20(7), 1529–1538. [PubMed: 19875677]
Feldman JI, Dunham K, Cassidy M, Wallace MT, Liu Y, & Woynaroski TG (2018). Audiovisual
multisensory integration in individuals with autism spectrum disorder: A systematic review and
Author Manuscript
[PubMed: 24613076]
Gaspar P, Cases O, & Maroteaux L (2003). The developmental role of serotonin: News from mouse
molecular genetics. Nature Reviews Neuroscience, 4(12), 1002–1012. [PubMed: 14618156]
Gervais H, Belin P, Boddaert N, Leboyer M, Coez A, Sfaello I, … Zilbovicius M (2004). Abnormal
cortical voice processing in autism. Nature Neuroscience, 7(8), 801–802. [PubMed: 15258587]
Ghanizadeh A, Sahraeizadeh A, & Berk M (2014). A head-to-head comparison of aripiprazole and
risperidone for safety and treating autistic disorders, a randomized double blind clinical trial. Child
Psychiatry and Human Development, 45 (2), 185–192. [PubMed: 23801256]
Gharaei S, Arabzadeh E, & Solomon SG (2018). Integration of visual and whisker signals in rat
superior colliculus. Scientific Reports, 8(1), 1–16. [PubMed: 29311619]
Ghazanfar AA, Chandrasekaran C, & Logothetis NK (2008). Interactions between the superior
temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys.
Journal of Neuroscience, 28(17), 4457–4469. [PubMed: 18434524]
Ghazanfar AA, Maier JX, Hoffman KL, & Logothetis NK (2005). Multisensory integration of dynamic
Author Manuscript
faces and voices in rhesus monkey auditory cortex. Journal of Neuroscience, 25(20), 5004–5012.
[PubMed: 15901781]
Ghazanfar AA, & Schroeder CE (2006). Is neocortex essentially multisensory? Trends in Cognitive
Sciences, 10(6), 278–285. [PubMed: 16713325]
Gleiss S, & Kayser C (2012). Audio-visual detection benefits in the rat. PLoS One, 7(9), e45677
10.1371/journal.pone.00456773445486 [PubMed: 23029179]
Gogolla N, Takesian AE, Feng G, Fagiolini M, & Hensch TK (2014). Sensory integration in mouse
insular cortex reflects GABA circuit maturation. Neuron, 83, 894–905. [PubMed: 25088363]
Gu Y, Angelaki DE, & DeAngelis GC (2008). Neural correlates of multisensory cue integration in
macaque MSTd. Nature Neuroscience, 11(10), 1201–1210. [PubMed: 18776893]
Author Manuscript
Hazen EP, Stornelli JL, O’Rourke JA, Koesterer K, & McDougle CJ (2014). Sensory symptoms in
autism spectrum disorders. Harvard Review of Psychiatry, 22(2), 112–124. [PubMed: 24614766]
Hillock AR, Powers AR, & Wallace MT (2011). Binding of sights and sounds: Age-related changes in
multisensory temporal processing. Neuropsychologia, 49(3), 461–467. 10.1016/
j.neuropsychologia.2010.11.0413140703 [PubMed: 21134385]
Hilton CL, Harper JD, Kueker RH, Lang AR, Abbacchi AM, Todorov A, & LaVesser PD (2010).
Sensory responsiveness as a predictor of social severity in children with high functioning autism
spectrum disorders. Journal of Autism and Developmental Disorders, 40(8), 937–945. 10.1007/
s10803-010-0944-8 [PubMed: 20108030]
Hirokawa J, Bosch M, Sakata S, Sakurai Y, & Yamamori T (2008). Functional role of the secondary
visual cortex in multisensory facilitation in rats. Neuroscience, 153(4), 1402–1417. 10.1016/
j.neuroscience.2008.01.011 [PubMed: 18440715]
Hirokawa J, Sadakane O, Sakata S, Bosch M, Sakurai Y, & Yamamori T (2011). Multisensory
information facilitates reaction speed by enlarging activity difference between superior colliculus
Author Manuscript
Irwin JR, Tornatore LA, Brancazio L, & Whalen D (2011). Can children with autism spectrum
disorders “hear” a speaking face? Child Development, 82(5), 1397–1403. [PubMed: 21790542]
Ishikawa T, Shimuta M, & Hausser M (2015). Multimodal sensory integration in single cerebellar
granule cells in vivo. eLife, 4, e12916. [PubMed: 26714108]
Jacklin DL, Cloke JM, Potvin A, Garrett I, & Winters BD (2016). The dynamic multisensory engram:
Neural circuitry underlying crossmodal object recognition in rats changes with the nature of object
experience. Journal of Neuroscience, 36(4), 1273–1289. [PubMed: 26818515]
Jiang W, Jiang H, & Stein BE (2002). Two corticotectal areas facilitate multisensory orientation
behavior. Journal of Cognitive Neuroscience, 14(8), 1240–1255. 10.1162/089892902760807230
[PubMed: 12495529]
Jiang W, Jiang H, & Stein BE (2006). Neonatal cortical ablation disrupts multisensory development in
superior colliculus. Journal of Neurophysiology, 95(3), 1380–1396. 10.1152/jn.00880.2005
[PubMed: 16267111]
Jiang W, Wallace MT, Jiang H, Vaughan JW, & Stein BE (2001). Two cortical areas mediate
Author Manuscript
Kayser C, Petkov CI, & Logothetis NK (2008). Visual modulation of neurons in auditory cortex.
Cerebral Cortex, 18(7), 1560–1574. [PubMed: 18180245]
Author Manuscript
Kern JK, Trivedi MH, Grannemann BD, Garver CR, Johnson DG, Andrews AA, … Schroeder JL
(2007). Sensory correlations in autism. Autism: The International Journal of Research and
Practice, 11(2), 123–134. 10.1177/1362361307075702 [PubMed: 17353213]
Knöpfel T, Sweeney Y, Radulescu CI, Zabouri N, Doostdar N, Clopath C, & Barnes SJ (2019). Audio-
visual experience strengthens multisensory assemblies in adult mouse visual cortex. Nature
Communications, 10(1), 1–15.
Kuroki S, Yoshida T, Tsutsui H, Iwama M, Ando R, Michikawa T, … Itohara S (2018). Excitatory
neuronal hubs configure multisensory integration of slow waves in association cortex. Cell
Reports, 22(11), 2873–2885. [PubMed: 29539417]
Kwakye LD, Foss-Feig JH, Cascio CJ, Stone WL, & Wallace MT (2011). Altered auditory and
multisensory temporal processing in autism spectrum disorders. Frontiers in Integrative
Neuroscience, 4, 129 10.3389/fnint.2010.001293024004 [PubMed: 21258617]
Lakatos P, Chen C-M, O’Connell MN, Mills A, & Schroeder CE (2007). Neuronal oscillations and
multisensory interaction in primary auditory cortex. Neuron, 53(2), 279–292. [PubMed:
Author Manuscript
17224408]
Lane AE, Young RL, Baker AE, & Angley MT (2010). Sensory processing subtypes in autism:
Association with adaptive behavior. Journal of Autism and Developmental Disorders, 40(1), 112–
122. [PubMed: 19644746]
Lang R, O’Reilly M, Healy O, Rispoli M, Lydon H, Streusand W, … Lancioni G (2012). Sensory
integration therapy for autism spectrum disorders: A systematic review. Research in Autism
Spectrum Disorders, 6(3), 1004–1018.
Laramee ME, Kurotani T, Rockland KS, Bronchti G, & Boire D (2011). Indirect pathway between the
primary auditory and visual cortices through layer V pyramidal neurons in V2L in mouse and the
effects of bilateral enucleation. European Journal of Neuroscience, 34(1), 65–78. 10.1111/
j.1460-9568.2011.07732.x [PubMed: 21676038]
Lau C, Manno FA, Dong CM, Chan KC, & Wu EX (2018). Auditory-visual convergence at the
superior colliculus in rat using functional MRI. Paper presented at the 2018 40th Annual
International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).
Leekam SR, Nieto C, Libby SJ, Wing L, & Gould J (2007). Describing the sensory abnormalities of
Author Manuscript
children and adults with autism. Journal of Autism and Developmental Disorders, 37(5), 894–
910. [PubMed: 17016677]
Lesicko AM, Hristova TS, Maigler KC, & Llano DA (2016). Connectional modularity of top-down
and bottom-up multimodal inputs to the lateral cortex of the mouse inferior colliculus. Journal of
Neuroscience, 36(43), 11037–11050. [PubMed: 27798184]
Lippert MT, Takagaki K, Kayser C, & Ohl FW (2013). Asymmetric multisensory interactions of visual
and somatosensory responses in a region of the rat parietal cortex. PLoS One, 8(5), e63631.
[PubMed: 23667650]
Lipton PA, Alvarez P, & Eichenbaum H (1999). Crossmodal associative memory representations in
rodent orbitofrontal cortex. Neuron, 22(2), 349–359. [PubMed: 10069340]
Lyamzin D, & Benucci A (2019). The mouse posterior parietal cortex: Anatomy and functions.
Neuroscience Research, 140, 14–22. [PubMed: 30465783]
Maier JX, Neuhoff JG, Logothetis NK, & Ghazanfar AA (2004). Multisensory integration of looming
signals by rhesus monkeys. Neuron, 43(2), 177–181. [PubMed: 15260954]
Author Manuscript
Marco EJ, Hinkley LB, Hill SS, & Nagarajan SS (2011). Sensory processing in autism: A review of
neurophysiologic findings. Pediatric Research, 69(5 Pt 2), 48R–54R. 10.1203/
PDR.0b013e3182130c543086654
Martínez K, Martínez-García M, Marcos-Vidal L, Janssen J, Castellanos FX, Pretus C, … Parellada M
(2019). Sensory-to-cognitive systems integration is associated with clinical severity in autism
spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 59(3),
422–433. [PubMed: 31260788]
Martuzzi R, Murray MM, Michel CM, Thiran J-P, Maeder PP, Clarke S, & Meuli RA (2007).
Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cerebral
Author Manuscript
window of integration results from changing task demands. PLoS One, 8(8), e71608. [PubMed:
23951203]
Meijer GT, Marchesi P, Mejias JF, Montijn JS, Lansink CS, & Pennartz CM (2020). Neural correlates
of multisensory detection behavior: Comparison of primary and higher-order visual cortex. Cell
Reports, 31(6), 107636. [PubMed: 32402272]
Meijer GT, Montijn JS, Pennartz CM, & Lansink CS (2017). Audiovisual modulation in mouse
primary visual cortex depends on cross-modal stimulus configuration and congruency. Journal of
Neuroscience, 37(36), 8783–8796. [PubMed: 28821672]
Meijer GT, Pie JL, Dolman TL, Pennartz C, & Lansink CS (2018). Audiovisual integration enhances
stimulus detection performance in mice. Frontiers in Behavioral Neuroscience, 12, 231.
[PubMed: 30337861]
Menzel RR, & Barth DS (2005). Multisensory and secondary somatosensory cortex in the rat. Cerebral
Cortex, 15(11), 1690–1696. 10.1093/cercor/bhi045 [PubMed: 15703251]
Meredith MA (2002). On the neuronal basis for multisensory convergence: A brief overview.
Author Manuscript
Miller J, Ulrich R, & Lamarre Y (2001). Locus of the redundant-signals effect in bimodal divided
attention: A neurophysiological analysis. Perception and Psychophysics, 63(3), 555–562.
[PubMed: 11414141]
Miller LM, & D’esposito M (2005). Perceptual fusion and stimulus coincidence in the cross-modal
integration of speech. Journal of Neuroscience, 25(25), 5884–5893. [PubMed: 15976077]
Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, & Foxe JJ (2002). Multisensory
auditory-visual interactions during early sensory processing in humans: a high-density electrical
mapping study. Cognitive Brain Research, 14(1), 115–128. 10.1016/s0926-6410(02)00066-6
[PubMed: 12063135]
Mongillo EA, Irwin JR, Whalen DH, Klaiman C, Carter AS, & Schultz RT (2008). Audiovisual
processing in children with and without autism spectrum disorders. Journal of Autism and
Author Manuscript
CRC Press.
Najafi F, Elsayed GF, Cao R, Pnevmatikakis E, Latham PE, Cunningham JP, & Churchland AK
(2019). Excitatory and inhibitory subnetworks are equally selective during decision-making and
emerge simultaneously during learning. Neuron, 105(1), 165–179. [PubMed: 31753580]
Nath AR, & Beauchamp MS (2012). A neural basis for interindividual differences in the McGurk
effect, a multisensory speech illusion. Neuroimage, 59(1), 781–787. [PubMed: 21787869]
Nath AR, Fava EE, & Beauchamp MS (2011). Neural correlates of interindividual differences in
children’s audiovisual speech perception. The Journal of Neuroscience, 31(39), 13963–13971.
10.1523/JNEUROSCI.2605-11.20113203203 [PubMed: 21957257]
Nestler EJ, & Hyman SE (2010). Animal models of neuropsychiatric disorders. Nature Neuroscience,
13(10), 1161–1169. [PubMed: 20877280]
Olcese U, Iurilli G, & Medini P (2013). Cellular and synaptic architecture of multisensory integration
in the mouse neocortex. Neuron, 79(3), 579–593. 10.1016/j.neuron.2013.06.010 [PubMed:
23850594]
Author Manuscript
Olive I, Tempelmann C, Berthoz A, & Heinze HJ (2015). Increased functional connectivity between
superior colliculus and brain regions implicated in bodily self-consciousness during the rubber
hand illusion. Human Brain Mapping, 36(2), 717–730. [PubMed: 25346407]
Ooi YP, Weng S-J, Kossowsky J, Gerger H, & Sung M (2017). Oxytocin and autism spectrum
disorders: A systematic review and meta-analysis of randomized controlled trials.
Pharmacopsychiatry, 50(01), 5–13. [PubMed: 27574858]
Parham LD, Roley SS, May-Benson TA, Koomar J, Brett-Green B, Burke JP, … Schaaf RC (2011).
Development of a fidelity measure for research on the effectiveness of the Ayres Sensory
Integration® intervention. American Journal of Occupational Therapy, 65(2), 133–142.
[PubMed: 21476360]
Patten E, Labban JD, Casenhiser DM, & Cotton CL (2016). Synchrony detection of linguistic stimuli
in the presence of faces: Neuropsychological implications for language development in ASD.
Developmental Neuropsychology, 41 (5–8), 362–374. [PubMed: 28059555]
Patten E, Watson LR, & Baranek GT (2014). Temporal synchrony detection and associations with
language in young children with ASD. Autism Research and Treatment, 2014, 1–8.
Author Manuscript
Pelphrey KA, Morris JP, & Mccarthy G (2004). Grasping the intentions of others: The perceived
intentionality of an action influences activity in the superior temporal sulcus during social
perception. Journal of Cognitive Neuroscience, 16 (10), 1706–1716. [PubMed: 15701223]
Perrault TJ, Vaughan JW, Stein BE, & Wallace MT (2003). Neuron-specific response characteristics
predict the magnitude of multisensory integration. Journal of Neurophysiology, 90(6), 4022–
4026. [PubMed: 12930816]
Perrault TJ, Vaughan JW, Stein BE, & Wallace MT (2005). Superior colliculus neurons use distinct
operational modes in the integration of multisensory stimuli. Journal of Neurophysiology, 93(5),
2575–2586. [PubMed: 15634709]
Petersen CC (2014). Cortical control of whisker movement. Annual Review of Neuroscience, 37, 183–
203.
Author Manuscript
Powers AR 3rd, Hevey MA, & Wallace MT (2012). Neural correlates of multisensory perceptual
learning. The Journal of Neuroscience, 32(18), 6263–6274. 10.1523/
JNEUROSCI.6138-11.20123366559 [PubMed: 22553032]
Powers AR, Hillock AR, & Wallace MT (2009). Perceptual training narrows the temporal window of
multisensory binding. Journal of Neuroscience, 29(39), 12265–12274. 10.1523/
Jneurosci.3501-09.2009 [PubMed: 19793985]
Raab DH (1962). Statistical facilitation of simple reaction times. Transactions of the New York
Academy of Sciences, 24, 574–590. [PubMed: 14489538]
Radvansky BA, & Dombeck DA (2018). An olfactory virtual reality system for mice. Nature
Communications, 9(1), 1–14.
Raposo D, Kaufman MT, & Churchland AK (2014). A category-free neural population supports
evolving demands during decision-making. Nature Neuroscience, 17, 1784–1792. [PubMed:
25383902]
Raposo D, Sheppard JP, Schrater PR, & Churchland AK (2012). Multisensory decision-making in rats
Author Manuscript
monoamine oxidase a and 5-ht transporter knock-out mice. The Journal of Neuroscience, 21(3),
884–896. [PubMed: 11157075]
Author Manuscript
Sartorato F, Przybylowski L, & Sarko DK (2017). Improving therapeutic outcomes in autism spectrum
disorders: Enhancing social communication and sensory processing through the use of interactive
robots. Journal of Psychiatric Research, 90, 1–11. [PubMed: 28213292]
Schaaf RC, Dumont RL, Arbesman M, & May-Benson TA (2018). Efficacy of occupational therapy
using Ayres Sensory Integration®: A systematic review. American Journal of Occupational
Therapy, 72(1), 7201190010p7201190011-7201190010p7201190010.
Schauder KB, Muller CL, Veenstra-VanderWeele J, & Cascio CJ (2015). Genetic variation in serotonin
transporter modulates tactile hyperresponsiveness in ASD. Research in Autism Spectrum
Disorders, 10, 93–100. [PubMed: 25558276]
Schlack A, Sterbing-D’Angelo SJ, Hartung K, Hoffmann KP, & Bremmer F (2005). Multisensory
space representations in the macaque ventral intraparietal area. Journal of Neuroscience, 25(18),
4616–4625. [PubMed: 15872109]
Schormans AL, & Allman BL (2018). Behavioral plasticity of audiovisual perception: Rapid
recalibration of temporal sensitivity but not perceptual binding following adult-onset hearing
Author Manuscript
Shaw LH, Freedman EG, Crosse MJ, Nicholas E, Chen AM, Braiman MS, … Foxe JJ (2020).
Operating in a multisensory context: Assessing the interplay between multisensory reaction time
facilitation and inter-sensory task-switching effects. Neuroscience, 436, 122–135. [PubMed:
32325100]
Sheppard JP, Raposo D, & Churchland AK (2013). Dynamic weighting of multisensory stimuli shapes
decision-making in rats and humans. Journal of Vision, 13(6), 4 10.1167/13.6.43649064
Sieben K, Röder B, & Hanganu-Opatz IL (2013). Oscillatory entrainment of primary somatosensory
cortex encodes visual control of tactile processing. The Journal of Neuroscience, 33 (13), 5736–
5749. 10.1523/JNEUROSCI.4432-12.2013 [PubMed: 23536087]
Siemann J, Muller C, Forsberg C, Blakely R, Veenstra-VanderWeele J, & Wallace M (2017). An
autism-associated serotonin transporter variant disrupts multisensory processing. Translational
Psychiatry, 7(3), e1067. [PubMed: 28323282]
Siemann JK, Muller CL, Bamberger G, Allison JD, Veenstra-VanderWeele J, & Wallace MT (2015). A
novel behavioral paradigm to assess multisensory processing in mice. Frontiers in Behavioral
Author Manuscript
Smiley JF, & Falchier A (2009). Multisensory connections of monkey auditory cerebral cortex.
Hearing Research, 258(1–2), 37–46. [PubMed: 19619628]
Author Manuscript
Smith EG, & Bennetto L (2007). Audiovisual speech integration and lipreading in autism. Journal of
Child Psychology and Psychiatry, and Allied Disciplines, 48(8), 813–821. 10.1111/
j.1469-7610.2007.01766.x
Song Y-H, Kim J-H, Jeong H-W, Choi I, Jeong D, Kim K, & Lee S-H (2017). A neural circuit for
auditory dominance over visual perception. Neuron, 93(4), 940–954.e946. [PubMed: 28162806]
Sparks DL, & Hartwich-Young R (1989). The deep layers of the superior colliculus. Reviews of
Oculomotor Research, 3, 213–255. [PubMed: 2486324]
Stanford TR, Quessy S, & Stein BE (2005). Evaluating the operations underlying multisensory
integration in the cat superior colliculus. The Journal of Neuroscience, 25(28), 6499–6508.
[PubMed: 16014711]
Stehberg J, Dang PT, & Frostig RD (2014). Unimodal primary sensory cortices are directly connected
by long-range horizontal projections in the rat sensory cortex. Frontiers in Neuroanatomy, 8, 93.
[PubMed: 25309339]
Stein BE (2012). The new handbook of multisensory processing. Cambridge, MA: MIT Press.
Author Manuscript
Stein BE, & Meredith MA (1993). The merging of the senses. Cambridge, MA: The MIT Press.
Stein BE, Meredith MA, Huneycutt WS, & McDade L (1989). Behavioral indices of multisensory
integration: orientation to visual cues is affected by auditory stimuli. Journal of Cognitive
Neuroscience, 1(1), 12–24. 10.1162/jocn.1989.1.1.12 [PubMed: 23968407]
Stein BE, & Stanford TR (2008). Multisensory integration: Current issues from the perspective of the
single neuron. Nature Reviews. Neuroscience, 9(4), 255–266. 10.1038/nrn2331 [PubMed:
18354398]
Stein BE, Stanford TR, & Rowland BA (2014). Development of multisensory integration from the
perspective of the individual neuron. Nature Reviews Neuroscience, 15(8), 520–535. [PubMed:
25158358]
Stein BE, Wallace MW, Stanford TR, & Jiang W (2002). Cortex governs multisensory integration in
the midbrain. The Neuroscientist, 8(4), 306–314. [PubMed: 12194499]
Stevenson R, Siemann J, Woynaroski T, Schneider B, Eberly H, Camarata S, & Wallace M (2013).
Brief report: Arrested development of audiovisual speech perception in autism spectrum
Author Manuscript
Stevenson RA, VanDerKlok RM, Pisoni DB, & James TW (2011). Discrete neural substrates underlie
complementary audiovisual speech integration processes. NeuroImage, 55(3), 1339–1345.
10.1016/j.neuroimage.2010.12.0633057325 [PubMed: 21195198]
Stevenson RA, & Wallace MT (2013). Multisensory temporal integration: Task and stimulus
dependencies. Experimental Brain Research, 227(2), 249–261. [PubMed: 23604624]
Stevenson RA, Wilson MM, Powers AR, & Wallace MT (2013). The effects of visual training on
multisensory temporal processing. Experimental Brain Research, 225(4), 479–489. [PubMed:
23307155]
Stevenson RA, Zemtsov RK, & Wallace MT (2012). Individual differences in the multisensory
temporal binding window predict susceptibility to audiovisual illusions. Journal of Experimental
Psychology. Human Perception and Performance., 38, 1517–1529. 10.1037/a0027339 [PubMed:
22390292]
Sugihara T, Diltz MD, Averbeck BB, & Romanski LM (2006). Integration of auditory and visual
communication information in the primate ventrolateral prefrontal cortex. Journal of
Author Manuscript
Wallace MT (2004). The development of multisensory processes. Cognitive Processing, 5(2), 69–83.
Wallace MT, Carriere BN, Perrault TJ, Vaughan JW, & Stein BE (2006). The development of cortical
multisensory integration. The Journal of Neuroscience, 26(46), 11844–11849 [PubMed:
17108157]
Wallace MT, Meredith MA, & Stein BE (1992). Integration of multiple sensory modalities in cat
cortex. Experimental Brain Research, 91(3), 484–488. [PubMed: 1483520]
Wallace MT, Meredith MA, & Stein BE (1993). Converging influences from visual, auditory, and
somatosensory cortices onto output neurons of the superior colliculus. Journal of
Neurophysiology, 69(6), 1797–1809. [PubMed: 8350124]
Wallace MT, Meredith MA, & Stein BE (1998). Multisensory integration in the superior colliculus of
the alert cat. Journal of Neurophysiology, 80(2), 1006–1010. [PubMed: 9705489]
Author Manuscript
Wallace MT, Ramachandran R, & Stein BE (2004). A revised view of sensory cortical parcellation.
Proceedings of the National Academy of Sciences of the United States of America, 101(7),
2167–2172. 10.1073/pnas.0305697101357070 [PubMed: 14766982]
Wallace MT, & Stein BE (1994). Crossmodal synthesis in the midbrain depends on input from cortex.
Journal of Neurophysiology, 71(1), 429–432. [PubMed: 8158240]
Wallace MT, & Stein BE (1996). Sensory organization of the superior colliculus in cat and monkey.
Progress in Brain Research, 112, 301–311. [PubMed: 8979837]
Wallace MT, & Stein BE (1997). Development of multisensory neurons and multisensory integration
in cat superior colliculus. The Journal of Neuroscience, 17(7), 2429–2444. [PubMed: 9065504]
Wallace MT, & Stein BE (2000). Onset of cross-modal synthesis in the neonatal superior colliculus is
gated by the development of cortical influences. Journal of Neurophysiology, 83(6), 3578–3582.
[PubMed: 10848574]
Wallace MT, & Stein BE (2001). Sensory and multisensory responses in the newborn monkey superior
colliculus. The Journal of Neuroscience, 21(22), 8886–8894. [PubMed: 11698600]
Author Manuscript
Wallace MT, & Stein BE (2007). Early experience determines how the senses will interact. Journal of
Neurophysiology, 97(1), 921–926. 10.1152/jn.00497.2006 [PubMed: 16914616]
Wallace MT, & Stevenson RA (2014). The construct of the multisensory temporal binding window and
its dysregulation in developmental disabilities. Neuropsychologia, 64, 105–123. [PubMed:
25128432]
Wallace MT, Wilkinson LK, & Stein BE (1996). Representation and integration of multiple sensory
inputs in primate superior colliculus. Journal of Neurophysiology, 76(2), 1246–1266. [PubMed:
8871234]
Wallace MT, Woynaroski TG, & Stevenson RA (2020). Multisensory integration as a window into
orderly and disrupted cognition and communication. Annual Review of Psychology, 71, 193–
219.
Wang Y, Celebrini S, Trotter Y, & Barone P (2008). Visuo-auditory interactions in the primary visual
cortex of the behaving monkey: Electrophysiological evidence. BMC Neuroscience, 9(1), 79.
[PubMed: 18699988]
Author Manuscript
Warren Z, Veenstra-VanderWeele J, Stone W, Bruzek JL, Nahmias AS, Foss-Feig JH, … Glasser AM
(2011). Therapies for children with autism spectrum disorders. Rockville (MD): Agency for
Healthcare Research and Quality (US).
Waterhouse BD, Azizi SA, Burne RA, & Woodward DJ (1990). Modulation of rat cortical area 17
neuronal responses to moving visual stimuli during norepinephrine and serotonin
microiontophoresis. Brain Research, 514(2), 276–292. [PubMed: 2357542]
Watkins S, Shams L, Tanaka S, Haynes J-D, & Rees G (2006). Sound alters activity in human V1 in
association with illusory visual perception. NeuroImage, 31(3), 1247–1256. [PubMed:
16556505]
Watling R, & Hauer S (2015). Effectiveness of Ayres Sensory Integration® and sensory-based
interventions for people with autism spectrum disorder: A systematic review. American Journal
of Occupational Therapy, 69(5), 6905180030p6905180031-6905180030p6905180012.
Watson LR, Patten E, Baranek GT, Poe M, Boyd BA, Freuler A, & Lorenzi J (2011). Differential
associations between sensory response patterns and language, social, and communication
measures in children with autism or other developmental disabilities. Journal of Speech,
Author Manuscript
Yaguchi A, & Hidaka S (2018). Distinct autistic traits are differentially associated with the width of the
multisensory temporal binding window. Multisensory Research, 31(6), 523–536. [PubMed:
Author Manuscript
31264616]
Yi F, Danko T, Botelho SC, Patzke C, Pak C, Wernig M, & Sudhof TC (2016). Autism-associated
SHANK3 haploinsufficiency causes Ih channelopathy in human neurons. Science, 352(6286),
aaf2669. [PubMed: 26966193]
Zhang M, Kwon SE, Ben-Johny M, O’Connor DH, & Issa JB (2020). Spectral Hallmark of auditory-
tactile interacttions in the mouse somatosensory cortex. Communications Biology, 3(1), 1–17.
[PubMed: 31925316]
Zhou H. y., Cai X. l., Weigl M, Bang P, Cheung EF, & Chan RC (2018). Multisensory temporal
binding window in autism spectrum disorders and schizophrenia spectrum disorders: A
systematic review and meta-analysis. Neuroscience and Biobehavioral Reviews, 86, 66–76.
[PubMed: 29317216]
Zilbovicius M, Meresse I, Chabane N, Brunelle F, Samson Y, & Boddaert N (2006). Autism, the
superior temporal sulcus and social perception. Trends in Neurosciences, 29(7), 359–366.
[PubMed: 16806505]
Author Manuscript
Author Manuscript
Author Manuscript
Table 1.
al., 2000; Nath et al., 2011; Nath & 1981; Ghazanfar, Chandrasekaran, & 2001; Wallace et al., Larue, Winer, & Barth, 2003; Lippert, 2018]
Beauchamp, 2012; Powers 3rd, Hevey, & Logothetis, 2008; Ghazanfar & 1992, 1993; Wallace Takagaki, Kayser, & Ohl, 2013; Menzel PFC: [Sharma &
Wallace, 2012; Stevenson & James, Schroeder, 2006; Gu, Angelaki, & et al., 2006; Wallace & Barth, 2005; Raposo, Kaufman, & Bandyopadhyay, 2019]
2009; Stevenson, VanDerKlok, Pisoni, & DeAngelis, 2008] & Stein, 1994; Churchland, 2014] V2L: [Meijer et al., 2020]
James, 2011] Parietal cortex: [Avillac, Hamed, & Wilkinson et al., PFC: [Lipton, Alvarez, & Eichenbaum, Insula: [Gogolla, Takesian,
Parietal cortex: [Calvert, Hansen, Duhamel, 2007; Cohen, 2009; 1996] 1999; Reid, Jacklin, & Winters, 2014] Feng, Fagiolini, & Hensch,
Iversen, & Brammer, 2001; Driver & Schlack, Sterbing-D’Angelo, Unisensory cortices: Insula: [Rodgers, Benison, Klein, & 2014]
Noesselt, 2008; Miller & D’esposito, Hartung, Hoffmann, & Bremmer, [Allman & Meredith, Barth, 2008] Unisensory cortices and border
2005] 2005] 2007] V2L: [Hirokawa et al., 2008; Schormans regions: [Cohen, Rothschild, &
PFC: [Calvert et al., 2001; Miller & PFC: [Fuster, Bodner, & Kroger, et al., 2017; Schormans & Allman, 2019; Mizrahi, 2011; Deneux et al.,
D’esposito, 2005] 2000; Romanski, Averbeck, & Diltz, Schormans, Typlt, & Allman, 2017] 2019; Knöpfel et al., 2019;
Unisensory cortices: [Calvert et al., 2005; Romanski & Goldman-Rakic, Unisensory cortices and border regions: McClure Jr & Polack, 2019;
1997; Foxe & Schroeder, 2005; Lakatos, 2002; Sugihara, Diltz, Averbeck, & [Barth, Goldberg, Brett, & Di, 1995; Meijer, Montijn, Pennartz, &
Chen, O’Connell, Mills, & Schroeder, Romanski, 2006] Bieler, Sieben, Schildt, Röder, & Lansink, 2017; Morrill &
2007; Martuzzi et al., 2007; Molholm et Unisensory cortices: [Foxe & Hanganu-Opatz, 2017; Maruyama & Hasenstaub, 2018; Olcese et
al., 2002; Watkins, Shams, Tanaka, Schroeder, 2005; Ghazanfar, Maier, Komai, 2018; Sieben, Röder, & Hanganu- al., 2013]
Haynes, & Rees, 2006] Hoffman, & Logothetis, 2005; Opatz, 2013; Wallace et al., 2004; Xu,
Lakatos et al., 2007; Rockland & Sun, Zhou, Zhang, & Yu, 2014]
Ojima, 2003; Wang, Celebrini,
Trotter, & Barone, 2008]
Subcortical SC: [Olive, Tempelmann, Berthoz, & SC: [Frens & Van Opstal, 1998; SC: [Meredith & SC: [Gharaei, Arabzadeh, & Solomon, SC: [Doykos, Gilmer, Person,
Heinze, 2015] Wallace et al., 1996; Wallace & Stein, 1986a; 2018; Hirokawa et al., 2011; Lau, Manno, & Felsen, 2020; Drager &
Stein, 1996; Wallace & Stein, 2000] Meredith & Stein, Dong, Chan, & Wu, 2018; May, 2006; Hubel, 1975a; Drager &
1986b] Sparks & Hartwich-Young, 1989] Hubel, 1975b]
Behavioral AV detection: [Calvert et al., 2004; AV detection: [Cappe et al., 2010; AV detection: AV detection: [Carandini & Churchland, AV detection: [Meijer, Pie,
Frassinetti, Bolognini, & Làdavas, 2002; Miller, Ulrich, & Lamarre, 2001; [Burnett, Henkel, 2013; Gleiss & Kayser, 2012; Hirokawa Dolman, Pennartz, & Lansink,
Miller, 1982; Raab, 1962] Wang et al., 2008] Stein, & Wallace, et al., 2008; Sakata, Yamamori, & 2018; Meijer et al., 2020;
AV discrimination: [Raposo, Sheppard, SJ and TOJ tasks: [Frens & Van 2002; Burnett, Stein, Sakurai, 2004] (Siemann et al., 2015]
Schrater, & Churchland, 2012; Sheppard, Opstal, 1998; Kayser, Petkov, & Chaponis, & AV discrimination: [Raposo et al., 2012,
Raposo, & Churchland, 2013] Logothetis, 2008] Wallace, 2004; Jiang 2014; Sheppard et al., 2013]
SJ and TOJ tasks: [Colonius & et al., 2002; Stein, SJ and TOJ tasks: [Schormans & Allman,
Diederich, 2004; Stevenson, Ghose, Meredith, Huneycutt, 2018; Schormans, Scott, et al., 2017;
Note. This is by far not an exhaustive list; there are more examples of studies that can be found under each category for each species, and there are other cortical, subcortical, and behavioral findings that
could be listed for each species, which are not highlighted. The purpose of this table is meant only to begin to highlight some of the neuronal and behavioral similarities across these animal species.
Abbreviations: AES: anterior ectosylvian sulcal cortex; AV: audiovisual; PFC: prefrontal cortex; SC: superior colliculus; SJ and TOJ: simultaneity judgment and temporal order judgment; STS: superior
temporal sulcus; V2L: lateral portion of V2 (secondary visual cortex).
Page 31
Author Manuscript Author Manuscript Author Manuscript Author Manuscript
Table 2.
Findings of Altered Multisensory Processing in Neural and Behavioral Domains for Mouse Models Associated with ASD
Note. Note the recency, yet relatively limited number of studies, demonstrating the importance of these types of studies for future investigations.