You are on page 1of 32

HHS Public Access

Author manuscript
Autism Res. Author manuscript; available in PMC 2021 September 01.
Author Manuscript

Published in final edited form as:


Autism Res. 2020 September ; 13(9): 1430–1449. doi:10.1002/aur.2375.

Approaches to Understanding Multisensory Dysfunction in


Autism Spectrum Disorder
Justin K. Siemann,
Department of Biological Sciences, Vanderbilt University, Nashville, Tennessee

Jeremy Veenstra-VanderWeele,
Department of Psychiatry, Columbia University, Center for Autism and the Developing Brain, New
Author Manuscript

York Presbyterian Hospital, and New York State Psychiatric Institute, New York, New York

Mark T. Wallace
Department of Psychiatry, Vanderbilt University, Nashville, Tennessee; Department of
Psychology, Vanderbilt University, Nashville, Tennessee; Department of Hearing and Speech
Sciences, Vanderbilt University, Nashville, Tennessee; Kennedy Center for Research on Human
Development, Vanderbilt University, Nashville, Tennessee

Abstract
Abnormal sensory responses are a DSM-5 symptom of autism spectrum disorder (ASD), and
research findings demonstrate altered sensory processing in ASD. Beyond difficulties with
processing information within single sensory domains, including both hypersensitivity and
Author Manuscript

hyposensitivity, difficulties in multisensory processing are becoming a core issue of focus in ASD.
These difficulties may be targeted by treatment approaches such as “sensory integration,” which is
frequently applied in autism treatment but not yet based on clear evidence. Recently,
psychophysical data have emerged to demonstrate multisensory deficits in some children with
ASD. Unlike deficits in social communication, which are best understood in humans, sensory and
multisensory changes offer a tractable marker of circuit dysfunction that is more easily translated
into animal model systems to probe the underlying neurobiological mechanisms. Paralleling
experimental paradigms that were previously applied in humans and larger mammals, we and
others have demonstrated that multisensory function can also be examined behaviorally in rodents.
Here, we review the sensory and multisensory difficulties commonly found in ASD, examining
laboratory findings that relate these findings across species. Next, we discuss the known
neurobiology of multisensory integration, drawing largely on experimental work in larger
Author Manuscript

mammals, and extensions of these paradigms into rodents. Finally, we describe emerging
investigations into multisensory processing in genetic mouse models related to autism risk. By
detailing findings from humans to mice, we highlight the advantage of multisensory paradigms
that can be easily translated across species, as well as the potential for rodent experimental
systems to reveal opportunities for novel treatments.

Lay Summary:

Address for correspondence and reprints: Justin K. Siemann, Department of Biological Sciences, Vanderbilt University, 8270 MRB III
BioSci Bldg, 465 21st Ave South, Nashville, TN 37235. justin.k.siemann@vanderbilt.edu.
Siemann et al. Page 2

Sensory and multisensory deficits are commonly found in ASD and may result in cascading
Author Manuscript

effects that impact social communication. By using similar experiments to those in humans, we
discuss how studies in animal models may allow an understanding of the brain mechanisms that
underlie difficulties in multisensory integration, with the ultimate goal of developing new
treatments.

Keywords
multisensory integration; autism spectrum disorder; mouse models; serotonin; visual processing;
auditory processing

Introduction
Autism spectrum disorder (ASD) represents a behaviorally defined cluster of complex
Author Manuscript

neurodevelopmental conditions characterized by impairment in social communication as


well as the presence of repetitive and restrictive behaviors [APA, 2013]. Approximately 1 in
59 children are diagnosed with ASD, with 3–4:1 higher prevalence in males than females
[CDC, 2019]. Beyond the classic social and repetitive behavior symptoms, abnormal sensory
responses are also highly prevalent, with estimates of up to 85% of individuals with ASD
exhibiting sensory features [Iarocci & McDonald, 2006; Robertson & Baron-Cohen, 2017].
Based upon this recognition, the DSM-5 now includes sensory disturbance as a symptom
within the restricted, repetitive behavior domain of ASD [APA, 2013].

There is clear evidence that sensory processing across multiple modalities (sight, sound,
touch, smell, and taste) is impacted in ASD [Baum, Stevenson, & Wallace, 2015a;
Robertson & Baron-Cohen, 2017]. Studies have shown that individuals with ASD can be
Author Manuscript

hypersensitive or hyposensitive to a variety of sensory stimuli, with both hypo-and


hypersensitivity possible even within a single individual (e.g. hypersensitive to sight and
hyposensitive to touch) [Hazen, Stornelli, O’Rourke, Koesterer, & McDougle, 2014; Lane,
Young, Baker, & Angley, 2010; Leekam, Nieto, Libby, Wing, & Gould, 2007]. While all
sensory modalities can be impacted in ASD [Robertson & Baron-Cohen, 2017], for the
scope of this review we will mainly focus on visual and auditory processing, as it has been
the major focus of multisensory work.

Founded on these differences in sensory function, a variety of sensory-based therapies and


interventions including the Ayres Sensory Integration intervention [Parham et al., 2011] have
been applied in ASD. Sensory-based therapies take a variety of approaches; including the
active implementation and engagement of sensory stimuli with play behavior with the goal
Author Manuscript

of improving adaptive behaviors, along with interventions focused on activating the arousal
system with more passive inclusion of sensory information in the child’s normal routine,
which may ameliorate some of the observed atypical responses to sensory stimuli (i.e.
hyper- or hypo-sensitivity) [Ayres, 1972; Case-Smith, Weaver, & Fristad, 2015]. In addition
to the Ayres Sensory Integration intervention, studies have evaluated these sensory-based
therapies with mixed results, largely concluding there is little efficacy of these treatments or
at least more evidence is still needed [Case-Smith et al., 2015; Lang et al., 2012; Schaaf,
Dumont, Arbesman, & May-Benson, 2018; Watling & Hauer, 2015]. In order to apply novel

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 3

sensory-based interventions, it is important to understand the underlying neural mechanisms


Author Manuscript

responsible for the atypical sensory responses consistently observed in ASD. Toward this
end, investigators have recently begun to harness studies on the neuronal, systems, and
behavioral levels to evaluate performance not only in response to stimuli within a single
sensory modality, but also when stimuli from multiple modalities are present [Baum et al.,
2015a].

Introduction to Multisensory Processing


Multisensory processing represents the merging of sensory information from different
modalities [Murray & Wallace, 2011]. We are constantly bombarded in our daily lives by a
variety of sensory stimuli yet are largely able to integrate this information seamlessly in
order to generate a coherent perception of the world. This has led to numerous studies
investigating how the brain is capable of combining information from the various senses into
Author Manuscript

a unified percept or gestalt [Calvert, Spence, & Stein, 2004].

Investigations in both humans and animals have shown that the combination of information
from multiple senses can result in behavioral enhancements, such as increased accuracies
and reduced reaction times, when compared to behavioral performances under unisensory
(i.e. visual, auditory, or tactile alone) conditions [Stevenson et al., 2014]. For example,
human psychophysical studies have shown that the added visual information of seeing a
speaker’s lip movements can facilitate speech intelligibility in a noisy environment [Sumby
& Pollack, 1954]. In addition to this work in the behavioral and perceptual realms, a great
deal of work has now characterized the neural circuits and processes that underlie
multisensory integration [Stein & Stanford, 2008]. Collectively, this work has shown that
multisensory inputs converge at many sites in the nervous system [Meredith, 2002; Wallace,
Author Manuscript

Meredith, & Stein, 1992; Wallace, Ramachandran, & Stein, 2004] and that this convergence
frequently results in dramatic changes in neuronal responses when stimuli from multiple
modalities are presented collectively [Meredith & Stein, 1983].

Atypical Multisensory Processing in Autism Spectrum Disorder


Based on the wealth of evidence highlighting disturbances across a number of sensory
systems in ASD, there has been keen interest in studying the integration of information
across these different sensory modalities [Beker, Foxe, & Molholm, 2018; Cuppini et al.,
2017; Feldman et al., 2018]. Such multisensory integration is fundamental in allowing us to
successfully interpret communicative and social signals. Numerous neural and behavioral
studies have now demonstrated atypical multisensory processing in individuals with ASD
[Baum et al., 2015a; Foss-Feig et al., 2010; Iarocci & McDonald, 2006; Kwakye, Foss-Feig,
Author Manuscript

Cascio, Stone, & Wallace, 2011; Marco, Hinkley, Hill, & Nagarajan, 2011; Russo et al.,
2010; Stevenson, Segers, Ferber, Barense, & Wallace, 2014; Wallace & Stevenson, 2014].
Evidence of decreased multisensory gain or benefit has been reported utilizing both simple
and complex stimuli [Collignon et al., 2013; Mongillo et al., 2008; Stevenson et al., 2014;
Stevenson et al., 2014]. Further, these multisensory deficits have been correlated with
symptom severity [Brandwein et al., 2015; Yaguchi & Hidaka, 2018], and the development
of multisensory function and its neural underpinnings may be atypical in ASD as well

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 4

[Beker et al., 2018; Brandwein et al., 2012; Cuppini et al., 2017; Foxe et al., 2015;
Author Manuscript

Stevenson et al., 2013; Taylor, Isaac, & Milne, 2010].

One common method to measure multisensory integration is with the use of cross-modal
illusions. The McGurk effect is a well-known audiovisual speech illusion that has been
evaluated in children, adults, as well as in individuals with neurodevelopmental disorders
[Mcgurk & Macdonald, 1976; van Wassenhove, Grant, & Poeppel, 2007; Woynaroski et al.,
2013]. In this illusion, a speaker typically mouths the phoneme “ga” while viewers
simultaneously hear an auditory “ba.” Under these conflicting audiovisual conditions,
individuals tend to perceive an entirely new speech percept, typically reporting that the
speaker uttered the phoneme “da” or “tha.” Under these illusory conditions, it is believed
that subjects are integrating the visual “ga” with the auditory “ba” to create the novel percept
[Mcgurk & Macdonald, 1976]. Brain imaging studies have demonstrated that the superior
temporal sulcus (STS) is integral for the perception of the McGurk effect [Nath &
Author Manuscript

Beauchamp, 2012; Nath, Fava, & Beauchamp, 2011], and when this brain region is disrupted
by transcranial magnetic stimulation (TMS) in typically developing individuals, illusory
percepts decrease dramatically [Beauchamp, Nath, & Pasalar, 2010].

A variety of studies has also evaluated the McGurk effect in clinical populations, including
in those with ASD. Overall, studies have demonstrated that individuals with ASD tend to
perceive the McGurk effect less frequently in comparison to TD individuals [Irwin,
Tornatore, Brancazio, & Whalen, 2011; Mongillo et al., 2008; Stevenson, Siemann, et al.,
2013; Stevenson, Siemann, Schneider, et al., 2014; Taylor et al., 2010], with the majority of
the studies describing this as a decrease in the strength or magnitude of multisensory
integration. In addition, studies have evaluated and found differences in the perception of the
McGurk effect across development in individuals with ASD [Stevenson, Siemann, et al.,
Author Manuscript

2013; Taylor et al., 2010]. Also, studies, utilizing various psychophysical behavioral
measures, have determined that the development of multisensory integration might be
delayed in these individuals [Beker et al., 2018; Brandwein et al., 2012; Foxe et al., 2015;
Stevenson, Siemann, et al., 2013; Taylor et al., 2010], along with evidence of sex-dependent
differences in audiovisual speech development in both typically developed and ASD
populations [Ross, Del Bene, Molholm, Frey, & Foxe, 2015]. Interestingly, neuroimaging
studies have found both structural and functional differences in the STS of those with ASD,
and this brain region, which has been shown to be important for the McGurk illusion, has
also been established as a neural hub for multisensory integration and social processing
[Beauchamp, Argall, Bodurka, Duyn, & Martin, 2004; Boddaert et al., 2004; Calvert,
Campbell, & Brammer, 2000; Gervais et al., 2004; Redcay, 2008; Stevenson & James, 2009;
Zilbovicius et al., 2006].
Author Manuscript

One of the critical factors in the integration of information across the different senses is
time, in that only when these stimuli are sufficiently close together in time are they
perceptually integrated. Specifically, there appears to be a window of time within which
events specified by the different sensory modalities are perceptually “bound” [Hillock,
Powers, & Wallace, 2011]. This multisensory temporal binding window (TBW) makes a
great deal of ethological sense, since sensory stimuli presented in close temporal proximity
are highly likely to be associated with the same event. To measure the TBW, auditory and

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 5

visual stimuli are presented at various asynchronies relative to one another (including at
Author Manuscript

simultaneity), and participants are asked to respond if they perceived these sensory signals
as synchronous or not [Stevenson, Ghose, Fister, et al., 2014]. Typically developing
participants tend to report high proportions of simultaneity for stimuli separated by about
300 ms, depending on the complexity of stimuli [Stevenson & Wallace, 2013]. However, it
has been shown that the TBW can be dependent on participant task demands [Mégevand,
Molholm, Nayak, & Foxe, 2013], audiovisual processing may be due to attentional
switching [Shaw et al., 2020], and these capabilities may be developmentally different for
typically developed individuals compared to individuals with ASD [Crosse, Foxe, &
Molholm, 2019]. These points highlight the fact that task design, motivation, and attentional
components are all highly relevant when selecting and utilizing psychophysical paradigms to
investigate multisensory processing, especially in clinical populations such as ASD. Lastly,
the width of the TBW has been shown to relate back to the perceptual illusions described
Author Manuscript

above, specifically in that individuals who have wider TBWs tend to report the McGurk
illusion less frequently [Stevenson, Zemtsov, & Wallace, 2012]. Such a finding suggests an
important relationship between the TBW and the strength or magnitude of the multisensory
binding process.

Emerging evidence suggests that the multisensory temporal binding window may be
abnormally wide in individuals with ASD [Baum et al., 2015a; Baum, Stevenson, &
Wallace, 2015b; Foss-Feig et al., 2010; Kwakye et al., 2011; Stevenson, Siemann,
Woynaroski, et al., 2014; Wallace & Stevenson, 2014], resulting in degraded perceptual
representations. For example, Stevenson and colleagues measured the TBW in both TD
children and individuals with ASD across three levels of stimulus complexity: simple visual
flashes and auditory beeps, more complex tool stimuli (i.e. a hammer hitting a nail) and a
speaker uttering speech syllables. Temporal binding windows increased in width as the
Author Manuscript

stimulus complexity increased for both groups, with the widest windows being observed for
speech stimuli. Significant differences in the TBW widths were observed between the
groups under the most complex conditions (i.e. speech), where individuals with ASD
demonstrated wider TBWs compared to TD individuals. Interestingly, no significant
differences in TBWs were observed between the groups for simple flashes and beeps and
tool stimuli. This suggests that multisensory temporal processing may be typical in ASD for
more simplistic audiovisual stimuli, however may become atypical as stimulus complexity
increases. An alternative interpretation is that by utilizing a speaker uttering syllables, there
may be intrinsic differences between the groups as there is evidence of atypical processing
of faces in ASD [Dalton et al., 2005; Dawson, Webb, & McPartland, 2005]. This study also
measured the McGurk effect, and individuals with ASD perceived the McGurk illusion less
frequently compared to TD individuals. Importantly, it was found that for individuals with
Author Manuscript

ASD, a wider TBW, regardless of the stimulus complexity, resulted in less perception of the
McGurk effect. Therefore, this study hypothesized those differences in multisensory
temporal processing for even low-level sensory stimuli (i.e. visual flashes and auditory
beeps) in individuals with ASD could result in potential cascading effects into higher order
domains such as speech or communication [Stevenson, Siemann, Schneider, et al., 2014].

In addition to the McGurk effect, the sound-induced flash illusion has also been examined in
individuals with ASD. In this illusion, one visual flash is paired with multiple auditory

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 6

beeps, resulting in the frequent reporting of multiple visual flashes [Shams, Kamitani, &
Author Manuscript

Shimojo, 2000]. Using this illusion, it was observed that children with ASD had a temporal
binding window that was almost twice as wide compared to TBWs of typically developing
controls [Foss-Feig et al., 2010].

Importantly, studies have now specifically demonstrated relationships between multisensory


temporal processing and severity of social and language impairments in individuals with
ASD [Feldman et al., 2018; Patten, Watson, & Baranek, 2014; Stevenson et al., 2018;
Wallace, Woynaroski, & Stevenson, 2020; Woynaroski et al., 2013]. Collectively, the body
of evidence points to altered multisensory temporal function in ASD, and these temporal
alterations are likely to have cascading effects on higher-order social communicative skills
that necessitate the rapid and accurate integration of multisensory information [Stevenson et
al., 2016, 2018; Yaguchi & Hidaka, 2018; Zhou et al., 2018].
Author Manuscript

In sum, numerous findings, using a variety of approaches, have demonstrated the presence of
multisensory and temporal processing deficits in individuals with autism spectrum disorder
[Baum et al., 2015b; Brandwein et al., 2015; Collignon et al., 2013; Foss-Feig et al., 2010;
Foxe et al., 2015; Irwin et al., 2011; Kwakye et al., 2011; Smith & Bennetto, 2007;
Stevenson, Siemann, et al., 2013; Stevenson, Siemann, Schneider, et al., 2014; Stevenson,
Siemann, Woynaroski, et al., 2014; Woynaroski et al., 2013]. In addition, there is emerging
evidence that (multi)sensory function may play a critical role in the establishment and
maintenance of more complex functions such as social communication [Baum et al., 2015a;
Robertson & Baron-Cohen, 2017; Stevenson et al., 2018; Stevenson, Siemann, Schneider, et
al., 2014; Thye, Bednarz, Herringshaw, Sartin, & Kana, 2018]. With mounting evidence of
atypical multisensory integration in ASD on both the behavioral and neural levels, cross-
species studies in animal models may allow for investigations to understand the underlying
Author Manuscript

neural mechanisms of multisensory processing, with the goal to translate that understanding
into testable hypotheses—and potential treatments—in the human population.

The Neurobiology of Multisensory Integration


The use of animal models has been integral in determining the development [Stein, Stanford,
& Rowland, 2014; Wallace & Stein, 1997, 2001; Wallace & Stein, 2007], neural
mechanisms [Jiang, Wallace, Jiang, Vaughan, & Stein, 2001; Wallace et al., 1992; Wallace &
Stein, 2000], and brain regions and circuits responsible for multisensory integration [Driver
& Noesselt, 2008; Stein, 2012]. From these studies, many of the connections and neural
structures critical for multisensory processing have been determined [Meredith & Stein,
1986b; Olcese, Iurilli, & Medini, 2013; Stein, Wallace, Stanford, & Jiang, 2002; Wallace &
Stein, 1994]. In addition, a body of work has begun to establish relationships between these
Author Manuscript

circuits and multisensory behaviors in a variety of species ranging from mice to non-human
primates [Cappe, Murray, Barone, & Rouiller, 2010; Hirokawa, Bosch, Sakata, Sakurai, &
Yamamori, 2008; Olcese et al., 2013; Wallace, Meredith, & Stein, 1998].

One of the best-studied models for this work has been the cat [Meredith & Stein, 1983].
Work in this model served to establish three main principles of multisensory integration
(spatial, temporal, and inverse effectiveness) dependent upon the physical characteristics of

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 7

the paired stimuli [Meredith, Nemitz, & Stein, 1987; Meredith & Stein, 1986a, 1986b]. The
Author Manuscript

spatial and temporal principles specify that stimuli from different sensory modalities
presented in close spatial and/or temporal proximity result in greater neural responses and
behavioral benefits compared to those presented in a disparate manner [Meredith et al.,
1987; Meredith & Stein, 1986a]. In addition, the principle of inverse effectiveness states that
as the effectiveness (i.e. loudness, brightness) of the unisensory stimuli decreases, the
resultant behavioral multisensory gain or benefit increases when stimuli are combined across
sensory modalities [Meredith & Stein, 1986b].

Two of the traditional means to measure and evaluate the gains conferred by multisensory
combinations are the interactive index (ii) and the mean statistical contrast (msc) [Stein &
Stanford, 2008]. The interactive index compares multisensory responses to the largest or
most effective unisensory response [Meredith & Stein, 1983, 1986a, 1986b]. The interactive
index can be either positive or negative depending on whether the stimulus combination
Author Manuscript

results in a gain (enhancement) or diminishment (depression) in response [Stein & Meredith,


1993].

Multisensory processing can be further evaluated by the mean statistical contrast. This
compares the responses to multisensory stimuli to the combined responses from the
individual sensory modalities [Perrault, Vaughan, Stein, & Wallace, 2003, 2005; Stanford,
Quessy, & Stein, 2005]. Three types of responses (i.e. subadditive, additive, or
superadditive) can be observed as a result of this evaluation.

In the cat, two of the best-studied structures from a multisensory perspective are the deep
layers of a subcortical structure, the superior colliculus (SC) [Meredith et al., 1987;
Meredith & Stein, 1986a, 1986b], and the anterior ectosylvian sulcal (AES) cortex [Wallace
Author Manuscript

et al., 1992; Wallace, Carriere, Perrault, Vaughan, & Stein, 2006]. Intriguingly, it is the
projections from the AES to the SC that gates the integration seen in SC neurons [Wallace,
Meredith, & Stein, 1993; Wallace & Stein, 1994]. Pharmacological [Wilkinson, Meredith, &
Stein, 1996] and deactivation studies [Jiang et al., 2001; Jiang, Jiang, & Stein, 2002; Wallace
& Stein, 1994] show that the AES is critical for this gating, in addition to being integral for
the behavioral enhancements observed under multisensory conditions.

Beyond studies in the cat; neural, circuit, and behavioral-based studies in the non-human
primate have been crucial in understanding the underlying mechanisms of multisensory
processing [Cappe et al., 2010; Cappe & Barone, 2005; Maier, Neuhoff, Logothetis, &
Ghazanfar, 2004; Rockland & Ojima, 2003]. Striking similarities have been shown in both
the development [Wallace, 2004], and neural response properties [Wallace & Stein, 2001] of
multisensory integration in the cat and monkey [Wallace & Stein, 1996]. As in the cat, work
Author Manuscript

in the non-human primate has shown the superior colliculus to be a major subcortical hub
for multisensory integration, and neurons adhere to the established principles of
multisensory integration [i.e. space, time, and effectiveness) (Wallace, Wilkinson, & Stein,
1996]. These similarities in development and neural response properties, between cat and
non-human primate models, suggest that multisensory integration may be highly conserved
across larger animal species [Table 1] [Stein & Stanford, 2008].

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 8

There are a variety of cortical regions in the non-human primate shown to receive
Author Manuscript

convergent input from multiple sensory modalities, and to demonstrate many of the classical
multisensory features [Ghazanfar & Schroeder, 2006; Kajikawa, Falchier, Musacchia,
Lakatos, & Schroeder, 2012]. These include the prefrontal cortex [Fuster et al., 2000;
Romanski et al., 2005; Romanski & Goldman-Rakic, 2002; Sugihara et al., 2006], parietal
cortex [Avillac et al., 2007; Cohen, 2009; Mullette-Gillman, Cohen, & Groh, 2005; Schlack
et al., 2005] and the medial superior temporal area [Bruce et al., 1981; Ghazanfar et al.,
2008; Gu et al., 2008]. Further, primary unisensory cortices such as visual and auditory
cortex have been shown to be modulated by multisensory stimulation, expanding greatly the
view of what constitutes a “multisensory” brain region [Foxe & Schroeder, 2005; Ghazanfar
et al., 2005; Lakatos et al., 2007; Wang et al., 2008]. In addition, non-human primate work
has identified both subcortical and cortical connections and circuits needed for multisensory
processing [Clavagnier, Falchier, & Kennedy, 2004; Falchier et al., 2010; Rockland &
Author Manuscript

Ojima, 2003; Smiley & Falchier, 2009]. Most relevant to this review are those studies that
then investigate the underlying neurophysiology and multisensory behavior in non-human
primates [Angelaki, Gu, & DeAngelis, 2009; Frens & Van Opstal, 1998; Gu et al., 2008;
Kayser et al., 2008; Miller et al., 2001], allowing for the mapping of neuron-behavior
relationships. Overall, these studies have demonstrated strikingly similar neuroanatomical
and functional connections between non-human primates and humans, see Table 1
[Ghazanfar & Schroeder, 2006; Kajikawa et al., 2012], and has provided critical insights into
the neural substrates needed for multisensory processing in an animal model that more
closely resembles the neural architecture found in humans.

While the original work that evaluated the underlying neurobiology and development of
multisensory processing was initially performed in animal models [Murray & Wallace,
2011; Stein et al., 2014; Stein & Stanford, 2008], numerous studies have identified the
Author Manuscript

necessary brain structures, and behavioral/perceptual benefits of multisensory integration in


humans as well (Table 1) [Calvert et al., 1997; Calvert et al., 2001; Colonius & Diederich,
2004; Driver & Noesselt, 2008; Foxe & Schroeder, 2005; Frassinetti et al., 2002; Lakatos et
al., 2007; Martuzzi et al., 2007; Miller, 1982; Miller & D’esposito, 2005; Olive et al., 2015;
Raab, 1962; Stein, 2012; Stevenson, Ghose, Fister, et al., 2014; Watkins et al., 2006].
Emerging work in humans now focuses on evaluating multisensory processing as it relates to
clinical disorders where sensory processing is known to be impacted [Baum et al., 2015a;
Wallace & Stevenson, 2014]. Although classical studies in larger animal models have been
instrumental in building our understanding of the neural and behavioral bases of
multisensory processing, it must be acknowledged that these species are somewhat limited
for modeling and studying disease. In contrast, the mouse and, more recently, the rat allow
facile creation of genetic models of disease risk [Nestler & Hyman, 2010]. Critically, one
Author Manuscript

key use of these animal models is the ability to back translate relevant clinical findings from
the human to ask questions focused on the mechanisms into how (multi)sensory systems
become atypical. Rodents may offer a translational bridge to evaluate the underlying
mechanisms for multisensory function in genetic models.

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 9

Multisensory Studies in Rats


Author Manuscript

In most rodent multisensory behavioral studies, rats have been used to assess multisensory
function based on their ability to complete complex operant tasks [Gleiss & Kayser, 2012;
Sakata et al., 2004; Sheppard et al., 2013]. Based on the relative ease of stimulus
presentation, and the relevance to the human population, audiovisual stimuli have been most
frequently used to assess multisensory function behaviorally [Carandini & Churchland,
2013; Sakata et al., 2004], despite the fact that rodents tend to have fairly poor vision and
more specialized somatosensory and olfactory systems [Ihara, Yoshikawa, & Touhara, 2013;
Petersen, 2014]. While these studies have differed on the variety of stimuli, specific stimulus
features, and task, they have all successfully demonstrated multisensory processing in rats,
finding similar effects to those found in larger animal species (Table 1) [Carandini &
Churchland, 2013; Gleiss & Kayser, 2012; Hirokawa et al., 2008; Raposo et al., 2012;
Raposo et al., 2014; Sakata et al., 2004].
Author Manuscript

Beyond behavioral studies, work has identified neural structures important for multisensory
processing in rats [Barth et al., 1995; Brett-Green et al., 2003; Hirokawa et al., 2008; Menzel
& Barth, 2005; Sieben et al., 2013; Tees, 1999], including specific border regions between
primary sensory cortices (visual, auditory, somatosensory) that contain a large number of
multisensory neurons and demonstrate similar multisensory characteristics to those found in
larger animal models [Schormans & Allman, 2019; Schormans, Typlt, & Allman, 2017;
Wallace et al., 2004; Xu et al., 2014]. In addition, studies have demonstrated that seemingly
irrelevant unisensory stimuli (i.e. auditory) can modulate responses in primary sensory
cortices (i.e. visual) [Bieler et al., 2017; Maruyama & Komai, 2018], which could be a result
of connectivity between primary sensory cortices [Stehberg, Dang, & Frostig, 2014]. In
addition to primary unisensory cortices and their overlapping border regions, studies have
Author Manuscript

identified the cerebellum [Ishikawa, Shimuta, & Hausser, 2015], insula [Rodgers et al.,
2008], parietal cortex [Brett-Green et al., 2003; Lippert et al., 2013; Menzel & Barth, 2005],
prefrontal cortex [Lipton et al., 1999; Reid et al., 2014], perirhinal cortex [Jacklin, Cloke,
Potvin, Garrett, & Winters, 2016], and the lateral entorhinal cortex [Doan, Lagartos-Donate,
Nilssen, Ohara, & Witter, 2019], as hubs for multisensory integration in rats. In addition to
these cortical structures, studies in rats have also investigated the role of the superior
colliculus (SC) in multisensory function on both the neural and behavioral level [Gharaei et
al., 2018; Hirokawa et al., 2011; Lau et al., 2018; May, 2006; Sparks & Hartwich-Young,
1989]. Importantly, many of these subcortical and cortical findings are similar to those that
have been observed in larger animal models (Table 1) [May, 2006; Wallace & Stein, 1996].
While similarities in responses to multisensory stimuli have been observed across animal
species in similar neural structures, it is important to note that there are large inter-species
Author Manuscript

differences in the functional roles of these regions. It is important to recognize that while
there may be structural similarities between species (i.e. regions that were once thought to
respond exclusively to unisensory stimuli, yet can respond under multisensory conditions)
this does not necessarily mean that these structures function in exactly the same context as in
larger animals and humans. Rodent studies allow us to gain insight into the cell-type
specificity, genetic underpinnings, and offer an initial view into the brain structures that may
be implicated. These findings are necessary as they can then be applied to larger animal

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 10

models, which share more analogous structural and functional brain regions to humans, with
Author Manuscript

the ultimate goal that these insights would lead to more clinically relevant applications.

A number of studies have utilized lesions, pharmacology, and/or cooling blockades to


disrupt and evaluate multisensory circuits and function in animal models [Burnett et al.,
2002; Burnett, Stein, Perrault, & Wallace, 2007; Jiang et al., 2001; Jiang, Jiang, & Stein,
2006; Wilkinson et al., 1996]. In the awake, behaving rat, Hirokawa and colleagues used
pharmacology to transiently disrupt and identify a border region between the primary visual
and auditory cortices, known as the lateral part of V2 or V2L, that is critical for audiovisual
processing in rats [Hirokawa et al., 2008]. Allman and colleagues recently investigated the
role of V2L in multisensory temporal processing [Schormans & Allman, 2018; Schormans,
Scott, et al., 2017]. Remarkably, rats are capable of performing classic psychophysical
simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks [Schormans, Scott, et
al., 2017], generating TBWs that appear to be similar to those found in the human
Author Manuscript

population [Wallace & Stevenson, 2014]. Interestingly, studies in the mouse have also shown
anatomical connections between primary auditory cortex (A1) and V2L, as well as primary
visual cortex (V1) and V2L [Charbonneau, Laramee, Boucher, Bronchti, & Boire, 2012;
Laramee, Kurotani, Rockland, Bronchti, & Boire, 2011], suggesting that V2L may be a
multisensory structure conserved across rodent species.

Emerging Multisensory Studies in Mice


Over the past decade, increasing interest has focused on multisensory research in the mouse
[Borges-Merjane & Trussell, 2015; Hornix, Havekes, & Kas, 2018; Olcese et al., 2013;
Radvansky & Dombeck, 2018; Reig & Silberberg, 2014]. Importantly, similar findings to the
rat have emerged in the mouse, where unisensory stimuli from other modalities can impact
Author Manuscript

neuronal responses in primary sensory cortices [Cohen et al., 2011; Deneux et al., 2019;
Knöpfel et al., 2019; McClure Jr & Polack, 2019; Meijer et al., 2017; Morrill & Hasenstaub,
2018; Zhang, Kwon, Ben-Johny, O’Connor, & Issa, 2020];results that may be supported by
growing evidence that primary sensory cortices may share underlying connections with one
another [Masse, Ross, Bronchti, & Boire, 2017; Morrill & Hasenstaub, 2018]. As observed
in the rat (Table 1), studies have also shown that the parietal cortex [Kuroki et al., 2018;
Lyamzin & Benucci, 2019; Najafi et al., 2019; Song et al., 2017], orbitofrontal cortex
[Sharma & Bandyopadhyay, 2019], cerebellum [Chabrol, Arenz, Wiechert, Margrie, &
DiGregorio, 2015] are neuronal hubs for multisensory integration in the mouse. Elegant
work identifying subcortical (i.e. inferior colliculus - [Dillingham, Gay, Behrooz, &
Gabriele, 2017], superior colliculus - [Doykos et al., 2020; Drager & Hubel, 1975a; Drager
& Hubel, 1975b] to cortical connections [Lesicko, Hristova, Maigler, & Llano, 2016],
Author Manuscript

cortical–cortical connections through transcranial electrical stimulation [Hishida, Kudoh, &


Shibuki, 2014], and multisensory neuronal hubs in association cortices via calcium imaging
[Kuroki et al., 2018] are beginning to demonstrate the underlying neural circuitry that
supports multisensory processing in the mouse.

In addition to these physiological and circuit-based findings, our lab was the first to
demonstrate multisensory processing on the behavioral level in mice [Siemann et al., 2015].
We found that mice from different background strains (i.e. C57 and 129 S4/S6) demonstrate

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 11

increased behavioral accuracies under paired audiovisual conditions compared to auditory or


Author Manuscript

visual conditions alone. Furthermore, this work in mice has begun to establish similarities in
the stimulus characteristics and principles that support these behavioral benefits [Siemann et
al., 2015]. Interestingly, a more recent study designed and utilized another behavioral
paradigm to demonstrate inverse effectiveness in mice as well, and with this task the authors
observed that stimulus detection is significantly improved under audiovisual conditions
[Meijer et al., 2018]. The establishment of these behavioral paradigms set the stage for work
structured to assess how molecular and circuitry manipulations impact multisensory
behavior.

To this point, most recent work is now attempting to connect the underlying neural
mechanisms with the associated multisensory behavior in the mouse model. It has been
demonstrated that neurons in the V2L, also known as area AL, are responsive to audiovisual
stimuli and critical for multisensory behavior [Meijer et al., 2020]. In this study, mice were
Author Manuscript

head fixed and performed a stimulus detection task in which auditory, visual, and combined
audiovisual stimuli were presented while simultaneously utilizing calcium imaging to record
from neurons in layer II/III in V2L or V1. Meijer et al., found that neural responses could be
elevated under audiovisual conditions in the V2L, and the neural population activity in this
region corresponded with behavioral performance under multisensory conditions. This work
is impactful as it identifies a novel neuronal hub, in the mouse, crucial for multisensory
behavior, and also demonstrates the similarities across rodent species (i.e. V2L is implicated
in both the rat and mouse), highlighting the possibility to utilize the mouse model in more
complex brain-behavior based studies that may be relevant for larger animal species.

Multisensory Studies in Mouse Models Associated with ASD


Author Manuscript

Based on emerging results in wild type mice, there have been a few investigations of
multisensory processing in genetically modified mice (Table 2) [Gogolla et al., 2014; Wada
et al., 2019]. Gogolla and colleagues evaluated the neural mechanisms of multisensory
processing in four mouse lines, including the Shank3 knockout mouse that models Phelan-
McDermid syndrome, which is robustly associated with autism diagnosis. They also
evaluated the Mecp2 null mouse model of Rett syndrome, the Gad65 knockout mouse model
of GABA deficiency, and the BTBR T+tf/J inbred strain of mouse that demonstrate
behaviors which serve as proxies for the human clinical phenotype of autism. However, it is
important to note caveats such as the BTBR T+tf/J inbred mouse strain represents no known
idiopathic form of ASD [Meyza & Blanchard, 2017], Rett syndrome itself is no longer
considered within the autism diagnosis [APA, 2013], and the Shank3 mouse model is
typically homozygous whereas in the clinical population, individuals display heterozygous
Author Manuscript

mutations [Yi et al., 2016].

The authors assessed multisensory processing, in anesthetized mice, in the insular cortex by
recording neural responses utilizing in vivo imaging of intrinsic flavoprotein fluorescence to
auditory (i.e. pure tones with varying frequencies and intensities to the contralateral ear) and
somatosensory stimuli (i.e. air puffs to the contralateral front paw) along with the combined
multisensory presentations. Multisensory responses were atypical and the magnitude of
multisensory enhancement was reduced in the four selected lines in comparison to wild type

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 12

animals. In addition, differences were observed in the size and densities of GAD65 puncta,
Author Manuscript

and atypical miniature inhibitory postsynaptic currents in pyramidal cells from layers 2/3 of
the insular cortex were found in response to auditory stimuli. Interestingly, the authors also
found that the maturation of responses to auditory stimuli was atypical in BTBR mice across
multiple developmental time periods. Based on these differences in inhibitory signaling and
altered developmental trajectories, the authors chronically administered diazepam, a
GABAA receptor positive allosteric modulator, between postnatal days 15–28. Multisensory
function was then assessed during adulthood, with neural responses normalizing under
multisensory conditions, compared to responses in control wild type mice, as well as a
normalization of the abnormal self-grooming behaviors seen in these models [Gogolla et al.,
2014]. In addition, the authors treated these mouse models with diazepam later in life, at
postnatal days 48–54, and found that diazepam did not normalize electrophysiological
responses to multisensory stimuli. These results suggest there are sensitive periods of
Author Manuscript

development in which manipulation of GABAergic circuitry can result in the normalization


of neuronal responses to multisensory stimuli. Therefore, this was the first study to
demonstrate atypical (multi)sensory processing on the neuronal level in mouse models
associated with ASD, demonstrated an altered developmental trajectory for multisensory
function, and provides intriguing evidence suggesting that that this multisensory dysfunction
could have cascading effects into core domains such as repetitive or restricted behaviors that
characterize ASD.

In addition, our lab demonstrated the first evidence of abnormal multisensory behavioral
responses in a genetically manipulated mouse (Table 2) [Siemann et al., 2017]. In this study,
we utilized a mouse with a point mutation in the serotonin transporter (SERT) in which a
glycine (Gly56) is substituted to an alanine (Ala56). In a study of multiplex ASD families,
the Ala56 allele was the most common of a group of rare amino acid coding variants in the
Author Manuscript

serotonin transporter [Sutcliffe et al., 2005]. The SERT Ala56 mouse shows elevated whole
blood serotonin levels, paralleling the hyperserotonemia found in 25% of children with
ASD, as well as abnormalities in social, communicative, and repetitive behaviors [Gabriele,
Sacco, & Persico, 2014; Muller, Anacker, & Veenstra-VanderWeele, 2016]. We found that
SERT Ala56 animals demonstrated a significant decrease in response accuracy to visual and
auditory targets; however, the most significant behavioral deficits were observed under
combined audiovisual conditions compared to wild type littermate controls. These
multisensory effects were larger compared to differences between groups for either auditory
or visual alone stimuli, however our results may also reflect poorer unisensory performance
in SERT Ala56 mice that is further compounded when auditory and visual stimuli are
combined. When examined in more detail, it was found that the behavioral gain normally
seen under multisensory conditions was eliminated in SERT Ala56 mice. Overall, these
Author Manuscript

findings represented the first to demonstrate atypical multisensory processing behaviorally


in a genetically manipulated mouse. These behavioral findings provide the foundation for
beginning to investigate the underlying cellular components and circuits that may lead to
altered multisensory processing related to genetic risk factors [Hornix et al., 2018].

The importance of these studies is reinforced by the fact that serotonin has been consistently
implicated in ASD [Cook & Leventhal, 1996; Mulder et al., 2004; Muller et al., 2016], has
been shown to be important for sensory cortical development [Cases et al., 1996; Gaspar,

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 13

Cases, & Maroteaux, 2003; Salichon et al., 2001], and can modulate responses to a variety
Author Manuscript

of sensory stimuli [Esaki et al., 2005; Hurley, Thompson, & Pollak, 2002; Waterhouse,
Azizi, Burne, & Woodward, 1990]. Most relevant to our work, there is evidence that
serotonin may play a role in cross-modal plasticity [Jitsuki et al., 2011]. Jitsuki and
colleagues demonstrated that under conditions of visual deprivation serotonin receptors are
critical for increased synaptic strength and more precise tuning of somatosensory responses
in the mouse barrel cortex. More recently, a study from Tang and colleagues demonstrated
that serotonin can modulate microcircuits in the dorsal cochlear nucleus to enhance
multisensory signaling by dampening auditory-related pathways [Tang & Trussell, 2017].
The loss of multisensory gain in the SERT Ala56 knock-in mouse further supports a role for
serotonin in multisensory processing, with the potential for future studies to assess the
ability of serotonin transporter blockade or other approaches to rescue this behavioral deficit
[Robson et al., 2018].
Author Manuscript

A Translational Bridge: From Human Clinical Findings to Rodent


Experimental Systems
Psychophysical studies show strikingly similar results and benefits under multisensory
conditions in humans, non-human primates, cats, rats, and mice using comparable
behavioral tasks (Table 1) [Carandini & Churchland, 2013; Raposo et al., 2012; Raposo et
al., 2014; Schormans, Scott, et al., 2017; Sheppard et al., 2013; Siemann et al., 2015;
Stevenson, Ghose, Fister, et al., 2014]. These studies demonstrate the potential utility of
rodent models for studying multisensory function and provide evidence that these behavioral
results could be compared across species, but a number of questions remain. By utilizing
behavioral tasks [Meijer et al., 2018; Siemann et al., 2015], future studies may assess
multisensory behavioral function for mice in order to mirror or relate these findings to those
Author Manuscript

being observed in human psychophysical tasks.

To this point, studies are already using very similar psychophysical experiments as observed
in humans to demonstrate multisensory temporal processing and temporal binding windows
(TBWs) in rats and have identified the neural structures needed for this multisensory
behavior [Schormans & Allman, 2018; Schormans, Scott, et al., 2017; Schormans, Typlt, &
Allman, 2017]. Therefore, can mouse studies be refined to evaluate multisensory temporal
processing, including temporal binding windows, and would these tasks implicate brain
regions (i.e. V2L) similar to those observed in the rat model? Interestingly, V2L has just
recently been identified in the mouse as a neural structure capable of responding to
audiovisual stimuli and critical for multisensory detection behavior [Meijer et al., 2020].
This would strongly suggest that V2L may have a critical role for multisensory temporal
Author Manuscript

processing in the mouse, highlighting the need for future studies to design novel behavioral
tasks to measure temporal binding windows in this animal model as well. As a result, a
number of important translational and clinically relevant studies could be evaluated. Prior
work has already begun to show atypical multisensory processing in mouse models
associated with ASD (Table 2) [Gogolla et al., 2014; Siemann et al., 2017; Wada et al.,
2019]. Would future investigations find similar multisensory temporal behavioral deficits in
genetic mouse models of ASD as are observed in the human clinical population?

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 14

In addition, mouse models allow for the screening and testing of specific pharmacological
Author Manuscript

agents that could provide important preclinical information in normalizing multisensory


function [Gogolla et al., 2014]. If genetic mouse models demonstrate atypical multisensory
function it may be possible to investigate associations between similar genetic mutations and
multisensory deficits in clinical subsets of ASD. For example, altered multisensory
processing has been shown in the SERT Ala56 mouse model [Siemann et al., 2017], do
individuals with ASD who have altered serotonin signaling (genetically or via
hyperserotonemia) also demonstrate atypical multisensory function or temporal processing?
This could allow for future studies focused on identifying associations between genetic
abnormalities and multisensory function. Hypothetically, if there were a subset of
individuals with ASD that had specific genetic alterations in serotonin signaling and also
demonstrated altered multisensory processing, could the use of pharmacological treatment
targeting the serotonin system (i.e. an SSRI), in this specific subset, improve altered
Author Manuscript

multisensory processing? By evaluating and targeting multisensory dysfunction as a


potential biomarker in animal models, it may be possible to ultimately use and apply this
information for the clinical ASD population. While these studies may be difficult and clearly
not as straightforward as proposed, they do highlight the advantages that genetic rodent
models allow for in the assessment of multisensory behavioral function, which may be
translated for larger animal species and ultimately could prove to be relevant for clinical
applications.

Translational Approaches for ASD: Relationships Between Multisensory


and Higher-order Cognitive Function
While most studies in humans have focused on treating the core symptoms of ASD, this
approach has had relatively limited success [Dove et al., 2012; Warren et al., 2011]. To this
Author Manuscript

point there are only two currently approved drugs for the treatment of ASD, yet these treat
co-occurring behavioral problems, rather than core symptoms [Ghanizadeh, Sahraeizadeh, &
Berk, 2014]. A variety of pharmacologic compounds have been found to rescue social or
repetitive behaviors observed in genetic or behavioral mouse models of ASD [Gogolla et al.,
2014; Silverman et al., 2012; Silverman et al., 2015];however, those compounds that have
been tested in translational clinical trials have not yet demonstrated comparable benefits in
humans with autism spectrum disorder [Ooi, Weng, Kossowsky, Gerger, & Sung, 2017;
Silverman & Crawley, 2014; Veenstra-VanderWeele et al., 2017].

Although the focus of this intervention research, both in humans and in model systems, has
focused on these core symptoms, we argue that it may be beneficial to test treatments or
interventions that target altered (multi)sensory function. Since multisensory integration is
Author Manuscript

critical for our perception of the world and is integral for the interpretation and
understanding of social communicative signals, it is possible that treatment that targets of
these sensory disturbances may have therapeutic effects that extend to core symptom
domains. Indeed, if multisensory integration is a key building block in the construction of
higher-order cognitive representations, it may serve as a tractable target for intervention that
would have benefits that have cascading impact across both the processing hierarchy as well
as in development [Baum et al., 2015a; Robertson & Baron-Cohen, 2017; Stevenson et al.,

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 15

2018; Stevenson, Siemann, Schneider, et al., 2014; Thye et al., 2018]. With this possibility, it
Author Manuscript

is important to carefully evaluate the potential relationships between multisensory


processing and clinical domains such as social communication in ASD [Cascio, Woynaroski,
Baranek, & Wallace, 2016; Feldman et al., 2018; Martínez et al., 2019; Patten, Labban,
Casenhiser, & Cotton, 2016; Sartorato, Przybylowski, & Sarko, 2017; Wallace et al., 2020;
Yaguchi & Hidaka, 2018].

Currently, the hypothesis that atypical multisensory processing underlies core ASD
symptoms is plausible, yet it has only some evidence in humans, and much of that evidence
is indirect. For example, Stevenson et al. demonstrated that atypical multisensory temporal
processing in individuals with ASD resulted in poorer performance in perceiving a
multisensory illusion that utilizes speech related stimuli [Stevenson, Siemann, Schneider, et
al., 2014]. In addition, the brain region (STS) shown to be critical in the perception of this
illusion [McGurk effect) (Beauchamp et al., 2010; Nath et al., 2011; Nath & Beauchamp,
Author Manuscript

2012], is also a key hub for the processing of social communicative stimuli [Pelphrey,
Morris, & Mccarthy, 2004; Stevenson et al., 2011; Zilbovicius et al., 2006], and has been
found to be both structurally and functionally atypical in ASD [Boddaert et al., 2004;
Redcay, 2008]. In addition, studies have found that sensory disturbances correlate with
greater deficits in social communication, along with the presence of more repetitive
behaviors in individuals with ASD [Boyd et al., 2010; Boyd, McBee, Holtzclaw, Baranek, &
Bodfish, 2009; Feldman et al., 2019; Foss-Feig, Heacock, & Cascio, 2012; Hilton et al.,
2010; Kern et al., 2007; Schauder, Muller, Veenstra-VanderWeele, & Cascio, 2015;
Stevenson et al., 2017; Stevenson et al., 2018; Watson et al., 2011; Zhou et al., 2018].

A number of future investigations could evaluate these relationships. Studies have shown
that multisensory training paradigms with feedback can result in the narrowing of the TBW
Author Manuscript

and increased functional connectivity between the STS and regions of the auditory and
visual cortices in typically developed individuals [De Niear, Gupta, Baum, & Wallace, 2018;
De Niear, Koo, & Wallace, 2016; Powers 3rd et al., 2012; Powers, Hillock, & Wallace,
2009]. Work has just begun to investigate if these same paradigms result in improved neural
and behavioral multisensory function in individuals with ASD [Feldman et al., 2020], and
plan to assess if these effects result in lasting changes to other domains such as social
communication. There is evidence that seemingly irrelevant sensory stimuli can impact
primary sensory systems [Ghazanfar & Schroeder, 2006]. Future studies could also utilize
sensory-based therapies and/or perceptual training to investigate the underlying mechanisms
for which improving unisensory function can result in increased multisensory function or
temporal processing [Stevenson, Wilson, et al., 2013], and if this ultimately leads to lasting
and positive changes in higher order domains for both typically developed and clinical
Author Manuscript

populations [Wallace et al., 2020].

Animal models could also be used to evaluate the relationship between multisensory
processing, social function, and repetitive behavior. Gogolla and colleagues demonstrated
that a pharmacological intervention in mouse models associated with ASD not only
normalized atypical multisensory neural responses, but it also reversed the presence of
repetitive behaviors found in these animal models [Gogolla et al., 2014]. They did not,
however, establish causality directly. Future studies should use optogenetics,

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 16

electrophysiology, and pharmacology to more directly investigate the known circuits and
Author Manuscript

neural populations that support multisensory function. For example, if relevant subcortical
(i.e. superior colliculus) and/or cortical brain regions (i.e. unisensory cortices, V2L, or
parietal cortex), critical for multisensory processing are selectively silenced either
pharmacologically, through cooling blockade or lesioning methods, and/or specific neuronal
populations (i.e. parvalbumin-positive interneurons) are reversibly silenced through
optogenetic techniques [Olcese et al., 2013], does this result in impaired social function or
the presence of repetitive/restricted behavior? The development of multisensory operant
tasks in mice [Meijer et al., 2018; Siemann et al., 2015] also raises the opportunity to
evaluate whether training in animal models can improve multisensory function, which may
parallel approaches being developed in humans that seek to capitalize on multisensory
training [De Niear et al., 2016; De Niear et al., 2018; Powers 3rd et al., 2012; Powers et al.,
2009].
Author Manuscript

Conclusion
Atypical multisensory integration may be a tractable phenotype that can be observed across
species and is becoming increasingly recognized as an important feature of ASD. In contrast
to social communicative behaviors, which are quite species-specific, or repetitive behaviors,
which take on very different forms in rodents than in humans, multisensory integration
appears to have preserved brain mechanisms and is readily tested through behavioral assays
that have been developed across species. With the coupling of these behavior paradigms and
genetic manipulations, it may be possible to investigate molecular, cellular, and systems-
level questions, and then attempt to translate these findings to the human clinical population.
This approach is further supported by emerging evidence that suggests that altered
(multi)sensory function has cascading effects that impact core symptoms, making this area
Author Manuscript

of study not just tractable, but also potentially highly impactful.

Acknowledgments
This work was supported by the Vanderbilt Kennedy Center HD1505 (MW), 5T32MH018921-24: Development of
Psychopathology: From Brain and Behavioral Science to Intervention (JKS), MH094604 (JVV), and MH096972
(JVV, MW).

Conflict of Interest

Consulting or Advisory Board: Roche, Novartis, SynapDx. Research funding: Roche, Novartis, SynapDx, Seaside
Therapeutics, Forest and editorial stipend: Springer, Wiley for JVV. The remaining authors report no financial
interests or potential conflicts of interest.

References
Author Manuscript

Allman BL, & Meredith MA (2007). Multisensory processing in “unimodal” neurons: cross-modal
subthreshold auditory effects in cat extrastriate visual cortex. Journal of Neurophysiology, 98(1),
545–549. 10.1152/jn.00173.2007 [PubMed: 17475717]
Angelaki DE, Gu Y, & DeAngelis GC (2009). Multisensory integration: Psychophysics,
neurophysiology, and computation. Current Opinion in Neurobiology, 19(4), 452–458. [PubMed:
19616425]
APA. (2013). Diagnostic and statistical manual of mental disorders DSM-5 development. Washington,
DC: American Psychiatric Association.

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 17

Avillac M, Hamed SB, & Duhamel J-R (2007). Multisensory integration in the ventral intraparietal
area of the macaque monkey. Journal of Neuroscience, 27(8), 1922–1932. [PubMed: 17314288]
Author Manuscript

Ayres AJ (1972). Sensory integration and learning disorders. Torrance, CA: Western Psychological
Services.
Barth DS, Goldberg N, Brett B, & Di S (1995). The spatiotemporal organization of auditory, visual,
and auditory-visual evoked potentials in rat cortex. Brain Research, 678(1), 177–190. [PubMed:
7620886]
Baum SH, Stevenson RA, & Wallace MT (2015a). Behavioral, perceptual, and neural alterations in
sensory and multisensory function in autism spectrum disorder. Progress in Neurobiology, 134,
140–160. [PubMed: 26455789]
Baum SH, Stevenson RA, & Wallace MT (2015b). Testing sensory and multisensory function in
children with autism spectrum disorder. JoVE (Journal of Visualized Experiments), 98, e52677–
e52677.
Beauchamp MS, Argall BD, Bodurka J, Duyn JH, & Martin A (2004). Unraveling multisensory
integration: Patchy organization within human STS multisensory cortex. Nature Neuroscience,
7(11), 1190–1192. [PubMed: 15475952]
Author Manuscript

Beauchamp MS, Nath AR, & Pasalar S (2010). fMRI-guided transcranial magnetic stimulation reveals
that the superior temporal sulcus is a cortical locus of the McGurk effect. The Journal of
Neuroscience, 30(7), 2414–2417. [PubMed: 20164324]
Beker S, Foxe JJ, & Molholm S (2018). Ripe for solution: Del-ayed development of multisensory
processing in autism and its remediation. Neuroscience and Biobehavioral Reviews, 84, 182–192.
[PubMed: 29162518]
Bieler M, Sieben K, Schildt S, Röder B, & Hanganu-Opatz IL (2017). Visual-tactile processing in
primary somatosensory cortex emerges before cross-modal experience. Synapse, 71(6), e21958.
Boddaert N, Chabane N, Gervais H, Good C, Bourgeois M, Plumet M, … Samson Y (2004). Superior
temporal sulcus anatomical abnormalities in childhood autism: A voxel-based morphometry MRI
study. Neuroimage, 23(1), 364–369. [PubMed: 15325384]
Borges-Merjane C, & Trussell LO (2015). ON and OFF unipolar brush cells transform multisensory
inputs to the auditory system. Neuron, 85(5), 1029–1042. [PubMed: 25741727]
Boyd BA, Baranek GT, Sideris J, Poe MD, Watson LR, Patten E, & Miller H (2010). Sensory features
Author Manuscript

and repetitive behaviors in children with autism and developmental delays. Autism Research, 3(2),
78–87. [PubMed: 20437603]
Boyd BA, McBee M, Holtzclaw T, Baranek GT, & Bodfish JW (2009). Relationships among repetitive
behaveiors, sensory features, and executive functions in high functioning autism. Research in
Autism Spectrum Disorders, 3(4), 959–966. [PubMed: 21475640]
Brandwein AB, Foxe JJ, Butler JS, Frey H-P, Bates JC, Shulman LH, & Molholm S (2015).
Neurophysiological indices of atypical auditory processing and multisensory integration are
associated with symptom severity in autism. Journal of Autism and Developmental Disorders,
45(1), 230–244. [PubMed: 25245785]
Brandwein AB, Foxe JJ, Butler JS, Russo NN, Altschuler TS, Gomes H, & Molholm S (2012). The
development of multisensory integration in high-functioning autism: High-density electrical
mapping and psychophysical measures reveal impairments in the processing of audiovisual inputs.
Cerebral Cortex, 23(6), 1329–1341. [PubMed: 22628458]
Brett-Green B, Fifkova E, Larue DT, Winer JA, & Barth DS (2003). A multisensory zone in rat
parietotemporal cortex: Intra- and extra-cellular physiology and thalamocortical connections.
Author Manuscript

Journal of Comparative Neurology, 460(2), 223–237. 10.1002/Cne.10637 [PubMed: 12687687]


Bruce C, Desimone R, & Gross CG (1981). Visual properties of neurons in a polysensory area in
superior temporal sulcus of the macaque. Journal of Neurophysiology, 46(2), 369–384. [PubMed:
6267219]
Burnett LR, Henkel CK, Stein BE, & Wallace MT (2002). A loss of multisensory orientation behaviors
is associated with the loss of parvalbumin-immunoreactive neurons in the superior colliculus.
FASEB Journal, 16(5), A735.

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 18

Burnett LR, Stein BE, Chaponis D, & Wallace MT (2004). Superior colliculus lesions preferentially
disrupt multisensory orientation. Neuroscience, 124(3), 535–547. 10.1016/
Author Manuscript

j.neuroscience.2003.12.026 [PubMed: 14980725]


Burnett LR, Stein BE, Perrault TJ, & Wallace MT (2007). Excitotoxic lesions of the superior colliculus
preferentially impact multisensory neurons and multisensory integration. Experimental Brain
Research, 179(2), 325–338. 10.1007/s00221-006-0789-8 [PubMed: 17146648]
Calvert G, Spence C, & Stein BE (2004). The handbook of multisensory processes. Cambridge, MA:
MIT Press.
Calvert GA, Bullmore ET, Brammer MJ, Campbell R, Williams SC, McGuire PK, … David AS
(1997). Activation of auditory cortex during silent lipreading. Science, 276 (5312), 593–596.
[PubMed: 9110978]
Calvert GA, Campbell R, & Brammer MJ (2000). Evidence from functional magnetic resonance
imaging of crossmodal binding in the human heteromodal cortex. Current Biology, 10(11), 649–
657. [PubMed: 10837246]
Calvert GA, Hansen PC, Iversen SD, & Brammer MJ (2001). Detection of audio-visual integration
sites in humans by application of electrophysiological criteria to the BOLD effect. NeuroImage,
Author Manuscript

14(2), 427–438. [PubMed: 11467916]


Cappe C, & Barone P (2005). Heteromodal connections supporting multisensory integration at low
levels of cortical processing in the monkey. European Journal of Neuroscience, 22(11), 2886–
2902. [PubMed: 16324124]
Cappe C, Murray MM, Barone P, & Rouiller EM (2010). Multisensory facilitation of behavior in
monkeys: Effects of stimulus intensity. Journal of Cognitive Neuroscience, 22 (12), 2850–2863.
10.1162/jocn.2010.21423 [PubMed: 20044892]
Carandini M, & Churchland AK (2013). Probing perceptual decisions in rodents. Nature Neuroscience,
16(7), 824–831. 10.1038/Nn.3410 [PubMed: 23799475]
Cascio CJ, Woynaroski T, Baranek GT, & Wallace MT (2016). Toward an interdisciplinary approach to
understanding sensory function in autism spectrum disorder. Autism Research, 9(9), 920–925.
[PubMed: 27090878]
Cases O, Vitalis T, Seif I, De Maeyer E, Sotelo C, & Gaspar P (1996). Lack of barrels in the
somatosensory cortex of monoamine oxidase A-deficient mice: Role of a serotonin excess during
the critical period. Neuron, 16(2), 297–307. [PubMed: 8789945]
Author Manuscript

Case-Smith J, Weaver LL, & Fristad MA (2015). A systematic review of sensory processing
interventions for children with autism spectrum disorders. Autism, 19(2), 133–148. [PubMed:
24477447]
CDC. (2019). Autism Spectrum Disorder (ASD): Data & Statistics—Prevalence.
Chabrol FP, Arenz A, Wiechert MT, Margrie TW, & DiGregorio DA (2015). Synaptic diversity enables
temporal coding of coincident multisensory inputs in single neurons. Nature Neuroscience, 18(5),
718–727. [PubMed: 25821914]
Charbonneau V, Laramee ME, Boucher V, Bronchti G, & Boire D (2012). Cortical and subcortical
projections to primary visual cortex in anophthalmic, enucleated and sighted mice. European
Journal of Neuroscience, 36(7), 2949–2963. 10.1111/j.1460-9568.2012.08215.x [PubMed:
22780435]
Clavagnier S, Falchier A, & Kennedy H (2004). Long-distance feedback projections to area V1:
Implications for multisensory integration, spatial awareness, and visual consciousness. Cognitive,
Affective, and Behavioral Neuroscience, 4(2), 117–126.
Author Manuscript

Cohen L, Rothschild G, & Mizrahi A (2011). Multisensory integration of natural odors and sounds in
the auditory cortex. Neuron, 72(2), 357–369. 10.1016/j.neuron.2011.08.019 [PubMed: 22017993]
Cohen YE (2009). Multimodal activity in the parietal cortex. Hearing Research, 258(1–2), 100–105.
[PubMed: 19450431]
Collignon O, Charbonneau G, Peters F, Nassim M, Lassonde M, Lepore F, … Bertone A (2013).
Reduced multisensory facilitation in persons with autism. Cortex, 49(6), 1704–1710. [PubMed:
22818902]

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 19

Colonius H, & Diederich A (2004). Multisensory interaction in saccadic reaction time: A time-
window-of-integration model. Journal of Cognitive Neuroscience, 16(6), 1000–1009.
Author Manuscript

10.1162/0898929041502733 [PubMed: 15298787]


Cook EH, & Leventhal BL (1996). The serotonin system in autism. Current Opinion in Pediatrics,
8(4), 348–354. [PubMed: 9053096]
Crosse MJ, Foxe JJ, & Molholm S (2019). Developmental recovery of impaired multisensory
processing in autism and the cost of switching sensory modality. bioRxiv, 565333.
Cuppini C, Ursino M, Magosso E, Ross LA, Foxe JJ, & Molholm S (2017). A computational analysis
of neural mechanisms underlying the maturation of multisensory speech integration in
neurotypical children and those on the autism spectrum. Frontiers in Human Neuroscience, 11,
518. [PubMed: 29163099]
Dalton KM, Nacewicz BM, Johnstone T, Schaefer HS, Gernsbacher MA, Goldsmith HH, … Davidson
RJ (2005). Gaze fixation and the neural circuitry of face processing in autism. Nature
Neuroscience, 8(4), 519–526. [PubMed: 15750588]
Dawson G, Webb SJ, & McPartland J (2005). Understanding the nature of face processing impairment
in autism: Insights from behavioral and electrophysiological studies. Developmental
Author Manuscript

Neuropsychology, 27(3), 403–424. [PubMed: 15843104]


De Niear MA, Gupta PB, Baum SH, & Wallace MT (2018). Perceptual training enhances temporal
acuity for multisensory speech. Neurobiology of Learning and Memory, 147, 9–17. [PubMed:
29107704]
De Niear MA, Koo B, & Wallace MT (2016). Multisensory perceptual learning is dependent upon task
difficulty. Experimental Brain Research, 234(11), 3269–3277. [PubMed: 27401473]
Deneux T, Harrell ER, Kempf A, Ceballo S, Filipchuk A, & Bathellier B (2019). Context-dependent
signaling of coincident auditory and visual events in primary visual cortex. eLife, 8, e44006.
[PubMed: 31115334]
Dillingham CH, Gay SM, Behrooz R, & Gabriele ML (2017). Modular-extramodular organization in
developing multisensory shell regions of the mouse inferior colliculus. Journal of Comparative
Neurology, 525(17), 3742–3756. [PubMed: 28786102]
Doan TP, Lagartos-Donate MJ, Nilssen ES, Ohara S, & Witter MP (2019). Convergent projections
from perirhinal and postrhinal cortices suggest a multisensory nature of lateral, but not medial,
entorhinal cortex. Cell Reports, 29(3), 617–627.e617. [PubMed: 31618631]
Author Manuscript

Dove D, Warren Z, McPheeters ML, Taylor JL, Sathe NA, & Veenstra-VanderWeele J (2012).
Medications for adolescents and young adults with autism spectrum disorders: A systematic
review. Pediatrics, 130(4), 717–726. [PubMed: 23008452]
Doykos TK, Gilmer JI, Person AL, & Felsen G (2020). Monosynaptic inputs to specific cell types of
the intermediate and deep layers of the superior colliculus. Journal of Comparative Neurology,
528, 2254–2268. [PubMed: 32080842]
Drager U, & Hubel DH (1975a). Responses to visual stimulation and relationship between visual,
auditory, and somatosensory inputs in mouse superior colliculus. Journal of Neurophysiology,
38(3), 690–713. [PubMed: 1127462]
Drager UC, & Hubel DH (1975b). Physiology of visual cells in mouse superior colliculus and
correlation with somatosensory and auditory input. Nature, 253, 203–204. [PubMed: 1110771]
Driver J, & Noesselt T (2008). Multisensory interplay reveals crossmodal influences on ‘sensory-
specific’ brain regions, neural responses, and judgments. Neuron, 57(1), 11–23. 10.1016/
j.neuron.2007.12.0132427054 [PubMed: 18184561]
Author Manuscript

Esaki T, Cook M, Shimoji K, Murphy DL, Sokoloff L, & Holmes A (2005). Developmental disruption
of serotonin transporter function impairs cerebral responses to whisker stimulation in mice.
Proceedings of the National Academy of Sciences of the United States of America, 102(15), 5582–
5587. [PubMed: 15809439]
Falchier A, Schroeder CE, Hackett TA, Lakatos P, Nascimento-Silva S, Ulbert I, … Smiley JF (2010).
Projection from visual areas V2 and prostriata to caudal auditory cortex in the monkey. Cerebral
Cortex, 20(7), 1529–1538. [PubMed: 19875677]

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 20

Feldman JI, Dunham K, Cassidy M, Wallace MT, Liu Y, & Woynaroski TG (2018). Audiovisual
multisensory integration in individuals with autism spectrum disorder: A systematic review and
Author Manuscript

meta-analysis. Neuroscience and Biobehavioral Reviews, 95, 220–234. [PubMed: 30287245]


Feldman JI, Dunham K, Conrad JG, Simon DM, Cassidy M, Liu Y, … Woynaroski TG (2020).
Plasticity of temporal binding in children with autism spectrum disorder: A single case
experimental design perceptual training study. Research in Autism Spectrum Disorders, 74,
101555. [PubMed: 32440308]
Feldman JI, Kuang W, Conrad JG, Tu A, Santapuram P, Simon DM, … Wallace MT (2019). Brief
report: Differences in multisensory integration covary with sensory responsiveness in children with
and without autism spectrum disorder. Journal of Autism and Developmental Disorders, 49(1),
397–403. [PubMed: 30043353]
Foss-Feig JH, Heacock JL, & Cascio CJ (2012). Tactile responsiveness patterns and their association
with core features in autism spectrum disorders. Research in Autism Spectrum Disorders, 6(1),
337–344. [PubMed: 22059092]
Foss-Feig JH, Kwakye LD, Cascio CJ, Burnette CP, Kadivar H, Stone WL, & Wallace MT (2010). An
extended multisensory temporal binding window in autism spectrum disorders. Experimental
Author Manuscript

Brain Research, 203(2), 381–389. 10.1007/s00221-010-2240-4 [PubMed: 20390256]


Foxe JJ, Molholm S, Del Bene VA, Frey H-P, Russo NN, Blanco D, … Ross LA (2015). Severe
multisensory speech integration deficits in high-functioning school-aged children with autism
spectrum disorder (ASD) and their resolution during early adolescence. Cerebral Cortex, 25(2),
298–312. [PubMed: 23985136]
Foxe JJ, & Schroeder CE (2005). The case for feedforward multisensory convergence during early
cortical processing. Neuroreport, 16(5), 419–423. [PubMed: 15770144]
Frassinetti F, Bolognini N, & Làdavas E (2002). Enhancement of visual perception by crossmodal
visuo-auditory interaction. Experimental Brain Research, 147(3), 332–343. [PubMed: 12428141]
Frens MA, & Van Opstal AJ (1998). Visual-auditory interactions modulate saccade-related activity in
monkey superior colliculus. Brain Research Bulletin, 46(3), 211–224. [PubMed: 9667814]
Fuster JM, Bodner M, & Kroger JK (2000). Cross-modal and cross-temporal association in neurons of
frontal cortex. Nature, 405(6784), 347–351. [PubMed: 10830963]
Gabriele S, Sacco R, & Persico AM (2014). Blood serotonin levels in autism spectrum disorder: A
systematic review and meta-analysis. European Neuropsychopharmacology, 24(6), 919–929.
Author Manuscript

[PubMed: 24613076]
Gaspar P, Cases O, & Maroteaux L (2003). The developmental role of serotonin: News from mouse
molecular genetics. Nature Reviews Neuroscience, 4(12), 1002–1012. [PubMed: 14618156]
Gervais H, Belin P, Boddaert N, Leboyer M, Coez A, Sfaello I, … Zilbovicius M (2004). Abnormal
cortical voice processing in autism. Nature Neuroscience, 7(8), 801–802. [PubMed: 15258587]
Ghanizadeh A, Sahraeizadeh A, & Berk M (2014). A head-to-head comparison of aripiprazole and
risperidone for safety and treating autistic disorders, a randomized double blind clinical trial. Child
Psychiatry and Human Development, 45 (2), 185–192. [PubMed: 23801256]
Gharaei S, Arabzadeh E, & Solomon SG (2018). Integration of visual and whisker signals in rat
superior colliculus. Scientific Reports, 8(1), 1–16. [PubMed: 29311619]
Ghazanfar AA, Chandrasekaran C, & Logothetis NK (2008). Interactions between the superior
temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys.
Journal of Neuroscience, 28(17), 4457–4469. [PubMed: 18434524]
Ghazanfar AA, Maier JX, Hoffman KL, & Logothetis NK (2005). Multisensory integration of dynamic
Author Manuscript

faces and voices in rhesus monkey auditory cortex. Journal of Neuroscience, 25(20), 5004–5012.
[PubMed: 15901781]
Ghazanfar AA, & Schroeder CE (2006). Is neocortex essentially multisensory? Trends in Cognitive
Sciences, 10(6), 278–285. [PubMed: 16713325]
Gleiss S, & Kayser C (2012). Audio-visual detection benefits in the rat. PLoS One, 7(9), e45677
10.1371/journal.pone.00456773445486 [PubMed: 23029179]
Gogolla N, Takesian AE, Feng G, Fagiolini M, & Hensch TK (2014). Sensory integration in mouse
insular cortex reflects GABA circuit maturation. Neuron, 83, 894–905. [PubMed: 25088363]

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 21

Gu Y, Angelaki DE, & DeAngelis GC (2008). Neural correlates of multisensory cue integration in
macaque MSTd. Nature Neuroscience, 11(10), 1201–1210. [PubMed: 18776893]
Author Manuscript

Hazen EP, Stornelli JL, O’Rourke JA, Koesterer K, & McDougle CJ (2014). Sensory symptoms in
autism spectrum disorders. Harvard Review of Psychiatry, 22(2), 112–124. [PubMed: 24614766]
Hillock AR, Powers AR, & Wallace MT (2011). Binding of sights and sounds: Age-related changes in
multisensory temporal processing. Neuropsychologia, 49(3), 461–467. 10.1016/
j.neuropsychologia.2010.11.0413140703 [PubMed: 21134385]
Hilton CL, Harper JD, Kueker RH, Lang AR, Abbacchi AM, Todorov A, & LaVesser PD (2010).
Sensory responsiveness as a predictor of social severity in children with high functioning autism
spectrum disorders. Journal of Autism and Developmental Disorders, 40(8), 937–945. 10.1007/
s10803-010-0944-8 [PubMed: 20108030]
Hirokawa J, Bosch M, Sakata S, Sakurai Y, & Yamamori T (2008). Functional role of the secondary
visual cortex in multisensory facilitation in rats. Neuroscience, 153(4), 1402–1417. 10.1016/
j.neuroscience.2008.01.011 [PubMed: 18440715]
Hirokawa J, Sadakane O, Sakata S, Bosch M, Sakurai Y, & Yamamori T (2011). Multisensory
information facilitates reaction speed by enlarging activity difference between superior colliculus
Author Manuscript

hemispheres in rats. PLoS One, 6(9), e25283 10.1371/journal.pone.00252833180293 [PubMed:


21966481]
Hishida R, Kudoh M, & Shibuki K (2014). Multimodal cortical sensory pathways revealed by
sequential transcranial electrical stimulation in mice. Neuroscience Research, 87, 49–55.
[PubMed: 25066521]
Hornix BE, Havekes R, & Kas MJ (2018). Multisensory cortical processing and dysfunction across the
neuropsychiatric spectrum. Neuroscience and Biobehavioral Reviews, 97, 138–151. [PubMed:
29496479]
Hurley LM, Thompson AM, & Pollak GD (2002). Serotonin in the inferior colliculus. Hearing
Research, 168(1), 1–11. [PubMed: 12117504]
Iarocci G, & McDonald J (2006). Sensory integration and the perceptual experience of persons with
autism. Journal of Autism and Developmental Disorders, 36(1), 77–90. 10.1007/
s10803-005-0044-3 [PubMed: 16395537]
Ihara S, Yoshikawa K, & Touhara K (2013). Chemosensory signals and their receptors in the olfactory
neural system. Neuroscience, 254, 45–60. [PubMed: 24045101]
Author Manuscript

Irwin JR, Tornatore LA, Brancazio L, & Whalen D (2011). Can children with autism spectrum
disorders “hear” a speaking face? Child Development, 82(5), 1397–1403. [PubMed: 21790542]
Ishikawa T, Shimuta M, & Hausser M (2015). Multimodal sensory integration in single cerebellar
granule cells in vivo. eLife, 4, e12916. [PubMed: 26714108]
Jacklin DL, Cloke JM, Potvin A, Garrett I, & Winters BD (2016). The dynamic multisensory engram:
Neural circuitry underlying crossmodal object recognition in rats changes with the nature of object
experience. Journal of Neuroscience, 36(4), 1273–1289. [PubMed: 26818515]
Jiang W, Jiang H, & Stein BE (2002). Two corticotectal areas facilitate multisensory orientation
behavior. Journal of Cognitive Neuroscience, 14(8), 1240–1255. 10.1162/089892902760807230
[PubMed: 12495529]
Jiang W, Jiang H, & Stein BE (2006). Neonatal cortical ablation disrupts multisensory development in
superior colliculus. Journal of Neurophysiology, 95(3), 1380–1396. 10.1152/jn.00880.2005
[PubMed: 16267111]
Jiang W, Wallace MT, Jiang H, Vaughan JW, & Stein BE (2001). Two cortical areas mediate
Author Manuscript

multisensory integration in superior colliculus neurons. Journal of Neurophysiology, 85(2), 506–


522. [PubMed: 11160489]
Jitsuki S, Takemoto K, Kawasaki T, Tada H, Takahashi A, Becamel C, … Ziff EB (2011). Serotonin
mediates crossmodal reorganization of cortical circuits. Neuron, 69(4), 780–792. [PubMed:
21338886]
Kajikawa Y, Falchier A, Musacchia G, Lakatos P, & Schroeder CE (2012). Audiovisual integration in
non-human primates In The neural bases of multisensory processes. Boca Raton, FL: CRC Press/
Taylor & Francis.

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 22

Kayser C, Petkov CI, & Logothetis NK (2008). Visual modulation of neurons in auditory cortex.
Cerebral Cortex, 18(7), 1560–1574. [PubMed: 18180245]
Author Manuscript

Kern JK, Trivedi MH, Grannemann BD, Garver CR, Johnson DG, Andrews AA, … Schroeder JL
(2007). Sensory correlations in autism. Autism: The International Journal of Research and
Practice, 11(2), 123–134. 10.1177/1362361307075702 [PubMed: 17353213]
Knöpfel T, Sweeney Y, Radulescu CI, Zabouri N, Doostdar N, Clopath C, & Barnes SJ (2019). Audio-
visual experience strengthens multisensory assemblies in adult mouse visual cortex. Nature
Communications, 10(1), 1–15.
Kuroki S, Yoshida T, Tsutsui H, Iwama M, Ando R, Michikawa T, … Itohara S (2018). Excitatory
neuronal hubs configure multisensory integration of slow waves in association cortex. Cell
Reports, 22(11), 2873–2885. [PubMed: 29539417]
Kwakye LD, Foss-Feig JH, Cascio CJ, Stone WL, & Wallace MT (2011). Altered auditory and
multisensory temporal processing in autism spectrum disorders. Frontiers in Integrative
Neuroscience, 4, 129 10.3389/fnint.2010.001293024004 [PubMed: 21258617]
Lakatos P, Chen C-M, O’Connell MN, Mills A, & Schroeder CE (2007). Neuronal oscillations and
multisensory interaction in primary auditory cortex. Neuron, 53(2), 279–292. [PubMed:
Author Manuscript

17224408]
Lane AE, Young RL, Baker AE, & Angley MT (2010). Sensory processing subtypes in autism:
Association with adaptive behavior. Journal of Autism and Developmental Disorders, 40(1), 112–
122. [PubMed: 19644746]
Lang R, O’Reilly M, Healy O, Rispoli M, Lydon H, Streusand W, … Lancioni G (2012). Sensory
integration therapy for autism spectrum disorders: A systematic review. Research in Autism
Spectrum Disorders, 6(3), 1004–1018.
Laramee ME, Kurotani T, Rockland KS, Bronchti G, & Boire D (2011). Indirect pathway between the
primary auditory and visual cortices through layer V pyramidal neurons in V2L in mouse and the
effects of bilateral enucleation. European Journal of Neuroscience, 34(1), 65–78. 10.1111/
j.1460-9568.2011.07732.x [PubMed: 21676038]
Lau C, Manno FA, Dong CM, Chan KC, & Wu EX (2018). Auditory-visual convergence at the
superior colliculus in rat using functional MRI. Paper presented at the 2018 40th Annual
International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).
Leekam SR, Nieto C, Libby SJ, Wing L, & Gould J (2007). Describing the sensory abnormalities of
Author Manuscript

children and adults with autism. Journal of Autism and Developmental Disorders, 37(5), 894–
910. [PubMed: 17016677]
Lesicko AM, Hristova TS, Maigler KC, & Llano DA (2016). Connectional modularity of top-down
and bottom-up multimodal inputs to the lateral cortex of the mouse inferior colliculus. Journal of
Neuroscience, 36(43), 11037–11050. [PubMed: 27798184]
Lippert MT, Takagaki K, Kayser C, & Ohl FW (2013). Asymmetric multisensory interactions of visual
and somatosensory responses in a region of the rat parietal cortex. PLoS One, 8(5), e63631.
[PubMed: 23667650]
Lipton PA, Alvarez P, & Eichenbaum H (1999). Crossmodal associative memory representations in
rodent orbitofrontal cortex. Neuron, 22(2), 349–359. [PubMed: 10069340]
Lyamzin D, & Benucci A (2019). The mouse posterior parietal cortex: Anatomy and functions.
Neuroscience Research, 140, 14–22. [PubMed: 30465783]
Maier JX, Neuhoff JG, Logothetis NK, & Ghazanfar AA (2004). Multisensory integration of looming
signals by rhesus monkeys. Neuron, 43(2), 177–181. [PubMed: 15260954]
Author Manuscript

Marco EJ, Hinkley LB, Hill SS, & Nagarajan SS (2011). Sensory processing in autism: A review of
neurophysiologic findings. Pediatric Research, 69(5 Pt 2), 48R–54R. 10.1203/
PDR.0b013e3182130c543086654
Martínez K, Martínez-García M, Marcos-Vidal L, Janssen J, Castellanos FX, Pretus C, … Parellada M
(2019). Sensory-to-cognitive systems integration is associated with clinical severity in autism
spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 59(3),
422–433. [PubMed: 31260788]

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 23

Martuzzi R, Murray MM, Michel CM, Thiran J-P, Maeder PP, Clarke S, & Meuli RA (2007).
Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cerebral
Author Manuscript

Cortex, 17(7), 1672–1679. [PubMed: 16968869]


Maruyama AT, & Komai S (2018). Auditory-induced response in the primary sensory cortex of
rodents. PLoS One, 13(12), e0209266. [PubMed: 30571722]
Masse IO, Ross S, Bronchti G, & Boire D (2017). Asymmetric direct reciprocal connections between
primary visual and somatosensory cortices of the mouse. Cerebral Cortex, 27(9), 4361–4378.
[PubMed: 27522075]
May PJ (2006). The mammalian superior colliculus: Laminar structure and connections. Progress in
Brain Research, 151, 321–378. [PubMed: 16221594]
McClure JP Jr., & Polack P-O (2019). Pure tones modulate the representation of orientation and
direction in the primary visual cortex. Journal of Neurophysiology, 121(6), 2202–2214.
[PubMed: 30969800]
Mcgurk H, & Macdonald J (1976). Hearing lips and seeing voices. Nature, 264(5588), 746–748.
[PubMed: 1012311]
Mégevand P, Molholm S, Nayak A, & Foxe JJ (2013). Recalibration of the multisensory temporal
Author Manuscript

window of integration results from changing task demands. PLoS One, 8(8), e71608. [PubMed:
23951203]
Meijer GT, Marchesi P, Mejias JF, Montijn JS, Lansink CS, & Pennartz CM (2020). Neural correlates
of multisensory detection behavior: Comparison of primary and higher-order visual cortex. Cell
Reports, 31(6), 107636. [PubMed: 32402272]
Meijer GT, Montijn JS, Pennartz CM, & Lansink CS (2017). Audiovisual modulation in mouse
primary visual cortex depends on cross-modal stimulus configuration and congruency. Journal of
Neuroscience, 37(36), 8783–8796. [PubMed: 28821672]
Meijer GT, Pie JL, Dolman TL, Pennartz C, & Lansink CS (2018). Audiovisual integration enhances
stimulus detection performance in mice. Frontiers in Behavioral Neuroscience, 12, 231.
[PubMed: 30337861]
Menzel RR, & Barth DS (2005). Multisensory and secondary somatosensory cortex in the rat. Cerebral
Cortex, 15(11), 1690–1696. 10.1093/cercor/bhi045 [PubMed: 15703251]
Meredith MA (2002). On the neuronal basis for multisensory convergence: A brief overview.
Author Manuscript

Cognitive Brain Research, 14 (1), 31–40. 10.1016/S0926-6410(02)00059-9 [PubMed: 12063128]


Meredith MA, Nemitz JW, & Stein BE (1987). Determinants of multisensory integration in superior
colliculus neurons. 1. Temporal factors. Journal of Neuroscience, 7(10), 3215–3229. [PubMed:
3668625]
Meredith MA, & Stein BE (1983). Interactions among converging sensory inputs in the superior
colliculus. Science, 221 (4608), 389–391. [PubMed: 6867718]
Meredith MA, & Stein BE (1986a). Spatial factors determine the activity of multisensory neurons in
cat superior colliculus. Brain Research, 365(2), 350–354. [PubMed: 3947999]
Meredith MA, & Stein BE (1986b). Visual, auditory, and somatosensory convergence on cells in
superior colliculus results in multisensory integration. Journal of Neurophysiology, 56(3), 640–
662. [PubMed: 3537225]
Meyza K, & Blanchard D (2017). The BTBR mouse model of idiopathic autism—Current view on
mechanisms. Neuroscience and Biobehavioral Reviews, 76, 99–110. [PubMed: 28167097]
Miller J (1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive
Psychology, 14(2), 247–279. [PubMed: 7083803]
Author Manuscript

Miller J, Ulrich R, & Lamarre Y (2001). Locus of the redundant-signals effect in bimodal divided
attention: A neurophysiological analysis. Perception and Psychophysics, 63(3), 555–562.
[PubMed: 11414141]
Miller LM, & D’esposito M (2005). Perceptual fusion and stimulus coincidence in the cross-modal
integration of speech. Journal of Neuroscience, 25(25), 5884–5893. [PubMed: 15976077]
Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, & Foxe JJ (2002). Multisensory
auditory-visual interactions during early sensory processing in humans: a high-density electrical
mapping study. Cognitive Brain Research, 14(1), 115–128. 10.1016/s0926-6410(02)00066-6
[PubMed: 12063135]

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 24

Mongillo EA, Irwin JR, Whalen DH, Klaiman C, Carter AS, & Schultz RT (2008). Audiovisual
processing in children with and without autism spectrum disorders. Journal of Autism and
Author Manuscript

Developmental Disorders, 38(7), 1349–1358. 10.1007/s10803-007-0521-y [PubMed: 18307027]


Morrill RJ, & Hasenstaub AR (2018). Visual information present in infragranular layers of mouse
auditory cortex. Journal of Neuroscience, 38(11), 2854–2862. [PubMed: 29440554]
Mulder EJ, Anderson GM, Kema IP, de Bildt A, van Lang ND, den Boer JA, & Minderaa RB (2004).
Platelet serotonin levels in pervasive developmental disorders and mental retardation: Diagnostic
group differences, within-group distribution, and behavioral correlates. Journal of the American
Academy of Child and Adolescent Psychiatry, 43 (4), 491–499.
10.1097/00004583-200404000-00016 [PubMed: 15187810]
Muller CL, Anacker AM, & Veenstra-VanderWeele J (2016). The serotonin system in autism spectrum
disorder: From biomarker to animal models. Neuroscience, 321, 24–41. [PubMed: 26577932]
Mullette-Gillman ODA, Cohen YE, & Groh JM (2005). Eye-centered, head-centered, and complex
coding of visual and auditory targets in the intraparietal sulcus. Journal of Neurophysiology,
94(4), 2331–2352. [PubMed: 15843485]
Murray MM, & Wallace MT (2011). The neural bases of multisensory processes. Boca Raton, FL:
Author Manuscript

CRC Press.
Najafi F, Elsayed GF, Cao R, Pnevmatikakis E, Latham PE, Cunningham JP, & Churchland AK
(2019). Excitatory and inhibitory subnetworks are equally selective during decision-making and
emerge simultaneously during learning. Neuron, 105(1), 165–179. [PubMed: 31753580]
Nath AR, & Beauchamp MS (2012). A neural basis for interindividual differences in the McGurk
effect, a multisensory speech illusion. Neuroimage, 59(1), 781–787. [PubMed: 21787869]
Nath AR, Fava EE, & Beauchamp MS (2011). Neural correlates of interindividual differences in
children’s audiovisual speech perception. The Journal of Neuroscience, 31(39), 13963–13971.
10.1523/JNEUROSCI.2605-11.20113203203 [PubMed: 21957257]
Nestler EJ, & Hyman SE (2010). Animal models of neuropsychiatric disorders. Nature Neuroscience,
13(10), 1161–1169. [PubMed: 20877280]
Olcese U, Iurilli G, & Medini P (2013). Cellular and synaptic architecture of multisensory integration
in the mouse neocortex. Neuron, 79(3), 579–593. 10.1016/j.neuron.2013.06.010 [PubMed:
23850594]
Author Manuscript

Olive I, Tempelmann C, Berthoz A, & Heinze HJ (2015). Increased functional connectivity between
superior colliculus and brain regions implicated in bodily self-consciousness during the rubber
hand illusion. Human Brain Mapping, 36(2), 717–730. [PubMed: 25346407]
Ooi YP, Weng S-J, Kossowsky J, Gerger H, & Sung M (2017). Oxytocin and autism spectrum
disorders: A systematic review and meta-analysis of randomized controlled trials.
Pharmacopsychiatry, 50(01), 5–13. [PubMed: 27574858]
Parham LD, Roley SS, May-Benson TA, Koomar J, Brett-Green B, Burke JP, … Schaaf RC (2011).
Development of a fidelity measure for research on the effectiveness of the Ayres Sensory
Integration® intervention. American Journal of Occupational Therapy, 65(2), 133–142.
[PubMed: 21476360]
Patten E, Labban JD, Casenhiser DM, & Cotton CL (2016). Synchrony detection of linguistic stimuli
in the presence of faces: Neuropsychological implications for language development in ASD.
Developmental Neuropsychology, 41 (5–8), 362–374. [PubMed: 28059555]
Patten E, Watson LR, & Baranek GT (2014). Temporal synchrony detection and associations with
language in young children with ASD. Autism Research and Treatment, 2014, 1–8.
Author Manuscript

Pelphrey KA, Morris JP, & Mccarthy G (2004). Grasping the intentions of others: The perceived
intentionality of an action influences activity in the superior temporal sulcus during social
perception. Journal of Cognitive Neuroscience, 16 (10), 1706–1716. [PubMed: 15701223]
Perrault TJ, Vaughan JW, Stein BE, & Wallace MT (2003). Neuron-specific response characteristics
predict the magnitude of multisensory integration. Journal of Neurophysiology, 90(6), 4022–
4026. [PubMed: 12930816]
Perrault TJ, Vaughan JW, Stein BE, & Wallace MT (2005). Superior colliculus neurons use distinct
operational modes in the integration of multisensory stimuli. Journal of Neurophysiology, 93(5),
2575–2586. [PubMed: 15634709]

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 25

Petersen CC (2014). Cortical control of whisker movement. Annual Review of Neuroscience, 37, 183–
203.
Author Manuscript

Powers AR 3rd, Hevey MA, & Wallace MT (2012). Neural correlates of multisensory perceptual
learning. The Journal of Neuroscience, 32(18), 6263–6274. 10.1523/
JNEUROSCI.6138-11.20123366559 [PubMed: 22553032]
Powers AR, Hillock AR, & Wallace MT (2009). Perceptual training narrows the temporal window of
multisensory binding. Journal of Neuroscience, 29(39), 12265–12274. 10.1523/
Jneurosci.3501-09.2009 [PubMed: 19793985]
Raab DH (1962). Statistical facilitation of simple reaction times. Transactions of the New York
Academy of Sciences, 24, 574–590. [PubMed: 14489538]
Radvansky BA, & Dombeck DA (2018). An olfactory virtual reality system for mice. Nature
Communications, 9(1), 1–14.
Raposo D, Kaufman MT, & Churchland AK (2014). A category-free neural population supports
evolving demands during decision-making. Nature Neuroscience, 17, 1784–1792. [PubMed:
25383902]
Raposo D, Sheppard JP, Schrater PR, & Churchland AK (2012). Multisensory decision-making in rats
Author Manuscript

and humans. The Journal of Neuroscience, 32(11), 3726–3735. 10.1523/


JNEUROSCI.4998-11.20123335889 [PubMed: 22423093]
Redcay E (2008). The superior temporal sulcus performs a common function for social and speech
perception: Implications for the emergence of autism. Neuroscience and Biobehavioral Reviews,
32(1), 123–142. [PubMed: 17706781]
Reid JM, Jacklin DL, & Winters BD (2014). Delineating prefrontal cortex region contributions to
crossmodal object recognition in rats. Cerebral Cortex, 24(8), 2108–2119. [PubMed: 23505287]
Reig R, & Silberberg G (2014). Multisensory integration in the mouse striatum. Neuron, 83, 1200–
1212. [PubMed: 25155959]
Robertson CE, & Baron-Cohen S (2017). Sensory perception in autism. Nature Reviews Neuroscience,
18, 671–684. [PubMed: 28951611]
Robson MJ, Quinlan MA, Margolis KG, Gajewski-Kurdziel PA, Veenstra-VanderWeele J, Gershon
MD, … Blakely RD (2018). p38a MAPK signaling drives pharmacologically reversible brain and
gastrointestinal phenotypes in the SERT Ala56 mouse. Proceedings of the National Academy of
Author Manuscript

Sciences, 115(43), E10245–E10254.


Rockland KS, & Ojima H (2003). Multisensory convergence in calcarine visual areas in macaque
monkey. International Journal of Psychophysiology, 50(1–2), 19–26. [PubMed: 14511833]
Rodgers KM, Benison AM, Klein A, & Barth DS (2008). Auditory, somatosensory, and multisensory
insular cortex in the rat. Cerebral Cortex, 18(12), 2941–2951. 10.1093/cercor/bhn054 [PubMed:
18424777]
Romanski LM, Averbeck BB, & Diltz M (2005). Neural representation of vocalizations in the primate
ventrolateral prefrontal cortex. Journal of Neurophysiology, 93(2), 734–747. [PubMed:
15371495]
Romanski LM, & Goldman-Rakic PS (2002). An auditory domain in primate prefrontal cortex. Nature
Neuroscience, 5(1), 15–16. [PubMed: 11753413]
Ross LA, Del Bene VA, Molholm S, Frey H-P, & Foxe JJ (2015). Sex differences in multisensory
speech processing in both typically developing children and those on the autism spectrum.
Frontiers in Neuroscience, 9, 185. [PubMed: 26074757]
Russo N, Foxe JJ, Brandwein AB, Altschuler T, Gomes H, & Molholm S (2010). Multisensory
Author Manuscript

processing in children with autism: High-density electrical mapping of auditory-somatosensory


integration. Autism Research: Official Journal of the International Society for Autism Research,
3(5), 253–267. 10.1002/aur.152 [PubMed: 20730775]
Sakata S, Yamamori T, & Sakurai Y (2004). Behavioral studies of auditory-visual spatial recognition
and integration in rats. Experimental Brain Research, 159(4), 409–417. 10.1007/
s00221-004-1962-6 [PubMed: 15249987]
Salichon N, Gaspar P, Upton AL, Picaud S, Hanoun NM, Hamon M, … Lesch KP (2001). Excessive
activation of serotonin (5-HT) 1B receptors disrupts the formation of sensory maps in

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 26

monoamine oxidase a and 5-ht transporter knock-out mice. The Journal of Neuroscience, 21(3),
884–896. [PubMed: 11157075]
Author Manuscript

Sartorato F, Przybylowski L, & Sarko DK (2017). Improving therapeutic outcomes in autism spectrum
disorders: Enhancing social communication and sensory processing through the use of interactive
robots. Journal of Psychiatric Research, 90, 1–11. [PubMed: 28213292]
Schaaf RC, Dumont RL, Arbesman M, & May-Benson TA (2018). Efficacy of occupational therapy
using Ayres Sensory Integration®: A systematic review. American Journal of Occupational
Therapy, 72(1), 7201190010p7201190011-7201190010p7201190010.
Schauder KB, Muller CL, Veenstra-VanderWeele J, & Cascio CJ (2015). Genetic variation in serotonin
transporter modulates tactile hyperresponsiveness in ASD. Research in Autism Spectrum
Disorders, 10, 93–100. [PubMed: 25558276]
Schlack A, Sterbing-D’Angelo SJ, Hartung K, Hoffmann KP, & Bremmer F (2005). Multisensory
space representations in the macaque ventral intraparietal area. Journal of Neuroscience, 25(18),
4616–4625. [PubMed: 15872109]
Schormans AL, & Allman BL (2018). Behavioral plasticity of audiovisual perception: Rapid
recalibration of temporal sensitivity but not perceptual binding following adult-onset hearing
Author Manuscript

loss. Frontiers in Behavioral Neuroscience, 12, 256. [PubMed: 30429780]


Schormans AL, & Allman BL (2019). Compensatory plasticity in the lateral extrastriate visual cortex
preserves audiovisual temporal processing following adult-onset hearing loss. Neural Plasticity,
2019, 1–20.
Schormans AL, Scott KE, Vo AM, Tyker A, Typlt M, Stolzberg D, & Allman BL (2017). Audiovisual
temporal processing and synchrony perception in the rat. Frontiers in Behavioral Neuroscience,
10, 246. [PubMed: 28119580]
Schormans AL, Typlt M, & Allman BL (2017). Crossmodal plasticity in auditory, visual and
multisensory cortical areas following noise-induced hearing loss in adulthood. Hearing Research,
343, 92–107. [PubMed: 27387138]
Shams L, Kamitani Y, & Shimojo S (2000). What you see is what you hear. Nature, 408, 788.
[PubMed: 11130706]
Sharma S, & Bandyopadhyay S (2019). Differential rapid plasticity in auditory and visual responses in
the primarily multisensory orbitofrontal cortex. Eneuro, 7(3), 869446 10.1523/
ENEURO.0061-20.2020
Author Manuscript

Shaw LH, Freedman EG, Crosse MJ, Nicholas E, Chen AM, Braiman MS, … Foxe JJ (2020).
Operating in a multisensory context: Assessing the interplay between multisensory reaction time
facilitation and inter-sensory task-switching effects. Neuroscience, 436, 122–135. [PubMed:
32325100]
Sheppard JP, Raposo D, & Churchland AK (2013). Dynamic weighting of multisensory stimuli shapes
decision-making in rats and humans. Journal of Vision, 13(6), 4 10.1167/13.6.43649064
Sieben K, Röder B, & Hanganu-Opatz IL (2013). Oscillatory entrainment of primary somatosensory
cortex encodes visual control of tactile processing. The Journal of Neuroscience, 33 (13), 5736–
5749. 10.1523/JNEUROSCI.4432-12.2013 [PubMed: 23536087]
Siemann J, Muller C, Forsberg C, Blakely R, Veenstra-VanderWeele J, & Wallace M (2017). An
autism-associated serotonin transporter variant disrupts multisensory processing. Translational
Psychiatry, 7(3), e1067. [PubMed: 28323282]
Siemann JK, Muller CL, Bamberger G, Allison JD, Veenstra-VanderWeele J, & Wallace MT (2015). A
novel behavioral paradigm to assess multisensory processing in mice. Frontiers in Behavioral
Author Manuscript

Neuroscience, 8, 456. [PubMed: 25628549]


Silverman J, Pride M, Hayes J, Puhger K, Butler-Struben H, Baker S, & Crawley J (2015). GABAB
receptor agonist R-baclofen reverses social deficits and reduces repetitive behavior in two mouse
models of autism. Neuropsychopharmacology, 40, 2228–2239. [PubMed: 25754761]
Silverman JL, & Crawley JN (2014). The promising trajectory of autism therapeutics discovery. Drug
Discovery Today, 19(7), 838–844. [PubMed: 24362109]
Silverman JL, Smith DG, Rizzo SJS, Karras MN, Turner SM, Tolu SS, … Ring RH (2012). Negative
allosteric modulation of the mGluR5 receptor reduces repetitive behaviors and rescues social
deficits in mouse models of autism. Science Translational Medicine, 4(131), 131ra51.

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 27

Smiley JF, & Falchier A (2009). Multisensory connections of monkey auditory cerebral cortex.
Hearing Research, 258(1–2), 37–46. [PubMed: 19619628]
Author Manuscript

Smith EG, & Bennetto L (2007). Audiovisual speech integration and lipreading in autism. Journal of
Child Psychology and Psychiatry, and Allied Disciplines, 48(8), 813–821. 10.1111/
j.1469-7610.2007.01766.x
Song Y-H, Kim J-H, Jeong H-W, Choi I, Jeong D, Kim K, & Lee S-H (2017). A neural circuit for
auditory dominance over visual perception. Neuron, 93(4), 940–954.e946. [PubMed: 28162806]
Sparks DL, & Hartwich-Young R (1989). The deep layers of the superior colliculus. Reviews of
Oculomotor Research, 3, 213–255. [PubMed: 2486324]
Stanford TR, Quessy S, & Stein BE (2005). Evaluating the operations underlying multisensory
integration in the cat superior colliculus. The Journal of Neuroscience, 25(28), 6499–6508.
[PubMed: 16014711]
Stehberg J, Dang PT, & Frostig RD (2014). Unimodal primary sensory cortices are directly connected
by long-range horizontal projections in the rat sensory cortex. Frontiers in Neuroanatomy, 8, 93.
[PubMed: 25309339]
Stein BE (2012). The new handbook of multisensory processing. Cambridge, MA: MIT Press.
Author Manuscript

Stein BE, & Meredith MA (1993). The merging of the senses. Cambridge, MA: The MIT Press.
Stein BE, Meredith MA, Huneycutt WS, & McDade L (1989). Behavioral indices of multisensory
integration: orientation to visual cues is affected by auditory stimuli. Journal of Cognitive
Neuroscience, 1(1), 12–24. 10.1162/jocn.1989.1.1.12 [PubMed: 23968407]
Stein BE, & Stanford TR (2008). Multisensory integration: Current issues from the perspective of the
single neuron. Nature Reviews. Neuroscience, 9(4), 255–266. 10.1038/nrn2331 [PubMed:
18354398]
Stein BE, Stanford TR, & Rowland BA (2014). Development of multisensory integration from the
perspective of the individual neuron. Nature Reviews Neuroscience, 15(8), 520–535. [PubMed:
25158358]
Stein BE, Wallace MW, Stanford TR, & Jiang W (2002). Cortex governs multisensory integration in
the midbrain. The Neuroscientist, 8(4), 306–314. [PubMed: 12194499]
Stevenson R, Siemann J, Woynaroski T, Schneider B, Eberly H, Camarata S, & Wallace M (2013).
Brief report: Arrested development of audiovisual speech perception in autism spectrum
Author Manuscript

disorders. Journal of Autism and Developmental Disorders, 44, 1–8. 10.1007/s10803-013-1992-7


Stevenson RA, Baum SH, Segers M, Ferber S, Barense MD, & Wallace MT (2017). Multisensory
speech perception in autism spectrum disorder: From phoneme to whole-word perception.
Autism Research, 10(7), 1280–1290. [PubMed: 28339177]
Stevenson RA, Ghose D, Fister JK, Sarko DK, Altieri NA, Nidiffer AR, … Wallace MT (2014). Identi-
fying and quantifying multisensory integration: A tutorial review. Brain Topography, 27(6), 707–
730. 10.1007/s10548-014-0365-7 [PubMed: 24722880]
Stevenson RA, & James TW (2009). Audiovisual integration in human superior temporal sulcus:
Inverse effectiveness and the neural processing of speech and object recognition. NeuroImage,
44(3), 1210–1223. [PubMed: 18973818]
Stevenson RA, Segers M, Ferber S, Barense MD, Camarata S, & Wallace MT (2016). Keeping time in
the brain: Autism spectrum disorder and audiovisual temporal processing. Autism Research, 9(7),
720–738. [PubMed: 26402725]
Stevenson RA, Segers M, Ferber S, Barense MD, & Wallace MT (2014). The impact of multisensory
integration deficits on speech perception in children with autism spectrum disorders. Frontiers in
Author Manuscript

Psychology, 5, 379 10.3389/Fpsyg.2014.00379 [PubMed: 24904448]


Stevenson RA, Segers M, Ncube BL, Black KR, Bebko JM, Ferber S, & Barense MD (2018). The
cascadeing influence of multisensory processing on speech perception in autism. Autism, 22(5),
609–624. [PubMed: 28506185]
Stevenson RA, Siemann JK, Schneider BC, Eberly HE, Woynaroski TG, Camarata SM, & Wallace MT
(2014). Multisensory temporal integration in autism spectrum disorders. Journal of Neuroscience,
34(3), 691–697. 10.1523/Jneurosci.3615-13.2014 [PubMed: 24431427]
Stevenson RA, Siemann JK, Woynaroski TG, Schneider BC, Eberly HE, Camarata SM, & Wallace MT
(2014). Evidence for diminished multisensory integration in autism spectrum disorders. Journal

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 28

of Autism and Developmental Disorders, 44, 3161–3167. 10.1007/s10803-014-2179-6 [PubMed:


25022248]
Author Manuscript

Stevenson RA, VanDerKlok RM, Pisoni DB, & James TW (2011). Discrete neural substrates underlie
complementary audiovisual speech integration processes. NeuroImage, 55(3), 1339–1345.
10.1016/j.neuroimage.2010.12.0633057325 [PubMed: 21195198]
Stevenson RA, & Wallace MT (2013). Multisensory temporal integration: Task and stimulus
dependencies. Experimental Brain Research, 227(2), 249–261. [PubMed: 23604624]
Stevenson RA, Wilson MM, Powers AR, & Wallace MT (2013). The effects of visual training on
multisensory temporal processing. Experimental Brain Research, 225(4), 479–489. [PubMed:
23307155]
Stevenson RA, Zemtsov RK, & Wallace MT (2012). Individual differences in the multisensory
temporal binding window predict susceptibility to audiovisual illusions. Journal of Experimental
Psychology. Human Perception and Performance., 38, 1517–1529. 10.1037/a0027339 [PubMed:
22390292]
Sugihara T, Diltz MD, Averbeck BB, & Romanski LM (2006). Integration of auditory and visual
communication information in the primate ventrolateral prefrontal cortex. Journal of
Author Manuscript

Neuroscience, 26(43), 11138–11147. [PubMed: 17065454]


Sumby WH, & Pollack I (1954). Visual contribution to speech intelligibility in noise. The Journal of
the Acoustical Society of America, 26(2), 212–215.
Sutcliffe JS, Delahanty RJ, Prasad HC, McCauley JL, Han Q, Jiang L, … Blakely RD (2005). Allelic
heterogeneity at the serotonin transporter locus (SLC6A4) confers susceptibility to autism and
rigid-compulsive behaviors. The American Journal of Human Genetics, 77(2), 265–279.
[PubMed: 15995945]
Tang Z-Q, & Trussell LO (2017). Serotonergic modulation of sensory representation in a central
multisensory circuit is pathway specific. Cell Reports, 20(8), 1844–1854. [PubMed: 28834748]
Taylor N, Isaac C, & Milne E (2010). A comparison of the development of audiovisual integration in
children with autism spectrum disorders and typically developing children. Journal of Autism and
Developmental Disorders, 40(11), 1403–1411. 10.1007/s10803-010-1000-4 [PubMed:
20354776]
Tees RC (1999). The effects of posterior parietal and posterior temporal cortical lesions on multimodal
spatial and nonspatial competencies in rats. Behavioural Brain Research, 106(1–2), 55–73.
Author Manuscript

10.1016/S0166-4328(99)00092-3 [PubMed: 10595422]


Thye MD, Bednarz HM, Herringshaw AJ, Sartin EB, & Kana RK (2018). The impact of atypical
sensory processing on social impairments in autism spectrum disorder. Developmental Cognitive
Neuroscience, 29, 151–167 [PubMed: 28545994]
van Wassenhove V, Grant KW, & Poeppel D (2007). Temporal window of integration in auditory-
visual speech perception. Neuropsychologia, 45(3), 598–607. 10.1016/
j.neuropsychologia.2006.01.001 [PubMed: 16530232]
Veenstra-VanderWeele J, Cook EH, King BH, Zarevics P, Cherubini M, Walton-Bowen K, …
Carpenter RL (2017). Arbaclofen in children and adolescents with autism spectrum disorder: A
randomized, controlled, phase 2 trial. Neuropsychopharmacology, 42(7), 1390–1398. [PubMed:
27748740]
Wada M, Ide M, Atsumi T, Sano Y, Shinoda Y, Furuichi T, & Kansaku K (2019). Rubber tail illusion is
weakened in Ca2+-dependent activator protein for secretion 2 (Caps 2)-knockout mice. Scientific
Reports, 9(1), 1–9 [PubMed: 30626917]
Author Manuscript

Wallace MT (2004). The development of multisensory processes. Cognitive Processing, 5(2), 69–83.
Wallace MT, Carriere BN, Perrault TJ, Vaughan JW, & Stein BE (2006). The development of cortical
multisensory integration. The Journal of Neuroscience, 26(46), 11844–11849 [PubMed:
17108157]
Wallace MT, Meredith MA, & Stein BE (1992). Integration of multiple sensory modalities in cat
cortex. Experimental Brain Research, 91(3), 484–488. [PubMed: 1483520]
Wallace MT, Meredith MA, & Stein BE (1993). Converging influences from visual, auditory, and
somatosensory cortices onto output neurons of the superior colliculus. Journal of
Neurophysiology, 69(6), 1797–1809. [PubMed: 8350124]

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 29

Wallace MT, Meredith MA, & Stein BE (1998). Multisensory integration in the superior colliculus of
the alert cat. Journal of Neurophysiology, 80(2), 1006–1010. [PubMed: 9705489]
Author Manuscript

Wallace MT, Ramachandran R, & Stein BE (2004). A revised view of sensory cortical parcellation.
Proceedings of the National Academy of Sciences of the United States of America, 101(7),
2167–2172. 10.1073/pnas.0305697101357070 [PubMed: 14766982]
Wallace MT, & Stein BE (1994). Crossmodal synthesis in the midbrain depends on input from cortex.
Journal of Neurophysiology, 71(1), 429–432. [PubMed: 8158240]
Wallace MT, & Stein BE (1996). Sensory organization of the superior colliculus in cat and monkey.
Progress in Brain Research, 112, 301–311. [PubMed: 8979837]
Wallace MT, & Stein BE (1997). Development of multisensory neurons and multisensory integration
in cat superior colliculus. The Journal of Neuroscience, 17(7), 2429–2444. [PubMed: 9065504]
Wallace MT, & Stein BE (2000). Onset of cross-modal synthesis in the neonatal superior colliculus is
gated by the development of cortical influences. Journal of Neurophysiology, 83(6), 3578–3582.
[PubMed: 10848574]
Wallace MT, & Stein BE (2001). Sensory and multisensory responses in the newborn monkey superior
colliculus. The Journal of Neuroscience, 21(22), 8886–8894. [PubMed: 11698600]
Author Manuscript

Wallace MT, & Stein BE (2007). Early experience determines how the senses will interact. Journal of
Neurophysiology, 97(1), 921–926. 10.1152/jn.00497.2006 [PubMed: 16914616]
Wallace MT, & Stevenson RA (2014). The construct of the multisensory temporal binding window and
its dysregulation in developmental disabilities. Neuropsychologia, 64, 105–123. [PubMed:
25128432]
Wallace MT, Wilkinson LK, & Stein BE (1996). Representation and integration of multiple sensory
inputs in primate superior colliculus. Journal of Neurophysiology, 76(2), 1246–1266. [PubMed:
8871234]
Wallace MT, Woynaroski TG, & Stevenson RA (2020). Multisensory integration as a window into
orderly and disrupted cognition and communication. Annual Review of Psychology, 71, 193–
219.
Wang Y, Celebrini S, Trotter Y, & Barone P (2008). Visuo-auditory interactions in the primary visual
cortex of the behaving monkey: Electrophysiological evidence. BMC Neuroscience, 9(1), 79.
[PubMed: 18699988]
Author Manuscript

Warren Z, Veenstra-VanderWeele J, Stone W, Bruzek JL, Nahmias AS, Foss-Feig JH, … Glasser AM
(2011). Therapies for children with autism spectrum disorders. Rockville (MD): Agency for
Healthcare Research and Quality (US).
Waterhouse BD, Azizi SA, Burne RA, & Woodward DJ (1990). Modulation of rat cortical area 17
neuronal responses to moving visual stimuli during norepinephrine and serotonin
microiontophoresis. Brain Research, 514(2), 276–292. [PubMed: 2357542]
Watkins S, Shams L, Tanaka S, Haynes J-D, & Rees G (2006). Sound alters activity in human V1 in
association with illusory visual perception. NeuroImage, 31(3), 1247–1256. [PubMed:
16556505]
Watling R, & Hauer S (2015). Effectiveness of Ayres Sensory Integration® and sensory-based
interventions for people with autism spectrum disorder: A systematic review. American Journal
of Occupational Therapy, 69(5), 6905180030p6905180031-6905180030p6905180012.
Watson LR, Patten E, Baranek GT, Poe M, Boyd BA, Freuler A, & Lorenzi J (2011). Differential
associations between sensory response patterns and language, social, and communication
measures in children with autism or other developmental disabilities. Journal of Speech,
Author Manuscript

Language, and Hearing Research, 54, 1562–1576.


Wilkinson LK, Meredith MA, & Stein BE (1996). The role of anterior ectosylvian cortex in cross-
modality orientation and approach behavior. Experimental Brain Research, 112(1), 1–10.
[PubMed: 8951401]
Woynaroski TG, Kwakye LD, Foss-Feig JH, Stevenson RA, Stone WL, & Wallace MT (2013).
Multisensory speech perception in children with autism spectrum disorders. Journal of Autism
and Developmental Disorders, 43(12), 2891–2902. [PubMed: 23624833]
Xu J, Sun X, Zhou X, Zhang J, & Yu L (2014). The cortical distribution of multisensory neurons was
modulated by multisensory experience. Neuroscience, 272, 1–9. [PubMed: 24813435]

Autism Res. Author manuscript; available in PMC 2021 September 01.


Siemann et al. Page 30

Yaguchi A, & Hidaka S (2018). Distinct autistic traits are differentially associated with the width of the
multisensory temporal binding window. Multisensory Research, 31(6), 523–536. [PubMed:
Author Manuscript

31264616]
Yi F, Danko T, Botelho SC, Patzke C, Pak C, Wernig M, & Sudhof TC (2016). Autism-associated
SHANK3 haploinsufficiency causes Ih channelopathy in human neurons. Science, 352(6286),
aaf2669. [PubMed: 26966193]
Zhang M, Kwon SE, Ben-Johny M, O’Connor DH, & Issa JB (2020). Spectral Hallmark of auditory-
tactile interacttions in the mouse somatosensory cortex. Communications Biology, 3(1), 1–17.
[PubMed: 31925316]
Zhou H. y., Cai X. l., Weigl M, Bang P, Cheung EF, & Chan RC (2018). Multisensory temporal
binding window in autism spectrum disorders and schizophrenia spectrum disorders: A
systematic review and meta-analysis. Neuroscience and Biobehavioral Reviews, 86, 66–76.
[PubMed: 29317216]
Zilbovicius M, Meresse I, Chabane N, Brunelle F, Samson Y, & Boddaert N (2006). Autism, the
superior temporal sulcus and social perception. Trends in Neurosciences, 29(7), 359–366.
[PubMed: 16806505]
Author Manuscript
Author Manuscript
Author Manuscript

Autism Res. Author manuscript; available in PMC 2021 September 01.


Author Manuscript Author Manuscript Author Manuscript Author Manuscript

Table 1.

Comparing Cortical, Subcortical, and Behavioral Multisensory Findings Across Species

Human Non-Human primate Cat Rat Mouse


Cortical STS: [Beauchamp et al., 2004; Calvert et STS: [Bruce, Desimone, & Gross, AES: [Jiang et al., Parietal cortex: [Brett-Green, Fifkova, Parietal cortex: [Kuroki et al.,
Siemann et al.

al., 2000; Nath et al., 2011; Nath & 1981; Ghazanfar, Chandrasekaran, & 2001; Wallace et al., Larue, Winer, & Barth, 2003; Lippert, 2018]
Beauchamp, 2012; Powers 3rd, Hevey, & Logothetis, 2008; Ghazanfar & 1992, 1993; Wallace Takagaki, Kayser, & Ohl, 2013; Menzel PFC: [Sharma &
Wallace, 2012; Stevenson & James, Schroeder, 2006; Gu, Angelaki, & et al., 2006; Wallace & Barth, 2005; Raposo, Kaufman, & Bandyopadhyay, 2019]
2009; Stevenson, VanDerKlok, Pisoni, & DeAngelis, 2008] & Stein, 1994; Churchland, 2014] V2L: [Meijer et al., 2020]
James, 2011] Parietal cortex: [Avillac, Hamed, & Wilkinson et al., PFC: [Lipton, Alvarez, & Eichenbaum, Insula: [Gogolla, Takesian,
Parietal cortex: [Calvert, Hansen, Duhamel, 2007; Cohen, 2009; 1996] 1999; Reid, Jacklin, & Winters, 2014] Feng, Fagiolini, & Hensch,
Iversen, & Brammer, 2001; Driver & Schlack, Sterbing-D’Angelo, Unisensory cortices: Insula: [Rodgers, Benison, Klein, & 2014]
Noesselt, 2008; Miller & D’esposito, Hartung, Hoffmann, & Bremmer, [Allman & Meredith, Barth, 2008] Unisensory cortices and border
2005] 2005] 2007] V2L: [Hirokawa et al., 2008; Schormans regions: [Cohen, Rothschild, &
PFC: [Calvert et al., 2001; Miller & PFC: [Fuster, Bodner, & Kroger, et al., 2017; Schormans & Allman, 2019; Mizrahi, 2011; Deneux et al.,
D’esposito, 2005] 2000; Romanski, Averbeck, & Diltz, Schormans, Typlt, & Allman, 2017] 2019; Knöpfel et al., 2019;
Unisensory cortices: [Calvert et al., 2005; Romanski & Goldman-Rakic, Unisensory cortices and border regions: McClure Jr & Polack, 2019;
1997; Foxe & Schroeder, 2005; Lakatos, 2002; Sugihara, Diltz, Averbeck, & [Barth, Goldberg, Brett, & Di, 1995; Meijer, Montijn, Pennartz, &
Chen, O’Connell, Mills, & Schroeder, Romanski, 2006] Bieler, Sieben, Schildt, Röder, & Lansink, 2017; Morrill &
2007; Martuzzi et al., 2007; Molholm et Unisensory cortices: [Foxe & Hanganu-Opatz, 2017; Maruyama & Hasenstaub, 2018; Olcese et
al., 2002; Watkins, Shams, Tanaka, Schroeder, 2005; Ghazanfar, Maier, Komai, 2018; Sieben, Röder, & Hanganu- al., 2013]
Haynes, & Rees, 2006] Hoffman, & Logothetis, 2005; Opatz, 2013; Wallace et al., 2004; Xu,
Lakatos et al., 2007; Rockland & Sun, Zhou, Zhang, & Yu, 2014]
Ojima, 2003; Wang, Celebrini,
Trotter, & Barone, 2008]
Subcortical SC: [Olive, Tempelmann, Berthoz, & SC: [Frens & Van Opstal, 1998; SC: [Meredith & SC: [Gharaei, Arabzadeh, & Solomon, SC: [Doykos, Gilmer, Person,
Heinze, 2015] Wallace et al., 1996; Wallace & Stein, 1986a; 2018; Hirokawa et al., 2011; Lau, Manno, & Felsen, 2020; Drager &
Stein, 1996; Wallace & Stein, 2000] Meredith & Stein, Dong, Chan, & Wu, 2018; May, 2006; Hubel, 1975a; Drager &
1986b] Sparks & Hartwich-Young, 1989] Hubel, 1975b]
Behavioral AV detection: [Calvert et al., 2004; AV detection: [Cappe et al., 2010; AV detection: AV detection: [Carandini & Churchland, AV detection: [Meijer, Pie,
Frassinetti, Bolognini, & Làdavas, 2002; Miller, Ulrich, & Lamarre, 2001; [Burnett, Henkel, 2013; Gleiss & Kayser, 2012; Hirokawa Dolman, Pennartz, & Lansink,
Miller, 1982; Raab, 1962] Wang et al., 2008] Stein, & Wallace, et al., 2008; Sakata, Yamamori, & 2018; Meijer et al., 2020;
AV discrimination: [Raposo, Sheppard, SJ and TOJ tasks: [Frens & Van 2002; Burnett, Stein, Sakurai, 2004] (Siemann et al., 2015]
Schrater, & Churchland, 2012; Sheppard, Opstal, 1998; Kayser, Petkov, & Chaponis, & AV discrimination: [Raposo et al., 2012,
Raposo, & Churchland, 2013] Logothetis, 2008] Wallace, 2004; Jiang 2014; Sheppard et al., 2013]
SJ and TOJ tasks: [Colonius & et al., 2002; Stein, SJ and TOJ tasks: [Schormans & Allman,
Diederich, 2004; Stevenson, Ghose, Meredith, Huneycutt, 2018; Schormans, Scott, et al., 2017;

Autism Res. Author manuscript; available in PMC 2021 September 01.


Fister, et al., 2014; Stevenson, Segers, & McDade, 1989; Schormans, Typlt, & Allman, 2017]
Ferber, et al., 2014; Stevenson, Siemann, Wallace et al., 1998]
et al., 2013; Stevenson, Siemann,
Schneider, et al., 2014; Stevenson,
Siemann, Woynaroski, et al., 2014;
Stevenson, Wilson, Powers, & Wallace,
2013]

Note. This is by far not an exhaustive list; there are more examples of studies that can be found under each category for each species, and there are other cortical, subcortical, and behavioral findings that
could be listed for each species, which are not highlighted. The purpose of this table is meant only to begin to highlight some of the neuronal and behavioral similarities across these animal species.

Abbreviations: AES: anterior ectosylvian sulcal cortex; AV: audiovisual; PFC: prefrontal cortex; SC: superior colliculus; SJ and TOJ: simultaneity judgment and temporal order judgment; STS: superior
temporal sulcus; V2L: lateral portion of V2 (secondary visual cortex).
Page 31
Author Manuscript Author Manuscript Author Manuscript Author Manuscript

Table 2.

Findings of Altered Multisensory Processing in Neural and Behavioral Domains for Mouse Models Associated with ASD

Mouse model Cortical Subcortical Behavioral


BTBR T+tf/J Insula: [Gogolla et al., 2014] N/A N/A
Siemann et al.

Shank3 KO Insula: [Gogolla et al., 2014] N/A N/A


MecP2 KO Insula: [Gogolla et al., 2014] N/A N/A
Gad65 KO Insula: [Gogolla et al., 2014] N/A N/A
SERT Ala56 N/A N/A AV detection: [Siemann et al., 2017]
Caps2 KO N/A N/A Rubber tail illusion: [Wada et al., 2019]

Note. Note the recency, yet relatively limited number of studies, demonstrating the importance of these types of studies for future investigations.

Autism Res. Author manuscript; available in PMC 2021 September 01.


Page 32

You might also like