Spatial Perception and Cognition in

Multichannel Audio for Electroacoustic

Sonic Arts Research Centre, Queen’s University Belfast, Belfast, BT7 1NN, UK

Listeners experience electroacoustic music as full of analogies and spatial metaphors run through our
significance and meaning, and they experience spatiality as language and our reasoning. It is no wonder then
one of the factors contributing to its meaningfulness. If we that space can be a potent component of meaning in
want to understand spatiality in electroacoustic music, we electroacoustic music.
must understand how the listener’s mental processes give
When considering multichannel loudspeaker systems
rise to the experience of meaning. In electroacoustic music as
in particular, an important part of the discussion should
in everyday life, these mental processes unite the peripheral
auditory system with human spatial cognition. In the be focused on the distinctive and idiomatic ways in
discussion that follows we consider a range of the listener’s which this particular mode of sound production con-
mental processes relating space and meaning from the tributes to and situates meaning. The listener’s experi-
perceptual attributes of spatial imagery to the spatial ence depends on many factors, among which are
reference frames for places and navigation. When considering acoustic factors that directly affect the listener’s spatial
multichannel loudspeaker systems in particular, an important hearing and ultimately influence the spatial meanings
part of the discussion is focused on the distinctive and understood by the listener. In particular, there are many
idiomatic ways in which this particular mode of sound idiosyncrasies to spatial perception with multiple loud-
production contributes to and situates meaning. These speakers that distinguish it from other forms of audio
include the phenomenon of image dispersion, the important
reproduction and which contribute to its distinctiveness.
consequences of the precedence effect and the influence
Concerts of electroacoustic music in multichannel
of source characteristics on spatial imagery. These are
discussed in close relation to the practicalities of artistic reproduction are also an artistic idiom in which the
practice and to the potential for artistic meaning listener’s prior experience and understanding play a key
experienced by the listener. role in the apprehension of artistic meaning.

1. INTRODUCTION 1.1. The chain of (dis-)connection
Listeners experience electroacoustic music as full Auditory spatial imagery and spatial meanings can arise
of significance and meaning, and they experience when listening, when remembering or when imagining.
spatiality as one of the factors contributing to its This tells us that the listener’s spatial capabilities are not
meaningfulness. Perceived sound is always spatial, simply a product of immediate perception. Listeners
and spatiality is an integral part of every auditory think spatially. The spatial meanings that the listener
experience. Sometimes the spatiality is in the fore- experiences when sitting in an audience are the final
ground of attention and is a primary conveyor of result of a complex chain of thoughts and actions.
meaning. At other times, it slips into the background. At one end of the chain is the work of the composer
If we want to understand spatiality in electroacoustic imagining spatial meanings and auditioning sound
music, we must understand how the listener’s mental while assembling the composition’s components. The
processes give rise to the experience of meaning. typical products of this effort are the tracks of sound
The feelings and thoughts that the listener associ- brought to the concert hall and the directions for how
ates with the experience of sound in space appear to these tracks are distributed among loudspeakers. At the
arise from a deeply embodied knowledge of space and time of the performance, there may be someone, pos-
spatial movement. This spatial knowledge is acquired sibly the composer, who manipulates the reproduction
through sensory experience that presumably starts system and performs live diffusion. This person brings
at the beginning of life and continues through inter- his or her own intentions for spatial meanings and
actions with the everyday world. Notions of space, nuance to the situation. The sounds projected into the
acquired and understood through bodily experience, hall provide the stimuli for the audience members’
underlie a great deal of our everyday thinking. Spatial experiences, which may be quite diverse. For one thing,
Organised Sound 15(3): 228–238 & Cambridge University Press, 2010. doi:10.1017/S1355771810000336

relationships is one of the ways in which humans the goal of a multichannel audio system should be to develop their multimodal conceptual understanding deliver acoustic signals to the ears of the listener that of events. should be to evoke experiences in the listener with The two may have little to do with one another. This enables us to discuss spatial experience of the listener. These are events that typically involve objects. we also under. The perceptual attributes of the spatial imagery tion. the interaction between immediate perception and our better we will be able to harness the capacities of such understanding of the world around us. He then goes on to describe the immersive place in space. the conceptual event provide the stimulus for such artistic spatial experi. there remain some ginal acoustic events and the signals that finally reach fundamental commonalities in listeners’ experiences. Importantly. There is a constant sound systems and the listener’s spatial thinking. Rumsey’s spatial attributes start with the dimensional obscure or unknowable. spatiality being the many details of multichannel perception in terms that aspect that is generally the least well preserved. meaning emerging one hand. actions and agents. Events and their object–action–agent constituents who often describe this as research into spatial quality. nology (Zacharov and Koivuniemi 2001. This mediating system may have Some of these commonalities arise from the general some effect on the fidelity and timbre of the perceived characteristics of spatial hearing. This is the same way that we relax our framework of 2. even though these constituents may be unobserved. perceives and understands can be most accurately described as events that take place in space (Kendall 2. given the com. Therefore. attributes among which he includes distance. They also have different events. different associations with events take place. SPATIAL AUDIO AND SPATIAL accommodate potential artistic meanings that arise COGNITION from unexpected or novel spatial relationships. There is a long history of studies that connect and one’s grasp of sensory experiences as events is the attributes of spatial imagery with specific termi- fundamentally multimodal. The spatiality of events spatial relationships when viewing a painting that When a listener hears sounds in space.2. are experienced through multiple sensory modalities. The more that we under. proposes a framework for a very concise vocabulary. as well as the likely physical context in which reasons for being present. the electroacoustic music. what the listener stretches and distorts space for expressive purposes. may influence and constrain the perceptual image: ences and understandings. the recording shaped by the situation. and reproduction system mediates between the ori- In the midst of all this diversity. Clearly what systems for artistic purposes. but it almost certainly has an effect on the uniform across all people. variations in the apprehension of meaning that are When one listens to recorded sound. actions and agents. in the context of electroacoustic art. The are applicable to all listeners. for example we are more likely to hear an airplane stand about the complex relationship between spatial above our heads than below us. Importantly. Rumsey 2003). On artistic meaning: in particular. the spatial properties of the conceptual event and the The goal of spatial audio in electroacoustic music spatial imagery that the listener experiences directly. the listener’s ears. and so on. These are conceptual properties the sound sources. the direct perception of auditory spatial from the spatiality of the perceived sound. On the other hand. Particularly instructive for our pur- they are able to make sense of these experiences as poses is an article by Rumsey (2002) in which he events that take place with objects. each person gradually learns to understand more produced with loudspeakers have been well studied complex and more nuanced relationships among these by researchers in the audio engineering community three. by intention and more. different levels of experience with of how the listener understands what is heard. which are relatively events. Over the course of human matura. we can to emerge from between a pair of stereo loudspeakers assume that the audience members have many shared or. the listener must relax the tight grip of plausibility to 2. Given the understanding to these shifts of scale and perspective specifics of any particular listening situation. The attributes of spatial imagery 2008). Also. just so many percepts and plausible understandings for This illuminates an important distinction between a listener who is attentive to the experience. Spatial Perception and Cognition in Multichannel Audio for Electroacoustic Music 229 these listeners are spread about in different locations scale of the actions. So it is not surprising that. There are bound to be conceptual event that the listener envisions. in the case of headphone listening. Listeners can often infer the physical attributes.1. there are with amazingly little effort. it may appear experiences and understandings of everyday spatial inside of the listener’s head. objects and agents that produce that affect their perceptions. width stand events as intrinsically spatial because events take and depth. we perceive can be influenced by what we expect. Berg and when listeners have sensory experiences of sound alone. Human beings adapt their relationships that evoke shared meanings. spatial image of an orchestra performing may appear monalities of living in the everyday world. which he separates into presence (‘being .

In order to itemise the spatial attributes of electro. aggregation. which face forward and create Rumsey’s terminology may only have been intended spectral differences between sound events from the front to describe the spatiality of conventional recordings. 2002: 662) and several Once again. Left–right judgments are the least error-prone and which was not relevant to Rumsey’s original discus. the left–right axis is fundamental. which is important for systems with of this axis. Both the orientation of Rumsey’s article is that these spatial attributes char. but his list of attributes is somewhat limited because of a focus on 2. Kendall relation to the spatial image and the other to the con- ceptual event. ensembles and sometimes listeners. inside of an enclosed space’. identifying a plurality of sound events as an ensemble signifies a category in which there is a likely commonality of behaviour. And yet. overlapping. and clearly the listener’s most important spatial framework is anchored in the body. also grouped together the three attributes that relate The front–back axis is another axis of bodily sym- to spatial extent as depicted in figure 1. the front–back distinction is very organised in terms of sources. Rooms can contain sources. For one thing. the perception of envelopment and presence may influ- Figure 1. macro-level. the list auditory spatial art depends greatly on the everyday must be slightly enlarged. it is the axis that aligns with are two fundamentally different ideas at work. the one axis of > Distance the body that is truly symmetrical. . we have added direction. the notion of room is not merely a particular kind of source with spatial attributes. these are ways in which the listener makes cognitive sense of auditory phenomena.230 Gary S. rooms and dependent on the dynamic movement of the head. Dimensional attributes: For example. dation for describing spatial imagery. and so forth. but and the back. especially in the absence of head motion. the separation of left from right defines > Direction a primary axis for the body’s space. the front–back distinction depends on the tained within rooms that are enclosed within scenes. the spatial imagery that is directly perceived kinds of envelopment (being surrounded by sound). too. We understand the > Extent: depth. In the second case. acoustics of the pinnae. stood are potentially quite distinct. width and height left–right axis first and foremost as a bodily distinc- Immersive attributes: tion. the eyes and the means of locomotion accentuate the acterise spatial images that range from the micro. For example. Front–back distinctions are somewhat his concept can easily be extended to describing spatial error prone. in audio reproduction the room can just as easily be reduced to a narrow point source with the listener on the outside. Importantly. We have is on the left or right. People seldom misjudge whether a sound loudspeakers at multiple levels of elevation. But as Kendall (2007) points out. The space around the body conventional recordings and reproduction systems. In the first case. experience of space. scenes. Illustration of the attributes of spatial imagery in ence the listener to assume a perspective from inside the loudspeaker reproduction. Rumsey proposes a scene-based paradigm In directional hearing. sources maybe grouped into ensembles that are con. To Rumsey’s original list. one in gravity. In directional hearing. How the listener experiences artistic meaning in acoustic music reproduced with loudspeakers. but also an understanding in which one thing contains another. and the conceptual spatial organisation that is under- Rumsey’s article provides a cogent conceptual foun. ensembles. Secondarily. A spatial image by itself may or may not have this conceptual characteristic. there other axes. room. the most important contribution of around in the physical world. metry and is fundamental to manoeuvring our way Undoubtedly. the greatest directional acuity is centred in the middle sion. and gravity imbues it with special significance. Then. and height.3. For Rumsey these are hierarchic groupings: which clarifies what is in front from what is behind. The separation of the ears along this axis gives > Presence rise to the interaural differences in time and intensity > Envelopment that are cues to the direction of sound the distinction between what is in front and what is behind. Rumsey’s levels provide a way of thinking about the spatial arrangement of images: inclusion. imagery in electroacoustic music (see Emmerson 1998 The up–down axis is experienced differently from the and Smalley 2007 for related concepts).

We rarely travel vertically in an comparison to the physical space (Tversky 2009: 202). Kendall (2007) discusses how the typical everyday spatial activities are taken in through the acoustic behaviours associated with the schema senses and integrated into and in accordance with our Containment can be violated by signal processing to ongoing mental representation of the world around situate listeners inside a container where they should us. the higher timbre. by one’s self. say changes in pitch and acteristics of the sound source. Roffler located. presumably a great deal spatial learning and testing Then. (This happens in Denis Smalley’s Empty physical world. A aroused when listening to electroacoustic music has corollary to the fact that ‘events’ are intrinsically its roots in a lifetime of embodied spatial experience. In directional hearing. In general. which are completely asymmetric from top to bottom. For a discussion of events and felt experience. it cannot simply be the acoustic behaviours that we associate with Object. At early stages of development. The listener’s felt experience of spatial motion It becomes imagined space and even metaphorical in electroacoustic music is typically imbued with the space. effort. vision on the plausibility of auditory spatial percep- objects. the higher the event. For example. structing and updating some form of spatial repre- The deeply meaningful sense of space that is sentation in which events have spatial persistence. the conceptual space is typically distorted in ‘floor’ of the space. and bright demonstrates that listeners must be mentally con- events appear more elevated than dark ones. Our schemas. Over a lifetime. it remains fixed relative to the environment. Dynamic the pinnae are confounded with the spectral char. The way in which such tional hearing below the body is almost non-existent. This helps us to understand the influence of to the experience of spatial events. relative positions are preserved helps us to construct a Importantly. If we turn during a sound event. high-frequency sound events appear Franssen effect (Hartmann and Rakerd 1989). may aid us in updating our understanding of the spectral energy distribution of the source. despite rotations of the body. Thinus-Blanc and Gaunet 1997). If one compares a acoustic behaviour expected of the schema for Room. One of these is the image schema Object that From our preceding discussion. there is music is curved like the movements of the body. This is verified in listening tests input to maintain our notion of where things are with a variety of filtered signals (Bloom 1977. actions and agents are most easily grasped tion. It should come as no surprise then that All of this reveals how the mind sustains spatial so much of the spatial movement in electroacoustic thinking. For a practical understanding of (Johnson 1987). acoustic environment or experience sound events One factor that aids listeners in maintaining their below us. Directional multiple events maintain their relative positions hearing above the head is very imprecise and direc.) example.) Listeners and rooms can also space of the mind is an active construct that is be separated from each other in violation of the dynamic and seeks conformation. The persistence of a plausible location in the and Butler 1968a.4. (See Kendall 2010 for codified at many different levels of specificity. multimodal is that we expect them to be multimodal. and it must be independent of any one perceptual In electroacoustic music. Even Elevation perception is the least accurate. Beneath the apparent continuity of everyday space If we see a glass drop to the floor. but we do not need constant perceptual the perceived elevation. In general. we expect to hear are the axes of the body giving structure and context it too. Gravity binds figure to ground and defines the space. The space of the physical world is Vessels when the listener’s perspective is shifted inside stable and requires no effort to be maintained. The of a large garden pot. Spatial Perception and Cognition in Multichannel Audio for Electroacoustic Music 231 It has forces at work on the body and the environ. constructed moment-to-moment from immediate such as the fact that actions can cause a sound to perception. But the space of the mind is not the space of the not be. location of things. 1968b. Susnik. Our understanding must have persistence emanate from the object’s location. one of the ways in which modality (though there is good evidence that vision spatial events take on artistic meaning is by violating plays a pivotal role in the evolution of our spatial the acoustic behaviours that we associate with image understanding. in the context of the body. elevation cues are spatial framework is their ability to keep track of the primarily provided by the acoustics of the pinnae. Lakoff and Johnson discuss image schemas. Lakoff 1987. changes in a sound event. The space of the mind 1999). too. which more elevated than low-frequency events. Events with visible sources are more plausible in terms of the familiar: objects that are manipulated than ones where nothing can be seen. Recurrent patterns of spatial thinking become flow dynamics of bodily effort. particular space as conceived with the actual physical ment. Sodnik and Tomazic face of contradictory acoustics is illustrated by the 2005). motion requires while we develop our ability to think in terms of space. it should be obvious captures the spatial commonalities of everyday objects that the axes of spatial hearing are quite different such as the fact that objects have a location in space from each other. which capture some of our most basic spatial patterns (Johnson 1987. Lakoff and Johnson 2. we have also learned the space around us to emerge. . the acoustic cues to elevation created by stable sketch of what is going on around us.

microphone space. tinuity of Path psychologically implies some continuity of agency or action. specific concert halls or specific forests. whe- actions and behaviours. the movement of objects. The con- fragmented movement or multiple spatial events. ingful spaces. we network of associations. but in fact this space. Then. anchor points in our spatial orientation. Frames and journeys catalogue of reference frames of various kinds. (c) and (d) illustrate violations of those schema easily produced through spatial DSP (after Kendall 2007). The electroacoustic listener not only also learn that certain places and certain contexts makes sense of immediate spatial events but also are characterised by particular spatial behaviours and experiences meaning when connecting these events we construct mental reference frames that capture to frames which situate the events within a complex these associations (Tversky 2009). How image schemas help us to make sense of events that are difficult to grasp is illustrated by the image schema Path. so forth. These examples are illustrated in figure 2. activating other frames in order to understand what is heard. How the schema Path gives spatial continuity to moving along a shared trajectory (figure 3). We develop basic mental ther they are real or imagined. We may envision a prototypical concert The movement of the body. listening to a without a sense of location? recording. Beside such general frames. Then. while at the same time they are metaphorical (Kendall 2010). How impoverished would associate the concert hall with a reference frame as it be to listen to Luc Ferrari’s Presque rien no 2 or well as the forest. we often have Infants actively explore the world and they experience prototype frames for familiar or particularly mean- space through actions requiring both time and effort. Path can pro- vide a thread of continuity to a combination of events Figure 3.232 Gary S. For example. Denis Smalley (2007) provides an extensive 2. We tree in Avatar. A journey does not need to lead for their physical location whether it is a concert hall to realistic consequences and may even be purely or a living room. hall or a prototypical forest. The space beyond the framework of the body may frames that he associates with terms such as gestural appear to be a priori to most people. which can imbue fragmented spatial movement with a sense of coherence. Our frame for the concert hall Hildegard Westerkamp’s Kit’s Beach Soundwalk could be evoked by being at a concert. The egocentric associated with signposts and landmarks that serve as perspective (body-centred) and the allocentric per. too.5. we may the movement of other people – all of these con. or just imagining something being played Of course. and space must be traversed to be properly grasped. whether it is spective (external landmark-oriented) appear to arise the fountain outside Avery Fisher Hall or the home quite spontaneously (Taylor and Tversky 1996). too. art does not carry the constraints of at a concert. Kendall Figure 2. Electroacoustic listeners hold a frame real world situations. The schemas for Container and Room are represented in (a) and (b). Such places often are frameworks for spatial perspective. have particularly rich frames associated with specific tribute to an understanding of space through spatial places. prospective space. but navigation does .

the listener is not only able to perceive spatial loudspeakers. This study demonstrates that. loudspeakers conformed to expectations and the rear Such combinations of spaces could be likened to a pair produced similar results. In either case. but most events will not. Ratliff (1974) studied the per- many other pieces. (Ratliff from being in an art gallery or a shopping mall. When listening specifically to electroacoustic music. front of the listener in stereo reproduction does not When listening to Barry Truax’s Island. 1974: 12) In multichannel reproduction. Separate places and their spa. The side quadrants exhibit a great degree of uncertainty. We know that some events systems: Holman 2008: 182–3). Spatial Perception and Cognition in Multichannel Audio for Electroacoustic Music 233 imply time and effort. determined that the direction of phantom images pro- duced by amplitude panning varied for every pair of 3. Treating loudspeakers and the room confirm Ratliff’s. dif- study was followed by Theile and Plenge (1977) ferent traditions for how the spatialisation is handled who investigated phantom images with loudspeakers and listeners also may adjust their frame depending 608 apart and at different angles in relationship to the on the extent that they recognise the idiomatics of listener. the listener will If quad loudspeakers with 908 of separation create probably group and interpret events within one or these issues. y we bring with us reference frames and associations There is a distinct threshold interchannel level-difference that situate the meanings we construct. The reference frame of multichannel It would appear that the ‘stereophonic-image’ phenom- reproduction enon breaks down at the side of the head y Also there is preferential reception of the front loudspeaker. It has been long known that the may anticipate upcoming events and signposts. even though these locations may be only In one sense it represents a localisation error. what the listener listener. we activate reference Even from the sweet spot. During the era of notion of when the next landmark will arrive. The view from the sweet spot from the sweet spot. side and series of connections. While striving to can be exploited for artistic purposes. Phantom images are that we are exploring a Pacific island. experiences depends on these mental reference frames In a study of localisation with an eight-channel as well as the acoustic and perceptual details of the loudspeaker system. the same result in all directions. IDIOSYNCRASIES OF MULTICHANNEL loudspeakers. In multi-loudspeaker repro- and follows a path of changing location within that duction. From the vantage point of the sweet may be reduced but is not eliminated by additional spot. but: cognitive collage (Tversky 1993). His data for the front pair of tial frameworks are abridged and connected together. Their eight-channel system was a familiar REPRODUCTION octophonic arrangement with a vertex to the front. but behaviour that we associate with phantom images in the time at which they arrive is probably uncertain. but we have no extremely fragile on the sides. In simple cases. In quadrophonic sound. Simon. There are. In another sense. Treating the loudspeakers and room relying on phantom images to the side and instead as a vehicle for creating illusory images and spaces to position loudspeakers to the left and right of the produces another. the sweet spot is equidistant differences observed in the certainty of the phantom from the two loudspeakers and set back to form an image’s location which indicate that image dispersion angle of 608. This is a context in which we know that enon of spatial images shifting and spreading between loudspeakers represent the most plausible locations of loudspeakers on the side we will call image dispersion. it is may conform to the spatial behaviours of the physical one of the idiosyncrasies of multichannel audio that world. The listener all loudspeakers. the sweet spot is roughly equidistant from framework. Mason and Rumsey (2009) multichannel reproduction. their results such traditions. even in the case of listening 3. . navigation is best described as a ception of phantom images to the front. Just being in a in this region. sound events.6. Important too are the In stereo reproduction. simple panning does not produce The listener’s ideal physical location is the sweet spot. of course. In respect to image dispersion. about which small variations cause concert hall situates meaning in a way that is different the image to jump towards the front or back. (Tom loosely related to spatial imagery and totally unrelated Holman describes it as a problem to avoid in 5. the listener imagery at the loudspeaker locations. then the obvious question is how close more spatial frames familiar from the real world or do the loudspeakers need to be to avoid this? Ratliff’s from electroacoustic art. we imagine continue around to the sides. but also phan- navigates space within a single spatial framework tom images in between.1 to conceptual sound events. understand the ongoing experience. y 2. The journey may rush or linger. This phenom- of plausibility. it is difficult to achieve a frames that also shape our expectations and our sense compact phantom image to the side. One consequence of Theile and as being the composition’s environment produces one Plenge’s results is their recommendation to avoid kind of result.1. back of a listener.

when the reproduction system is scaled up from a tribution is largely suppressed. Because the difference in distance cedence is said to be released. arrival-time differences (ATD) in the relevant range for tener and two loudspeakers. But the size of the circle makes a huge differ- When there is no ATD between the two loudspea.7 10ms 11. From 1ms to at in proportion to the relative size of the circles. but the arrival–time difference (ATD) is the most critical for called the echo threshold. at some point after 5ms there is a sound system with loudspeakers at 08. the acoustic signals reaching the listener 20ms 22. Illustration of arrival-time difference (ATD) at is better able to localise the sound source despite the ears for two loudspeakers at different distances. For transient material between the two loudspeakers and listener is greater such as clicks. Yost and arrangement of eight loudspeakers in a circle shown Guzman (1999). The view from outside the sweet spot Outside of the sweet spot.9 will differ in arrival time. Depending on the died phantom imagery in a five-channel surround source material. pairs B–C and H–B for the listener halfway between dually moving the ATD from 0 to 1ms. Haas 1951. circle.5 6. Corey and Quesnel (1999) stu- dominates the perceived direction. listeners typically perceived a single phantom illustrates the ADT and intensity ratio of loudspeaker image located between the loudspeakers. the amplitude ratios between pairs summing localisation. Sound emanating from precedence.34 When identical signals emanate from the two loud. Consider the common effect is offered by Litovsky. the point at which pre- our discussion. At the sweet spot. What makes this study interesting for us is sound events at each of the loudspeakers. Blauert 1971) has a crucial impact on the listener’s spatial imagery.234 Gary S. This is that it investigated both interchannel amplitude and . When comparing location of the phantom image will shift toward the typical small and large circles (diameters of 4m and loudspeaker that is leading in time. As shown in Table 1. Table 1. ADTs in the range affected Precedence is a multifaceted phenomenon and by precedence occur when loudspeakers are offset an excellent contemporary review of the precedence by relatively small distances. differences. This least 5ms there is a region of fusion in which a single is a clear demonstration of why precedence can sound event is perceived at the location of the leading dramatically change a composition’s spatial imagery loudspeaker while the lagging loudspeaker’s con. either loudspeaker will arrive at the listener’s ears ATD feet metres with interaural time and intensity differences. ence for listeners outside of the sweet spot. When encountering a combination of direct sound and indirect sound.6 1.3 3. of localisation dominance in which the leading sound Martin. the apparent the sweet spot and loudspeaker C. Woszczyk. Offset in the distance of two loudspeakers for Consider the simple situation in which there is a lis. This is also the region small room to a large room. Colburn. 1ms 1. intensity and direction. The precedence effect varies depending on these three factors and the source’s characteristics. as shown in figure 4. 1/2 308 and transition region in which the listener begins to hear 1/2 1208. When gra. This is the zone of 12m respectively). the first arriving sound. in which the contributions of of loudspeakers are the same. Table 2 kers. the auditory system effectively gives precedence to the direct sound. part of the directional cues to the loudspeakers’ locations.4 speakers. Precedence is often descri- bed as the auditory system’s mechanism for clarify- ing directional hearing in reverberant environments.2. every listener is closer to one loudspeaker than another. the echo threshold begins around 5ms. the presence of sound reflections. Kendall 3. speakers arrives at the same time and at the same Consider the case in which our two loudspeakers are relative intensity regardless of the diameter of the located to the front-left and front-right of the listener. and the precedence effect (Wallach. than the distance between the ears. but the ADTs change both loudspeakers are integrated. The perceptual consequence is that the listener Figure 4. sound from all loud- with the perceived direction of the sound event. But precedence is the same whether the delayed sound comes from an environmental reflection or a distant loudspeaker. the a higher order of magnitude than interaural time echo threshold may range from 20 up to 50ms. 5ms 5. ATDs are of For sonic material with more continuous content. One facet of precedence has to do in figure 5. Newman and Rosenzweig 1949.1 0.

4 23 1. Perrott. 2007). (1999) and is simply a different way of conceptualising their results. the faders positions a stereo source signal simultaneously they are spatially segregated. the sound (Adair. The study provides an insight into There is a flip side to the influence of the pre- image dispersion in the context of precedence. . The relationship of image dispersion to ATDs in the case of a front and a rear loudspeaker both on the same side. This can be accom- in live sound diffusion of electroacoustic music.5 spread in relation to a range of ATDs. The interchannel time differences produce ATDs at the listener’s position 3.6 3. with the high and low fre. but two identical sounding sources which are seriously compromised.6 11. they point out that when even when the rear loudspeaker has a 2ms lead. interchannel time differences.6ms.3 25. the listener perceives not of the images as appearing from a single location is one. the operator will experience image dispersion that spreads the sound images from the front to the back on both sides. How this phenomenon can quencies appearing to originate from different direc.) have micro-variations or other ongoing differences Knowledge of image dispersion and precedence that cause them to be decorrelated while leaving the enables us to explain a practice that occurs regularly identity of the source unaffected. its location dominates and Manligas (1987) point out that there are impor- the listener’s judgement of direction until the ADT is tant conditions under which the precedence effect around 0. the authors confirm that ‘the integrity uncorrelated to the first. Adjust- Table 2. As an example. Saji. (See Ootomo. the intensity ratios and the arrival-time differences changes imagery across the listening space and also (ATDs) for a listener halfway between the sweet spot and affects the listener’s impression of being enveloped by loudspeaker C and for two different diameters of the circle. Alcorn and Corrigan 2008). Two listener positions are represented. image dispersion occurs over a wide range of ATDs. hardly affected by ATDs. From the vantage point of the sweet spot. The Diameter B/C B–C H/B H–B image dispersion effect is strongest for broadband in metres dB ms dB ms signals and source signals made up of a mixture of events (as happens with live diffusion of stereo tracks). Spatial Perception and Cognition in Multichannel Audio for Electroacoustic Music 235 Figure 6. one noise signal is presented with another noise signal Importantly. But since central sweet spot and one halfway between the sweet spot and loudspeaker C. From image width to envelopment akin to those produced by having loudspeakers at different distances. The figure is extrapolated from Martin et al.3. When cedence effect on spatial imagery. Tanno.9 Figure 6 represents the way in which spatial images 12 23 4. The prerequisite and Hatano 2007 for a related study with trade-offs is that the individual versions of the source signal between ATDs and intensity ratios. Familiar arrangement of eight loudspeakers in a Every other listener is in a different location with circle. Figure 5.1 25. plished either during synthesis or by post-processing. For the loudspeaker arrangement shown in ing the relative balance of the front and rear channels figure 5. Consider the situation in which the person operating When such signals are routed to separate loudspeakers. one in the differing distances from the loudspeakers. At that point. in a front pair and a rear pair of loudspeakers. Strybel the 308 loudspeaker leads. be extended to arbitrary source signals is thoroughly tions’ (1999: 7–8). most listeners will perceive the effect even though the imagery probably varies with each location. Huang discussed by Kendall (1995. the localisation spread does not occur: there is no fusion and no localisation increases dramatically and is not markedly reduced dominance.

In general. Karlheinz Stock. 3. Belfast. The distribution of sub. utilise the basement loudspeakers. the less that azimuthal panning works similarly to the since the relative intensity ratios are constant when horizontal plane. the shorter the delay natural thing to do! A characteristic of the many large. are to lateral fluctuation’ (Bass 2010). But it is well worth mentioning that ears are not as sensitive to height fluctuations as they signal processing can reshape the situation. Interaural relationships above the size of the reproduction system scales up. This is probably Etlinger (2009) demonstrated that the sweet spot could why so many composers find it practical to organise effectively be moved by adjusting the outputs of the their tracks in stereo pairs. Kendall The absence of precedence in this situation These include the ZKM Klangdom.4. source material. Clearly ceiling and base- ment loudspeakers work in a way that is different from common expectations. During an Rumsey (2002: 660) points out. Rakerd and Hartmann (1986) found that subjects were able to 3.236 Gary S. limited signals (Blauert and Col 1992). sources are localised in the ceiling loud- speakers and listeners. the sweet spot in multichannel reproduction is before. The notion that spatial percep. the used elevated loudspeakers. Then work well only with broadband sources. without regard to ATDs. Broadband signals space multichannel systems emerging today is the generally have a shorter echo threshold than band- use of loudspeakers at multiple levels of elevation. narrow-band sources and sources with between the perceived spatial imagery. One of images with extraordinary width. BEAST and the improves the scalability of spatial imagery between Sonic Lab at the Sonic Arts Research Centre. and the relative locations of loud. the richest stimulation to the listener’s perceptual woofer. Then. It must have seemed a more transient the source. Sharp attack loudspeaker orchestras such as the Acousmonium transients create a momentary burst of energy across (Gayou 2007).5. but they are very good at localising sine sound event is spread. For relative intensities of the individual loudspeakers example. Referring again to Table 2. waves with transient attacks. the acoustic sharp attack transients are localised below the floor. For decorrelated sig. an the attribute we call source width become another electroacoustic composer was asked why he did not one called envelopment?’ The lack of a clear bound. The influence of source characteristics Image dispersion and signal decorrelation are two ways in which sound artists can create spatial Spatial hearing is complex and multifaceted. This also helps us its interesting facets is how onset characteristics affect to understand an important aspect of the practice of the listener’s directional judgements. small rooms and large rooms. An multiple loudspeakers (or that spatial perception is insightful comment from the same electroacoustic independent of the characteristics of the sound source) composer about elevated loudspeakers: ‘I think my is far too simple. too. istics of the source. This is continuum between single. fused images and imagery especially true for ‘de-elevated’ loudspeakers such as spread among all active loudspeakers. degraded the localisation accuracy of continuous hausen (Toop 1981) and Edgar Varèse (Treib 1996). In the Sonic Lab. hearing and the least prone to error. Experience with producing electroacoustic music in nals. For exam. In fact. And. loudspeakers to produce zero ATDs and unity intensity ratios at other locations within the audience space. imagery head are quite different! Vertical panning appears to created with uncorrelated signals scales up too. as those below the floor in the Sonic Lab. the early pioneers of electroacoustic music They also found that room reflections dramatically Pierre Schaeffer (Teruggi 2007). the higher the elevation of the loudspeakers. Pause to reflect perception is confounded with the spectral character- The above discussion of image dispersion and pre. ‘at what point does interview concerning live diffusion in this space.6. elevation 3. before the release of precedence. signal processing can control the degree of elevated loudspeakers sound images are frequently decorrelation among signals so as to create a spatial disassociated from the loudspeaker locations. Beyond the horizontal plane localise sine waves with short onsets in the range While elevated loudspeakers are not an everyday 0–100ms despite the presence of room reflections. only low- cedence illustrates the complexity of the relationship frequency. speakers only when they have relatively high fre- tion is uniform as signals are uniformly panned around quencies or when they have sharp attack transients. As discussed ple. mid-range and tweeter loudspeakers in space system. His answer: ‘But it ary encourages artistic exploration. the Gmebaphone (Clozier 2001) or the the whole range of audible frequencies and provide BEAST (Harrison 1999). and very . experience. the left–right axis is fundamental to spatial located where it is only in the absence of moving it. all sounds without these sharp attacks. for again. As discussed before. In a study of precedence with loudspeakers and room reflections. Listeners are generally very poor at localising creates a canvas over which the spatial image of the sine waves. imagery is determined by the directions and such a venue teaches some important lessons. doesn’t work’ (Bass 2010).

1992. and Rumsey. 2003.. In the absence of an attack transient. The Gmelaphone Concept and most of these spatial images appear closer to the lis. Banff: 184–98. M. I. J. Spatial meaning emerges from limitation of being reproduced only in quad. the listener’s impression of the location can spatial perception and also acknowledge that listeners be stuck or can lag behind what acoustics would are making meaningful sense of spatial events. Christian. This is especially true Computer Music Journal 25(4): 81–90. influenced by the source’s spectrum. J. one location (generally at the loudspeaker) while the Bass. phantom images are often elevated above the Blauert. Cazals. the cratic characteristics of multichannel reproduction. We need to engage spatial audio in all of its tion. Proceedings of the AES 24th Depending on the spectral characteristics of the International Conference. the Cybernephone Instrument. S. Irregularities in the Pre- loudspeakers. 1971. C.P. Creating Source Elevation by Spectral Lab. sound events localise somewhere in between. If sharp the embodied aspect of spatial perception. below the floor and only very high narrow-band Psychoacoustic data. The initial spatial impression triggered by an attack transient can have a decisive impact on the listener’s judgement of where a sound event is located. P. what- that the listener’s initial judgement of the event ever one thinks about the aesthetic issues. Belfast: ICMA. 2003. beginning section of Chowning’s Turenas create a There is a distinctive and intricate linkage between vivid sense of a moving sound source despite the meaning and space. and. it is sometimes difficult to know when a signal is Manipulation. CONCLUSION first impressions tend to persist. 2001. Horner head and below the body is so imprecise. Slow onsets greater than will be able to search out those new and unique 50–100ms are generally sufficient to avoid the tran- possibilities for meaning that are yet to be discovered sient localisation effect. No indicate is the instantaneous location (Hartmann and viable approach to spatial audio can ignore how the Rakerd 1989. trans. S. Localization and the Law of the First height of the loudspeakers (Ratliff 1974: 12). few seconds will certainly avoid it (Rakerd and Hartmann 1986). Personal correspondence. tener than to the loudspeakers. and sound events with ongoing transients tend intricacies. Journal of the Audio Engineering Society 25: 560–5. Auditory Physiology and Perception. the technical location remains in effect until some countervailing schemes or the compositional methodologies. and it emerges from the idiosyn- situation. the locali. Alcorn. and Corrigan. listener’s localisation judgement depends on the the nature of the artistic medium itself. L. but onsets in the range of a through spatial audio. A Study This effect also occurs with many acoustic sources Into the Perception of Envelopment in Electroacoustic that contain both transient and continuous compo.J. Systematic Evaluation of continuous sound is elsewhere. together (Gregorio 2009). In the Sonic Bloom. The to our time ought to be based in knowledge of human rapid attacks of the high-frequency tone in the perception and cognition. The Precedence Only very low-frequency narrow-band signals localise Effect For Noise Bursts of Different Bandwidths. . J. Blauert and Djelani 2003). and Djelani. 26–28 June. (eds. Similar illusory tical Society of America 50(2): 466–70. Music. it appears approaches to multichannel reproduction. Oxford: sation of signals from these loudspeakers is easily Pergamon Press. Proceedings of the 2008 International Computer nents. Perceived Spatial Quality. J.. Journal of the Acous- well known to recording engineers. T. source. then slow onsets produce quite the opposite spatial experience. elevations can be created with ceiling and sub-floor Blauert. On the other hand. 1977. Acoustical Science and Technology signals localise at the ceiling. Whatever the charac- Electroacoustic music embraces a multiplicity of teristics of the continuing acoustic signal. it is difficult to quantify the influence of slow onsets with broadband REFERENCES noise because the perceived source tends to break up into separate frequency bands in different locations. a fact Wavefront in the Median Plane. 2010. every cue triggers a change. originating from the ceiling or from below the floor! Braasch. Adair. nor can it limit its con- of a new transient in an otherwise continuous event siderations simply to the physical acoustics of repro- can trigger an update of the event’s perceived loca- duction. Justice Olsson. we ing at the listener’s ears. It emerges transients assist the listener in localising sound in ways that are idiomatic to spatial thinking and events. Sometimes the transient component localises in Music Conference. Damany and K. Possibly because localisation above the cedence Effect. Through the relationship of the continuous acoustic signals arriv- combination of technical and artistic exploration. And 4. But the introduction listener perceives and thinks. Etlinger 2009). 2008. and Clozier. In Y. Most typical musical 24(5): 233–41. In the case of slow continuous approach ought to be informed by the realities of changes. F. Spatial Perception and Cognition in Multichannel Audio for Electroacoustic Music 237 narrow-band signals weaken localisation dominance when ceiling and sub-floor loudspeakers are used (Braasch. Any approach to spatial audio appropriate to continually update the listener’s judgement. Blauert. J.). Berg. and Col. J. 531–38.

H. Interface 10: New York: Basic Books. Copenhagen: ICMA. Martin. Theile. J. W. PhD dissertation. Schloss Acoustical Society of America 80(6): 1.U. and Johnson.. NJ: Society of America 106(4): 1. 38. W. G.366–73. Journal of the Acoustical Society of America 43: and Early Reflections Sound Processing. Chicago. Tanno.K. tion. H. D. 2009. . and Tversky. G. 1968b. and Gaunet. G. P. 2002. Treib.. R. A Musically Motivated Approach to ment No. Kendall. Robbins and M. The Artistic Play of Spatial Organiza. Psychological Bulletin 121(1): 20–42. G. Cognitive Collages. Based Paradigm. Cambridge University Press. Strybel.. Journal of the Acoustical Society of International Computer Music Conference. and Sound Source Localization in a Five-Chanel Surround Spatial Mental Models. Thoughts on the ‘What’. 2008. Litovsky. Sodnik.R. Perspective in Spatial its Impact on Spatial Imagery. Zacharov.R. Organised Sound 3(2): 117–27. G. M. 1981.K. Preprint No. Smalley. Locali- Diffusion. The Body in the Mind: The Bodily Basis Elevation in Acoustic Image of Space. Tversky. 1974. and Rosenzweig. Journal of the Acoustical Pavilion. Spatial Information Theory: A Theoretical Basis Engineering Society 107th Convention. 1999. Ootomo. 1987. Unraveling the Rakerd.. Organised Sound 12(3): 203–11. Organised Sound 12(3): 213–31. American May Or May Not Occur. 1977. D. 1996. J. Spatial Cognition. 1996. Germany. Toop. Localization of speaker Array. J. R.. Journal of the Audio Engineering Harrison.. Space Calculated in Seconds: The Philips 1999. T. The Decorrelation of Audio Signals and Taylor. Proceedings of the 2008 Phantom Sources. R. and Quesnel. Aydede (eds. Aural Landscape: Musical Space. Embodied and 2007. Properties of Hearing Related to Organised Sound 3(2): 135–40.).H. Princeton. M. Roffler.C. A. S. Spatial Quality Evaluation for Repro- Haas.633–54. Localization on Evanston.J. and Manligas. and Plenge. W. and Butler. S. Marc. Princeton University Press. Sound. Jonty. Metaphor and Gist. Ratliff. 1993. Journal of Auditory Research Journal of Psychology 57: 315–36. Philosophy in the Flesh. 1989. Representation the Everyday Mind. H.. Chicago. J. Quadraphonic Reproduction. 2008. 1986. Edgard Vare`se. Meaning. G. 1987.260–6. III. Preprint No. 1949. Acustica 1: 49–58. 2009. Richard. 4994. Audio Engineering Society 30th International Cambridge Handbook of Situated Cognition. 371–91. 2005. M. Acoustical Society of America 86(4): 1. H.. E. Journal of the Gayou. E. Proceedings in Rooms. Spatial Impression Effect in a Per. R. Localization of Lateral Circumstances. 2007. T. Sketches and Worksheets from 1952–1967. Woszczyk. IL: University of Chicago Press. The Precedence Effect. Cognitive Maps.M. and Koivuniemi. F.. C. Finland. Proceedings of the 2007 International Musicales and Their Implication in Musical Composi- Computer Music Conference. S. Northwestern University. B. Journal of the Convention. Kendall Emmerson. 1968a. 149–97. 2007.. Coding of Johnson. Colburn. 2009. 2009. Sculpture: Some Society 50(9): 651–66. 201–17.S. S. Conditions Under Which the Haas Precedence Effect The Precedence Effect in Sound Localization.L. and Hatano. 2010. Mason. Wallach. 1951. Audio Engineering Society 127th Sound in Rooms IV: The Franssen Effect. Tonal Stimuli in the Vertical Plane. S. Campari Sound Reproduction System. for GIS. Journal of Memory and Language 35: 19(4): 71–87. Scene Analysis and Auditory Technical Developments of the Groupe de Recherches Spatial Schemata. 2001. Kendall.B. In P. The GRM: Landmarks on a Historic Acoustical Society of America 43: 1.255–9. ence the Localization of Sound in the Vertical formance Situation: Multi-Channel Reproduction System Plane. 1. Berlin: Springer-Verlag. F. Technology and Musique Concrete: The tion: Spatial Attributes. Localization of Sound Perception of Spatial Sound Reproduction. and Rumsey. Saariselkä.B. and a Scene- Intelligibility of Speech. Queen’s University Belfast. C. Emlau. America 24(4): 196–200. Cambridge: Conference. Newman. and Rakerd. Susnik. 2007. Roffler. G. G. B. 1995.A. Amsterdam: Elsevier. Belfast: ICMA. L. K. IL. Journal of the of the AES 19th International Conference. R.S. Acoustics: Pro- of Meaning. Holman. Yost. Sonic Arts Research Centre.A. G. N. W. Audio (eds. B. Fire and Dangerous Things. Organised Sound 15(1): 63–74. On the Influence of a Single Echo on the duced Sound: Terminology. and Butler. and Hartmann.238 Gary S. of Space in Blind Persons: Vision as a Spatial Sense? Lakoff. Huang. 1999.. D. Masters report. BBC Research Depart- Etlinger. Frank and I. Stockhausen’s Electronic Works: Lakoff. Route. Corey. Kendall. Kendall. F. Spatial Audio for Large Venues. B. W. Women. Surround Sound Up and Running. R. sation Curves for a Regularly-Spaced Octagon Loud- Hartmann. Tversky. What is an Event? The Event Schema. ‘How’ and ‘Why’ of Sound Simon. IL: ceedings of the Annual Conference of the Australian University of Chicago Press. 1998. Organised Sound 12(1): 35–58. The Precedence Effect of Sound From Side Situated. and Tomazic. D. Onset and Duration Effects. Saji. B.). S. Perrott.R.Z. and Reason. K. Space. Rumsey. Computer Music Journal Descriptions. Factors that Influ- Gregorio. Space-Form and the Acousmatic Image.695–706. and Guzman. 1997. Acoustical Society 2005: 145–50. 2007. Meaning in Electroacoustic Music and Thinus-Blanc. 1999. 1987. Imagination. In A.A. Teruggi. Le Corbusier.A. 27: 59–72. 7915. The Direction.