You are on page 1of 13

Virtual Worlds: Audio and

Other Senses
VR Worlds: Output Overview
• Visual Displays:
– Visual depth cues
– Properties
– Kinds: monitor, projection, head-based, handheld
• Aural Displays:
– Aural cues
– Properties, including spatialization
– Kinds: headphones, speakers
• Haptic (touch) Displays:
– Properties
– Kinds
• Other Displays: vestibular, olfactory
Audio Displays: Human Perception
Outer ear collects sound as air vibrations,
converts them to mechanical vibrations in
the middle ear (vibrations of the tympanic
membrane produce vibrations in the 3
bones called the ossicles) which transmit
vibrations to the inner ear’s cochlea (filled
with fluid, motion of the fluid causes the
hairs of the basilar membrane to vibrate)
and then sent via nerve impulses to the
brain
Audio Displays: Sound
• Sound is the propagation of waves (pressure variations)
through a medium; humans hear sounds from 20 Hz
(cycles per second) to 20 kHz; low frequency are deeper
• Sound properties include amplitude, frequency (rate at
which the pressure varies) and phase (where they are in
time), spectral components which determine timbre (type
or quality)
• Sounds have a temporal component; there can be
masking
• Pitch and loudness are perceptions related to the sound
properties (frequency and amplitude); for example, the A
that orchestras tune to is 440 Hz
• Humans can do auditory scene analysis (grouping into
discrete objects) and auditory stream segregation
(isolating a sequence of sounds as one event) and
cocktail party effect
• Localization of sound: ability to pinpoint the source
Cues for Localization of Sound
• Interaural level difference (ILD): different
volumes reach each ear; better for high
frequencies where the head provides
interference
• Interaural time difference (ITD): same
sound reaches the ears at different times;
better for low frequencies (wavelength is
large relative to head size)
Called duplex theory; one problem is the
cone of confusion where ITD is the same;
head movement can help
Cues for Localization of Sound
(con’t)
• People with hearing loss in one ear can
localize so there are other factors
• Doppler effects: intensity increases means
getting closer; intensity decreases =
receding
• Reverberation/reflection
• Acoustic characteristics of speech
• Pinna filtering and head-related transfer
functions (HTRF)
Head-Related Transfer Functions: HRTF
• Pinna filtering (outer ear): distorts
incoming sounds; depends on frequencies
and position of sound
• Experiments done: microphones to
measure the changes that happen from
different locations (~30 years ago)
• Produced mathematical functions to
change the sound
• “Trick” the ear into localization
• Should be individualized but can be
general; not as good when sound is
behind or in low frequency range
Auralization: simulation of sounds
• Room or environment has acoustic properties
(RIR: room impulse response); uses wave-
based modeling and geometric modeling (sound
is traced)
• Acoustic environment and the listener together
are the BRIR (binaural room impulse response):
generally done by combining the RIR and the
HTRF, with the majority falling on the HTRF
• Generally use standard HTRF; doesn’t take into
account different size pinna, diff ways to
measure HTRF, perturbation of microphones,
changes in head position
• Another techniques is to have arrays of
microphones to reproduce sounds
Aural Presentation Properties
• Less computationally intensive than CG
• Number of channels: 1, 2 or more
• Sound stage: world referenced
(loudspeakers) or head-referenced
(headphones)
• Localization: HRFT, can simulate
interaural differences, reverb
• Masking
• Amplification
Aural Displays
• Headphones: immersive, cables, HRTF, tracking
needed
• Headphones can isolate listener but
cumbersome and only good for one person and
sounds originate “inside the head”
• Loudspeakers: environment, masking by
projection screens, world-referenced, good for
more people
• Loudspeakers: more difficult to get diff signals
for diff ears, amplitude panning to simulate ILD
(volume)
• Loudspeakers with wave field synthesis; large
number of closely-packed loudspeakers
Aural Logistic Properties
• Noise pollution
• User mobility because of cables
• Interference from tracking equipment
• Environment needs: reflections
• Combination with other displays (visual, haptic)
• Portability
• Throughput: number of people, time to change
headphones
• Encumbrance
• Safety
• Cost
Vestibular and Other Senses
• Vestibular sense: inner ear – to sense
equilibrium, acceleration, gravity
– Flight simulators, motion platforms, shaking,
low-gravity
– Issues with nausea, dizziness
• Olfactory: difficult to describe, individual
variance
• Taste?
Olfactory Sense
• Hardware to generate odors: inkjet,
solenoid valves, mass-flow controllers,
autosamplers- blending of odor
components
• Odor concentration
• Odor duration and strength: continuous air
flow, air pump, headsets with nose
interface
• Odor sensing an individual differences

You might also like