The Role of Rest Frames in Ve tion, Presen e and Motion

Si kness

by
Jerrold D. Prothero

A dissertation submitted in partial ful llment
of the requirements for the degree of

Do tor of Philosophy

University of Washington
1998

Approved by
(Chairperson of Supervisory Committee)
Program Authorized
to O er Degree
Date

In presenting this dissertation in partial ful llment of the requirements for the Do toral degree at the University of Washington, I agree that the Library shall make
its opies freely available for inspe tion. I further agree that extensive opying of
this dissertation is allowable only for s holarly purposes, onsistent with \fair use"
as pres ribed in the U.S. Copyright Law. Requests for opying or reprodu tion of
this dissertation may be referred to University Mi ro lms, 1490 Eisenhower Pla e,
P.O. Box 975, Ann Arbor, MI 48106, to whom the author has granted \the right
to reprodu e and sell (a) opies of the manus ript in mi roform and/or (b) printed
opies of the manus ript made from mi roform."

Signature
Date

University of Washington
Abstra t
The Role of Rest Frames in Ve tion, Presen e and Motion Si kness

by Jerrold D. Prothero
Chairperson of Supervisory Committee:

Professor Thomas A. Furness
Industrial Engineering

A framework is presented for omprehending partly parti ipants' spatial per eption in virtual environments. Spe i hypotheses derived from that framework in lude: simulator si kness should be redu ible through visual ba kground manipulations; and the sense of presen e, or of \being in" a virtual environment, should be
in reased by manipulations that fa ilitate per eption of a virtual s ene as a per eptual
rest frame. Experiments to assess the simulator si kness redu tion hypothesis demonstrated that ongruen e between the visual ba kground and inertial ues de reased
reported simulator si kness and per-exposure postural instability. Experiments to
assess the presen e hypothesis used two measures: self-reported presen e and visualinertial nulling. Results indi ated that a meaningful virtual s ene, as opposed to a
random one, in reased both reported presen e and the level of inertial motion required to over ome per eived self-motion eli ited by s ene motion. The simulator
si kness resear h implies that visual ba kground manipulations may be a means to
redu e the prevalent unwanted side-e e ts of simulators. The presen e resear h introdu es a pro edure, possibly based on brain-stem level neural pro essing, to measure
the salien e of virtual environments. Both lines of resear h are entral to developing
e e tive virtual interfa es whi h have the potential to in rease the human- omputer

bandwidth, and thus to partially address the information explosion.

TABLE OF CONTENTS
List of Figures

iv

List of Tables

v

Glossary

vii

Prefa e

xiv

Chapter 1:

Introdu tion

1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 2:

Literature Review

1
1

3

2.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

2.2 Introdu tion to Presen e . . . . . . . . . . . . . . . . . . . . . . . . .

3

2.3 Area I: Presen e Measures . . . . . . . . . . . . . . . . . . . . . . . .

5

2.4 Area II: Presen e Manipulations . . . . . . . . . . . . . . . . . . . . .

14

2.5 Area III: Motion Si kness . . . . . . . . . . . . . . . . . . . . . . . .

23

Chapter 3:

Introdu tion to the Resear h

26

3.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

26

3.2 The Rest Frame Constru t . . . . . . . . . . . . . . . . . . . . . . . .

26

3.3 Impli ations of the Rest Frame Constru t . . . . . . . . . . . . . . . .

28

3.4 Division of the Resear h . . . . . . . . . . . . . . . . . . . . . . . . .

37

3.5 Guide to the Experiments . . . . . . . . . . . . . . . . . . . . . . . .

38

3.6 Experimental Con gurations . . . . . . . . . . . . . . . . . . . . . . .

41

3.7 Equipment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

44

3.8 General Methods and Measures . . . . . . . . . . . . . . . . . . . . .

50

Chapter 4:

Area I: Presen e Measures

55

4.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

55

4.2 General Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

58

4.3 Experiment AIE1: Narrow Field-of-View (Reported Presen e) . . . .

67

4.4 Experiment AIE2: Meaningful/Random . . . . . . . . . . . . . . . .

73

Chapter 5:

Area II: Presen e Manipulations

84

5.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

84

5.2 Experiment AIIE1: Foreground O lusions and Reported Presen e I .

85

5.3 Experiment AIIE2: Foreground O lusions and Reported Presen e II

89

5.4 Inside-Out Displays and Foreground O lusions . . . . . . . . . . . .

92

Chapter 6:

Area III: Motion Si kness

6.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93
93

6.2 Experiment AIIIE1: Independent Visual Ba kground for Low-End Systems I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

94

6.3 Experiment AIIIE2: Independent Visual Ba kground for Low-End Systems II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.4 General Dis ussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

Chapter 7:

General Dis ussion

110

7.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
7.2 Area I: Presen e Measures . . . . . . . . . . . . . . . . . . . . . . . . 110
7.3 Area II: Presen e Manipulations . . . . . . . . . . . . . . . . . . . . . 114
ii

7.4 Area III: Motion Si kness . . . . . . . . . . . . . . . . . . . . . . . . 115
7.5 Sele ted Rest Frames and Cognition . . . . . . . . . . . . . . . . . . . 116

Chapter 8:

Future Resear h

119

8.1 Introdu tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
8.2 Area I: Presen e Measures . . . . . . . . . . . . . . . . . . . . . . . . 119
8.3 Area II: Presen e Manipulations . . . . . . . . . . . . . . . . . . . . . 122
8.4 Area III: Motion Si kness . . . . . . . . . . . . . . . . . . . . . . . . 123

Chapter 9:

Con lusion

124

9.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Bibliography

126

Appendix A: CogE1: The In uen e of Meaning on Presen e

139

Appendix B: Area I Pilot Studies: Visual-Inertial Nulling Presen e
Measures
141
B.1 Initial Pilot Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
B.2 Pilot Studies Related to Experiment AIE1 . . . . . . . . . . . . . . . 144

Appendix C: Area II Pilot Studies: Foreground O lusions

148

C.1 Foreground O lusions In rease Presen e . . . . . . . . . . . . . . . . 148
C.2 Inside-Out Display Pilot Studies . . . . . . . . . . . . . . . . . . . . . 149

Appendix D: Area III Pilot Studies: Motion Si kness

155

Appendix E: Foreground O lusions and Bino ular Rivalry

157

Appendix F: Simulator Si kness Terminology

162

iii

Appendix G: Simulator Si kness Questionnaire

iv

164

LIST OF FIGURES
2.1 Foreground O lusion . . . . . . . . . . . . . . . . . . . . . . . . . . .

15

2.2 Inside-Out and Outside-In Roll Representations . . . . . . . . . . . .

20

3.1 Presen e Measure Experimental Con guration . . . . . . . . . . . . .

42

3.2 Independent Visual Ba kground (IVB) . . . . . . . . . . . . . . . . .

43

3.3 Rotating Chair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47

C.1 Inside-Out Display Pit h Representations . . . . . . . . . . . . . . . . 154

v

LIST OF TABLES
3.1 Dissertation Experiments . . . . . . . . . . . . . . . . . . . . . . . . .

40

4.1 Experiment AIE1: Field-of-View (Reported Presen e) . . . . . . . . .

68

4.2 Parti ipant Overview for Experiments AIE1 . . . . . . . . . . . . . .

69

4.3 Reported Presen e Data for Experiment AIE1 . . . . . . . . . . . . .

71

4.4 Experiment AIE1 ANOVA Table for Reported Presen e Data . . . .

72

4.5 Experiment AIE2: Meaningful/Random (Nulling Measure) . . . . . .

75

4.6 Parti ipant Overview for Experiment AIE2 . . . . . . . . . . . . . . .

76

4.7 Nulling and Reported Presen e Data for Experiment AIE2 . . . . . .

78

4.8 Experiment AIE2 ANOVA Table for Cross-Over Data . . . . . . . . .

79

4.9 Experiment AIE2 ANOVA Table for Reported Presen e Data . . . .

80

4.10 Correlations Between EFT S ores and the Two Dependent Measures

80

5.1 Experiment AIIE1: Foreground O lusions and Reported Presen e I .

86

5.2 Parti ipant Overview for Experiment AIIE1 . . . . . . . . . . . . . .

87

5.3 Experiment AIIE2: Foreground O lusions and Reported Presen e II

90

5.4 Parti ipant Overview for Experiment AIIE2 . . . . . . . . . . . . . .

91

6.1 Experiment AIIIE1: Independent Visual Ba kground I (Low-End) . .

95

6.2 Parti ipant Overview for Experiment AIIIE1 . . . . . . . . . . . . . .

96

6.3 Data From Experiment AIIIE1 . . . . . . . . . . . . . . . . . . . . .

99

6.4 Experiment AIIIE2: Independent Visual Ba kground II (Low-End) . 102
6.5 Parti ipant Overview for Experiment AIIIE1 . . . . . . . . . . . . . . 103
vi

6.6 Data From Experiment AIIIE2 . . . . . . . . . . . . . . . . . . . . . 106
B.1 Cross-Over Data for Experiment AIP3 . . . . . . . . . . . . . . . . . 146
B.2 Cross-Over Data for Experiment AIP4 . . . . . . . . . . . . . . . . . 147
G.1 Computation of SSQ S ores . . . . . . . . . . . . . . . . . . . . . . . 165

vii

GLOSSARY
ANCHOR EFFECT:

The psy hologi al phenomenon whereby di erent starting points

yield di erent estimates, whi h are biased towards the initial values.
AREA I:

As used in this dissertation, the line of resear h whi h seeks to measure

the sense of presen e in a virtual environment in terms of the degree to whi h
the sele ted rest frame is in uen ed by the virtual environment.
AREA II:

As used in this dissertation, the line of resear h whi h seeks to manipu-

late the sense of presen e in a virtual environment by altering the sele ted rest
frame.
AREA III:

As used in this dissertation, the line of resear h whi h seeks to redu e

simulator si kness by redu ing the disagreement between the rest frames implied
by visual and vestibular ues.
ATAXIA:

La k of mus ular oordination. Used in this do ument to indi ate pos-

tural instability.
CI:

Content-of-interest.

CLASS A MEASURES:

In Brindley's de nition [15℄, psy hologi al measures whi h

an be expressed solely in terms of a omparison of sensory stimuli. Stated
di erently, measures in whi h the parti ipant's response is expressed in a way
whi h is ongruent to the input stimuli. For instan e, having a parti ipant
viii

adjust a light intensity until it mat hes the per eived brightness of another
light would be a Class A measure.
CLASS B MEASURES:

In Brindley's de nition [15℄, psy hologi al measures whi h

an not be expressed solely in terms of a omparison of sensory stimuli. Stated
di erently, measures whi h require the parti ipant to transform the input into a
di erent form before responding. For instan e, magnitude estimation of ve tion.
Magnitude estimation requires the parti ipant to evaluate an impression of selfmotion.
CONTENT-OF-INTEREST:

The usual s ene displayed by a virtual environment. See

\independent visual ba kground."
CROSS-OVER AMPLITUDE:

As used in this do ument, the inertial amplitude at

whi h parti ipants \ ross-over" between visual and inertial dominan e. That
is, between having the sense of self-motion determined by visual or ( on i ting)
inertial self-motion ues. See \visual-inertial nulling".
DEMAND CHARACTERISTICS:

A situation in whi h a observer's response is in u-

en ed more by the resear h setting than by the independent variable.
EFT:

Embedded Figures Test.

EMBEDDED FIGURES TEST:

A standardized measure of ognitive style and ana-

lyti al ability. Used in the resear h des ribed here as a measure of eld dependen y. (See Se tion 3.8.2.)
FIELD DEPENDENCE:

The degree to whi h an observer has diÆ ulty analyzing

part of an organized eld independently of the eld.
ix

FIELD-OF-VIEW:

The instantaneous angular extent of the virtual s ene within the

visual eld.
FOREGROUND OCCLUSION:

An obstru tion mounted in front of a display in su h

a way that only the display (and not its boundary) is visible through the obstru tion, and su h that the display is at a greater visual distan e than all other
available visual ues.
FOREGROUND OCCLUSION EFFECT:

A psy hologi al or performan e hange pro-

du ed by a foreground o lusion.
FOV:
HITL:

Field-of-view.
Human Interfa e Te hnology Laboratory at the University of Washington.

The site for the work des ribed in this dissertation.
HMD:

Head-mounted display.

HUMAN FACTORS:

The study of human apabilities and limitations as they per-

tain to the design of tools and environments.
IMMERSIVE:

A system whi h displays information to a large per entage of human

sensory hannels, and whi h additionally may a ept ommands from a wide
range of human motor output.
INDEPENDENT VISUAL BACKGROUND:

A visual s ene made to appear behind the

ontent-of-interest of a virtual environment, and ontrolled independently of the
ontent-of-interest. Used to redu e motion si kness.

x

INDUCED MOVEMENT:

The apparent motion of a stimulus aused by motion of

nearby stimuli.
INERTIAL DETECTION THRESHOLD:

The amplitude of inertial motion at whi h it

is just possible to dete t the movement.
INTERFACE SICKNESS:

The omponent of simulator si kness whi h results from

an imperfe t simulation, for instan e due to lag, poor inter-o ular adjust, poor
resolution, et . (This term is introdu ed by myself and Mark Draper. For a
dis ussion, see Appendix F.)
INTERFACE STUDIES:

The study of the information boundary between systems.

It en ompasses the intera tions between humans, software and hardware.
IVB:

Independent visual ba kground.

LUNING:

A bino ular rivalry e e t o urring in HMD's. If there is a partial over-

lap between the s enes displayed to the right and left eyes, the nasal edge of the
s reen for ea h eye may be per eived as bands in the ommon visual eld. The
right and left bands form a \moon-like" en losure, hen e the term \luning".
MAGNITUDE ESTIMATION:

Psy hophysi al s aling method in whi h the observer

assigns numbers a ording to the apparent magnitudes of the stimuli.
MOTION SICKNESS:

A pattern of symptoms in luding nausea, heada hes and dis-

orientation whi h result under some onditions from exposure to motion stimuli.
MULTI-MODAL NULLING:

The measurement of the e e t of stimuli to one sen-

sory modality in terms of per eptual hanges observed from a se ond sensory
modality. See \visual-inertial nulling", a spe i ase of multi-modal nulling.
xi

PRESENCE:

Informally, the sense of \being somewhere", usually used with re-

spe t to the sense of \being in" a omputer-generated spa e. This dissertation
suggests a de nition in terms of the rest frame onstru t. See \presen e hypothesis."
PRESENCE HYPOTHESIS:

The sense of presen e in an environment re e ts the

degree to whi h that environment in uen es the sele ted rest frame.
PROPRIOCEPTION:

The re eption of stimuli produ ed within the organism. Gen-

erally used in the ontext of re ognizing the lo ation of or for es on parts of the
body.
PSYCHOPHYSICS:

The study of the apability of an organism to dete t, quantify,

or identify a stimulus.
RANGE EFFECT:

The psy hologi al phenomenon whereby magnitude estimations

are strongly in uen ed by the range of stimuli to whi h parti ipants are exposed.
REFERENCE FRAME:

A oordinate system with respe t to whi h positions, orien-

tations and motions an be judged.
REST FRAME:

The parti ular referen e frame whi h a given observer takes to be

stationary.
REST FRAME CONSTRUCT:

The nervous system has a ess to many rest frames.

Under normal onditions, one of these is sele ted by the nervous system as the
omparator for spatial judgments. We all this the \sele ted rest frame". In
some ases, the nervous system is not able to sele t a single rest frame.
RFC:

Rest Frame Constru t.
xii

RT:

Rea tion-time. Used as a dependent measure in the inside-out display re-

sear h.
SEE-THROUGH:

(In onjun tion with HMD's.) A display in whi h a virtual im-

age is seen super-imposed on the real environment, for instan e by means of
proje ting the image on a half-silvered mirror.
A rest frame sele ted by the nervous system from envi-

SELECTED REST FRAME:

ronmental stimuli. See Rest Frame Constru t.
SHARPENED ROMBERG:

A stan e in whi h a person stands with heel-to-toe, arms

folded a ross the hest and hin up. It is intended to make balan e more diÆ ult,
and to therefore make more dete table any balan e problems.
SIMULATOR SICKNESS:

The generi feeling of si kness resulting from exposure

to a omputer-generated spa e. (The de nition of this term is moderately
ontroversial. For a dis ussion, see Appendix F.)
SIMULATOR SICKNESS QUESTIONNAIRE:

A standard questionnaire and s oring pro-

edure for re ording and analyzing reported simulator si kness symptoms [55℄.
SSQ:

Simulator Si kness Questionnaire.

SPATIAL PERCEPTION:

The per eption of the position, angular orientation and

motion both of the observer and of obje ts external to the observer.
STD:

Standard deviation.

TEXTURE MAP:

An image, either digitized or synthesized, whi h is mapped onto

a surfa e.
xiii

VECTION:

The visually-indu ed per eption of self-motion.

VISUAL-INERTIAL NULLING:

A pro edure whereby on i ting inertial and visual

ues are presented simultaneously. The observers' responses indi ates the extent
to whi h their per eption is determined by the visual or the inertial ues. By
varying inertial stimulus amplitude, the \point of subje tive equality" ( rossover between visual and inertial dominan e) an be determined. Consequently,
the e e tiveness of di erent visual stimuli an be s aled in terms of inertial
stimulus amplitude. (This te hnique is introdu ed as a presen e measure in
Chapter 4.)
VIRTUAL ENVIRONMENT:

A omputer-generated spa e whi h is: 1. immersive;

and 2. an indu e a sense of presen e.
VIRTUAL REALITY:
VIRTUAL IMAGE:

A synonym for virtual environment.

A image whi h appears to ome from a point in spa e where it

does not a tually originate.

xiv

PREFACE
How an he possibly be humble? He hasn't done anything yet.

| Albert Einstein
This dissertation was motivated by the need to manage omplex information.
The wealth of the industrial e onomies is in reasingly in the form of information,
rather than of physi al materials. In the United States, information servi es have
grown relative to the rest of the e onomy sin e 1860, with a sharp a eleration after
1940 [53℄. For a re ent a ount of the impli ations of this evolution for the workfor e,
see Hunt [51℄. A monograph surveying the underlying trends, and proje ting their
so ial and politi al impli ations, an be found in Prothero [79℄.
A similar evolution a e ts engineering. Engineering will in reasingly be limited
less by the properties of materials and more by the omplexity of the tasks undertaken.
This is be ause more powerful tools ontinually tilt the balan e of diÆ ulty further
from the me hani s of onstru tion and towards design and maintenan e. As an
indi ation of where engineering as a whole may be headed, onsider the dis ipline
most limited by omplexity (and least by physi al onstraints), software engineering.
The entrality of the omplexity problem to software engineering is well-understood
by its pra titioners. A ording to Gelernter, \If you're a software designer and you
an't master and subdue monumental omplexity, you're dead: your ma hines don't
work" [27℄ (p. 52). Similarly, Fredri k Brooks, in the 1995 update of his lassi
software engineering book The

Mythi al Man-Month

[26℄, states that \ omplexity is

the business we are in, and omplexity is what limits us." (p. 226; emphasis in the
original.)
xv

The frequent examples of omplexity-related software failures are often summarized in the Asso iation for Computing Ma hinery's

Risks Digest

[6℄. In the May

1997 issue, for instan e, we read of the 15-month delay in the opening of the \world's
most advan ed air-traÆ - ontrol entre" as a result of software bugs buried in 1.82
million lines of ode [58℄.
One parti ularly intri ate bran h of software engineering, ompiler onstru tion,
has evolved a mini-mythology to express the omplexity problem. Su essive editions
of \Compilers: Prin iples, Te hniques and Tools" [4℄ have featured a knight armed
with the tools of ompiler onstru tion doing battle with a \dragon of omplexity".
It is a reature whose hot breath will be ome in reasingly familiar to engineers of all
dis iplines as we progress into the information age. This dissertation is, in a sense, a
joust with the dragon.
It is possible to leave omplexity into two related (and overlapping) areas of
resear h. The rst deals with the manipulation of information inside of oherent
modules. This is in essen e the study of algorithms, the domain of omputer s ien e. The other is the ow of information a ross the boundary of modules: the
study of interfa es. As yet, there is no established a ademi setting for the study of
interfa es1 . It in orporates parts of various dis iplines, su h as: omputer s ien e;
edu ation; ele tri al engineering; ergonomi s/human fa tors; industrial engineering;
mathemati s; psy hology; and te hni al ommuni ation. While the general de nition
of an interfa e in ludes the information boundary between any two or more systems,
for instan e two software or hardware modules, this dissertation is on erned only
with the human- omputer interfa e.
Comparing algorithms to interfa es, I suggest that interfa es are in the long run
the harder problem. The reason is that omplexity is related to un ertainty. The
1 Note,

for instan e, that this dissertation was written for a Me hani al Engineering degree.

xvi

greatest un ertainty o urs where distin t systems ome together (thus losing internal
onsisten y):

i.e.

, at the interfa e. It follows that the design of e e tive interfa es is

entral to onfronting the omplexity problem in engineering.
It is unfortunate, therefore, that we do not know more about how to design interfa es. The development of an interfa e for a new problem is guided either by general
rules, whi h provide little pre ision for the problem at hand [92℄, or by usability testing [23℄, whi h has to be repeated for ea h new interfa e. At present, interfa e design
is generally a trial-and-error pro ess, with the onsequent osts to development time
and quality. As the tasks we need to perform be ome more omplex, both the requirements pla ed on interfa es, and the diÆ ulty of a hieving these requirements,
will in rease.
The (perhaps unattainable) ideal would be to have a mature eld of \interfa e
engineering", whi h would allow one to systemati ally onstru t interfa es with predi table properties, in terms of their ability to onvey information both dire tions
a ross the human- omputer boundary. To the extent that su h a bran h of engineering were a hievable, it would augment the urrent te hniques for building interfa es
based on general rules, experien e, intuition and usability testing.
Engineering (in our ase, the hypotheti al \interfa e engineering") does not stri tly
require, but does bene t from, an underlying s ienti theory whi h provides a basis
for predi ting performan e. S ien e in turn requires measures whi h an be used
to a urately assess the out ome of experiments. A goal of this dissertation, therefore, was to investigate measures related to the goodness of interfa es. (For a brief
survey of existing interfa e goodness measures, urrently in a rudimentary state of
development, see [78℄.)
This dissertation fo uses on \virtual interfa es", a somewhat ill-de ned term. I
hoose to de ne a \virtual interfa e" as one whi h has two properties: \sensory
xvii

immersion" (meaning that the interfa e makes use of a large per entage of the human
sensory bandwidth); and the ability to indu e a sense of \presen e", or of \being in",
an environment implied by the interfa e2 . There are at least three reasons why the
study of virtual interfa es is important. The rst reason is that the sensory immersion
whi h is hara teristi of virtual interfa es has the potential to in rease the human omputer bandwidth, an important point in view of the omplexity problem. The
se ond reason is that virtual interfa es will be ome in reasingly ommon as te hnology
advan es. The third reason is that psy hologi ally, virtual interfa es an be easier to
study than traditional interfa es, in that by \tri king" the per eptual system virtual
interfa es an evoke stronger psy hologi al responses.
The fundamental advantage of virtual interfa es is that, by making better use of
natural human sensory inputs, they have the potential to greatly in rease the human omputer ommuni ation bandwidth. The fundamental disadvantage of virtual interfa es is that (pre isely be ause they an make ompelling use of the human per eptual
system) virtual interfa es have the apability to ause unwanted side-e e ts. These
side-e e ts in lude simulator si kness and postural instability.
Two ru ial human fa tors problems limit the e e tive use of virtual interfa es.
The rst problem is the la k of robust measures for how good virtual interfa es are,
without whi h it is diÆ ult to build the knowledge needed for systemati and highquality interfa e engineering. The se ond problem is side-e e ts. It is suggested here
that useful light an be shed on both of these problems (and quite a few others) by a
2 The

use of the word \virtual" in \virtual interfa e" is related to the term \virtual images" in

opti s. A virtual image is one whi h appears to originate from a lo ation whi h is not physi ally
o upied by the sour e of the image. For instan e, an obje t seen in a mirror appears to be
behind the mirror, but is not [37℄. Similarly, a virtual interfa e (or, somewhat more generally,
a virtual environment) reates a strong impression that something \really" exists whi h in fa t
la ks a physi al instantiation.

xviii

single, on ise framework: the rest frame onstru t3 . The purpose of this dissertation
is to introdu e the rest frame onstru t and its appli ations.

3 In

prior publi ations, I have referred to the \rest frame onstru t" as the \rest frame hypothesis".

The hange to \rest frame onstru t" was made on the re ommendation of my do toral ommittee,
to emphasize that the intent is more to nd a onvenient summary of the existing literature on
spatial per eption than to form a testable hypothesis.

xix

ACKNOWLEDGMENTS
This dissertation required the support and guidan e of many people. I am
pleased to a knowledge their e orts. I hope that the nal result is worthy of
their involvement.
I would like to thank the members of my supervisory ommittee: Dr. Thomas
Furness ( hair), Dr. Linda Brubaker, Dr. Earl Hunt, Dr. Kailash Kapur,
Dr Gregory Miller and Dr. Donald Parker. Dr. Maxwell Wells informally
played a similar advisory role, as well as administering the resear h grant
under whi h most of this work was funded.
I owe additional thanks to Drs. Furness, Hunt, Parker and Wells for providing detailed omments on several drafts of this dissertation. The nal version
is greatly improved as a result of their involvement. Of ourse, I must take
personal responsibility for remaining aws.
My resear h was most in uen ed by Drs. Furness, Parker and Wells. Together, they provided a truly impressive expertise in per eptual psy hology
and the human fa tors of virtual environments. While the framework developed here is primarily of my own devising, it ertainly would not have arisen
without their insight and support. I owe a parti ular debt to Dr. Parker for
introdu ing me to the literature on per eption, for guidan e on the design of
the visual-inertial nulling experiments, and for helping to arrange for the loan
of a rotating hair from the National Aeronauti s and Spa e Administration
(NASA).
xx

While I was the lead investigator for all experiments reported here ex ept
Experiment CogE1 in Appendix A, many of these experiments were ondu ted
as ollaborations. In parti ular, Dr. Hunter Ho man played a very a tive role
in the reported presen e foreground o lusion experiments (AIIP1, AIIE1 and
AIIE2) and Mark Draper was similarly involved in the \low-end" simulator
si kness experiments (AIIIE1, AIIIP1 and AIIIE2). I learned a great deal
from both and enjoyed their ompany.
Joris Groen, a visiting student from Leiden University in the Netherlands,
was kind enough to administer Experiment AIIE2, whi h followed a doubleblind pro edure.
A short pilot study (AIIIP2) was ondu ted using a driving simulator at
Hughes Resear h Laboratories (HRL). This study was done in ollaboration
with Dr. Wells, and with the assistan e of HRL sta . I am parti ularly grateful
to Dr. Peter Tinker of HRL for te hni al assistan e, and to HRL for allowing
use of this fa ility.
Another pilot study was arried out at the University of Washington's
psy hology department, with the aid of Dr. Ho man, Jennifer M Lean, and
Dr. Geo rey Loftus. While the study is not reported in this do ument it did
help to guide later resear h. I thank them for their assistan e.
Te hni al support was of ourse ru ial to all of my dissertation resear h.
I am parti ularly grateful for the software development of Paul S hwartz.
Mr. S hwartz provided working software prototypes for many of the experiments des ribed in this dissertation, whi h I subsequently enhan ed as needed.
He showed a remarkable patien e, diligen e and good humor throughout.
The visual-inertial nulling experiments of Chapter 4 and Appendix B were
te hni ally hallenging, due both to the inherent diÆ ulties of the design and
xxi

due to repeated hardware failures. The ele troni s expertise of Robert Burstein
was often of great use. Others who ontributed to the hardware support of
these experiments in luded Chris Airola, Arthur Gonzales, Jaswant Jabal and
Herb Kramer.
I am grateful to Toni Emerson and her ex ellent assistants in the Human
Interfa e Te hnology Laboratory (HITL) library. Good information is essential
to resear h, and I feel very fortunate to have had a ess to a resear h library
devoted to virtual interfa es.
This resear h was ondu ted at the HITL. While most of the HITL sta
were not dire tly involved in this resear h, this dissertation would not have
been possible without them. Rather than single out individuals, I would like
to thank them all for their ne work.
While I am grateful for the assistan e of many people from outside of
the University of Washington, I would like to a knowledge two in parti ular.
One is Dr. John Jahnke, who visited from the University of Miami in Ohio.
Dr. Jahnke suggested having parti ipants indi ate the per eived inertial endpoints of their motion in the visual-inertial nulling experiments. This was a
great improvement over the te hnique I had previously tried for measuring visual or inertial dominan e. The se ond is Dr. Robert Patterson of Washington
State University, for dis ussions of bino ular rivalry (see Appendix E).
My appre iation also goes out to the numerous experimental parti ipants
who, with remarkable good humor, took part in a wide variety of odd experiments.
It is usual in dissertations to a knowledge the friends and family members
who one to varying degrees deserted during the Ph.D. pro ess, and who nevertheless provided their un agging support. I have ome to understand why
xxii

this tradition arose. Of the many to whom my thanks are due, I mention
in parti ular my an ee, Rita Solon; my parents, John and Joy e Prothero;
my brother, Je Prothero; my sister, Ja ky Je ery; and the members of my
immediate \ hess ir le", Philipp and Vera Frenkel, Drayton Harrison, Ken
Plesset and David Zi k.
The experiments des ribed in Chapter 4 made use of a rotating hair (Contraves Goerz Co. Dire t Drive Rate Table Series 800 and a Neurokineti s, In .
Motion Simulator Controller). This was gra iously loaned to Dr. Parker by
NASA.
The Division ProVision 100's used in Experiments AIIP1, AIIE1 and AIIE2
were a quired as the result of a grant from the U.S. West Foundation to a HITL
edu ational proje t.
This resear h was supported in part by the Air For e OÆ e of S ienti
Resear h (Grant F49620-93-1-0339; Dr. John Tangney, ontra t monitor) and
NASA Grant NAS 0-703.
Finally, I would like to thank the U.S. taxpayers who, whether or not they
would approve, have funded the following resear h.

xxiii

DEDICATION
To the memory of my grandparents, and to Rita.

xxiv

Chapter 1

INTRODUCTION
The more losely and pre isely one observes parti ulars, the sooner one
arrives at a per eption of the whole.
| J.W. Goethe

1.1 Overview
Virtual environments have the potential to in rease the human- omputer ommuni ation bandwidth by mimi king natural human intera tion through appropriate
stimulation of sensory hannels. However, virtual interfa es are onfronted with two
fundamental problems. The rst problem is the la k of robust measures for the quality
of interfa es, without whi h it is diÆ ult to build the knowledge needed for systemati and high-quality interfa e engineering. The se ond problem is that, be ause they
e e tively stimulate human per eptual systems, virtual interfa es have the apability to onfuse the nervous system with unpleasant onsequen es, in luding simulator
si kness.
It is suggested that a useful (if partial) measure for the quality of an interfa e is
the degree to whi h it an indu e presen e. \Presen e" refers to the sense of \being
in" a omputer-generated spa e. It is hara teristi of virtual interfa es that they
produ e the sense that one is not simply looking at an image, but is instead inside of
a spa e generated by the omputer. Manipulations whi h in rease the ease-of-use or

2
intuitiveness of an interfa e also tend to in rease the sense of presen e. Conversely, a
good measure for presen e should tell us something about the quality of the interfa e.
There are ertainly limitations to presen e as a measure for the quality of an
interfa e. It is a general and somewhat ambiguous measure: it does not repla e taskspe i performan e measures. And higher presen e is not always a good thing. For
instan e, if the task requires swit hing rapidly between two displays, high presen e in
either may impede a smooth transfer. Nevertheless, presen e is an important variable
to study in the sear h for interfa e goodness measures.
Both presen e and simulator si kness deal with omplex psy hologi al issues. It is
likely that a useful approa h to these problems will have to be based on fundamental
prin iples. The approa h des ribed in this dissertation is based on an emerging
understanding of multisensory spatial per eption. This dissertation summarizes a
good deal of the literature on spatial per eption through the \rest frame onstru t"
(RFC). The RFC suggests that spatial judgments are made with respe t to a \rest
frame" (de nition of what is stationary) whi h is not physi ally determined, but
whi h is arefully maintained by the nervous system.
The most important appli ations of the RFC are to the measurement of presen e
based on spatial per eption and to the redu tion of simulator si kness. However,
the RFC also predi ts that foreground o lusions should a e t presen e and spatial
per eption. Resear h on all of these topi s is des ribed.

Chapter 2

LITERATURE REVIEW
Books are lighthouses ere ted in the great sea of time.
| E.P. Whipple

2.1 Introdu tion
The resear h in this dissertation is broken into 3 areas: the measurement of presen e; e e ts of foreground o lusions; and simulator si kness. (In the formulation to
be introdu ed in Chapter 3, these three areas orrespond respe tively to the measurement, manipulation and malformation of sele ted rest frames.) This hapter
summarizes the relevant literature.
Se tion 2.2 introdu es the literature on presen e, and suggests why the study of
presen e may provide a useful (if partial) measure for the quality of an interfa e.
Se tion 2.3 surveys the literature on psy hologi al measures, in parti ular as they
relate to presen e. Se tion 2.4 reviews e e ts of foreground o lusions. Finally,
Se tion 2.5 introdu es the literature on motion si kness and simulator si kness.

2.2 Introdu tion to Presen e
Informally, \presen e" refers to the sense of \being somewhere": usually in the
sense of being in a omputer-generated or omputer-mediated environment. A possible re nement to this de nition, relating presen e to the spatial per eption literature,
will be introdu ed in Se tion 3.3.3. The urrent se tion reviews what is known about
presen e and suggests why it may be of importan e.

4
The view that presen e is of primary importan e to the study of virtual interfa es
is not universally held. Ellis points out that in reasing presen e in one part of a task
may de rease global performan e (by inhibiting smooth transitions between distin t
environments). He states that \the design of a teleoperation or virtual environment
system should generally fo us on the eÆ ient ommuni ation of ausal intera tion.
In this view the sense of presen e, that is of a tually being at the simulated or remote
workpla e, is an epiphenomen[on℄ of se ondary importan e for design." [24℄
Nevertheless, presen e seems worthy of spe ial attention for the following two
reasons. The rst reason is that presen e is a ommon property of virtual interfa es.
Hen e, any attempt to explain the psy hology of interfa es must explain presen e.
The se ond reason is that while presen e may not be dire tly related to the e e tiveness of an interfa e, it appears to re e t many things whi h are. Thus, measuring
presen e may be a onvenient means to assess the importan e of these fa tors. Examples of fa tors whi h resear h implies in rease the sense of presen e in lude many
whi h make an interfa e more natural to use (more intuitive). The literature overs,
for instan e: intera tivity and pi torial realism [108℄; update rate [7, 108℄; eld-ofview (FOV) [36, 82℄; head-tra king and stereos opi ues [39℄; spatialized sound [40℄;
the addition of ta tile ues [42℄; and the degree to whi h the ontent is engaging [75℄.
Resear h reported in this dissertation (Appendix A; and at greater length in [41℄)
suggests that presen e is related to the amount of meaning extra ted from a display,
with all other variables held onstant. Presen e in a virtual environment may also
improve transfer of training to the real world [16℄.
The degree to whi h presen e an measure the over-all e e tiveness of an interfa e
of ourse depends on the strength of the indire t link between presen e and e e tiveness. The strength of this link an only be established empiri ally, by examining ases
in whi h good measures for both presen e and over-all e e tiveness are available.
Be ause it appears losely related to the nature and e e tiveness of virtual interfa es, presen e is a primary fo us of this dissertation.

5

2.3 Area I: Presen e Measures
This se tion gives a ba kground on psy hologi al measures, parti ularly as they
relate to the measurement of presen e. It summarizes the literature relevant to the
resear h reported in Chapter 4.

2.3.1 Overview of Psy hologi al Measures
This se tion on eptually introdu es Class A and Class B measures, physiologi al
measures, threshold measures and multi-modal nulling. Appli ations to the measurement of presen e appear in subsequent se tions.
A measurement, by de nition, requires that one thing be ompared with another.
Spatial omparisons an be made with respe t to a standard unit of distan e, su h
as a meter sti k. What are the psy hologi al equivalents of the meter sti k?
An important property of a good measure is that it be very dire t: that is, that
the measure require few transformations of its input. This is be ause ea h transformation introdu es possible sour es of noise. For sensory experiments, Brindley [15℄
introdu ed a useful terminology for dis ussing how dire t a psy hologi al measure is.
The most dire t he termed \Class A" measurements, whi h an be expressed solely
in terms of a omparison of sensations. For instan e, a Class A measure for ability
to distinguish olors might involve asking an observer to indi ate whi h two of three
adja ent olored lights mat hed ea h other.
In many ases, Class A psy hologi al measures are not possible. For example
(to sti k with sensory experiments, for whi h Brindley developed his terminology)
one might be interested in measuring how observers per eive olors to be related
to ea h other. It turns out that humans are omfortable naming the olor spa e
with ombinations of four (but not three) olors. A ross ultures, the preferred
olors orrespond to \red, yellow, green and blue" [60, 52, 11℄. The olor naming
experiments whi h determined this preferen e (and whi h ontributed to the now

6
well-established opponent-pro ess theory) are not based on a Class A measure for
distan es between olors. The experiments require parti ipants to ompare a sensory
stimulus ( olor) to a mental model: the experiment an not be expressed solely
in terms of sensations. Brindley termed \Class B" all measures whi h an not be
expressed solely in terms of sensations1 .
Class B measures an be quite useful, as in the above example, but be ause
Class B measures must pass through observers' mental models they are less dire t
and more error-prone. Class B measures risk errors of interpretation and estimation.
Further, it is known that a great deal of ltering o urs as sensory data moves to
more abstra t levels of the nervous system. Consequently, Class A measures, whi h
do not require a ess to mental models, may be more sensitive than Class B measures.
Class A measures may be able to dete t trends whi h would only be ome apparent
with stronger stimuli using Class B measures. These issues are dis ussed more fully
below in referen e to presen e questionnaires (see Se tion 2.3.3).
The terms \obje tive" and \subje tive" are widely used in pla e of \Class A"
and \Class B". This notation is poor, however, as \obje tive" and \subje tive" have
too many meanings. For instan e, administering a questionnaire asking for presen e
ratings is a Class B measure, in that it requires a parti ipant to evaluate a mental
state. But one ould argue that the results are \obje tive", from the point-of-view
of the experimenter. If the experimenter did not in uen e the out ome of the experiment, and does not impose a bias on the analysis of the data, is not the out ome
\obje tive"?
Distin t from the Class A/Class B lassi ation are the physiologi al measures,
su h as heart-rate or postural hanges indu ed by sensory stimulus. See Se tion 2.3.4.
Also distin t from the Class A/Class B lassi ation are threshold and multi1 Another

way to view the Class A/Class B distin tion is that in the ase of Class A measures

the parti ipant makes a response whi h is dire tly ongruent to the input, whereas with Class B
measures a mental transformation is required.

7
modal nulling measures. Whereas Class A measures require an observer to ompare
two sensations, threshold and multi-modal nulling measures do not. For threshold
measures, one evaluates an observer's ability to barely dete t some stimulus under
di erent onditions. For multi-modal nulling, one evaluates the degree to whi h
parti ular stimuli an per eptually overwhelm on i ting stimuli from a di erent
sensory hannel. See Se tion 2.3.5. A spe i ase of multi-modal nulling is visualinertial nulling, as used in the presen e measure resear h of Chapter 4.

2.3.2 Class A Presen e Measures
It is sometimes suggested that presen e should be measured in terms of the degree
to whi h a real s ene is indistinguishable from a virtual s ene, or in terms of the
amount of noise needed to make a real s ene indistinguishable from a virtual s ene
[62, 97, 91, 94℄. These might be alled \realism" measures. Su h measures are
\Class A": they require only a omparison of sensory input, between the real and
virtual environments. However, I do not believe that they are satisfa tory. The
reason is that realism does not appear to be the key fa tor determining presen e. For
instan e, movies based on animation are no less engaging (or pro table) than those
based on realisti imagery. Paus h et al., in summarizing Disney Imagineering's
experien e with a virtual environment based on the movie \Aladdin", mention the
importan e of ontent (not simply realism) to the sense of presen e [75℄. Consistent
with this, Wel h et al. [108℄ have re ently published results in whi h pi torial realism
was less important to reported presen e than either intera tivity or short delays.

2.3.3 Class B Presen e Measures
The most ommon means to measure presen e is by asking observers to report
their sense of presen e on a numeri s ale, or to adjust a devi e (su h as a lever) to
indi ate their sense of presen e. This te hnique is known as \magnitude estimation".

8
It is an example of a \Class B" measure, sin e it requires the parti ipant to evaluate a
mental state to respond. Magnitude estimation has the advantage of simpli ity: this
is why it is so widely used (in luding for the resear h reported in this dissertation).
Unfortunately, magnitude estimation also has fundamental aws.
Numeri verbal magnitude estimation of presen e is likely to produ e results of
marginal validity. Following its introdu tion by Stevens [98℄, magnitude estimation
pro edures were widely used and produ ed many useful observations. However, numerous limitations soon be ame apparent [76, 25℄. One of these is the \range e e t"
[100℄: parti ipants' numeri al ratings are strongly in uen ed by the range of physi al
stimuli to whi h they are exposed. The possible in uen e of range e e ts an be ontrolled or evaluated for ases where the physi al stimulus dimension is well des ribed;

e.g., when relating physi al sound intensity to per eived loudness. Similar ontrol or
evaluation is not available when the domain for whi h verbal magnitude estimations
are being provided is not des ribed. Presen e is a produ t of the observer's ognitive
pro esses; physi al manipulations to appropriately manipulate per eived presen e
are only vaguely known. Consequently, we are unable to evaluate possible range
e e ts when performing magnitude estimation of presen e. (Similar limitations are
en ountered when using magnitude estimation of assessment of ognitively-mediated
per epts su h as pain and motion si kness.)
A se ond diÆ ulty with magnitude estimation is an hor e e ts [103℄, in whi h the
value observers assign to a given ondition may depend on the onditions to whi h it
is ompared.
The existen e of range and an hor e e ts in magnitude estimation has serious
onsequen es. These e e ts sharply limit our ability to draw valid on lusions from
omparisons between data gathered in separate experiments with di erent onditions.

9

2.3.4 Physiologi al Presen e Measures
Physiologi al measures for presen e are in prin iple very attra tive, as at least
some of them an be re orded fairly unobtrusively while the subje t is parti ipating
in the virtual environment. This would potentially allow for a real-time response to
the subje t's level of presen e. A list of possible physiologi al measures for presen e
is given by Bar eld and Weghorst [8℄ and in ludes posture, mus le tension and ardiovas ular and o ular responses to virtual events. Neurologi al measures might also
be onsidered.
Held and Durla h [38℄ suggest a measure for presen e based on the ability of an
environment to produ e a \startle response" to unexpe ted stimuli. More generally, to
what extent an a virtual environment produ e responses whi h imply that observers
interpret it as the real environment?
Unfortunately, there is urrently little eviden e that physiologi al measures orrelate well with presen e. Sheridan [93℄ (p. 209) mentions physiologi al measures,
stating that \It is natural to seek an obje tive measure or riterion that an be used
to say that telepresen e or virtual presen e have been a hieved. However telepresen e
(or virtual presen e) is a subje tive sensation, mu h like mental workload, and it is
a mental model | it is not so amenable to obje tive physiologi al de nition and
measurement."
Nevertheless, the use of postural measures have re eived attention. Ohmi [69℄
laims a good orrelation between the angle of body tilt and the subje tive impression
of a eleration. Hoshino et al. [43℄ used body sway to measure the presen e of
observers exposed to various frequen ies of a rolling stimulus displayed on an HMD;
on a omsumer 3D television; and on a large 3D display. Body sway was measured
in terms of head displa ements. They reported highest body sway in the ase of the
HMD with a .33 Hz stimulus2 .
2 In

general, one needs to be areful when omparing an HMD to a non-HMD with a body-sway

10

2.3.5 Threshold and Multi-Modal Nulling Measures
The measures des ribed in this se tion seek to evaluate spatial per eption as
dire tly as possible. They are similar in spirit to Brindley's \Class A" measures, but
need not involve a sensory omparison. Rather, the e e t of a stimulus is gauged in
terms of its ability to alter per eption in some way.
Many su h measures take advantage of the lose onne tion between visual and
inertial per eption. Per eption of self-orientation and self-motion is served by a multisensory system that re eives inputs from inertial re eptors, in luding the vestibular
apparatus and somati re eptors, as well as from the eyes [28, 77, 73, 72℄. Changes
in inertial re eptor response are usually interpreted as altered self-motion or selforientation. Appropriate manipulation of stimuli to inertial re eptors may result in
self-motion per eption, su h as those produ ed by pit hing the head forward while
rotating about an earth-verti al axis (i.e., \ ross oupling" { see Howard [44℄) or by
skin pressure ues from a \g seat" [65℄. Similarly, visual eld ow is usually asso iated
with self-motion. To the extent that the visual surround is interpreted as indi ating what is stationary, visual ow with respe t to an inertially stationary observer
may eli it per eived self-motion. The phenomenon of visually-indu ed per eived selfmotion (ve tion) has been examined in numerous experiments and is the basis for
many motion simulators (see Rolfe and Staples [88℄).
Multisensory integration of visual and inertial stimuli o urs in the vestibular
nu lei. For example, Waespe and Henn [105℄ demonstrated that vestibular nu leus
neurons may be ex ited by lo kwise inertial rotation. Similar ex itation from the
same neuron may be evoked by ounter- lo kwise rotation of the visual surround
measure. The HMD may, simply by altering the e e tive mass of the head, a e t body sway
(parti ularly head sway). This would need to be ontrolled for, for instan e by omparing body
sway with eyes losed with and without the HMD. It is not lear from the available abstra t
whether the resear hers did so.

11
(an optokineti drum). These multisensory neurons, whi h may underlie self-motion
per eption, apparently do not distinguish between inertial and visual motion stimuli. Further, be ause these neurons are in the brain stem, self-motion per eption
asso iated with their response may provide a mu h more robust measure of spatial
per eption than self-report.
The multisensory nature of the self-motion per eption system suggests pro edures
in whi h the visual salien e of a virtual environment is measured in terms of its ability
to overwhelm on i ting inertial per eption ues.

Ve tion Measures
Ve tion refers to the per eption of self-motion indu ed by visual stimuli. An
example of this sensation is that if one is seated in a train parked at the station while
the train next to it pulls out from the station, one may have a sustained sensation
that one is moving in the opposite dire tion. A possible lose relationship between
presen e and ve tion will be introdu ed in Se tion 3.3.3. In view of this relationship,
prior work on ve tion measures is of spe ial interest. This se tion borrows from [84℄.
The simplest types of ve tion are ir ular and linear. To indu e ir ular ve tion,
parti ipants may be seated in a hair surrounded by a ylinder (often painted with
stripes) whi h rotates around the parti ipant. Linear ve tion is typi ally indu ed by
a display in whi h obje ts seem to be approa hing or re eding.
There is a good dis ussion of ve tion measures in Carpenter-Smith et al. [17℄.
Most previous ve tion studies have been based on a magnitude estimation measure, in
whi h a parti ipant is requested to assign numbers or joysti k positions to per eptions.
The de ien ies of magnitude estimation were dis ussed above.
There is a small but interesting literature on erning other measures for ve tion.
This literature an be thought of as dividing into threshold [111, 12℄ and nulling [112,
48, 49, 50℄ studies. In threshold studies, one investigates how visual stimuli a e t the
minimally dete table magnitude of inertial motion (or onversely, how inertial motion

12
a e ts the onset of ve tion). Young et al. [111℄ examined the intera tion of visual
and inertial rotation ues, by pla ing parti ipants on a rotatable hair surrounded
by a stripe pattern rotating at onstant angular velo ity. Among their ndings were
higher thresholds for the dete tion of inertial a eleration when the inertial ues
on i t with the ve tion ues. Berthoz et al. [12℄ pla ed parti ipants in a art whi h
moved linearly and indu ed ve tion by providing moving images in the lateral eld.
Their report in ludes ve tion onset thresholds in the range of the threshold for visual
motion dete tion, indi ating the importan e of vision to the per eption of motion.
A disadvantage of threshold studies is possible varian e due to parti ipants adopting
di erent on den e riteria for reporting threshold.
In nulling studies, one set of stimuli are opposed by another and parti ipants are
asked to determine the point at whi h the two stimuli ounterbalan e ea h other.
Za harias and Young [112℄ set up a ir ular ve tion nulling experiment. Parti ipants
were asked to maintain a stationary position by adjusting their inertial rotation, in the
presen e of a rotating visual surround and inertial disturban e. Other resear h using
visual-inertial nulling is des ribed by Huang and Young: for yaw rotation [48℄; lateral
motion [49℄; and roll and pit h [50℄. These papers developed models for visual-inertial
intera tion.
Related but distin t is the resear h by Carpenter-Smith et al. [17℄. Prone subje ts
were translated along their head x-axis (fore-aft). In the presen e of various inertial
and visual surround onditions, parti ipants were asked to report their dire tion of
motion. By running many trials for ea h parti ipant in ea h ondition, a point of
subje tive equality (PSE), at whi h parti ipants would think themselves as at rest,
ould be determined mathemati ally for ea h ondition. Shifts in PSE as a result of
hanges in the visual surround were used to develop, for the rst time, a s ale for
linear ve tion. (This is not to say that a s ale for ve tion annot be developed using
the more traditional te hniques, only that it had not been previously done.)
The presen e measurement experiments des ribed in Chapter 4 are an extension

13
and re nement of the pro edure reported by Carpenter-Smith and his olleagues. An
attempt is also made to relate visually-indu ed self motion, indi ated by the visualinertial nulling pro edure, to self-reported presen e.

Threshold and Nulling Presen e Measures
Many measures for the \visual e e tiveness" of a virtual environment are possible
in whi h the ability of the visual ues to in uen e spatial per eption is evaluated.
One an, for instan e, measure the extent to whi h the per eption of the gravitational
verti al is in uen ed by on i ting visual verti al ues. For an explanation of why I
onsider these to be presen e measures, see the dis ussion of the \presen e hypothesis"
in Se tion 3.3.3.
Two examples involving the per eption of gravity are outlined below.
Hatada et al. [36℄ examined the ability of a display to indu e a hange in the
per eived verti al as a fun tion of horizontal FOV and s ene ontent. They report
very small tilts for FOV's less than 20{30Æ , and a saturation at about a 5Æ disturban e
to the per eived verti al when the display FOV rea hes 80{90Æ.
Nemire et al. [68℄ reported an experiment to measure \simulation delity" in
terms of the ability of a VE to indu e a hange in per eption of gravity-referen ed
eye level (GREL). A pit hed opti al array an bias a subje t's estimate of eye-level.
The authors report that a physi al array biases GREL more than an identi al virtual
array. However, the addition of longitudinal (into the distan e) lines to the virtual
array removed the performan e di eren e.
The resear h des ribed in Chapter 4 deliberately fa tored gravity out by restri ting
motion ues to the horizontal plane. The reason is that gravity is a very strong ue:
it is diÆ ult for visual ues to overwhelm it, parti ularly when the visual ues are
provided by a poor virtual environment. A more sensitive measure is possible when
the in uen e of gravity is ontrolled for.

14

2.4 Area II: Presen e Manipulations
It was mentioned at the top of this hapter that the three areas whi h it reviews
will be related in Chapter 3 to the measurement, manipulation and malformation of
sele ted rest frames. This se tion surveys the literature relevant to the manipulation
of the sele ted rest frame through foreground o lusions.
There are two aspe ts to this dis ussion. The rst is the e e t of foreground o lusions on presen e. The se ond is whether the e e t of foreground o lusions on the
sele ted rest frame an be measured through a spatial performan e task (spe i ally,
a roll- orre tion task asso iated with inside-out displays).
This se tion summarizes the literature relevant to the resear h reported in Chapter 5. It borrows in part from [83℄.

2.4.1 Foreground O lusions
A \foreground o lusion" an be thought of as a \peephole" through whi h to
view a display, possibly a peephole for ea h eye3 (see Figure 2.1). More formally, a
\foreground o lusion" is de ned here to be \an obstru tion mounted in front of a
display in su h a way that only the display (and not its boundary) is visible through
the obstru tion, and su h that the display is at a greater visual distan e than all
other available visual ues."
Be ause a foreground o lusion pla es the display at a greater distan e than other
visual ues, it en ourages the observer to interpret the display as the visual ba kground. The importan e of this is dis ussed below.
3 The

term \foreground o lusion", seems preferable to \peephole", be ause the latter has strong

mono ular onnotations.

15

Boundary of Visual Display

Visual Display

Boundary of Visual Display

Foreground Occlusion

Figure 2.1: Foreground O lusion
The foreground o lusion blo ks the view of the boundary of the display.
In general, foreground o lusions an be bino ular.

16

2.4.2 The Foreground O lusion E e t
This dissertation develops a somewhat new theoreti al framework for dis ussing
foreground o lusions, and emphasizes their appli ation to presen e and HMD design.
However, the existen e of a psy hologi al e e t asso iated with foreground o lusions
is not a new observation. The below introdu es the literature on foreground o lusions
from the elds of ve tion and the monos opi viewing of 2D, stati pi tures.
For those wishing to experien e the foreground o lusion e e t, a simple te hnique
is to hold up a ardboard box with a hole in it whi h just blo ks the edges of a VCR
monitor. For best results, play on the monitor a s ene with onsiderable movement4 .
Compare one's impressions looking through the hole with one eye to an unobstru ted
view of the monitor at the same distan e, again with only one eye.

Foreground O lusions and Ve tion
Traditionally, it was believed that a ne essary ondition for ve tion was the stimulation of peripheral vision through a wide FOV display. Andersen and Braunstein
[5℄ showed that \ entral ve tion" (ve tion as a result of stimulating only the entral
visual eld with angles as small as 7.5Æ ) was possible. A similar non-dependen e
of ve tion on peripheral vision was reported by Howard and He kmann [47℄. Other
resear h [14, 71, 70℄ has suggested that a riti al issue in determining ve tion is the
apparent relative motion between the self and the per eived visual ba kground.
Be ause ve tion is in uen ed by the visual ba kground, and be ause foreground
o lusions make a display appear to be the visual ba kground, one would expe t that
a foreground o lusion would in rease ve tion. Howard and He kmann [47℄ make
essentially this point. \The on guration in whi h we obtained good entre- onsistent
ve tion, namely a moving s ene seen at some distan e through a window in a large
stationary surround, is typi al of situations in whi h the world is seen through the
4I

re ommend the faster parts of the videotape Over Washington, possibly played on fast-forward.

17
window of a moving vehi le. The fa t that good ve tion may be obtained under these
ir umstan es is a point to be borne in mind by those who wish to avoid the high
ost of produ ing wide- eld displays to indu e onvin ing sensations of self-motion
in air raft simulators."
Similarly, Mergner and Be ker [67℄ reported that when parti ipants were exposed
to a 30Æ by 30Æ ve tion stimulus in entral vision, the parti ipants never reported
ve tion when the limited FOV was reated by masking (blanking) part of the s reen
(by putting a box with a small opening over the proje tion system). In ontrast,
when the same FOV restri tion was reated by a mask worn on spe ta les, the parti ipants did report ve tion. In the latter ase, the parti ipants felt the ve tion to
be qualitatively less ogent than with full- eld stimuli; however, their quantitative
estimates were only slightly redu ed.

Mono ular Foreground O lusions and Stati Pi tures
The above suggested that the e e t of foreground o lusions on ve tion may be
due to the ability of foreground o lusions to alter the per eived visual ba kground.
It appears that another fa tor, depth per eption, may also be involved in the e e t
of mono ular foreground o lusions on the per eption of stati , two-dimensional (2D)
pi tures. The literature on mono ular foreground o lusions for viewing stati , 2D
pi tures is reviewed by Rogers [87℄.
It has long been observed that the appearan e of three dimensions in a
pi ture is more striking under ertain viewing onditions, su h as viewing
the pi ture with one eye only and by using a peephole or a lens in a redu tion s reen or viewbox. A number of old viewing devi es make use of
one method or another to produ e their sometimes powerful e e ts. The
usual explanation is that the restri ted view enhan es the e e tiveness of
pi torial-depth information by redu ing the on i ting atness informa-

18
tion that spe i es a pi ture's obje tive surfa e...
Peephole viewing is mono ular and head motion is prevented, and this
should enhan e per eived depth as des ribed above. A peephole or a lens
also restri ts the observer's view of the pi ture itself, hiding the frame
and surrounding surfa es. Loss of the visible frame and dis ontinuous
surrounding surfa es redu e information for the pi ture as a at obje t
(perhaps even for the presen e of a surfa e at all), potentially enhan ing
the illusion of depth in the pi ture. This is the oft- ited reason for the
su ess of the various pi ture-viewing devi es.
The following is a personal observation, in keeping with Rogers' dis ussion of
the e e t of mono ular foreground o lusions on depth per eption in 2D pi tures.
When viewing Claude Monet's Boats at Argenteuil through a mono ular foreground
o lusion, the omponents of the image appear larger than when viewed mono ularly
from the same position without the foreground o lusion. A possible explanation
is that sizes are diÆ ult to judge mono ularly; that surrounding ues, for instan e
from the pi ture frame, would ordinarily a e t mono ular size per eption; and that
with these ues removed, the per eived size of the pi ture omponents in reases. The
overall s aling-up of the image omponents may stret h the per eived depth.
Two possible me hanisms for the e e t of foreground o lusions have been presented in the last two se tions. For ve tion, literature involving bino ular foreground
o lusions suggested that their e e t may be due to an alteration of the per eived
visual ba kground. For mono ular foreground o lusions applied to 2D, stati images, the alteration of per eived depth was also mentioned. One might ask: \is the
alteration of per eived depth a signi ant fa tor in the e e t of bino ular foreground
o lusions?"
I suggest that depth per eption is not a signi ant fa tor for bino ular foreground
o lusions. The reason is that strong stereos opi depth ues are present both with

19
and without the bino ular foreground o lusion. Consistent with this, I do not remember any of the parti ipants run in the experiments des ribed in Chapter 5 spontaneously mentioning that depth per eption was a e ted by a bino ular foreground
o lusion. Nor did I have that impression myself.

2.4.3 Inside-Out Displays
Inside-out displays o asionally produ e an interesting failure of spatial interpretation known as \ ontrol reversals". For reasons given in Chapter 3, it was thought
that foreground o lusions might redu e this problem, resulting in a performan e
measure for the foreground o lusion e e t. The literature on inside-out displays is
summarized below.
The two simplest ways to represent the roll of an air raft on a o kpit display
are either: to keep a representation of the ground stationary and roll an air raft
i on; or to keep the air raft i on stationary and roll the ground representation (see
Figure 2.2). The former is referred to as an outside-in display; the latter, as an insideout display. The terminology arises as follows. If one views a plane from above and
behind (\outside"), a roll is seen as the plane moving with respe t to the ground.
Hen e, a display in whi h the air raft i on moves is alled \outside-in". Conversely,
if one sees the roll of the plane from inside looking out, what one sees with respe t
to the plane is the ground moving up either the left or right window of the o kpit.
Hen e, a display in whi h the ground representation moves is alled \inside-out"5.
Despite the ongruen e between inside-out displays and the view out the window
of the o kpit, inside-out displays are prone to a misinterpretation known as a \ ontrol reversal". When a ontrol reversal o urs, the pilot inadvertently attempts to
5 There

is a re nement alled a \frequen y-separated" display whi h ombines the \outside-in"

and \inside-out" approa hes. There are data pointing to the superiority of frequen y-separated
displays; see, for instan e, [10℄. However, frequen y-separated displays are beyond the s ope of
the urrent dis ussion.

20

A

B

Figure 2.2: Inside-Out and Outside-In Roll Representations
A Inside-out; and B outside-in depi tions of a plane rolled down towards
the right.

21
ontrol the representation of the ground, rather than of the plane. For instan e, in
Figure 2.2 (A) we see a representation of a plane rolled with its right wing down. To
orre t for the roll, the pilot should bank the plane to the left. However, in a \ ontrol reversal" pilots will instead a t as if they should orre t for the roll by \pushing
down" on the ground representation (i.e., banking the plane to the right).
Of ourse, pilots are well-trained to interpret inside-out displays orre tly, and
ontrol reversals are rare. Nevertheless, Ros oe mentions that \it is possible { even
for airline pilots { to onfuse the moving horizon bar of the gyros opi attitude
indi ator and the xed airplane symbol when they nd themselves suddenly and
unexpe tedly in an unusual ight attitude." [89℄ Ros oe suggests that this ondition
may be impli ated in \graveyard spirals", whi h ause approximately 100 deaths
per year in the U.S. in general aviation, as well as o asional ommer ial disasters,
perhaps in luding the 1994 USAir 737 rash.
Given that the inside-out display is ongruent to the view out the window of the
o kpit, and that ontrol reversals do not o ur when looking out the window, why
do ontrol reversals o ur when using inside-out displays?
There is a good dis ussion of this issue by Ros oe et al. [90℄ (p. 70):
The problem of pilot errors on moving-horizon attitude displays may
be explained in the ontext of the psy hologi al phenomenon of gure
and ground. Although gure-ground de nitions emphasize stati aspe ts
of the visual eld, dynami aspe ts dominate the ight situation. Psy hologi ally, the part of the eld of view that appears to be stationary
is ustomarily alled the ba kground, and the obje t that is moving is
alled the gure. When the entire visual eld moves in relation to the observer's eye, as o urs with head movement, the observer usually per eives
orre tly that the ba kground is stationary.
The question then be omes: do the gure and ground relationships

22
between the air raft and the outside world hange when the pilot shifts
attention from the outside world to the attitude indi ator on the panel
inside the o kpit? If the pilot's frame of referen e hanges when a small,
abstra t instrument representation of the outside world is all that is available, as opposed to the outside world itself, this hange must involve a
shift in the gure-ground relationship. Spe i ally, the air raft's instrument panel or even the framed aperture of an individual display fa e
be omes the ba kground against whi h the display elements move.
Ros oe et al. [90℄ (p. 70) mention an interesting te hnique for removing ontrol
reversals, whi h was the subje t of Pilot Study AIIP2.
The highly resolved, dynami , literal image in full olor presented on
a display s reen by a proje tion-type ight peris ope onsistently yields
the same stable gure-ground relationships for all pilots regardless of display size. With display s reens subtending visual angles ranging from 30
degrees down to 7.5 degrees (a 2-in h s reen viewed from 15 in hes) and
presenting a forward looking view as narrow as 3.75 degrees (x2 magni ation), no ontrol reversal was observed during more than 135 hours of
ight experimentation involving more than 25 di erent pilots of widely
varying experien e.
Another method for redu ing the ontrol reversal problem is the \Mal olm
horizon" [64℄, in whi h a light representing the horizon is extended a ross the o kpit.
Aside from being easier to attend to than a small horizon display, this approa h makes
\use of the neural programming whi h naturally orients us with the horizon." The
authors dis uss the importan e of stimulating peripheral vision in order to a hieve
intuitive orientation with the horizon.

23

2.5 Area III: Motion Si kness
This se tion is on erned with simulator si kness, an unfortunate side-e e t of
virtual interfa es mentioned in Chapter 1. The literature summarized here is relevant
to the resear h reported in Chapter 6. This se tion borrows in part from [80℄.
Simulator si kness refers to the experien e of malaise arising from exposure to
omputer-generated or omputer-mediated environments. It is a major issue impeding the introdu tion of advan ed interfa es for both non-in lusive simulators as well
as in lusive virtual environments. Simulator si kness imposes limitations on the use
of these interfa es for training and entertainment appli ations as well as raising liability on erns. An extensive survey of the literature on simulator si kness is available
in [57℄. Also available are abstra ts from a re ent onferen e on motion si kness [3℄.
One an think of simulator si kness as having two omponents. The rst arises
from imperfe tions in the te hnology, su h as lag or geometri distortions. This
omponent might be termed \interfa e si kness" (see Appendix F for a dis ussion
of this terminology). Interfa e si kness is a serious problem, but one whi h is likely
to be ome less so as te hnology advan es. For an extended dis ussion, in luding a
te hnique for measuring the e e t of display imperfe tions on the vestibulo-o ular
re ex, see [22℄.
The se ond omponent of simulator si kness arises from the a urate presentation of stimuli whi h are inherently nauseogeni 6 . This omponent is alled \motion
si kness". Thus, a part of simulator si kness is believed to be losely related to other
types of motion si kness su h as sea si kness, ar si kness, and spa e si kness. The
relationship between various forms of motion si kness is dis ussed in [54℄.
Motion si kness refers to a pattern of symptoms in luding nausea, heada hes
6 For

instan e, the presentation of visual motion ues in the absen e of inertial self-motion ues

from the hair in whi h the observer is seated.

24
and disorientation7. A ording to the standard sensory rearrangement theory, \all
situations whi h provoke motion si kness are hara terized by a ondition of sensory
rearrangement in whi h the motion signals transmitted by the eyes, the vestibular
system and the non-vestibular proprio eptors are at varian e either with one another
or with what is expe ted from previous experien e" [29, 85, 86℄.
Crampton has restated the sensory rearrangement theory in a manner whi h
losely parallels an appli ation of the rest frame onstru t to be introdu ed in Se tion 3.3.5. A ording to Crampton [19℄ (p. 30):
Animals have an orientation onstan y su h that brain me hanisms
support a per eption of up and down, right and left, of the stable environment in whi h they maneuver. This per eption is a omplex one
that in ludes the distin tion between the animal moving its eyes or body
and the movement of obje ts in its spa e. The per eption is supported
by neural me hanism or memory that is ontinually updated or tested
by vestibular, visual, auditory, hemi al, kinestheti , and somatosensory
inputs that are often asso iated with motor a tivity su h as visual sear h,
limb movement, and lo omotion. This orientation onstan y may be extended to in lude geographi al features important to migration and navigation.
In the ourse of a spe ies' history, orientation onstan y is assembled
through sele tion over generations and then modi ed by ea h individual's
experien e until almost all ombinations of inputs are interpreted su h
that the animal an e e tively perform within its spa e environment to
7 It

is puzzling that vomiting would be a response to motion si kness. An evolutionary hypothesis

put forward by Treisman [102℄ suggests that food poisoning produ es abnormal per eptions, whi h
are therefore linked with vomiting. Motion si kness mimi s the abnormal per eptions due to food
poisoning, and therefore triggers the vomiting response.

25
sustain, defend and reprodu e its own kind. If a new sensory input is
so dis ordant with the orientation onstan y (a mismat h) that the performan e of the animal is degraded, ertain spe ies display symptoms of
lethargy, anorexia, salivation, and then vomit. In addition, man shows
fa ial pallor and reports nausea.
In a ordan e with sensory rearrangement theory, motion si kness is likely to
be an ever-in reasing problem as virtual interfa es be ome more ompelling. More
ompelling stimuli from the virtual interfa e ause the nervous system to give the
stimuli more weight, in reasing the si kness aused by any sensory on i ts.
Finally, while this dissertation is the rst full-s ale presentation of the rest frame
onstru t and its impli ations, its parts have been aired previously and have begun
to work their way into the literature. A group led by Deborah Harm at NASA has
applied the ideas from Chapter 3 to develop the rst su essful predi tor for spa e
si kness based on per eptual style [34, 33℄. Astronauts were divided into two groups:
those whose sele ted rest frame in weightlessness was determined more by the visual
s ene (VS), and those whose sele ted rest frame was determined more by their own
body axis (IZ). \It was found that orientation referen e type had a signi ant e e t,
resulting in an estimated 3-fold in rease in the expe ted motion si kness s ore on
ight day 1 for VS astronauts. Estimated probabilities of no symptoms ranged from
0.46 (Flight Day 1) to 0.79 (Flight Day 7) for IZ astronauts, and from 0.31 (Flight
Day 1) to 0.62 (Flight Day 7) for VS astronauts."

Chapter 3

INTRODUCTION TO THE RESEARCH
To say that the world is resting on the wheel of spa e or on the wheel of
wind is not the truth of the self or the truth of others. Su h a statement
is based on a small view. People speak this way be ause they think that it
must be impossible to exist without having a pla e on whi h to rest.
| Dogen, 13th Century [21℄

3.1 Introdu tion
The purpose of this hapter is to introdu e the rest frame onstru t (RFC); to
lassify its appli ations; and to brie y summarize and index the experiments des ribed in subsequent hapters. The dissertation is organized logi ally rather than
hronologi ally. The intent is to explain on isely rather than in a tual sequen e the
work ondu ted as part of this resear h. To streamline the presentation substantial
e ort is left undes ribed, due to a la k of de nitive results or be ause subsequent
experiments were more lear.

3.2 The Rest Frame Constru t
The RFC is a way of summarizing mu h of the literature on spatial per eption.
While the idea it aptures is not fundamentally new, formulating it in a parti ular
way is onvenient for urrently purposes. The RFC fairly naturally suggests lines of
resear h involving visual-inertial nulling presen e measures; te hniques for manipulating presen e; and te hniques for redu ing simulator side-e e ts.

27
The RFC derives from the observation that humans have a strong per eption
that ertain things are stationary. For instan e, we ordinarily per eive the Earth as
stationary. We typi ally interpret relative motions between the Earth's surfa e and
obje ts on or near it (su h as bi y les or air raft) as implying that the obje t in
question is moving, while the Earth remains stationary (\at rest").
From a mathemati al point-of-view, our strong per eption that ertain things are
stationary is quite strange. Given a relative motion between two entities, it makes as
mu h sense to interpret either (or neither) as stationary. Our nervous system ould,
in prin iple, hoose to agree with Coperni us that the Sun is stationary and provide
us with a omplex impression of our motion on the Earth with respe t to the Sun.
Or, in prin iple, if one moves one's hand ba k and forth in a room, one ould per eive
the hand as stationary and the room as moving in the opposite manner. Both of
these possibilities are ompletely legitimate, mathemati ally, but are also ompletely
foreign to our per eption.
The suggestion made here is that the nervous system sele ts ertain things as
being stationary in order to minimize its al ulations. For instan e, if one is primarily
on erned with navigating on the Earth, it is useful to assume the Earth's surfa e to
be stationary and to use it as the basis for spatial omparisons. Similarly, it is mu h
more eÆ ient to ompute the motion of one's hand with respe t to a room whi h is
assumed to be stationary than the onverse. (For similar reasons, mathemati ians and
physi ists are often on erned to hoose a spe i oordinate system whi h simpli es
a parti ular problem.)
The RFC simply formalizes this dis ussion. Borrowing from physi s, a oordinate
system used to de ne positions, angular orientations and motions is alled a \referen e
frame". The parti ular referen e frame whi h a given observer takes to be stationary
is alled the \rest frame" for that observer1 .
1 As

an example of a referen e frame whi h is not a rest frame, onsider a train whi h is per eived

as moving through a lands ape. One might noti e where some person is on the train. Sin e one

28
The \rest frame onstru t" states that:
The nervous system has a ess to many rest frames. Under normal onditions, one of these is sele ted by the nervous system as the omparator
for spatial judgments. We all this the \sele ted rest frame." In some
ases, the nervous system is not able to sele t a single rest frame.

3.3 Impli ations of the Rest Frame Constru t
The RFC has a number of dire t impli ations. These are listed in the below
subse tions.

3.3.1 The Importan e of the Visual Ba kground
If the sele ted rest frame is pi ked in su h a way as to minimize al ulations,
we would expe t it to be heavily in uen ed by what is per eived to be the visual
ba kground. The reason is that the visual ba kground generally de nes the largest
set of oherent spatial ues in the environment. Consequently, al ulations performed
by the nervous system may be simpli ed by assigning this set of ues to be the rest
frame, and thus as forming the omparator for spatial judgments.
One should note, however, that the primary issue is simplifying al ulations, not
an inherent need to make use of the visual ba kground. For ases in whi h one's task
involves the visual foreground one would expe t this foreground to determine the
sele ted rest frame. One example of this is that while the night stars are learly in
the visual ba kground, one usually per eives a hange in the position of the stars, not
the Earth. The reason, presumably, is that the task of on ern to the nervous system
is navigating with respe t to the surfa e of the Earth, not with respe t to the stars.
Hen e, it is mu h more pra ti al to take a rest frame aligned with the surfa e of the
is making a judgment with respe t to the train, whi h is per eived as moving, the train a ts as a
moving referen e frame.

29
Earth. The relative motion between the Earth and the stars is therefore interpreted as
a motion of the stars2 . Similarly, when he king the instrument panel of an air raft,
one will tend to per eive the panel as being at rest (de ning the sele ted rest frame).
It is only when one again looks out the window and swit hes to a task of navigating
visually with respe t to the Earth's surfa e that one again per eives the air raft (and
the panel) as moving.
The \indu ed motion" of the visual ba kground reported in Pilot Study AIIIP2
(Appendix D, involving a driving simulator) is also of interest in this regard. In
this ase, a on i t between the motion of the visual ba kground and of the visual
foreground was per eived as a motion of the ba kground, even though it was the
ba kground whi h was in agreement with the inertial ues. The driving-simulator task
under study required the parti ipant to navigate with respe t to the visual foreground.
Apparently, this was suÆ ient to result in the foreground being sele ted as the rest
frame, even in the presen e of on i ting rest frame ues from the ba kground and
from inertial ues.

3.3.2 Six Types of Spatial \Illusions"
The sele ted rest frame is hypothesized to underlie our per eption of position,
angular orientation and motion, both for self and for external obje ts. Crossing \self"
or \external obje t" with \position", \orientation" or \motion", we would expe t that
visual ba kground manipulations whi h alter the sele ted rest frame should produ e
6 types of \illusions"3.
2 Mike

Weissman, a HITL sta member, gives an extreme example of this. He reports that when

on a boat in the Pa i O ean, after a ouple of days when looking up at the night sky the swaying
of the boat was per eived as a swaying of the stars.
3 These

six onditions are termed \illusions" here be ause they are often so labelled, and for

la k of a better term in general ir ulation. If one a epts the physi s viewpoint that absolute
motion, position and angular orientation are meaningless, then it is in orre t to think of these six

30
For 5 of these 6 ases (the sixth being \self and position") it is well-known that
visual ba kground manipulations an produ e the indi ated illusion. The six possibilities are itemized below.
. The visually-indu ed per eption of self-motion is alled \ve -

Self and Motion

tion". For instan e, if one is seated in a stationary ar at a traÆ light, and the
adja ent ar rolls ba kwards, one may have a sustained impression that one is moving forwards. Re ent work implies that ve tion is heavily in uen ed by one's relative
motion with respe t to the per eived visual ba kground. See Se tion 2.4.1.
. If one moves the ba kground behind an obje t,

External Obje ts and Motion

one often has the impression that the obje t is moving in the opposite dire tion to
the ba kground. This is alled \indu ed motion" [106℄.
. If one is pla ed in a room in whi h the walls are tilted,

Self and Orientation

one tends to per eive that one is tilted in the opposite dire tion [45℄.
. A tilted ba kground an alter one's per-

External Obje ts and Orientation

eption of the orientation of an obje t in the foreground with respe t to gravity [45℄.
. It is suggested that presen e, the sense of \being somewhere",

Self and Position

en ompasses this kind of illusion. See the dis ussion of the \presen e hypothesis",
below.
. Visual ba kground manipulations may alter

External Obje ts and Position

the per eived distan e to obje ts, whi h is a form of position illusion. For instan e,
in the \ orridor illusion", linear perspe tive is used to alter the per eived depth (and
therefore size) of an obje t [60℄.
onditions as re e ting illusions. They are simply di erent per eptions. (Although a ase ould
be made for the \ orridor illusion" being a true illusion, sin e relative distan es

are physi ally

meaningful, at least for small velo ities.) I would prefer to des ribe these six onditions with the
terms \rest frame phenomena" or \rest frame shifts".

31

3.3.3 The Presen e Hypothesis and Presen e Measures
Given the suggested importan e of sele ted rest frames to per eption, an interesting question is whether we have any orresponding subje tive onstru ts. That is,
to what degree are we aware of the sele ted rest frame we are using? One possibility
is that the sense of presen e is related to the hoi e of rest frame. If one thinks of
presen e as the sense of \being somewhere", and the sense of \being somewhere" as
referring to one's sense of position, angular orientation and motion, then the possible
link between presen e and the sele ted rest frame be omes lear. This way of thinking
brings to mind the following \presen e hypothesis", whi h states that:
The sense of presen e in an environment re e ts the degree to whi h that
environment in uen es the sele ted rest frame.
Thus, in this formulation, the sense of presen e en ompasses the six spatial illusions dis ussed above.
One advantage of the presen e hypothesis is that it links the sense of presen e
squarely to something whi h appears to be quite fundamental to spatial per eption.
A se ond advantage is that it makes possible a wide range of presen e measures. If
the presen e hypothesis is true, then the degree of presen e in a virtual environment
an be measured through per eptual experiments in whi h the rest frame implied
by the virtual environment on i ts with the rest frame implied by the external
environment. The ease with whi h the virtual rest frame overwhelms the external
rest frame per eptually an then be used as a presen e measure. Su h a measure
might provide a more robust method than questionnaires for determining the best
ombinations of FOV, resolution, update rate, et . to engender presen e.
The presen e hypothesis also suggests why presen e may be related to su h issues
as the enjoyability of an interfa e, or the degree of meaning extra ted from it. To the
extent that a virtual interfa e is able to engage the attention, it will de ne the task

32
at hand and thus in uen e the sele ted rest frame4. The apture of the sele ted rest
frame is in turn eviden ed by the sense of presen e.
It must be emphasized, however, that the presen e hypothesis is an hypothesis:
the posited link between presen e (as a phenomenon observers are aware of and as
measured by self-reports) and measures based on rest frame on i t needs to be tested
experimentally. Experiments along these lines are reported in this dissertation.
The obje tion is sometimes made that the \presen e hypothesis" is too simple to
apture all of what we mean by presen e. This argument takes two forms. The rst
form is that presen e in a virtual environment is known to depend on a multitude
of fa tors, in luding, for instan e: the observer's state of mind going into the virtual
environment; the weight of the HMD (if there is one); the power of the omputer
supporting the virtual environment; the sensory modalities used; and the ontent of
the virtual environment. How ould something whi h depends sensitively on so many
things have a simple de nition?
A response is that it is quite ommon for many fa tors to in uen e something,
while that \something" is itself very simple. For instan e, the speed of an automobile
depends on a wide range of variables, su h as the mood of the driver; aerodynami s;
and the quality of the engine, tires and road. But the speed itself is entirely trivial,
and an be measured with a simple speedometer.
The se ond form of the \too simple" obje tion is that the presen e hypothesis
leaves ertain things out. The best example is perhaps the sense of presen e we feel
in a story when reading a novel. Is it laimed that this, too, has to do with sele ted
rest frames?
This depends how one wants to draw the lines. It would not greatly hurt the
basi argument, whi h is on erned with spatial per eption, to arve o text-based
presen e as something separate. I am in lined, however, to suggest that text-based
4 In

a ordan e with the argument given above that the sele tion of rest frame is driven by eÆ ien y

onsiderations linked to the urrent task.

33
presen e an be thought of in terms of sele ted rest frames, although the rest frame
implied by a novel is abstra t and does not lend itself to the kind of visual-inertial
nulling measures to be des ribed below.
In the end, the presen e hypothesis is not so mu h true or false as useful or useless.
Emphasizing sele ted rest frames suggests ertain kinds of resear h questions. It is
on the depth and out ome of these questions that the presen e hypothesis should
ultimately be judged.

3.3.4 The Importan e of Foreground O lusions
The importan e of the visual ba kground to determining the sele ted rest frame
was dis ussed above. In a ordan e with the presen e hypothesis, one would expe t
that the visual ba kground would also play a role in determining the sense of presen e.
It is known that in reasing the FOV of a display, everything else being equal,
in reases the sense of presen e. This is onsistent with the above argument: as
the FOV of a display is in reased, it absorbs more of the visual eld, whi h should
in rease the likelihood that the ontents of the display will be interpreted as de ning
the visual ba kground. If the ontents of the display are interpreted as de ning the
visual ba kground, they will in uen e the sele ted rest frame; this should in turn
in rease the sense of presen e.
The importan e of FOV is well-known to the virtual environments ommunity.
Less well-known is the importan e of a foreground o lusion (see Se tion 2.4.1). Be ause a foreground o lusion pla es the display at a greater distan e than all other
visual ues, it be omes more likely that the ontents of the display will be interpreted
as de ning the visual ba kground.
One might onsequently predi t that a foreground o lusion will in uen e the
sele ted rest frame and (therefore) in rease reported presen e. Two sequen es of
experiments on this question are reported in this dissertation. The rst sequen e
examined the e e t of foreground o lusions on reported presen e.

34
The se ond sequen e, adapted from the problem of ontrol reversals in inside-out
displays (see Se tion 2.4.3), sought a performan e measure related to the in uen e of
a foreground o lusion on the sele ted rest frame. Ros oe et al.'s \ gure and ground"
dis ussion of the ontrol reversal problem has a natural expression in terms of the
RFC. In order for an inside-out display to be interpreted orre tly, a di eren e in the
orientation of the ground representation and the air raft i on has to be interpreted
as a roll of the air raft i on, not of the ground representation. That is, the ground
representation has to de ne the sele ted rest frame. This happens automati ally
when looking out the window of the real air raft: the real ground de nes the visual
ba kground, and hen e the sele ted rest frame, and hen e a hange in the roll angle
between the real ground and the real air raft is interpreted as a roll of the real air raft.
The interpretation of a roll depi ted on an inside-out display is mu h less intuitive
than a roll seen out of a real o kpit window. The ground representation on the insideout display does not have a wide FOV and does not de ne the visual ba kground.
The sele ted rest frame tends to be determined by the o kpit as a whole. Sin e the
air raft i on on the inside-out display is xed with respe t to the o kpit, a hange
in the roll angle between the ground representation of the inside-out display and the
air raft i on is interpreted as a roll of the ground representation. In the absen e of
training or under stressful onditions, this an lead to ontrol reversals.
In view of this line of thinking, it is interesting to onsider the two remedies for
ontrol reversals dis ussed in Se tion 2.4.3. The use of a \highly resolved, dynami ,
literal image in full olor presented on a display s reen" is onsistent with fa tors
known to in rease the sense of presen e (see Se tion 2.2). The presen e hypothesis
suggests the following: the \literal image" in reases the sense of presen e, whi h
equivalently maps the sele ted rest frame into the natural referen e frame for the
display (the ground representation), whi h onsequently redu es ontrol reversals5 .
5 However,

an alternative explanation is possible, whi h is that the more detailed ues simply make

the ground representation easier to interpret, for low-level visual per eption reasons. The more

35
The se ond remedy dis ussed was the Mal olm horizon, whi h extends the ground
representation of the inside-out display a ross the o kpit. In part, the value of the
Mal olm horizon arises from providing a wide FOV stimulus whi h makes it easier to
dete t small angular motions. However, re ent resear h on narrow FOV ve tion (see
Se tion 2.4.1) shows that motion per eption depends heavily on what one per eives
to be the visual ba kground. It therefore seems plausible that the Mal olm horizon
also has a more indire t in uen e: expanding the FOV in reases the han e that the
ground representation will be interpreted as the visual ba kground, and hen e that
the ground representation will in uen e the sele ted rest frame.
More generally, one might expe t that any manipulation whi h in reases the sense
that the ground representation of the inside-out display is in the visual ba kground
would serve to redu e ontrol reversals. One su h manipulation is to provide a foreground o lusion. In view of the Mal olm horizon, and in view of the above dis ussion
of the parallel between in reasing the FOV and providing a foreground o lusion, one
might predi t that pla ing a foreground o lusion in front of an inside-out display
would serve to redu e ontrol reversals.
This idea is hardly of pra ti al value for real o kpits, sin e a foreground o lusion
whi h blo ked out other visual ues at the same or greater distan e as the inside-out
display (i.e., the rest of the o kpit) would not be useful. Whether a foreground
o lusion an in prin iple redu e ontrol reversals is an interesting question, however,
for at least two reasons.
1. It provides a possible performan e measure for the foreground o lusion e e t.
2. It is a test of the theory that the per eived visual ba kground is a determining
fa tor in the o urren e of ontrol reversals. This might be of relevan e to the
design of virtual o kpits presented on a display, in whi h the ground represendetailed ues may make it easier to dete t the angle between the plane and the ground.

36
tation ould be put at a greater distan e than the rest of the virtual o kpit
using stereos opi , a ommodative, or other depth ues.
Just as the frequen y of ontrol reversals is a potential measure for the foreground
o lusion e e t, it is also a potential measure for any other presen e manipulation.
This suggests that the e e t of a wide variety of display manipulations on presen e
ould be measured by evaluating the frequen y of ontrol reversals.

3.3.5 A Te hnique for Redu ing Simulator Side-E e ts
This se tion borrows in part from [80℄.
Given the hypothesized importan e of rest frames to spatial per eption, we would
expe t that a diÆ ulty in sele ting a onsistent rest frame would have serious onsequen es. The rest frame onstru t suggests that motion si kness does not arise from
on i ting motion signals per se6 , but rather from on i ting rest frames dedu ed
from those motion signals. That is, what is ru ial is not the full set of motion ues
in an environment, but rather how those motion ues are interpreted to in uen e
one's sense of what is and is not stationary. For instan e, if one is seated on a ben h
wat hing a o k of birds approa hing, one has on i ting motion signals (the birds
indi ate a relative motion, the inertial ues do not). However, one is very unlikely to
be ome motion si k be ause one's per eptual system is unlikely to interpret the o k
of birds as de ning the stationary rest frame, and, as su h, indi ating self-motion.
Other visual rest frame ues, from the ground or the sky, are more in uential than
the o k of birds.
From this point-of-view, the key to avoiding motion si kness is not to remove
all on i ting motion ues, but rather to remove those dis repan ies whi h indi ate
on i ting rest frames. It was argued above that the sele ted rest frame is heavily
in uen ed by the per eived visual ba kground. Therefore, a foreground/ba kground
6 As

suggested by the standard sensory rearrangement theory. See Se tion 2.5.

37
division of the visual eld ould be introdu ed as a means to test the appli ation of
the rest frame onstru t to motion si kness.
A moving visual ba kground (for instan e, a wide FOV simulator display) ombined with an absen e of inertial motion ues is likely to in rease motion si kness
symptoms. The rest frame indi ated by the visual ba kground indi ates self-motion,
whereas the inertial rest frame ues do not.
This suggests that providing a visual ba kground in agreement with inertial ues
may serve to redu e simulator si kness, even when the foreground ues are not in
agreement with inertial ues. The appli ation to simulator design is that providing
an \independent visual ba kground" (IVB) whi h appears behind the simulator's
usual visual \ ontent-of-interest" (CI) may provide a simple te hnique for redu ing
simulator si kness. The IVB an be made onsistent with the inertial rest frame even
if the CI (foreground) is not. Two experiments to test this predi tion are reported in
this dissertation7 .

3.4 Division of the Resear h
The rest frame onstru t and presen e hypothesis together suggest three losely
related areas of resear h. These have to do with the measurement, manipulation and
malformation of sele ted rest frames.
Area I.

Presen e re e ts sele ted rest frame de isions. This suggests that it is

possible to measure presen e by reating a on i t between real and virtual rest frame
ues and then evaluating the relative in uen e of the virtual ues on the sele ted rest
frame. Thus, a s ale for presen e an be onstru ted in terms of the ability of a
virtual environment to per eptually overwhelm on i ting real stimuli.
7 It

should be emphasized that the IVB te hnique an only redu e the omponent of simulator

si kness whi h is due to rest frame on i t (i.e., motion si kness). It an not redu e interfa e
si kness. See Se tion 2.5 and Appendix F for a dis ussion of this distin tion.

38

Area II.

What stimuli in uen e sele ted rest frame de isions? This leads to

the study of te hniques for manipulating the sele ted rest frame and the sense of
presen e in virtual environments. In parti ular, the use of foreground o lusions was
investigated.
Area III.

What are the onsequen es when the appropriate sele ted rest frame is

un lear? This produ es at least two interesting sub ases.

Area III A.

Visual ues

determine the sele ted rest frame, despite on i ting inertial ues. As dis ussed in
Se tion 3.3.2, this suggests a useful way of linking together presen e and six lasses
No fun tional sele ted rest frame is formed at all. As
a slight re nement to sensory rearrangement theory, it was suggested in Se tion 3.3.5

of visual illusions.

Area III B.

that this ondition is the underlying ause of motion si kness.
A typi al dissertation takes as its topi a single, often narrow, line of resear h
and pursues it to a de nitive on lusion. The present dissertation resear h was not
ondu ted in that fashion, as a matter of deliberate poli y. The reason is that the
usefulness of the RFC lies pre isely in its ability to show relationships, and to allow
borrowings, between distin t areas of resear h. The test of a unifying formulation
does not lie in one spe i line of resear h, but rather in its ability to onsistently
provide useful guidan e a ross its domain of appli ability. Hen e, as des ribed below,
this dissertation reports on separate lines of resear h in ea h of Areas I, II and III, as
well as an experiment on the in uen e of ognitive fa tors on presen e and preliminary
work on the appli ation of foreground o lusions to redu ing \luning" (an unfortunate
side-e e t of ertain bino ular displays).

3.5 Guide to the Experiments
Table 3.1 provides a taxonomy of the experiments reported in this dissertation
and where their des riptions an be found. All but two of these studies (CogE1 and
BRP1) addressed questions in Areas I, II, or III. The experiments are labelled with

39
a short index, a ording to the following s heme. The index begins with the area of
resear h (\AI", \AII" and \AIII", for Areas I, II and III; \Cog" for ognitive fa tors
on presen e; and \BR" for bino ular rivalry. The next letter is either \P" for pilot
studies (intended only for guidan e, rather than to a hieve de nitive results) or \E"
(for experiments whi h are intended to be fairly de nitive, at least for the parti ular
onditions under study). Des riptions of the experiments are usually found in the
main hapters; the pilot studies are reported in abbreviated form in the appendi es.
The nal part of the experiment index is a number, giving the logi al order of the
experiment within its area of resear h.
. Des ribed under this topi is an experiment showing

Importan e of Presen e

that presen e is a e ted by the meaningfulness of the stimuli, with all other variables
held onstant. Conversely, presen e may be a partial measure for the meaningfulness
of the stimuli. This strengthens the assertion of this dissertation that presen e is
worthy of serious study.
Area I: Presen e Measures

. Des ribed under this topi is a te hnique for

measuring presen e based on multi-modal nulling.
. Des ribed under this topi are experiments

Area II: Presen e Manipulations

on the manipulation of the sense of presen e, as applied to HMD's and inside-out
displays.
Area III: Motion Si kness

. Des ribed under this topi is a te hnique for

redu ing simulator side-e e ts using an independent visual display. (However, AIIIP2
found an interesting indu ed motion e e t of the independent visual ba kground
whi h may be useful as a presen e measure.)
Bino ular Rivalry.

Des ribed under this topi is a literature review and pilot

study on the appli ation of foreground o lusions to redu ing a bino ular rivalry
problem know as \luning". Luning is a signi ant problem for some HMD's whi h
partially overlap the s ene presented to the two eyes.

40

Table 3.1: Dissertation Experiments

Label

Title

Lo ation

Importan e of Presen e

CogE1

Cognitive In uen es on Presen e

Appendix A

Area I: Presen e Measures

AIP1

Wide Field-of-View

Appendix B

AIP2

Orientation

Appendix B

AIE1

Narrow Field-of-View (Reported Presen e)

Chapter 4

AIP3

Narrow Field-of-View (Nulling Measure I)

Appendix B

AIP4

Narrow Field-of-View (Nulling Measure II)

Appendix B

AIE2

Meaningful/Random

Chapter 4

Area II: Presen e Manipulations

AIIP1

Foreground O lusions and Field-of-View

Appendix C

AIIE1

Foreground O lusions and Reported Presen e I

Chapter 5

AIIE2

Foreground O lusions and Reported Presen e II

Chapter 5

AIIP2

Inside-Out Displays and Opti al Flow

Appendix C

AIIP3

Inside-Out Displays and Foreground O lusions I

Appendix C

AIIE3

Inside-Out Displays and Foreground O lusions II Appendix C
Area III: Motion Si kness

AIIIE1

Independent Visual Ba kground I (low-end)

Chapter 6

AIIIP1

Undisturbed Postural Stability

Appendix D

AIIIE2

Independent Visual Ba kground II (low-end)

Chapter 6

AIIIP2

Independent Visual Ba kground III (high-end)

Appendix D

Bino ular Rivalry

BRP1

Foreground O lusions and Bino ular Rivalry

Appendix E

41

3.6 Experimental Con gurations
This se tion gives logi al diagrams and brief des riptions of the experimental
on gurations used in this dissertation.

3.6.1 Area I: Presen e Measures
The experiments of Chapter 4 measured the relative e e t of di erent visual onditions on the per eption of self-motion. A visual-inertial nulling te hnique was used
in whi h the strength of the visual stimuli was measured by its ability to overwhelm
on i ting inertial motion ues. The te hnique is outlined in Figure 3.1.

3.6.2 Area II: Presen e Manipulations
The experiments of Chapter 5 measured the e e t of a foreground o lusion on
reported presen e and on ontrol reversals in an inside-out display. A foreground

o lusion is de ned in this dissertation as \an obstru tion mounted in front of a
display in su h a way that only the display (and not its boundary) is visible through
the obstru tion, and su h that the display is at a greater visual distan e than all
other available visual ues." This is diagramed in Figure 2.1 on page 15.
The experiments of Chapter 5 ompared onditions with and without a foreground
o lusion, keeping the FOV onstant a ross onditions.

3.6.3 Area III: Motion Si kness
The experiments of Chapter 6 measured the ability of a stationary visual ba kground to redu e simulator side-e e ts resulting from a moving visual foreground.
The onditions are outlined in Figure 3.2.

42

111 Virtual Scene
000
000
111
0001
111
0
0001
111
0

Visual
Inertial
Perceived

A

B
Motion
Direction

Visual
Inertial
Perceived

p
Time

?

?

C

D

Figure 3.1: Presen e Measure Experimental Con guration
Parti ipant is seated in a hair whi h os illates in yaw (i.e., in the
horizontal plane) while wearing an HMD. The HMD shows a s ene whi h
also os illates in yaw. B. In the real world, when one turns to the right the
visual eld ows to the left. The sense of self-motion is towards the right.
C. The visual motion in the HMD an be set to turn in the same dire tion
as the inertial motion. In this ase, the inertial ues make one think one is
moving to the right, whereas the visual ues make one think one is moving
to the left. The sense of self-motion depends on their relative strength.
D. Any visual-inertial phase angle is possible.
A.

43

1111111111
0000000000
0000000000
1111111111

Independent Visual Background

Scene

A

B

Figure 3.2: Independent Visual Ba kground (IVB)
In both onditions, a s ene rotating at 60Æ /se around the verti al axis
was shown in an HMD. The s ene was re orded on videotape by rotating
a amera on a tripod in the ounter- lo kwise dire tion as viewed from
above in a plaza on the University of Washington ampus. The IVB
ondition A provides a real ba kground visible through the half-silvered
mirror of the HMD whi h is onsistent with the la k of inertial self-motion
ues. The IVB was rated by parti ipants as being less visible than the
HMD s ene. The non-IVB ondition B pla ed an o lusion behind the
half-silvered mirror su h that only the rotating s ene was visible.

44

3.7 Equipment
Many of the experiments des ribed in subsequent hapters used the same equipment. This se tion summarizes their features.

3.7.1 Computer Software and Hardware Platforms
The presen e measure experiment of Chapter 4 made use of a standard PC and
\Virtual TV" (VTV) software from Warp, Ltd. [1℄ whi h allows a very large image to
be stored in memory. The VTV software \pre omput[es℄ the rendering of a s ene from
all possible view orientations. This permits unlimited pan/tilt/roll/zoom freedom at
high framerates on standard VGA ards." [107℄ Instead of dynami ally omputing
updates to the visual s ene, os illations were implemented more eÆ iently by indexing into di erent portions of the image. This allowed for a frame-rate of 60 Hz.
Unfortunately, it also meant that the s ene ould not be displayed in stereo: the same
indexed image was sent to both eyes of the HMD. To keep the frame-rate onsistently
at 60 Hz in all onditions, an \approximate dewarp" mode was used, whi h does not
pre isely simulate os illations around a xed enter-point. For the same reason, a
rather low resolution of 240x320 pixels was used.
The reported presen e/foreground o lusion resear h reported in Chapter 5 made
use of the Division ProVision 100. The ProVision 100 is a hardware/software platform
designed spe i ally for supporting virtual environments. It ombines an Intel 486
platform with dedi ated stereo graphi s, three-dimensional audio8 , and low-laten y
virtual world intera tivity in a single hassis. It an be a essed by standard UNIX
appli ations. It omes with dVS, Division's software environment, whi h provides a
distributed foundation for appli ations and a high level obje t-oriented programming
interfa e. It is used in onjun tion with the Division dVisor HMD des ribed below.
The environment used for the resear h des ribed in Chapter 5 was \SharkWorld".
8 The

audio apability was not used for the experiments reported in this dissertation.

45
\SharkWorld" was developed by Division, Ltd. and features a texture-mapped underwater s ene with a sunken ship and various moving sea reatures. The parti ipants
tried to at h sharks using a virtual net whi h followed real hand position.
The resear h involving ontrol reversals (see Appendix C) made use of images
generated by a Sili on Graphi s Reality Engine II and displayed on a Sili on Graphi s
20" monitor.

3.7.2 Driving Simulator
As des ribed in Appendix D, a 1-day pilot study was ondu ted using the Hughes
Resear h Laboratories driving simulator. For a full des ription of the simulator, see
[101℄. The simulator features the front half of a ar mounted before an LCD front
proje tion s reen with an FOV of 160Æ horizontal by 35Æ verti al.

3.7.3 Head-Mounted Displays
Three HMD's were used during the ourse of the dissertation experiments. The
below spe i ation are taken from [96℄. Spe i ations sometimes vary slightly depending on the sour e.
For the reported presen e experiments des ribed in Chapter 5, the Division dVisor
was used. The dVisor features 294x141 pixels, horizontal FOV 105Æ , verti al FOV
41Æ , and 46Æ stereo overlap. It weighs 8.8 pounds.
The presen e measure experiments des ribed in Chapter 4 used the Virtual Resear h VR4 HMD. The VR4 has the advantages over the dVisor of higher resolution;
of being mu h less heavy and bulky; and of being light-tight. Unfortunately, the VR4
has a lower FOV than the dVisor. The VR4 features 247x230 pixels, horizontal FOV
48Æ , verti al FOV 36Æ , and 100% stereo overlap. It weighs 33 oun es.
The experiments reported in Chapter 6 made use of the Virtual i-O i-glasses!
VTV/VPC. This HMD features 263x230 olor pixels and a 30Æ horizontal by 24Æ

46
verti al FOV with 100% overlap between the two eyes. It weighs 8 oun es. The image
is displayed on a semi-transparent half-silvered mirror. The half-silvered mirror has
the ru ial advantage (for the experiments reported in Chapter 6) of allowing both
a see-through and o luded mode. In the see-through mode, the image on the halfsilvered mirror is overlaid on the external environment. By mounting an o lusion
behind the mirror, the HMD an also be used in a mode in whi h only the display
itself is visible.

3.7.4 Rotating Chair
The experiments in Se tion 4 and the pilot studies in Appendix B involved os illating parti ipants in yaw (i.e., around the verti al axis) while providing an HMD
s ene whi h os illated with the same period but on i ting phase. This resear h made
use of a hair mounted on a Contraves Goerz Co. Dire t Drive Rate Table Series 800
ontrolled by a Neurokineti s, In . Motion Simulator Controller (see Figure 3.3).
Parti ipants were seated with their spine aligned with the gravitational verti al. The
hair moved in yaw (i.e., around the verti al axis).
The hair is tted with a 5-point harness. Sin e the experiments des ribed in this
dissertation were ondu ted at low amplitudes, only the lap belt was used.

3.7.5 Balan e Platform
For Experiment AIIIE2, reported in Chapter 6, a devi e was used to measure
postural stability before and after exposure to a ondition. This devi e was a Chatte x
Balan e System platform whi h permits the determination of the parti ipant's enterof-gravity based on signals from for e plates under ea h foot. Cal ulations were
based on the dispersion of the parti ipant's enter-of-gravity in entimeters in the two
horizontal dimensions over a 10 se ond period sampled at 100 Hz. Lower dispersion
values indi ate greater stability.

47

Figure 3.3: Rotating Chair
Rotating hair used in the visual-inertial nulling experiments. It allows for
os illations in the horizontal plane under program ontrol. This equipment was gra iously loaned to Dr. Parker by the National Aeronauti s
and Spa e Administration.

48

3.7.6 Foreground O lusions
Chapter 5 and Appendix C des ribes two sequen es of experiments: the rst involving reported presen e measures and the se ond a s reen-based spatial orientation
measure. For the rst sequen e, a very small foreground o lusion was needed whi h
ould t easily under an HMD. In the se ond, it was possible to mount a larger devi e on a tabletop in front of the monitor. Consequently, two ompletely di erent
foreground o lusions were used for the two sequen es of experiments, as dis ussed
below. In both ases, the foreground o lusion was learly per eived as being in front
of the monitor.

Reported Presen e Foreground O lusion Experiments
The sequen e of experiments des ribed in Se tion 5.2 made use of a pair of Lu as
Produ ts Corporation white \Super Sunnies" goggles, of the kind ommonly used in
tanning booths. These are worn dire tly over the eyes, and therefore t easily under
an HMD. The 1.27 m diameter entral ultra-violet prote tor in front of ea h eye
was removed: this had the e e t of providing lear vision in an FOV limited to a
entral ir le. As the goggles rest lose to the eyes, translation of the pupil during
eye rotation results in a di eren e between \dire t FOV" (the range one an foviate
on by turning the eyes but not the head) and \peripheral FOV" (the total range
one an see using peripheral vision while looking straight ahead without turning the
head)9 . We measured dire t FOV for the foreground o lusion at 40Æ , peripheral at
60Æ .
To avoid infe tion, in all experiments the goggles were washed with rubbing al ohol
between parti ipants.
To ompare the foreground o lusion ondition (i.e., the ondition in whi h the
9 The

terms \dire t" and \peripheral" FOV were invented for the purposes of this experiment. I

do not know of established terms for these ideas.

49
parti ipants wore the tanning goggles) to the same FOV without the foreground
o lusion, bla k onstru tion paper was pla ed over the HMD s reens with holes
orresponding to what was visible through the tanning goggles.

Inside-Out Display Foreground O lusion Experiments
The sequen e of experiments des ribed in Se tion C.2 explored the e e t of a foreground o lusion on an inside-out display roll- orre tion task. For these experiments,
the foreground o lusion was a bla k, mono ular iris mounted on a desk in front of
a 20" monitor. The iris ould be adjusted to narrow or widen the FOV. The iris
was mounted in su h a way that its height ould be easily altered to provide ea h
parti ipant with a omfortable seated view of the monitor through the foreground
o lusion. An adjustable padded hinrest was also provided. The iris and hinrest
were surrounded by bla k tarp whi h blo ked out any peripheral visual ues from the
laboratory. After hoosing whi h eye to use for the trials, parti ipants were tted
with an eyepat h whi h blo ked the other eye. This avoided bino ular rivalry distra tions without the need for squinting. Room lights were turned o so that illumination
ame almost entirely from the monitor.
The monitor displayed a ir ular s ene surrounded by a bla k annulus whi h overed the remainder of the s reen. In the foreground o lusion ondition, parti ipants
were asked to narrow the iris so that they ould just barely see a red ir le just inside
the annulus. Thus, the iris blo ked out all visual ues loser than the s ene. To provide mat hed trials without the foreground o lusion, parti ipants expanded the iris
to mat h a red ir le su h that the bla k annulus surrounding the s ene was learly
visible. In both onditions, the red ir le disappeared prior to the start of the trials.
The monitor was viewed from a distan e of about 20 m. The FOV of the s ene
was about 30Æ . The red ir le for the foreground o lusion ondition was set at about
27Æ . In the wider ondition, the FOV was about 45Æ .

50

3.8 General Methods and Measures
Many of the experiments des ribed in subsequent hapters used similar te hniques.
This se tion summarizes their ommon features. Spe ializations are presented as
needed for the individual experiments.
Every experiment was ondu ted with the prior approval of the University of
Washington's Human Subje ts Review Committee, and prior informed onsent was
obtained from all parti ipants. Ex ept were stated, data taken from the experimenters
were not used, and all parti ipants were unpaid volunteers not familiar with the
experimental hypotheses.

3.8.1 Presen e Questionnaire
For a dis ussion of presen e questionnaires in general, see Se tion 2.3.3.
With minor variations for s ene di eren es, the following questionnaire was used
in the foreground o lusion reported presen e experiments des ribed in Chapter 5 and
in Appendix C; and in the visual-inertial nulling experiments of Chapter 4 and in
Appendix B. Reported presen e was measured by taking the average of the responses
to the below questions.
1. In sharkworld, I felt like ... (1 = I was standing in the laboratory, wearing a
virtual reality helmet.) (7 = I was in some sort of o ean, near a shark-infested
shipwre k.)
2. How real did the virtual world seem to you? (1 = about as real as an imagined
world.) (7 = indistinguishable from the real world.)
3. To what extent were there times when you felt that the virtual world be ame
the "reality" for you, and you almost forgot about the real world outside? (1
= at no time.) (7 = almost all the time.)

51
4. Did the virtual world seem more like something you saw or somepla e you
visited? (1 = something I saw.) (7 = some pla e I visited.)
5. Did the virtual world seem more like a pi ture or more like a s ene looked at
through a window? (1 = like a pi ture.) (7 = like looking through a window.)
As the above questions produ ed similar patterns of responses, and sin e a shorter
questionnaire was desired whi h ould be onveniently used during exposure, rather
than as a post-test, the experiments des ribed in Chapter 4 relied on a version of the
rst question. This question appears to most readily apture the informal des ription of presen e as the sense of \being somepla e". (See the \presen e hypothesis",
Se tion 3.3.3.)

3.8.2 Embedded Figures Test
The experiment reported in Chapter 4 made use of an embedded gures test
(EFT) provided by Consulting Psy hologists [2℄. This is a standardized measure of
ognitive style and analyti al ability. The test requires nding simple forms whi h
are embedded in larger gures. The s ore is the average time in se onds to dete t
the simple forms. Thus, higher s ores re e t greater diÆ ulty in analyzing a part
separately from a wider pattern. (Or, viewed more positively, a greater tenden y to
per eive omplete patterns rather than their separate omponents.)
The EFT is losely asso iated with other per eptual measures whi h require the
parti ipant to analyze part of an organized eld independently of the whole [110℄.
These in lude tests of per eption of orientation to the verti al (the rod-and-frame
test and the body adjustment test); of ertain illusions and reversible perspe tive;
of similar auditory and ta tile disembedding tasks; and of problem-solving whi h
requires \disembedding" a part from its urrent environment.
The link between the EFT and \ eld dependen e" is parti ularly interesting, for
urrent purposes. Field dependen e measures the degree to whi h an observer has

52
diÆ ulty analyzing part of an organized eld independently of the eld. For instan e,
the \rod-and-frame" test requires one to sit in a totally darkened room and adjust
to the gravitational verti al a tilted luminous rod entered within a tilted luminous
frame, while the frame remains in its initial position of tilt. \Re e ting in ea h ase
the strong in uen e of the immediately surrounding eld upon the way in whi h one
of its parts is per eived, the person who takes very long to dis over the simple gure
in the omplex EFT design is also likely to tilt the rod far toward the tilted frame
and his own body far toward the tilted room." [110℄
The visual-inertial on i t set up by the resear h of Chapter 4 required parti ipants to extra t inertial motion ues from on i ting visual motion ues. This bears
a resemblan e to the rod-and-frame test. A question addressed in Chapter 4 is therefore whether between-subje t di eren es in the ease of visual-inertial disambiguation
is related to EFT s ores.
More generally, presen e re e ts the degree to whi h one is \pulled into" a (usually) visual environment, despite external ues. By analogy with the rod-and-frame
experiment, one might expe t between-subje t di eren es in any valid presen e measure to have at least some orrelation with EFT s ores.

3.8.3 Simulator Si kness Questionnaire
The simulator si kness questionnaire (SSQ) [55℄ is an extensively resear hed proto ol for measuring reported simulator si kness. It breaks simulator si kness into three
omponents: nausea (nausea, stoma h awareness, in reased salivation, burping); o u-

lomotor (eyestrain, diÆ ulty fo using, blurred vision, heada he) and disorientation
(dizziness, vertigo). The omponents an be ombined to give a total SSQ s ore. The
SSQ was used as a measure in the simulator si kness experiments of Chapter 6.
See Appendix G for the full spe i ation of the SSQ.

53

3.8.4 The PEST Convergen e Pro edure
The experiments des ribed in Chapter 4 and in Appendix B used su essive approximations to nd the inertial amplitude at whi h parti ipants rossed over between
inertial and visual dominan e (i.e., between having their per eption of self-motion determined by inertial and visual ues). The inertial amplitude adjustments after ea h
trial were made following the Parameter Estimation by Sequential Testing (PEST)
pro edure [99℄, whi h is a simple onvergen e algorithm developed for psy hophysi s
experiments.
The PEST pro edure obeys the following four rules:
1. On every reversal of step dire tion, halve the step size.
2. The se ond step in a given dire tion, if alled for, is the same size as the rst.
3. The fourth and subsequent steps in a given dire tion are ea h double their
prede essor (ex ept that... large steps may be disturbing to a human observer
and an upper limit on permissible step size may be needed).
4. Whether a third su essive step in a given dire tion is the same as or double the
se ond depends on the sequen e of steps leading to the most re ent reversal. If
the step immediately pre eding that reversal resulted from a doubling, then the
third step is not doubled, while if the step leading to the most re ent reversal
was not the result of a doubling, then this third step is double the se ond.
The PEST pro edure ends when the step size drops to a pre-determined amount.
The estimate is the level alled for by the last step10 .
10 A tually,

in Chapter 4 I hose to report the range between the last amplitude tested and the

next determined by the PEST pro edure.

54
In pra ti e, the fa t that ea h trial in the visual-inertial nulling experiments required a few minutes for ed a fairly rapid onvergen e, whi h in turn required a nal
step size suÆ ient to allow for a rapid onvergen e (this was taken to be an amplitude
of 5Æ /se in the experiment reported in Chapter 4, or 3Æ /se if the two onditions run
in parallel would otherwise be tied). Hen e, the subtleties of rules 3 and 4, above,
rarely ame into play.

Chapter 4
AREA I: PRESENCE MEASURES

Door meten tot weten. (To knowledge by measurement.)

| Kammerlingh, Dut h low temperature physi ist
4.1

Introdu tion

This hapter addresses the \presen e hypothesis" from Se tion 3.3.3 and its appli ation to the measurement of presen e. The \presen e hypothesis" laims that the
degree to whi h the rest frame implied by a virtual environment per eptually overwhelms a on i ting rest frame implied by the real (external) environment should
onstitute a presen e measure1 .
The following questions are addressed in this hapter.
1. How should a rest frame on i t measure be implemented?
2. Is the presen e hypothesis orre t? That is, does a measure based on real-virtual
rest frame on i t evaluate our subje tive sense of presen e?
3. What is the test-retest orrelation of a rest frame on i t measure2 ?
1 See

Se tion 2.3 for an introdu tion to psy hologi al measures in general, as well as their previous

appli ation to presen e and ve tion. See Se tion 3.7 for a des ription of the equipment.
2 Test-retest

orrelation evaluates the self- onsisten y of a measure. A high test-retest orrelation

is ru ial to a good measure, be ause a measure an be no more a urate than it is onsistent.

56
4. How is the rest frame on i t measure related to eld dependen y? Presen e
and eld dependen y both measure the degree to whi h one is drawn into a
visual s ene. It might therefore be expe ted of a good presen e measure that
between-subje t variation would orrelate with between-subje t variation in
eld dependen y3 .
In prin iple, many real-virtual rest frame on i t measures are possible. See
Se tion 3.3.2 for a lassi ation of spatial \illusions" whi h an be indu ed by visual
stimuli. Any of these an be harnessed to reate a rest frame on i t measure, by
implying a rest frame with the virtual ues whi h on i ts with the rest frame implied
by the real ues. The rest frame on i t measure then re ords the degree to whi h
the observer's spatial per eption is dominated by the virtual, rather than real, ues.
A straight-forward rest frame on i t measure an be based on altering the per eption of the verti al. Visual ues from a display may be set to indi ate the verti al
to be di erent from gravity. Measuring the per eption of the verti al, for instan e
by having the observer adjust a rod to indi ate the per eived verti al4, measures the
degree to whi h the visual rest frame has overwhelmed the inertial rest frame.
While rest frame on i t measures based on per eived orientation are quite possible, they fa e the diÆ ulty that the inertial per eption of gravity is very strong. This
tends to make the in uen e of the visual s ene diÆ ult to measure, parti ularly for
virtual environments whi h are not very ompelling. The resear h reported in this
hapter therefore fo used on a measure in whi h gravity is not a fa tor: on i ting
inertial and visual self-motion ues in the horizontal plane. Given on i ting inertial
and visual self-motion ues, one's per eption of self-motion may be determined by
3 See

Se tion 3.8.2 for a further dis ussion of these issues.

4 See

[45℄ for a dis ussion of the per eption of the verti al, Se tion 2.3.2 for prior resear h on

presen e measures of this type, and Se tion 3.8.2 for a des ription of the rod-and-frame test and
its relationship to other psy hologi al measures.

57
the inertial ues, by the visual ues, or by some ombination. As the inertial amplitude is lowered, it be omes in reasingly likely that the visual ues will determine the
per eption of self-motion.
The approa h was to develop a \visual-inertial nulling measure" for nding the
inertial amplitude at whi h parti ipants ross-over between inertial and visual dominan e of the per eption of self-motion. The hypothesis was that a visual ondition
whi h was more \presen e indu ing" would require stronger inertial ues to mat h the
visual ues per eptually, and that this would be re e ted in a higher visual-inertial
ross-over amplitude.
The te hnique developed here is intended primarily as a proof-of-prin iple: to
establish a rest frame on i t measure whi h fa tors out gravity, and to examine
its relationship to a reported presen e measure and the test-retest orrelation of
both measures. The measure used in this hapter is not a nal answer to the need
for sensitive and onvenient presen e measures. In parti ular, it is umbersome (it
requires the full attention of the parti ipant to perform the measurement task) and is
not suited to intera tive virtual environments (sin e the time- ourse of both the visual
and inertial motion ues have to be tightly ontrolled by the experimenter). Pilot
Study AIIIP2 (see Appendix D) raised the possibility of a more gra eful rest frame
on i t measure, based on the indu ed motion of a ba kground grid. See Chapter 8
for a dis ussion of this possibility.
Two experiments are des ribed in this hapter. The rst experiment, AIE1, examined three FOV's: 16Æ, 32Æ and 48Æ FOV5 . It was predi ted that wider FOV would
result in higher presen e on both a self-report and visual-inertial nulling measure. The
se ond experiment, AIE2, fo used on a meaningful/random visual s ene manipulation. The random ondition was reated by randomizing the pixels in the meaningful
s ene. The meaningful/random manipulation appeared more likely to result in a lear
5 48Æ

was the widest FOV available for the Virtual Resear h VR4 HMD used in these studies.

58
main e e t on both measures than the FOV manipulation des ribed above. The main
e e t was predi ted to indi ate higher presen e on both measures in the meaningful
ondition.
The meaningful/random manipulation reported here is similar to the meaningful/meaningless manipulation des ribed in Appendix A. The meaningful/meaningless
manipulation ompared meaningful with mat hed meaningless hess positions. This
found a di eren e in reported presen e on a purely ognitive variable, in favor of
higher presen e for the meaningful ondition.
The meaningful/random manipulation des ribed in Experiment AIE2 is not a
purely ognitive manipulation. In addition to a e ting level of meaning, it also affe ts low-level per eptual fa tors su h as number and distribution of edges. Thus,
while the results of Experiment AIE2 ould be taken as supporting the on lusion of
Appendix A that meaning a e ts presen e, it does not do so unambiguously.
4.2

General Methods

See Se tion 3.7 for a des ription of the equipment.
In all resear h des ribed in this hapter, parti ipants were seated upright in a
hair whi h os illated sinusoidally in yaw (i.e., in the horizontal plane). A fan was
mounted on the ba k of the hair to mask any vibratory or auditory noise it might
produ e. Parti ipants wore a Virtual Resear h VR4 HMD whi h displayed a (nonstereographi ) s ene whi h also os illated sinusoidally in yaw. The HMD blo ked o
all visual ues from the laboratory. The image motion was ontrolled by Warp Ltd.'s
\Virtual TV" software, as des ribed in Se tion 3.7. Both inertial ( hair) and visual
(HMD) os illations were always at .1 Hz. However, the phase angle between the
inertial and visual motion was varied.
The primary image used was taken from Maui, and onsisted of tropi al vegetation
overlooking water. In separate experiments, this image was manipulated either by

59
varying the FOV or by randomizing the lo ation of ea h pixel in the image.
Reported simulator si kness data was gathered before and after every session,
using the pre- and post-exposure SSQ. These data were used only as an informal
diagnosti to examine whether the experiment was ausing malaise and needed to be
re-designed. Si kness did not appear to a problem, as long as exposures in ea h trial
were kept to one or two minutes and separated by a brief break while onditions were
altered. During the break, parti ipants were requested to lose their eyes and relax,
and the hair was brought smoothly to rest.
4.2.1

Overview of the Visual-Inertial Nulling Measure

This se tion introdu es the rest frame on i t measure studied below. A s ale for
the visual e e t of a stimulus is de ned in terms of its ability to in uen e the inertial
per eption of motion. This \visual-inertial nulling" te hnique draws its strength
from the lose onne tion between the visual and inertial motion per eption systems
in humans. Be ause these two systems are so losely related, it may be possible to
measure subtle hanges in the input to one system in terms of a hange in per eption
arising from the other. See Se tion 2.3 for further details.
Des ription of Visual-Inertial Phase Angles

The resear h des ribed below manipulates the phase angle between the visual and
inertial motions in order to produ e on i ting self-motion ues. A onfusing point
about visual-inertial phase angles is that, ordinarily, visual and inertial motion ues
are 180Æ out of phase (see Figure 3.1, on page 42). For instan e, when one turns to
the right, the visual eld ows to the left. More expli itly, if one stares perpendi ular
to one's shoulders while turning to the right, after having turned, a point whi h one
was initially observing will have moved to one's left.
Thus, the \ordinary" situation is for inertial and visual stimuli to be 180Æ out

60
of phase. The \most ontradi tory" ondition o urs when the inertial and visual
stimuli are exa tly in phase. For instan e, if the visual ba kground moves to one's
right, one tends to feel that one is moving to the left. If the inertial motion is also to
the right, then one has ompletely on i ting motion ues.
It is onfusing to use the term \in phase" to des ribe inertial and visual stimuli
whi h are ompletely on i ting in terms of their e e t on self-motion per eption.
To avoid this problem, the phase angles in this hapter are given between the ve tion
and inertial urves rather than between the visual and inertial urves. That is, a
phase angle of zero implies a onsistent sense of self-motion from the two kinds of
stimuli. The ve tion urve di ers from the visual urve by 180Æ. A phase angle of
+90Æ is de ned here to imply that one's inertial sense of self-motion is 90Æ ahead of
one's visual sense of self-motion. Conversely, a phase angle of 90Æ implies that one's
inertial sense of self-motion is 90Æ behind one's visual sense of self-motion.
The word \ onsistent" is used to indi ate a ve tion-inertial phase angle of zero,
and \ on i ting" to indi ate all other ve tion-inertial phase angles.
Visual and Vestibular Motion Dete tion

The visual-inertial nulling measure involves the intera tion between visual and inertial motion ues. An important inertial motion dete tor for humans is the vestibular
system in the inner ear6. This se tion summarizes the relationship between visual
and vestibular motion per eption as it a e ts the visual-inertial nulling measure.
All of the nulling experiments des ribed in this dissertation involved both visual
and inertial sinusoidal os illations in the horizontal plane at .1 Hz. The .1 Hz frequen y was pi ked for two reasons. The rst had to do with the dynami s of the
semi- ir ular anals of the vestibular system, whi h dete t angular a elerations of
the head. As taken from Howard [46℄, natural head motions o ur in the frequen y
6 Humans

have other inertial motion dete tors, for instan e in the torso [104℄.

61
range of .1 Hz to 5 Hz. The semi- ir ular anals are tuned for these frequen ies. In
this range, the dete tion of inertial os illations should have a gain of lose to 1 and
very little phase o set. This implies that parti ipants should be able to a urately
dete t the inertial os illation in this frequen y range (provided that the amplitude is
suÆ ient). At higher frequen ies, the anals exhibit phase lag; at lower frequen ies,
phase lead. To test the ability of visual ues to overwhelm inertial ues, we needed
parti ipants to be re eiving a urate inertial information. This required that the
experiment be ondu ted somewhere in the .1 Hz to 5 Hz range.
A se ond onsideration was to pi k a range in whi h both the visual and vestibular
systems are used for motion dete tion. Otherwise, an e e tive on i t between the
two ould not be reated. The visual system dominates at very low frequen ies; the
vestibular system, at high frequen ies. The gain for the visual and inertial systems
are equal at about .02 Hz [112℄. This was a reason for pi king the lowest value in the
.1 Hz to 5 Hz range: it is loser to .02 Hz.
Aside from frequen y onsiderations, the measurement of inertial amplitude is
also an important issue. In prin iple, inertial amplitude an be measured in many
ways, for instan e: angular displa ement; peak angular velo ity; peak angular a eleration, et . The best hoi e appears to be the maximum peak-to-peak di eren e
in velo ity a ross a omplete y le. This is be ause while the semi ir ular anals
respond to a elerations in their individual planes of rotation, they integrate these
a elerations to report velo ities [46℄. Benson et al. [9℄ mention that \The frequen y
oded signals from the ampullary re eptors [of the semi ir ular anals℄ are mu h more
losely related to the angular velo ity of the head than to its angular a eleration."
They add that \many experiments (reviewed by Guedry [31℄) in whi h thresholds for
the dete tion of whole-body rotation were determined have shown that threshold is
primarily dependent upon the hange of angular velo ity (_) a hieved by the test
stimulus rather than by its a eleration, provided the duration of the stimulus does

62
not appre iably ex eed the integration time onstant of the semi ir ular anals."7
In the same paper, Benson et al. [9℄ give the mean threshold for the dete tion of
whole body yaw rotations in 30 subje ts as 1.5Æ/se . The velo ity stimulus followed
a single osine bell traje tory.
The Determination of Inertial Cross-Over Amplitudes

The pro edure des ribed here determines the relative in uen e of visual and inertial stimuli on the sense of self-motion. See Figure 3.1, on page 42, for a diagram
illustrating the pro edure.
Both inertial ( hair) and visual (HMD) os illations were at .1 Hz, but the phases
on i ted. The degree to whi h parti ipants identi ed with the visual vs. inertial
self-motion ues was determined by having them indi ate with a toggle swit h the
per eived right and left extremes of the inertial motion. If the inertial amplitude is
suÆ iently low, one tends to inadvertently indi ate the ve tion ues, rather than the
inertial motion of the hair (even though, with one's eyes losed, one ould orre tly
follow the hair motion at the same inertial amplitude). The term \inertial dominan e" will be used when the observer orre tly indi ates the inertial motion, and the
term \visual dominan e" when the observer indi ates the ve tion stimulus, despite
attempting to signal the hair motion.
Using the PEST pro edure (see Se tion 3.8.4) to adjust the inertial amplitude after
ea h trial, the inertial amplitude at whi h the parti ipant swit hed between visual
7 Without

going into the details of vestibular physiology, the integration time onstant refers to the

time period over whi h a stimulus dete ted by the semi ir ular anals ontinues to be signaled.
The integration time onstant is partly a e ted by a re overy time within the semi ir ular anals
(about 4 se onds) and partly, apparently, is due to \the time taken for entral neural events to
subside" [46℄. The ombined integration time onstant is over 12 se onds. By omparison, the
duration of one y le of an inertial wave at .1 Hz, as used for the experiments des ribed here, is
10 se onds.

63
and inertial dominan e was determined. This is termed the \ ross-over amplitude".
The presen e hypothesis implies that the ross-over amplitude should be a presen e
measure. The resear h des ribed below investigated this measure.
The experiments des ribed in this hapter ran trials of two visual onditions in
parallel in ea h session. To ounter-balan e order e e ts, trials from the two onditions followed an ABBA pattern within ea h session. In addition, the ondition
with the rst trial was ounter-balan ed a ross parti ipants, and within parti ipants
a ross sessions.
The visual amplitude for all trials was 30Æ/se . This was pi ked for salien y after
informal trials. The experiment began with an inertial amplitude of 15Æ/se . The
initial step-size for the PEST pro edure was 10Æ/se . The termination ondition was
a step-size of at most 5Æ/se 8 . This value were pi ked to onverge fairly qui kly, in
order to keep sessions in the 1 hour range, ounting introdu tions, administering the
Embedded Figures Test, simulator si kness questionnaire data olle tion, et .
The ross-over data reported below are the midpoints of the range to whi h the
inertial amplitude was narrowed in Æ /se . This is half the maximum peak-to-peak
velo ity di eren e over one y le. See above for a dis ussion of these units.
The phase di eren e between the inertial and ve tion ues was always 90Æ. A
90Æ magnitude phase di eren e is useful in that the two urves are suÆ iently far
apart that the di eren e between visual and inertial dominan e is readily apparent in
the data, but not so far apart that the ve tion and inertial self-motion ues be ome
learly distin t. A magnitude of 90Æ was pi ked on the basis of informal trials.
In Pilot Study AIP3, a random hoi e of either 90Æ phase angle was used, to avoid
a possible learning e e t whi h might o ur from holding the phase angle onstant.
This resulted in a poor test-retest orrelation (.38), possibly due to a systemati
8 In

a few ases, only a resolution of 10Æ /se ould be a hieved due to equipment or s heduling

problems.

64
di eren e in the diÆ ulty of the two phase angles9.
Pilot Study AIP4 used the same ve tion-inertial phase angle of +90Æ for all trials.
For a small parti ipant sample (n = 4), this resulted in a test-retest orrelation of
.99. There was not a within-subje ts drop in the ross-over amplitude from the rst
to the se ond session. All 4 of the parti ipants believed (in orre tly) that the phase
angle was hanging a ross trials10. This suggested that a learning e e t might not be
a major issue with a single phase angle.
Ea h parti ipant was given an initial pra ti e trial with the HMD turned o at
an inertial amplitude of 10Æ/se . This served both to introdu e the parti ipant to the
pro edures and to he k their inertial motion dete tion. Only one parti ipant was
s reened out on the basis of this test11 .
Between trials, while the hair was at rest, parti ipants were asked to relax and
lose their eyes. Before ea h new trial, after starting the hair in motion, parti ipants
were asked to ount down by 7's with their eyes losed for 25 se onds from an arbitrary
number sele ted by the experimenter. This provided a distra tion whi h prevented
the parti ipant from \lo king in" to the inertial motion. They then ontinued to
ount down for an additional 25 se onds with their eyes open. This gave the ve tion
ues time to build up an e e t. Next, parti ipants were asked to stop ounting and
start signaling the per eived endpoints of the hair's inertial motion while attending
to the visual s ene.
In written and oral instru tions, parti ipants were asked to
1. Pay attention to the visual s ene when your eyes are open.
2. Signal how you think the hair is physi ally moving.
9 It

should be remembered that .1 Hz is at the bottom of the ideal angular motion-dete tion range

for the vestibular hannels. Below this frequen y, phase lead be omes an issue. See above.
10 A

possible reason for this in orre t per eption will be given in Se tion 4.4.5, below.

11 Re all

from above that the mean threshold for dete tion of whole body yaw rotations is 1.5Æ /se .

65
3. Close your eyes after ea h trial until asked to open them.
A trial always ran for at least 4 signals. Often the trial was halted after four
signals, as all four would indi ate either lear visual or lear inertial dominan e. If
not, approximately 30 se onds to a minute of signals would be re orded to examine
whether a lear trend developed. The trial would then be halted and the average
phase distan e of the signals from the inertial and ve tion stimuli would be re orded.
After the trial, the hair was brought smoothly down to zero velo ity.
At inertial amplitudes markedly below or above the ross-over amplitude, parti ipants signaled per eived endpoints of their inertial motion whi h were roughly a
phase distan e of 10Æ from either the ve tion or inertial ues, depending on whether
the parti ipant was in inertial or visual dominan e12. As the ross-over amplitude was
approa hed, the phase distan es equalized in a predi table if somewhat haphazard
manner. This raised the issue of what de ision to make for the PEST pro edure if
the phase angles between the parti ipant's signals and the two motion stimuli were
indistinguishable (de ned to mean: within 10Æ of ea h other). The de ision made
was as follows: if the ross-over amplitude had already been determined to within a
5Æ/se range, simply take the urrent amplitude as the ross-over amplitude. If not,
retest.
4.2.2

Overview of the Reported Presen e Measure

Reported presen e data was gathered in both experiments des ribed in this hapter. The order of onditions was ounter-balan ed a ross parti ipants and (in Experiment AIE2) sessions. Ea h experiment gathered two reported presen e ratings per
ondition, in order to examine test-retest reliability. Both experiments used the same
12 The

a ura y of the inertial motion signaling was limited by human response, rather than equip-

ment onsiderations. Under ideal onditions, simply wat hing the hair moving with no distra tions, it was not diÆ ult to signal the endpoints of the hair motion to within 5Æ , often less.

66
pro edure, ex ept that Experiment AIE1 re orded two presen e ratings per ondition
in a single session, and Experiment AIE2 re orded one presen e rating per ondition
in ea h of two sessions. In Experiment AIE1, initial presen e ratings were obtained
for ea h of three onditions in random order. Immediately afterwards13 , presen e
ratings were obtained for the same three onditions in an independent random order.
Parti ipants were familiarized with the general idea of presen e prior to the experiment by the following written des ription.
Experien ing a virtual world an lead to a on i t in where you fo us
your attention. For instan e, you may feel as if you are in the pla e
suggested by the virtual world. At the same time, you may be aware
of the ontradi tory fa t that you are in the laboratory, wearing a headmounted display. The following question assesses the extent to whi h
you feel \immersed" in the virtual world, and whether the intensity of
this feeling is di erent in the various onditions. There are no \ orre t"
answers. Please make your ratings as honestly as possible. Pi k a number
between 1 and 7, where \1" means \very little" and \7" means \very
mu h so".
Parti ipants were also familiarized (prior to the experiment) with the parti ular
presen e question to be asked. The question was as follows:
1. I feel like I am in ... (1 = The laboratory, wearing a head-mounted display.) (7
= In the virtual world.)
For a dis ussion of this presen e question, see Se tion 3.8.1.
When reported presen e data was gathered, parti ipants were seated in a hair
whi h os illated sinusoidally at .1 Hz, with an amplitude of 20Æ/se . They wore a
13 Within

minutes.

67
Virtual Resear h VR4 HMD showing the (non-stereographi ) s ene os illating sinusoidally at .1 Hz, with an amplitude of 30Æ/se . The inertial and ve tion stimuli
were onsistent (i.e., what one would expe t in the real world). The visual amplitude
was hosen for salien y, and the inertial amplitude was hosen to be in the upper
range of inertial amplitudes likely to be experien ed during the visual-inertial nulling
measure trials. The non-equality of the inertial and visual amplitude was deliberate,
to be onsistent with the normal state of a airs in the visual-inertial nulling measure
trials14.
Parti ipants were asked to observe ea h ondition, whi h were shown for one
minute ea h. Immediately afterwards, parti ipants were exposed to the same onditions, and, after 10 se onds, asked the presen e question about ea h ondition.
Before obtaining repeated measures for the same ondition, parti ipants were
instru ted to answer the question in the same way if they felt the same as the rst
time, and di erently if their impressions had hanged.
4.3

Experiment AIE1: Narrow Field-of-View (Reported Presen e)

4.3.1

Introdu tion

Experiment AIE1 and Pilot Studies AIP3 and AIP4 jointly examined the reported presen e and visual-inertial nulling measures for three FOV's (16Æ, 32Æ, and
48Æ). AIE1 gathered reported presen e data for referen e against the ross-over data
gathered in Pilot Studies AIP3 and AIP4. Half of the 20 parti ipants from Experiment AIE1 pro eeded to the more lengthy nulling studies, AIP3 or AIP415 . It was
14 It

is interesting that, judging from informal dis ussions, parti ipants never had a lear per eption

of what the di eren e was between the inertial and visual amplitudes, or even whether there was
a di eren e.
15 Running

more parti ipants in AIE1 than AIP3 and AIP4 took advantage of the relative briefness

of AIE1 to gather more data on reported presen e, and served to gain pra ti al experien e with

68
hypothesized that wider FOV's would result in higher reported presen e.
4.3.2

Summary

See Table 4.1.
Table 4.1: Experiment AIE1: Field-of-View (Reported Presen e)
Ba kground Investigated the e e t of FOV on reported presen e.
Hypothesis Reported presen e will be higher at wider FOV's.
Methods
16Æ, 32Æ, and 48Æ FOV's were ompared. There were 1-minute
pre-exposures to ea h ondition, order ounter-balan ed, with 2
per-exposure presen e ratings re orded.
Results
48Æ FOV produ ed signi antly higher reported presen e than
the other two onditions. However, about 1=3 of parti ipants
did not rate the widest FOV as invoking the highest presen e.
Con lusions The predi ted main e e t was found. The 1=3 of parti ipants
who did not follow this pattern may re e t a foreground o lusion e e t at narrow FOV.

4.3.3

Methods

There were 20 adult volunteers (16 male, 4 female) all but 1 naive to the experimental hypothesis. Fifteen reported more than 10 minutes prior experien e with
virtual environments. General data about parti ipants are given in Table 4.2.
For general methods, in luding the image used and how the per-exposure presen e
ratings were taken, see Se tion 4.2. Parti ipants were given a one-minute pre-exposure
to ea h ondition, in random order, during whi h they were asked to examine the s ene
new equipment.

69

Table 4.2: Parti ipant Overview for Experiments AIE1
Id

Gn

Age

Prev

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

M
M
M
M
M
M
M
F
M
M
F
M
M
M
F
M
M
F
M
M

41
29
43
42
26
36
29
40
27
34
43
25
36
21
27
27
27
38
53
48

1
0
1
1
1
1
1
1
1
1
0
1
0
0
1
0
1
1
1
1

Id is the parti ipant number. Gn gives the gender, Age the age in years,
and Prev is \1" if the parti ipant had more than 10 minutes of prior
experien e in virtual environments, \0" otherwise.

70
and gather their impressions. Subsequently, per-exposure presen e ratings were taken
for ea h ondition in an independent random order.
Three FOV onditions were ompared: 16Æ, 32Æ, and 48Æ (the widest available on
the HMD used). FOV was restri ted by bla king pixels around the boundary of the
display (by the same proportion verti ally as horizontally).
4.3.4

Results

See Table 4.3 for a data summary. The test-retest orrelation a ross all three
onditions is .81.
Data from Table 4.3 were analyzed using an ANOVA pro edure. The ANOVA
Table is given in Table 4.4. A main e e t was found for FOV (F(2,38) = 6.2; p <
.01). On Tukey's pro edure, the di eren e between the 48Æ and 16Æ FOV onditions
was signi ant (p < .01) as was the di eren e between the 48Æ and 32Æ onditions (p
< .01). The di eren e between the 32Æ and 16Æ FOV onditions was not signi ant.
There was a trend for in reasing reported presen e over time (F(1,19) = 3.4; p <
.10).
While there is a main e e t for highest reported presen e in the widest FOV ondition, about a third of the parti ipants did not follow this pattern. Five parti ipants
(6, 8, 11, 12, 14) reported highest presen e not at the widest FOV. An additional
two parti ipants (10,18) reported that the widest FOV produ ed the same presen e
as at least one of the narrower two FOV's. For all of these seven, the narrowest FOV
resulted in the same or greater reported presen e as the other two onditions.
4.3.5

Dis ussion

Despite the rst and se ond reported presen e ratings for ea h ondition having
been taken within minutes of ea h other, there was signi ant variation (.81 orrelation). This is indi ative of the volatility of reported presen e.

71
Table 4.3: Reported Presen e Data for Experiment AIE1
Id

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

16Æ (1)

16Æ (2)

32Æ (1)

32Æ (2)

48Æ (1)

48Æ (2)

3
5
5
5
6
6
3
2
3
3
4
4
2
2
4
5
6
4
1
1
1
1
4
1
2
2
3
4
5
5
5
6
5
6
3
6
4
4
3
5
5
5
5
5
3
3
4
4
1
1
3
3
7
7
5
5
5
5
5
5
6
6
3
4
5
5
5
5
5
5
3
5
3
2
5
5
6
6
7
7
4
5
3
4
1
2
2
3
6
6
3
5
5
5
5
6
5
5
4
5
6
6
2
4
1
2
3
3
2
4
3
3
6
6
1.5
1
2
1.5
3.5
1.5
3.3(1.8) 3.7(1.9) 3.5(1.3) 3.9(1.4) 4.8(1.3) 4.8(1.6)
Id is the parti ipant number. The last 6 olumns give the rst and se ond
reported presen e rating (1|7 s ale) for the 16Æ, 32Æ, and 48Æ FOV onditions, respe tively. The nal row gives the mean and (in parentheses)
standard deviation for ea h olumn.

72

Table 4.4: Experiment AIE1 ANOVA Table for Reported Presen e Data
Sour e

TREAT
SESSION
TREAT x SESSION
P
TREAT x P
SESSION x P
TREAT x SESSION x P
Total

Sum of Squares

df

37.4
2.4
1.3
135.8
112.4
12.8
15.8
318.0

2
1
2
19
38
19
38
119

Mean Square

F

Sig.

18.7 6.2 p < .01
2.4 3.4 p < .10
.6 1.5 p > .10
7.1
3.0
.7
.4

ANOVA table for reported presen e data from Experiment
AIE1.
\TREAT" is the treatment ondition with 3 levels (16Æ, 32Æ, and 48Æ
FOV), \SESSION" is the session number with 2 levels, and \P" is parti ipants with 20 levels. TREAT and SESSION were ea h tested by their
intera tion with P. The TREAT x SESSION intera tion was tested by
TREAT x SESSION x P.

73
The over-all reported presen e di eren e in favor of the 48Æ ondition, and the
la k of di eren e between the two narrower onditions, is onsistent with a ommon
belief in the VE ommunity that one approa hes a \presen e threshold" at about 60Æ
FOV.
However, about a third of the parti ipants did not rate the widest FOV ondition
as having higher presen e than the other two onditions. This may re e t a foreground
o lusion e e t (see Se tion 3.3.4): at the narrowest FOV, the moving s ene may
have appeared more distant and onsequently a more salient rest frame. It would be
interesting to apply the same proto ol to the study of 3 wider FOV's, su h as 60Æ,
80Æ and 100Æ, where the boundary is less noti eable and a foreground o lusion e e t
is less likely to o ur.
Two pilot studies were ondu ted whi h examined a visual-inertial nulling measure for the same 3 FOV onditions. These are des ribed in Se tion B.2. A primary
purpose of the resear h des ribed in this hapter was to measure a main e e t for
treatment with the nulling measure in agreement with predi tion and reported presen e. Be ause of the ambiguity in response to the three FOV onditions des ribed
above, it was de ided that a di erent set of onditions would provide a better opportunity to establish the nulling measure. For this reason, the pilot studies des ribed
in Se tion B.2 were not pursued. Instead, a meaningful/random manipulation was
examined, as des ribed below.
4.4
4.4.1

Experiment AIE2: Meaningful/Random
Introdu tion

Experiment AIE1, whi h studied 16Æ, 32Æ and 48Æ FOV onditions, did not show a
uniform pattern in the reported presen e data. Nor was there a uniform pattern in the
visual-inertial nulling data for these onditions, as reported in Se tion B.2. In order
to test whether a lear main e e t for both measures ould be found in the predi ted

74
dire tion, it was de ided to study a di erent manipulation. The manipulation hosen
was to ompare a meaningful to a random s ene. It was hypothesized that the
meaningful ondition would produ e higher values on both the nulling and reported
presen e measures.
This meaningful/random manipulation was motivated in part by Experiment CogE1
(see Appendix A), whi h showed a main e e t in the dire tion of higher reported
presen e for meaningful as ompared to meaningless stimuli. Unlike the meaningful/meaningless manipulation of Experiment CogE1, the meaningful/random manipulation studied here a e ts low-level per eptual fa tors, su h as number and distribution of edges.
4.4.2

Summary

See Table 4.5.
4.4.3

Methods

There were 12 adult parti ipants (9 male, 3 female) all but 1 naive to the experimental hypothesis. Six reported more than 10 minutes prior experien e with virtual
environments. Due to the availability of funds, and to ompensate parti ipation near
the end of an a ademi term, the last ve parti ipants (28, 29, 30, 31 and 32) were
paid $10/session. General data about parti ipants are given in Table 4.6.
For general methods, in luding the image used, see Se tion 4.2. As with Experiment AIE1, parti ipants were given a one-minute pre-exposure to ea h ondition, in
random order, during whi h they were asked to examine the s ene and gather their
impressions. Per-exposure presen e ratings were then taken for ea h ondition in an
independent random order. Subsequently, visual-inertial ross-over amplitudes were
found a ording to the pro edure in Se tion 4.2. Parti ipants were asked to attend
to the visual s ene while signaling the per eived inertial endpoints of their motion.

75
Table 4.5: Experiment AIE2: Meaningful/Random (Nulling Measure)
Ba kground Investigated the e e t of a meaningful/random manipulation on
the reported presen e and nulling measures.
Hypothesis Both measures will re ord higher values for the meaningful ondition.
Methods
A meaningful and random s ene were ompared, both at 48Æ
FOV. There were two exposures to ea h ondition for ea h measure. A +90Æ ve tion-inertial phase angle was used on all nulling
measure trials.
Results
The test-retest orrelation for both measures was about .80. The
nulling measure found the predi ted e e t (p < .05). There was
a trend for reported presen e in the predi ted dire tion (p <
.10).
Con lusions For the meaningful/random onditions, the nulling measure is
self- onsistent and agrees with predi tion and a trend in the
reported presen e data.
A 48Æ FOV was used (the widest the HMD allowed). In the \random" ondition,
the lo ation of ea h pixel in the image was randomized. The \meaningful" ondition
was the image as originally taken16 .
The experiment was run in two sessions, ea h omparing the meaningful to the
random ondition with both nulling and reported presen e dependent measures. The
two sessions were pro edurally identi al, ex ept that an Embedded Figures Test was
administered following one of these sessions and the introdu tion was not repeated.
16 Thus,

the \meaningful" ondition from the urrent experiment was identi al to the \48Æ" ondi-

tion from Experiment AIE1.

76

Table 4.6: Parti ipant Overview for Experiment AIE2
Id

Gn

Age

Prev

EFT

21 M 36 1 24
22 M 32 0 15
23 F 28 1 37
24 M 33 0 21
25 M 55 1 15
26 M 23 1 11
27 M 31 0 30
28 M 44 1 29
29 F 28 1
7
30 F 44 0 38
31 M 21 0 26
32 M 24 0 10
Id is the parti ipant number. Gn gives the gender, Age the age in years,
and Prev is \1" if the parti ipant had more than 10 minutes of prior
experien e in virtual environments, \0" otherwise. EFT gives Embedded
Figures Test s ores.

77
Ea h session began by obtaining one per-exposure reported presen e rating for ea h
ondition a ording to the pro edure des ribed in Se tion 4.2.2.
4.4.4

Results

The ANOVA Table for the ross-over data is given in Table 4.8. A main e e ts
was found for treatment, with the \meaningful" ondition higher than the \random"
ondition (F(1,11) = 8.7; p < .05).
The ANOVA Table for the reported presen e data is given in Table 4.9. A trend
was found for treatment, with the \meaningful" ondition higher than the \random"
ondition (F(1,11) = 2.7; p < .10).
The test-retest orrelation between sessions is .83 for the visual-inertial nulling
measure and .80 for the reported presen e measure.
Table 4.10 gives the orrelation between the EFT s ores for ea h parti ipant and
the mean a ross sessions of the meaningful and random data for both measures. At
the urrent sample size, none of these orrelations are signi antly di erent from zero
(p < .05).
The orrelation of the ross-over data with the reported presen e data for ea h
subje t is .0617 .
The orrelation of the di eren e between onditions for the visual-inertial nulling
measure with the di eren e between onditions for the reported presen e measure is
.3818. There is a trend for this orrelation being signi antly di erent from zero (p
< .07).
In the post-experiment debrief, 10 of the 12 parti ipants reported (in orre tly) believing that the ve tion-inertial phase angle varied between trials. One was un ertain
17 This

was omputed by omparing all data between the two measures with mat hed parti ipant

identi er, treatment ondition, and session number.
18 This

was omputed by nding the di eren e a ross treatment onditions for mat hed parti ipants

and session numbers, then orrelating these di eren es a ross measures.

78
Table 4.7: Nulling and Reported Presen e Data for Experiment AIE2
Cross-Over (Æ /se )

Id

Reported Presen e (1-7)

1st Session

2nd Session

1st Session

2nd Session

MCO

MCO

MRP

MRP

RCO

RCO

RRP

RRP

21 35 35 22.5 17.5 1
1
2
1
22 1.5 1.5 1.5 1.5 2
1
1
1
23
9
9 22.5 22.5 1
1
1
1
24
3 1.5 1.5 1.5 5
5
4
5
25 30 30 35 35
1
1
1
1
26
9
8
4
1.5 3
2
2
1
27 32.5 17.5 15 15
2
1
2
1
28 7.5 2.5 6.5 1.5 2
1
2
1
29
9
4
8
4
4
2
3
2
30 35 35 35 35
3
7
7
7
31
9 6.5 12 6.5 5
3
5
3
32 7.5 7.5 7.5 7.5 2
2
5
3
Mean 15.7 13.2 14.3 12.4 2.6 2.3 2.9 2.3
STD 13.2 13.0 12.0 12.7 1.4 1.9 1.9 2.0
Data from Experiment AIE2. Cross-over gives the inertial amplitude at
whi h parti ipants swit h between visual and inertial dominan e. (These
data are the mean of ross-over ranges, as des ribed in Se tion 4.2, ex ept
where
an upper bound was not found. In these ases, the lower bound of
35Æ/se was used.) Higher values indi ate more visual apture. Reported
Presen e gives the parti ipant's per exposure report of their sense of
\being in" the visual s ene. Id is the parti ipant number, equivalent to
the parti ipant numbers in Table 4.6. MCO and RCO give the rossover amplitudes for the meaningful and random onditions, respe tively.
MRP and RRP give the reported presen e ratings for the meaningful and
random onditions, respe tively. The bottom two rows give the mean and
STD for ea h olumn.

79

Table 4.8: Experiment AIE2 ANOVA Table for Cross-Over Data
Sour e

TREAT
SESSION
TREAT x SESSION
P
TREAT x P
SESSION x P
TREAT x SESSION x P
Total

Sum of Squares

df

56.3
14.1
1.3
6402.4
71.0
555.3
64.8
7165.3

1
1
1
11
11
11
11
47

Mean Square

F

Sig.

56.3 8.7 p < .05
14.1 0.3 p > .10
1.3 0.2 p > .10
582.0
6.5
50.5
5.9

ANOVA table for ross-over data from Experiment AIE2. \TREAT" is
the treatment ondition with 2 levels (meaningful/random), \SESSION"
is the session number with 2 levels, and \P" is parti ipants with 12 levels.
TREAT and SESSION were ea h tested by their intera tion with P. The
TREAT x SESSION intera tion was tested by TREAT x SESSION x P.

80

Table 4.9: Experiment AIE2 ANOVA Table for Reported Presen e Data
Sour e

Sum of Squares

df

3.0
.3
.3
119.0
12.0
9.7
5.7
150.0

1
1
1
11
11
11
11
47

TREAT
SESSION
TREAT x SESSION
P
TREAT x P
SESSION x P
TREAT x SESSION x P
Total

Mean Square

F

Sig.

3.0 2.7 p < .10
.3 .3 p > .10
.3 .6 p > .10
10.8
1.1
.9
.5

ANOVA table for ross-over data from Experiment AIE2. \TREAT" is
the treatment ondition with 2 levels (meaningful/random), \SESSION"
is the session number with 2 levels, and \P" is parti ipants with 12 levels.
TREAT and SESSION were ea h tested by their intera tion with P. The
TREAT x SESSION intera tion was tested by TREAT x SESSION x P.

Table 4.10: Correlations Between EFT S ores and the Two Dependent Measures
Cross-Over

Reported Presen e

Meaningful Random Meaningful Random
EFT .45(.14) .42(.18) .04(.90) .28(.37)

The orrelation between the Embedded Figures Test s ores and the rossover and reported presen e data for ea h of the meaningful and random
onditions. The signi an e of ea h orrelation is given in parentheses, for
the null hypothesis that the orrelation is zero. While the ross-over orrelations appear higher, none of these orrelations are signi antly di erent
from zero at the urrent sample size.

81
and one believed ( orre tly) that the phase angle was always the same.
4.4.5

Dis ussion

The visual-inertial nulling measure found a main e e t for treatment in the predi ted dire tion of meaningful higher than random (p < .05). The reported presen e
measure showed a trend in the same dire tion (p < .10). Both measures showed
reasonable test-retest reliability (.83 and .80, respe tively).
It is not surprising that no relationship (.06 orrelation) was found between the
magnitude of the ross-over data and the magnitude of reported presen e. This is
onsistent with the la k of a standard s ale, between parti ipants, governing how to
assign numbers to mental states. However, there is a weak relationship (.38 orrelation) between di eren es a ross treatment onditions a ross the two measures. Thus,
parti ipants who reported a large di eren e in presen e between the two onditions
showed a weak tenden y to exhibit a orresponding large di eren e in the nulling
measure.
While the urrent sample size did not show statisti al signi an e, the orrelation of the visual-inertial nulling measure with the EFT s ores appears to be higher
than the orrelation of the reported presen e measure with the EFT s ores. This is
onsistent with the suggestion that the visual-inertial nulling measure is more losely
related to eld dependen y, a fa tor whi h would be expe ted to in uen e betweensubje t variation in presen e levels. This implies that the between-subje t variation
in the visual-inertial nulling measure may be more a urate than the between-subje t
variation in the reported presen e measure at re ording real di eren es in presen e.
A on ern was that a learning e e t would result from using the same ve tioninertial phase angle for all trials. This on ern appears to be unfounded. Ten out of
12 of the parti ipants believed that the phase angle varied between trials, with one
more un ertain. (Additionally, all 4 of the parti ipants in Pilot Study AIP4 believed
the phase angle varied between trials.) Nor was there a main e e t for session number

82
in the ANOVA.
Why did parti ipants not noti e the onstant phase relationship? One possibility
is as follows. As the inertial amplitude dropped, the visual in uen e on the sense
of motion in reased, whi h made it more diÆ ult to distinguish the inertial from
the visual self-motion ues. This may have been interpreted as a de rease in the
ve tion-inertial phase di eren e. Consistent with this (as mentioned at the end of
Se tion 4.2.1), parti ipants did not tend to make a sudden jump from inertial to visual
dominan e, but rather approa hed the ross-over by signaling progressively further
from the inertial peak as the inertial amplitude was lowered.
The ontinuous approa h to the ross-over amplitude is supportive of the view
that the sense of presen e in an environment is a gradated, rather than all-or-nothing,
phenomenon. It is also onsistent with resear h on visual-proprio eptive dis repan ies. Wel h and Warren [109℄, reviewing the literature, report that \visual bias of
proprio eption is not an all-or-none phenomenon."
4.4.6

Con lusion

This preliminary resear h has shown that the visual-inertial nulling measure is
apable of nding a main e e t in the predi ted dire tion whi h agrees with a trend
in reported presen e s ores, and that the test-retest reliabilities for the two measures
are similar. It nds a weak trend for a stronger relationship between the visual-inertial
nulling measure and eld dependen y (as measured by the Embedded Figures Test)
than between reported presen e and eld dependen y.
A nulling presen e measure has onsiderable advantages over a presen e measure
based on self-report. Humans were evolved to make per eptual judgments. We rarely
arti ulate spatial judgments; however our a tion reveals those \judgments". Spatial
per eption supports a tion, primarily by representing what a tions the environment
will permit. We were not evolved to rate presen e on a numeri s ale.
Be ause the nulling measure is more deeply rooted than reported presen e, ross-

83
over amplitudes found in di erent experiments may be less distorted by an hor effe ts than are reported presen e ratings. This would allow knowledge to be built
up systemati ally by pooling data a ross experiments, rather than being limited to
within-experiment omparisons.
The predi tion that the nulling measure is less prone to an hor e e ts than the
reported presen e measure ould be tested by omparing a ondition A to a ondition
B in one experiment, and by omparing ondition A to a quite di erent ondition
C in a se ond experiment with di erent parti ipants. The predi tion would be that
ondition A would have similar values in the two experiments on the nulling measure,
but signi antly di erent values in the two experiments on the reported presen e
measure.
Finally, a measure whi h is mu h more onvenient than the visual-inertial nulling
measure des ribed here, but whi h preserves its desirable properties, may be possible.
This measure would be based on the indu ed-motion of a ba kground grid, as reported
in Appendix D and dis ussed in Chapter 8.

Chapter 5
AREA II: PRESENCE MANIPULATIONS

5.1

Introdu tion

In the previous hapter, the measurement of sele ted rest frames through visualinertial nulling was examined as a means to evaluate the sense of presen e. In this
hapter, the manipulation of sele ted rest frames is studied as a means to alter the
sense of presen e. Spe i ally, this hapter studies the e e t of foreground o lusions1.
The following questions are addressed in this hapter.
1. Can a foreground o lusion in rease reported presen e?
2. Can a performan e measure be found for the e e t of a foreground o lusion?
Experiment AIIE1 and Experiment AIIE2 sought to measure the e e t of a foreground o lusion on reported presen e. Resear h on a performan e measure for foreground o lusions is dis ussed in Appendix C, in referen e to inside-out displays.
1 See

Se tion 3.7.6 for ba kground. Related pilot studies and an experiment on inside-out displays

are des ribed in Appendix C. The des ription of Experiments AIIE1 and AIIE2 borrows from
[83℄ and [82℄. While I was the lead investigator, this resear h was arried out in ollaboration
with Dr. Hunter Ho man.

85
5.2

Experiment AIIE1:

Foreground O lusions and Reported Pres-

en e I
5.2.1

Introdu tion

Previous resear h had suggested that a foreground o lusion an in rease ve tion.
Experiment AIIE1 addressed the question of whether it an also in rease reported
presen e. This question is important for both theoreti al and pra ti al reasons. The
theoreti al reason is that the RFC and the presen e hypothesis predi t a lose relationship between ve tion and presen e. Spe i ally, ve tion results from one aspe t of
the sele ted rest frame (the implied self-motion) whereas presen e re e ts the sele ted
rest frame in general. Hen e, if foreground o lusions in rease ve tion (for possible
reasons dis ussed in Se tion 3.7.6), then it follows that foreground o lusions should
also in rease presen e. One purpose of Experiment AIIE1 was to test this theoreti al
predi tion.
The pra ti al reason for investigating whether foreground o lusions a e t the
sense of presen e has to do with HMD design. Traditionally, it has been believed
that a wide FOV is ne essary to indu e a strong sense of presen e. There is a tradeo between FOV and resolution. Hen e, if one wants to in rease the sense of presen e
by in reasing the FOV, one needs to lower the resolution. If a foreground o lusion
mounted inside an HMD an in rease the sense of presen e, then it may be possible
to have a high level of presen e at a low FOV. Experiment AIIE1 investigates this
possibility as well.
5.2.2

Summary

See Table 5.1.

86
Table 5.1: Experiment AIIE1: Foreground O lusions and Reported Presen e I
Ba kground Foreground o lusions have been shown to in rease reported ve tion. If presen e and ve tion are losely related, foreground o lusions should also in rease reported presen e.
Hypothesis Foreground o lusions in rease reported presen e ompared to
the same FOV without a foreground o lusion.
Methods
The same 60Æ FOV was provided in an HMD either with or
without a foreground o lusion mounted in front of the s reen.
Reported presen e was assessed with a presen e questionnaire
following 2.5 minute exposures to ea h ondition.
Results
Reported presen e was higher in the foreground o lusion ondition (p < .01).
Con lusions The results support the predi tion that foreground o lusions
an in rease reported presen e.
5.2.3

Methods

There were 26 adult volunteers (19 male, 7 female). the experimental hypothesis.
Three reported more than 10 minutes prior experien e with virtual environments.
The parti ipants are summarized in Table 5.2.
See Se tion 3.7 for an overview of the equipment and Se tion 3.8.1 for a dis ussion of the presen e questionnaire. Parti ipants were exposed to the \Sharkworld"
environment run on a Division ProVision 100 using a dVisor HMD. A foreground
o lusion was provided with a pair of Lu as Produ ts \Super Sunnies" tanning goggles with the entral ultra-violet prote tor removed. As the tanning goggles were
worn dire tly over the eyes and blo ked peripheral vision, they removed visual ues
surrounding the s reen in the HMD. Thus, the tanning goggles met the \foreground

87

Table 5.2: Parti ipant Overview for Experiment AIIE1
Id

Gn

Prev

Id

Gn

Prev

33 M 0 46 M 0
34 F 0 47 F 0
35 M 0 48 F 0
36 M 1 49 F 0
37 M 0 50 F 0
38 M 1 51 M 0
39 M 0 52 M 0
40 M 0 53 M 0
41 M 0 54 M 0
42 M 0 55 M 0
43 M 0 56 F 0
44 M 0 57 F 0
45 M 0 58 M 1
Id is the parti ipant number, Gn gives the gender, and Prev is \1" if the
parti ipant had more than 10 minutes of prior experien e in virtual environments, \0" otherwise. Age data was not olle ted for this experiment,
beyond noting that all parti ipants were over 18.

88
o lusion" de nition of blo king out all ues at the same or greater distan e than the
display2.
The tanning goggles limited the FOV to about 60Æ. A mat hing FOV for the
non-foreground o lusion ondition was provided by masking the HMD s reen with
onstru tion paper aÆxed dire tly to the s reen's surfa e.
Ea h parti ipant was run for 2.5 minutes in ea h of the foreground o lusion
and non-foreground o lusion onditions with the ondition order ounterbalan ed
a ross parti ipants. They were shown how to at h moving sharks with a virtual
net atta hed to the hand. However, their a tivity in the virtual environment was
not onstrained by any parti ular task. After viewing both onditions, parti ipants
ompleted a presen e questionnaire.
Before lling out the presen e questionnaire, but after exposure to the virtual
environment, parti ipants were introdu ed to the idea of presen e with the following
paragraph.
Experien ing a virtual world an lead to a on i t in where you fo us
your attention. For instan e, in the Sharkworld, you may feel like you
are in the o ean, near a shark-infested shipwre k. At the same time,
you may be aware of the ontradi tory fa t that you are just standing in
the laboratory, wearing a virtul reality helmet. The following questions
assess the extent to whi h you felt \immersed" in the virtual world, and
the relative intensity of this feeling when wearing the tanning goggles
ompared to when the s reen was masked with paper. There are no
\ orre t" answers, please make your ratings as honestly as possible. Cir le
one of the numbers.
2 A tually,

due to the design of the dVisor and of the tanning goggles, part of the nasal edge of

the s reen was visible in the foreground o lusion ondition. The e e t found below would be
predi ted to have been even stronger if the foreground o lusion had perfe tly blo ked the s reen
boundary.

89
Results

Statisti al analysis used a paired, two-tailed, non-parametri Wil oxon signedrank test. The responses averaged a ross all 5 presen e questions were larger (Z = 3:1,
p = :002) for foreground o lusion (M = 4:2, SD = 1:1) than for non-foreground
o lusion (M = 3:3, SD = 1:0). When the responses to ea h question were analyzed
separately, signi ant di eren es between viewing onditions were found for all 5
questions (Z = 2:69, p = :007; Z = 3:26, p = :001; Z = 2:49, p = :01; Z = 2:79,
p = :005; and Z = 2:36, p = :02, respe tively).
Dis ussion

The results of Experiment AIIE1 support the hypothesis that foreground o lusions in rease reported presen e. Further, they suggests that a foreground o lusion
mounted in an HMD may be of pra ti al bene t for in reasing presen e.
While Experiment AIIE1 found a large di eren e between onditions, the possibility existed that the results were due to subtle ues pi ked up from the experimenters.
Experiment AIIE2 was a repli ation of Experiment AIIE1 whi h ontrolled for this
possibility, by using an experimenter blind to the hypothesis.
5.3

Experiment AIIE2: Foreground O lusions and Reported Presen e
II

5.3.1

Introdu tion

Experiment AIIE2 was run in the same manner as Experiment AIIE1 ex ept that
additional steps were taken to remove possible experimental bias. It was hypothesized
that the foreground o lusion would in rease reported presen e.

90
5.3.2

Summary

See Table 5.3.
Table 5.3: Experiment AIIE2: Foreground O lusions and Reported Presen e II
Ba kground Experiment AIE1 found an e e t of foreground o lusion on
reported presen e. However, demand hara teristi s may have
played a role.
Hypothesis Foreground o lusions in rease reported presen e ompared to
the same FOV without a foreground o lusion.
Methods
The experimenter was naive to the hypothesis. The same 60Æ
FOV was provided in an HMD either with or without a foreground o lusion mounted in front of the s reen. Reported
presen e was assessed with a presen e questionnaire following
2.5 minute exposures to ea h ondition.
Results
Reported presen e was higher in the foreground o lusion ondition (p < .04).
Con lusions The results support the predi tion that foreground o lusions
an in rease reported presen e.

5.3.3

Methods

There were 13 adult volunteers (9 male, 4 female). One reported more than 10
minutes prior experien e with virtual environments. The parti ipants are summarized
in Table 5.4.
The pro edure for Experiment AIIE2 was identi al to Experiment AIIE1 ex ept
for two modi ations. Firstly, Experiment AIIE2 was run using a double-blind pro edure in whi h the experimenter (as well as the parti ipants) was kept unfamiliar

91

Table 5.4: Parti ipant Overview for Experiment AIIE2
Id

Gn

Prev

59 F 0
60 M 0
61 M 1
62 M 0
63 M 0
64 F 0
65 M 0
66 M 0
67 F 0
68 M 0
69 M 0
70 F 0
71 M 0
Id is the parti ipant number, Gn gives the gender. Prev is \1" if the
parti ipant had more than 10 minutes of prior experien e in virtual environments, \0" otherwise. Age data was not olle ted for this experiment,
beyond noting that all parti ipants were over 18.

92
with the hypothesis. The experimenter was a new visiting student from Europe. He
administered the questionnaires in sealed envelopes and was not shown their ontent, or allowed to dis uss the experiment, until after it was on luded. In addition,
parti ipants were instru ted not to dis uss the experiment with the experimenter.
Se ondly, the order of presen e questions about the two onditions was randomized
a ross parti ipants.
5.3.4

Results

Statisti al analysis used a paired, two-tailed, non-parametri Wil oxon signedrank test. For Experiment AIIE2 (with half the number of parti ipants of Experiment AIIE1) the responses averaged a ross all 5 questions were larger (Z = 2:1,
p = :03) for foreground o lusion (M = 4:3, SD = :93) than for non-foreground
o lusion (M = 3:4, SD = :93).
5.3.5

Dis ussion

The results from Experiment AIIE2 appear to rule out demand hara teristi s
as a possible explanation of the results for Experiment AIIE1. The results from
Experiments AIIE1 and AIIE2 are onsistent with the predi tion of Se tion 3.7.6
that foreground o lusions in rease presen e.
5.4

Inside-Out Displays and Foreground O lusions

Se tion 3.3.4 suggested that foreground o lusions might redu e ontrol reversals
asso iated with inside-out displays. Approximately 3,000 trials were arried out to
address this question. As no e e t was found, this line of investigation is reported in
abbreviated form in Appendix C.

Chapter 6
AREA III: MOTION SICKNESS

Praise the sea, but keep on the land.

| Herbert
6.1

Introdu tion

The previous two hapters were intended to study the measurement and manipulation of sele ted rest frames. This hapter is on erned with the onsequen es when
a onsistent sele ted rest frame an not be formed1. It examines a situation ommon
with simulators, in whi h the visual ues imply self-motion, but the inertial ues do
not.
The following questions are addressed in this hapter.
1. Can an independent visual ba kground (IVB) in agreement with inertial ues
redu e reported simulator si kness and related side-e e ts?
2. If so, an it do so without redu ing the subje tive impa t of the s ene?
3. Can an IVB be e e tive even when attention is dire ted to the visual foreground,
in whi h visual self-motion ues disagree with the inertial ues?
1 See

Se tion 3.3.5 for ba kground, and Se tion 3.7 for a des ription of the equipment. This hapter

borrows from [80℄ and [81℄. While I was the lead investigator, this resear h was arried out in
ollaboration with Mark Draper.

94
Experiment AIIIE1 explored questions 1 and 2, above. Experiment AIIIE2 was
a follow-up study whi h added a visual task whi h for ed attention into the visual
foreground. This allowed the 3rd question to be addressed as well.
Both Experiment AIIIE1 and Experiment AIIIE2 involved a \low-end" virtual
environment with limited FOV. A brief pilot study involving a \high-end" driving
simulator, intended as a motion si kness experiment, is des ribed in Appendix D
be ause of its possible impli ations for an improved presen e measure. See Chapter 7
for a dis ussion.
6.2

Experiment AIIIE1: Independent Visual Ba kground for Low-End
Systems I

6.2.1

Introdu tion

Experiment AIIIE1 was the initial study of the e e tiveness of an IVB for redu ing
simulator side-e e ts. A low-end system was used, omparing the see-through (IVB)
and o luded (non-IVB) modes of a Virtual i-O HMD. A ir ular ve tion stimulus was
provided in both onditions. Illumination levels were su h that the IVB was rated
by parti ipants as being less visible than the ve tion stimulus. It was hypothesized
that the IVB ondition would produ e lower simulator side-e e ts than the non-IVB
ondition.
6.2.2

Summary

See Table 6.1.
6.2.3

Methods

There were 15 adult volunteers (10 male, 5 female). Twelve reported more than 10
minutes prior experien e with virtual environments. The parti ipants are summarized
in Table 6.2.

95
Table 6.1: Experiment AIIIE1: Independent Visual Ba kground I (Low-End)
Ba kground Motion si kness may arise from on i ting rest frames, rather
than from on i ting motion ues per se. The visual rest frame
is heavily in uen ed by the visual ba kground.
Hypothesis A visual ba kground in agreement with inertial ues should redu e simulator side-e e ts, even if the visual foreground is not
in agreement with inertial ues.
Methods
A ir ular ve tion stimuli was provided both with and without a
stationary visual ba kground. Measures in luded per-exposure
postural instability and post-exposure reported simulator si kness.
Results
Both postural instability (p < .03) and SSQ (p < .05) were
signi antly lower with a stationary visual ba kground.
Con lusions The results support the hypothesis.
A pre-exposure SSQ was administered to he k for pre-existing nausea or related
symptoms. No parti ipants or data were removed on this basis.
See Se tion 3.7 for a des ription of the equipment. The experiment was ondu ted
as a within-subje ts design. The order of onditions was ounter-balan ed a ross
parti ipants.
A ir ular motion visual stimulus was reated by pla ing a video amera on a
tripod in an open plaza on the University of Washington ampus and ontinuously
rotating the amera through 360Æ in yaw with a period of six se onds. The lo ation
was pi ked for visual salien e, having a variety of sharp edges and both verti al and
horizontal features. The angular velo ity of 60Æ/se was pi ked in a ordan e with
literature indi ating that motion si kness peaks at this value [30℄. Other sour es

96

Table 6.2: Parti ipant Overview for Experiment AIIIE1
Id

Gn

Age

Prev

72 M 23 1
73 F 25 1
74 M 22 1
75 M 24 1
76 M 32 1
77 M 41 0
78 F 33 0
79 M 22 1
80 F 44 1
81 M 24 0
82 F 41 1
83 F 25 1
84 M 41 1
85 M 42 1
86 M 29 1
Id is the parti ipant number. Gn gives the gender, Age the age in years,
and Prev is \1" if the parti ipant had more than 10 minutes of prior
experien e in virtual environments, \0" otherwise.

97
suggest that ve tion in reases with stimulus velo ity up to about 90Æ/se [20℄, at
least for stimuli onsisting of verti al stripes.
The head-mounted display (HMD) used to display the videotape was a Virtual iO i-glasses! VTV/VPC (see Se tion 3.7.3 for HMD spe i ations). The HMD was
masked so that no visual ues were available ex ept those on or through the display.
(That is, the edges of the opti s and HMD were o luded.)
For ea h parti ipant, there were two separate three-minute sessions in whi h
he/she was exposed to the ir ular motion stimulus, one session in ea h of the seethrough (IVB) and o luded (no IVB) modes (see Figure 3.2 on page 43). The
sessions were separated by 5 minutes. During the rst and third minutes of ea h
session, parti ipants were asked to stand in the \Sharpened Romberg" stan e. This
stan e onsists of pla ing one foot in front of the other, heel tou hing toe, weight
evenly distributed between the legs, arms folded a ross the hest and hin up [32℄.
In the se ond minute, to avoid fatigue, parti ipants were allowed to stand in a relaxed posture, but were instru ted to ontinue to look forward. Half-way through
the rst, se ond and third minutes, parti ipants were asked to roll their heads down
to their right shoulder, then to their left shoulder, then return their head to the upright position. This a tion was intended to in rease motion si kness by introdu ing
a visual-vestibular intera tion.
Parti ipants were instru ted to keep their eyes open and to look forward. They
were told that they ould break the Sharpened Romberg stan e if ne essary, but that
they should get ba k into it as qui kly as possible. Other than maintaining postural
stability and looking forward, parti ipants were not given a task in Experiment AIIIE1.
A verbal rating of the relative visibility of the ba kground in the see-through
ondition was re orded in the post-test questionnaire. This was intended to make
sure that the IVB in the see-through ondition was not simply \washing-out" the
ir ular ve tion stimulus provided by the HMD. A 1{5 s ale was used, with the low

98
end indi ating that only the foreground s ale was visible, and the high end indi ating
that only the ba kground was visible.
Three dependent measures were re orded in Experiment AIIIE1 to measure the
e e t of the IVB. The rst was a measure of per-exposure ataxia: the total number of
stan e breaks in the rst (SB1) and third (SB3) minutes of ea h session. A break was
de ned as a translation of either foot and/or an un rossing of the arms. Parti ipants
were asked to maintain the Sharpened Romberg stan e, but if a break was ne essary
they were to asked to get ba k into Sharpened Romberg as soon as possible.
The se ond dependent measure was the post-exposure SSQ, to re ord self-reported
symptoms of simulator si kness.
The third dependent measure was a post-exposure rating of ve tion, de ned in
terms of the following question. \While in the virtual environment, did you get the
feeling of motion (i.e., did you experien e a ompelling sensation of self-motion as
though you were a tually moving)?". The endpoints of the 1{7 s ale were an hored
as \not at all" and \very mu h so", respe tively.
6.2.4

Results

Data from one parti ipant were not analyzed due to inability to perform the
Sharpened Romberg task, parti ularly in the o luded ondition. The data from the
remaining fourteen parti ipants are summarized in Table 6.2.4.
The stan e break data passed tests of normality and homogeneity. Total stan e
breaks, SB1 and SB3 were analyzed with a 2-tail paired t-test. The questionnaire
data, SSQ and ve tion ratings, were analyzed using a non-parametri , 2-tail paired
Wil oxon.
Both SSQ (p < .05) and total stan e breaks (p < .03) were signi antly lower in
the see-through than the o luded ondition. Pooling a ross onditions, there were
signi antly fewer stan e breaks in SB1 than SB3 (p < .03), indi ating in reased
ataxia as the experiment progressed. There was a weak trend for higher ve tion in

99

Table 6.3: Data From Experiment AIIIE1
Condition
Measure

See-through

O luded

SSQ *
15:0(16:0) 18:7(24:5)
Total Stan e Breaks * 4:1(4:8)
9:1(8:2)
SB1
1:1(1:5)
3:9(3:2)
SB3
3:0(3:6)
5:2(5:5)
Ve tion
3:5=7:0(1:7) 3:5=7:0(1:9)
Ba kground Visibility 2:0=5:0(0:6)
N/A
\SSQ" is a reported simulator si kness value [55℄. \SB1" and \SB3" are
the total number of stan e breaks in the rst and third minute, respe tively. For the skewed data (SSQ, ve tion and visibility ratings), medians
are reported. Other values are means. Standard deviations are given in
parentheses. \*" implies that the di eren e between onditions is signi ant at p < .05.

100
the o luded ondition (p < .14).
A weak trend was found for an overall order e e t for total stan e breaks, with
the rst ondition more diÆ ult (p < .13).
6.2.5

Dis ussion

Experiment AIIIE1 examined whether an IVB (see-through ondition) ould redu e simulator side-e e ts. The introdu tion of an IVB signi antly redu ed ataxia
and SSQ ratings while viewing the ve tion stimulus. Given that ve tion ratings were
not signi antly lower in the see-through ondition, an IVB may be able to redu e
negative e e ts of simulators without signi antly a e ting the subje tive impa t of
the virtual environment.
Two questions posed by the ataxia data from Experiment AIIIE1 are: 1. \what
would the baseline number of stan e breaks be without a visual stimulus?" and, 2.
\was the in rease in stan e breaks from the rst to the third minute due to a build-up
of e e t or due to physiologi al fatigue from holding the Sharpened Romberg stan e?"
Pilot Study AIIIP1 was ondu ted to examine these questions (see Appendix D). It
was on luded that the baseline number of stan e breaks is essentially zero, and that
the in rease in stan e breaks from the rst to the third minute was primarily due to
a build-up of e e t.
6.3

Experiment AIIIE2: Independent Visual Ba kground for Low-End
Systems II

6.3.1

Introdu tion

Four possible short- omings were identi ed with Experiment AIIIE1.
First, the visual ba kground might have been simply \washing out" the ve tion
stimulus in the CI. This seems unlikely, given that parti ipants rated the IVB as less
visible than the CI (mean rating of 2, with 1 indi ating only the CI was visible, 5

101
indi ating only the IVB was visible).
Se ond, performan e of the head rolls by the parti ipant often varied within and
between sessions, as well as between parti ipants.
Third, the time between sessions may not have been long enough to prevent
arryover e e ts.
Fourth, it was possible that subje ts were simply ignoring the CI in the seethrough ondition. An IVB is only useful if it onfers value while the subje t is
primarily attending to the CI, where the simulator task is being performed.
Experiment AIIIE2 was similar to Experiment AIIIE1, with a few re nements
des ribed in the Methods se tion. The re nements were intended to address the
above issues. Most importantly, a visual task was provided to dire t attention into
the visual foreground. This ontrolled for the degree to whi h parti ipants attended
to the visual foreground. It also reated a stronger link to a real simulator task, in
whi h attention would have to be fo used on the visual foreground, rather than the
IVB.
As with Experiment AIIIE1, it was hypothesized that the IVB ondition would
produ e less simulator side-e e ts than the non-IVB ondition.
6.3.2

Summary

See Table 6.4.
6.3.3

Methods

There were 21 adult volunteers (15 male, 6 female). Four reported more than 10
minutes prior experien e with virtual environments. The parti ipants are summarized
in Table 6.5.
None had parti ipated in Experiment AIIIE1. Parti ipant s reening was done
more rigorously than in Experiment AIIIE1. Parti ipants were asked whether they

102
Table 6.4: Experiment AIIIE2: Independent Visual Ba kground II (Low-End)
Ba kground Experiment AIIIE1 found an e e t for a stationary visual ba kground redu ing simulator side-e e ts. But this redu tion might
disappear if attention is for ed into the visual foreground.
Hypothesis A visual ba kground in agreement with inertial ues should redu e simulator side-e e ts, even if the visual foreground is not
in agreement with inertial ues.
Methods
A ir ular ve tion stimuli was provided both with and without a
stationary visual ba kground. Measures in luded per-exposure
postural in stability and post-exposure reported simulator si kness. A visual task for ed attention into the visual foreground.
Results
Postural instability was signi antly lower with a stationary visual ba kground (p < .03).
Con lusions The results support the hypothesis.
were over 18, had had al ohol or drugs in the last 24 hours, had any olor-blindness,
were he ked for at-soled shoes, asked about urrent si kness or si kness in the last
week, whether they had a history of epilepsy or any un orre ted visual/vestibular
problems, and were he ked for 20/25 or better visual a uity.
The methods for Experiment AIIIE2 were similar to Experiment AIIIE1, with the
re nements des ribed below.
For Experiment AIIIE2, a se ond videotape was made, similar in all respe ts to
the rst ex ept that a performan e task was added to for e attention into the CI.
Two olleagues, who were separated by 180Æ, ea h held up a large sheet of posterboard. Ea h posterboard was white on one side and either bright green or bright
red on the other. When the amera swept by, ea h olleague displayed the white

103
Table 6.5: Parti ipant Overview for Experiment AIIIE1
Id

Gn

Age

Prev

87 M 31 0
88 F 20 0
89 F 37 0
90 M 18 1
91 M 21 0
92 M 27 0
93 F 27 1
94 M 20 0
95 F 18 0
96 M 21 0
97 M 21 0
98 M 22 0
99 M 23 1
100 M 22 0
101 M 18 0
102 M 30 0
103 F 24 0
104 M 39 0
105 M 24 0
106 M 21 0
107 F 34 1
Id is the parti ipant number. Gn gives the gender, Age the age in years,
and Prev is \1" if the parti ipant had more than 10 minutes of prior
experien e in virtual environments, \0" otherwise.

104
side approximately 75% of the time, and the olored side the other 25% of the time,
with a random pattern. Parti ipants were asked to all out the olor when a olored
sign appeared in the videotape. Responses were re orded on audiotape to he k task
performan e.
Instru tions to parti ipants in luded the following:
You will wear a Virtual i-O HMD for this experiment. The HMD will show
a rotating s ene. Red and green signs will be displayed periodi ally; your
task will be to all out \red" or \green" when you see a \red" or \green"
sign. You will be asked to stand in the Sharpened Romberg stan e for
one minute while viewing the image. (It's okay if you need to break the
Sharpened Romberg stan e; just get ba k into it.) You will then have one
minute to relax while viewing the rotating image, followed by 1 minute
more in the Sharped Romberg stan e, then a minute and a half to relax
while viewing the rotating image.
In addition to the three dependent measures used in Experiment AIIIE1 (stan e
breaks, SSQ and reported ve tion), pre- and post-exposure ataxia were re orded. The
pre- and post-exposure ataxia measures were made with a Chatte x Balan e System
(see Se tion 3.7.5). Postural measurements were taken with subje ts in the Sharpened
Romberg stan e with eyes open.
The average of three trials on the Balan e System prior to ea h ondition was
used as the baseline ataxia measure. The average of two trials on the Balan e System
after ea h ondition was used as the post-exposure ataxia measure. (As the intent
was to test for rapidly-de aying after-e e ts, it was not thought that more than two
post-exposure readings would be useful.) Measurements were taken with parti ipants
in the Sharpened Romberg stan e with eyes open. Prior to ea h measure, parti ipants mounted the apparatus and were given a few se onds to attain their balan e.

105
Parti ipants stepped o the apparatus and stood at ease for approximately 30 se onds between trails while the results were al ulated. Post-exposure measures were
re orded immediately after exposure. Parti ipants were asked to lose their eyes while
the HMD was removed, then open their eyes and walk dire tly to the Balan e System,
lo ated a ouple of steps away.
Exposure in ea h ondition was for 4.5 minutes, with the rst and third minutes in
Sharpened Romberg. The extra 1.5 minutes at the end ompared to Experiment AIIIE1 was intended to in rease the han e of nding an ataxia after-e e t. There was
a 15 minute break between onditions, during whi h subje ts relaxed and walked
around the building to mitigate any arryover e e ts between onditions.
Head rolls were removed, both to redu e individual variations in how the rolls
were performed and for fear that with attention for ed into the spinning CI, motion
si kness and ataxia might otherwise be too large.
The design of the HMD masking was improved to remove a slight breathing restri tion, whi h might have elevated SSQ ratings over both onditions in Experiment AIIIE1.
The visibility-of-the-ba kground rating was hanged to a 1-7 s ale in Experiment AIIIE2, for onsisten y with the ve tion s ale and prior work.
The Balan e System was also used to he k that parti ipants were ba k to their
baseline level of postural stability before leaving the experiment.

6.3.4 Results
Sharpened Romberg data from three subje ts were not analyzed due to ex essive
diÆ ulty with the task, parti ularly in the o luded ondition. However, their data
were not ex eptional on other variables and were therefore in luded in the rest of the

106

Table 6.6: Data From Experiment AIIIE2
Condition
Measure

See-through

O luded

SSQ

15:0(16:2)

18:7(15:0)

Total Stan e Breaks *

1:6(1:9)

2:6(2:7)

SB1

0:6(0:8)

1:0(1:2)

SB3

1:0(1:6)

1:6(1:9)

Ve tion

3:0=7:0(1:7) 4:0=7:0(1:9)

Ba kground Visibility 2:5=7:0(0:9)

N/A

Pre-Exposure Ataxia

6:2(1:9)

6:8(2:3)

Post-Exposure Ataxia

7:4(3:8)

8:3(3:9)

\SSQ" is a reported simulator si kness value [55℄. \SB1" and \SB3" are
the total number of stan e breaks in the rst and third minute, respe tively. For the skewed data (SSQ, ve tion and visibility ratings), medians
are reported. Other values are means. Standard deviations are given in
parentheses. \*" implies that the di eren e between onditions is signi ant at p < .05.

analysis2 . One outlier was removed from the visibility data (the subje t rated the
visual ba kground as quite visible in both the see-through and o luded onditions).
The three pre-exposure ataxia trials were averaged, as were the two post-exposure
ataxia trials (after determining that the post-exposure trials were not signi antly
di erent). Data on the CI visual task ( alling out the olored signs) were not formally
analyzed as all subje ts performed perfe tly or near-perfe tly in both onditions.
The data are summarized in Table 6.3.4.
Total stan e breaks, and pre- and post-exposure ataxia data were normal and
2 Stan e breaks are a dis rete measure, and be ome meaningless in the

The same problem does not ai t the other measures.

ase of ontinuous instability.

107
therefore no transformations were required. These data were analyzed with a 2-tail
paired t-test. The questionnaire data, SSQ and ve tion ratings, were analyzed using
a non-parametri , 2-tail paired Wil oxon, as these data were markedly skewed.
Total stan e breaks were signi antly lower in the see-through than o luded
onditions (p < .03).
No di eren e was found a ross onditions for SSQ, ve tion or post-exposure ataxia
s ores. The SSQ s ores after both onditions were signi antly higher than the pretest SSQ s ores (p < .03), indi ating that the stimulus did have an e e t, simply not a
ondition-spe i e e t. After nding no di eren e between onditions, pre- and postexposure ataxia data were ea h pooled a ross onditions, resulting in signi antly
greater ataxia for post-exposure than pre-exposure (p

<

.04). This indi ated the

existen e of in reased ataxia after only a 4.5 minute exposure to a ve tion stimulus.
Pooling a ross onditions, no di eren e was found between SB1 and SB3.

6.3.5 Dis ussion
Experiment AIIIE2 found that a strong e e t for redu ed per-exposure ataxia
remained in the presen e of a CI task. This was true even though the IVB was again
rated as less visible than the CI (mean rating 2.5, with 1 indi ating only the CI was
visible, 7 indi ating only the IVB was visible). This indi ates that the IVB does not
have to be overpowering to be e e tive.
There was a sharp drop in the overall ataxia for both onditions from Experiment AIIIE1 to Experiment AIIIE2 (See-through: 4.1 vs. 1.6; O luded: 9.1 vs.
2.6). We hypothesize that this was due to the absen e of head rolls.
The absen e of head rolls, redu ing overall e e ts, may partially explain the la k
of a signi ant di eren e on the SSQ and SB1 vs. SB3 tests (whi h were signi antly
di erent in Experiment AIIIE1).

108

6.4

General Dis ussion

The ndings from the experiments in this hapter suggest that IVB's may be e e tive for redu ing simulator si kness and simulator-indu ed ataxia. It is en ouraging
that strong e e ts were found in this initial work, using short exposure durations, a
low-end HMD with limited FOV, and a simple manifestation of the IVB on ept.
Rest frame sele tion appears to be a pre- ons ious operation: we are not generally
aware of how our per eptual system determines what will and will not be interpreted
as stationary. Consequently, it may be possible to indu e a visual rest frame whi h
avoids motion si kness and simulator side-e e ts without impinging on attention, and
thus distra ting from the task at hand.
It is noteworthy that a signi ant de rease in total stan e breaks (in both experiments) and SSQ ratings (in Experiment AIIIE1) was possible without a signi ant
di eren e in ve tion ratings between onditions. In the experiments, we used a postrather than per-exposure ve tion measure, to avoid an additional load on parti ipants
during trials. It is possible that a more sensitive per-exposure ve tion measure would
have found a di eren e between the two onditions: this is an interesting topi for
future resear h. However, we believe that urrent data are suÆ ient to indi ate that
there was not a large di eren e in ve tion between the two onditions. If one a epts
that ve tion is related to the sense of presen e in a virtual environment (see Chapter 3) this suggests that an IVB an produ e performan e and simulator si kness
bene ts without substantially redu ing the sense of presen e in a simulator.
The signi ant di eren e between pooled pre- and post-exposure ataxia s ores
in Experiment AIIIE2 is important, given that exposure durations were only 4.5
minutes per ondition and only a low-end system was used. This is an indi ation
of how easily VE's an produ e after-e e ts. However, these after-e e ts seemed
to disappear qui kly: no di eren e was found for pre-exposure ataxia between the
rst and se ond onditions, with a 15 minute break before the se ond ondition.

109
Furthermore, subje ts were required to perform at their baseline levels of postural
stability before leaving the area. This o urred within 10 minutes after the se ond
exposure for all subje ts.
An improved use of the IVB might be to put it in the periphery of the display. In
the urrent implementation, subje ts often reported that they lost tra k of the visual
ba kground when fo using on the CI. Pla ing the IVB in the periphery might allow
the IVB to be pro essed non- ompetitively and unobtrusively in peripheral vision.
This is a topi for future resear h.

Chapter 7

GENERAL DISCUSSION
Here about the bea h I wander'd, nourishing a youth sublime
With the fairy tales of s ien e, and the long result of Time.
| Alfred Lord Tennyson, Lo ksley Hall
7.1

Introdu tion

This hapter gives a high-level dis ussion of the resear h des ribed in the previous
hapters. It follows the division into 3 areas whi h was introdu ed in Chapter 3.
Suggestions for future resear h appear in Chapter 8.
7.2

Area I: Presen e Measures

The following resear h questions were posed at the top of Chapter 4:
1. How should a rest frame on i t measure be implemented?
2. Is the presen e hypothesis orre t? That is, does a measure based on real-virtual
rest frame on i t evaluate our subje tive sense of presen e?
3. What is the test-retest orrelation of a rest frame on i t measure?
4. How is the rest frame on i t measure related to eld dependen y?
The resear h des ribed in this dissertation has made signi ant progress towards
answering these questions. A measure based on visual-inertial nulling in the horizontal

111
plane was introdu ed; in Experiment AIE2, it was used to nd a main e e t in
agreement with predi tion and with a trend in reported presen e; a reasonable testretest orrelation (.83) was found; and data was olle ted on the orrelation between
the nulling measure and the Embedded Figures Test.
That said, one must point to the limitations of this resear h. In Table 4.7 (page 78)
there is no di eren e between onditions on the nulling measure for 13 mat hed pairs
out of 241 . This implies that the measure was having diÆ ulty distinguishing the
onditions. And this for a meaningful/random omparison, whi h one might expe t
to produ e large di eren es between onditions.
This la k of sensitivity was probably due in part to equipment limitations. The
resolution was set quite low (240x320 pixels) to maintain a 60 frames-per-se ond
update rate, and the FOV, at 48Æ , was somewhat below what is usually onsidered
the threshold for high presen e. Both of these may have introdu ed oor e e ts.
However, the la k of intera tivity inherent to the pro edure used in Chapter 4
may also have played a role. An environment whi h does not support intera tivity
may not tend to \draw people in" with an intensity needed to learly demonstrate
di eren es between onditions. In addition to introdu ing a possible oor e e t, the
la k of intera tivity limits what the pro edure of Chapter 4 an be applied to. While
it is suited to investigating general display fa tors su h as the relative importan e of
FOV and resolution, or measuring the foreground o lusion e e t, intera tive virtual
environments fall outside of its s ope. A possible visual-inertial nulling measure
suited to intera tive environments, based on the \indu ed motion" e e t found in
1 The

distribution is similar for the reported presen e measure. Again from Table 4.7 (page 78),

for 9 mat hed pairs out of 24 there is no di eren e between onditions, for 13 mat hed pairs the
meaningful ondition was rated higher, and in 2 the random ondition was rated higher. This
suggests that the frequent la k of di eren e in the visual-inertial nulling measure data a ross
onditions was due to the nature of the experiment rather than to a spe i la k of sensitivity for
visual-inertial nulling.

112
Pilot Study AIIIP2 (see Appendix D), will be dis ussed in Chapter 8. The underlying
per eptual phenomenon will be dis ussed below in Se tion 7.4.
A possible (if untested) bene t of nulling presen e measures may be a redu ed
an hor e e t (see Se tion 2.3.3). A ost of the an hor e e t is that one an not readily
make omparisons between s ores on onditions whi h were not dire tly ompared in
the same experiment. This makes it diÆ ult to in rementally build knowledge by a
series of disjoint experiments.
As an example of the an hor e e t, onsider the reported presen e ratings in Experiment AIE1 (Table 4.3, page 71) and Experiment AIE2 (Table 4.7, page 78). The
ondition labeled \48Æ " in Experiment AIE1 was identi al to the ondition labeled
\Meaningful" (MRP) in Experiment AIE2. Yet the reported presen e for this ondition from the two experiments is quite di erent. Averaged a ross rst and se ond
reports, the value for the \48Æ " ondition in Experiment AIE1 was 4.8, whereas the
value for the \Meaningful" ondition in Experiment AIE2 was 2.75. This di eren e
is highly signi ant on a two-sample t-test (p

<

.001). The di eren e presumably

arises be ause in Experiment AIE2 the omparison was against a random s ene, whi h
tended to lower over-all presen e ratings. In Experiment AIE1, the omparison was
against two similar onditions of slightly narrower FOV.
An hor e e ts for reported presen e are not surprising. Humans were not evolved
to assign numbers to mental states and have no robust means for doing so. While
there are urrently no data on this, it may be that a nulling presen e measure, being
more deeply-rooted psy hologi ally than self-reported presen e values, may exhibit
mu h more onsisten y a ross experiments.
Both Experiment AIE2 (Table 4.7, page 78) and Pilot Study AIP4 (Table B.2,
page 147) found roughly a fa tor of 10 between-parti ipant di eren e in where the
ross-over amplitude o urred. Similar di eren es were found in Pilot Studies AIP1
and AIP2, using di erent equipment. A fa tor of 10 is a onservative estimate: parti ipants ex eeded the range of measurement in both dire tions. A possibility is that

113
the between-parti ipant variation in the ross-over amplitude re e ts a more sensitive
measure of eld dependen y than has previously been available. Like the EFT and
the rod-and-frame test, the visual-inertial nulling measure requires parti ipants to extra t a signal from a on i ting or distra ting visual pattern. But the visual-inertial
nulling pro edure uses a more ompelling visual stimulus than either of the other
two tests; and, in addition, the visual-inertial nulling pro edure avoids the strong
gravitational ue whi h is present in the rod-and-frame test.
The visual-inertial nulling pro edure might also be used lini ally as a means to
diagnose vestibular damage.
About a third of the parti ipants in Experiment AIE1 did not follow the general
pattern that the 48Æ ondition produ ed higher reported presen e than the other
two onditions. This suggests that for a minority of the parti ipants, the e e t of a
foreground o lusion in reasing presen e may have been stronger than the e e t of
wider FOV in reasing presen e, at least for relatively narrow FOV's.
The proto ol for Experiment AIE1 and Experiment AIE2 alled for reported presen e data to be gathered with the visual and inertial motions ongruent, but with
di erent sinusoidal amplitudes (30Æ /se and 20Æ /se , respe tively). An informal observation is that parti ipants did not seem to be aware of what the relationship was
between the visual and inertial amplitudes, even to the extent of knowing whether
or not they were equal. While visual-inertial phase di eren es are very apparent,
amplitude di eren es are not, at least at the ons ious level.
SSQ data were gathered before and after every session. These data were not
formally analyzed, as they were gathered primarily to make a rough he k that parti ipants were not experien ing serious malaise. Simulator si kness did not appear to
be a serious problem2. The la k of strong symptoms in an experiment whi h involved
a lear visual-inertial sensory on i t may seem surprising. It is probably due to
2 In

a few ases, parti ipants reported feeling better after the experiment than before. This may

have been a bene ial side-e e t of the instru tion to \relax and lose your eyes" between trials.

114
the short duration of exposures to the on i ting stimuli. Exposures tended to be
about a minute long, separated by a rest with eyes losed of one or two minutes while
onditions were hanged. It appeared that parti ipants exposed to on i ting stimuli
for longer periods of time did develop symptoms of simulator si kness.
7.3

Area II: Presen e Manipulations

The following resear h questions were posed at the top of Chapter 5:
1. Can a foreground o lusion in rease reported presen e?
2. Can a performan e measure be found for the e e t of a foreground o lusion?
Experiment AIIE1 and Experiment AIIE2 found the predi ted e e t of a foreground o lusion in reasing reported presen e. While this result is to be expe ted
from the ve tion literature, it appeared to ome as a surprise to most virtual environments resear hers at the time. These experiments served the twin purposes of
supporting the hypothesized link between ve tion and presen e and of pointing out
an appli ation of foreground o lusions to HMD design.
The literature and pilot study reported in Appendix E point to another appli ation
of foreground o lusions to HMD design, besides in reasing presen e. Foreground
o lusions may also be useful to redu e bino ular rivalry for displays with a partial
overlap between the s enes displayed to the two eyes.
Appendix C reports on a series of studies whi h sought a performan e measure
for foreground o lusions, in terms of a spatial orientation task. While two pilot
studies were promising, the full experiment (AIIE3) failed to nd an e e t. Possible
modi ations to Experiment AIIE3 are dis ussed in Appendix C. However, based
on urrent data, it appears doubtful that the foreground o lusion e e t will be
su essfully measured with experiments similar to AIIE3.

115
Given the subje tive strength of the foreground o lusion e e t (as measured in
Experiments AIIE1 and AIIE2) it would be surprising if it ould only be dete ted
by self-report measures. The sear h for a foreground o lusion performan e measure
might pursue a spatial task whi h more learly requires a swit h of the sele ted rest
frame away from the laboratory and into the s ene shown on the display. However,
it is not immediately lear what an appropriate task would be. Alternatively, an
attempt might be made to measure the foreground o lusion e e t with a nulling
measure.
7.4

Area III: Motion Si kness

The following resear h questions were posed at the top of Chapter 6:
1. Can an independent visual ba kground (IVB) in agreement with inertial ues
redu e reported simulator si kness and related side-e e ts?
2. If so, an it do so without redu ing the subje tive impa t of the s ene?
3. Can an IVB be e e tive even when attention is dire ted into the visual foreground, in whi h visual self-motion ues disagree with the inertial ues?
Experiments AIIIE1 and AIIIE2 answered all of these questions in the aÆrmative.
However, these experiments dealt only with a \low-end" system based on the Virtual
i-O i-glasses!. An important open problem is to investigate the usefulness of the IVB
te hnique for high-end systems.
Pilot Study AIIIP2 (see Appendix D) was a very brief investigation of a simple
ba kground grid as an IVB for a high-end driving simulator. It failed to nd a redu tion in simulator si kness. Pilot Study AIIIP2 is reported be ause it serendipitiously
found a possible nulling presen e measure suitable for intera tive environments. The

116
possible measure will be dis ussed in Chapter 8. The below omments on the underlying phenomenon.
Pilot Study AIIIP2 used a simple ba kground grid for the IVB, displayed in the
sky portion of the simulator s ene. The grid was kept xed with respe t to the
laboratory at all times. There was a fas inating e e t in whi h the ba kground grid
appeared to rotate in the opposite dire tion when the CI turned, even though the
ba kground grid was stationary with respe t to the laboratory3. This is known as
\indu ed motion": the apparent motion of a stimulus aused by motion of nearby
stimuli (see Se tion 3.3.2).
While it is usually a motion of the visual ba kground whi h produ es an indu ed
motion of a foreground obje t, the reverse is not unknown. Levine and Shefner [60℄
give the following example: \... onsider a loudy night sky with the moon du king in
and out of the drifting louds. The moon is a tually stationary relative to the louds,
but be ause the louds take up so mu h more room in the visual eld than the moon,
they appear to be stationary while the moon seems to move in the opposite dire tion
from them." (p. 382.)
See Se tion 3.3.1 for a dis ussion of why this indu ed motion may have o urred.
It seems likely that a more ompelling visual ba kground than a simple grid would
serve to redu e simulator si kness. But this remains a topi for future resear h.
7.5

Sele ted Rest Frames and Cognition

Sele ted rest frames have been presented in this dissertation as a purely per eptual
phenomenon, although a e ted by ognitive issues su h as the fo us of attention (see
Se tion 3.3.1). An interesting question is to what extent sele ted rest frames operate
3 For

myself, at least, the impression was that the CI and the ba kground grid were mounted on

independent (not visible) disks, whi h were free to rotate independently. The disk the ba kground
grid was on appeared to slide in the opposite dire tion during turns.

117
on prin iples similar to ognition.
An analogy an be drawn between the operation of the sele ted rest frame and of
hunking. \Chunks" refer to familiar patterns stored in long-term memory. Human
information pro essing makes heavy use of hunking. A omplex pattern of information whi h has been hunked is often easier to pro ess than a smaller amount of
unfamiliar information. An interesting re ent example of this is provided by Lu k
and Vogel [63℄, who mention that
...obje ts de ned by a onjun tion of four features an be retained in
working memory just as well as single-feature obje ts, allowing sixteen
individual features to be retained when distributed a ross four obje ts.
Thus, the apa ity of visual working memory must be understood in terms
of integrated obje ts rather than individual features...
Chunks underlie memory of the environment. Chase and Simon [18℄ found that
while expert hess players are better at remembering meaningful hess positions than
are novi es, the two groups perform equally poorly on random hess positions. Experts hess players (and, presumably, humans in general) remember by breaking the
environment into previously-learned patterns.
The same hunking whi h drives both short-term and long-term memory is also
a tive in visual per eption. There are numerous visual illusions whi h imply that
the per eptual system seeks familiar patterns ( hunks), to the extent that it an be
fooled when the pattern is not entirely there4 .
It seems likely that the nervous system does not merely look for and remember
hunks: it also represents unfamiliar information in terms of its di eren es from
known hunks. It is more eÆ ient to view new information as \almost like su h-andsu h ex ept for..." than it is to analyze new information from s rat h. Viewed in this
4 See,

for instan e, the dis ussion of \illusions dependent on per eptual hypotheses" in Levine and

Shefner [60℄.

118
way, one's set of hunks do not simply de ne a set of isolated points: they de ne a
oordinate system in terms of whi h new information an be represented5 .
This resembles the role hypothesized for the sele ted rest frame. The sele ted
rest frame is hypothesized to serve as the omparitor for spatial judgments. Spatial
judgments are made by des ribing how things relate to (are di erent from) the sele ted
rest frame.
This suggests that the sele ted rest frame an be thought of as the \underlying
hunk" for spatial per eption. Sele ted rest frames may therefore be losely related
to the operation of hunking in ognition.

5 That

is, new information an be thought of as being de ned in terms of its proje tion onto known

hunks, whi h a t as basis ve tors.

Chapter 8

FUTURE RESEARCH
If we knew what it was we were doing, it would not be alled resear h,
would it?
| Albert Einstein
8.1

Introdu tion

The pre eding hapter summarized the dissertation experiments. This hapter
suggests future resear h. It follows the division into 3 areas whi h was introdu ed in
Chapter 3.
8.2

Area I: Presen e Measures

8.2.1 Improved Equipment
As dis ussed in the previous hapter, the visual-inertial nulling measure appeared
to su er from a oor e e t whi h may have been due in part to equipment limitations.
Simply improving the equipment should make the visual-inertial nulling measure better able to dete t di eren es between onditions. This might in lude faster pro essing
to allow for higher resolution; the use of stereo images; a wider FOV HMD; and the
use of (possibly spatialized) auditory ues.

120

8.2.2 Individual Variation
Experiment AIE2 and Pilot Study AIP4 found at least a fa tor of 10 betweenparti ipant variation in the ross-over amplitude. What is the sour e of this variation?
It was suggested in the previous hapter that this variation might re e t a sensitive
measurement of eld dependen y. A se ond possibility is that it re e ts individual
variation in the degree to whi h the parti ipant attended to the visual s ene. Those
who wat hed the s ene more arefully would be expe ted to be more in uen ed by
the ve tion stimulus, whi h would be re e ted in a higher ross-over amplitude.
The relative in uen e of inherent individual di eren es and of variation in attention level ould be assessed by providing a visual task. For instan e, attention level
ould be ontrolled for by having parti ipants push a button every time a red dot
appeared on the s reen1 .

8.2.3 A Possible Indu ed Motion Measure
A primary advantage of the visual-inertial nulling pro edure developed in Chapter 4 is that it fa tors out the strong inertial ues from gravity. However, this visualinertial nulling pro edure has serious drawba ks: it requires the full attention of
the parti ipant and ompletely determining the visual and inertial motion. While
the visual-inertial nulling measure an address display fa tors su h as the in uen e of
FOV, resolution, et ., it is not suited to the study of intera tive virtual environments.
An important question, therefore, is whether less burdensome per eptual measures
are possible whi h preserve the advantages of the visual-inertial nulling measure.
A possiblity for su h a measure is suggested by Pilot Study AIIIP2 (see Appendix D). The \indu ed motion" phenomenon found in Pilot Study AIIIP2 was
1 It

should be noted that this pro edure might introdu e noise into the ross-over data, sin e

parti ipants would be fa ed with a divided attention task. They would need to respond to both
the inertial motion extremes and the appearan e of visual targets.

121
dis ussed in Se tion 7.4. Brie y, a grid in the visual ba kground of the driving simulator whi h was stationary with respe t to the laboratory appeared to rotate in the
opposite dire tion when the CI in the foreground turned.
An interpretation of this phenomenon is that the sele ted rest frame was determined more by the CI than by the ba kground grid or by the (la k of) inertial ues.
The relative motion between the CI and the ba kground grid was therefore per eived
as a motion of the ba kground grid. A ording to the presen e hypothesis, the degree
to whi h the sele ted rest frame is determined by the CI is related to the level of presen e in the CI. This suggests that a per eptual presen e measure may be onstru ted
by measuring the amount of indu ed motion of the ba kground grid.
A possible te hnique for doing so is as follows. Give parti ipants a dial to turn,
and instru t them to adjust the dial so that the ba kground grid appears to be
stationary during turns in the simulator. The dial has the e e t of introdu ing a
ounter-rotation of the ba kground grid to balan e out the indu ed motion. The
magnitude of ounter-rotation needed to make the grid appear to be stationary is the
per eptual presen e measure.
The dial ould be mounted on a navigational devi e for the simulator s ene (e.g.,
a steering wheel) where it would be easy for the parti ipant to adjust. If the level of
presen e is fairly onstant the dial might not require updating frequently, in whi h
ase it would require little attention from the parti ipant.

8.2.4 An hor E e t
See Se tion 7.2 for an example of an an hor e e t in the reported presen e data
from the urrent resear h, and a dis ussion of why the an hor e e t limits the ombination of data a ross experiments. It was suggested in that dis ussion that visualinertial nulling measures might be less prone to the an hor e e t, as they are based
on a more dire t measure of spatial per eption.
The hypothesis that nulling measures (su h as the visual-inertial nulling measure

122
introdu ed in Chapter 4) might be less prone to the an hor e e t is easily testable.
This ould be done by a series of experiments in whi h the same ondition \A" is
ompared to quite di erent onditions. For instan e, Experiment 1 might ompare
ondition \A" to ondition \B", Experiment 2 might ompare ondition \A" to ondition \C", et . The onsisten y of the values obtained for ondition \A" ould then be
ompared a ross experiments. A omparison should be made between the onsisten y
of a nulling measure and a self-report measure for presen e.

8.2.5 Appli ation to Interfa e Issues
The above subse tions have suggested various ways to evaluate or improve the
visual-inertial nulling measure. In the end, however, measures exist to be used. A
nulling measure an and should be applied to systemati ally evaluate the in uen e
of may fa tors, su h as FOV, foreground o lusions, resolution, auditory ues, intera tivity, et .
8.3

Area II: Presen e Manipulations

As dis ussed in Se tion 7.3, foreground o lusions may be useful additions to
HMD's both to in rease presen e at a onstant FOV and to redu e bino ular rivalry
e e ts. This is an option whi h HMD manufa turers might want to onsider more
seriously. There are ergonomi and safety issues asso iated with using a foreground
o lusion whi h would have to be evaluated for ea h HMD design.
The resear h of Appendix C failed to nd a performan e e e t for foreground
o lusions in terms of a simple spatial orientation task. However, a foreground o lusion e e t might be measurable with a nulling measure. The relative e e t on
presen e of a foreground o lusion and wider FOV would be an interesting topi of
study. The results of Experiment AIE1 suggest that for about a third of parti ipants,
the foreground o lusion e e t had as strong or stonger an e e t on presen e as wider

123
FOV, at least using a reported presen e measure over a range of fairly narrow FOV's.
8.4

Area III: Motion Si kness

Experiments AIIIE1 and AIIIE2 suggested that an IVB an redu e simulator
side-e e ts even when attention is for ed into the simulator s ene. However, these
experiments were ondu ted using a \low-end" system. The most important potential
appli ation of the IVB idea would be to \high-end" systems involving wide FOV
displays and intera tivity.
To what degree an IVB might be useful for su h systems, and how it should be
implemented, are open questions. Spe i questions in lude the following.
1. What should the properties of the IVB be?
2. What is the e e t of the IVB on task-performan e?
The rst question raises IVB design issues su h as: what should the FOV, brightness, s ene ontent, et ., be; does it need to be in entral vision or an it be on ned to
the periphery where it would provide less of a distra tion; should it be user-adjustable,
and if so, whi h parameters should vary; whi h depth ues should be used to ause
the IVB to be per eived to be behind the CI; should the CI/IVB mixing be done
opti ally or in software?
The se ond question asks what an IVB osts. It is lear that, in the limit at
least, the IVB te hnique an redu e simulator side-e e ts. In the limit, a strong IVB
simply washes out the CI, avoiding simulator side-e e ts at the ost of rendering
the simulator useless. The question, therefore, is to what degree an IVB an redu e
simulator side-e e ts without degrading task performan e. (It is also possible that
an IVB may improve task performan e, to the extent that it redu es malaise.)

Chapter 9

CONCLUSION
A set o' dull, on eited hashes
Confuse their brains with ollege lasses,
They gang in stirks and ome out asses,
Plain truth to speak;...
| Robert Burns, Epistle to J. Lapraik
9.1

Overview

This dissertation sought to address key problems fa ing the human fa tors of virtual interfa es in a uni ed way. To this end, the RFC was introdu ed as a onvenient
summary of a onsiderable part of the literature on spatial per eption. The RFC was
applied to the problems of understanding, measuring, and manipulating presen e,
and to the alleviation of simulator si kness.
Presen e orrelates with high-level aspe ts of the quality of a virtual interfa e,
in luding its intuitiveness and its ability to onvey meaning. Therefore, a robust
presen e measure supports the systemati development of knowledge about how to
onstru t virtual interfa es. This dissertation introdu ed a presen e measure based
on spatial per eption, possibly at the brain-stem level. By evaluating presen e at
su h a fundamental level the problems asso iated with self-report measures, su h as
range and an hor e e ts, may be redu ed or eliminated.
Simulator si kness poses a riti al problem for virtual interfa es. As virtual interfa es be ome more ompelling, they be ome more able to ause unwanted side-e e ts.

125
To address this problem, a link through the RFC between the spatial per eption literature and the motion si kness literature was exploited. This link was used to suggest
a te hnique for redu ing simulator si kness through manipulations to the visual ba kground.
This dissertation is, of ourse, no more than a beginning. It does not rea h
losure; it seeks to steer resear h in parti ular dire tions. It is based on the belief
that fundamental problems should be approa hed from fundamental prin iples; in
this ase, fundamental prin iples of spatial per eption.
There are many interesting ways in whi h this resear h ould be further pursued.
These have been summarized in the pre eding two hapters. The most pressing are
improvements to the presen e measure; its use for the systemati study of the fa tors a e ting presen e in virtual environments; and the appli ation of independent
visual ba kgrounds to redu ing simulator si kness in high-end virtual environments.
Beyond this basi resear h lies the appli ation of the more sophisti ated virtual interfa es whi h it will engender. In many appli ation domains, well-designed virtual
interfa es provide a means to onfront the omplexity explosion by in reasing the
human- omputer bandwidth. But these are stories whi h remain to be told.

BIBLIOGRAPHY
[1℄ Warp, Ltd. S.S. Vallejo, Varda Landing, Sausalito, CA 94965 USA.
http://www.warp. om/.
[2℄ Consulting Psy hologists Press, In . 3803 East Bayshore Road, P.O. Box 10096,
Palo Alto, CA 94303. tel:1-800-624-1765.
[3℄ International Workshop on Motion Si kness: Medi al and Human Fa tors.
Medi al Resear h Coun il, Human Movement and Balan e Unit. National Hospital for Neurology and Neurosurgery,Queen Square, London, WCIN 3BG, UK.,
1997. See http://www.ion.bpmf.a .uk/ dizzymr /motsi k.html.
[4℄ A.V. Aho, R. Sethi, and J.D. Ullman. Compilers: prin iples, te hniques and

tools. Addison-Wesley Publishing Company, 1986.
[5℄ G.J. Andersen and M.L. Braunstein. Indu ed self-motion in entral vision.

Journal of Experimental Psy hology, 11(2):122{132, 1985.
[6℄ Asso iation for Computing Ma hinery. The Risks Digest: Forum on Risks

to the Publi in Computers and Related Systems.
http:// atless.n l.a .uk/Risks/.

Available online from

[7℄ W. Bar eld and C. Hendrix. The e e t of update rate on the sense of presen e within virtual environments. Virtual Reality: Resear h, Development and

Appli ations, 1(1), 1995.

127
[8℄ W. Bar eld and S. Weghorst. The sense of presen e within virtual environments: a on eptual framework. In Human omputer intera tion: software and

hardware interfa es. Elsevier S ien e Publishers, 1993.
[9℄ A.J. Benson, E.C.B. Hutt, and S.F. Brown. Threshold for the per eption of
whole body angular movement about a verti al axis. Aviation, Spa e and En-

vironmental Medi ine, 60(3):205{213, 1989.
[10℄ D.B. Beringer, R.C. Williges, and S.N. Ros oe. The transition of experien ed
pilots to a frequen y-separated air raft attitude display. Human Fa tors, 4,
1975.
[11℄ B. Berlin and P. Kay. Basi olor terms, their universality and evolution. University of California Press, 1969.
[12℄ A. Berthoz, B. Pavard, and L.R. Young. Per eption of linear horizontal selfmotion indu ed by peripheral vision (linear ve tion): basi hara teristi s and
visual vestibular intera tions. Experimental Brain Resear h, 23:471{489, 1975.
[13℄ F. Bio a. Will simulation si kness slow down the di usion of virtual environments te hnology? Presen e, 3(1):334{343, 1992.
[14℄ T. Brandt, J. Di hgans, and E. Koenig. Foreground and ba kground in dynami
spatial orientation. Per eption and Psy hophysi s, 17:497{503, 1975.
[15℄ G.S. Brindley. Physiology of the Retina and Visual Pathway. Williams and
Wilkens, 1970.
[16℄ A.S. Carlin, H.G. Ho man, and S. Weghorst. Virtual reality and ta tile augmentation in the treatment of of spider phobia: a ase report. Behaviour Resear h

and Therapy, pages 153{158, 1997.

128
[17℄ T.R. Carpenter-Smith, R.G. Futamura, and D.E. Parker. Inertial a eleration
as a measure of linear ve tion: an alternative magnitude estimation. Per eption

and Psy hophysi s, 57(1):35{42, 1995.
[18℄ W.G. Chase and H.A. Simon. The mind's eye in hess. In Visual information

pro essing. New York:A ademi Press, 1973.
[19℄ G.H. Crampton. Neurophysiology of motion si kness. In Motion and spa e

si kness. CRC Press, 1990.
[20℄ J. Di hgans and T. Brandt. Visual vestibular intera tion: E e ts on self-motion
per eption and postural ontrol. In Handbook of sensory physiology (Vol. VIII).
Springer-Verlag, 1978.
[21℄ Dogen. 13th entury Zen tea her. Quoted from Harper's Magazine, Mar h 1997.
[22℄ M.H. Draper. The adaptive e e ts of virtual interfa es: vestibulo-o ular re ex

and simulator si kness. PhD thesis, University of Washington, Seattle, USA,
1998.
[23℄ J.S. Dumas and J.C. Redish. A pra ti al guide to usability testing. Ablex
Publishing Co., 1993.
[24℄ S.R. Ellis. Presen e of mind. Presen e, 5(2):247{259, 1996.
[25℄ J.C. Falmagne. Psy hophysi al measurement and theory. In K.R. Bo , L. Kaufman, and J.P. Thomas, editors, Handbook of per eption and human perfor-

man e, volume 1, pages 1{66. New York:John Wiley and Sons, 1986.
[26℄ Jr. F.P. Brooks. The mythi al man-month. Adison Wesley, 1995.

129
[27℄ D.H. Gelernter. Mirror worlds, or the day software puts the universe in a

shoebox|: how it will happen and what it will mean. Oxford University Press,
1991.
[28℄ J.J. Gibson. The senses onsidered as per eptual systems. Boston:Houghton
Miin, 1966.
[29℄ M.J. GriÆn. Handbook of Human Vibration. A ademi Press, 1990.
[30℄ M.J. GriÆn. Physi al hara teristi s of stimuli provoking motion si kness. In

AGARD Le ture Series. Motion Si kness: Signi an e in Aerospa e Operations
and Prophylaxis, 1991.
[31℄ F.E. Guedry. Psy hophysi s of vestibular sensation. In H.H. Kornhuber, editor,

Handbook of sensory physiology, Vol 6(ii). Berlin:Springer-Verlag, 1974.
[32℄ K.M. Hamilton, L. Kanto, and L.E. Magee. Limitations of postural equilibrium
tests for examining simulator si kness. Aviation, Spa e and Environmental

Medi ine, 60(3):246{51, 1989.
[33℄ D.L. Harm, D.E. Parker, M.F. Res hke, and N.C. Skinner. Relationship between sele ted orientation rest frame, ir ular ve tion and spa e motion si kness. Brain Resear h Bulletin (in press).
[34℄ D.L. Harm, D.E. Parker, M.F. Res hke, and N.C. Skinner. Relationship between sele ted orientation rest frame, ir ular ve tion and spa e motion si kness. In International Workshop on Motion Si kness: Medi al and Human

Fa tors. Medi al Resear h Coun il, Human Movement and Balan e Unit. National Hospital for Neurology and Neurosurgery,Queen Square, London, WCIN
3BG, UK, 1997. See http://www.ion.bpmf.a .uk/ dizzymr /motsi k.html.

130
[35℄ E.C. Haseltine. Vignetting te hnique for improving appearan e of partially
overlapped bino ular displays. U.S. Patent Number 05488508, 1996.
[36℄ T. Hatada, H. Sakata, and H. Kusaka. Psy hophysi al analysis of the sensation
of reality. SMPTE Journal, 89:560{569, 1980.
[37℄ E. He ht and A. Zaja . Opti s. Adison-Wesley, 1979.
[38℄ R. Held and N. Durla h. Telepresen e, time delay and adaptation. In S.R. Ellis,
editor, Pi torial ommuni ation in virtual and real environments. Taylor and
Fran is, 1991.
[39℄ C. Hendrix and W. Bar eld. Presen e within virtual environments as a fun tion
of visual display parameters. Presen e, 5(3):274{289, 1996.
[40℄ C. Hendrix and W. Bar eld. The sense of presen e within auditory virtual
environments. Presen e, 5(3):290{301, 1996.
[41℄ H. Ho man, J. Prothero, M. Wells, and J. Groen. Virtual hess: Meaning
enhan es users' sense of presen e in virtual environments. International Journal

of Human-Computer Intera tions (in press).
[42℄ H.G. Ho man, J. Groen, S. Rousseau, A. Hollander, W. Winn, M. Wells, and
T. Furness III. Ta tile augmentation: enhan ing presen e in virtual reality with
ta tile feedba k from real obje ts. In Pro eedings of the Ameri an Psy hologi al

So iety, 1996.
[43℄ M. Hoshino, M. Takahashi, K. Oyamada, M. Ohmi, and T. Yoshizawa. Body
sway indu ed by 3d images. In IS&T/SPIE Symposium on Ele troni Imaging

S ien e and Te hnology, page 64, 1997.

131
[44℄ I.P. Howard. Human visual orientation. New York:John Wiley and Sons, 1982.
[45℄ I.P. Howard. The per eption of posture, self motion and the visual verti al. In

Handbook of Per eption and Human Performan e. John Wiley and Sons, 1986.
[46℄ I.P. Howard. The vestibular system. In Handbook of Per eption and Human

Performan e. John Wiley and Sons, 1986.
[47℄ I.P. Howard and T. He kmann. Cir ular ve tion as a fun tion of the relative
sizes, distan es, and positions of two ompeting visual displays. Per eption,
18(5):657{665, 1989.
[48℄ J.K. Huang and L.R. Young. Sensation of rotation about a verti al axis with a
xed visual eld in di erent illuminations and in the dark. Experimental Brain

Resear h, 41:172{183, 1981.
[49℄ J.K. Huang and L.R. Young. In uen e of visual and motion ues on manual
lateral stabilization. Aviation, Spa e and Environmental Medi ine, 58(12):1197{
1204, 1987.
[50℄ J.K. Huang and L.R. Young. Visual eld in uen e on manual roll and pit h stabilization. Aviation, Spa e and Environmental Medi ine, 59(7):611{619, 1988.
[51℄ E. Hunt. Will we be smart enough? A ognitive analysis of the oming work-

for e. New York:Russell Sage Foundation, 1995.
[52℄ D.B. Judd. Basi orrelates of the visual stimulus. In S.S. Stevens, editor,

Handbook of experimental psy hology. John Wiley and Sons, 1951.
[53℄ R.L. Katz. The Information So iety: An International Perspe tive. New
York:Praeger, 1988.

132
[54℄ R.S. Kennedy, J.M. Drexler, and K.S. Berbaum. Methodologi al and measurement issues for identi ation of engineering features ontributing to virtual
reality si kness. In IMAGE VII, pages 245{254, 1994.
[55℄ R.S. Kennedy, N.E. Lane, K.S. Berbaum, and M.G. Lilienthal. Simulator si kness questionnaire: an enhan ed method for quantifying simulator si kness.

International Journal of Aviation Psy hology, 3(3):203{220, 1993.
[56℄ V. Klymenko, R.W. Verona, H.H. Beasley, and John S. Martin. Convergent and
divergent viewing a e t luning, visual thresholds and eld-of-view fragmentation in partial bino ular overlap helmet mounted displays. SPIE, 2218:82{96,
1994.
[57℄ E.M. Kolasinski. Simulator si kness in virtual environments. Te hni al report,
United States Army Resear h Institute for Behavioral and So ial S ien es, 1995.
TR 1027.
[58℄ P.B. Ladkin. Software problems with new-generation air-traÆ ontrol enter.

The Risks Digest: Forum on Risks to the Publi in Computers and Related Systems. Available online from http:// atless.n l.a .uk/Risks/19.18.html#subj1.1.
[59℄ W.J.M. Levelt. On bino ular rivalry. The Hauge, Paris, Mouton, 1968.
[60℄ M.W. Levine and J.M. Shefner. Fundamentals of sensation and per eption.
Brooks/Cole Publishing, 1991.
[61℄ N.K. Logothetis, D.A. Leopold, and D.L. Sheinberg. What is rivalling during
bino ular rivalry? Nature, 380(6575):621{624, 1996.
[62℄ J.M. Loomis. Presen e and distal attribution: Phenomenology, determinants

133
and assessment. In SPIE Vol. 1666: Human Vision, Visual Pro essing, and

Digital Display III, pages 590{595, 1992.
[63℄ S.J. Lu k and E.K. Vogel. The apa ity of visual working memory for features
and onjun tions. Nature, 390:279{281, Nov 1997.
[64℄ R. Mal olm, K.E. Money, and P. Andersen. Peripheral vision arti ial horizon
display. In AGARD Conferen e Pro eedings No. 145: Vibration and ombined

stresses in advan ed systems, 1974.
[65℄ G.R. M Millan, E.A. Martin, J.M. Fla h, and G.E. Ri io. Advan ed dynami
seats: An alternative to platform motion? In Pro eedings of the 7th Interser-

vi e/Industry Te hni al Equipment Conferen e (Arlington, VA: The Ameri an
Defense Preparedness Asso iation), pages 193{203, 1985.
[66℄ J.E. Melzer and K. MoÆtt. An e ologi al approa h to partial bino ular-overlap.

SPIE, 1456:124{131, 1991.
[67℄ T. Mergner and W. Be ker. Per eption of horizontal self-rotation: Multisensory
and ognitive aspe ts. In Per eption and ontrol of self-motion, pages 219{263.
Hillsdale, NJ:Lawren e Erlbaum Asso iates, 1990.
[68℄ K. Nemire, R.H. Ja oby, and S.R. Ellis. Simulation delity of a virtual environment display. Human Fa tors, 36(1):79{93, 1994.
[69℄ M. Ohmi. Ego entri per eption through intera tion among many sensory systems. Cognitive Brain Resear h, 5:87{96, 1996.
[70℄ M. Ohmi and I.P. Howard. E e t of stationary obje ts on illusory forward
self-motion indu ed by a looming display. Per eption, 17:5{12, 1988.

134
[71℄ M. Ohmi, I.P. Howard, and J.P. Landolt. Cir ular ve tion as a fun tion of
foreground-ba kground relationships. Per eption, 16(1):17{22, 1987.
[72℄ D.E. Parker. Human vestibular fun tion and weightlessness. Journal of Clini al

Pharma ology, 31:904{910.
[73℄ D.E. Parker. The vestibular apparatus. S ienti Ameri an, 243(5), 1980.
[74℄ R. Paus h, T. Crea, and M. Conway. A literature survey for virtual environments: military ight simulator visual systems and simulator si kness. Presen e,
3(1):344{363, 1992.
[75℄ R. Paus h, J. Snoddy, R. Taylor, S. Watson, and Eri Haseltine. Disney's
aladdin: rst steps towards storytelling in virtual reality. In SIGGRAPH Pro-

eedings, pages 193{203, 1996.
[76℄ E.C. Poulton. The new psy hophysi s: six models for magnitude estimation.

Psy hologi al Bulletin, 69:1{19, 1968.
[77℄ W. Pre ht. Vestibular me hanisms. Annual Review of Neuros ien e, 2:265{89,
1979.
[78℄ J.D. Prothero.

A survey of interfa e goodness measures.

port, University of Washington, 1994.

Te hni al re-

R-94-1. Available online from

http://www.hitl.washington.edu/publi ations.
[79℄ J.D. Prothero.

The politi al s ien e of the internet.

port, University of Washington, 1996.

Te hni al re-

R-96-2. Available online from

http://www.hitl.washington.edu/publi ations/r-96-2.html.

135
[80℄ J.D. Prothero, M.H. Draper, T.A. Furness, D.E. Parker, and M.J. Wells. The
use of an independent visual ba kground to redu e simulator side-e e ts. Submitted for journal publi ation.
[81℄ J.D. Prothero, M.H. Draper, T.A. Furness, D.E. Parker, and M.J. Wells. Do
visual ba kground manipulations redu e simulator si kness? In Pro eedings of

the International Workshop on Motion Si kness, pages 18{21, 1997. Available
online from http://www.hitl.washington.edu/publi ations/index.html.
[82℄ J.D. Prothero and H.G. Ho man.

Widening the eld-of-view in reases

the sense of presen e in immersive virtual environments.
port, University of Washington, 1995.

Te hni al re-

R-95-5. Available online from

http://www.hitl.washington.edu/publi ations.
[83℄ J.D. Prothero, H.G. Ho man, D.E. Parker, T.A. Furness III, and M.J. Wells.
Foreground/ba kground manipulations a e t presen e. In Pro eedings of the

Human Fa tors and Ergonomi s So iety 39th Annual Meeting, 1995. Available
online from http://www.hitl.washington.edu/publi ations.
[84℄ J.D. Prothero, D.E. Parker, T.A. Furness III, and M.J. Wells. Towards a
robust, quantitative measure for presen e. In Pro eedings of the Interna-

tional Conferen e on Experimental Analysis and Measurement of Situation
Awareness. Embry-Riddle Aeronauti al University, 1995. Available online from
http://www.hitl.washington.edu/publi ations.
[85℄ J.T. Reason. Motion si kness: a spe ial ase of sensory rearrangement. Ad-

van ement of S ien e, pages 386{393, 1970.
[86℄ J.T. Reason. Motion si kness adaptation: a neural mismat h model. Journal

of the Royal So iety of Medi ine, 71:819{829, 1978.

136
[87℄ S. Rogers. Per eiving pi torial spa e. In W. Epstein and S. Rogers, editors,

Per eption of Spa e and Motion. San Diego:A ademi Press, 1995.
[88℄ J.M. Rolfe and K.J. Staples, editors. Flight Simulation. Cambridge:Cambridge
University Press, 1986.
[89℄ S.N. Ros oe. Horizon ontrol reversals and the graveyard spiral. CSERIAC

Gateway, 7:1{4, 1997.
[90℄ S.N. Ros oe, S.L. Johnson, and R.C. Williges. Display motion relationships. In
S.N. Ros oe, editor, Aviation Psy hology. Iowa State University Press / AMES,
1980.
[91℄ D.W. S hloerb. A quantitative measure of telepresen e. Presen e, 4(1):64{80,
1995.
[92℄ B. S hneiderman. Designing the User Interfa e. Addison-Wesley, 1992.
[93℄ T.B. Sheridan, editor. Teleroboti s, automation, and human supervisory ontrol.
MIT Press, 1992.
[94℄ T.B. Sheridan. Further musings on the psy hophysi s of presen e. Presen e,
5(2):241{246, 1996.
[95℄ S. Shimojo and K. Nakayama. Real world o lusion onstraints and bino ular
rivalry. Vision Resear h, 30(1):69{80, 1990.
[96℄ Sta . Headmounted displays. VR NEWS, 4(4):21{29, 5 1995.
[97℄ J. Steuer. De ning virtual reality: dimensions determining telepresen e. Jour-

nal of Communi ation, 42(4):73{93, 1992.

137
[98℄ S.S. Stevens. To honor fe hner and repeal his law. S ien e, 133:80{6, 1961.
[99℄ M.M. Taylor. Pest: EÆ ient estimates on probability fun tions. Journal of the

A ousti al So iety of Ameri a, 41(4):782{787, 1967.
[100℄ R. Teghtsoonian. On the exponents in stevens' power law and the onstant in
ekman's law. Psy hologi al Review, 78:71{80, 1971.
[101℄ P. Tinker, R. Azuma, C. Hein, and M. Daily. Driving simulation for rash
avoidan e warning system evaluation. In 29th ISATA, pages 367{374, 1996.
[102℄ M. Treisman. Motion si kness: an evolutionary hypothesis. S ien e, 197:493{
495, 1977.
[103℄ A. Tversky and D. Kahneman. Judgment under un ertainty: Heuristi s and
biases. S ien e, 185:1124{1131, Sep 1974.
[104℄ H.E. von Gierke and D.E. Parker. Di eren es in otolight and abdominal vis era gravi eptor dynami s: inpli ations for motion si kness and per eived body
position. Aviation, Spa e and Environmental Medi ine, pages 747{751, Aug
1996.
[105℄ W. Waespe and V. Henn. The velo ity response of vestibular nu leus neurons
during vestibular, visual and ombined angular a elerations. Experimental

Brain Resear h, 37:337{47, 1979.
[106℄ H. Walla h. The per eption of motion. S ienti Ameri an, Jul 1959.
[107℄ Warp Ltd. VTV Software Development Kit Referen e Manual V. 2.0 (Win95),
1996.

138
[108℄ R.B. Wel h, T.T. Bla kmon, A. Lin, B.A. Mellers, and L.W. Stark. The e e ts
of pi torial realism, delay of visual feedba k, and observer intera tivity on the
subje tive sense of presen e. Presen e, 5(3):263{273, 1996.
[109℄ R.B. Wel h and D.H. Warren. Immediate per eptual response to intersensory
dis repan y. Psy hologi al Bulletin, 88(3):638{667, 1980.
[110℄ Herman A. Witkin, Philip K. Oltman, Evelyn Raskin, and Stephen Karp. A

manual for the embedded gures tests. Consulting Psy hologists Press, In .,
1971.
[111℄ L.R. Young, J. Di hgans, R. Murphy, and T. Brandt. Intera tions of optokineti
and vestibular stimuli in motion per eption. A ta Otolaryngologi a, 76:24{31,
1973.
[112℄ G.L. Za harias and L.R. Young. In uen e of ombined visual and vestibular
ues on human per eption and ontrol of horizontal rotation. Experimental

Brain Resear h, 41:159{171, 1981.

Appendix A

COGE1: THE INFLUENCE OF MEANING ON
PRESENCE
See Se tion 2.2 for a review of presen e. Experiment CogE1 addressed the importan e of presen e, by showing that (in addition to many other fa tors related to the
quality of an interfa e) presen e is linked to the meaningfulness of stimuli. This adds
importan e to the sear h for robust presen e measures des ribed in Chapter 4.
This resear h was led by Dr. Ho man. As I was se ond author on this experiment,
and as its role (for urrent purposes) is only supportive to resear h in Areas I, II and
III, full details are deferred to a journal arti le, in press [41℄.
Title: CogE1: Cognitive In uen es on Presen e. Ba kground: While presen e

is hara teristi of virtual environments, the fa tors in uen ing it, and its onsequen es, remain un lear. Previous resear h has fo used primarily on the omputer
side of the human- omputer interfa e: i.e., display issues su h as FOV and lag. The
urrent experiment studied the human side of the interfa e. We studied the e e t of
meaningfulness on presen e and re all. To the extent that presen e an be demonstrated to be a ognitive phenomenon, it be omes more important to study, be ause
of the higher probability that presen e will be linked to su h things as performan e
and training transfer. Hypothesis: In reasing the level of meaningfulness while
other variables are held onstant in reases the level of presen e. Higher levels of
expertise may orrelate with a stronger e e t for meaningfulness on both presen e
and memory performan e. Methods: Meaningfulness was manipulated by omparing master hess positions with mat hed random positions. Positions were shown in

140
a Division dVisor HMD. Positions were labelled meaningful or meaningless in the
virtual environment for the bene t of the non-players used as ontrols, and to thus
mat h any possible demand hara teristi s between hess players and non-players.
Four levels of hess expertise were used, ranging from those not knowing the rules of
hess to strong tournament players. The \old-new" memory re ognition task measured a ura y in reporting whi h positions had been seen previously. The experiment was performed double-blind, with the experimenter blind to both the hypothesis
and the hess strength of parti ular subje ts. Thirty-three unpaid adult volunteers
parti ipated. Results: Presen e ratings were signi antly higher for meaningful
than meaningless positions for all parti ipants ex ept non-players. No di eren e in
presen e ratings was found between the three levels of expertise above non-players.
On the old-new memory re ognition task, re ognition rates were signi antly higher
(and near-perfe t) for the meaningful positions, but only for the strongest group
of hess players. The meaningful/meaningless intera tion with expertise was signi ant. Con lusions: The results show that the amount of presen e felt by individuals
depends on whether the ontents were meaningful to them, independent of display
parameters. This demonstrates that presen e is related to ognitive, not simply per eptual issues. Only a small level of expertise was needed to produ e a large e e t
of meaning on presen e. The memory re all results are onsistent with Chase and
Simon [18℄, extended to a di erent memory task.

Appendix B
AREA I PILOT STUDIES: VISUAL-INERTIAL NULLING
PRESENCE MEASURES

B.1

Initial Pilot Studies

This se tion des ribes the rst two pilot studies ondu ted on the use of visualinertial nulling as a possible presen e measure. Di erent equipment was used in
these studies than in those des ribed in Chapter 4, or in the next se tion. Pilot
Studies AIP1 and AIP2 made use of a Division ProVision 100 and dVisor HMD to
provide images generated in real-time and syn hronized to the hair motion. This
approa h led to syn hronization problems and a low frame rate. The pro edure of
Chapter 4, whi h made use of a very large, pre omputed image, was introdu ed as
an improvement.
Title:

AIP1: Wide Field-of-View.

Ba kground:

See Se tion 3.3.3 for a general

introdu tion to the use of rest frame on i t as a presen e measure. The urrent study
was an initial investigation of the following questions:

1.

an a per eptual presen e

measure be based on rest frame on i t?; and 2. if so, is the measure losely related
to reported presen e, as implied by the presen e hypothesis? A visual-inertial selfmotion on i t was investigated. Pilot Study AIP1 explored whether the amplitude
of inertial motion (from the external environment) needed to overwhelm on i ting
visual motion ues (from the virtual environment) ould be used as a presen e measure. To avoid the strong inertial ue provided by gravity, the visual-inertial on i t
was restri ted to the horizontal plane.

Hypothesis:

Within subje ts, omparing

two FOV's (40Æ and 105Æ ) in an HMD, the FOV with the higher reported presen e

142
will orrelate with the FOV with the higher inertial amplitude needed to per eptually overwhelm on i ting visual self-motion ues from an HMD.

Methods:

There

were 10 unpaid adult parti ipants (7 male, 3 female). None were familiar with the
hypothesis. Parti ipants were seated in a hair whi h os illated at .1 Hz in yaw (the
horizontal plane). They wore a Division dVisor HMD whi h displayed a s ene of a
kit hen whi h also os illated in yaw at .1 Hz. The virtual environment was generated
in real-time by a Division ProVision 100, and ran at about 10 frames-per-se ond.
Head-tra king was disabled, so that visual position updates o urred only as a onsequen e of the programmed sinusoidal os illations, not due to user a tions. The FOV
was restri ted by pla ing a ir ular bla k o lusion in the virtual s ene in front of
the virtual eye position. The phase angle between the inertial and visual was set
to reate a on i t between the inertial and visual self-motion ues. Parti ipants
were asked to attend to the visual s ene but signal the per eived right/left inertial
extremes of their motion with a slider potentiometer. A ounting pro edure was
used as a distra tor before ea h trial to prevent parti ipants from \lo king in" to the
inertial motion. The inertial amplitude was varied following a PEST pro edure to
dis over the \ ross-over" inertial amplitude at whi h parti ipants swit hed from orre tly indi ating the inertial motion of the hair (inertial dominan e) to inadvertently
signaling the visual motion (visual dominan e). To ontrol for learning e e ts, the
two onditions were run in parallel, alternating on an ABBA pattern. As this was an
exploratory pilot study, the magnitude and s hedule of phase angles was varied a ross
subje ts, although they were mat hed a ross onditions for ea h subje t. Subsequent
to the visual-inertial nulling experiment, reported presen e in the two onditions was
assessed with a questionnaire (see Se tion 3.8.1).

Results:

Even with the rather

rude virtual environment used, it was found that it was not diÆ ult to get parti ipants into visual dominan e. The mean ross-over amplitude a ross onditions and
subje ts was an order of magnitude above the inertial dete tion threshold in the dark.
For four of the parti ipants, there was a fa tor of two or more di eren e between the

143
ross-over values between the two onditions. However, reported presen e di eren es
between onditions did not di er mu h between onditions, and di eren es were not
in lear agreement with the nulling data, even for parti ipants with the largest differen es between onditions on the visual-inertial nulling measure.

Con lusions:

This pilot study established that large visual dominan e e e ts an be reated in the
horizontal plane. That large e e ts were found between onditions on the ross-over
experiment was quite en ouraging for this sort of measure. However, the la k of
onsisten y between the visual-inertial nulling and reported presen e measures left
open the question of whether the two measures are re ording fundamentally the same
psy hologi al phenomenon, as predi ted by the presen e hypothesis.
Title:

AIP2: Orientation.

Ba kground:

Pilot Study AIP1 demonstrated that

visual dominan e ould be easily a hieved in a horizontal os illation experiment using
standard VE equipment, and that large di eren es between onditions ould be measured. However, di eren es in reported presen e were small and not onsistent with
the nulling data. This is a hallenge to the presen e hypothesis. One possibility was
that the nulling measure was more sensitive than reported presen e in AIP1. Perhaps
per eptual di eren es existed whi h were not strong enough to rea h the ons ious
level a essible to the reported presen e measure. If so, the la k of orrelation between the two measures might re e t noise in the reported presen e measure. This
hypothesis ould be tested by strengthening the reported presen e signal. Given the
result of Experiment Cog1 (see Appendix A), a reasonable try for a strong presen e
di eren e was to ompare a meaningful to a meaningless stimulus. Another possible
riti ism of AIP1 is that the ross-over data did not re e t a di eren e in presen e
between onditions, but rather a low-level per eptual fa tor: total area of retinal
stimulation. To ontrol for this, it was desirable to ompare a \meaningful" to a
\meaningless" visual ondition whi h were as alike as possible in low-level per eptual
issues. A andidate manipulation, easily implemented with the available software,
was to ompare the kit hen used in AIP1 in upright (\meaningful") and upside-

144
down (\meaningless") on gurations.

Hypothesis:

The upright ondition should

produ e both higher inertial ross-over and higher reported presen e s ores than the
upside-down ondition.

Methods:

As in AIP1, with the ex eption of the di erent

onditions. There were 10 unpaid adult parti ipants (4 male, 6 female), all naive
to the experimental hypothesis. None had parti ipated in AIP1.

Results:

Unlike

AIP1, no large di eren es were found between onditions on the nulling measure. Nor
were large di eren es found on the reported presen e measure. Nor was there a lear
relationship within subje ts a ross the two measures.

Con lusions:

The la k of a

lear di eren e between the onditions on either measure suggest that upside-down
is not the same as meaningless. The same information is present: it is simply harder
to interpret. For some, the diÆ ulty of interpreting the upside-down s ene de reased
reported presen e somewhat; for others, the added attention required interpret the
upside-down s ene in reased presen e. But the e e ts (on both measures) appear to
be small.
B.2

Pilot Studies Related to Experiment AIE1

The below pilot studies examined the same onditions as Experiment AIE1, but
with a nulling measure rather than reported presen e. See Chapter 4 for a des ription
of the onditions and general methods. The primary result of these pilot studies was
to establish the te hniques used in Experiment AIE2.
Title:

AIP3: Narrow Field-of-View (Nulling Measure I).

Ba kground:

This

experiment applied the visual-inertial nulling measure to the same onditions as Experiment AIE1.

Hypothesis:

The nulling measure should have good test-retest or-

relation. Within subje ts, ross-over data should be onsistent with reported presen e
data a ross onditions, with higher ross-over values orresponding to higher reported
presen e values. A ross subje ts, the average ross-over value should orrelate with
EFT s ores.

Methods:

For general methods, see Se tion 4.2. There were 6 unpaid

145
adult parti ipants (4 male, 2 female) sele ted from parti ipants in Experiment AIE1
(numbered 3, 8, 9, 14, 15 and 16 in Table 4.2). All but one were naive to the hypothesis. In ea h of three sessions, parti ipants were exposed to two of the three
onditions from Experiment AIE1 in alternate trials, seeing ea h ondition twi e over
the ourse of the three sessions. A visual-inertial per eived self-motion on i t was

reated by randomly pi king either a (90Æ ) phase di eren e between the ve tion

and inertial stimulus for ea h trial. Two phase angles were used to redu e possible
learning e e ts. An EFT was administered following one of the three sessions.
sults:

Re-

See Table B.1. Test-retest onsisten y for the ross-over data was poor (.38

orrelation). In view of this low orrelation, indi ating unreliable results, other data
were not formally analyzed.

Con lusions:

The la k of test-retest onsisten y for the

nulling measure as implemented in AIP3 renders it useless. In examining the pattern
of responses, it appeared that there might be a systemati di eren e in the diÆ ulty

of inertial tra king as a fun tion of whi h of the two phase angles was used (90Æ ).
Pilot Study AIP4 was therefore ondu ted with a single phase angle.
Title:

AIP4: Narrow Field-of-View (Nulling Measure II).

Ba kground:

An

examination of the ross-over data from AIP3 suggested that the in onsisten y of
test-retest s ores might be a ounted for by a systemati di eren e in the diÆ ulty
of inertial tra king between the two ve tion-inertial phase angles used. A ordingly,
AIP4 was repeated using only a single phase angle throughout. This raised the possible problem that parti ipants would \ gure out" the relation between the visual
and inertial urves, ons iously or un ons iously, resulting in the task be oming progressively easier over time.

Hypothesis:

As with AIP3.

Methods:

As with AIP3,

ex ept that only a single ve tion-inertial phase angle (90Æ ) was used. There were 4
unpaid parti ipants (3 male, 1 female) sele ted from parti ipants in Experiment AIE1
(numbered 17, 18, 19 and 20 in Table 4.2).

Results:

See Table B.2. The test-retest

orrelation for the ross-over data was .99. Retest ross-over amplitudes were not
lower than initial ross-over amplitudes, and all four parti ipants reported believing

146

Table B.1: Cross-Over Data for Experiment AIP3
Id

16Æ (1)

16Æ (2)

32Æ (1)

32Æ (2)

48Æ (1)

48Æ (2)

EFT

3

6.5

6.5

11.5

6.5

17.5

9

8

8

12.5

7.5

12.5

12.5

2.5

22.5

55

9

2.5

12.5

2.5

17.5

7.5

2.5

7

14

6.5

9

12.5

9

7.5

6.5

15

15

23.5

22.5

12.5

17.5

23.5

27.5

12

16

6.5

2.5

9

7.5

4

12.5

16

9.7(7.5) 10.1(6.9) 10.1(3.9) 11.8(4.9) 10.4(8.3) 13.4(9.7)
Id is the parti ipant number ( onsistent with the numbers in Experiment AIE1). The next 6 olumns give the rst and se ond ross-over
amplitudes (in Æ /se ) for the 16Æ , 32Æ , and 48Æ FOV onditions, respe tively. (These data are the mean of ross-over ranges, as des ribed in
Se tion 4.2.) Higher values indi ate more visual apture. The last olumn gives Embedded Figures Test s ores. The nal row gives the mean
and (in parentheses) standard deviation for ea h olumn.

147

Table B.2: Cross-Over Data for Experiment AIP4
Id

16Æ (1)

16Æ (2)

32Æ (1)

32Æ (2)

48Æ (1)

48Æ (2)

EFT

17

2.5

6.5

9

6.5

9

7.5

32

18

1.5

3

4

4

3

3

10

19

27.5

30

30

30

30

30

18

20

1.5

3

4

2

4

1.5

8

8.25(12.8) 10.6(13.0) 11.8(12.4) 10.6(13.0) 11.5(12.6) 10.5(13.2)
Id is the parti ipant number ( onsistent with the numbers in Experiment AIE1). The next 6 olumns give the rst and se ond ross-over
amplitudes (in Æ /se ) for the 16Æ , 32Æ , and 48Æ FOV onditions, respe tively. (These data are the mean of ross-over ranges, as des ribed in
Se tion 4.2, ex ept where an upper bound was not found. In these ases,
the lower bound of 30Æ /se was used.) Higher values indi ate more visual
apture. The last olumn gives Embedded Figures Test s ores. The nal row gives the mean and (in parentheses) standard deviation for ea h
olumn.

(in orre tly) that the phase angle varied a ross trials. This suggests that a learning
e e t related to using a single phase angle was not a signi ant fa tor. While the
mean ross-over amplitude was lower for the narrowest FOV than for the other two
onditions, the di eren e was too small to be signi ant with available data. There
was a between-subje ts di eren e in ross-over amplitudes of at least a fa tor of ten,
with no omparable di eren e in the reported presen e data from Experiment AIE1.
Con lusions:

A single phase angle (90Æ) proto ol is workable, and learly superior

to a proto ol with both 

90Æ phase angles.

The test-retest orrelation of .99 was

unexpe tedly high, and probably related to the small sample size.

Appendix C
AREA II PILOT STUDIES: FOREGROUND
OCCLUSIONS

C.1

Foreground O lusions In rease Presen e

For a more omplete des ription of Pilot Study AIIP1, see Prothero and Ho man
[82℄.
Title:

AIIP1: Foreground O lusions and Field-of-View.

Ba kground:

A

theoreti al argument based on sele ted rest frames put forward in this dissertatation
suggests that foreground o lusions (as well as FOV) should in uen e presen e. How
are the two manipulations related? If presen e depends heavily on whether the display
ontents are interpreted as being the visual ba kground, it might be that FOV is irrelevant if a foreground o lusion is used to blo k out depth ues at the same or greater
distan e than the display ontents.

Hypothesis:

If a foreground o lusion is in-

stalled, presen e does not depend of FOV. Methods: Thirty-eight parti ipants were
exposed to the intera tive \Sharkworld" environment on a Division ProVision 100
using a dVisor HMD. Exposures were for 2.5 minutes in ea h of two onditions. In
one ondition, the FOV was an unrestri ted 105Æ , with no foreground o lusion. In
the other ondition, the FOV was restri ted with a foreground o lusion to 40Æ (whi h
ould be foviated on) or equivalently 60Æ (whi h ould be seen with peripheral vision).
The order of onditions was ounter-balan ed a ross subje ts. After the experiment,
a written questionnaire was used to measure reported presen e in the two onditions.
Results:

Presen e ratings were analyzed with a non-parametri Wil oxon signed-

rank test. Presen e ratings were signi antly lower in the foreground o lusion than

149
the full FOV ondition (Z = -2.4, p = 0.02).

Con lusions:

FOV apparently has

a positive e e t on level of reported presen e even when a foreground o lusion is
used. For this reason, FOV was ontrolled for in subsequent experiments examining
the e e t of foreground o lusions (AIIE1, AIIE2, AIIP2, AIIP3 and AIIE3).
C.2

Inside-Out Display Pilot Studies

Title:

AIIP2: Inside-Out Displays and Opti al Flow.

Ba kground:

A sen-

sitive measure for ontrol-reversals would be useful as a means to investigate the
possible relationship between presen e and sele ted rest frames, and to measure the
e e t of parti ular manipulations on presen e. AIIP2 examined the sensitivity of a
parti ular ontrol-reversal measure by testing its ability to dete t the in uen e of a
manipulation known to a e t ontrol reversals (the use or absen e of moving textures on the ground representation of an inside-out display).

Hypothesis:

Either

a ura y or rea tion-time (RT) or both should be improved on an inside-out display
roll- orre tion task when a moving texture is present on the ground representation.
Methods:

Twenty-two adults volunteered, none familiar with the hypothesis. The

task onsisted of orre ting an inside-out depi tion of an air raft roll on a omputer
monitor by pressing either the left or right arrow key. A ura y and RT were measured. The independent variables were presen e/absen e of a moving texture of the
ground portion of the display; roll angle; and pit h angle. The roll/pit h ombinations

used were 60Æ /-20Æ, 100Æ /-20Æ, 130Æ /-10Æ and 160Æ /-10Æ . There were two repli ations of ea h roll/pit h ombination for ea h ondition, for a total of 2x2x8 = 32
trials per parti ipant. There was a blo k on moving vs. stati ground representations.
(The blo k was to allow for a \build-up of e e t" in the moving ondition.) Order
of angle ombinations were randomized within ea h blo k. RT-a ura y studies have
a data analysis problem if the two dependent measures vary in opposite ways a ross
onditions. In AIIP2, a ura y was ontrolled for by only analyzing RT's for par-

150
ti ipants who pi ked the orre t roll at least 14/16 times in both onditions. This
was onsistent with preliminary data suggesting that this a ura y level would generally be met, and with the expe tation that any signi an e would be in the RT
data, rather than in a ura y. To maintain ounter-balan ing a ross ondition orders
at the required a ura y threshold, su essive parti ipants were added as needed to
either order.

Results:

In the ondition order with the moving texture ground rep-

resentation rst, 8/8 parti ipants met the a ura y threshold. In the ondition order
with the stati representation rst, only 7/14 parti ipants did so. This di eren e is
signi ant on Fisher's exa t test (p < .02). For those above the a ura y threshold,

there was a strong order e e t for a ura y (p < .003) and a trend towards an order
e e t for rea tion-times (p

< .07) with the se ond ondition performing better in

ea h ase. There was no di eren e on a paired t-test of median RT's between onditions for those above the a ura y threshold. On questionnaire data, 11 parti ipants
reported believing they performed better with the moving texture; 5 indi ated no
preferen e; and 6 stated that the moving texture was a ounter-produ tive distra tor.

Con lusions:

The di eren e in a ura y-threshold between the two ondition

orders, together with the strong order e e t, imply that the stati ondition was
measurably more diÆ ult that the moving texture ondition. This suggested that a
ontrol-reversal measure might be viable, with re nements to redu e the strong order
e e t, whi h may obs ure di eren es between onditions.
Title:

AIIP3: Inside-Out Displays and Foreground O lusions I.

Ba kground:

AIIP2 implied that the ontrol-reversal measure ould distinguish between movingtexture and stati ground onditions. This study applied the measure to another
pair of onditions: foreground o lusion/no foreground o lusion. Modi ations were
made to attempt to redu e the order e e t found in AIIP1.

Hypothesis:

The use of a

foreground o lusion, by making the inside-out display the visual ba kground, should
in rease the likelihood that the ground representation of the inside-out display will
de ne the sele ted rest frame. This should improve performan e on a roll- orre tion

151
task.

Methods:

Similar to AIIP2. Three adult volunteers, all male. One was

familiar with (but did not believe) the hypothesis. The images of a plane and of a
ground representation with a moving texture were displayed on a omputer monitor.
The images were in a ir le, surrounded by a bla k annulus whi h overed the rest of
the monitor. In front of the monitor was mounted an adjustable amera iris at eye
level, surrounded by bla k tarp to remove peripheral ues. The diameter of the iris
was adjustable. In the foreground o lusion ondition, the inner edge of the bla k
annulus was obs ured by the foreground o lusion. In the non-foreground o lusion
ondition, the FOV was expanded so that one ould see part of the bla k annulus.
The monitor was viewed from a distan e of about 22 m. The FOV of the s ene
was about 30Æ . In the foreground o lusion ondition, the FOV was set at about
27Æ . In the non-foreground o lusion ondition, the FOV was about 45Æ , with the
outside 15Æ showing the bla k annulus1 . Several modi ations were made to make the
task more diÆ ult than in AIIP2, to redu e the order e e t. The rst modi ation
was to allow both the ground and the plane to roll, with the task being to nd the
shortest dire tion (left or right) to align the plane with the ground. The se ond
modi ation was to in rease the number of possible angles, sele ting randomly with 

(55Æ, 65Æ, 75Æ, 85Æ, 95Æ, 105Æ,
of ground rotation angles (60Æ , 70Æ ,

repla ement from the set of plane rotation angles
115Æ , 125Æ, 135Æ, 145Æ , 155Æ ); from the set

80Æ , 90Æ , 100Æ, 110Æ , 120Æ , 130Æ , 140Æ , 150Æ, 160Æ ); and from the set of pit h angles
1 For

those interested in repli ating AIIP3 or AIIE3, I should emphasize the importan e of not

providing useful referen e frame ues surrounding the display in the non-foreground o lusion
ondition. In preliminary studies, not reported in this dissertation, the air raft i on was kept
xed and the wide FOV ondition allowed spatial orientation ues from the laboratory. These
studies found a strong e e t against the foreground o lusion ondition. A possible interpretation,
suggested by Dr. Parker, is that one needs to know two orientations to solve the inside-out roll
orre tion problem: the angle of the ground and the angle of the plane. The spatial ues from the
laboratory were in agreement with the air raft i on, whi h made the orientation of the air raft
i on easier to interpret, whi h improved performan e in the non-foreground o lusion ondition.

152 

(0Æ ,

10Æ , 20Æ ). The third modi ation was that a longitudinal design was used

with 8 blo ks of 16 trials in ea h of the foreground o lusion and non-foreground
o lusion onditions. Ea h parti ipant had 2 sessions of 4 blo ks ea h, ondu ted on
separate days. The ondition blo ks followed an ABBA pattern, ounter-balan ed
a ross parti ipants.

Results:

Examining adja ent pairs of trial blo ks with a paired

two-tail t-test on error rate and RT, the error rate was signi antly lower in the
foreground o lusion ondition. (Foreground o lusion: median = 1.2/16, STD =
1.5/16; non-foreground o lusion: median = 1.8/16, STD = 1.5; p
was no di eren e in RT's.

Con lusions:

< .04).

There

The nding that there was a signi ant

di eren e in errors, but not rea tion-times, was onsistent with AIIP2. A possible
interpretation is that parti ipants thought of the two onditions as being identi al,
and therefore responded at the same rate. The di eren e between the two onditions
therefore showed up in the error rate. This interpretation is onsistent with the
fa t that none of the parti ipants reported feeling a di eren e in their performan e
between onditions.
Title:

AIIE3: Inside-Out Displays and Foreground O lusions II. Ba kground:

AIIP3 found an e e t of foreground o lusion to redu e the error rate for a roll orre tion task. However, this was based on a longitudinal study of only 3 parti ipants. AIIE3 sought to repli ate this result with a larger sample size.

Hypothesis:

The use of a foreground o lusion, by making the inside-out display the visual ba kground, should in rease the likelihood that the ground representation of the inside-out
display will de ne the sele ted rest frame. This should improve performan e on a roll orre tion task.

Methods:

Nine adults volunteered (6 male, 3 female). None were

familiar with the hypothesis. The pro edure was identi al to AIIP3, ex ept that a
study of the error distribution from AIIP3 was used to bias the trials towards the
most diÆ ult ases. (This was done with the intent to in rease the low error rate from
AIIP3, and thus to allow a di eren e between the onditions to show more learly.)
In AIIP3, The trials in whi h plane and ground had opposite rolls produ ed 80% of

153
the errors. There were twi e as many errors in pit h up as pit h down onditions (see
Figure C.1. To try to in rease the error rate, therefore, the random sele tion of angles
was biased 80% towards opposing rolls, and 80% towards pit h 

0Æ .

As in AIIP3,

parti ipants were run through 16 blo ks of 16 trials. As there were 9 parti ipants

in AIIE3, the below results summarize 9x16x16=2304 trials.

Results:

Examining

adja ent pairs of trial blo ks with a paired two-tail t-test on error rate and RT, no
signi ant di eren e was found between the foreground o lusion and non-foreground
o lusion onditions on either error rate (Foreground o lusion: median = 1.6, STD =
2.0; non-foreground o lusion: mean = 1.7, STD = 2.4) or on median RT in se onds
(Foreground o lusion: median = .29, STD = .18; non-foreground o lusion: median
= .28, STD = .18). The possibility was onsidered that an e e t existed in the early
blo ks whi h was obs ured by a performan e asymptote in later blo ks. However, no
di eren e between onditions was found in the rst 8 blo ks, nor in the rst 4 blo ks.
Con lusions:

It is noteworthy that the angle sele tion pro edure des ribed above

did not su eed in in reasing the error rate from AIIP3 to AIIE3. This suggests that
the task is overly simple, and that a performan e asymptote is qui kly rea hed. If
so, a better approa h than used here might be to alternate trials with and without
the foreground o lusion, rather than taking blo ks of 16 trials in ea h ondition.
This was not done in AIIE3 for two reasons: the diÆ ulty of qui kly re-adjusting
the apparatus, and to allow for a \build-up of e e t" within ea h ondition blo k. A
further try (using the inside-out display metaphor) would be to use parti ipants with
no training on inside-out displays and re ord their intuitive response for orre ting a
roll as a fun tion of the presen e or absen e of a foreground o lusion. One would
hypothesize (if foreground o lusions do a e t the sele ted rest frame) that there
would be a stronger tenden y to mis-interpret inside-out displays in the absen e of a
foreground o lusion.

154

Pitch Down

Pitch Up

Pitch Level
Figure C.1: Inside-Out Display Pit h Representations

Appendix D
AREA III PILOT STUDIES: MOTION SICKNESS

Title:

AIIIP1: Undisturbed Postural Stability.

Ba kground:

Experiment AI-

IIE1 found an in rease in ataxia (measured in terms of stan e breaks from the Sharpened Romberg position) from the rst to the third minute when exposed to a ir ular
ve tion stimulus. The urrent pilot study measured the ba kground level of stan e
breaks without the ve tion stimulus.

Hypothesis:

In the absen e of a ve tion stim-

ulus, few stan e breaks o ur under the proto ol of Experiment AIIIE1. Nor do the
number of stan e breaks in rease signi antly over a period of 3 minutes.

Methods:

Eight parti ipants were asked to follow the proto ol of Experiment AIIIE1 for one
3{minute session, while wearing the Virtual i-O HMD but without being exposed to
moving visual stimuli. Five of these parti ipants had their eyes open, simulating the
see-through ondition. Three had their eyes losed, simulating the o luded ondition.

Results:

Seven of the 8 had no stan e breaks at all. The 8th, with eyes losed,

had 3 stan e breaks in the rst minute and 1 in the third minute.

Con lusions:

The

stan e breaks found in Experiment AIIIE1 were due to the ve tion stimulus. The
in rease in stan e breaks from the rst to third minutes in Experiment AIIIE1 was
due to a build-up of e e t, rather than to fatigue.
Title:

AIIIP2: Independent Visual Ba kground III (high-end).

Ba kground:

Experiments AIIIE1 and AIIIE2 report initial ndings that an IVB an be useful
for redu ing simulator side-e e ts for low-end systems. The urrent pilot study investigated whether the same is true for a high-end driving simulator, in whi h the
nauseogeni stimulus is mu h stronger.

Hypothesis:

Providing an IVB onsistent

with the inertial rest frame may redu e simulator side-e e ts, even when the simula-

156
tor's ontent-of-interest (CI) is not onsistent with the inertial rest frame.

Methods:

Eight subje ts drove gure-eights in the Hughes Resear h Laboratories driving simulator1 for 10 minutes at simulated speeds of 15-30 miles-per-hour. Inertial ues were
not used. In separate onditions, a gray grid with about 2Æ spa ing was either not
present, appeared as an independent visual ba kground (only in the sky, with elements from the simulator s ene sweeping over and o luding the grid during turns) or
(sin e it was an easy manipulation to try) in the foreground, as a mesh overlaying the
entire simulator s ene. In all ases, the grid was stationary with respe t to the laboratory. The experimental design was between subje ts, to avoid demand hara teristi s.
Simulator si kness questionnaire [55℄ and ataxia data were re orded.

Results:

The

resear h time available was too short to rea h de nitive on lusions. Informal observations in luded the following: 1. The ba kground grid did not appear to redu e
simulator si kness. 2. The ba kground grid was often per eived as ounter-rotating
in the opposite dire tion during turns, despite the fa t that the ba kground grid was
stationary with respe t to the laboratory. 3. A foreground grid did appear to be
somewhat e e tive in redu ing reported simulator si kness, perhaps by breaking up
the s ene and thus redu ing its believability.

Con lusions:

The \indu ed motion" of

the ba kground grid suggests that the sele ted rest frame was determined by the CI,
not the IVB. In a ordan e with the presen e hypothesis (Se tion 3.3.3), measuring
the amount of indu ed motion of the ba kground grid may provide a useful per eptual measure for the degree of presen e in the CI. Unlike the \ ross-over" measure
of Chapter 4, the \indu ed motion" measure is potentially suitable for intera tive
environments. Further, the \indu e motion" measure may provide only a minimal
extra load on the parti ipant. See Chapter 8 for a dis ussion.

1 See

Se tion 3.7.2 for a des ription of the simulator.

Appendix E
FOREGROUND OCCLUSIONS AND BINOCULAR
RIVALRY

The below literature review and pilot study is labelled \BRP1" in Se tion 3.5.
This se tion di ers from the rest of the dissertation in that it is not dire tly
related to presen e or to the RFC. It is in luded be ause of its pra ti al importan e,
and be ause it points to another important use for foreground o lusions in HMD's,
besides in reasing the sense of presen e (see Se tion 3.3.4 and Chapter 5)1 . The
dis ussion is primarily based on resear h by others and on theoreti al arguments.
Only informal new data is presented here.
HMD's typi ally provide a separate s reen for ea h eye. In the simplest ase, there
is a total overlap between the images presented in the two s reens. That is, ea h s reen
shows the same s ene, although from a slightly di erent angle orresponding to the
distan e between the two eyes.
However, there is a strong temptation in HMD design to only partially overlap
the images from the two eyes. In this on guration, only the entral 40Æ or so of
the FOV is shared between the two displays. Outside of this region, ea h eye sees a
peripheral region whi h the other eye an not see.
The advantage of partial overlap is that it allows a wider FOV to be seen with
the same equipment. Furthermore, sin e bino ular vision is only e e tive in the
entral region where both eyes an onverge easily, there should in prin iple be little
1 The

appli ation des ribed here of foreground o lusions to the bino ular rivalry problem was

suggested by Furness (personal ommuni ation).

158
per eptual ost to partial overlap.
The disadvantage of partial overlap is an unfortunate bino ular rivalry e e t [66,
56, 35℄. The shared bino ular region is anked on both sides by mono ular regions.
At ea h boundary between the mono ular and bino ular regions, one eye sees the
nasal (inner) edge of a s reen, and the other eye sees the mono ular ontinuation of
the s ene. Thus, the two eyes provide di erent information about the same point in
the visual eld.
This presents the brain with a on i t: what is \really" out there? The edge of a
s reen (as indi ated by one eye) or the s ene displayed to the other eye? The on i t
tends to be resolved in favor of the edge of the s reen. One per eives two dark bands,
orresponding to the nasal edge of ea h s reen, in the shared visual eld. When the
boundaries of the s reen are ir ular this e e t is known as \luning".
Why is the on i t resolved in favor of the edge of the s reen? Why does one
not instead per eive a ontinuous s ene, with the display edges suppressed? This
question is usually addressed in terms of ontrast gradients. Levelt [59℄, for instan e,
did an extensive and quantitative study of bino ular rivalry. In general, if on i ting
images are presented to the two eyes, the one with the higher ontrast gradient is
the one per eived (weighted, to some degree, by eye dominan e). In the ase of an
HMD, the edge of the s reen is usually dark bla k, whereas the s reen itself is brightly
illuminated. This reates a very strong ontrast, whi h per eptually dominates the
ontinuous s ene viewed in the other eye. Consequently, the edge of the s ene is
per eived, resulting in the luning e e t.
A method to ameliorate the luning e e t is therefore to redu e the ontrast gradient at the edge of a s reen. Haseltine [35℄ re ommends a omplishing this by
pla ing an aperture stop near the front surfa e of ea h eyepie e of an
HMD so that the bino ular boundaries of the left and right elds of view
are substantially out of fo us. The aperture stop is out of fo us, and

159
thus relatively in onspi uous, be ause the observer is fo using on distant
virtual images, and also be ause the aperture stop may be loser to the
eye than the shortest distan e at whi h the eye is able to fo us.
Lighting the boundary of the s reen, so that the s reen fades o slowly into bla kness, might also be onsidered.
It is ertainly true that ontrast gradients play a role in bino ular rivalry, and
hen e that redu ing ontrast gradients at the edge of the s reen should redu e the
luning e e t. However, a di erent approa h to the luning problem is suggested by
thinking of bino ular rivalry as a high-level phenomenon, asso iated with ompeting
per eptual interpretations, rather than as a low-level phenomenon asso iated with
ompeting mono ular stimuli.
Logothetis et al. [61℄ mention that \bino ular rivalry is thought to re e t ompetition between mono ular neurons within the primary visual ortex. However, neurons
whose a tivity orrelates with per eption during rivalry are found mainly in higher
orti al areas, and respond to input from both eyes. Thus rivalry may involve ompetition between alternative per eptual interpretations at a higher level of analysis."
They des ribe an experiment in whi h alternating images ompete independently of
the eye from whi h they are dete ted.
Similarly, Shimojo and Nakayama [95℄ note that real world o lusions result in
unpaired regions: the orresponding parts of the two eyes re eive di erent images in
the region of the o lusion. \The authors report a demonstration and experiments to
show that opto- geometri ally `valid' unpaired regions are seen as ontinuous with the
rear plane and es ape intero ular suppression, whereas `invalid' unpaired regions are
per eived as loser and are suppressed vigorously. An additional experiment indi ates
that the results annot be understood in terms of orresponden e solving, but require
neural me hanisms that embody real-world o lusion onstraints."
These observations suggest that the luning e e t does not arise from on i ting

160
bino ular ues per se, but rather from on i ting per eptual interpretations. (This
bears a resemblan e to the interpretation of motion si kness put forward in Se tion 3.3.5.) From this point-of-view, the fundamental problem is not that there is a
sharp ontrast at the nasal s reen boundary whi h does not orrespond to the s ene
in the other eye. The problem is that the per eptual system an not nd a onsistent
interpretation of these disparate stimuli.
One might summarize the problem as follows.
1. In onsistent images are provided to paired regions of the two eyes. (S reen
boundary in one eye, visual s ene in the other.)
2. The per eptual system has no onsistent interpretation whi h explains this dis repan y. Consequently, the problem an not be resolved smoothly.
3. The more ompelling stimulus should \win" and be pla ed in the per eived
visual eld, overwhelming the less ompelling stimulus.
4. The more ompelling image is the s reen boundary. (That is, the s reen boundary has a higher ontrast gradient. Possibly it would be useful to formulate this
information-theoreti ally.)
5. Consequently, the s reen boundary is pla ed in the per eived visual eld, resulting in the luning e e t.
In keeping with Shimojo and Nakayama [95℄, one might expe t that providing
the nervous system with an e ologi ally valid interpretation of why the in onsisten y
o urs would remove the luning e e t. An e ologi ally valid interpretation an be
provided simply by supplying a foreground o lusion mounted lose to ea h eye individually whi h blo ks the view of the nasal s reen boundary. The s reen boundary
is in e e t moved lose to ea h eye. Sin e the boundary is lose to ea h eye, it would

161
not be expe ted to be visible by the other eye. Consequently, there is an e ologi ally
valid interpretation on erning why the boundary is visible in one eye but not in the
other. One might therefore expe t the luning e e t to be redu ed or removed entirely.
Given an HMD with partial overlap, this hypothesis an be tested quite easily.
My own observations were made with a Division dVisor (see Se tion 3.7.3) and the
bla k ap of a Write Brothers Papermate pen (medium point). Holding the ap of
the pen under the HMD lose to the eye, in su h a way as to blo k the nasal edge of
the s reen, greatly redu es or eliminates the luning e e t.
The olor of the pen ap was hosen to mat h the bla k edge of the s reen (and
hen e the ontrast gradient). However, the spatial frequen y ould not be ontrolled.
That is, a pen ap lose to the eye is blurrier than the more distant edge of the
s reen, and onsequently the e e tiveness of the pen ap ould be attributed to a
spatial frequen y e e t, rather than to the e ologi al argument2 .
It is an open question, so far as I know, but I nd the e ologi al interpretation more
plausible than the spatial frequen y interpretation. The e ologi al interpretation
addresses a real problem whi h the nervous system must solve ( reating a single
onsistent per eived visual eld from bino ular input). Furthermore, the e ologi al
argument is onsistent with re ent ndings, des ribed above, that bino ular rivalry
is a fairly high-level phenomenon.
From the point-of-view of pra ti al HMD design, however, it does not mu h matter
whether one favors the spatial frequen y or the e ologi al interpretation. In addition
to presen e onsiderations, des ribed in Chapter 5, the luning e e t provides a se ond
reason for mounting foreground o lusions in HMD's.

2 Dr.

Patterson, a visual psy hologist at Washington State University spe ializing in stereovision,

has taken the \spatial frequen y" position in our own dis ussions. Ironi ally, I am in debt to him
for the Shimojo and Nakayama [95℄ referen e.

Appendix F
SIMULATOR SICKNESS TERMINOLOGY

There is an ambiguity in the use of the term \simulator si kness". In informal
usage, \simulator si kness" tends to refer to the generi experien e of feeling si k as a
result of exposure to omputer-generated stimuli. However, it is frequently used in a
more restri ted sense, as in luding only the si kness aused by poor simulations. For
instan e, Paus h et al. [74℄ mention that \The term simulator si kness is typi ally
used to refer to si kness aused by the in orre t aspe ts of the simulation, not si kness
aused by a orre t simulation of a nauseating experien e, su h as a turbulent airplane ight." Elsewhere in the same spe ial issue on simulator si kness, one may nd
\simulator si kness" used in the more generi sense. For instan e, in the pre eding
arti le Bio a [13℄ states that \Simulator si kness is the term that has been atta hed
to a host of symptoms asso iated with visual and vestibular disturban es that resemble motion si kness." The generi usage of \simulator si kness" is impli it in the
title of the spe ial issue, \Spotlight On: Simulator Si kness" ( overing all si kness
symptoms indu ed by simulators).
To take another example of the generi usage, Kennedy et al.'s widely-used \Simulator Si kness Questionnaire" [55℄ re ords motion si kness symptoms.
It appears that there are three ideas present for whi h only two terms are in
wide-spread use. The best solution is to introdu e a third term. The three ideas are:
1. The generi feeling of si kness resulting from exposure to a omputer-generated
spa e.
2. The omponent of \1" whi h is inherent to the stimulus itself, and whi h would

163
be present even if the simulation were a perfe t representation of the real world.
3. The omponent of \1" whi h results from an imperfe t simulation, for instan e
due to lag, poor inter-o ular adjust, poor resolution, et .
There is general agreement that \2" should be referred to as \motion si kness".
The problem lies with \1" and \3". Both are important ideas, and the term \simulator
si kness" tends to os illate between them depending on the topi of dis ussion. For
our own work, Mark Draper and I at the HITL found it onvenient to use \simulator
si kness" to refer to \1", and to introdu e the term \interfa e si kness" to refer to
\3". \Simulator si kness" is thus used in the generi sense (\1") in this dissertation.

Appendix G
SIMULATOR SICKNESS QUESTIONNAIRE

The Simulator Si kness Questionnaire (SSQ) introdu ed by Kennedy et al.
[55℄ was used as a measure in the simulator si kness experiments of Chapter 6. The
symptoms used, and their weightings, are given in Table G (adapted from [55℄). The
SSQ is based on three omponents: nausea, o ulomotor problems, and disorientation.
These an be ombined to produ e a total SSQ s ore, as des ribed in Table G.

165

Table G.1: Computation of SSQ S ores
Weight
SSQ Symptom

Nausea O ulomotor Disorientation

General dis omfort

1

1

0

Fatigue

0

1

0

Heada he

0

1

0

Eyestrain

0

1

0

DiÆ ulty fo using

0

1

1

In reased salivation

1

0

0

Sweating

1

0

0

Nausea

1

0

1

DiÆ ulty on entrating

1

1

0

Fullness of head

0

0

1

Blurred vision

0

1

1

Dizzy (eyes open)

0

0

1

Dizzy (eyes losed)

0

0

1

Vertigo

0

0

1

Stoma h awareness

1

0

0

Burping

1

0

0

Parti ipants report the degree to whi h they experien e ea h of the above
symptoms as one of \None", \Slight", \Moderate" and \Severe". These
are s ored respe tively as 0, 1, 2 and 3. To ompute the s ale s ores
for ea h olumn, the reported value for ea h symptom is multiplied by
the weight in ea h olumn and then summed down the olumns. The
total SSQ s ore is obtained by adding the s ale s ores a ross the three
olumns and multiplying by 3.74. Weighted s ale s ores for ea h olumn
individually an be found by multiplying the \Nausea" s ale s ore by 9.54;
the \O ulomotor" subs ale by 7.58; and the \Disorientation" subs ale by
13.92.

VITAE

Jerrold Douglas Prothero

Human Interfa e Te hnology Laboratory
University of Washington
Seattle, WA USA 98105
protherohitl.washington.edu
HITL: 1 (206) 616-1437
1 (206) 521-9660
CITIZENSHIP

Canada and the United Kingdom.
Resident Alien, United States.
EDUCATION

Ph.D. Me hani al Engineering, 1998. University of Washington.
Dissertation title: The Role of Rest Frames in Ve tion, Presen e and Motion Si kness.
M.S.E. Industrial Engineering, 1993. University of Washington.
Thesis title: The Treatment of Akinesia Using Virtual Images.
This thesis was the rst study of a pioneering te hnique for redu ing Parkinson's
disease symptoms using visual displays. The te hnique has sin e been the frequent
topi of televised do umentaries.
B.S. Physi s (with honors), 1986. University of Washington.
B.S. Computer S ien e, 1986. University of Washington.
ACADEMIC EMPLOYMENT

Resear h Assistant/Asso iate, 1991-1998. University of Washington.
OTHER DISTINCTIONS

167
Top 0.5% on SAT and PSAT (U.S. pre- ollegiate tests).
United States and International (FIDE)-rated hess master.
RESEARCH INTERESTS 

Motion si kness and simulator si kness. 

Psy hophysi al measures for presen e. 

Ve tion. 

Use of motion ues to redu e Parkinson's disease symptoms. 

Visualization of omplex information. 

Online ommer e and ommunities. 

So ial impli ations of te hnology.

PUBLICATIONS

Ho man, H, Prothero, J, Wells, M & Groen, J. Virtual hess: meaning enhan es
users' sense of presen e in virtual environments. International Journal of HumanComputer Intera tion. In press.
Prothero, J & Parker, D. (1998). A Uni ed Approa h to Presen e and Motion Si kness. A epted book hapter for L. Hettinger and M. Haas (Eds). Psy hologi al
Issues in the Design and the Use of Adaptive, Virtual Interfa es.
Prothero, J, Draper, M, Furness, T, Parker, D & Wells, M. The use of an independent
visual ba kground to redu e simulator side-e e ts. Submitted journal arti le.
Prothero, J, Draper, M, Furness, T, Parker, D & Wells, M (1997). Do visual ba kground manipulations redu e simulator si kness? Presented at the International
Workshop on Motion Si kness, May 26-28, 1997. HITL Te hni al Report, R-97-12.
Prothero, J (1997). Rest frames and `Class A' presen e measures. Abstra t and
poster, 41st Annual Meeting of the Human Fa tors and Ergonomi s So iety (refereed).

168
Draper, M, Prothero, J, Viirre, E (1997). Physiologi al adaptations to virtual interfa es: results of initial explorations. Abstra t and poster, 41st Annual Meeting of
the Human Fa tors and Ergonomi s So iety (refereed).
Prothero, J (1996). The politi al s ien e of the internet. HITL Te hni al Report
R-96-2.
Prothero, JD (1996). Annotations from the 1996 Washington State Chess Championship. Northwest Chess.
Prothero, J (1996). Three queen knight pawns. Northwest Chess.
Prothero, J (1996). An adventure with the Frenkel Fren h. Northwest Chess.
Prothero, J, Ho man, H, Parker, D, Furness, T, & Wells, M (1995). Foreground/ba kground
manipulations a e t presen e. Paper presented at the 39th Annual Meeting of the
Human Fa tors and Ergonomi s So iety (refereed).
Prothero, J, Parker, D, Furness, T & Wells, M (1995). Towards a robust, quantitative
measure for presen e. In Pro eedings of the Conferen e on Experimental Analysis and
Measurement of Situation Awareness, 359-366.
Prothero, J & Ho man, H (1995). Widening the eld-of-view in reases the sense of
presen e in immersive virtual environments. HITL Te hni al Report R-95-5.
Weghorst, S, Prothero J, Furness, T, Anson, D, & Reiss, T. (1994). Virtual images
in the treatment of Parkinson's disease akinesia. Medi ine Meets Virtual Reality II,
San Diego, California.
Prothero, J (1994). A Survey of interfa e goodness measures. HITL Te hni al Report
R-94-1.
Emerson, T, Prothero, J & Weghorst, S (1994). A resour e guide to VR in medi ine.
Arti ial Intelligen e in Medi ine. 6:335-49, 1994.
Emerson, T, Prothero, J & Weghorst, S (1994). Medi ine and virtual reality: A guide
to the literature. HITL Bibliography B-94-1.
Prothero, JD & Prothero, JW (1985). TOPPER, a software pa kage in FORTRAN
for s aling studies. International Journal of Biomedi al Computing. 17(3-4):185-191.
INTELLECTUAL PROPERTY

Sole prin ipal developer (1995). Invention dis losure \Devi es and te hniques whi h
in rease the sense of presen e or fa ilitate a rest frame swit h and appli ations
thereof." With T. Furness and M. Wells.
SELECTED PUBLISHED ACKNOWLEDGMENTS

169
Alstyne, M (1997). Higher edu ation's information hallenge. Exe utive Strategies
2(4), May, 1997 pp. 1-14. (A knowledged for dis ussions relating to my essay \The
Politi al S ien e of the Internet.")
Ma edonia, M, Zyda, M, Pratt, D, Barham, P & Zeswitz, S (1994). NPSNET: a
network software ar hite ture for large-s ale virtual environments. Presen e 3(4):265287. (A knowledged for an unpublished survey of software supporting distributed
virtual environments.)

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.