You are on page 1of 2


A System to Study 3D Perception for Diagnosing Schizophrenia and Assessing Treatment Results
Jordan T. Ash, James M. Hughes, Thomas V. Papathomas
Department of Biomedical Engineering and Laboratory of Vision Research Rutgers University, New Brunswick, New Jersey, 08854

AbstractIndividuals with schizophrenia (SZ) perceive various visual stimuli differently than healthy controls. This is especially true of the hollow-face illusion. We have created a virtual rendering system that exploits this well-known fact. We also explain the addition of a head-tracking feature that provides a more realistic spatio-temporal interaction for our virtual-reality apparatus. Our system is capable of quantifying the amount of time a person perceives the illusion and affords a comparison between SZ patients and controls, thus offering a diagnostic tool for SZ.
Figure 1. Left panel: All masks (a-e) face left in this top view. The vertical rotation axis is at circles center. Mask d rotates around an axis located at the tip of the nose. Right panel: Diagram specifying the notation. Origin is at the circles center.

I. I NTRODUCTION Previous studies [1, 2] have shown that individuals with SZ rely on data-driven processes for perception. Consequently, they under-utilize concept-driven processes that rely on past experiences. This is particularly clear when people with SZ observe hollow-face illusion (HFI) stimuli [3]. In this study we discuss a virtual-reality (VR) system that we developed to create experiments that are impossible to achieve with realworld (RW) stimuli. We show that healthy controls respond similarly to VR and RW stimuli, even when relative motion cues are accounted for. We used the concept of depth reversal [3, 4] to measure the time observers perceive the HFI in a pilot study. We offer a proposal to use virtual HFI stimuli to diagnose SZ, and to assess the efcacy of treatment methods.

Our software parametrization:





< x, y, z > = < R cos(), R sin(), 0 >


II. M ETHODS Eight nave subjects participated in the pilot VR study. Ten nave subjects participated in the pilot RW study. None of the participants were known to have any cognitive abnormalities. The VR stimulus was a mask created with the FaceGen [5] tool for human head object le generation. The head was then modied in 3D Studio Max [6] to create a hollow mask, mapped with lifelike features on both sides using three highresolution image les: one of the face (1024x1024 pixels), and two images of the left and right eye (256x256 pixels each). It was presented on a 15-inch LCD monitor supplied with a laptop computer. This mask was loaded as an object le into our virtual environment using the MOGL features of PsychToolBox [7, 8, 9]. In our pilot study, we varied the distance between the nose-tip and the vertical axis of rotation, as shown in Fig 1.

This parametrization has many advantages, most notably the ability to easily change whether the mask is facing inwards or outwards by simply changing the sign of R. The mask was rotated by incrementing by d in each iteration; the sign and magnitude of d allowed for rotation direction and angular speed to be changed. The amount of time spent in the illusion was tracked, with the user pressing the right or left arrow keys, corresponding to whether he or she perceived the mask as rotating clockwise or counterclockwise as seen from above (same as RW experiment). We assessed the strength of the illusion by the ratio J/M , where J and M are themselves ratios. J is the ratio of iterations spent in the illusion and total iterations. M is the length of the time-interval possible to spend in the illusion (i.e., when the concave side was facing the observer) divided by the duration of the entire trial. J is given by (2), where R is the distance from the mask to the center of rotation and D is the distance from the center of rotation to the observer. M= 1 arcsin(R/D) 2 (2)

A sum (+) corresponds to a trial in which the mask is facing away from the axis of rotation, while a difference () corresponds to the mask facing towards the axis of rotation. Our RW system mirrored the VR system closely, and involved a similarly varying rotational axis. The same formulas were applied for statistical analysis. A painted plastic mask, afxed

to a motorized turntable was used as the stimulus, and a MATLAB [10] program was used to record intervals during which observers witnessed the illusion. Each observer saw the stimulus in all possible combinations of rotation radius, mask direction, and rotation direction. The order of these conditions was arranged pseudo-randomly. d was kept at .2 radians per iteration, resulting in 10 iterations per full rotation. The viewing distance was kept at 25 units, to allow for the mask to t comfortably on the screen during rotation. In each trial, the mask rotated a total of four revolutions (8 radians), justifying the use of (2) which only applies for integer multiples of radians. Although not utilized in the pilot study, our software is also capable of real-time head tracking (Fig 2). Such functionality allows the combination of observer self-motion and object motion in future studies. A Nintendo Wii remote control [11, 12] senses the positions of battery-powered infrared light emitting diodes (LEDs) that are mounted on a headband worn by of our observers.

the contrary, our data from both VR and RW experiments show that the illusion strength did not vary signicantly with the position of the axis of rotation. This shows a higher dependence on top-down, concept-driven processes, such as the experience of convex faces, rather than data-driven MP signals. There was no signicant difference between the responses of observers in the RW and the VR experiment, conrming that virtual stimuli can replace real-world stimuli for more exible and quantiable measurements in the future. It was clear that all observers saw the illusion strongly. However, many observers were hesitant to respond promptly, either for fear of an incorrect answer or because they had a poor understanding of their task. We are currently developing better techniques for training observers, based on our observations from the pilot studies. Nevertheless, the main nding remains that the illusion is obtained across all conditions. IV. F UTURE W ORK Future experiments will utilize the head-tracking apparatus described above to assess the observers position in real space and time. We already have a working head-tracking system as shown in Fig. 2. We have successfully developed software that affords a wide spectrum of interesting studies on the perception of 3D objects, because our software is not limited to mask stimuli. On the contrary, it can handle any 3D object model. These resources enable us to design experiments to compare 3D object perception between patients with SZ and healthy controls. Since the hollow-mask illusion correlates with the severity of SZ, similar experiments can be performed on SZ patients who are undergoing treatment to assess the efcacy of the treatment method. V. ACKNOWLEDGMENTS We thank Chris Kourtev for technical support. R EFERENCES
[1] Dima, D., et al., Understanding why patients with schizophrenia do not perceive the hollow-mask illusion using dynamic causal modelling, NeuroImage (2009), doi:10.1016/j.neuroimage.2009.03.033 [2] Silverstein, S. M., Hatashita-Wong, M. H., Schenkel. L. S., Kovcs, I., Feher, A., Smith, T. E., Goicochea, C., & Uhlhaas, P. (2006). Reduced top-down inuences in contour detection in schizophrenia. Cognitive Neuropsychiatry, 11, 112-132 [3] Papathomas, TV and Bono, L. (2004). Experiments with a hollow mask and a reverspective: Top-down inuences in the inversion effect for 3-D stimuli, Perception, 33, 1129-1138. [4] Papathomas, TV (2007). Art pieces that move in our minds an explanation of illusory motion based on depth reversal, Spatial Vision, 21, 7995. [5] Singular Inversions. FaceGen 3D Human Faces Version 3.5. Available: [6] Autodesk. 3DSMax 2011. Available: [7] Brainard DH (1997). The Psychophysics Toolbox, Spatial Vision 10, 433-436. [8] Pelli DG (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies, Spatial Vision, 10, 437-442. [9] Kleiner M, Brainard D, Pelli D, 2007, "Whats new in Psychtoolbox-3?" Perception 36 ECVP Abstract Supplement. [10] MATLAB version 2010. Natick, Massachusetts: The MathWorks Inc., 2010. [11] Lee, JC (2008). Hacking the Nintendo Wii Remote, Pervasive Computing, July-September 2008. [12] Peek, Brian., WiiLAB version 1.1. Available:

Figure 2. scale).

Top view of rear projection head tracking system (not drawn to

We render the virtual scene in real time by changing the viewing point according to the sensed observers position. To further enhance realism in the VR system, a rear screen projection system was installed to allow for larger images. This created an environment for studies less restrictive than the study performed on the laptop. Calibration of this system was necessary to accurately redraw the virtual environment for every change in the participants position. This gain calibration variable was assessed using an object in the RW that was also constructed in the VR world. We marked locations on the laboratory oor where similar features of the RW and VR object looked identical. This enabled us to express the amount of rotation for the VR object as a linear function of the viewers displacement. This was a necessary step to increase the realism of the VR stimulus view that changed as a result of the observers self-motion. III. R ESULTS The rationale for varying the position of the rotation axis was to vary the strength of the motion parallax (MP) signals. The perceived feature-to-observer distance varies inversely with MP. Thus, mask d of Fig 1 should produce a weak illusion because the nose-tip, having zero MP, should be seen further than features with large MP, such as the cheeks. On