A TACTILE VISION SUBSTITUTION SYSTEM* Paul Bach-y-Ritat, Carter C, Collins}, Benjamin Whites, Frank A. Saunders§, Lawrence Scadden||, and Robert Blomberg® Smith-Kettlewell Institute of Visual Sciences Pacific Medical Center, San Francisco, California ABSTRACT A prototype of a tactile television system has been developed, in which a television camera image is projected onto the skin of the back by means of vibrating electromechanical stimulators, Blind subjects, within a few hours of training, have learned to identify geometric forms, move- ment in depth, environmental objects, photographs of faces, and block letter words. A vision substitution system is being developed both as a practical system for the blind and as a means of studying the way afferent information is proc- essed by the central nervous system. The theoretical neurophysiological basis! and the physical concept of the instrumentation? have been discussed previously, and results obtained with 3 x 3 and 20 x 20 stimulus arrays have been briefly reported. A detailed technical description of the apparatus will appear else- where', This report will present a summary of results obtained from 16 blind subjects during a three-month period. Four hundred solenoid stimulators are arranged in a 20 x 20 array built into a chair. The stimulators are spaced 12 mm apart, with 1 mm diameter Teflon tips which vibrate against the skin of the back. Their on-off activity can be monitored visually on an oscilloscope as a two-dimensional pictorial display. Of the 16 subjects, 14 were studied for one session of approximately four hours. Two subjects, selected for their verbal abilities (one congenitally blind 19-year-old college girl and one 29-year-old psychologist blind since age 4 years), have each spent approximately 50 hours in training. This time has been spent primarily in the exploration and development of training methods, rather than in intensive training by any single method. Indeed, continued im- provement in performance by these subjects indicates that maximum proficiency with this instrument has not been reached. No sensory fatigue has been report- ed by any subject. Fourteen subjects were introduced to the chair and to the characteristics of tactile stimuli through a videotape presentation of three simple geometric forms (circle, square and triangle) in a random series of 54 trials. After one hour of experience with the apparatus, they were asked to report their judgments of the forms. ‘Read before the Annual Meeting of the American Academy of Optometry, Beverly Hills, Cali- fornia, December 8, 1968. For publication in the February, 1969, issue of the AMERICAN JOURNAL OF OPTOMETRY AND ARCHIVES OF AMERICAN ACADEMY OF OPTOMETRY. Supported by DHEW, Social and Rehabilitation Service Grant No. RD-2444-S: USPHS Research Career Award No. 5 K3 NB-14,094; Rosenberg Foundation Grant; USPHS Gen- eral Research Support Grant No. 5 $01 FR 05566; T. B. Walker Foundation; and Trust in Memory of Beatrice and James J. Ingels. {Neuro-physiologist, M-D., Acting Director of Institute. +4Biophysicist, Ph.D., Senior Research Member. §Psychologist, Ph. [[Psychologist, M.A. Psychologist, B.A. 109TACTILE VISION SUBSTITUTION SYSTEM—BACH-Y-RITA, et al. These 14 subjects achieved 49.6%, 58.7% and 67.1% correct responses in the first, second and third blocks of 18 trials, respectively. The average per- cent correct response for all 54 trials was 58.6%. Even from the first block, per- formance significantly (p<0.05) above chance was obtained; a significant (p<0.05) and rapid learning effect occurred across trials. A later training procedure has eliminated the presentation of videotaped material and has allowed the subject to explore real objects with the TV camera from the beginning of the first session. This procedure has resulted in no ap- parent difference in the rapidity with which subjects learn to identify objects. Subjective description of movement in depth was initially gathered by pre- senting a 10 cm disc with 2, 3 or 4 short white pencils protruding vertically from equidistant points on its circumference; the disc was rotated at a speed of 6 rpm in a horizontal plane. Some subjects with acquired blindness have spontaneously reported rotation, while our congenitally blind subjects have all reported oscillatory movement in only one plane. Simulated objects (wagon, ice cream cone, etc.) constructed of geometric forms have been identified and correctly named by subjects; this task requires the subject to manipulate the TV camera mounted upon a tripod and to syn- thesize the perceived components into a meaningful whole Real objects (chairs, telephone, stairs, etc.) have been identified after as little as four hours of training. Subjects are guided in their judgments, as they pan the camera vertically and horizontally, by the trainer, who views a monitor presenting a visual display of tactor activity; the trainer gives feedback as to the accuracy of contour judgments, indicates areas of the object which need further exploration by the subject, and helps him to develop techniques of visual analysis for independent exploration. A zoom lens under the subject's control allows close analysis of detail. Recognition of block letters (upper case) comes immediately to subjects who have reasonable familiarity with letter configurations. With minimal train- ing, rate and accuracy improve greatly. Within two hours, words of five and six letters become recognizable with a single pass of the camera at an accuracy of approximately 80%. Monocular cues of depth are readily perceived by subjects using the vision substitution system. Movement within the environment is spontaneously re- ported. Many subjects, especially those who are congenitally blind, have never experienced changes in an object's apparent size as a function of its distance: after experience with the vision substitution system, this concept becomes a reality. The path of movement as the trainer moves toward and away from the camera is easily determined, and subjects can quantify the distance traveled by the trainer. Apparent motion and interposition of objects are available as cues for judging relative position and distance of targets. The complex problem of linear perspective, essential for the perception of depth, is learned with training and is acquired more readily by the subjects with acquired blindness than by the congenitally blind. Our two most experienced subjects, after some 40 hours of training, spon- taneously reported the external localization of stimuli in that information 110TACTILE VISION SUBSTITUTION SYSTEM—BACH-Y-RITA, et al. seemed to come from the object in front of the camera, rather than from the vibrotactors on their back. After sufficient experience, the use of the vision sub- stitution system thus appears to become an automatic extension of the sensory apparatus. REFERENCES 1, Bach-y-Rita, P. Sensory plasticity: Applications to a vision substitution system. Acta Neurologica ‘Scandinavica 43: 417-426, 1967. 2. Collins, C. C. Tactile image projection (abstract). National Symposium on Information Display 8: 290, May, 1967. 3. Bach-y-Rita, P., and Collins, C. C. Sensory plasticity and tactile image projection _(ab- stract). Invest, Ophthal. 6: 669, 1967. Bach-y-Rita, P., Collins, C. C., White, B., Saun- ders, F., Blomberg, R., and Scadden, L. Tactile television: Studies with blind ‘subjects (abstract). Invest. Ophthal., in press 4, Collins, C. C., Holmlund, G., Acker, R., and Bach-y-Rita, P. A. video-tactile image pro- jector. ‘In preparation,