A Seminar Report on

³Human Machine Interfacing using TOUNGE-DRIVED SENSOR´

Submitted for partial fulfillment of requirement of award of

Degree In

Electrical&Electronics Engineering ASHWINI KUMAR PATEL
Roll No. 0806321025


SESSION: 2010-2011



CERTIFICATE Certified that this is a bonafide record of seminar entitled ³ HUMAN MACHINE INTERFACE USING TONGUE DRIVED SENSOR´ done by the following student ASHWINI KUMAR PATEL of the vith semester. Mr. SACHIN GOPAL SONI SEMINAR GUIDE .B Technical University.Electrical and Electronics Engineering in the year 2011 in partial fulfillment of the requirements to the award of Degree of Bachelor of Technology in Electrical and Electronics Engineering of G. Lucknow.

ACKNOWLEDGEMENT I thank my seminar guide Mr. Lecturer for his proper guidance. ASHWINI KUMAR PATEL ROLL NO. and valuable suggestions. I once again extend my sincere thanks to all of them. I am indebted to other faculty members for giving me an opportunity to learn and present the seminar.0806321025 . Sachin gopal soni. If not for the above mentioned people my seminar would never have been completed successfully.


called ³Tongue Drive´.Abstract We have developed a noninvasive. For human trials. A small permanent magnet secured on the tongue using a tongue clip.5 s. The magnetic field variations d ue to the marker movements are detected by an array of magnetic sensors mounted on a headset outside the mouth or an orthodontic brace inside. These commands can be used to access a computer by substituting the mouse or keyboard functions. or tongue piercing is utilized as a marker to track tongue movements. or other equipments. tissue adhesive. . The sensor outputs are then processed and translated into different user control commands after being wirelessly transmitted to a portable computer (PC or PDA). a phone. unobtrusive magnetic wireless tongue-computer interface. we have developed a prototype system with 6 direct commands on a baseball helmet and successfully tested it. They can also be customized to operate a powered wheelchair. to provide people with severe disabilities with flexible and effective computer access and environment control. The Tongue Drive system response time for >95% correctly completed commands is about 1.

sight. Also Known As: Man-Machine Interface Examples: A typical computer station will have four human-machine interfaces. It is sensitive and kept moist by saliva. as much of the upper surface of the tongue is covered in papillae and taste buds.what is a sensor? A sensor is a device that measures a physical quantity and converts it into a signal which can be read by an observer or by an instrument. the . It is the area of the human and the area of the machine that interact during a given task. a mercury-in-glass thermometer converts the measured temperature into expansion and contraction of a liquid which can be read on a calibrated glass tube. Lastly. and then converts this condition into analog or digital representation´. Sensors measure real-world conditions. heat transference or any other physical or cognitive function. It is the primary organ of taste. In sort we defiened as´ A device that receives and responds to a stimulus or signal. For example. A secondary function of the tongue is speech. sound. most sensors are calibrated against known standards. and is richly supplied with nerves and blood vessels to help it move. such as heat or light. the tongue also serves as a natural mea ns of cleaning one's teeth What is human machine interface? Definition: The Human-Machine Interface is quite literally where the human and the machine meet. What is tongue ? The tongue is a muscle on the floor of the mouth that manipulates food for chewing and swallowing (deglutition).1. Interaction can include touch. A thermocouple converts temperature to an output voltage which can be read by a voltmeter. For accuracy.

keyboard (hand). A company in Japan is us ing the same kind of technology to allow paralyzed people to control their wheelchairs using their mind. sending instructions or commands which are executed by the machine. It's the closet thing mankind has developed now to telepathy. That's an extreme example of something which can be as simple as a switch on a computer. the mouse (hand). but there's also more practical applications. It's being developed as both a cool gaming application called the Force Trainer. There's actually a new technology that can transmit your brainwaves wirelessly into a machine. and it's already on the market. . A human-machine interface is just what it sounds like: something that allows a human to communicate with a machine. the monitor (eyes) and the speakers (ears).

Similarly. Many studies have demonstrate d that comparable subjective images are experienced by blind persons using one of several sensory substitution systems . Besides using camera data to restore sight. where it is turned into electrical impulses in th e optic nerve.Whether or not the blind subjects are really seeing is address ed in . such as sound and balance. Providing informati on from artificial receptors offers an opportunity to restore function In the intact visual syst em. information from an artificial receptor iscoupled to the brain via . but retain central visual mechanisms. the perceived image is re -created in thebrain . have produced auditory and vestibular substitution . Usually. sensors to replace other lost sensory infor mation. In su ch restoration of lost senses.Introduction Persons who become blind do not lose the capacity to see. they lose the peripheral sensory system (the retina). The input from a sensory substitution system can reach many brain struc tures including those anatomically and physiologically related to the lost sensory modality . persons who become deaf or are without balance usually lose only the peripheral structures relating to sound transduction (the cochlea) or positional orientation(the vestibular apparatus). the optical image goes only to the retina.

However.In the past. This recent explos ion of interest in sensory substitution suggests that now is a good time to review progress in the area.a human±machine interface(HMI). METHODS 2. Sensory substitution is thus only possible because of brain plasticity . sensory substitution studies were purely academic. From Faradays law the voltage drop across an inductance can be found as: 2 = -L di/dt = . leading to the establishment of new research and development efforts in many countries. none of the d evice sever reached the market. and uses variable inductance techniques.1 Theory The detection method used in this work is based on Faraday s law of induction for a coil. with rare exceptions. This is stimulating the interest of both research groups and industry. 2. replacing information usually carried to the brainfrom an intact sense organ. recent technological advances have led to the pos sibility of new prosthetic devices being potentially accessible at much lower cost to millions of patients.0 ‡ r ‡ N ‡A/l ‡di/dt Where .

The method resembles the known techniques used for displacement sensors . 1). of constant peak-peak amplitude.L= 0 ‡ r ‡ n ‡A/l 2 L = inductance 0 r = vacuum permeability = relative magnetic permeability of the core material N= number of turns l = is the average length of the magnetic flux path When only air is present as the core of the inductance. which stays increased. Applying a sine wave current. is obtained across the coil L. a until the material is removed. ferromagnetic material is placed r =1. Introduction of the ferromagnetic material into the air gap of the coil. results in an increase of . This will be utilized for activation of a command in the inductive tongue control system (Fig. . i. the core becomes a combination of air and ferromagnetic material. a constant amplitude voltage drop. and r changes according to the magnetic permeability of the ferromagnetic material. . As a in the coil.

and fabrication of the activation unit Using dental acrylic.Figure 1. The Inductive tongue control system. The inner diameter of the inductor coils was 4mm. c: the inductors.2. 2 . a: The activation unit. and the coils had 90-150 turns. The tongue activates the sensors by placing the tongue-mounted activation unit at or inside a coil. b: the palatal plate. see Fig. nine air cored inductors were placed on a palatal plate resembling the ones used as dental retainers. see Fig. A silicone tube was fixed to the palatal plat e and carried the wires out of the mouth. Mounting of Sensors at the Palatal Plate.

upper: The palatal plate (placed on a mould of the upper part of the mouth) with inductive coils and a silicone tube leading the wires out of the mouth . Figure 2. of 20006500. .2 mm and the height was 2 mm. r.The diameter of the cylinder was 3.lower : The activation unit glued to the tongue.A small ferromagnetic metal cylinder was fabricated from stainless steel of the type SUS447J1. The steel had a maximum magneticpermeabili ty.

The palatal plate with the inductors was placed at the hard palate. The signals form the 9 inductors were amplified and rectified.03 mA amplitude was applied to the inductors from a galvanically isolated current source. A 50 kHz sine wave current with a 0. Experimental Setup The ITCS was tested in 3 healthy male subjects. Fig. . 2. The activation unit was glued to the tongue using tissue glue. The subject activated the inductive sensors by positioning the tongue in a manner that placed the activating unit in the centre of the different inductor coils. age 22 -29 years. Then the signals from 4 of the inductors were inverted before the inductor outputs were connected to obtain 4 channels with 2 -3 coils in series.3. hour on three consecutive days.

The baselines of the sensor signals corresponding to no activation of the inductive sensors were subtracted. 3 In this way the subject was typing given sequences of the characters ³ABCDEFGHI on of each inductor. Using online thresholding of the inductor signals. . 3.Figure 3. what to type in white and what have been typed by the subject in black 2. see Fig. Measurements were performed for intervals of 30 seconds Then the signals were sampled and processed using the Matlab DAQ toolbox. 4. The inductor leads comes out of the subjects mouth. RESULTS The subjects could activate desired sensors as shown in Fig. one of the characters: ³ABCDEFGHI´ was related to the activati and typed on the visual display of a computer (Fig. Experimental set-up. 3) when the corresponding inductor was activated. The subject is provided with a visual feedback showing the position of the coils/characters on the dental palate .4 Signal Processing The measured signals were amplified and rectified to obtain envelopes of the signals. the subject could see which sensor had been activated. From the visual display.

without having a visual display of the position of the characters resulted in a typing speed of 32-42 characters per minute with an error rate of 14-25%. The typed characters are shownabove each of the two graphs. Example of the signals from the ITCS during typing. dotted: signal from the inductor related to the character ³C´. Typing a random sequence of characters. but only the activated inductors show amplitudes deviating from zero. that has not previously been typed. that had been repeated approximately 40 times during the 3 days of measurements. On the top graph. On the bottom graphs the signal from the ITCS is shown during repetitive typing of B. black: signal from the inductor related to the character ³B´. resulted in a typing speed of 30-57 characters per minute with an error rate of 15-29%. Repetitive typing of the same character was performed with maximal typing speeds of 48-114 characters per minute. . lower case characters are incorrectly typed characters.Figure: Figure 4. and an error rate of 0-0. again without visual display. The data are from the third and final day of measurement. ABC is typed repeatedly: Grey: signal from the inductor related to the character ³A´. On both graphs signals from all 9 inductors are included.1%. Typing sequences. including all nine characters. Capital characters are correctly typed characters. The baseline potential corresponding to no activation of the coils has been subtracted.

4 29.Provides an efficient and quick activation of the desired function These requirements may be met by future applications of this new inductive tongue-computer interface.6 0 4. The increase in the typing rate between consecutive measurement days indicate that learning may still taking place . which may lead to substantial increase in e. about the perception of usage of the visual display (1= no use.8 10. DISCUSSION For a future control system to be truly successful it has.g. toys and prosthesis. minute correct characters on day 3 ma mean minimum Subject x 1 2 3 48 114 100 18. Typing rate of correct characters pr. This may imply that the system: .The typing rate of correct characters is shown in table 1. the tongue palatal pressure needed for activation of the units (1=no pressure at all.Is cosmetically acceptable in and outside the home of the user ± preferably invisible . The small size of the sensor -coils opens up for the possibility of having the whole alphabet as separate ³buttons´ on the palatal plate. . 10=maximum obtainable pressure) to 2 -3 and the difficulty in using the system to 3-5 (1=very easy 10 = impossible).Can be used/worn all day and night . in reality to be a help for the user. 10=constant use) the subjects rated the use of the display to 1-3. e.6 41 13. wheelchairs.Can be used to control a wide range of equipment. The typing rate of 30 to 57 characters per minute with 15-30% error rate suggests that the system may be quite efficient after sufficient training. computers.When asked on a scale from 1 to 10. In aver age the mean typing rate of correct characters of all measurements from one day.Is easy to use and induces a low degree of fatigue . the rate of writing for quadriplegics. improved with 117% from day one to day 2 and 22% from day 2 to day 3.g. Table 1.

Finally. that the activation of the inductive sensors only demands a low degree of tongue-palatal pressure. In addition the subjective experience was. rd Circuit diagram for tongue drive sensor . Future work will focus on implementing inductive tongue control interfaces with more than 9 inductive sensors and explore the possibility of implementation of a mouse or joystick function.after the 3 day. The subjective experience of the subjects using the system suggests that the system may be used without visual display. which may mean that longer training will reduce the error rate and further increase the typing rate. potentially making it quite mobile. Therefore longer training is needed to get a more acceptable error rate. Further. thereby reducing the risk of fatigue. The high typing rate of up to 114 activations pr. incorporation of wireless control and development of command strategies to control a wide range of devices will be considered. which can be activated by repetitive inductor activation. minute related to repetitive typing suggest the implementation of an interface having multiple functions for each inductive sensor. the Inductive tongue computer interface will be tested by people with motor disabilities.

y .

S. vol. 47 3. Comparison of Computer Interface Devices for Persons with Severe Physical Disabilities . Sensory substitution and thehuman ±machine interface by Paul Bach -y-Rit 4. The American Journal of Occupational Theraphy. Andreasen Strui 2. Human-Based Sensing ± Sensor Systems to Complement Human Perception by Peter Wide . C Lau and S O Leary. Tongue-computer interface for disabled people by Lotte N.REFERENCES 1.

Sign up to vote on this title
UsefulNot useful