JOURNAL OF COMPUTING, VOLUME 4, ISSUE 9, SEPTEMBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.

ORG

41

Feature scruitinization for hand gestures

Sumaira kausar, M.Younis Javed, samabia tehsin
Abstract Sign language recognition is applicable to a diversity of areas. That is the reason it is getting high attention of researchers in latest years. This paper focuses on two critical phases of automatic sign language recognition i.e. feature selection and feature extraction. As a matter of fact, a sign language recognizer can never give good results if it does not rely on correct feature selection and their accurate extraction. This paper aimed at presenting some of the possible feature sets and some popular extraction mechanisms with a critical view. This paper anlayzed different geometrical feature sets on the basis of a quality metric. So researchers specially, novice one can get one point access to the issues associated with these two phases for sign language recognition

1 INTRODUCTION Hand gesture recognition is an area of increasing interest for the researchers due to its applicability in many areas. A lot of work has already been done in the area of hand gesture recognition [5,6,7,8,9,10,11,12], but still a lot more to do. Majority of the developed systems lack robustness. [13,14,15,16] . The main problem in such system is lack of robust feature set. The most critical pahse in hand gesture recognition is the correct selection of features. Accurate classification of hand gesture has a lot of dependence on corrrect selection and accurate extraction of these features because no matter how efficient and reliable the classifier, used for hand gesture recognition, but if the classifier is not provided with carefully selected and accurately extracted features, the accuracy of classifier goes into vain. So a lot of attention is required by the researchers in the selection criteria and extraction techniques of features for hand gesture recognition systems. If the phase of feature selection and extraction is given due attention, even a very simple classifier can show very good results. This paper represents some possible feature sets for static hand gesture recognition, their respective extraction techniques along with their associated problems. The aim of this paper is to provide researchers of the field with a taxonomy of possible feature sets based on geometric features of signs and corresponding extraction techniques. It is an observed fact that researchers of the field usually presents their feature sets but they normally does not explicitly narrate the associated problems. So for novice researchers in the field may have to go through a long hit and try period to select and extract feature set with required reliability and robustness. This paper is for such resaearchers so they can get one point access to some potential feature sets for hand gesture recognition along with their strengths and weaknesses. Indicated problems of the feature sets can guide researchers of the field in the selection of feature sets for their hand gesture recognition systms. 2 ANALYSIS There are two broad categories of hand gesture recognition systems: vision based and non-vision based systems. In non-vision based systems data gloves[2] or colored gloves [1] are used to extract feature points for the system. This makes feature extraction phase quite easy but it is unnatural to wear data gloves or colored gloves. Hence such methods for feature extraction limit down the area of its applicability. Vision based feature extraction is difficult for researchers to implement but is more natural and hance have wider applicability.when talking about vision based features again two broad categories of techniques: geometric features and statistical features. This paper presents geometrical

2012 Journal of Computing Press, NY, USA, ISSN 2151-9617

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 9, SEPTEMBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

42

features in detail. Geometrical features of hand are features on the basis of geometry of hands e.g approximation of hand shape, finger tip detection on the basis of curvature property, angles between different geometrical points of hand etc. The feature sets in this paper are analyzed in two steps, first feature selection phase is addressed and then comes feature extraction phase. 3 FEATURE SELECTION Feature selection deals with selecting appropriate feature points that can be used to distinguish classes succesfully by using some classification method. Selection of feature set for vision based hand gestures is very crucial and a difficult decision. Because inappropriate feature selection leads to incorrect classification. Different feature sets are analyzed in this paper according to following metric. Number of feature points Size of feature vector Suitability for variety of gestures

For a better feature set it is required to have a small number of required feature points, smallet feature vector and should be able to have recognition suitability for a large variety of gestures i.e high suitability rate. Different signs can be recognized on the basis of following features: number of open and closed fingers, identification of the fingers which are open and vice versa, level of opennes i.e. bending level, orientation of hand and orientation of fingers. If these features are correctly extracted then all possible signs of a particulart sign language under consideration can be accurately recognized. But the problem is faced by the researchers is that machine unlike humans not capable of recognizing these features that easily. For these features, angles between different geometric features can be selected to acquire all above mentioned feautures. Some of the researchers [1,2,3] have used such angles for sign language recognition but it is a matter of fact that these researchers are not capable of recognizing large size of sign dictionary. Figure 1. is showing four different sets of feature points named as FV1,FV2,FV3,FV4 shown with blue marks on front side of hand. These geometric points are then used to calculate angles and distance between different selected points and then these angles and distance are used as feature vector for classification purpose. In FV1 features are, angle between each finger tip and corresponding finger first joint and second joint , angle between first finger joint and corresponding second finger joint of each finger, distance between palm centre point and each finger tips, distance (positive/ negative to indicate orientation of hand) between wrist point and each finger tip. Hence size of feature vector FV1 would be 20. In FV2 features are, angle between each finger tip and corresponding finger joint, distance between palm centre point and each finger tips, distance between wrist point and each finger tip. In FV3 features are, distance and angle between palm centre point and each finger tips, distance and angle between three wrist points and each finger tip. Two additional wrist points are to accommodate those signs which are for signs having side view of hand. In FV4 features are, distance and angle between wrist point and corresponding finger tip. In FV4, five points in wrist respectively correspond to five fingers i.e. angle and distance measures are taken between corresponding wrist point and finger tip.

2012 Journal of Computing Press, NY, USA, ISSN 2151-9617

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 9, SEPTEMBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

43

(FV1)

(FV2)

(FV3)

(FV4) Fig. 1 Sets of feature points Following table is summarizing above mentioned features according to the metric mentioned at the start of the topic. Suitability rate (SR) is calculated as a ratio between recoignizable signs and total signs. SR= rs / n Where And rs= recognizable signs with given feature vector n= total size of sign dictionary

Signs considered for this test are taken from Pakistani sign language (PSL). Total one hundered and forty signs are considered, all these signs are static signs. These signs are thirty seven PSL alphabet signs, three special alphabets and hundered numertation signs from one to hundered. Signs are taken from the book published by most authenticated and renowned deaf association of Pakistan i.e Sirsyed Deaf Association [4]. Figure 2 shows signs which are used for analysis for this paper. Table 1 is enlisting the metric for feature selection while calculating SR without considering the possiblility of extracting required feature points. As human can estimate the occluded feature points in a given sign, same estimation can be adopted by machines as in many sign these feature points are not visible. For this paper estimation is performed by assuming that the size of fully stretched hand should be visible ,all finger tips should be visible and point on wrist/ palm centre should be visible, then rest of the feature points which may not be visible in different signs,b can be estimated by using available points. So with this supposition that machine recognizer can estimate missing feature points, Figure 3 is showing graph corresponding to table 1. Feature vector Number of feature points Size of vector feature Suitability rate for dictionary %

2012 Journal of Computing Press, NY, USA, ISSN 2151-9617

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 9, SEPTEMBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

44

FV1 17 25 38 FV2 12 15 38 FV3 9 25 50 FV4 10 10 50 Table 1: feature selection metric for PSL recognition with extraction estimation

50 Number of feature points Size of feature vector Suitability rate for % dictionary 4FV 3FV 2FV 1FV 40 30 20 10 0

Figure 3: feature selection metric for PSL recognition with extraction estimation : Table 2 is summaring feature selection metric with the calculation for SR considering that feature points which are not visible in a sign image can not be estimated. So recognition is based only with image, the feature points which are visible in a scene. Figure 4 is giving graph corresponding to Table 2. Feature vector Number of feature Size of feature Suitability rate for points vector dictionary % FV1 17 25 4.3 FV2 12 15 5.7 FV3 9 25 39 FV4 10 10 45 Table 2: feature selection metric for PSL recognition without extraction estimation :

50 Number of feature points Size of feature vector Suitability rate for % dictionary 4FV 3FV 2FV 1FV 40 30 20 10 0

Figure 4: feature selection metric for PSL recognition without extraction estimation :

By looking at talbe 1 and 2, it is evident that automatic recognition can not be successful until feature points estimation is used. FV4 with estimation and even without estimation outperforms as it has smaller feature vector and higher SR. But as it is visible with the above analysis that even with estimation, geometric features are not giving encouraging results. This is because of t the fact that

2012 Journal of Computing Press, NY, USA, ISSN 2151-9617

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 9, SEPTEMBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

45

geometric features are difficult to extract because the whole set of required features is not visible in every sign. Estimation of features has its own limits. This estimation can be achieved through many mean. The overall reliability of the recognizer is dependent upon the reliability of the estimator of the missing features. So to avoid these inherent problems associated with geometric features, statistical features can be used. 4 FEATURE SELECTION: Representative and accurate feature selection phase has got a lot of importance in the overall sign language recognition system and equally important is the extraction of these feature accurately because no matter how good and representative features are selected if extraction is not accurate all effort is useless. In all of the above mentioned feature points, finger tip is a must. Features points apart from finger tip, can be easily extracted while using some distance measures and relative distance measures. But finger tip extraction has a lot of associated problems. So this paper is presenting here problems associated with the two most commonly used methods to identify finger tips in a hand image. Researchers have used these methods to identify finger tips but because of some inherent problems with the methods, these two are not much suitable for a large sign dictionary recognition. These two methods are based on the curvature property of the finger tip. These two methods are k-curvature based on derivatives and k-curvature based on angle between vectors [3]. Figure 4 shows result of identifying finger tips and finger joints with palm using derivatives based kcurvature to calculate valleys and peaks with varying values of k.

(a) (b)

(c)

Figure 3.(a-c) k-curvature using derivatives with k values 30, 25 and 20 respectively Figure 3 shows result of identifying finger tips and finger joints with palm using vector angles based k-curvature to calculate valleys and peaks with different values of k.

(a) (b) Figure 4.(a-b) k-curvature using vector angles with k values 20 and 30 respectively

2012 Journal of Computing Press, NY, USA, ISSN 2151-9617

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 9, SEPTEMBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

46

By varying threshhold value and value for k, results can vary drastically i.e visible in figure 3 and Figure 4.So it is very difficult to adjust these parameters as different parameter values are suitable for different types of signs. That is the reason that researchers have used these methods for very small sized sign dictionaries. Apart from these parameter tuning problems that lead to an unreliable and unrobust sign language recognizer, there are some other inherent problems of both these methods. K-curvature using vector angle is that, this method is suitable only angles between vectors less than 180 degree. This is a big issue while considering sign language recognition as angle between points on curvature of hands vary depending upon the sign and hence its not possible that these angles can always be less than 180 degree. Similarly derivatives based k-curvature is more suitable for corners not for curves like finger tips.

5 CONCLUSION This paper is an effort to guide novice researchers in the field of sign language recognition about the feature selection and extraction for sign language recognition. So they can direct their research in the light of issues pointed out in this paper without wasting their precious time on ideas which have some associated and inherent problems which are not explicitly told by the researchers who have selected such features. So this paper can be a great help to the researchers of the field.

References: [1] Sumaira Kausar, M. Younus,Javed, Shaleeza,Sohail: Recognition of gestures in Pakistani sign language using fuzzy classifier , 8th conference on Signal processing, computational geometry and artificial vision 2008, 20-22 August 2008, Rhodes, Greece, Pages 101-105 [2] Eunseok Jeong, Jaehong Lee, DaeEun Kim: " Finger-gesture Recognition Glove using Velostat" , 11th international conference on control, automation and systems, Oct. 26-29, 2011 in Kintex, Gyeonggi-do,Korea. [3]Omer Rashid, Ayoub Al-Hamidi and Brend Michaelis, " Utilizing Invariant Descriptors for finger spelling American Sign Language Using SVM", ISVC 2010, Part I,LNCS 6453, pp. 253-263.2010 [4] "Pakistan Sign Language", 3rd edition, by Sirsyed Deaf Association, Regd No. VSWA/ICT/40-1988. [5] Yang R., Sarkar S., Loeding B., Handling Movement Epenthesis and Hand Segmentation Ambiguities in Continuous Sign Language Recognition Using Nested Dynamic Programming ,Pattern Analysis and Machine Intelligence, IEEE Transactions on March 2010,volume 32 Issue:3 pp 462 477. [6] Anup Nandy, Jay Shankar Prasad, Soumik Mondal, Pavan Chakraborty and G. C. Nandi, Recognition of Isolated Indian Sign Language Gesture in Real Time ,Information Processing and Management Communications in Computer and Information Science, 2010, Volume 70, 102-107, DOI: 10.1007/978-3-642-12214-9_18

2012 Journal of Computing Press, NY, USA, ISSN 2151-9617

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 9, SEPTEMBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

47

[7] Ali Karami, ahman Zanj and Azadeh kiani, Persian sign language (PSL) recognition using wavelet transform and neural networks ,Expert Systems with Applications Volume 38, Issue 3, March 2011, Pages 2661-2667 [8] J. Ravikiran, Kavi Mahesh, Suhas Mahishi, R. Dheeraj, S. Sudheender and Nitin V. Pujari, Automatic Recognition of Sign Language Images ,Intelligent Automation and Computer Engineering Lecture Notes in Electrical Engineering, 2010, Volume 52, 321-332, DOI: 10.1007/978-90481-3517-2_25 Computer Science, 2011, Volume 6495/2011, 665-676, [9] kelly D., McDonald J., Markham C., A person independent system for recognition of hand postures used in sign language, Pattern Recognition Letters Volume 31, Issue 11, 1 August 2010, Pages 1359-1368 [10] Kelly D., McDonald J., Markham C., Continuous recognition of motion based gestures in sign language , Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on Sept. 27 2009-Oct. 4 2009, Kyoto,pp 1073 1080. [11] Zafrulla Z., Brashear H., Hamilton H., Starner T.,"A novel approach to American Sign Language (ASL) phrase verification using reversed signing", Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference San Francisco, CA,, 13-18 June 2010 , pp 48 55. [12] Justus Piater, Thomas Hoyoux, Wei Du, Video Analysis for Continuous Sign Language Recognition,4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, 2010 (Workshop at the 7th International Conference on Language Resources and Evaluation (LREC), Malta) [13] Zahoor Zafrulla, Helene Brashear, Pei Yin , Peter Presti, Thad Starner, Harley Hamilton, American Sign Language Phrase Verification in an Educational Game for Deaf Children ,20th International Conference on Pattern Recognition, Istanbul, Turkey ,August 23-August 26 2010. [14] Chung-Lin Huang; Bo-Lin Tsai; , A Vision-Based Taiwanese Sign Language Recognition, Pattern Recognition (ICPR), 2010 20th International Conference,Istanbul ,Turkey , 23-26 Aug. 2010, PP 3683 3686. [15] Kelly D., McDonald J., Markham C., Evaluation of threshold model HMMS and Conditional Random Fields for recognition of spatiotemporal gestures in sign language ,Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on Sept. 27 2009-Oct. 4 2009, Kyoto, pp 490 497. [16] Tan Dat Nguyen and Surendra Ranganath, Recognizing Continuous Grammatical Marker Facial Gestures in Sign Language Video ,Computer Vision ACCV 2010 Lecture Notes in Computer Science, 2011, Volume 6495/2011, 665-676

2012 Journal of Computing Press, NY, USA, ISSN 2151-9617