Sign Language Recognition based on Hands Symbols Classification n~ 9 ¢ ~~ Guided by: Dr.S.R.Balasundaram_ ¢ (Professor) Presented by: Triloki Gupta M.Tech(Data Analytics) 205217006 hello 17/2019 Department of Computer ApplicationContent e Introduction e Motivation e Problem statement e Objective e Literature review e Dataset description e Proposed work e Result e@ Conclusion and Future work e References 1/17/2019 Department of Computer ApplicaIntroduction e@ The world is hardly live without communication, no matter whether it is in the form of texture, voice or visual expression. e The communication among the deaf and dumb people is carried by text and visual expressions. e@ Gestural communication is always in the scope of confidential and secure communication. e Hands and facial parts are immensely influential to express the thoughts of human in confidential communication. zing Department of Computer ApplicationMotivation e Sign language is learned by deaf and dumb, and usually it is not known to normal people, so it becomes a challenge for communication between a normal and hearing impaired person. e Its strike to our mind to bridge the between hearing impaired and normal people to make the communication easier. e Sign language recognition (SLR) system takes an input expression from the hearing impaired person gives output to the normal person in the form text or voice. zing Department of Computer ApplicationProblem Statement e Understanding the exact context of symbolic expressions of deaf and dumb people is the challenging job in real life until unless it is properly specified. zing Department of Computer ApplicationObjective e Communication is always having a great impact in every domain and how it is considered the meaning of the thoughts and expressions that attract the researchers to bridge this gap for every living being. e@ The objective of this project is to identify the symbolic expression through images so that the communication gap between a normal and hearing impaired person can be easily bridged. zing Department of Computer ApplicationLiterature Review Author Publication | Year Problem Methodolog | Remark y Naresh Kumar] ICCCA 2017 | Hand Sign SVM&LDA | Classification Language to recognition Recognition sign language symbols (97.3%). Tse-Yu Pan, IEEE 2016 | Real-time Sign | SVM,PCA & _ | Images of the Li-Yun Lo, Language LDA same gesture Chung-Wei Recognition were captured Yeh, Jhe-Wei in different Li, Hou-Tim lighting Liu, Min- Conditions Chun Hu (94%) 1/17/2019 7 Department of Computer ApplicationCont. Author Publication | Year | Problem Methodolog | Remark y. Salem Ameen, | Universityof | 2017 | Classify American CNN ‘This paper Sunil Vadera Salford Sign Language explores the Fingerspelling from applicability of Depth and Colour deep learning for Images interpreting sign language, precision of 82% and recall of 80%. Anbic sign IEEE 2017 | 3D Convolutional CNN ‘The system language recognition with 3D convolutional neural networks Neural Network (CNN) was used to recognize 25 gestures from an Arabic sign language dictionary achieved 98% accuracy for observed data and 85% average accuracy for new data. 1/17/2019 Department of Computer ApplicationDataset description @ We analyze 4,800 images of sign images which is ISL of the English alphabet, which have a spread of 26 class labels assigned to them. Each class label is a set of sign images of the English alphabet. @ All the images are resized to 640 x 480 pixels, and we perform both the model optimization and predictions on these downscaled images 1/17/2019 Department of Computer ApplicationCont. @ Below figure shows an example from every class of sign images dataset. A B Cc D E F G iH & wih! eal ols Rl 1 K -L M N © P @ R s oT vu vow x ¥ 1/17/2019 Department of Computer Application 12Proposed Work @ In this work, we proposed an idea for feasible communication between hearing impaired and normal person with the help of- © Deep Leami BONN BH AlexNet Machine Leaming Msvm @ Random Forest EKNN 1/17/2019 Department of Computer Application1/17/2019 Cont. Work flow Diagram: Data Acquisition Image Preprocessing Image Preprocessing (resize & segmentation) ese) ee Data Augmentation (shape, texture, histogram, shift) | I [Apply SVM, Random Forest Apply CNN, AlesNet ron J i Ciasetaten aseaten Department of Computer ApplicationCont. Subsamping Fuly conected Figure. Typical Convolutional Neural Network Architecture 1/17/2019 Department of Computer ApplicationCont. Input Coma. Cond Coma Coma Cons Fee FCT FR Figure. Typical AlexNet Architecture 1/17/2019 Department of Computer ApplicationResult e CNN + SGD optimizer with learning rate 0.01 and dropout 0.25 + Model Accuracy ; ecg + Model Loss ” Loss: 6.53% : 7“ i 1/17/2019 Department of Computer ApplicationCont. e AlexNet 1/17/2019 SGD optimizer with learning rate 0.01, momentum 0.9, nesterov and dropout 0.25 Model Accuracy stingy @ Accuracy: 99.79% =“ =e Model Loss ‘a @ Loss: 0.79% Department of Computer ApplicationCont. Comparison table of CNNs and AlexNet: EE a. -: = cerry Perry 1/17/2019 Department of Computer ApplicationCont. e@ Machine Learning + SVM: 91.226% + Random Forest: 95.719 % + KNN:87.52% | accuracy eww sum ‘Random Forest Machine learning techsiques 1/17/2019 Department of Computer Application 1sConclusion and Future Work e In this project, we proposed an idea for feasible communication between hearing impaired and normal person with the help of deep learning and machine learning approach. e@ This proposed work ensures the accuracy of 91.22% using SVM, 95.71% using Random Forest, 87.54% using KNN, 98.74% using CNN and 99.79 using AlexNet. e There is ever the sounding challenge to develop a sign language system in data the collection remains invariant of the unconstraint environment. This project can be extended to the real time data. 1/17/2019 Department of Computer ApplicationReferences [1] Ameen, S., & Vadera, S. (2017). A convolutional neural network to classify American Sign Language fingerspelling from depth and colour images. Expert Systems. [2] Naresh Kumar(2017). Sign Language Recognition for Hearing Impaired People based on Hands Symbols Classification. International Conference on Computing, Communication and Automation (ICCCA2017) [3] Menna ElBadawy, A. S. Elons, Howida A. Shedeed and M. F. Tolba. Arabic sign language recognition with 3D convolutional neural networks. 2017 Eighth International Conference on Intelligent Computing and Information Systems (ICICIS) [4] Pan, T. Y., Lo, L. Y., Yeh, C. W., Li, J. W., Liu, H. T., & Hu, M. C.(2016, April). Real-time sign language recognition in complex background scene based on a hierarchical clustering classification method. In Multimedia Big Data (BigMM), 2016 IEEE Second International Conference on (pp. 64-67). IEEE. 1/17/2019 Department of Computer ApplicationCont. [5] Pigou, L., Dieleman, S., Kindermans, P. J., & Schrauwen, B. (2014, September). Sign language recognition using convolutional neural networks. In Workshop at the European Conference on Computer Vision (pp. 572-578). Springer International Publishing. [6] Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of initialization and momentum in deep learning. In: Proceedings of the 30th International Conference on Machine Learning (ICML-13). pp. 1139{1147 (2013) [7] Tao Liu, Wengang Zhou, and Hougiang Li. Sign Language Recognition With Long Short-Term Memory . 2016 IEEE International Conference on Image Processing (ICIP) 1/17/2019 Department of Computer Application1/17/2019 Saying “Thank You” a Department of Computer Application