Sign Language Recognition based on Hands symbols Classification

Communication is always having a great impact in every domain and how it is considered the meaning of the thoughts and expressions that attract the researchers to bridge this gap for every living being.

The objective of this project is to identify the symbolic expression through images so that the communication gap between a normal and hearing impaired person can be easily bridged.
Github Link:https://github.com/TrilokiDA/Hand_Sign_Language

1. Sign Language Recognition based on Hands Symbols Classification Guided by: Dr.S.R.Balasundaram (Professor) Presented by: Triloki Gupta M.Tech(Data Analytics) 205217006 Department of Computer Application 1/17/2019 ‹#›

2. Content Introduction Motivation Problem statement Objective Literature review Dataset description Proposed work Result Conclusion and Future work References 1/17/2019 ‹#› Department of Computer Application

3. Introduction The world is hardly live without communication, no matter whether it is in the form of texture, voice or visual expression. The communication among the deaf and dumb people is carried by text and visual expressions. Gestural communication is always in the scope of confidential and secure communication. Hands and facial parts are immensely influential to express the thoughts of human in confidential communication. 1/17/2019 ‹#› Department of Computer Application

4. Motivation Sign language is learned by deaf and dumb, and usually it is not known to normal people, so it becomes a challenge for communication between a normal and hearing impaired person. Its strike to our mind to bridge the between hearing impaired and normal people to make the communication easier. Sign language recognition (SLR) system takes an input expression from the hearing impaired person gives output to the normal person in the form text or voice. 1/17/2019 ‹#› Department of Computer Application

5. Problem Statement Understanding the exact context of symbolic expressions of deaf and dumb people is the challenging job in real life until unless it is properly specified. 1/17/2019 ‹#› Department of Computer Application

6. Objective Communication is always having a great impact in every domain and how it is considered the meaning of the thoughts and expressions that attract the researchers to bridge this gap for every living being. The objective of this project is to identify the symbolic expression through images so that the communication gap between a normal and hearing impaired person can be easily bridged. 1/17/2019 ‹#› Department of Computer Application

7. Literature Review 1/17/2019 ‹#› Author Publication Year Problem Methodology Remark Naresh Kumar ICCCA 2017 Hand Sign Language Recognition SVM & LDA Classification to recognition sign language symbols (97.3%). Tse-Yu Pan, Li-Yun Lo, Chung-Wei Yeh, Jhe-Wei Li, Hou-Tim Liu, Min-Chun Hu IEEE 2016 Real-time Sign Language Recognition SVM, PCA & LDA Images of the same gesture were captured in different lighting Conditions (94%) Department of Computer Application

8. Cont. Author Publication Year Problem Methodology Remark Salem Ameen, Sunil Vadera University of Salford 2017 Classify American Sign Language Fingerspelling from Depth and Colour Images CNN This paper explores the applicability of deep learning for interpreting sign language, precision of 82% and recall of 80%. Arabic sign language recognition with 3D convolutional neural networks IEEE 2017 3D Convolutional Neural Network (CNN) was used to recognize 25 gestures from an Arabic sign language dictionary CNN The system achieved 98% accuracy for observed data and 85% average accuracy for new data. 1/17/2019 ‹#› Department of Computer Application

9. Dataset description 1/17/2019 ‹#› We analyze 4,800 images of sign images which is ISL of the English alphabet, which have a spread of 26 class labels assigned to them. Each class label is a set of sign images of the English alphabet. All the images are resized to 640 x 480 pixels, and we perform both the model optimization and predictions on these downscaled images. Department of Computer Application

10. Cont. 1/17/2019 ‹#› Below figure shows an example from every class of sign images dataset. Department of Computer Application

11. Proposed Work In this work, we proposed an idea for feasible communication between hearing impaired and normal person with the help of- Deep Learning CNN AlexNet Machine Learning SVM Random Forest KNN 1/17/2019 ‹#› Department of Computer Application

12. Cont. Work flow Diagram: 1/17/2019 ‹#› Department of Computer Application

13. Cont. Figure. Typical Convolutional Neural Network Architecture 1/17/2019 ‹#› Department of Computer Application

14. Cont. Figure. Typical AlexNet Architecture 1/17/2019 ‹#› Department of Computer Application

15. Result CNN SGD optimizer with learning rate 0.01 and dropout 0.25 Model Accuracy Accuracy: 98.74% Model Loss Loss: 6.53% 1/17/2019 ‹#› Department of Computer Application

16. Cont. AlexNet SGD optimizer with learning rate 0.01, momentum 0.9, nesterov and dropout 0.25 Model Accuracy Accuracy: 99.79% Model Loss Loss: 0.79% 1/17/2019 ‹#› Department of Computer Application

17. Cont. Comparison table of CNNs and AlexNet: 1/17/2019 ‹#› Model Optimizer Dropout Learning rate Momentum Nestrove Accuracy % CNNs SGD 0.25 0.01 --- --- 98.74 CNNs SGD 0.50 0.01 --- --- 98.74 AlexNet SGD 0.25 0.01     98.42 AlexNet SGD 0.50 0.01 --- --- 95.70 AlexNet SGD 0.25 0.01 0.9 True 99.79 AlexNet SGD 0.50 0.01 0.9 True 99.79 AlexNet Adam 0.25 0.001 --- --- 99.69 AlexNet Adam 0.50 0.001 --- --- 99.58 Department of Computer Application

18. Cont. 1/17/2019 ‹#› Machine Learning SVM: 91.226% Random Forest: 95.719% KNN: 87.542% Department of Computer Application

19. Conclusion and Future Work In this project, we proposed an idea for feasible communication between hearing impaired and normal person with the help of deep learning and machine learning approach. This proposed work ensures the accuracy of 91.22% using SVM, 95.71% using Random Forest, 87.54% using KNN, 98.74% using CNN and 99.79 using AlexNet. There is ever the sounding challenge to develop a sign language system in data the collection remains invariant of the unconstraint environment. This project can be extended to the real time data. 1/17/2019 ‹#› Department of Computer Application

20. References [1] Ameen, S., & Vadera, S. (2017). A convolutional neural network to classify American Sign Language fingerspelling from depth and colour images. Expert Systems. [2] Naresh Kumar(2017). Sign Language Recognition for Hearing Impaired People based on Hands Symbols Classification. International Conference on Computing, Communication and Automation (ICCCA2017) [3] Menna ElBadawy, A. S. Elons, Howida A. Shedeed and M. F. Tolba. Arabic sign language recognition with 3D convolutional neural networks. 2017 Eighth International Conference on Intelligent Computing and Information Systems (ICICIS) [4] Pan, T. Y., Lo, L. Y., Yeh, C. W., Li, J. W., Liu, H. T., & Hu, M. C.(2016, April). Real-time sign language recognition in complex background scene based on a hierarchical clustering classification method. In Multimedia Big Data (BigMM), 2016 IEEE Second International Conference on (pp. 64-67). IEEE. 1/17/2019 ‹#› Department of Computer Application

21. Cont. [5] Pigou, L., Dieleman, S., Kindermans, P. J., & Schrauwen, B. (2014, September). Sign language recognition using convolutional neural networks. In Workshop at the European Conference on Computer Vision (pp. 572-578). Springer International Publishing. [6] Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of initialization and momentum in deep learning. In: Proceedings of the 30th International Conference on Machine Learning (ICML-13). pp. 1139{1147 (2013) [7] Tao Liu, Wengang Zhou, and Houqiang Li. Sign Language Recognition With Long Short-Term Memory . 2016 IEEE International Conference on Image Processing (ICIP) 1/17/2019 ‹#› Department of Computer Application

22. 1/17/2019 ‹#› Department of Computer Application