Document Type : Research Paper
Authors
Affiliation: Control and Systems Engineering Department, University of Technology -Iraq.
Abstract
An American Sign Language (ASL) is a complex language. It is depending on the special gesture stander of marks. These marks are represented by hands with assistance by facial expression and body posture. ASL is the main communication language of deaf and people who have hard hearing from North America and other parts of the world. In this paper, Gesture recognition is proposed of static ASL using Deep Learning. The contribution consists of two solutions to the problem. The first one is resized with Bicubic static ASL binary images. Besides that, good recognition results in of detection the boundary hand using the Robert edge detection method. The second solution is to classify the 24 alphabets static characters of ASL using Convolution Neural Network (CNN) and Deep Learning. The classification accuracy equals to 99.3 % and the error of loss function is 0.0002. According to 36 minutes with 15 seconds of elapsed time result and 100 iterations. The training is fast and gives the very good results, in comparison with other related works of CNN, SVM, and ANN for training
Keywords
[1] P.A. Nanivadekar and V. Kulkarni, “Gesture recognition: a revolutionary tool,” International Journal of Technological Advancement and Research, Vol. 3 Issue. 3, 2013.
[2] N. Patel and S. JingHe, “A survey on hand gesture recognition techniques, methods and tools,” International Journal of Research in Advent Technology, Vol. 6, No. 6, 2018.
[3] A.L.C. Barczak, N.H. Reyes, M. Abastillas, A. Piccio, and T. Susnjak, “A new 2D static hand gesture colour image dataset for ASL gestures,” Res. Lett. Inf. Math. Sci., Vol. 15, pp. 12-20, 2011.
[4] A. Julka and S. Bhargava, “A static hand gesture recognition based on local contour sequence,” International Journal of Advanced Research in Computer Science and Software Engineering, Vol. 3, Issue 7, 2013.
[5] A.K. Gautam and A. Kaushik, “American sign language recognition system using image processing method ,” International Journal on Computer Science and Engineering (IJCSE), Vol. 9 No.07, 2017.
[6] F.A. Raheema and H.A. Raheem, “ASL recognition quality analysis based on sensory gloves and MLP neural network,” American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS), 2018.
[7] S. Biradar, and A.M. Tuppad, “A static hand gesture classification system for american sign language (ASL) finger spelling and digits, “International Journal of Lastest Trends in Engineering and Technology (IJTET), Vol. 7, Issue 1, 2016.
[8] T.N. Nguyen, H.H. Huynh, and J. Meunier, “Static hand gesture recognition using artificial neural network,” Journal of Image and Graphics, Vol. 1, No.1, 2013.
[9] O.K. Oyedotun and A. Khashman, “Deep learning in vision-based static hand gesture recognition ,” Springer, Neural Computing and Applications, 2016.
[10]. J.L. Flores C.E. Gladys Cutipa, R.L. Enciso, “Application of convolutional neural networks for static hand gestures recognition under different invariant features,” IEEE, 2017
[11] J. Bamwend and M. Özerdem, "Recognition of static hand gesture with using ANN and SVM",” Dicle University Journal of Engineering, 2019
[12] J. Pansare and M. Ingle, “Vision-based approach for American sign language recognition using edge orientation histogram,” International Conference on Image, Vision and Computing, 2016
[13] S. Nagarajan and T. Subashini," Static Hand Gesture Recognition for Sign Language Alphabets using Edge Oriented Histogram and Multi Class SVM", International Journal of Computer Applications, Vol. 82, 2013.
[14].A. Prajapati, S. Naik, and S. Mehta, “Evaluation of different image interpolation algorithms ,” International Journal of Computer Applications (0975 – 8887),Vol. 58, No. 12, 2012.
[15] M.B. Hisham, S.N. Yaakob, R.A. Raof, A.B. Nazren, and N.M. Wafi, “An analysis of performance for commonly used interpolation method ,” American Scientific Publishers Advanced Science Letters, United States of America, 2015.
[16] S.A. Alrubaie and A.H. Hameed, “Dynamic weights equations for converting grayscale image to RGB image,” Journal of University of Babylon for Pure and Applied Sciences,Vol. 26, No.8, 2018.
[17] C. Saravanan, “Color image to grayscale image conversion,” IEEE, Second International Conference on Computer Engineering and Applications, pp. 196-199, 2010.
[18] S.J. Pise, “An outlet for a creative mind, thinkquest,” Springer Science & Business Media, Proceedings of the First International Conference on Contours of Computing Technology, 2011.
[19] P. Selvakumar and S. Hariganesh “The performance analysis of edge detection algorithms for image processing,” International Conference on Computing Technologies and Intelligent Data Engineering, 2016
[2] N. Patel and S. JingHe, “A survey on hand gesture recognition techniques, methods and tools,” International Journal of Research in Advent Technology, Vol. 6, No. 6, 2018.
[3] A.L.C. Barczak, N.H. Reyes, M. Abastillas, A. Piccio, and T. Susnjak, “A new 2D static hand gesture colour image dataset for ASL gestures,” Res. Lett. Inf. Math. Sci., Vol. 15, pp. 12-20, 2011.
[4] A. Julka and S. Bhargava, “A static hand gesture recognition based on local contour sequence,” International Journal of Advanced Research in Computer Science and Software Engineering, Vol. 3, Issue 7, 2013.
[5] A.K. Gautam and A. Kaushik, “American sign language recognition system using image processing method ,” International Journal on Computer Science and Engineering (IJCSE), Vol. 9 No.07, 2017.
[6] F.A. Raheema and H.A. Raheem, “ASL recognition quality analysis based on sensory gloves and MLP neural network,” American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS), 2018.
[7] S. Biradar, and A.M. Tuppad, “A static hand gesture classification system for american sign language (ASL) finger spelling and digits, “International Journal of Lastest Trends in Engineering and Technology (IJTET), Vol. 7, Issue 1, 2016.
[8] T.N. Nguyen, H.H. Huynh, and J. Meunier, “Static hand gesture recognition using artificial neural network,” Journal of Image and Graphics, Vol. 1, No.1, 2013.
[9] O.K. Oyedotun and A. Khashman, “Deep learning in vision-based static hand gesture recognition ,” Springer, Neural Computing and Applications, 2016.
[10]. J.L. Flores C.E. Gladys Cutipa, R.L. Enciso, “Application of convolutional neural networks for static hand gestures recognition under different invariant features,” IEEE, 2017
[11] J. Bamwend and M. Özerdem, "Recognition of static hand gesture with using ANN and SVM",” Dicle University Journal of Engineering, 2019
[12] J. Pansare and M. Ingle, “Vision-based approach for American sign language recognition using edge orientation histogram,” International Conference on Image, Vision and Computing, 2016
[13] S. Nagarajan and T. Subashini," Static Hand Gesture Recognition for Sign Language Alphabets using Edge Oriented Histogram and Multi Class SVM", International Journal of Computer Applications, Vol. 82, 2013.
[14].A. Prajapati, S. Naik, and S. Mehta, “Evaluation of different image interpolation algorithms ,” International Journal of Computer Applications (0975 – 8887),Vol. 58, No. 12, 2012.
[15] M.B. Hisham, S.N. Yaakob, R.A. Raof, A.B. Nazren, and N.M. Wafi, “An analysis of performance for commonly used interpolation method ,” American Scientific Publishers Advanced Science Letters, United States of America, 2015.
[16] S.A. Alrubaie and A.H. Hameed, “Dynamic weights equations for converting grayscale image to RGB image,” Journal of University of Babylon for Pure and Applied Sciences,Vol. 26, No.8, 2018.
[17] C. Saravanan, “Color image to grayscale image conversion,” IEEE, Second International Conference on Computer Engineering and Applications, pp. 196-199, 2010.
[18] S.J. Pise, “An outlet for a creative mind, thinkquest,” Springer Science & Business Media, Proceedings of the First International Conference on Contours of Computing Technology, 2011.
[19] P. Selvakumar and S. Hariganesh “The performance analysis of edge detection algorithms for image processing,” International Conference on Computing Technologies and Intelligent Data Engineering, 2016
[20] S.M. Sharef, F.A. Rahem, and S.S. Jouma'a, “Implementation of fuzzy logic techniques in detecting edges for noisy images,” The Second Engineering Conference of Control, Computers and Mechatronics Engineering (ECCCM2), pp. 154-162, 2014.
[21] P. Kim, “MATLAB deep learning ITH machine learning, neural networks and artificial intelligence, “ APress, 2017.
[22] S. Skansi, “Introduction to deep learning from logical calculus to artificial intelligence,” Springer, 2018.
[23].A. Gibson and J. Patterson, “Deep Learning,” O'Reilly Media, Inc., 2017.
[24] G.S. Chadha, E. Meydani, and A. Schwung, “Regularizing neural networks with gradient monitoring,” Springer, Recent Advances in Big Data and Deep Learning: Proceedings of the INNS BDDL, Sestri Levante, Geneva, Italy, 2019.
[25] S. Gong, C.L. J,B. Z.Y. Li, and H. Dong, “Advanced image and video processing using MATLAB,” Springer, 2019.
[26] S. Khan, H. Rahmani, S.A. Shah, and M. Bennamoun, “A guide to convolutional neural networks for computer vision,” Morgan & Claypool Publishers series, 2018.
[27] M.Z. Alom, T.M. Taha, C. Yakopcic, S. Westberg, P. Sidike, M. S. Nasrin, B. C. Esesn, and A.S, “The history began from AlexNet: a comprehensive survey on deep learning approaches,” Cornell University, arXiv.org, cs, arXiv:1803.01164, Computer Science, Computer Vision and Pattern Recognition, 2018.
[28] S. Ameen and S. Vadera, “A convolutional neural network to classify american sign language finger spelling from depth and colour images,” John Wiley & Sons, Ltd., 2017.
[29] K. H. Zhang and S.R Sun, “Delving deep into rectifiers: surpassing human-level performance on image net classification,” Cornell University, arXiv:1502.01852v1, Computer Science, Computer Vision and Pattern Recognition, 2015.
[30] J. Nagi, F. Ducatelle, G.A. DiCaro, D. Ciresan, U. Meier, A. Giusti, F. Nagi, J. Schmidhuber, and L.M. Gambardella, “Max-pooling convolutional neural networks for vision-based hand gesture recognition,” IEEE International Conference on Signal and Image Processing Applications (ICSIPA2011), 2011.
[31] C. M. Bishop, “Pattern recognition and machine learning,” Springer, New York, NY, 2006.
[32] I. Sutskever, J. Martens, G. Dahl, and G.Hinton," On the importance of initialization and momentum in deep learning,” Proceedings of the 30 th International Conference on Machine Learning, Atlanta, Georgia, USA, 2013.
[33] W. Tangsuksant, S. Adhan, and C. Pintavirooj, “American sign language recognition by using 3D geometric invariant feature and ANN classification ,” IEEE, The Biomedical Engineering International Conference, 2014.
[21] P. Kim, “MATLAB deep learning ITH machine learning, neural networks and artificial intelligence, “ APress, 2017.
[22] S. Skansi, “Introduction to deep learning from logical calculus to artificial intelligence,” Springer, 2018.
[23].A. Gibson and J. Patterson, “Deep Learning,” O'Reilly Media, Inc., 2017.
[24] G.S. Chadha, E. Meydani, and A. Schwung, “Regularizing neural networks with gradient monitoring,” Springer, Recent Advances in Big Data and Deep Learning: Proceedings of the INNS BDDL, Sestri Levante, Geneva, Italy, 2019.
[25] S. Gong, C.L. J,B. Z.Y. Li, and H. Dong, “Advanced image and video processing using MATLAB,” Springer, 2019.
[26] S. Khan, H. Rahmani, S.A. Shah, and M. Bennamoun, “A guide to convolutional neural networks for computer vision,” Morgan & Claypool Publishers series, 2018.
[27] M.Z. Alom, T.M. Taha, C. Yakopcic, S. Westberg, P. Sidike, M. S. Nasrin, B. C. Esesn, and A.S, “The history began from AlexNet: a comprehensive survey on deep learning approaches,” Cornell University, arXiv.org, cs, arXiv:1803.01164, Computer Science, Computer Vision and Pattern Recognition, 2018.
[28] S. Ameen and S. Vadera, “A convolutional neural network to classify american sign language finger spelling from depth and colour images,” John Wiley & Sons, Ltd., 2017.
[29] K. H. Zhang and S.R Sun, “Delving deep into rectifiers: surpassing human-level performance on image net classification,” Cornell University, arXiv:1502.01852v1, Computer Science, Computer Vision and Pattern Recognition, 2015.
[30] J. Nagi, F. Ducatelle, G.A. DiCaro, D. Ciresan, U. Meier, A. Giusti, F. Nagi, J. Schmidhuber, and L.M. Gambardella, “Max-pooling convolutional neural networks for vision-based hand gesture recognition,” IEEE International Conference on Signal and Image Processing Applications (ICSIPA2011), 2011.
[31] C. M. Bishop, “Pattern recognition and machine learning,” Springer, New York, NY, 2006.
[32] I. Sutskever, J. Martens, G. Dahl, and G.Hinton," On the importance of initialization and momentum in deep learning,” Proceedings of the 30 th International Conference on Machine Learning, Atlanta, Georgia, USA, 2013.
[33] W. Tangsuksant, S. Adhan, and C. Pintavirooj, “American sign language recognition by using 3D geometric invariant feature and ANN classification ,” IEEE, The Biomedical Engineering International Conference, 2014.
)
