Engineering and Technology Journal

Detection of arabic sign language by machine learning techniques with PCA and LDA

Document Type : Research Paper

Authors

1 Electrical Engineering Dept., University of Technology-Iraq, Alsina’a street, 10066 Baghdad, Iraq.

2 Communications Engineering Dept., University of Technology-Iraq, Alsina’a street, 10066 Baghdad, Iraq.

Abstract

People need a communication channel and a way to communicate with a person or group. Deaf people communicate with others through sign language. The remarkable and rapid development in image and video recognition systems has made researchers use this development to solve many problems, including sign language for the deaf, and reduce their suffering in communicating with ordinary people. This work aims to use and apply machine learning technology to build an automatic recognition system for Arabic Sign Language (ArSL). In this work, images of ArSL characters were recognized using four classification techniques (Naïve Bayes (NB), Decision Trees (DTs), and Adaptive Boosting), and K-Nearest Neighbor (KNN)) with the Python library and using two feature extraction algorithms (PCA & LDA). Data pre-processing steps,  including grayscale conversion,  Gaussian blur, histogram equalization, and resizing, are applied to enhance the data's suitability for training and testing. The work was tested with five experiments chosen with multiple ratios for training and test data. The first training is 90%, the second training is 80% of the data, the third is 75%, the fourth is 70%, and the last is 60%. The work also played a good role in interpreting ArSL, and the accuracy of the work considers the KNN algorithm more accurate in prediction.

Graphical Abstract

Highlights

  • An automatic system translated Arabic Sign Language to improve deaf-hearing communication.
  • KNN algorithm had 86.4% accuracy in recognizing 32 Arabic sign language letters.
  • The system bridges the communication gap between sign language users and non-users.

Keywords

Main Subjects

References
  1. Alnahhas, B. Alkhatib, N. Al-Boukaee, N. Alhakim, O. Alzabibi, and N. Ajalyakeen, Enhancing the recognition of Arabic sign language by using deep learning and leap motion controller, Int. J. Sci. Technol. Res, 9 (2020) 1865–1870.
  2. Das, S. Gawde, K. Suratwala, and D. Kalbande, Sign language recognition using deep learning on custom processed static gesture images, Int. Conf. Smart City Emerg. Technol., (2018) 1–6. https://doi.org/10.1109/ICSCET.2018.8537248
  3. Jain, A. Jain, A. Chauhan, S. S. Kotla, and A. Gautam, American sign language recognition using support vector machine and convolutional neural network, Int. J. Inf. Technol., 13 (2021) 1193–1200.
  4. N. Saha, S. Tapadar, S. Ray, S. K. Chatterjee, and S. Saha, A machine learning based approach for hand gesture recognition using distinctive feature extraction, in 2018 IEEE 8th Annual Computing and Communication Workshop and Conference, (2018) 91–98. https://doi.org/10.1109/CCWC.2018.8301631
  5. Krishnaraj, M. G. Kavitha, T. Jayasankar, and K. V. Kumar, A Glove based approach to recognize Indian Sign Languages, Int. J. Recent Technol. Eng. IJRTE, 7 (2019) 1419–1425.
  6. M. Mohammed and S. M. Kadhem, A review on Arabic sign language translator systems, J. Phys. Conf. Ser., 1818 (2021) 12033. https://doi.org/10.1088/1742-6596/1818/1/012033
  7. Aloysius and M. Geetha, Understanding vision-based continuous sign language recognition, Multimed. Tools Appl., 79 (2020) 22177– 22209. https://doi.org/10.1007/s11042-020-08961-z
  8. Alshomrani, L. Aljoudi, and M. Arif, Arabic and American sign languages alphabet recognition by convolutional neural network, Adv. Sci. Technol. Res. J., 15 (2021) 136 –148. https://doi.org/10.12913/22998624/142012
  9. Huang, W. Zhou, H. Li, and W. Li, Attention-based 3D-CNNs for large-vocabulary sign language recognition, Trans. Circuits Syst. Video Technol., 29 (2019) 2822–2832. https://doi.org/10.1109/TCSVT.2018.2870740
  10. Hameed et al., Privacy-preserving British sign language recognition using deep learning, 44th Annual International Conference of the Eng. Med. Biol. Soc., (2022) 4316 – 4319. https://doi.org/10.36227/techrxiv.19170257.v1
  11. AL-Muifraje, T. Saeed, and M. Hassan, Design and Analysis of American Sign Language Classifier Based on n-tuple Technique, Eng. Technol. J., 36 (2018) 172–178. https://doi.org/10.30684/etj.36.2A.8
  12. H. Alrubayi et al., A pattern recognition model for static gestures in Malaysian sign language based on machine learning techniques, Comput. Electr. Eng., 95 (2021) 107383. https://doi.org/10.1016/j.compeleceng.2021.107383
  13. Kasapbaşi, A. E. A. Elbushra, A.-H. Omar, and A. Yilmaz, DeepASLR: A CNN based human-computer interface for American Sign Language recognition for hearing-impaired individuals, Comput. Methods Programs Biomed. Update., 2 (2022) 100048. https://doi.org/10.1016/j.cmpbup.2021.100048
  14. R. Sanmitra, V. V. S. Sowmya, and K. Lalithanjana, Machine Learning Based Real Time Sign Language Detection, Int. J. Res. Eng. Sci. Manag., 4 (2021) 137–141.
  15. A. A. Youssif, A. E. Aboutabl, and H. H. Ali, Arabic sign language (arsl) recognition system using hmm, Int. J. Adv. Comput. Sci. Appl., 2 (2011). https://dx.doi.org/10.14569/IJACSA.2011.021108
  16. S. Elons, GPU implementation for Arabic sign language real-time recognition using multi-level multiplicative neural networks, Int. Conf. Comput. Eng. Syst., (2014) 360 –367. https://doi.org/10.1109/ICCES.2014.7030986
  17. D. Rosero-Montalvo et al., Sign language recognition based on intelligent glove using machine learning techniques, Third Ecuador Technical Chapters Meeting (ETCM), 2018, 1–5. https://dx.doi.org/10.1109/ETCM.2018.8580268
  18. Elpeltagy, M. Abdelwahab, M. E. Hussein, A. Shoukry, A. Shoala, and M. Galal, Multi‐modality‐based Arabic sign language recognition, IET Comput. Vis., 12 (2018) 1031–1039. https://doi.org/10.1049/iet-cvi.2017.0598
  19. Alzohairi, R. Alghonaim, W. Alshehri, and S. Aloqeely, Image-based Arabic sign language recognition system, Int. J. Adv. Comput. Sci. Appl., 9 (2018). http://dx.doi.org/10.14569/IJACSA.2018.090327
  20. Tharwat, A. M. Ahmed, and B. Bouallegue, Arabic sign language recognition system for alphabets using machine learning techniques, J. Electr. Comput. Eng., 2021 (2021) 1–17. https://doi.org/10.1155/2021/2995851
  21. Latif, N. Mohammad, J. Alghazo, R. AlKhalaf, and R. AlKhalaf, ArASL: Arabic alphabets sign language dataset, Data Br., 23 (2019) 103777. https://doi.org/10.1016/j.dib.2019.103777
  22. A. H. Alrubaie and A. H. Hameed, Dynamic weights equations for converting grayscale image to RGB image, J. Univ. Babylon Pure Appl. Sci., 26 (2019) 122–129. https://doi.org/10.29196/jubpas.v26i8.1677
  23. Panigrahy, A. Seal, N. K. Mahato, and D. Bhattacharjee, Differential box-counting methods for estimating fractal dimension of greyscale images: A survey, Chaos, Solitons & Fractals, 126 (2019) 178–202. https://doi.org/10.1016/j.chaos.2019.06.007
  24. Flusser, S. Farokhi, C. Höschl, T. Suk, B. Zitova, and M. Pedone, Recognition of images degraded by Gaussian blur, IEEE Trans. Image Process., 25 (2015) 790 – 806. https://doi.org/10.1109/TIP.2015.2512108
  25. Patel and M. Goswami, Comparative analysis of Histogram Equalization techniques, International Conference on Contemporary Computing and Informatics, (2014) 167–168. https://doi.org/10.1109/IC3I.2014.7019808
  26. Dubey and M. Dixit, Image Enhancement Techniques: An Exhaustive Review, Int. Conf. Sust. Innov. Sol. Current Chall. Eng. Technol., (2019) 363 – 375. http://dx.doi.org/10.1007/978-3-030-44758-8_34
  27. S. Prasanna and P. V Virparia, A comparative analysis of image interpolation algorithms, Int. J. Adv. Res. Comput. Commun. Eng., 5 (2016) 29 – 34. http://dx.doi.org/10.17148/IJARCCE.2016.5107
  28. W. Adiyasa, A. P. Prasetyono, A. Yudianto, P. P. W. Begawan, and D. Sultantyo, Bilinear interpolation method on 8x8 pixel thermal camera for temperature instrument of combustion engine, J. Phys. Conf. Ser., 1700 (2020) 12076. http://dx.doi.org/10.1088/1742-6596/1700/1/012076
  29. M. Ebied, Feature extraction using PCA and Kernel-PCA for face recognition, 2012 8th International Conference on Informatics and Systems (INFOS), 2012, MM–72.
  30. Aly, S. Aly, and S. Almotairi, User-independent American sign language alphabet recognition based on depth image and PCANet features, IEEE Access, 7 (2019) 123138 –123150. http://dx.doi.org/10.1109/ACCESS.2019.2938829
  31. M. Angona et al., Automated Bangla sign language translation system for alphabets by means of MobileNet, TELKOMNIKA (Telecommunication Comput. Electron. Control., 18 (2020) 1292–1301. http://dx.doi.org/10.12928/TELKOMNIKA.v18i3.15311
  32. Sharma and K. K. Paliwal, Linear discriminant analysis for the small sample size problem: an overview, Int. J. Mach. Learn. Cybern., 6 (2015) 443–454. https://doi.org/10.1007/s13042-013-0226-9
  33. Deriche, S. O. Aliyu, and M. Mohandes, An intelligent Arabic sign language recognition system using a pair of LMCs with GMM based classification, IEEE Sens. J., 19 (2019) 8067–8078. https://doi.org/10.1109/JSEN.2019.2917525
  34. Hisham and A. Hamouda, Supervised learning classifiers for Arabic gestures recognition using Kinect V2, SN Appl. Sci., 1 (2019) 1–21. https://doi.org/10.1007/s42452-019-0771-2
  35. Silahudin and A. Holidin, Model expert system for diagnosis of COVID-19 using naïve Bayes classifier, IOP Conf. Ser. Mater. Sci. Eng., 1007 (2020) 12067. https://doi.org/10.1088/1757-899X/1007/1/012067
  36. Cherfi, K. Nouira, and A. Ferchichi, Very fast C4. 5 decision tree algorithm, Appl. Artif. Intell., 32 (2018) 119–137. https://doi.org/10.1080/08839514.2018.1447479
  37. Hisham and A. Hamouda, Arabic sign language recognition using Ada-Boosting based on a leap motion controller, Int. J. Inf. Technol., 13 (2021) 1221–1234. https://doi.org/10.1007/s41870-020-00518-5
  38. Taherkhani, G. Cosma, and T. M. McGinnity, AdaBoost-CNN: An adaptive boosting algorithm for convolutional neural networks to classify multi-class imbalanced datasets using transfer learning, Neurocomputing, 404 (2020) 351–366. https://doi.org/10.1016/j.neucom.2020.03.064
  39. Shahraki, M. Abbasi, and Ø. Haugen, Boosting algorithms for network intrusion detection: A comparative evaluation of Real AdaBoost, Gentle AdaBoost and Modest AdaBoost, Eng. Appl. Artif. Intell., 94 (2020) 103770. https://doi.org/10.1016/j.engappai.2020.103770
  40. Utaminingrum, I. K. Somawirata, and G. D. Naviri, Alphabet Sign Language Recognition Using K-Nearest Neighbor Optimization., J. Comput., 14 (2019) 63–70. https://doi.org/10.17706/jcp.14.1.63-70
  41. K. Hayder and A. L. Behadili, Classification Algorithms for Determining Handwritten Digit, Electr. Electron. Eng., 12 (2016) 96–102. https://doi.org/10.37917/ijeee.12.1.10
  42. Zhang, X. Li, M. Zong, X. Zhu, and R. Wang, Efficient kNN classification with different numbers of nearest neighbours, IEEE Trans. Neural networks Learn. Syst., 29 (2017) 1774 –1785. https://doi.org/10.1109/TNNLS.2017.2673241
  43. A. Alani, & Cosma, G., ArSL-CNN: a convolutional neural network for Arabic sign language gesture recognition. Indones. J. Electr. Eng. Comput. Sci., 22 (2021). http://doi.org/10.11591/ijeecs.v22.i2.pp1096-1107
Engineering and Technology Journal
Volume 42, Issue 2
Electrical Engineering
6 articles
February 2024
Page 298-311
Files
Share
Export Citation
Statistics