Dactylology is a technique used by individuals who are deaf or heard of hearing to communicate by making signs with their fingers, particularly in manual alphabets. The goal of this project is to create a functional, real-time American Sign Language (ASL) recognition system using vision-based methods through finger spelling gestures and provide real-time text or speech outputs for individuals who are deaf and mute. A convolution neural network (CNN) algorithm has been employed. A major benefit of CNNs is their ability to perform image classification with minimal pre-processing when compared to other algorithms. Unlike other approaches that use manually designed filters, CNNs learn these filters automatically through training. By properly displaying the ASL symbols and ensuring adequate lighting without background noise, the system was able to detect nearly all of the symbols accurately. The proposed methodology achieved an accuracy of 94.8% for the 26 letters of the alphabet.
Top1. Introduction
Dactylology communication is a visual-spatial language used by hard of hearing and deaf individuals to impart. Since various gesture-based communications are used in various nations and locales, it's anything but a general language. For instance, Spanish sign language in used in Spain, BSL in the United Kingdom, and Auslan is used in Australia. In this paper, we aimed to foster a framework that can consequently decipher communication through signing and give text or speech results for mute and deaf people for fingerspelling-based motions to frame a total word by joining each signal. In the existing system a Linear Discriminant Examination (LDA) algorithm was employed for gesture recognition, and the perceived signal is changed over into text and voice design (Muthu Mariappan & Gomathi, 2019). A language barrier is created when communicating between individuals who use sign language and those who do not, as the structure of sign language is different from that of normal text (Ojha et al., 2020). As a result, individuals who are deaf or hard of hearing rely on vision-based communication to interact with others (Narang & Sharma, 2021). Assuming that there was a typical channel that could change over communication through signing into text, gestures could be handily inferred by people who don't use sign language (Subbarayudu et al., 2017). Subsequently, research has been led to foster a vision based interface framework that would allow individuals who are deaf or hard of hearing to communicate without knowing each other's language. American sign language is a prevalent gesture-based communication and is broadly utilized by individuals everywhere in the world. The ASL letter sets displayed in Figure 1 comprise of manual letters from a to z, with each letter representing a specific information (Bhamare et al., 2022; Chong & Lee, 2018; Narasimha Prasad et al., 2013). In contrast, the act of gesturing can communicate a complete emotional message (Anisha, Reddy, & Nguyen, 2021). By making a human interface that accepts ASL as input and converts it into audio, people all around the world can speak with one another with practically no language boundaries (Hoffmeister & Gee, 1983). Convolution Neural Networks (CNNs) are artificial neural networks that are specifically designed to require minimal pre-processing (Rao et al., 2018). One of the main advantages of CNNs is that they require somewhat little pre-processing contrasted with other picture characterization algorithms (Reddy, 2012). Instead of relying on hand-engineered filters, the network learns these filters on its own, which eliminates the need for previous knowledge and human involvement in feature design. The existing algorithm for sign language detection using CNN showed an accuracy of 99% but it was used for Spanish based sign language which is the drawback of that system. The proposed system addresses this drawback where we employed CNN algorithm to detect and recognize ASL.
The remaining contents of the paper is organized as follows 2) gives the framework of the model and its methodology 3) gives the results and discussion of the proposed model 4) gives the concluding statements followed by the references.
Figure 1. American Sign Language (ASL)