Abstract

This research anticipates the issue of real-time sign language recognition as a means of facilitating the daily communication of the hearing-impaired persons. Improvement in the rate of real time sign language recognition will help in boosting interaction in the society and access to services for the hearing-impaired persons. This paper outlines a step-by-step guide on fine tuning an object detection model for the AASL using the SSD architecture. The AASL dataset which includes total of 7,857 samples of more than 200 patients are first preprocessed where the images are scaled, normalized, and augmented. In SSD model, the base of the network is the VGG16 network and few extra layers for feature extraction and for auxiliary and prediction of objects are added. Particularly in object detection, Intersection over Union (IoU), and mean Average Precision (mAP) are used in addition to the confusion matrix metrics comprising accuracy, precision, recall, and F1-score. The proposed model provides good recognition accuracy of 98 percent. This feature checks its capability of real-time identification of sign languages and shows 25% efficiency. The efficiency of the proposed method is explained when comparing the results of the modified VGG16-based SSD with other methodologies. As for the improvement of communication for the hearing-impaired individuals, the present work demonstrates that the deep learning methods can be significantly effective; however, suggesting that more research efforts should be directed to real-time solutions and that the datasets should be expanded.