Indian Sign Language Detection and Translation using Deep Learning and Text-to-Speech

Poovannapoikayil, Aji Vishwambharan (2024) Indian Sign Language Detection and Translation using Deep Learning and Text-to-Speech. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (1MB) | Preview

Abstract

This paper explores the integration of YOLOv10, a cutting-edge object detection model, for real-time static sign recognition and translation of Indian Sign Language (ISL) into regional Hindi text and speech. Motivated by the demand for more effective communication tools for the deaf community in India, particularly in non-English speaking regions, this research compares the performance of YOLOv10 against the established YOLOv5 model. The study focused on key metrics such as accuracy, Mean Average Precision (mAP), precision, and inference time to assess the efficacy of YOLOv10 in ISL detection. The results demonstrate that YOLOv10 significantly improves upon the mAP@50:95 accuracy of YOLOv5, which was 89%, achieving 99% mAP@50:95 accuracy with 25 epochs for trained ISL words, along with superior precision and faster inference times. However, the false positive cases suggest potential overfitting, indicating the need for future work to refine the model. The findings further suggest that YOLOv10 offers enhanced real-time performance, making it a viable solution for improving accessibility and communication in both rural and urban areas of India. However, the research also identifies limitations, particularly related to the availability of diverse, high-quality data and the time-intensive nature of manual annotation. Future work will address these challenges by expanding the dataset to include a wider range of words and dynamic gestures, and by exploring the integration of Long Short-Term Memory (LSTM) networks to better capture complex sign language elements. This study not only advances the field of sign language recognition but also holds significant potential for commercial applications, particularly in developing assistive communication tools tailored to the Indian context.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Hamill, David UNSPECIFIED
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science P Language and Literature > P Philology. Linguistics > Semiotics > Language. Linguistic theory > Gesture. Sign language Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning
Divisions:	School of Computing > Master of Science in Data Analytics
Depositing User:	Ciara O'Brien
Date Deposited:	25 Aug 2025 10:05
Last Modified:	25 Aug 2025 10:05
URI:	https://norma.ncirl.ie/id/eprint/8613

Actions (login required)

View Item