Development of Gesture Recognition System for Converting Indian Sign Language Videos to Speech

Abstract

In a world marked by diverse communication methods, the gap between hearing and deaf communities remains a significant challenge. Millions who rely on Indian Sign Language (ISL) face barriers in education, employment, and social interaction. This paper presents a machine learning–based gesture recognition system designed to convert ISL videos into text and speech in real time. Unlike systems that rely solely on pre-trained models, the proposed approach utilizes a self-trained and dynamically adaptive model, built using annotated ISL video data and continuously improved through user interaction. The system addresses key challenges such as dataset collection, ethical considerations, and the complexity of interpreting hand gestures, facial expressions, and body movements. By integrating computer vision techniques with deep learning models and text-to-speech synthesis, the system achieves high accuracy and real-time performance. This work contributes toward developing an inclusive communication framework that enhances accessibility and bridges the communication gap for individuals relying on ISL.