HGR-FYOLO: a robust hand gesture recognition system for the normal and physically impaired person using frozen YOLOv5
Abir Sen,Shubham Dombe,Tapas Kumar Mishra,Ratnakar Dash
DOI: https://doi.org/10.1007/s11042-024-18464-w
IF: 2.577
2024-02-14
Multimedia Tools and Applications
Abstract:Hand gesture recognition is important in human-machine interaction (HMI), enabling interaction with the systems without physically touching them. But current methodologies regarding gesture recognition face different challenges, such as poor lighting conditions, complex backgrounds, lower detection rates, slower speed, etc. The physically handicapped people with fewer fingers find it difficult to engage with desktop applications in case of complex backgrounds. To overcome these concerns, we have chosen the YOLOv5s model, one small version of YOLOv5 (You Only Look Once) object detection algorithm, to detect and classify the hand portion in a real-time scenario. We have fine-tuned the YOLOv5s architecture by freezing some convolutional layers in the backbone portion, which reduces the number of parameters, model size, and inference time. Two datasets, one public (American sign language) and one custom dataset, named 'NITR-Hand gesture' dataset have been utilized to evaluate the suggested work. There is always a trade-off between accuracy and detection speed. Therefore, the accuracy has been slightly reduced after freezing the backbone of YOLOv5s architecture. but the inference time and detection speed have both improved. Experimental results exhibit that our suggested frozen YOLOv5s model has achieved a mean average precision (mAP@50-95) of 92.60%, and the average speed has reached more than 55 frames per second (fps). We have conducted a comparative analysis of our used models with other state-of-the-art methods, and it is noticed that the fine-tuned YOLOv5s have outperformed other models in terms of mAP and inference speed. This prototype is very fruitful for physically impaired persons to interact with the systems in real-time.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering