Abstract:To better meet the communication needs of hearing impaired people and the public, it is of great significance to recognize sign language more quickly and accurately on embedded platforms and mobile terminals. YOLOv3, raised by Joseph Redmon and Ali Farhadi in 2018, achieved a great improvement in detection speed with considerable accuracy by optimizing Yolo. However, YOLOv3 is still too bloated to use on mobile terminals. A static sign language recognition method based on the ShuffleNetv2-YOLOv3 lightweight model was proposed. The ShuffleNetv2-YOLOv3 lightweight model makes the network lightweight by using ShuffleNetv2 as the backbone network of YOLOv3. The lightweight network improved the recognition speed steeply. Combing with the CIoU loss function, the ShuffleNetv2-YOLOv3 keeps the recognition accuracy while improving the recognition speed. Recognition effectiveness of the self-made sign language images and public database by the ShuffleNetv2-YOLOv3 lightweight model was evaluated by F1 score and mAP value. The performance of the ShuffleNetv2-YOLOv3 model was compared with that of the YOLOv3-tiny, SSD, Faster-RCNN, and YOLOv4-tiny model, respectively. The experimental results show that the proposed ShuffleNetv2-YOLOv3 model achieved a good balance between the accuracy and speed of the gesture detection under the premise of model lightweight. The F1 score and mAP value of the ShuffleNetv2-YOLOv3 model were 99.1% and 98.4%, respectively. The gesture detection speed on the GPU reaches 54 frames per second, which is better than other models. The mobile terminal application of the proposed lightweight model was also evaluated. The minimal inference speed of single frame images on the CPU and GPU is 0.14 and 0.025 s per image, respectively. It is only 1/6.5 and 1/8.5 of the running speed of the original YOLOv3 model. The ShuffleNetv2-YOLOv3 lightweight model is conducive to quick, real time, and similar static sign language gesture recognition, laying a good foundation for real-time gesture recognition in the embedded platforms and mobile terminals.

Research on Gesture Recognition Based on YOLOv5

Computer Interactive Gesture Recognition Model Based on Improved YOLOv5 Algorithm

Gesture Detection and Recognition Based on Object Detection in Complex Background

Research on Dynamic Gesture Recognition with Low-Pixel ToF-Sensors

Gesture Recognition Controls Image Style Transfer Based on Improved YOLOV5s Algorithm

HGR-FYOLO: a robust hand gesture recognition system for the normal and physically impaired person using frozen YOLOv5

Multimodal Gesture Recognition with Spatio-Temporal Features Fusion Based on YOLOv5 and MediaPipe

A real-time hand gesture recognition method

Human Action Recognition Based on YOLOv7

Novel Human Machine Interface via Robust Hand Gesture Recognition System using Channel Pruned YOLOv5s Model

Football referee gesture recognition algorithm based on YOLOv8s

YOLO Series for Human Hand Action Detection and Classification from Egocentric Videos

Gesture Recognition Based on Localist Attractor Networks with Application to Robot Control [Application Notes]

Object Detection Method for Grasping Robot Based on Improved YOLOv5

Hand Gesture Control for Human–Computer Interaction with Deep Learning

Deep Learning Methods for Human Behavior Recognition.

ShuffleNetv2-YOLOv3: a real-time recognition method of static sign language based on a lightweight network

Facial Expression Recognition Using YOLO

YOLO5Face: Why Reinventing a Face Detector

Research on Automatic Driving Target Detection Based on YOLOv5s