Abstract:In this paper, a hand pose estimation method is introduced that combines MobileNetV3 and CrossInfoNet into a single pipeline. The proposed approach is tailored for mobile phone processors through optimizations, modifications, and enhancements made to both architectures, resulting in a lightweight solution. MobileNetV3 provides the bottleneck for feature extraction and refinements while CrossInfoNet benefits the proposed system through a multitask information sharing mechanism. In the feature extraction stage, we utilized an inverted residual block that achieves a balance between accuracy and efficiency in limited parameters. Additionally, in the feature refinement stage, we incorporated a new best-performing activation function called “activate or not” ACON, which demonstrated stability and superior performance in learning linearly and non-linearly gates of the whole activation area of the network by setting hyperparameters to switch between active and inactive states. As a result, our network operated with 65% reduced parameters, but improved speed by 39% which is suitable for running in a mobile device processor. During experiment, we conducted test evaluation on three hand pose datasets to assess the generalization capacity of our system. On all the tested datasets, the proposed approach demonstrates consistently higher performance while using significantly fewer parameters than existing methods. This indicates that the proposed system has the potential to enable new hand pose estimation applications such as virtual reality, augmented reality and sign language recognition on mobile devices.

FastHand: Fast monocular hand pose estimation on embedded systems

Dual Regression for Efficient Hand Pose Estimation

Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data

Real-Time Energy Efficient Hand Pose Estimation: A Case Study

Fast and Robust Hand Tracking Using Detection-Guided Optimization

Estimate Hand Poses Efficiently from Single Depth Images

Efficient 2.5D Hand Pose Estimation via Auxiliary Multi-Task Training for Embedded Devices

Lightweight Architecture for Real-Time Hand Pose Estimation with Deep Supervision

Fast Single Shot Detection and Pose Estimation

Efficient Hand Gesture Recognition for Human-Robot Interaction

HandOS: 3D Hand Reconstruction in One Stage

Hand LightWeightNet: an optimized hand pose estimation for interactive mobile interfaces

FastDepth: Fast Monocular Depth Estimation on Embedded Systems

Attention! A Lightweight 2D Hand Pose Estimation Approach

Fast and Accurate 3D Hand Pose Estimation via Recurrent Neural Network for Capturing Hand Articulations

A Real-Time Hand Pose Estimation System with Retrieval.

SmartHand: Towards Embedded Smart Hands for Prosthetic and Robotic Applications

Fast 3D Hand Estimation for Mobile Interactions

Light and Fast Hand Pose Estimation from Spatial-Decomposed Latent Heatmap

Using a single RGB frame for real time 3D hand pose estimation in the wild

Fast and Robust Dynamic Hand Gesture Recognition via Key Frames Extraction and Feature Fusion