Abstract:In this paper, we propose to estimate 3D hand pose by recovering the 3D coordinates of joints in a group-wise manner, where less-related joints are automatically categorized into different groups and exhibit different features. This is different from the previous methods where all the joints are considered holistically and share the same feature. The benefits of our method are illustrated by the principle of multi-task learning (MTL), i.e., by separating less-related joints into different groups (as different tasks), our method learns different features for each of them, therefore efficiently avoids the negative transfer (among less related tasks/groups of joints). The key of our method is a novel binary selector that automatically selects related joints into the same group. We implement such a selector with binary values stochastically sampled from a Concrete distribution, which is constructed using Gumbel softmax on trainable parameters. This enables us to preserve the differentiable property of the whole network. We further exploit features from those less-related groups by carrying out an additional feature fusing scheme among them, to learn more discriminative features. This is realized by implementing multiple 1x1 convolutions on the concatenated features, where each joint group contains a unique 1x1 convolution for feature fusion. The detailed ablation analysis and the extensive experiments on several benchmark datasets demonstrate the promising performance of the proposed method over the state-of-the-art (SOTA) methods. Besides, our method achieves top-1 among all the methods that do not exploit the dense 3D shape labels on the most recently released FreiHAND competition at the submission date. The source code and models are available at https://github.com/ moranli-aca/LearnableGroups-Hand.

Semi-Supervised Joint Learning for Hand Gesture Recognition from a Single Color Image

Semi-supervised RGB-D Hand Gesture Recognition Via Mutual Learning of Self-supervised Models

Semi-Supervised Learning for Surface EMG-based Gesture Recognition.

Hand Pose Estimation through Semi-Supervised and Weakly-Supervised Learning

Semg-Based Gesture Recognition with Embedded Virtual Hand Poses and Adversarial Learning.

Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM

Deep Attention Network for Joint Hand Gesture Localization and Recognition Using Static RGB-D Images.

Learning a Deep Predictive Coding Network for a Semi-Supervised 3D-Hand Pose Estimation

Simultaneous 3D Hand Detection and Pose Estimation Using Single Depth Images

3D Hand Pose Estimation Using Synthetic Data and Weakly Labeled RGB Images.

Interacting Two-Hand 3D Pose and Shape Reconstruction from Single Color Image

Extreme Learning Machine Based Hand Posture Recognition in Color-Depth Image.

Exploiting Learnable Joint Groups for Hand Pose Estimation

3D Hand Shape and Pose Estimation from a Single RGB Image

A Survey of Deep Learning-based Hand Pose Estimation

Learning Hand Latent Features for Unsupervised 3D Hand Pose Estimation

Hand Gesture Recognition with Skin Detection and Deep Learning Method

Weakly-Supervised 3d Hand Pose Estimation From Monocular Rgb Images

A 3D Hand Attitude Estimation Method for Fixed Hand Posture Based on Dual-View RGB Images

Hand gesture segmentation, recognition and application

Semi-supervised Learning for RGB-D Object Recognition.