Abstract:Three-dimensional hand detection from a single RGB-D image is an important technology which supports many useful applications. Practically, it is challenging to robustly detect human hands in unconstrained environments because the RGB-D channels can be affected by many uncontrollable factors, such as light changes. To tackle this problem, we propose a 3D hand detection approach which improves the robustness and accuracy by adaptively fusing the complementary features extracted from the RGB-D channels. Using the fused RGB-D feature, the 2D bounding boxes of hands are detected first, and then the 3D locations along the z-axis are estimated through a cascaded network. Furthermore, we represent a challenging RGB-D hand detection dataset collected in unconstrained environments. Different from previous works which primarily rely on either the RGB or D channel, we adaptively fuse the RGB-D channels for hand detection. Specifically, evaluation results show that the D-channel is crucial for hand detection in unconstrained environments. Our RGB-D fusion-based approach significantly improves the hand detection accuracy from 69.1 to 74.1 comparing to one of the most state-of-the-art RGB-based hand detectors. The existing RGB- or D-based methods are unstable in unseen lighting conditions: in dark conditions, the accuracy of the RGB-based method significantly drops to 48.9, and in back-light conditions, the accuracy of the D-based method dramatically drops to 28.3. Compared with these methods, our RGB-D fusion based approach is much more robust without accuracy degrading, and our detection results are 62.5 and 65.9, respectively, in these two extreme lighting conditions for accuracy.

Towards interpretable and robust hand detection via pixel-wise prediction

A Pixel-Level Explainable Approach of Convolutional Neural Networks and Its Application

Dual Regression for Efficient Hand Pose Estimation

A Robust Context Attention Network for Human Hand Detection

Joint Hand Detection and Rotation Estimation by Using CNN

Interpretable Learning: A Result-Oriented Explanation for Automatic Cataract Detection

A Lightweight Hand-Gesture Recognition Network With Feature Fusion Prefiltering and FMCW Radar Spatial Angle Estimation

Improving Network Interpretability via Explanation Consistency Evaluation

Robust Hand Tracking Based on Online Learning and Multi-Cue Flocks of Features

A novel art gesture recognition model based on two channel region-based convolution neural network for explainable human-computer interaction understanding

Robust 3D Hand Detection from a Single RGB-D Image in Unconstrained Environments

SqueezeNet and Fusion Network-Based Accurate Fast Fully Convolutional Network for Hand Detection and Gesture Recognition

Efficient Fully Convolutional Network and Optimization Approach for Robotic Grasping Detection Based on RGB-D Images

SPMHand: Segmentation-guided Progressive Multi-path 3D Hand Pose and Shape Estimation

Detection Method of Manipulator Grasp Pose Based on RGB-D Image

Visual Hand Gesture Recognition With Convolution Neural Network

Deeper Interpretability of Deep Networks

Tensorized Multi-View Low-Rank Approximation Based Robust Hand-Print Recognition

Pixel-wise Regression: 3D Hand Pose Estimation via Spatial-form Representation and Differentiable Decoder

First-Person Hand Action Recognition Using Multimodal Data

Detecting and Restoring Non-Standard Hands in Stable Diffusion Generated Images