Abstract:Three-dimensional hand detection from a single RGB-D image is an important technology which supports many useful applications. Practically, it is challenging to robustly detect human hands in unconstrained environments because the RGB-D channels can be affected by many uncontrollable factors, such as light changes. To tackle this problem, we propose a 3D hand detection approach which improves the robustness and accuracy by adaptively fusing the complementary features extracted from the RGB-D channels. Using the fused RGB-D feature, the 2D bounding boxes of hands are detected first, and then the 3D locations along the z-axis are estimated through a cascaded network. Furthermore, we represent a challenging RGB-D hand detection dataset collected in unconstrained environments. Different from previous works which primarily rely on either the RGB or D channel, we adaptively fuse the RGB-D channels for hand detection. Specifically, evaluation results show that the D-channel is crucial for hand detection in unconstrained environments. Our RGB-D fusion-based approach significantly improves the hand detection accuracy from 69.1 to 74.1 comparing to one of the most state-of-the-art RGB-based hand detectors. The existing RGB- or D-based methods are unstable in unseen lighting conditions: in dark conditions, the accuracy of the RGB-based method significantly drops to 48.9, and in back-light conditions, the accuracy of the D-based method dramatically drops to 28.3. Compared with these methods, our RGB-D fusion based approach is much more robust without accuracy degrading, and our detection results are 62.5 and 65.9, respectively, in these two extreme lighting conditions for accuracy.

Robust 3-D Human Detection in Complex Environments With a Depth Camera.

A Two-Stage Human Body Detector on Depth Data

Real-time human detection and tracking in complex environments using single RGBD camera

Robust Real-Time Human Perception with Depth Camera.

Reliably Detecting Humans in Crowded and Dynamic Environments Using RGB-D Camera

Robust Human Detection with Super-Pixel Segmentation and Random Ferns Classification Using RGB-D Camera

Fast Human Detection in RGB-D Images Based on Color-Depth Joint Feature Learning.

Detecting and tracking people in real time with RGB-D camera

Physical Blob Detector and Multi-Channel Color Shape Descriptor for Human Detection.

Detecting More Objects Indoors: a Depth-guided Detector for Mobile Robots

An ultra-fast human detection method for color-depth camera

Human detection using depth information

Efficient Detection and Tracking of Human Using 3D LiDAR Sensor

A Robust Approach for Person Localization in Multi-camera Environment

Dynamic Depth Fusion and Transformation for Monocular 3D Object Detection.

Expandable YOLO: 3D Object Detection from RGB-D Images

DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries

Deep learning based 3D target detection for indoor scenes

Robust 3D Hand Detection from a Single RGB-D Image in Unconstrained Environments

Multi-Class Detection and Segmentation of Objects in Depth

Depth Is All You Need for Monocular 3D Detection