Abstract:3D hand pose estimation is a crucial subject in the domain of computer vision. Recently researchers transform a single depth image into multiple virtual view depth images. By projecting a single depth image through point cloud transformation and using the depth images of multiple virtual views together for hand pose estimation, these methods can effectively improve the estimation accuracy. However, current methods have issues with distorted generated depth images, insufficient usage of the depth image of each view, and high computational overhead. To overcome these problems, we introduce a multi-virtual view scoring network (MVSN). Our proposed MVSN consists of a single virtual view estimation module, virtual view feature encoding module, and virtual view scoring module. To generate an intermediate feature map suitable for virtual view scoring, the single virtual view estimation module uses a feature map offset loss function and enhance information interaction between channels in the backbone network. The virtual view feature encoding module adopts a two-branch structure to capture information about all joints and single joints from the intermediate feature map, respectively. This structure effectively improves model sensitivity to each view, better integrates information from each virtual view, and obtains a more appropriate scoring feature for each virtual view. The virtual view scoring module scores each view based on the scoring feature, and gives a higher score to the more accurately estimated virtual view. We also propose a dynamic virtual view removal strategy to remove poor quality views in the training process. Our model is tested on the NYU and ICVL datasets, and the mean joint error is 6.21 mm and 4.53 mm, respectively, exhibiting better estimation accuracy than existing methods.

MVPointNet: Multi-View Network for 3D Object Based on Point Cloud

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

Multi-virtual View Scoring Network for 3D Hand Pose Estimation from a Single Depth Image

PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition.

MVHANet: Multi-view Hierarchical Aggregation Network for Skeleton-Based Hand Gesture Recognition

Local and Global Point Cloud Reconstruction for 3D Hand Pose Estimation

MM-Point: Multi-View Information-Enhanced Multi-Modal Self-Supervised 3D Point Cloud Understanding

Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layer

PVRNet: Point-View Relation Neural Network for 3D Shape Recognition

V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map

Deep Learning Based Pose Estimation Method Using 3D Point Clouds

Hand3D: Hand Pose Estimation using 3D Neural Network

A novel GCN-based point cloud classification model robust to pose variances

A PointNet-Based Solution for 3D Hand Gesture Recognition

Center point to pose: Multiple views 3D human pose estimation for multi-person

MANet: Multimodal Attention Network based Point- View fusion for 3D Shape Recognition

Multi-Head Self-Attention for 3D Point Cloud Classification

View Invariant Human Body Detection and Pose Estimation from Multiple Depth Sensors

Deep Multi-scale Learning on Point Sets for 3D Object Recognition.

Real-Time 3D Hand Pose Estimation with 3D Convolutional Neural Networks

NETWORKS EFFECTIVELY UTILIZING 2D SPATIAL INFORMATION FOR ACCURATE 3D HAND POSE ESTIMATION