Abstract:3D hand pose estimation is a crucial subject in the domain of computer vision. Recently researchers transform a single depth image into multiple virtual view depth images. By projecting a single depth image through point cloud transformation and using the depth images of multiple virtual views together for hand pose estimation, these methods can effectively improve the estimation accuracy. However, current methods have issues with distorted generated depth images, insufficient usage of the depth image of each view, and high computational overhead. To overcome these problems, we introduce a multi-virtual view scoring network (MVSN). Our proposed MVSN consists of a single virtual view estimation module, virtual view feature encoding module, and virtual view scoring module. To generate an intermediate feature map suitable for virtual view scoring, the single virtual view estimation module uses a feature map offset loss function and enhance information interaction between channels in the backbone network. The virtual view feature encoding module adopts a two-branch structure to capture information about all joints and single joints from the intermediate feature map, respectively. This structure effectively improves model sensitivity to each view, better integrates information from each virtual view, and obtains a more appropriate scoring feature for each virtual view. The virtual view scoring module scores each view based on the scoring feature, and gives a higher score to the more accurately estimated virtual view. We also propose a dynamic virtual view removal strategy to remove poor quality views in the training process. Our model is tested on the NYU and ICVL datasets, and the mean joint error is 6.21 mm and 4.53 mm, respectively, exhibiting better estimation accuracy than existing methods.

Efficient Virtual View Selection for 3D Hand Pose Estimation

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

Multi-virtual View Scoring Network for 3D Hand Pose Estimation from a Single Depth Image

CAMInterHand: Cooperative Attention for Multi-View Interactive Hand Pose and Mesh Reconstruction

Recurrent 3D Hand Pose Estimation Using Cascaded Pose-Guided 3D Alignments

Efficient Encoding and Aligning Viewpoints for 6D Pose Estimation of Unseen Industrial Parts

Efficient Hierarchical Multi-view Fusion Transformer for 3D Human Pose Estimation

Zero-Shot 3d Pose Estimation of Unseen Object by Two-Step Rgb-D Fusion

Simultaneous 3D Hand Detection and Pose Estimation Using Single Depth Images

Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views

View Invariant 3D Human Pose Estimation

Depth-Based 3D Hand Pose Estimation: from Current Achievements to Future Goals

View Consistency Aware Holistic Triangulation for 3D Human Pose Estimation

Human Pose Regression Through Multiview Visual Fusion.

3D Hand Pose Estimation in Everyday Egocentric Images

Single-to-Dual-View Adaptation for Egocentric 3D Hand Pose Estimation

VoxelKeypointFusion: Generalizable Multi-View Multi-Person Pose Estimation

Accurate 3D Hand Pose Estimation Network Utilizing Joints Information.

NETWORKS EFFECTIVELY UTILIZING 2D SPATIAL INFORMATION FOR ACCURATE 3D HAND POSE ESTIMATION

Human Pose Estimation with Regression by Fusing Multi-View Visual Information