QMGR-Net: quaternion multi-graph reasoning network for 3D hand pose estimation
Haomin Ni,Shengli Xie,Pingping Xu,Xiaozhao Fang,Weijun Sun,Ribo Fang
DOI: https://doi.org/10.1007/s13042-023-01879-6
2023-06-21
International Journal of Machine Learning and Cybernetics
Abstract:Abstract3D Hand Pose Estimation (HPE) is a task of recognizing the 3D joints of the hand from static images or dynamic video frames, which is an important topic in human-computer interaction and computer vision. Even though the rise of deep learning promotes the accuracy quite a lot, previous CNN-based methods still suffer from two drawbacks in nature. On one hand, real-valued CNNs break the original structure of the RGB images, and learn color features dependencies and hidden relations mixedly resulting in prediction difficulty when skin is similar. On the other hand, the feature representation for each pixel is only learned by local convolution while the global context information is ignored. To alleviate these problems, we propose a deep learning architecture, named Quaternion Multi-Graph Reasoning Network (QMGR-Net), which contains two novel modules: Quaternion Separation Learning Module (RSLM) and Global Quaternion Reasoning Module (GQRM). The RSLM is proposed to learn color feature dependencies and hidden relations such as edges or shapes at the different level where features can be learned in the quaternion space. Besides, the GQRM is proposed to explicitly capture the interactions among hand joints and model the relationship of non-adjacent nodes. Compared with other state-of-the-art (SOTA) methods on the STB dataset and the RHD dataset, we achieve the best results with 7.27 EPE and 18.59 EPE, respectively, which exhibit the effectiveness of our method.
computer science, artificial intelligence