HTNet: Human Topology Aware Network for 3D Human Pose Estimation

Jialun Cai,Hong Liu,Runwei Ding,Wenhao Li,Jianbing Wu,Miaoju Ban

DOI: https://doi.org/10.48550/arXiv.2302.09790

2023-02-20

Abstract:3D human pose estimation errors would propagate along the human body topology and accumulate at the end joints of limbs. Inspired by the backtracking mechanism in automatic control systems, we design an Intra-Part Constraint module that utilizes the parent nodes as the reference to build topological constraints for end joints at the part level. Further considering the hierarchy of the human topology, joint-level and body-level dependencies are captured via graph convolutional networks and self-attentions, respectively. Based on these designs, we propose a novel Human Topology aware Network (HTNet), which adopts a channel-split progressive strategy to sequentially learn the structural priors of the human topology from multiple semantic levels: joint, part, and body. Extensive experiments show that the proposed method improves the estimation accuracy by 18.7% on the end joints of limbs and achieves state-of-the-art results on Human3.6M and MPI-INF-3DHP datasets. Code is available at <a class="link-external link-https" href="https://github.com/vefalun/HTNet" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Human-Computer Interaction,Machine Learning

What problem does this paper attempt to address?

This paper attempts to address the issue in 3D human pose estimation where the complexity of the human topology, especially the dependencies between joints, leads to cumulative estimation errors at the limb end joints. Specifically:

1. **Error Accumulation**: Since the human body is a linked structure, limb joints are highly dependent on their parent nodes (as shown in Figure 1(c)). Therefore, the estimation error from the trunk center hip joint (root joint) to the limb end joints accumulates. Inspired by the backtracking mechanism in automatic control systems, the authors designed an Intra-Part Constraint (IPC) module, which uses the parent joint within the part as a reference to constrain the high-degree-of-freedom joints, thereby reducing error accumulation.

2. **Hierarchical Structure**: Human joint movement is closely related to the hierarchical organization of human topology, including three levels: joints, parts, and body. To fully utilize these hierarchical structures, the authors proposed a novel Human Topology aware Network (HTNet), which learns human topology representations from multiple semantic levels (joints, parts, and body) through a channel-splitting progressive design.

With these designs, HTNet achieved state-of-the-art results on the Human3.6M and MPI-INF-3DHP datasets, particularly improving the estimation accuracy of limb end joints by 18.7%.