Unsupervised Prior Learning: Discovering Categorical Pose Priors from Videos

Ziyu Wang,Shuangpeng Han,Mike Zheng Shou,Mengmi Zhang

2024-10-05

Abstract:A prior represents a set of beliefs or assumptions about a system, aiding inference and decision-making. In this work, we introduce the challenge of unsupervised prior learning in pose estimation, where AI models learn pose priors of animate objects from videos in a self-supervised manner. These videos present objects performing various actions, providing crucial information about their keypoints and connectivity. While priors are effective in pose estimation, acquiring them can be difficult. We propose a novel method, named Pose Prior Learner (PPL), to learn general pose priors applicable to any object category. PPL uses a hierarchical memory to store compositional parts of prototypical poses, from which we distill a general pose prior. This prior enhances pose estimation accuracy through template transformation and image reconstruction. PPL learns meaningful pose priors without any additional human annotations or interventions, outperforming competitive baselines on both human and animal pose estimation datasets. Notably, our experimental results reveal the effectiveness of PPL using learnt priors for pose estimation on occluded images. Through iterative inference, PPL leverages priors to refine estimated poses, regressing them to any prototypical poses stored in memory. Our code, model, and data will be publicly available.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the problem of learning pose priors from videos in an unsupervised manner to enhance the accuracy of pose estimation. Specifically, the paper proposes a new method called Pose Prior Learner (PPL), which can self-supervisedly learn general pose priors from video data without requiring additional manual annotations or interventions. These prior information can be used to guide pose estimation tasks for different categories of objects (such as humans and animals) and perform particularly well on occluded images.

PPL uses hierarchical memory to store different components of prototype poses and extracts general pose priors from them. Experimental results show that PPL not only surpasses existing methods on multiple benchmark datasets but also generates interpretable visual prior information. Additionally, during the inference phase, PPL employs an iterative strategy to gradually optimize the estimated poses, making them regress to the nearest prototype poses stored in memory, thereby improving the accuracy of pose estimation in occluded scenes.