Abstract:This paper presents a novel and efficient head pose estimation framework based on particle swarm optimized contrastive learning and multimodal entangled graph convolution network, aiming to solve the complexity nonlinear mapping from 2D feature space to 3D pose space. Head pose estimation is an especially challenging task due to the complexity nonlinear mapping from 2D feature space to 3D pose space. To address the above issue, this paper presents a novel and efficient head pose estimation framework based on particle swarm optimized contrastive learning and multimodal entangled graph convolution network. Firstly, a new network, the region and difference‐aware feature pyramid network (RD‐FPN), is proposed for 2D keypoints detection to alleviate the background interference and enhance the feature expressiveness. Then, particle swarm optimized contrastive learning is constructed to alternatively match 2D and 3D keypoints, which takes the multimodal keypoints matching accuracy as the optimization objective, while considering the similarity of cross‐modal positive and negative sample pairs from contrastive learning as a local contrastive constraint. Finally, multimodal entangled graph convolution network is designed to enhance the ability of establishing geometric relationships between keypoints and head pose angles based on second‐order bilinear attention, in which point‐edge attention is introduced to improve the representation of geometric features between multimodal keypoints. Compared with other methods, the average error of our method is reduced by 8.23%, indicating the accuracy, generalization, and efficiency of our method on the 300W‐LP, AFLW2000, BIWI datasets.

Head Pose Estimation Using Covariance of Oriented Gradients

A Study of Two Image Representations for Head Pose Estimation

FDN: Feature Decoupling Network for Head Pose Estimation.

Direct Alignment with Generalized Correspondences: A Unified Framework for Structure-Based Visual Pose Estimation.

Head Pose Estimation Based on a Second Order Histogram of the Orientation Gradient

Partially Occluded Head Posture Estimation for 2D Images Using Pyramid HoG Features

Geometric and appearance model based approach for head pose recovery in monocular image sequence

Head-Pose Estimation Based on Lateral Canthus Localizations in 2-D Images

2DHeadPose: A simple and effective annotation method for the head pose in RGB images and its dataset

Head pose estimation from low-resolution image with hough forest

Head pose estimation with particle swarm optimization‐based contrastive learning and multimodal entangled GCN

COGE: A Novel Binary Feature Descriptor Exploring Anisotropy and Non-uniformity.

Head Pose Estimation on Top of Haar-Like Face Detection: A Study Using the Kinect Sensor

Feature Decomposition-Based Gaze Estimation with Auxiliary Head Pose Regression

Covariance descriptors on a Gaussian manifold and their application to image set classification

Robust Head-Pose Estimation Based on Partially-Latent Mixture of Linear Regressions

Light-weight Head Pose Invariant Gaze Tracking

Robust Head Pose Estimation Using LGBP

More About Covariance Descriptors for Image Set Coding: Log-Euclidean Framework based Kernel Matrix Representation

Person-independent head pose estimation based on random forest regression

Head yaw estimation from asymmetry of facial appearance.