Abstract:Head pose estimation methods can be generally classified into two categories: model-based and appearance-based methods. The model-based approach relies on facial landmarks for three-dimensional reconstruction, aiming to achieve high-precision results. However, this method is heavily dependent on the accuracy of these landmarks. The appearance-based approach utilizes images as input and employs feature extraction and calculations to generate outcomes. While the appearance-based method boasts greater robustness, its accuracy falls short of the former. In this paper, a new and effective hybrid method is proposed. This hybrid approach combines the strengths of both methods. Unlike the conventional model-based methods, the proposed method regards the facial landmarks in 2D images as a sequence of neural network inputs and then obtains the head pose estimation results for users by neural network regression. The proposed method solves the fuzzy rotation labeling problem by using a rotation matrix representation, introducing a 6D rotation matrix representation as an intermediate state of the rotation matrix to achieve effective direct regression. Introducing face processing enhances the robustness of the model in cross-dataset scenarios. The proposed method achieves remarkable results based on imprecise face recognition and a simplistic model. The proposed method can be divided into three parts. First, the proposed method applies face processing on the input image; second, the method detects facial landmarks; and third, it converts these facial landmarks into sequences and obtains the 6D rotation representation of the head pose by regression. Extensive experiments on the publicly available BIWI, PRIMA, and DrivFace datasets show that this method is functional and performs better than other state-of-the-art methods. Compared to other methods, this approach demonstrates an average performance improvement of at least 10% across the dataset.

Rotation Prediction Based Representative View Locating Framework for 3D Object Recognition

RINet: Efficient 3D Lidar-Based Place Recognition Using Rotation Invariant Neural Network

MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images

Direction-Sensitivity Features Ensemble Network for Rotation-Invariant Face Detection.

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

3D-SSD: Learning Hierarchical Features from RGB-D Images for Amodal 3D Object Detection

A Unified Feature Representation and Learning Framework for 3D Shape

On Representation of 3D Rotation in the Context of Deep Learning

Variable-Viewpoint Representations for 3D Object Recognition

3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

Learning Disentangled Representation for Multi-View 3D Object Recognition.

ReINView: Re-interpreting Views for Multi-view 3D Object Recognition

Reinforced Axial Refinement Network for Monocular 3D Object Detection

A New Rotation-Invariant Deep Network for 3D Object Recognition

Deep Projective Rotation Estimation through Relative Supervision

R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object

6DFLRNet: 6D rotation representation for head pose estimation based on facial landmarks and regression

Learning to Predict the 3D Layout of a Scene

VI-Net: Boosting Category-level 6D Object Pose Estimation via Learning Decoupled Rotations on the Spherical Representations

Recognizing Multi-View Objects with Occlusions Using a Deep Architecture