Abstract:Head pose estimation methods can be generally classified into two categories: model-based and appearance-based methods. The model-based approach relies on facial landmarks for three-dimensional reconstruction, aiming to achieve high-precision results. However, this method is heavily dependent on the accuracy of these landmarks. The appearance-based approach utilizes images as input and employs feature extraction and calculations to generate outcomes. While the appearance-based method boasts greater robustness, its accuracy falls short of the former. In this paper, a new and effective hybrid method is proposed. This hybrid approach combines the strengths of both methods. Unlike the conventional model-based methods, the proposed method regards the facial landmarks in 2D images as a sequence of neural network inputs and then obtains the head pose estimation results for users by neural network regression. The proposed method solves the fuzzy rotation labeling problem by using a rotation matrix representation, introducing a 6D rotation matrix representation as an intermediate state of the rotation matrix to achieve effective direct regression. Introducing face processing enhances the robustness of the model in cross-dataset scenarios. The proposed method achieves remarkable results based on imprecise face recognition and a simplistic model. The proposed method can be divided into three parts. First, the proposed method applies face processing on the input image; second, the method detects facial landmarks; and third, it converts these facial landmarks into sequences and obtains the 6D rotation representation of the head pose by regression. Extensive experiments on the publicly available BIWI, PRIMA, and DrivFace datasets show that this method is functional and performs better than other state-of-the-art methods. Compared to other methods, this approach demonstrates an average performance improvement of at least 10% across the dataset.

Real-Time Head Pose Estimation and Face Modeling from a Depth Image

Face Alignment Based on 3D Face Shape Model and Markov Random Field.

Learning a Deep Regression Forest for Head Pose Estimation from a Single Depth Image

From Stereo Pair to 3D Face Model

Cascade of Forests for Face Alignment

Joint Head Pose and Facial Landmark Regression from Depth Images

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Robust 3D face modeling and tracking from RGB-D images

Head pose estimation from low-resolution image with hough forest

Person-independent head pose estimation based on random forest regression

Collaborative 3D face alignment and head pose estimation with frontal face constraint based on RGB and sparse depth

Human Head Pose Estimation Through Temporal Enhanced and Accurate Self-Supervised Depth Prediction

Face-from-Depth for Head Pose Estimation on Depth Images

Depth-images-based pose estimation using regression forests and graphical models

Real-Time Accurate 3D Head Tracking and Pose Estimation with Consumer RGB-D Cameras

6DFLRNet: 6D rotation representation for head pose estimation based on facial landmarks and regression

Deep3DPose: Realtime Reconstruction of Arbitrarily Posed Human Bodies from Single RGB Images

3D Hand Pose Estimation Using Randomized Decision Forest with Segmentation Index Points

Head Pose Estimation through Keypoints Matching between Reconstructed 3D Face Model and 2D Image

Face Pose Estimation Based on 3D Models and Affine Correspondences