Abstract:Facial landmark detection, head pose estimation, and facial deformation analysis are typical facial behavior analysis tasks in computer vision. The existing methods usually perform each task independently and sequentially, ignoring their interactions. To tackle this problem, we propose a unified framework for simultaneous facial landmark detection, head pose estimation, and facial deformation analysis, and the proposed model is robust to facial occlusion. Following a cascade procedure augmented with model-based head pose estimation, we iteratively update the facial landmark locations, facial occlusion, head pose and facial de- formation until convergence. The experimental results on benchmark databases demonstrate the effectiveness of the proposed method for simultaneous facial landmark detection, head pose and facial deformation estimation, even if the images are under facial occlusion.
What problem does this paper attempt to address?
The problems that this paper attempts to solve are several key tasks in facial behavior analysis - facial landmark detection, head pose estimation, and facial deformation analysis. These tasks become more complex and challenging when the face is occluded. Existing methods usually perform each task independently and sequentially, ignoring the interactions among them. To solve this problem, the authors propose a unified framework that can perform facial landmark detection, head pose estimation, and facial deformation analysis simultaneously, and the model is robust to facial occlusion.
Specifically, the goals of the paper include:
1. **Unified framework**: Develop a method that can handle facial landmark detection, head pose estimation, and facial deformation analysis simultaneously, and utilize the joint relationships among these tasks to improve the performance of all tasks.
2. **Robustness**: Ensure that the proposed method is still effective when the face is partially or completely occluded.
3. **Iterative update**: Through a cascading process, combined with model - based head pose estimation, iteratively update the facial landmark positions, facial occlusion, head pose, and facial deformation until convergence.
4. **Experimental verification**: Prove the effectiveness of the proposed method through experimental results on benchmark databases, especially in the case of facial occlusion.
The main contributions of the paper are:
- Propose an iterative cascading method that can perform facial landmark detection, pose, and deformation estimation simultaneously, which is different from most existing methods that are processed independently or sequentially.
- Systematically integrate learning - based facial landmark detection with model - based head pose and facial deformation estimation without 3D annotations.
- Explicitly estimate facial occlusion, which is helpful for landmark detection, pose, and deformation estimation in the case of facial occlusion.
- Experimental results show the effectiveness of the proposed method for facial landmark detection, pose estimation, and deformation estimation in the case of facial occlusion.
In conclusion, this paper aims to solve the comprehensive challenges of multiple tasks in facial behavior analysis under facial occlusion conditions through a unified and robust framework.