Abstract:This paper presents a partition-based surface registration for 3D morphable model(3DMM). In the 3DMM, it often requires to warp a handcrafted template model into different captured models. The proposed method first utilizes the landmarks to partition the template model then scale each part and finally smooth the boundaries. This method is especially effective when the disparity between the template model and the target model is huge. The experiment result shows the method perform well than the traditional warp method and robust to the local minima.

What problem does this paper attempt to address?

This paper attempts to address the problem of how to deform a handcrafted template model onto different captured models in 3D facial modeling. Specifically, when there is a significant difference between the template model and the target model, traditional non-rigid registration methods often do not work well and are prone to getting stuck in local minima. Therefore, the paper proposes a partition-based non-rigid registration method that effectively solves this problem by segmenting the template model into multiple parts using landmarks, then scaling and smoothing the boundaries of each part. ### Main Issues 1. **Limitations of Traditional Non-Rigid Registration Methods**: - When there is a large difference between the template model and the target model, traditional methods are prone to getting stuck in local minima. - Traditional methods perform poorly when dealing with facial models of different ages, genders, and races. 2. **Specific Challenges to be Addressed**: - How to achieve accurate non-rigid registration when there is a significant difference between the template model and the target model. - How to avoid getting stuck in local minima during the optimization process to ensure the accuracy of the final result. ### Solution 1. **Partition-Based Non-Rigid Registration Method**: - **Partitioning**: Segment the template model into multiple parts using landmarks. - **Scaling**: Scale each part separately to approximate the target model. - **Smoothing Boundaries**: Smooth the boundaries between parts to ensure overall consistency of the model. 2. **Improved Energy Function**: - In addition to the traditional distance term, stiffness term, and landmark term, a normal vector term is introduced to further improve the robustness and accuracy of the registration. ### Experimental Results - By comparing with the traditional non-rigid ICP method (NICP), experimental results show that the proposed method performs better in handling complex facial features (such as the nose and mouth) and effectively avoids getting stuck in local minima. - Using a partitioning strategy with 68 Dlib landmarks, the template model was successfully registered to 3D models with different expressions. ### Conclusion The paper proposes a partition-based non-rigid registration method that significantly shortens the optimization path and improves the robustness and accuracy of the registration. Future work can further optimize this method to achieve better 3D facial model reconstruction results.

Partition-based Nonrigid Registration for 3D Face Model

Face Alignment Based on 3D Face Shape Model and Markov Random Field.

An Automatic Multi-Sample 3d Face Registration Method Based on Thin Plate Spline and Deformable Model

Non-rigid 3D Shape Registration using an Adaptive Template

Conformal Surface Registration with Applications on Face Morphing

A Modeling Method for the Human Body Model with Facial Morphology

Landmark Weighting for 3DMM Shape Fitting

On Learning 3D Face Morphable Model from In-the-wild Images

Topology-Adaptive Non-Rigid Registration for 3D Facial Scans

A Deformation Model to Reduce the Effect of Expressions in 3D Face Recognition

3D3M: 3D Modulated Morphable Model for Monocular Face Reconstruction

An Efficient Volumetric Method for Non-Rigid Registration

A Non-Rigid Registration Algorithm for Compatible Skeletonization

Nonrigid-Deformation Recovery for 3D Face Recognition Using Multiscale Registration

MMFace: A Multi-Metric Regression Network for Unconstrained Face Reconstruction

Automatic and robust 3D face registration using multiresolution Spherical Depth Map

3D face registration by depth-based template matching and active appearance model

Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape

3D Craniofacial Registration Using Thin-Plate Spline Transform and Cylindrical Surface Projection.

Building a 3d Morphable Face Model by Using Thin Plate Splines for Face Reconstruction

An Automatic Method of Building 3D Morphable Face Model