Abstract:Facial beauty analysis is an important topic in human society. It may be used as a guidance for face beautification applications such as cosmetic surgery. Deep neural networks (DNNs) have recently been adopted for facial beauty analysis and have achieved remarkable performance. However, most existing DNN‐based models regard facial beauty analysis as a normal classification task. They ignore important prior knowledge in traditional machine learning models which illustrate the significant contribution of the geometric features in facial beauty analysis. To be specific, landmarks of the whole face and facial organs are introduced to extract geometric features to make the decision. Inspired by this, we introduce a novel dual‐branch network for facial beauty analysis: one branch takes the Swin Transformer as the backbone to model the full face and global patterns, and another branch focuses on the masked facial organs with the residual network to model the local patterns of certain facial parts. Additionally, the designed multi‐scale feature fusion module can further facilitate our network to learn complementary semantic information between the two branches. In model optimisation, we propose a hybrid loss function, where especially geometric regulation is introduced by regressing the facial landmarks and it can force the extracted features to convey facial geometric features. Experiments performed on the SCUT‐FBP5500 dataset and the SCUT‐FBP dataset demonstrate that our model outperforms the state‐of‐the‐art convolutional neural networks models, which proves the effectiveness of the proposed geometric regularisation and dual‐branch structure with the hybrid network. To the best of our knowledge, this is the first study to introduce a Vision Transformer into the facial beauty analysis task.

A Jointly Learned Deep Architecture for Facial Attribute Analysis and Face Detection in the Wild

Deep Learning Face Attributes in the Wild

Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment

A Survey of Deep Facial Attribute Analysis

Joint Multi-View Face Alignment in the Wild

Landmarks-assisted Collaborative Deep Framework for Automatic 4D Facial Expression Recognition.

Learning Semantic Representations via Joint 3D Face Reconstruction and Facial Attribute Estimation

Bio-Inspired Deep Attribute Learning Towards Facial Aesthetic Prediction

Multi-Task Deep Neural Network for Joint Face Recognition and Facial Attribute Prediction

Boosting Facial Action Unit Detection Through Jointly Learning Facial Landmark Detection and Domain Separation and Reconstruction

A Real-Time Multi-Task Learning System for Joint Detection of Face, Facial Landmark and Head Pose

Learning Deep Sharable and Structural Detectors for Face Alignment

Learning Deep Representation for Face Alignment with Auxiliary Attributes

A Unified Deep Model for Joint Facial Expression Recognition, Face Synthesis, and Face Alignment.

Collaborative Learning Network For Face Attribute Prediction

ACWFace: efficient and lightweight face detector based on RetinaFace

Enhancing Facial Classification and Recognition using 3D Facial Models and Deep Learning

Joint 3d Face Reconstruction And Dense Face Alignment Via Deep Face Feature Alignment

Deep Structured Prediction for Facial Landmark Detection

A Good Practice Towards Top Performance of Face Recognition: Transferred Deep Feature Fusion

Geometric Prior Guided Hybrid Deep Neural Network for Facial Beauty Analysis