Look Through Masks: Towards Masked Face Recognition with De-Occlusion Distillation

Chenyu Li,Shiming Ge,Daichi Zhang,Jia Li

2024-09-19

Abstract:Many real-world applications today like video surveillance and urban governance need to address the recognition of masked faces, where content replacement by diverse masks often brings in incomplete appearance and ambiguous representation, leading to a sharp drop in accuracy. Inspired by recent progress on amodal perception, we propose to migrate the mechanism of amodal completion for the task of masked face recognition with an end-to-end de-occlusion distillation framework, which consists of two modules. The \textit{de-occlusion} module applies a generative adversarial network to perform face completion, which recovers the content under the mask and eliminates appearance ambiguity. The \textit{distillation} module takes a pre-trained general face recognition model as the teacher and transfers its knowledge to train a student for completed faces using massive online synthesized face pairs. Especially, the teacher knowledge is represented with structural relations among instances in multiple orders, which serves as a posterior regularization to enable the adaptation. In this way, the knowledge can be fully distilled and transferred to identify masked faces. Experiments on synthetic and realistic datasets show the efficacy of the proposed approach.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The paper attempts to address the problem of effectively recognizing masked faces in real-world applications, such as video surveillance and urban management. Due to masks covering part of the facial features, traditional face recognition techniques experience a significant drop in accuracy. To solve this problem, the authors propose a new end-to-end framework that improves the recognition performance of masked faces through De-Occlusion Distillation.

Specifically, the framework includes two main modules:

1. **De-Occlusion Module**:
- Uses Generative Adversarial Network (GAN) for face completion, restoring the parts covered by the mask and eliminating appearance ambiguity.
- Introduces an attention mechanism to enable the model to focus on information-rich areas.

2. **Distillation Module**:
- Employs a pre-trained general face recognition model as a teacher model, transferring the teacher model's knowledge to the student model through knowledge distillation to recognize the completed face.
- The knowledge of the teacher model is represented as the structural relationships between instances, which serve as posterior regularization to help the student model adapt to the task of masked face recognition.

By combining these two modules, the method not only restores the occluded facial content but also leverages the knowledge of the pre-trained model to improve recognition accuracy. Experimental results show that this method achieves significant improvements on both synthetic and real datasets.