Towards A Comprehensive Visual Saliency Explanation Framework for AI-based Face Recognition Systems

Yuhang Lu,Zewei Xu,Touradj Ebrahimi

2024-07-08

Abstract:Over recent years, deep convolutional neural networks have significantly advanced the field of face recognition techniques for both verification and identification purposes. Despite the impressive accuracy, these neural networks are often criticized for lacking explainability. There is a growing demand for understanding the decision-making process of AI-based face recognition systems. Some studies have investigated the use of visual saliency maps as explanations, but they have predominantly focused on the specific face verification case. The discussion on more general face recognition scenarios and the corresponding evaluation methodology for these explanations have long been absent in current research. Therefore, this manuscript conceives a comprehensive explanation framework for face recognition tasks. Firstly, an exhaustive definition of visual saliency map-based explanations for AI-based face recognition systems is provided, taking into account the two most common recognition situations individually, i.e., face verification and identification. Secondly, a new model-agnostic explanation method named CorrRISE is proposed to produce saliency maps, which reveal both the similar and dissimilar regions between any given face images. Subsequently, the explanation framework conceives a new evaluation methodology that offers quantitative measurement and comparison of the performance of general visual saliency explanation methods in face recognition. Consequently, extensive experiments are carried out on multiple verification and identification scenarios. The results showcase that CorrRISE generates insightful saliency maps and demonstrates superior performance, particularly in similarity maps in comparison with the state-of-the-art explanation approaches.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the lack of interpretability in current artificial - intelligence - based face recognition systems. Although deep convolutional neural networks have made remarkable progress in face recognition technology, these models are usually regarded as "black boxes" and their decision - making processes are difficult to understand. This not only affects the credibility of the system but also raises public concerns about privacy and data protection. Therefore, the paper proposes a comprehensive visual saliency explanation framework, aiming to improve the interpretability of AI face recognition systems.

Specifically, the main contributions of the paper include:
1. **Define visual saliency map explanation**: The paper provides a detailed definition of visual saliency map explanation for learning - based face recognition systems, especially considering the two most common recognition situations, namely face verification and identification.
2. **Propose a new explanation method**: The paper proposes a new model - independent explanation method - CorrRISE (Correlation - based Randomized Input Sampling for Explanation), which is used to generate saliency maps and reveal the similar and dissimilar regions between any given face images.
3. **New evaluation method**: The paper proposes a new objective evaluation method for quantitatively measuring and comparing the performance of different state - of - the - art saliency - map - based face recognition explanation methods.
4. **Experimental verification**: The paper conducts extensive experiments in multiple face verification and identification scenarios and makes a detailed quantitative comparison with the current state - of - the - art explanation methods, demonstrating the effectiveness of the proposed method.

Through these contributions, the paper aims to fill the gaps in existing research in explaining face recognition models, especially providing a more comprehensive and general explanation framework in face recognition tasks.