Abstract:Advances in deep learning and convolutional neural networks (ConvNets) have driven remarkable face recognition (FR) progress recently. However, the black-box nature of modern ConvNet-based face recognition models makes it challenging to interpret their decision-making process, to understand the reasoning behind specific success and failure cases, or to predict their responses to unseen data characteristics. It is, therefore, critical to design mechanisms that explain the inner workings of contemporary FR models and offer insight into their behavior. To address this challenge, we present in this paper a novel template-inversion approach capable of reconstructing high-fidelity face images from the embeddings (templates, feature-space representations) produced by modern FR techniques. Our approach is based on a novel Deep Face Decoder (DFD) trained in a regression setting to visualize the information encoded in the embedding space with the goal of fostering explainability. We utilize the developed DFD model in comprehensive experiments on multiple unconstrained face datasets, namely Visual Geometry Group Face dataset 2 (VGGFace2), Labeled Faces in the Wild (LFW), and Celebrity Faces Attributes Dataset High Quality (CelebA-HQ). Our analysis focuses on the embedding spaces of two distinct face recognition models with backbones based on the Visual Geometry Group 16-layer model (VGG-16) and the 50-layer Residual Network (ResNet-50). The results reveal how information is encoded in the two considered models and how perturbations in image appearance due to rotations, translations, scaling, occlusion, or adversarial attacks, are propagated into the embedding space. Our study offers researchers a deeper comprehension of the underlying mechanisms of ConvNet-based FR models, ultimately promoting advancements in model design and explainability.

Understanding Deep Face Representation Via Attribute Recovery

Designing One Unified Framework for High-Fidelity Face Reenactment and Swapping

Exploring Features and Attributes in Deep Face Recognition Using Visualization Techniques

Deep Learning Face Attributes in the Wild

Deep Identity-aware Transfer of Facial Attributes

A Survey of Deep Facial Attribute Analysis

Deep Attribute Guided Representation for Heterogeneous Face Recognition

Generating Face Images with Attributes for Free.

Adv-Attribute: Inconspicuous and Transferable Adversarial Attack on Face Recognition

Deep Face Decoder: Towards understanding the embedding space of convolutional networks through visual reconstruction of deep face templates

Identifying individual facial expressions by deconstructing a neural network

Learning Semantic Representations via Joint 3D Face Reconstruction and Facial Attribute Estimation

Robust RGB-D Face Recognition Using Attribute-Aware Loss

Identity-Aware Face Super-Resolution for Low-Resolution Face Recognition

Detailed 3D Face Reconstruction from Single Images Via Self-supervised Attribute Learning

Decoding Face Identity: A Reverse-Correlation Approach Using Deep Learning

Towards Interpretable Face Recognition

Sparse representation via deep learning features based face recognition method

A Comprehensive and Reliable Feature Attribution Method: Double-sided Remove and Reconstruct (DoRaR)

Face Super-Resolution via Discriminative-Attributes.

Facial Attribute Classification by Deep Mining Inter‐attribute Correlations