Abstract:Face recognition algorithms based on deep convolutional neural networks (DCNNs) have made progress on the task of recognizing faces in unconstrained viewing conditions. These networks operate with compact feature-based face representations derived from learning a very large number of face images. While the learned features produced by DCNNs can be highly robust to changes in viewpoint, illumination, and appearance, little is known about the nature of the face code that emerges at the top level of such networks. We analyzed the DCNN features produced by two face recognition algorithms. In the first set of experiments we used the top-level features from the DCNNs as input into linear classifiers aimed at predicting metadata about the images. The results show that the DCNN features contain surprisingly accurate information about the yaw and pitch of a face, and about whether the face came from a still image or a video frame. In the second set of experiments, we measured the extent to which individual DCNN features operated in a view-dependent or view-invariant manner. We found that view-dependent coding was a characteristic of the identities rather than the DCNN features - with some identities coded consistently in a view-dependent way and others in a view-independent way. In our third analysis, we visualized the DCNN feature space for over 24,000 images of 500 identities. Images in the center of the space were uniformly of low quality (e.g., extreme views, face occlusion, low resolution). Image quality increased monotonically as a function of distance from the origin. This result suggests that image quality information is available in the DCNN features, such that consistently average feature values reflect coding failures that reliably indicate poor or unusable images. Combined, the results offer insight into the coding mechanisms that support robust representation of faces in DCNNs.

Exploring Features and Attributes in Deep Face Recognition Using Visualization Techniques

Deep Clustering and Visualization for End-to-End High-Dimensional Data Analysis.

Deeper Interpretability of Deep Networks

Explaining Deep Face Algorithms through Visualization: A Survey

DeepVisage: Making face recognition simple yet with powerful generalization skills

Studies advanced in face recognition technology based on deep learning

Towards Better Analysis of Deep Convolutional Neural Networks

Discriminative Deep Feature Visualization for Explainable Face Recognition

Deep Face Decoder: Towards understanding the embedding space of convolutional networks through visual reconstruction of deep face templates

Deep Convolutional Neural Network Features and the Original Image

Face representation by deep learning: a linear encoding in a parameter space?

Deblurring Method of Face Recognition AI Technology Based on Deep Learning

Classification Mechanism of Convolutional Neural Network for Facial Expression Recognition.

Deep Learning Face Attributes in the Wild

Facial Expression Recognition Research Based on Deep Learning

Understanding Beauty via Deep Facial Features

A Survey of Deep Facial Attribute Analysis

Multi-channel Deep 3D Face Recognition

Decoding Face Identity: A Reverse-Correlation Approach Using Deep Learning

Concurrent emergence of view invariance, sensitivity to critical features, and identity face classification through visual experience: Insights from deep learning algorithms

Recognizing Semantic Features in Faces using Deep Learning