Abstract:Face recognition algorithms based on deep convolutional neural networks (DCNNs) have made progress on the task of recognizing faces in unconstrained viewing conditions. These networks operate with compact feature-based face representations derived from learning a very large number of face images. While the learned features produced by DCNNs can be highly robust to changes in viewpoint, illumination, and appearance, little is known about the nature of the face code that emerges at the top level of such networks. We analyzed the DCNN features produced by two face recognition algorithms. In the first set of experiments we used the top-level features from the DCNNs as input into linear classifiers aimed at predicting metadata about the images. The results show that the DCNN features contain surprisingly accurate information about the yaw and pitch of a face, and about whether the face came from a still image or a video frame. In the second set of experiments, we measured the extent to which individual DCNN features operated in a view-dependent or view-invariant manner. We found that view-dependent coding was a characteristic of the identities rather than the DCNN features - with some identities coded consistently in a view-dependent way and others in a view-independent way. In our third analysis, we visualized the DCNN feature space for over 24,000 images of 500 identities. Images in the center of the space were uniformly of low quality (e.g., extreme views, face occlusion, low resolution). Image quality increased monotonically as a function of distance from the origin. This result suggests that image quality information is available in the DCNN features, such that consistently average feature values reflect coding failures that reliably indicate poor or unusable images. Combined, the results offer insight into the coding mechanisms that support robust representation of faces in DCNNs.

Convolutional Neural Networks Features: Principal Pyramidal Convolution.

Aggregating Hierarchical Binary Activations for Image Retrieval

Research on Image Classification Method of Features of Combinatorial Convolution

Advances in Convolutional Neural Networks

Learning Feature Embedding with Strong Neural Activations for Fine-Grained Retrieval

A FEATURE EMBEDDING STRATEGY FOR HIGH-LEVEL CNN REPRESENTATIONS FROM MULTIPLE CONVNETS

A PCA-Based Convolutional Network

Convolutional Neural Networks Exploiting Attributes of Biological Neurons

Deep Convolutional Neural Network Features and the Original Image

Convolutional Channel Features

Convolutional Neural Pyramid for Image Processing

Convolutional Neural Network Based on Spatial Pyramid for Image Classification

Feature Extraction and Image Recognition with Convolutional Neural Networks

Good Practice in CNN Feature Transfer

Elliptical Convolution Kernel: More Real Visual Field

Transform-Invariant Convolutional Neural Networks for Image Classification and Search

PointCNN: Convolution on X-Transformed Points.

Cross-convolutional-layer Pooling for Generic Visual Recognition.

High-Resolution Remote Sensing Image Retrieval Based on CNNs from a Dimensional Perspective.

Dual Complementary Dynamic Convolution for Image Recognition

The Treasure Beneath Convolutional Layers: Cross-Convolutional-Iayer Pooling For Image Classification