Abstract:State-of-the-art deep CNN face matchers are typically created using extensive training sets of color face images. Our study reveals that such matchers attain virtually identical accuracy when trained on either grayscale or color versions of the training set, even when the evaluation is done using color test images. Furthermore, we demonstrate that shallower models, lacking the capacity to model complex representations, rely more heavily on low-level features such as those associated with color. As a result, they display diminished accuracy when trained with grayscale images. We then consider possible causes for deeper CNN face matchers "not seeing color". Popular web-scraped face datasets actually have 30 to 60% of their identities with one or more grayscale images. We analyze whether this grayscale element in the training set impacts the accuracy achieved, and conclude that it does not. We demonstrate that using only grayscale images for both training and testing achieves accuracy comparable to that achieved using only color images for deeper models. This holds true for both real and synthetic training datasets. HSV color space, which separates chroma and luma information, does not improve the network's learning about color any more than in the RGB color space. We then show that the skin region of an individual's images in a web-scraped training set exhibits significant variation in their mapping to color space. This suggests that color carries limited identity-specific information. We also show that when the first convolution layer is restricted to a single filter, models learn a grayscale conversion filter and pass a grayscale version of the input color image to the next layer. Finally, we demonstrate that leveraging the lower per-image storage for grayscale to increase the number of images in the training set can improve accuracy of the face recognition model.

Improving the Performance of Color Spaces for Face Recognition by Color Space Normalization and Rotation

Face Recognition based on scale invariant feature transform and Spatial Pyramid Representation

Evaluation of Color Spaces for Person Re-Identification

Color Space Learning for Cross-Color Person Re-Identification

Robust 3D Face Recognition by Local Shape Difference Boosting

Learning the Color Space for Effective Fabric Defect Detection

Illumination-normalized face recognition using Chromaticity Intrinsic Image

SCSNet: an Efficient Paradigm for Learning Simultaneously Image Colorization and Super-resolution

Using Eigencolor Normalization for Illumination-Invariant Color Object Recognition

ColorNet: Investigating the importance of color spaces for image classification

Fast Neighbourhood Component Analysis with Spatially Smooth Regulariser for Robust Noisy Face Recognition

An Optimized SIFT Algorithm Based on Color Space Normalization

Learning Spatially Decoupled Color Representations for Facial Image Colorization

Impact of Color Space and Color Resolution on Vehicle Recognition Models

What's color got to do with it? Face recognition in grayscale

Robust Face Recognition Via Gradient-Based Sparse Representation.

Gray-Scale Super-Resolution For Face Recognition From Low Gray-Scale Resolution Face Images

Influence of Color Spaces for Deep Learning Image Colorization

Face recognition using fast neighborhood component analysis with spatially smooth regularizer

A Face Detection and Recognition System in Color Image Series

MultiColor: Image Colorization by Learning from Multiple Color Spaces