Abstract:Color vision is essential for human visual perception, but its impact on machine perception is still underexplored. There has been an intensified demand for understanding its role in machine perception for safety-critical tasks such as assistive driving and surgery but lacking suitable datasets. To fill this gap, we curate multipurpose datasets ColorSense, by collecting 110,000 non-trivial human annotations of foreground and background color labels from popular visual recognition benchmarks. To investigate the impact of color vision on machine perception, we assign each image a color discrimination level based on its dominant foreground and background colors and use it to study the impact of color vision on machine perception. We validate the use of our datasets by demonstrating that the level of color discrimination has a dominating effect on the performance of mainstream machine perception models. Specifically, we examine the perception ability of machine vision by considering key factors such as model architecture, training objective, model size, training data, and task complexity. Furthermore, to investigate how color and environmental factors affect the robustness of visual recognition in machine perception, we integrate our ColorSense datasets with image corruptions and perform a more comprehensive visual perception evaluation. Our findings suggest that object recognition tasks such as classification and localization are susceptible to color vision bias, especially for high-stakes cases such as vehicle classes, and advanced mitigation techniques such as data augmentation and so on only give marginal improvement. Our analyses highlight the need for new approaches toward the performance evaluation of machine perception models in real-world applications. Lastly, we present various potential applications of ColorSense such as studying spurious correlations.

A New Color Constancy Model for Machine Vision

Computer Color Constancy in Two Steps

The Influence Factors and Nonlinear Modeling of Color Constancy

A Retinal Mechanism Inspired Color Constancy Model

A Color Constancy Model with Double-Opponency Mechanisms.

A Virtual Color Space Based on Physical Model

A Color-Opponency Based Biological Model for Color Constancy

Color constancy models inspired by human visual system: Survey paper

A Supervised Combination Strategy for Illumination Chromaticity Estimation

Color in machine vision and its application

Data Based Color Constancy.

Investigating Color Illusions from the Perspective of Computational Color Constancy

Evaluating Combinational Color Constancy Methods on Real-World Images

Color Constancy Model Based on Topology Resolve-Map in Medical Image Processing

Research on Color Constancy under Open Illumination Conditions

Edge-moment-based color constancy using illumination-coherent regularized regression.

Color Constancy Based on Image Derivative Framework and Non-negative Sparse Coding

Improved Color Opponent Contour Detection Model Based on Dark and Light Adaptation.

Color Image Edge Detection and Classification Based on Physical Model

ColorSense: A Study on Color Vision in Machine Visual Recognition

Combining Bottom-Up and Top-Down Visual Mechanisms for Color Constancy under Varying Illumination