Abstract:Deep neural networks (DNNs) are increasingly proposed as models of human vision, bolstered by their impressive performance on image classification and object recognition tasks. Yet, the extent to which DNNs capture fundamental aspects of human vision such as color perception remains unclear. Here, we develop novel experiments for evaluating the perceptual coherence of color embeddings in DNNs, and we assess how well these algorithms predict human color similarity judgments collected via an online survey. We find that state-of-the-art DNN architectures - including convolutional neural networks and vision transformers - provide color similarity judgments that strikingly diverge from human color judgments of (i) images with controlled color properties, (ii) images generated from online searches, and (iii) real-world images from the canonical CIFAR-10 dataset. We compare DNN performance against an interpretable and cognitively plausible model of color perception based on wavelet decomposition, inspired by foundational theories in computational neuroscience. While one deep learning model - a convolutional DNN trained on a style transfer task - captures some aspects of human color perception, our wavelet algorithm provides more coherent color embeddings that better predict human color judgments compared to all DNNs we examine. These results hold when altering the high-level visual task used to train similar DNN architectures (e.g., image classification versus image segmentation), as well as when examining the color embeddings of different layers in a given DNN architecture. These findings break new ground in the effort to analyze the perceptual representations of machine learning algorithms and to improve their ability to serve as cognitively plausible models of human vision. Implications for machine learning, human perception, and embodied cognition are discussed.

Exploring the Categorical Nature of Colour Perception: Insights from Artificial Networks

Exploring the categorical nature of colour perception: Insights from artificial networks

Emergent color categorization in a neural network trained for object recognition

Colour Terms: a Categorisation Model Inspired by Visual Cortex Neurons

Coordinating perceptually grounded categories through language: a case study for colour

Spatial and Colour Opponency in Anatomically Constrained Deep Networks

Divergences in color perception between deep neural networks and humans

A neural field model for color perception unifying assimilation and contrast

Capturing human categorization of natural images at scale by combining deep networks and cognitive models

Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation

Modeling the emergence of universality in color naming patterns

Categorisation of Colour Terms Using New Validation Tools: A Case Study and Implications

The Nature of Language and Cognition Interactions in Color Perception

ColorNet: Investigating the importance of color spaces for image classification

Spatial relation categorization in infants and deep neural networks

Contrastive learning explains the emergence and function of visual category-selective regions

Causal Influence of Linguistic Learning on Perceptual and Conceptual Processing: A Brain-Constrained Deep Neural Network Study of Proper Names and Category Terms

Origin of the ease of association of color names: Comparison between humans and AI

Rapid Color Categorization in the Brain Revealed by Frequency-tagging-based EEG

An Algorithm for Categorising Colours into Universal Colour Names.