Learning a Deep Color Difference Metric for Photographic Images

Haoyu Chen,Zhihua Wang,Yang Yang,Qilin Sun,Kede Ma

2023-03-27

Abstract:Most well-established and widely used color difference (CD) metrics are handcrafted and subject-calibrated against uniformly colored patches, which do not generalize well to photographic images characterized by natural scene complexities. Constructing CD formulae for photographic images is still an active research topic in imaging/illumination, vision science, and color science communities. In this paper, we aim to learn a deep CD metric for photographic images with four desirable properties. First, it well aligns with the observations in vision science that color and form are linked inextricably in visual cortical processing. Second, it is a proper metric in the mathematical sense. Third, it computes accurate CDs between photographic images, differing mainly in color appearances. Fourth, it is robust to mild geometric distortions (e.g., translation or due to parallax), which are often present in photographic images of the same scene captured by different digital cameras. We show that all these properties can be satisfied at once by learning a multi-scale autoregressive normalizing flow for feature transform, followed by the Euclidean distance which is linearly proportional to the human perceptual CD. Quantitative and qualitative experiments on the large-scale SPCD dataset demonstrate the promise of the learned CD metric.

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

The paper aims to address the problem of measuring color difference (CD) in photographic images. Traditionally, most widely used color difference measurement methods are manually designed and calibrated for uniform color patches, and these methods perform poorly when dealing with photographic images that have the complexity of natural scenes. To overcome this limitation, the researchers propose a deep learning framework—CD-Flow, for learning a color difference measurement method with the following four ideal characteristics:

1. **Consistent with the color perception mechanism in the visual cortex**: The design of this method should respect the inseparable view of color and shape processing in the visual cortex.
2. **Satisfies the metric properties in the mathematical sense**: That is, non-negativity, symmetry, identity of indiscernibles, and the triangle inequality.
3. **Accurately predicts human color perception differences in photographic images**: This method is not only suitable for photographic images but also generalizes well to uniform color patches.
4. **Robust to slight geometric distortions**: Such as translations or distortions caused by parallax, which commonly appear in images of the same scene taken by different digital cameras.

The study achieves feature transformation through multi-scale autoregressive normalization flow (a variant based on RealNVP) and uses Euclidean distance as the metric in the transformed space. Experimental results show that the proposed CD-Flow method outperforms 15 existing color difference measurement methods on the large-scale SPCD dataset and demonstrates stronger robustness in the presence of geometric distortions. Additionally, the method also shows good generalization ability on other datasets.