Contrastive Learning for Image Complexity Representation

Shipeng Liu,Liang Zhao,Dengfeng Chen,Zhanping Song

2024-08-06

Abstract:Quantifying and evaluating image complexity can be instrumental in enhancing the performance of various computer vision tasks. Supervised learning can effectively learn image complexity features from well-annotated datasets. However, creating such datasets requires expensive manual annotation costs. The models may learn human subjective biases from it. In this work, we introduce the MoCo v2 framework. We utilize contrastive learning to represent image complexity, named CLIC (Contrastive Learning for Image Complexity). We find that there are complexity differences between different local regions of an image, and propose Random Crop and Mix (RCM), which can produce positive samples consisting of multi-scale local crops. RCM can also expand the train set and increase data diversity without introducing additional data. We conduct extensive experiments with CLIC, comparing it with both unsupervised and supervised methods. The results demonstrate that the performance of CLIC is comparable to that of state-of-the-art supervised methods. In addition, we establish the pipelines that can apply CLIC to computer vision tasks to effectively improve their performance.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem this paper attempts to address is how to accurately quantify and evaluate image complexity while avoiding the expensive cost of manual annotation and subjective bias. Specifically, although existing supervised learning methods can effectively learn image complexity features from well-annotated datasets, creating such datasets requires high manual annotation costs and may introduce human subjective bias. Therefore, the paper proposes a method based on a contrastive learning framework—CLIC (Contrastive Learning for Image Complexity), which utilizes unlabeled data to learn image complexity representations, thereby eliminating the need for expensive annotations and completely avoiding human subjective bias. Additionally, the paper proposes a technique called Random Crop and Mix (RCM) to generate positive samples containing both local and global features to enhance the effect of contrastive learning. Experimental results show that CLIC's performance is close to state-of-the-art supervised methods and significantly outperforms supervised methods in scenarios with a small number of samples. Furthermore, CLIC, as auxiliary prior information, can also improve the performance of fundamental computer vision tasks such as object detection and semantic segmentation.