Abstract:Image clustering is an important and open-challenging task in computer vision. Although many methods have been proposed to solve the image clustering task, they only explore images and uncover clusters according to the image features, thus being unable to distinguish visually similar but semantically different images. In this paper, we propose to investigate the task of image clustering with the help of a visual-language pre-training model. Different from the zero-shot setting, in which the class names are known, we only know the number of clusters in this setting. Therefore, how to map images to a proper semantic space and how to cluster images from both image and semantic spaces are two key problems. To solve the above problems, we propose a novel image clustering method guided by the visual-language pre-training model CLIP, named \textbf{Semantic-Enhanced Image Clustering (SIC)}. In this new method, we propose a method to map the given images to a proper semantic space first and efficient methods to generate pseudo-labels according to the relationships between images and semantics. Finally, we propose performing clustering with consistency learning in both image space and semantic space, in a self-supervised learning fashion. The theoretical result of convergence analysis shows that our proposed method can converge at a sublinear speed. Theoretical analysis of expectation risk also shows that we can reduce the expected risk by improving neighborhood consistency, increasing prediction confidence, or reducing neighborhood imbalance. Experimental results on five benchmark datasets clearly show the superiority of our new method.

An Image Semantic Retrieval Algorithm Based on Learning Clustering

Semantic Image Retrieval Based on Multiple-Instance Learning

Image Emotional Semantic Retrieval Based on ELM

Learning A Semantic Space from User'S Relevance Feedback for Image Retrieval

Research on high-level semantic image retrieval

An Image Retrieval System with Color Emotion Query

Image Annotations Based on Semi-supervised Clustering with Semantic Soft Constraints.

Clustering-based Semantic Retrieval Algorithm

Image Retrieval Based on Fuzzy Semantic Relevance Matrix

Computer Image Content Retrieval Considering K-Means Clustering Algorithm

Clustering Guided SVM for Semantic Image Retrieval

Modeling Image Data for Effective Indexing and Retrieval in Large General Image Databases.

An Image Clustering Algorithm in Collaborative Tagging System

A New Approach for Automatic Image Annotation Based on Semantic Clustering and Graph Algorithm

ImageSaker : A Semantic-based Image Retrieval System Refining with Concept Model

Semantic-Enhanced Image Clustering

ICICLE: A Semantic-Based Retrieval System for WWW Images

Real-Time Image Semantic Retrieval Based on VQ

Supervised Clustering-Algorithm-based Visual Information Features Classification

Semantic Classification and Annotation of Images

A memory learning framework for effective image retrieval.