Abstract:Multi-modal image clustering focuses on exploring and exploiting related information across various modals of input images to obtain clear image cluster patterns. Recent multi-modal/view clustering methods have shown promising performance in solving the image clustering problem. However, most existing methods fail to properly deal with the multi-modal image data with massive number of clusters and high dimensionality of features in real-world applications, such as image retrieval, multi-modal autonomous driving perception and industrial automation. We call this problem "Big Clusters and Features" for short just as "Big Data" for large number of samples. To fix this challenging problem, we in this article design a general multi-modal image clustering framework which integrates cluster-wise weight learning, feature learning, and clustering structure learning. Under this framework, we further propose a new Cluster-weighted Multi-modal Information Bottleneck Co-clustering (CMIBC) method, and it could effectively measure image cluster importance information and discriminative features of each modal for obtaining satisfactory image clustering performance. Unlike existing cluster weight learning methods only considering intra-cluster similarity or cross-cluster dissimilarity, we design a novel cluster-wise weight learning strategy by jointly considering and enjoying the best of both worlds. Many carefully designed experiments on various multi-modal image datasets with big clusters and features reveal the competitive advantages of the CMIBC algorithm over lots of compared single/multi-modal clustering methods, particularly the notable improvement of 3.12% and 5.28% on the Leaves Plant Species dataset in terms of accuracy and normalized mutual information. Owing to the promising performance, the proposed CMIBC may be extended to many other practical applications, e.g., multi-modal medical analysis and video recognition.

Cross-Modal Image Clustering Via Canonical Correlation Analysis

Regularized Canonical Correlation Analysis with Unlabeled Data

Towards Improving Canonical Correlation Analysis for Cross-modal Retrieval.

Cross-media retrieval by cluster-based correlation analysis

Cross-modal correlation learning for clustering on image-audio dataset.

Nice to meet images with Big Clusters and Features: A cluster-weighted multi-modal co-clustering method

Cross-Modal Clustering With Deep Correlated Information Bottleneck Method

Multi-level Cross-modal Alignment for Image Clustering

Deep Comprehensive Correlation Mining for Image Clustering

Nonnegative Constrained Graph Based Canonical Correlation Analysis for Multi-view Feature Learning

A Local Method for Canonical Correlation Analysis.

Clustering-driven Deep Adversarial Hashing for scalable unsupervised cross-modal retrieval

CCL: Cross-modal Correlation Learning with Multi-grained Fusion by Hierarchical Network.

AnANet: Modeling Association and Alignment for Cross-modal Correlation Classification

Cross-modal image sentiment analysis via deep correlation of textual semantic

Cross-Modal Image-Tag Relevance Learning for Social Images

Cross-Media Retrieval Based on Canonical Correlation Analysisand Decision Tree

An output aggregation system for large scale cross-modal retrieval

Label Guided Correlation Hashing for Large-Scale Cross-Modal Retrieval

Multi-Level Correlation Adversarial Hashing for Cross-Modal Retrieval.

Exploring Shared Subspace and Joint Sparsity for Canonical Correlation Analysis.