Abstract:Aligning multiple heterogeneous modalities in a parameter-sharing encoder to mine consistent information is a core idea of multimodal learning. However, two drawbacks hinder the development of such methods for clustering tasks: 1) each modality contains a considerable amount of superfluous information that cannot be aligned, impeding the mining of consistent information and 2) one-to-one alignment is contradictory to the clustering principle of minimum intra-cluster distance, leading to suboptimal clustering results. In this paper, we propose a novel Consistency-Guided Multimodal Clustering method (ConGMC) to remove superfluous information within the modalities unsupervised through information theory while improving one-to-one alignment for the clustering task. ConGMC contains multiple unimodal encoders and a multimodal shared encoder, where the former learns unimodal representation while the latter aligns multiple modalities to learn the cluster partition. Specifically, we first construct a mutual information maximin function to distinguish consistent information from superfluous information, in which the consistent and superfluous information are maximally retained and removed, respectively. Then a Clustering-Friendly Alignment strategy (CF-Align) is designed to address the contradiction between the alignment and clustering tasks. CF-Align dynamically adjusts the set of negative samples according to the learned cluster partition to avoid increasing the intra-cluster distance. Finally, we consider the cluster partition as a consistent constraint to optimize the multimodal shared encoder, enabling consistent information to guide the training process iteratively. Moreover, a variational optimization algorithm is proposed to ensure that ConGMC converges to a local optimum. Numerous experimental results on twelve real-world datasets validate that the proposed ConGMC method outperforms the state-of-the-art multimodal clustering methods.

Cluster Number Estimation by Adaptively Identifying Ambiguously Clustered Pairs

Combining multiple clusterings via k-modes algorithm

Regime Switching Model Estimation: Spectral Clustering Hidden Markov Model

Clustering populations by mixed linear models

Model based clustering of multinomial count data

Enhanced Bayesian Gaussian hidden Markov mixture clustering for improved knowledge discovery

Consensus Cluster Center Guided Latent Multi-kernel Clustering

Probabilistic Cluster Structure Ensemble

How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis

DBHC: Discrete Bayesian HMM Clustering

Clusterer ensemble

ConGMC: Consistency-Guided Multimodal Clustering via Mutual Information Maximin

CNAK : Cluster Number Assisted K-means

A Probabilistic Approach to Latent Cluster Analysis

Clustering hidden Markov models with variational HEM

Hidden Markov Models on Variable Blocks with a Modal Clustering Algorithm and Applications

Multimodal data consistency anomaly detection based on sparse subspace clustering

Clustering sequence data with mixture Markov chains with covariates using multiple simplex constrained optimization routine (MSiCOR)

A New Unsupervised Binning Approachfor Metagenomic Sequences Based onN-grams and Automatic Feature Weighting.

CLAMS: A Cluster Ambiguity Measure for Estimating Perceptual Variability in Visual Clustering

A Graph-based Approach to Estimating the Number of Clusters