Abstract:Aligning multiple heterogeneous modalities in a parameter-sharing encoder to mine consistent information is a core idea of multimodal learning. However, two drawbacks hinder the development of such methods for clustering tasks: 1) each modality contains a considerable amount of superfluous information that cannot be aligned, impeding the mining of consistent information and 2) one-to-one alignment is contradictory to the clustering principle of minimum intra-cluster distance, leading to suboptimal clustering results. In this paper, we propose a novel Consistency-Guided Multimodal Clustering method (ConGMC) to remove superfluous information within the modalities unsupervised through information theory while improving one-to-one alignment for the clustering task. ConGMC contains multiple unimodal encoders and a multimodal shared encoder, where the former learns unimodal representation while the latter aligns multiple modalities to learn the cluster partition. Specifically, we first construct a mutual information maximin function to distinguish consistent information from superfluous information, in which the consistent and superfluous information are maximally retained and removed, respectively. Then a Clustering-Friendly Alignment strategy (CF-Align) is designed to address the contradiction between the alignment and clustering tasks. CF-Align dynamically adjusts the set of negative samples according to the learned cluster partition to avoid increasing the intra-cluster distance. Finally, we consider the cluster partition as a consistent constraint to optimize the multimodal shared encoder, enabling consistent information to guide the training process iteratively. Moreover, a variational optimization algorithm is proposed to ensure that ConGMC converges to a local optimum. Numerous experimental results on twelve real-world datasets validate that the proposed ConGMC method outperforms the state-of-the-art multimodal clustering methods.

Multimodal data consistency anomaly detection based on sparse subspace clustering

Multi-View Clustering Via Simultaneously Learning Shared Subspace And Affinity Matrix

Distributed Online One-Class Support Vector Machine for Anomaly Detection over Networks

A Clustering Algorithm for Multi-Modal Heterogeneous Big Data With Abnormal Data

Consistency and Complementarity Jointly Regularized Subspace Support Vector Data Description for Multimodal Data

Memoryless Multimodal Anomaly Detection via Student–Teacher Network and Signed Distance Learning

Anomaly detection algorithm based on FrFT transform and total variation regularization

Clustering-based anomaly detection in multivariate time series data

Spatial-temporal trajectory anomaly detection based on an improved spectral clustering algorithm

Spatiotemporal consistency-enhanced network for video anomaly detection

Consistency-Induced Multiview Subspace Clustering.

An anomaly detection algorithm for hyperspectral images using subspace sparse representation

A Novel Backbone Network Anomaly Detector Via Clustering In Sketch Space

ConGMC: Consistency-Guided Multimodal Clustering via Mutual Information Maximin

Towards a Unified Framework of Clustering-based Anomaly Detection

A Bottleneck Network with Light Attention for Multimodal Clustering

A novel anomaly detection method for multimodal WSN data flow via a dynamic graph neural network

Co-regularized Multiview Subspace Clustering

Unsupervised Multimodal Clustering for Semantics Discovery in Multimodal Utterances

Consistent multi-view subspace clustering with local structure information

SPACE: SPAtial-aware Consistency rEgularization for anomaly detection in Industrial applications