Abstract:Deep clustering is a crucial task in machine learning and data mining that focuses on acquiring feature representations conducive to clustering. Previous research relies on self-supervised representation learning for general feature representations, such features may not be optimally suited for downstream clustering tasks. In this paper, we introduce MICCF, a framework designed to bridge this gap and enhance clustering performance. MICCF enhances feature representations by combining mutual information at different levels and employs an auxiliary alignment mutual information module for learning clustering-oriented features. To be specific, we propose a dual mutual information constraints module, incorporating minimal mutual information constraints at the feature level and maximal mutual information constraints at the instance level. This reduction in feature redundancy encourages the neural network to extract more discriminative features, while maximization ensures more unbiased and robust representations. To obtain clustering-oriented representations, the auxiliary alignment mutual information module utilizes pseudo-labels to maximize mutual information through a multi-classifier network, aligning features with the clustering task. The main network and the auxiliary one work in synergy to jointly optimize feature representations that are well-suited for the clustering task. We validate the effectiveness of our method through extensive experiments on six benchmark datasets. The results indicate that our method performs well in most scenarios, particularly on fine-grained datasets, where our approach effectively distinguishes subtle differences between closely related categories. Notably, our approach achieved a remarkable accuracy of 96.4% on the ImageNet-10 dataset, surpassing other comparison methods. The code is available at https://github.com/Li-Hyn/MICCF.git .

An Adversarial Training-Based Mutual Information Constraint Method

Exploring adversarial examples and adversarial robustness of convolutional neural networks by mutual information

Towards a rigorous analysis of mutual information in contrastive learning

Mutual Information Multinomial Estimation

Improving Adversarial Robustness via Mutual Information Estimation

MICCF: A Mutual Information Constrained Clustering Framework for Learning Clustering-Oriented Feature Representations

Cross-domain Mutual Information Adversarial Maximization

Learning Unbiased Rewards with Mutual Information in Adversarial Imitation Learning

DeepMI: A Mutual Information Based Framework For Unsupervised Deep Learning of Tasks

Dual Mutual Information Constraints for Discriminative Clustering.

A robust estimator of mutual information for deep learning interpretability

Improving Mutual Information Estimation with Annealed and Energy-Based Bounds

Mutual Information calculation on different appearances

Enhancing 3D Robotic Vision Robustness by Minimizing Adversarial Mutual Information through a Curriculum Training Approach

L_DMI: A Novel Information-theoretic Loss Function for Training Deep Nets Robust to Label Noise

Detecting Adversarial Examples on Deep Neural Networks with Mutual Information Neural Estimation

On Variational Bounds of Mutual Information

Mutual Information Maximization for Effective Lip Reading

Enhancing Adversarial Transferability via Information Bottleneck Constraints

Adversarial Mutual Information for Text Generation

Addressing Implicit Bias in Adversarial Imitation Learning with Mutual Information.