A Novel Approach for Effective Multi-View Clustering with Information-Theoretic Perspective

Chenhang Cui,Yazhou Ren,Jingyu Pu,Jiawei Li,Xiaorong Pu,Tianyi Wu,Yutao Shi,Lifang He

2023-09-25

Abstract:Multi-view clustering (MVC) is a popular technique for improving clustering performance using various data sources. However, existing methods primarily focus on acquiring consistent information while often neglecting the issue of redundancy across multiple views. This study presents a new approach called Sufficient Multi-View Clustering (SUMVC) that examines the multi-view clustering framework from an information-theoretic standpoint. Our proposed method consists of two parts. Firstly, we develop a simple and reliable multi-view clustering method SCMVC (simple consistent multi-view clustering) that employs variational analysis to generate consistent information. Secondly, we propose a sufficient representation lower bound to enhance consistent information and minimise unnecessary information among views. The proposed SUMVC method offers a promising solution to the problem of multi-view clustering and provides a new perspective for analyzing multi-view data. To verify the effectiveness of our model, we conducted a theoretical analysis based on the Bayes Error Rate, and experiments on multiple multi-view datasets demonstrate the superior performance of SUMVC.

Machine Learning

What problem does this paper attempt to address?

This paper attempts to solve two main problems in multi - view clustering (MVC):

1. **Redundant information**: Existing multi - view clustering methods usually focus on obtaining consistent information and often overlook the redundancy problem among multiple views. This redundant information not only increases the computational complexity but may also affect the clustering performance.

2. **Insufficient information extraction**: Existing methods are insufficient in extracting valuable information from multi - view data, especially in maintaining consistency among views while reducing redundant information.

To solve these problems, the paper proposes a new method - Sufficient Multi - View Clustering (SUMVC), which re - examines the multi - view clustering framework from the perspective of information theory. Specifically, the SUMVC method consists of the following two parts:

1. **Simple Consistent Multi - View Clustering (SCMVC)**: A simple and reliable method is developed by generating consistent information through variational analysis.

2. **Sufficient representation lower bound**: A sufficient representation lower bound is proposed to enhance the consistent information and minimize the redundant information among views.

Through these two parts, the SUMVC method aims to improve the performance of multi - view clustering and provide a new perspective for analyzing multi - view data. To verify the effectiveness of this method, the author conducted a theoretical analysis based on the Bayes error rate and carried out experiments on multiple multi - view datasets. The results show that the SUMVC method is superior to existing methods in clustering performance.