Intrinsic Weight Learning Approach for Multi-view Clustering

Feiping Nie,Jing Li,Xuelong Li
DOI: https://doi.org/10.48550/arXiv.1906.08905
2019-06-21
Abstract:Exploiting different representations, or views, of the same object for better clustering has become very popular these days, which is conventionally called multi-view clustering. Generally, it is essential to measure the importance of each individual view, due to some noises, or inherent capacities in description. Many previous works model the view importance as weight, which is simple but effective empirically. In this paper, instead of following the traditional thoughts, we propose a new weight learning paradigm in context of multi-view clustering in virtue of the idea of re-weighted approach, and we theoretically analyze its working mechanism. Meanwhile, as a carefully achieved example, all of the views are connected by exploring a unified Laplacian rank constrained graph, which will be a representative method to compare with other weight learning approaches in experiments. Furthermore, the proposed weight learning strategy is much suitable for multi-view data, and it can be naturally integrated with many existing clustering learners. According to the numerical experiments, the proposed intrinsic weight learning approach is proved effective and practical to use in multi-view clustering.
Machine Learning
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to effectively measure the importance of each view in multi - view clustering. Specifically, the author points out the limitations of traditional methods in dealing with view weights and proposes a new intrinsic weight - learning method to improve the effect of multi - view clustering. ### Problem Background Multi - view clustering refers to using different representations or views of the same object for cluster analysis. Since different views may have noise or differences in descriptive ability, it is necessary to measure the importance of each view. Traditional methods usually achieve this by assigning a weight to each view, but these methods have problems such as being sensitive to hyper - parameters and failing to fully explore the relationships between views. ### Core Problems of the Paper The paper proposes a new intrinsic weight - learning method, aiming to solve the following problems: 1. **Hyper - parameter Sensitivity**: Traditional methods are very sensitive to the selection of hyper - parameters, resulting in unstable results. 2. **Insufficient Mining of View Relationships**: Traditional methods fail to fully explore the coupling relationships between views, thus affecting the clustering effect. ### Solutions The solutions proposed in the paper include: - **New Weight - learning Paradigm**: By introducing the re - weighting idea, a new intrinsic weight - learning method is proposed. - **Unified Laplacian Rank - Constrained Graph**: All views are connected by exploring a unified Laplacian rank - constrained graph to better encode the relationships between instances. - **Theoretical Analysis and Experimental Verification**: The working mechanism, convergence, and time complexity of the new method are analyzed in detail, and its effectiveness and practicality are verified through a large number of experiments. ### Formula Presentation The formulas involved in the paper are presented in Markdown format as follows: #### Objective Function \[ \min_{x} \sum_{v = 1}^{M} \Phi_v^{\frac{p}{2}}(x) \quad \text{s.t.} \quad x\in C_x, \quad 0 < p\leq2 \] #### Lagrange Function \[ \min_{x} \sum_{v = 1}^{M} \Phi_v^{\frac{p}{2}}(x)+G(\Lambda, x) \] #### Weight Update Formula \[ \alpha_v=\frac{p}{2} \Phi_v^{\frac{p - 2}{2}}(x) \quad \forall 1\leq v\leq M \] ### Experimental Results Through experiments on multiple datasets, the paper verifies the effectiveness of the new method. For example, on the synthetic dataset, the CLR - IW method can recover a completely clean block - diagonal matrix, and the learned weights are 0.528/0.472, indicating the effectiveness of multi - view learning. In addition, on actual datasets such as MSRC - v1, Caltech101, etc., the new method also significantly outperforms the baseline methods. In conclusion, this paper solves the problem of measuring the importance of views in multi - view clustering by proposing a new intrinsic weight - learning method and improves the clustering effect.