A Unified View On Clustering Analysis

Qiangsheng Zhu,Yanquan Zhou,Huacan He,Ying Tian

2006-01-01

Abstract:This paper searches the existing research on clustering analysis from the point of mathematics. A unified view is presented by examining the connections among various clustering methods. Our goal is not supporting a comprehensive survey, but trying to give an essence of data analysis.

What problem does this paper attempt to address?

Multi-View Clustering Via Simultaneously Learning Shared Subspace And Affinity Matrix

Nan Xu,Yanqing Guo,Jiujun Wang,Xiangyang Luo,Xiangwei Kong

DOI: https://doi.org/10.1177/1729881417745677

IF: 1.714

2017-01-01

International Journal of Advanced Robotic Systems

Abstract:Due to the existence of multiple views in many real-world data sets, multi-view clustering is increasingly popular. Many approaches have been investigated, among which the subspace clustering methods finding the underlying subspaces of data have been developed recently. Although the subspace-based multi-view methods can achieve promising performance, the shared subspace information has not been fully utilized. To address this problem, a novel multi-view clustering model by simultaneously learning shared subspace and affinity matrix is proposed. In our method, a shared subspace is learned to preserve the effective consensus information of all views. Then, a subspace-based affinity matrix with adaptive neighbors is learned to assign the most suitable cluster to each data point. An iterative strategy is developed for solving this problem. Moreover, experiments on four benchmark data sets demonstrate that our algorithm outperforms other state-of-the-art algorithms.
A Survey on Incomplete Multi-view Clustering

Jie Wen,Zheng Zhang,Lunke Fei,Bob Zhang,Yong Xu,Zhao Zhang,Jinxing Li

DOI: https://doi.org/10.1109/TSMC.2022.3192635

2022-08-17

Abstract:Conventional multi-view clustering seeks to partition data into respective groups based on the assumption that all views are fully observed. However, in practical applications, such as disease diagnosis, multimedia analysis, and recommendation system, it is common to observe that not all views of samples are available in many cases, which leads to the failure of the conventional multi-view clustering methods. Clustering on such incomplete multi-view data is referred to as incomplete multi-view clustering. In view of the promising application prospects, the research of incomplete multi-view clustering has noticeable advances in recent years. However, there is no survey to summarize the current progresses and point out the future research directions. To this end, we review the recent studies of incomplete multi-view clustering. Importantly, we provide some frameworks to unify the corresponding incomplete multi-view clustering methods, and make an in-depth comparative analysis for some representative methods from theoretical and experimental perspectives. Finally, some open problems in the incomplete multi-view clustering field are offered for researchers.

Machine Learning,Multimedia
Analyzing Popular Clustering Algorithms from Different Viewpoints

钱卫宁,周傲英

2002-01-01

Journal of Software

Abstract:Clustering is widely studied in data mining community. It is used to partition data set into clusters so that intra-cluster data are similar and inter-cluster data are dissimilar. Different clustering methods use different similarity definition and techniques. Several popular clustering algorithms are analyzed from three different viewpoints: (1) clustering criteria, (2) cluster representation, and (3) algorithm framework. Furthermore, some new built algorithms, which mix or generalize some other algorithms, are introduced. Since the analysis is from several viewpoints, it can cover and distinguish most of the existing algorithms. It is the basis of the research of self-tuning algorithm and clustering benchmark.
A New Fuzzy Text Clustering Method Based On Nonnegative Factor Analysis

Qiangsheng Zhu,Yanquan Zhou,Huacan He,Ying Tian

2006-01-01

Abstract:As an exploring data analysis method, text clustering is very important in text mining. Just as we know there seldom exists document that only belongs to one topic, so fuzzy text clustering methods are more scientific than those crisp ones. Inspired by the nonnegative matrix factorization algorithm, we put forward a fuzzy text clustering method based on nonnegative factor analysis. This not only makes the results easy to interpret, also gives a new way for fuzzy clustering.
An Overview of Recent Multi-View Clustering.

Lele Fu,Pengfei Lin,Athanasios V. Vasilakos,Shiping Wang

DOI: https://doi.org/10.1016/j.neucom.2020.02.104

IF: 6

2020-01-01

Neurocomputing

Abstract:With the widespread deployment of sensors and the Internet-of-Things, multi-view data has become more common and publicly available. Compared to traditional data that describes objects from single perspective, multi-view data is semantically richer, more useful, however more complex. Since traditional clustering algorithms cannot handle such data, multi-view clustering has become a research hotspot. In this paper, we review some of the latest multi-view clustering algorithms, which are reasonably divided into three categories. To evaluate their performance, we perform extensive experiments on seven real-world data sets. Three mainstream metrics are used, including clustering accuracy, normalized mutual information and purity. Based on the experimental results and a large number of literature reading, we also discuss existing problems in current multi-view clustering and point out possible research directions in the future. This research provides some insights for researchers in related fields and may further promote the development of multi-view clustering algorithms.
Solutions to General Clustering Algorithmic Issues

姜园,张朝阳,仇佩亮,戚玉鹏

DOI: https://doi.org/10.3969/j.issn.1007-0249.2004.03.021

2004-01-01

Abstract:Clustering is widely used in several fields such as statistics, machine learning, pattern recognition and numerical analysis. Recently, more and more attention has been paid to it. In this paper, five issues commonly concerned are discussed, they are: assessment of clustering results, estimation of total number of clusters, data preparation, measures of data proximity and outlier handling. Representative solutions to these issues are surveyed, conclusions are summed up, development trend of algorithms to deal with these five issues is forecasted.
Multi-view clustering: A survey

Hao Wang,Yan Yang

DOI: https://doi.org/10.26599/BDMA.2018.9020003

2018-04-12

Big Data Mining and Analytics

Abstract:In the big data era, the data are generated from different sources or observed from different views. These data are referred to as multi-view data. Unleashing the power of knowledge in multi-view data is very important in big data mining and analysis. This calls for advanced techniques that consider the diversity of different views, while fusing these data. Multi-view Clustering (MvC) has attracted increasing attention in recent years by aiming to exploit complementary and consensus information across multiple views. This paper summarizes a large number of multi-view clustering algorithms, provides a taxonomy according to the mechanisms and principles involved, and classifies these algorithms into five categories, namely, co-training style algorithms, multi-kernel learning, multiview graph clustering, multi-view subspace clustering, and multi-task multi-view clustering. Therein, multi-view graph clustering is further categorized as graph-based, network-based, and spectral-based methods. Multi-view subspace clustering is further divided into subspace learning-based, and non-negative matrix factorization-based methods. This paper does not only introduce the mechanisms for each category of methods, but also gives a few examples for how these techniques are used. In addition, it lists some publically available multi-view datasets. Overall, this paper serves as an introductory text and survey for multi-view clustering.

Mathematics,Computer Science
New Approaches in Multi-View Clustering

Fanghua Ye,Zitai Chen,Hui Qian,Rui Li,Chuan Chen,ZibinZheng

DOI: https://doi.org/10.5772/intechopen.75598

2018-01-01

Abstract:Many real-world datasets can be naturally described by multiple views.Due to this, multiview learning has drawn much attention from both academia and industry.Compared to single-view learning, multi-view learning has demonstrated plenty of advantages.Clustering has long been serving as a critical technique in data mining and machine learning.Recently, multi-view clustering has achieved great success in various applications.To provide a comprehensive review of the typical multi-view clustering methods and their corresponding recent developments, this chapter summarizes five kinds of popular clustering methods and their multi-view learning versions, which include k-means, spectral clustering, matrix factorization, tensor decomposition, and deep learning.These clustering methods are the most widely employed algorithms for single-view data, and lots of efforts have been devoted to extending them for multi-view clustering.Besides, many other multi-view clustering methods can be unified into the frameworks of these five methods.To promote further research and development of multi-view clustering, some popular and open datasets are summarized in two categories.Furthermore, several open issues that deserve more exploration are pointed out in the end.
Unified Multi-View Orthonormal Non-Negative Graph Based Clustering Framework

Liangchen Liu,Qiuhong Ke,Chaojie Li,Feiping Nie,Yingying Zhu

DOI: https://doi.org/10.48550/arXiv.2211.02883

2022-12-01

Abstract:Spectral clustering is an effective methodology for unsupervised learning. Most traditional spectral clustering algorithms involve a separate two-step procedure and apply the transformed new representations for the final clustering results. Recently, much progress has been made to utilize the non-negative feature property in real-world data and to jointly learn the representation and clustering results. However, to our knowledge, no previous work considers a unified model that incorporates the important multi-view information with those properties, which severely limits the performance of existing methods. In this paper, we formulate a novel clustering model, which exploits the non-negative feature property and, more importantly, incorporates the multi-view information into a unified joint learning framework: the unified multi-view orthonormal non-negative graph based clustering framework (Umv-ONGC). Then, we derive an effective three-stage iterative solution for the proposed model and provide analytic solutions for the three sub-problems from the three stages. We also explore, for the first time, the multi-model non-negative graph-based approach to clustering data based on deep features. Extensive experiments on three benchmark data sets demonstrate the effectiveness of the proposed method.

Computer Vision and Pattern Recognition
Multiple Clusterings: Recent Advances and Perspectives.

Guoxian Yu,Liangrui Ren,Jun Wang,Carlotta Domeniconi,Xiangliang Zhang

DOI: https://doi.org/10.1016/j.cosrev.2024.100621

2024-01-01

Abstract:Clustering is a fundamental data exploration technique to discover hidden grouping structure of data. With the proliferation of big data, and the increase of volume and variety, the complexity of data multiplicity is increasing as well. Traditional clustering methods can provide only a single clustering result, which restricts data exploration to one single possible partition. In contrast, multiple clustering can simultaneously or sequentially uncover multiple non-redundant and distinct clustering solutions, which can reveal multiple interesting hidden structures of the data from different perspectives. For these reasons, multiple clustering has become a popular and promising field of study. In this survey, we have conducted a systematic review of the existing multiple clustering methods. Specifically, we categorize existing approaches according to four different perspectives (i.e., multiple clustering in the original space, in subspaces and on multi-view data, and multiple co-clustering). We summarize the key ideas underlying the techniques and their objective functions, and discuss the advantages and disadvantages of each. In addition, we built a repository of multiple clustering resources (i.e., benchmark datasets and codes). Finally, we discuss the key open issues for future investigation.
Three-way Clustering: Foundations, Survey and Challenges

Pingxin Wang,Xibei Yang,Weiping Ding,Jianming Zhan,Yiyu Yao

DOI: https://doi.org/10.1016/j.asoc.2023.111131

IF: 8.7

2024-01-01

Applied Soft Computing

Abstract:Clustering, as an unsupervised data mining technique, allows us to classify similar objects into the same cluster according to certain criteria. It helps us identify patterns between objects, reveal the associations between objects, and discover hidden structures. Traditional two-way clustering (2W clustering) algorithms represent one cluster by one set and only two types of relationships are considered between a sample and a cluster, namely, belonging to and not belonging to. Two-way decision is not always feasible especially in situations that are characterized by uncertainty and lack of information. Guided by the principle of three-way decision (3WD) as thinking in threes, three-way clustering (3W clustering) addresses the information uncertainty problem using core and the fringe regions to character a cluster. The universe is split into three sections by these two sets, which capture three kinds of relationships between objects and a cluster, namely, belonging to, partially belonging to, and not belonging-to. Compared with 2W clustering methods, 3W clustering incorporates the fringe region to describe the uncertain relationship between objects and clusters, which provides more information about the clustering structure. This survey points out the historical developments of three-way clustering and makes an overview of the achievements in the field of three-way clustering. In addition, to reap a clearer grasp of the development and research significance of three-way clustering, we divide the existing three-way clustering approaches into two categories and present the bibliometric analysis of related approaches. Finally, we point out some challenges and future research topics in three-way clustering. It is hoped that this review can serve as a reference and provide convenience for scholars and practitioners in the field of three-way clustering.
Multiple clusterings: Recent advances and perspectives

Guoxian Yu,Liangrui Ren,Jun Wang,Carlotta Domeniconi,Xiangliang Zhang

DOI: https://doi.org/10.1016/j.cosrev.2024.100621

IF: 8.757

2024-01-01

Computer Science Review

Abstract:Clustering is a fundamental data exploration technique to discover hidden grouping structure of data. With the proliferation of big data, and the increase of volume and variety, the complexity of data multiplicity is increasing as well. Traditional clustering methods can provide only a single clustering result, which restricts data exploration to one single possible partition. In contrast, multiple clustering can simultaneously or sequentially uncover multiple non -redundant and distinct clustering solutions, which can reveal multiple interesting hidden structures of the data from different perspectives. For these reasons, multiple clustering has become a popular and promising field of study. In this survey, we have conducted a systematic review of the existing multiple clustering methods. Specifically, we categorize existing approaches according to four different perspectives (i.e., multiple clustering in the original space, in subspaces and on multi -view data, and multiple co -clustering). We summarize the key ideas underlying the techniques and their objective functions, and discuss the advantages and disadvantages of each. In addition, we built a repository of multiple clustering resources (i.e., benchmark datasets and codes). Finally, we discuss the key open issues for future investigation.
Multi-view Subspace Clustering Based on Unified Measure Standard

Kewei Tang,Xiaoru Wang,Jinhong Li

DOI: https://doi.org/10.1007/s11063-022-11136-6

IF: 2.565

2023-01-01

Neural Processing Letters

Abstract:In recent years, multi-view subspace clustering has attracted extensive attention. In order to improve the clustering performance, the previous work tries to explore the consistency and specificity between different views by making the common representation matrix as close to the representation matrix learned in each view as possible. However, the values of the elements corresponding to a similar degree of the strong or weak relationship often have different magnitude levels in the representation matrix learned in each view. In this situation, the above strategy will make the information of some views ignored or magnified. To overcome this limitation, we propose a novel multi-view subspace clustering method in this paper. Because our proposed method can normalize the degree of the strong or weak relationship in each view to the unified measure standard by scaling the representation matrix learned in each view, the consistency and specificity between different views will be mined more effectively. In addition, we provide a theoretical analysis of the convergence and computation complexity of our numerical algorithm. The experimental results on several benchmark data sets indicate that our proposed method is not only effective but also efficient for the problem of multi-view subspace clustering.
Research on Cluster Aggregation Approaches

YANG Cao-yuan,LIU Da-you,YANG Bo,CHI Shu-zhen,JIN Di

DOI: https://doi.org/10.3969/j.issn.1002-137x.2011.02.038

2011-01-01

Computer Science

Abstract:Clustering aggregation can offer a partition that could better reflect the inherent structure of the data set by studying and integrating many clustering results of the original data set.Clustering aggregation could detect and deal with the isolated points preferably,which improves the quality of clustering.This paper made an overview of the relevant knowledge of the clustering aggregation,presented the concepts and advantages of clustering aggregation.It pre-sented three approaches to get clustering members according to the uesd cluster algorithms;analysed their respective advantages,disadvantages and application conditions;presented the existing consensus functions;explained the basic principles and pointed out their limitations.Finally,it discussed the future research directions.
Individuality Meets Commonality: A Unified Graph Learning Framework for Multi-View Clustering

Zhibin Gu,Songhe Feng

DOI: https://doi.org/10.1145/3532612

IF: 4.157

2023-02-20

ACM Transactions on Knowledge Discovery from Data

Abstract:Multi-view clustering, which aims at boosting the clustering performance by leveraging the individual information and the common information of multi-view data, has gained extensive consideration in recent years. However, most existing multi-view clustering algorithms either focus on extracting the multi-view individuality or emphasize on exploring the multi-view commonality, neither of which can fully utilize the comprehensive information from multiple views. To this end, we propose a novel algorithm named V iew-specific and C onsensus G raph A lignment (VCGA) for multi-view clustering, which simultaneously formulates the multi-view individuality and the multi-view commonality into a unified framework to effectively partition data points. To be specific, the VCGA model constructs the view-specific graphs and the shared graph from original multi-view data and hidden latent representation, respectively. Furthermore, the view-specific graphs of different views and the consensus graph are aligned into an informative target graph, which is employed as a crucial input to the standard spectral clustering method for clustering. Extensive experimental results on six benchmark datasets demonstrate the superiority of our method against other state-of-the-art clustering algorithms.

computer science, information systems, software engineering
A Survey on Incomplete Multiview Clustering

Jie Wen,Zheng Zhang,Lunke Fei,Bob Zhang,Yong Xu,Zhao Zhang,Jinxing Li

DOI: https://doi.org/10.1109/tsmc.2022.3192635

2022-01-01

IEEE Transactions on Systems Man and Cybernetics Systems

Abstract:Conventional multiview clustering seeks to partition data into respective groups based on the assumption that all views are fully observed. However, in practical applications, such as disease diagnosis, multimedia analysis, and recommendation system, it is common to observe that not all views of samples are available in many cases, which leads to the failure of the conventional multiview clustering methods. Clustering on such incomplete multiview data is referred to as incomplete multiview clustering (IMC). In view of the promising application prospects, the research of IMC has noticeable advances in recent years. However, there is no survey to summarize the current progresses and point out the future research directions. To this end, we review the recent studies of IMC. Importantly, we provide some frameworks to unify the corresponding IMC methods and make an in-depth comparative analysis for some representative methods from theoretical and experimental perspectives. Finally, some open problems in the IMC field are offered for researchers. The related codes are released at https://github.com/DarrenZZhang/Survey_IMC.
Deep Clustering: A Comprehensive Survey

Yazhou Ren,Jingyu Pu,Zhimeng Yang,Jie Xu,Guofeng Li,Xiaorong Pu,Philip S. Yu,Lifang He

DOI: https://doi.org/10.48550/arXiv.2210.04142

2022-10-09

Abstract:Cluster analysis plays an indispensable role in machine learning and data mining. Learning a good data representation is crucial for clustering algorithms. Recently, deep clustering, which can learn clustering-friendly representations using deep neural networks, has been broadly applied in a wide range of clustering tasks. Existing surveys for deep clustering mainly focus on the single-view fields and the network architectures, ignoring the complex application scenarios of clustering. To address this issue, in this paper we provide a comprehensive survey for deep clustering in views of data sources. With different data sources and initial conditions, we systematically distinguish the clustering methods in terms of methodology, prior knowledge, and architecture. Concretely, deep clustering methods are introduced according to four categories, i.e., traditional single-view deep clustering, semi-supervised deep clustering, deep multi-view clustering, and deep transfer clustering. Finally, we discuss the open challenges and potential future opportunities in different fields of deep clustering.

Machine Learning
A Survey on Multi-View Clustering

Guoqing Chao,Shiliang Sun,Jinbo Bi

DOI: https://doi.org/10.1109/tai.2021.3065894

2021-01-01

IEEE Transactions on Artificial Intelligence

Abstract:Clustering is a machine learning paradigm of dividing sample subjects into a number of groups such that subjects in the same groups are more similar to those in other groups. With advances in information acquisition technologies, samples can frequently be viewed from different angles or in different modalities, generating multi-view data. Multi-view clustering, that clusters subjects into subgroups using multi-view data, has attracted more and more attentions. Although MVC methods have been developed rapidly, there has not been enough survey to summarize and analyze the current progress. Therefore, we propose a novel taxonomy of the MVC approaches. Similar to other machine learning methods, we categorize them into generative and discriminative classes. In discriminative class, based on the way of view integration, we split it further into five groups: Common Eigenvector Matrix, Common Coefficient Matrix, Common Indicator Matrix, Direct Combination and Combination After Projection. Furthermore, we relate MVC to other topics: multi-view representation, ensemble clustering, multi-task clustering, multi-view supervised and semi-supervised learning. Several representative real-world applications are elaborated for practitioners. Some benchmark multi-view datasets are introduced and representative MVC algorithms from each group are empirically evaluated to analyze how they perform on benchmark datasets. To promote future development of MVC approaches, we point out several open problems that may require further investigation and thorough examination.
A New Model and Algorithm for Clustering

Guohong Liang,Ying Li,Junqing Feng

DOI: https://doi.org/10.1007/978-3-319-69096-4_11

2017-01-01

Abstract:Clustering is a unsupervised pattern recognition method and an important research content in data mining and artificial intelligence. A new mathematical model of clustering is proposed using graph theory. It is proved theoretically that the model is a submodular function. According to it, a corresponding algorithm is proposed.
New approaches for clustering high dimensional data

Wei Wang,Jinze Liu

2006-01-01

Abstract:Clustering is one of the most effective methods for analyzing datasets that contain a large number of objects with numerous attributes. Clustering seeks to identify groups, or clusters, of similar objects. In low dimensional space, the similarity between objects is often evaluated by summing the difference across all of their attributes. High dimensional data, however, may contain irrelevant attributes which mask the existence of clusters. The discovery of groups of objects that are highly similar within some subsets of relevant attributes becomes an important but challenging task. My thesis focuses on various models and algorithms for this task. We first present a flexible clustering model, namely OP-Cluster (Order Preserving Cluster). Under this model, two objects are similar on a subset of attributes if the values of these two objects induce the same relative ordering of these attributes. OP-Clustering algorithm has demonstrated to be useful to identify co-regulated genes in gene expression data. We also propose a semi-supervised approach to discover biologically meaningful OP-Clusters by incorporating existing gene function classifications into the clustering process. This semi-supervised algorithm yields only OP-clusters that are significantly enriched by genes from specific functional categories. Real datasets are often noisy. We propose a noise-tolerant clustering algorithm for mining frequently occurring itemsets. This algorithm is called approximate frequent itemsets (AFI). Both the theoretical and experimental results demonstrate that our AFI mining algorithm has higher recoverability of real clusters than any other existing itemset mining approaches. Pair-wise dissimilarities are often derived from original data to reduce the complexities of high dimensional data. Traditional clustering algorithms taking pair-wise dissimilarities as input often generate disjoint clusters from pair-wise dissimilarities. It is well known that the classification model represented by disjoint clusters is inconsistent with many real classifications, such gene function classifications. We develop a Poclustering algorithm, which generates overlapping clusters from pair-wise dissimilarities. We prove that by allowing overlapping clusters, Poclustering fully preserves the information of any dissimilarity matrices while traditional partitioning algorithms may cause significant information loss.

A Unified View On Clustering Analysis

Multi-View Clustering Via Simultaneously Learning Shared Subspace And Affinity Matrix

A Survey on Incomplete Multi-view Clustering

Analyzing Popular Clustering Algorithms from Different Viewpoints

A New Fuzzy Text Clustering Method Based On Nonnegative Factor Analysis

An Overview of Recent Multi-View Clustering.

Solutions to General Clustering Algorithmic Issues

Multi-view clustering: A survey

New Approaches in Multi-View Clustering

Unified Multi-View Orthonormal Non-Negative Graph Based Clustering Framework

Multiple Clusterings: Recent Advances and Perspectives.

Three-way Clustering: Foundations, Survey and Challenges

Multiple clusterings: Recent advances and perspectives

Multi-view Subspace Clustering Based on Unified Measure Standard

Research on Cluster Aggregation Approaches

Individuality Meets Commonality: A Unified Graph Learning Framework for Multi-View Clustering

A Survey on Incomplete Multiview Clustering

Deep Clustering: A Comprehensive Survey

A Survey on Multi-View Clustering

A New Model and Algorithm for Clustering

New approaches for clustering high dimensional data