Dynamic Graph Learning for Dimension Reduction and Data Clustering

Liehuang Zhu,Jingjing Li,Zheng Zhang

2024-01-01

Synthesis Lectures on Computer Science

Abstract:This book illustrates how to achieve effective dimension reduction and data clustering using dynamic graph learning.

What problem does this paper attempt to address?

Dynamic Graph Learning for Data Clustering

Lei Zhu,Jingjing Li,Zheng Zhang

2023-01-01

Synthesis Lectures on Computer Science

Abstract:Clustering is one of the fundamental techniques in machine learning. It has been widely applied to various research fields, such as gene expression [1], face analysis [2], image annotation [3], and recommendation [4, 5]. In the past decades, many clustering approaches have been developed, such as k-means [6], spectral clustering [7–11] spectral embedded clustering [12] and normalized cut [13].
Self-Adaptive Clustering of Dynamic Multi-Graph Learning

Zhou Bo,Li Yangding,Huang Xincheng,Li Jiaye

DOI: https://doi.org/10.1007/s11063-020-10405-6

2021-01-01

Abstract:In the process of graph clustering, the quality requirements for the structure of data graph are very strict, which will directly affect the final clustering results. Enhancing data graph is the key step to improve the performance of graph clustering. In this paper, we propose a self-adaptive clustering method to obtain a dynamic fine-tuned sparse graph by learning multiple static original graph with different sparsity degrees. By imposing a constrainted rank on the corresponding Laplacian matrix, the method utilizes the eigenvectors of the Laplacian matrix to create a new and simple data sparse matrix to have exactly k connected components, so that the method can quickly and directly learn the clustering results. The experimental results on synthetic and multiple public datasets verify that the proposed method is meaningful and beneficial to discover the real cluster distribution of datasets.
Deep Clustering and Visualization for End-to-End High-Dimensional Data Analysis.

Lirong Wu,Lifan Yuan,Guojiang Zhao,Haitao Lin,Stan Z. Li

DOI: https://doi.org/10.1109/tnnls.2022.3151498

IF: 14.255

2023-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:High-dimensional data analysis for exploration and discovery includes two fundamental tasks: deep clustering and data visualization. When these two associated tasks are done separately, as is often the case thus far, disagreements can occur among the tasks in terms of geometry preservation. Namely, the clustering process is often accompanied by the corruption of the geometric structure, whereas visualization aims to preserve the data geometry for better interpretation. Therefore, how to achieve deep clustering and data visualization in an end-to-end unified framework is an important but challenging problem. In this article, we propose a novel neural network-based method, called deep clustering and visualization (DCV), to accomplish the two associated tasks end-to-end to resolve their disagreements. The DCV framework consists of two nonlinear dimensionality reduction (NLDR) transformations: 1) one from the input data space to latent feature space for clustering and 2) the other from the latent feature space to the final 2-D space for visualization. Importantly, the first NLDR transformation is mainly optimized by one Clustering Loss, allowing arbitrary corruption of the geometric structure for better clustering, while the second NLDR transformation is optimized by one Geometry-Preserving Loss to recover the corrupted geometry for better visualization. Extensive comparative results show that the DCV framework outperforms other leading clustering-visualization algorithms in terms of both quantitative evaluation metrics and qualitative visualization.
Dynamic Graph Learning for Feature Selection

Lei Zhu,Jingjing Li,Zheng Zhang

DOI: https://doi.org/10.1007/978-3-031-42313-0_3

2023-01-01

Synthesis Lectures on Computer Science

Abstract:In the era of big data, data presents multi-view, high-dimensional and complex characteristics. For one thing, with multi-view features, the data could be characterized more precisely and comprehensively from different perspectives. For another, high-dimensional multi-view features inevitably generate expensive computation costs and cause massive storage costs. Moreover, since raw data generally contains adverse noise, outlying entries, irrelevant and redundant features, the intrinsic dimension of the data may be much lower than the dimension of the raw data. The low-dimensional and robust data representation can effectively improve model efficiency as well as accuracy and thus is vital for downstream tasks in various fields.
Dynamic Graph Learning for Feature Projection

Lei Zhu,Jingjing Li,Zheng Zhang

2023-01-01

Synthesis Lectures on Computer Science

Abstract:High-dimensional features have gained widespread usage in various research fields such as multimedia computing, data mining, pattern recognition, and machine learning. However, the presence of high-dimensional features often gives rise to the “curse of dimensionality" problem and places significant computational burdens on machine learning models. To alleviate these issues, dimensionality reduction techniques are employed to identify low-dimensional latent subspaces that retain the data similarities observed in the original high-dimensional space. Two common paradigms used for dimensionality reduction are feature selection and feature projection. Feature selection involves identifying a subset of the original features as low-dimensional representations by discarding irrelevant and noisy features. On the other hand, feature projection utilizes a specific transformation matrix to generate projected dimensions that preserve the intrinsic data characteristics. Based on their reliance on semantic labels, feature projection can be further categorized into two families: unsupervised and supervised feature projection.
Dynamic Structural Clustering on Graphs

Boyu Ruan,Junhao Gan,Hao Wu,Anthony Wirth

DOI: https://doi.org/10.48550/arXiv.2108.11549

2021-08-26

Abstract:Structural Clustering ($DynClu$) is one of the most popular graph clustering paradigms. In this paper, we consider $StrClu$ under two commonly adapted similarities, namely Jaccard similarity and cosine similarity on a dynamic graph, $G = \langle V, E\rangle$, subject to edge insertions and deletions (updates). The goal is to maintain certain information under updates, so that the $StrClu$ clustering result on~$G$ can be retrieved in $O(|V| + |E|)$ time, upon request. The state-of-the-art worst-case cost is $O(|V|)$ per update; we improve this update-time bound significantly with the $\rho$-approximate notion. Specifically, for a specified failure probability, $\delta^*$, and every sequence of $M$ updates (no need to know $M$'s value in advance), our algorithm, $DynELM$, achieves $O(\log^2 |V| + \log |V| \cdot \log \frac{M}{\delta^*})$ amortized cost for each update, at all times in linear space. Moreover, $DynELM$ provides a provable "sandwich" guarantee on the clustering quality at all times after \emph{each update} with probability at least $1 - \delta^*$. We further develop $DynELM$ into our ultimate algorithm, $DynStrClu$, which also supports cluster-group-by queries. Given $Q\subseteq V$, this puts the non-empty intersection of $Q$ and each $StrClu$ cluster into a distinct group. $DynStrClu$ not only achieves all the guarantees of $DynELM$, but also runs cluster-group-by queries in $O(|Q|\cdot \log |V|)$ time. We demonstrate the performance of our algorithms via extensive experiments, on 15 real datasets. Experimental results confirm that our algorithms are up to three orders of magnitude more efficient than state-of-the-art competitors, and still provide quality structural clustering results. Furthermore, we study the difference between the two similarities w.r.t. the quality of approximate clustering results.

Data Structures and Algorithms
Adaptive Dimension Reduction for Clustering High Dimensional Data

C Ding,XF He,HY Zha,HD Simon

DOI: https://doi.org/10.1109/icdm.2002.1183897

2002-01-01

Abstract:It is well-known that for high dimensional data clustering, standard algorithms such as EM and K-means are often trapped in a local minimum. Many initialization methods have been proposed to tackle this problem, with only limited success. In this paper we propose a new approach to resolve this problem by repeated dimension reductions such that K-means or EM are performed only in very low dimensions. Cluster membership is utilized as a bridge between the reduced dimensional subspace and the original space, providing flexibility and ease of implementation. Clustering analysis performed on highly overlapped Gaussians, DNA gene expression profiles and Internet newsgroups demonstrate the effectiveness of the proposed algorithm.
Semi-Supervised Clustering via Dynamic Graph Structure Learning

Huaming Ling,Chenglong Bao,Xin Liang,Zuoqiang Shi

DOI: https://doi.org/10.48550/arxiv.2209.02513

2022-01-01

Abstract: Most existing semi-supervised graph-based clustering methods exploit the supervisory information by either refining the affinity matrix or directly constraining the low-dimensional representations of data points. The affinity matrix represents the graph structure and is vital to the performance of semi-supervised graph-based clustering. However, existing methods adopt a static affinity matrix to learn the low-dimensional representations of data points and do not optimize the affinity matrix during the learning process. In this paper, we propose a novel dynamic graph structure learning method for semi-supervised clustering. In this method, we simultaneously optimize the affinity matrix and the low-dimensional representations of data points by leveraging the given pairwise constraints. Moreover, we propose an alternating minimization approach with proven convergence to solve the proposed nonconvex model. During the iteration process, our method cyclically updates the low-dimensional representations of data points and refines the affinity matrix, leading to a dynamic affinity matrix (graph structure). Specifically, for the update of the affinity matrix, we enforce the data points with remarkably different low-dimensional representations to have an affinity value of 0. Furthermore, we construct the initial affinity matrix by integrating the local distance and global self-representation among data points. Experimental results on eight benchmark datasets under different settings show the advantages of the proposed approach.
Incomplete Multi-View Clustering Based on Dynamic Dimensionality Reduction Weighted Graph Learning

Yaosong Yu,Dongpu Sun

DOI: https://doi.org/10.1109/access.2024.3358681

IF: 3.9

2024-02-10

IEEE Access

Abstract:Aiming at the existing incomplete multi-view clustering methods that usually ignore the noise and redundancy of the original data, hide the valuable information in the missing views, and the different importance of each view, this paper proposes the incomplete multi-view clustering based on dynamic dimensionality reduction weighted graph learning (ARDGL), which is mainly divided into two parts: learning the similarity matrix by dynamic dimensionality reduction weighted graph and fusion of the self-weighted graph. In the process of learning the similarity matrix, the noise and redundancy of the original data are effectively filtered by the dynamic dimensionality reduction weighted graph learning, and the influence of incomplete data on the clustering results is attenuated. In the process of self-weighted graph fusion, multiple views are valued as having different importance by introducing view weights, and non-negative orthogonality constraints are added to improve the quality of the consensus matrix, so that clustering results can be obtained without post-processing. To solve the objective function, an alternating iteration algorithm is proposed. Experiments are conducted on four datasets and the results show that the algorithm has better performance.

computer science, information systems,telecommunications,engineering, electrical & electronic
Graph Clustering with Dynamic Embedding

Carl Yang,Mengxiong Liu,Zongyi Wang,Liyuan Liu,Jiawei Han

DOI: https://doi.org/10.48550/arXiv.1712.08249

2017-12-22

Abstract:Graph clustering (or community detection) has long drawn enormous attention from the research on web mining and information networks. Recent literature on this topic has reached a consensus that node contents and link structures should be integrated for reliable graph clustering, especially in an unsupervised setting. However, existing methods based on shallow models often suffer from content noise and sparsity. In this work, we propose to utilize deep embedding for graph clustering, motivated by the well-recognized power of neural networks in learning intrinsic content representations. Upon that, we capture the dynamic nature of networks through the principle of influence propagation and calculate the dynamic network embedding. Network clusters are then detected based on the stable state of such an embedding. Unlike most existing embedding methods that are task-agnostic, we simultaneously solve for the underlying node representations and the optimal clustering assignments in an end-to-end manner. To provide more insight, we theoretically analyze our interpretation of network clusters and find its underlying connections with two widely applied approaches for network modeling. Extensive experimental results on six real-world datasets including both social networks and citation networks demonstrate the superiority of our proposed model over the state-of-the-art.

Social and Information Networks,Physics and Society
Graph Embedding with Constraints

Xiaofei He,Ming Ji,Hujun Bao

2009-01-01

Abstract:Recently graph based dimensionality reduction has received a lot of interests in many fields of information processing. Central to it is a graph structure which models the geometrical and discriminant structure of the data manifold. When label information is available, it is usually incorporated into the graph structure by modifying the weights between data points. In this paper, we propose a novel dimensionality reduction algorithm, called Constrained Graph Embedding, which considers the label information as additional constraints. Specifically, we constrain the space of the solutions that we explore only to contain embedding results that are consistent with the labels. Experimental results on two real life data sets illustrate the effectiveness of our proposed method.
Robust Dimensionality Reduction via Low-rank Laplacian Graph Learning

Mingjian Cai,Xiangjun Shen,Stanley Ebhohimhen Abhadiomhen,Yingfeng Cai,Sirui Tian

DOI: https://doi.org/10.1145/3582698

IF: 5

2023-04-01

ACM Transactions on Intelligent Systems and Technology

Abstract:Manifold learning is a widely used technique for dimensionality reduction as it can reveal the intrinsic geometric structure of data. However, its performance decreases drastically when data samples are contaminated by heavy noise or occlusions, which leads to unsatisfying data processing performance. We propose a novel robust dimensionality reduction method via low-rank Laplacian graph learning for classification and clustering tasks to solve the above problem. First, we construct a low-rank Laplacian graph by combining manifold learning and subspace learning. This graph can capture both global and local structural information of the data. And we introduce rank constraints for the Laplacian graph to make it more discriminative. Second, we put the learning of projection matrix and sample affinity graph into a unified framework. The projection matrix is embedded into a robust low-rank Laplacian graph so that the low-dimensional mapping of data can maintain the structural information in the graph well. Finally, we add a regularization term to the projection matrix to make it have the ability of both feature extraction and feature selection. Therefore, the proposed model can resist the interference of noise or data damage to learn the optimal projection to achieve better performance in dimensionality reduction through such a data dimensionality reduction joint framework. Comprehensive experiments on various benchmark datasets with varying degrees of occlusions or corruptions are carried out to evaluate the performance of the proposed method. Compared with the state-of-the-art dimensionality reduction methods in the literature, the experimental results are inspiring, showing our method’s effectiveness and robustness in classification and clustering, especially in object recognition scenarios with noise or occlusions.

computer science, information systems, artificial intelligence
Optimization of Graph Clustering Inspired by Dynamic Belief Systems

Huijia Li,Haobin Cao,Yuhao Feng,Xiaoyan Li,Jian Pei

DOI: https://doi.org/10.1109/tkde.2023.3274547

IF: 9.235

2023-01-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:Graph clustering is essential to understand the nature and behavior of real world such as social network, technical network and transportation network. Different from the existing studies, we propose a new Markov clustering method inspired by belief dynamical system which can be used in general for optimization of different quality measures. By a rigorous theoretical proof, it has been shown that the quality function's global maximum is a dynamical system's asymptotically stable fixed point. Under specified conditions, the trajectory of the dynamical converges to the cluster labels of corresponding nodes. Particularly, a general formulation can unite well-known methodologies and the quality functions that correspond to them. The algorithm is fast and its computational complexity is nearly linear with the scale of sparse networks. Finally, we thoroughly evaluate our methodology on a variety of synthetic and real-world networks with various network properties, particularly on the dynamical networks. The results demonstrate that when compared to the current state-of-the-art algorithms, our method performs better on these networks.

computer science, information systems, artificial intelligence,engineering, electrical & electronic
Learning Robust Graph for Clustering

Zheng Liu,Wei Jin,Ying Mu

DOI: https://doi.org/10.1002/int.22901

IF: 8.993

2022-01-01

International Journal of Intelligent Systems

Abstract:Graph is a popular technique to explore the structure of data. Many related algorithms directly construct graphs based on the original data. Actually, the samples collected in real life usually contain noise. Besides, some unimportant features probably exist in high-dimensional data. Therefore, this way cannot assure a high-quality graph and furthermore brings some adverse influence to the following tasks. In this paper, we incorporate robust graph learning and dimensionality reduction into a unified framework which also seamlessly integrates the clustering task. On the basis of the framework, Euclidean distance-based robust graph (EDBRG) and self-expressiveness-based robust graph (SEBRG) are presented. Both EDBRG and SEBRG contain clustering information from which the clustering results can be obtained directly. By projecting the original data into a discriminative subspace where the negative effect of redundant features and noise is removed, EDBRG and SEBRG are informative, robust, and sparse. During the whole mapping process, the main energy of data is preserved. Finally, some data sets are adopted to test the performances of EDBRG and SEBRG. Extensive experiments illustrate that the proposed methods have many advantages for the task of clustering, comparing with the state-of-the-art algorithms.
Discriminative Embedded Clustering: A Framework for Grouping High-Dimensional Data.

Chenping Hou,Feiping Nie,Dongyun Yi,Dacheng Tao

DOI: https://doi.org/10.1109/TNNLS.2014.2337335

IF: 14.255

2015-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:In many real applications of machine learning and data mining, we are often confronted with high-dimensional data. How to cluster high-dimensional data is still a challenging problem due to the curse of dimensionality. In this paper, we try to address this problem using joint dimensionality reduction and clustering. Different from traditional approaches that conduct dimensionality reduction and cl...
DRGraph: An Efficient Graph Layout Algorithm for Large-scale Graphs by Dimensionality Reduction

Minfeng Zhu,Wei Chen,Yuanzhe Hu,Yuxuan Hou,Liangjun Liu,Kaiyuan Zhang

DOI: https://doi.org/10.48550/arXiv.2008.07799

2020-08-18

Abstract:Efficient layout of large-scale graphs remains a challenging problem: the force-directed and dimensionality reduction-based methods suffer from high overhead for graph distance and gradient computation. In this paper, we present a new graph layout algorithm, called DRGraph, that enhances the nonlinear dimensionality reduction process with three schemes: approximating graph distances by means of a sparse distance matrix, estimating the gradient by using the negative sampling technique, and accelerating the optimization process through a multi-level layout scheme. DRGraph achieves a linear complexity for the computation and memory consumption, and scales up to large-scale graphs with millions of nodes. Experimental results and comparisons with state-of-the-art graph layout methods demonstrate that DRGraph can generate visually comparable layouts with a faster running time and a lower memory requirement.

Social and Information Networks
Deep Temporal Graph Clustering

Meng Liu,Yue Liu,Ke Liang,Wenxuan Tu,Siwei Wang,Sihang Zhou,Xinwang Liu

2024-04-11

Abstract:Deep graph clustering has recently received significant attention due to its ability to enhance the representation learning capabilities of models in unsupervised scenarios. Nevertheless, deep clustering for temporal graphs, which could capture crucial dynamic interaction information, has not been fully explored. It means that in many clustering-oriented real-world scenarios, temporal graphs can only be processed as static graphs. This not only causes the loss of dynamic information but also triggers huge computational consumption. To solve the problem, we propose a general framework for deep Temporal Graph Clustering called TGC, which introduces deep clustering techniques to suit the interaction sequence-based batch-processing pattern of temporal graphs. In addition, we discuss differences between temporal graph clustering and static graph clustering from several levels. To verify the superiority of the proposed framework TGC, we conduct extensive experiments. The experimental results show that temporal graph clustering enables more flexibility in finding a balance between time and space requirements, and our framework can effectively improve the performance of existing temporal graph learning methods. The code is released: <a class="link-external link-https" href="https://github.com/MGitHubL/Deep-Temporal-Graph-Clustering" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Artificial Intelligence
A Brief Survey on Representation Learning based Graph Dimensionality Reduction Techniques

Akhil Pandey Akella

DOI: https://doi.org/10.48550/arXiv.2211.05594

IF: 5.414

2022-10-13

Machine Learning

Abstract:Dimensionality reduction techniques map data represented on higher dimensions onto lower dimensions with varying degrees of information loss. Graph dimensionality reduction techniques adopt the same principle of providing latent representations of the graph structure with minor adaptations to the output representations along with the input data. There exist several cutting edge techniques that are efficient at generating embeddings from graph data and projecting them onto low dimensional latent spaces. Due to variations in the operational philosophy, the benefits of a particular graph dimensionality reduction technique might not prove advantageous to every scenario or rather every dataset. As a result, some techniques are efficient at representing the relationship between nodes at lower dimensions, while others are good at encapsulating the entire graph structure on low dimensional space. We present this survey to outline the benefits as well as problems associated with the existing graph dimensionality reduction techniques. We also attempted to connect the dots regarding the potential improvements to some of the techniques. This survey could be helpful for upcoming researchers interested in exploring the usage of graph representation learning to effectively produce low-dimensional graph embeddings with varying degrees of granularity.
Adaptive Sparse Graph Learning Based Dimensionality Reduction for Classification.

Puhua Chen,Licheng Jiao,Fang Liu,Zhiqiang Zhao,Jiaqi Zhao

DOI: https://doi.org/10.1016/j.asoc.2019.04.029

IF: 8.7

2019-01-01

Applied Soft Computing

Abstract:To preserve the sparsity structure in dimensionality reduction, sparsity preserving projection (SPP) is widely used in many fields of classification, which has the advantages of noise robustness and data adaptivity compared with other graph based method. However, the sparsity parameter of SPP is fixed for all samples without any adjustment. In this paper, an improved SPP method is proposed, which has an adaptive parameter adjustment strategy during sparse graph construction. With this adjustment strategy, the sparsity parameter of each sample is adjusted adaptively according to the relationship of those samples with nonzero sparse representation coefficients, by which the discriminant information of graph is enhanced. With the same expectation, similarity information both in original space and projection space is applied for sparse representation as guidance information. Besides, a new measurement is introduced to control the influence of each sample’s local structure on projection learning, by which more correct discriminant information should be preserved in the projection space. With the contributions of above strategies, the low-dimensional space with high discriminant ability is found, which is more beneficial for classification. Experimental results on three datasets demonstrate that the proposed approach can achieve better classification performance over some available state-of-the-art approaches.
A dynamical systems based framework for dimension reduction

Ryeongkyung Yoon,Braxton Osting

DOI: https://doi.org/10.48550/arXiv.2204.08155

IF: 5.414

2022-04-18

Machine Learning

Abstract:We propose a novel framework for learning a low-dimensional representation of data based on nonlinear dynamical systems, which we call dynamical dimension reduction (DDR). In the DDR model, each point is evolved via a nonlinear flow towards a lower-dimensional subspace; the projection onto the subspace gives the low-dimensional embedding. Training the model involves identifying the nonlinear flow and the subspace. Following the equation discovery method, we represent the vector field that defines the flow using a linear combination of dictionary elements, where each element is a pre-specified linear/nonlinear candidate function. A regularization term for the average total kinetic energy is also introduced and motivated by optimal transport theory. We prove that the resulting optimization problem is well-posed and establish several properties of the DDR method. We also show how the DDR method can be trained using a gradient-based optimization method, where the gradients are computed using the adjoint method from optimal control theory. The DDR method is implemented and compared on synthetic and example datasets to other dimension reductions methods, including PCA, t-SNE, and Umap.

Dynamic Graph Learning for Dimension Reduction and Data Clustering

Dynamic Graph Learning for Data Clustering

Self-Adaptive Clustering of Dynamic Multi-Graph Learning

Deep Clustering and Visualization for End-to-End High-Dimensional Data Analysis.

Dynamic Graph Learning for Feature Selection

Dynamic Graph Learning for Feature Projection

Dynamic Structural Clustering on Graphs

Adaptive Dimension Reduction for Clustering High Dimensional Data

Semi-Supervised Clustering via Dynamic Graph Structure Learning

Incomplete Multi-View Clustering Based on Dynamic Dimensionality Reduction Weighted Graph Learning

Graph Clustering with Dynamic Embedding

Graph Embedding with Constraints

Robust Dimensionality Reduction via Low-rank Laplacian Graph Learning

Optimization of Graph Clustering Inspired by Dynamic Belief Systems

Learning Robust Graph for Clustering

Discriminative Embedded Clustering: A Framework for Grouping High-Dimensional Data.

DRGraph: An Efficient Graph Layout Algorithm for Large-scale Graphs by Dimensionality Reduction

Deep Temporal Graph Clustering

A Brief Survey on Representation Learning based Graph Dimensionality Reduction Techniques

Adaptive Sparse Graph Learning Based Dimensionality Reduction for Classification.

A dynamical systems based framework for dimension reduction