Fast C-K-R Partitions of Sparse Graphs

Manor Mendel,Chaya Schwob

DOI: https://doi.org/10.48550/arXiv.0809.1902

2009-09-09

Abstract:We present fast algorithms for constructing probabilistic embeddings and approximate distance oracles in sparse graphs. The main ingredient is a fast algorithm for sampling the probabilistic partitions of Calinescu, Karloff, and Rabani in sparse graphs.

Data Structures and Algorithms

What problem does this paper attempt to address?

Optimal Time Complexity Algorithms for Computing General Random Walk Graph Kernels on Sparse Graphs

Krzysztof Choromanski,Isaac Reid,Arijit Sehanobish,Avinava Dubey

2024-10-15

Abstract:We present the first linear time complexity randomized algorithms for unbiased approximation of the celebrated family of general random walk kernels (RWKs) for sparse graphs. This includes both labelled and unlabelled instances. The previous fastest methods for general RWKs were of cubic time complexity and not applicable to labelled graphs. Our method samples dependent random walks to compute novel graph embeddings in $\mathbb{R}^d$ whose dot product is equal to the true RWK in expectation. It does so without instantiating the direct product graph in memory, meaning we can scale to massive datasets that cannot be stored on a single machine. We derive exponential concentration bounds to prove that our estimator is sharp, and show that the ability to approximate general RWKs (rather than just special cases) unlocks efficient implicit graph kernel learning. Our method is up to $\mathbf{27\times}$ faster than its counterparts for efficient computation on large graphs and scales to graphs $\mathbf{128 \times}$ bigger than largest examples amenable to brute-force computation.

Machine Learning
Heterogeneous Graph Sparsification for Efficient Representation Learning

Chandan Chunduru,Chun Jiang Zhu,Blake Gains,Jinbo Bi

DOI: https://doi.org/10.48550/arXiv.2211.07518

2022-11-15

Abstract:Graph sparsification is a powerful tool to approximate an arbitrary graph and has been used in machine learning over homogeneous graphs. In heterogeneous graphs such as knowledge graphs, however, sparsification has not been systematically exploited to improve efficiency of learning tasks. In this work, we initiate the study on heterogeneous graph sparsification and develop sampling-based algorithms for constructing sparsifiers that are provably sparse and preserve important information in the original graphs. We have performed extensive experiments to confirm that the proposed method can improve time and space complexities of representation learning while achieving comparable, or even better performance in subsequent graph learning tasks based on the learned embedding.

Machine Learning
A Fast Random Sampling Algorithm for Sparsifying Matrices

Sanjeev Arora,Elad Hazan,Satyen Kale

DOI: https://doi.org/10.1007/11830924_26

2006-01-01

Abstract:We describe a simple random-sampling based procedure for producing sparse matrix approximations. Our procedure and analysis are extremely simple: the analysis uses nothing more than the Chernoff-Hoeffding bounds. Despite the simplicity, the approximation is comparable and sometimes better than previous work.Our algorithm computes the sparse matrix approximation in a single pass over the data. Further, most of the entries in the output matrix are quantized, and can be succinctly represented by a bit vector, thus leading to much savings in space.
Shortcut Partitions in Minor-Free Graphs: Steiner Point Removal, Distance Oracles, Tree Covers, and More

Hsien-Chih Chang,Jonathan Conroy,Hung Le,Lazar Milenkovic,Shay Solomon,Cuong Than

2023-08-01

Abstract:The notion of shortcut partition, introduced recently by Chang, Conroy, Le, Milenković, Solomon, and Than [CCLMST23], is a new type of graph partition into low-diameter clusters. Roughly speaking, the shortcut partition guarantees that for every two vertices $u$ and $v$ in the graph, there exists a path between $u$ and $v$ that intersects only a few clusters. They proved that any planar graph admits a shortcut partition and gave several applications, including a construction of tree cover for arbitrary planar graphs with stretch $1+\varepsilon$ and $O(1)$ many trees for any fixed $\varepsilon \in (0,1)$. However, the construction heavily exploits planarity in multiple steps, and is thus inherently limited to planar graphs. In this work, we breach the "planarity barrier" to construct a shortcut partition for $K_r$-minor-free graphs for any $r$. To this end, we take a completely different approach -- our key contribution is a novel deterministic variant of the cop decomposition in minor-free graphs [And86, AGG14]. Our shortcut partition for $K_r$-minor-free graphs yields several direct applications. Most notably, we construct the first optimal distance oracle for $K_r$-minor-free graphs, with $1+\varepsilon$ stretch, linear space, and constant query time for any fixed $\varepsilon \in (0,1)$. The previous best distance oracle [AG06] uses $O(n\log n)$ space and $O(\log n)$ query time, and its construction relies on Robertson-Seymour structural theorem and other sophisticated tools. We also obtain the first tree cover of $O(1)$ size for minor-free graphs with stretch $1+\varepsilon$, while the previous best $(1+\varepsilon)$-tree cover has size $O(\log^2 n)$ [BFN19].

Data Structures and Algorithms
Making Fast Graph-based Algorithms with Graph Metric Embeddings

Andrey Kutuzov,Mohammad Dorgham,Oleksiy Oliynyk,Chris Biemann,Alexander Panchenko

DOI: https://doi.org/10.48550/arXiv.1906.07040

2019-06-17

Abstract:The computation of distance measures between nodes in graphs is inefficient and does not scale to large graphs. We explore dense vector representations as an effective way to approximate the same information: we introduce a simple yet efficient and effective approach for learning graph embeddings. Instead of directly operating on the graph structure, our method takes structural measures of pairwise node similarities into account and learns dense node representations reflecting user-defined graph distance measures, such as <a class="link-external link-http" href="http://e.g.the" rel="external noopener nofollow">this http URL</a> shortest path distance or distance measures that take information beyond the graph structure into account. We demonstrate a speed-up of several orders of magnitude when predicting word similarity by vector operations on our embeddings as opposed to directly computing the respective path-based measures, while outperforming various other graph embeddings on semantic similarity and word sense disambiguation tasks and show evaluations on the WordNet graph and two knowledge base graphs.

Computation and Language
Scalable and Robust Community Detection with Randomized Sketching

Mostafa Rahmani,Andre Beckus,Adel Karimian,George Atia

DOI: https://doi.org/10.1109/TSP.2020.2965818

2022-12-04

Abstract:This article explores and analyzes the unsupervised clustering of large partially observed graphs. We propose a scalable and provable randomized framework for clustering graphs generated from the stochastic block model. The clustering is first applied to a sub-matrix of the graph's adjacency matrix associated with a reduced graph sketch constructed using random sampling. Then, the clusters of the full graph are inferred based on the clusters extracted from the sketch using a correlation-based retrieval step. Uniform random node sampling is shown to improve the computational complexity over clustering of the full graph when the cluster sizes are balanced. A new random degree-based node sampling algorithm is presented which significantly improves upon the performance of the clustering algorithm even when clusters are unbalanced. This framework improves the phase transitions for matrix-decomposition-based clustering with regard to computational complexity and minimum cluster size, which are shown to be nearly dimension-free in the low inter-cluster connectivity regime. A third sampling technique is shown to improve balance by randomly sampling nodes based on spatial distribution. We provide analysis and numerical results using a convex clustering algorithm based on matrix completion.

Social and Information Networks,Machine Learning
Fast and Accurate Network Embeddings via Very Sparse Random Projection

Haochen Chen,Syed Fahad Sultan,Yingtao Tian,Muhao Chen,Steven Skiena

DOI: https://doi.org/10.1145/3357384.3357879

2019-11-03

Abstract:We present FastRP, a scalable and performant algorithm for learning distributed node representations in a graph. FastRP is over 4,000 times faster than state-of-the-art methods such as DeepWalk and node2vec, while achieving comparable or even better performance as evaluated on several real-world networks on various downstream tasks. We observe that most network embedding methods consist of two components: construct a node similarity matrix and then apply dimension reduction techniques to this matrix. We show that the success of these methods should be attributed to the proper construction of this similarity matrix, rather than the dimension reduction method employed. FastRP is proposed as a scalable algorithm for network embeddings. Two key features of FastRP are: 1) it explicitly constructs a node similarity matrix that captures transitive relationships in a graph and normalizes matrix entries based on node degrees; 2) it utilizes very sparse random projection, which is a scalable optimization-free method for dimension reduction. An extra benefit from combining these two design choices is that it allows the iterative computation of node embeddings so that the similarity matrix need not be explicitly constructed, which further speeds up FastRP. FastRP is also advantageous for its ease of implementation, parallelization and hyperparameter tuning. The source code is available at https://github.com/GTmac/FastRP.
Escape times for subgraph detection and graph partitioning

Zachary M. Boyd,Nicolas Fraiman,Jeremy L. Marzuola,Peter J. Mucha,Braxton Osting

DOI: https://doi.org/10.48550/arXiv.2212.12839

2022-12-25

Social and Information Networks

Abstract:We provide a rearrangement based algorithm for fast detection of subgraphs of $k$ vertices with long escape times for directed or undirected networks. Complementing other notions of densest subgraphs and graph cuts, our method is based on the mean hitting time required for a random walker to leave a designated set and hit the complement. We provide a new relaxation of this notion of hitting time on a given subgraph and use that relaxation to construct a fast subgraph detection algorithm and a generalization to $K$-partitioning schemes. Using a modification of the subgraph detector on each component, we propose a graph partitioner that identifies regions where random walks live for comparably large times. Importantly, our method implicitly respects the directed nature of the data for directed graphs while also being applicable to undirected graphs. We apply the partitioning method for community detection to a large class of model and real-world data sets.
Distributed Balanced Partitioning via Linear Embedding

Kevin Aydin,MohammadHossein Bateni,Vahab Mirrokni

DOI: https://doi.org/10.48550/arXiv.1512.02727

2015-12-09

Abstract:Balanced partitioning is often a crucial first step in solving large-scale graph optimization problems, e.g., in some cases, a big graph can be chopped into pieces that fit on one machine to be processed independently before stitching the results together. In other cases, links between different parts may show up in the running time and/or network communications cost. We study a distributed balanced partitioning problem where the goal is to partition the vertices of a given graph into k pieces so as to minimize the total cut size. Our algorithm is composed of a few steps that are easily implementable in distributed computation frameworks. The algorithm first embeds nodes of the graph onto a line, and then processes nodes in a distributed manner guided by the linear embedding order. We examine various ways to find the first embedding, e.g., via a hierarchical clustering or Hilbert curves. Then we apply four different techniques including local swaps, minimum cuts on the boundaries of partitions, as well as contraction and dynamic programming. As our empirical study, we compare the above techniques with each other, and also to previous work in distributed graph algorithms, e.g., a label propagation method, FENNEL and Spinner. We report our results both on a private map graph and several public social networks, and show that our results beat previous distributed algorithms: e.g., compared to the label propagation algorithm, we report an improvement of 15-25% in the cut value. We also observe that our algorithms allow for scalable distributed implementation for any number of partitions. Finally, we apply our techniques for the Google Maps Driving Directions to minimize the number of multi-shard queries with the goal of saving in CPU usage. During live experiments, we observe an ~40% drop in the number of multi-shard queries when comparing our method with a standard geography-based method.

Distributed, Parallel, and Cluster Computing
Approximate Counting of Matchings in Sparse Uniform Hypergraphs

Marek Karpinski,Andrzej Rucinski,Edyta Szymanska

DOI: https://doi.org/10.48550/arXiv.1204.5335

2012-04-24

Abstract:In this paper we give a fully polynomial randomized approximation scheme (FPRAS) for the number of matchings in k-uniform hypergraphs whose intersection graphs contain few claws. Our method gives a generalization of the canonical path method of Jerrum and Sinclair to hypergraphs satisfying a local restriction. Our proof method depends on an application of the Euler tour technique for the canonical paths of the underlying Markov chains. On the other hand, we prove that it is NP-hard to approximate the number of matchings even for the class of k-uniform, 2-regular and linear hypergraphs, for all k >= 6, without the above restriction.

Data Structures and Algorithms,Discrete Mathematics
Distance Reconstruction of Sparse Random Graphs

Paul Bastide

2024-07-24

Abstract:In the distance query model, we are given access to the vertex set of a $n$-vertex graph $G$, and an oracle that takes as input two vertices and returns the distance between these two vertices in $G$. We study how many queries are needed to reconstruct the edge set of $G$ when $G$ is sampled according to the $G(n,p)$ Erdős-Renyi-Gilbert distribution. Our approach applies to a large spectrum of values for $p$ starting slightly above the connectivity threshold: $p \geq \frac{2000 \log n}{n}$. We show that there exists an algorithm that reconstructs $G \sim G(n,p)$ using $O( \Delta^2 n \log n )$ queries in expectation, where $\Delta$ is the expected average degree of $G$. In particular, for $p \in [\frac{2000 \log n}{n}, \frac{\log^2 n}{n}]$ the algorithm uses $O(n \log^5 n)$ queries.

Combinatorics,Data Structures and Algorithms
Approximating sparse graphs: The random overlapping communities model

Samantha Petti,Santosh S. Vempala

DOI: https://doi.org/10.1002/rsa.21081

2022-03-26

Random Structures and Algorithms

Abstract:How can we approximate sparse graphs and sequences of sparse graphs (with unbounded average degree)? We consider convergence in the first k moments of the graph spectrum (equivalent to the numbers of closed k‐walks) appropriately normalized. We introduce a simple random graph model that captures the limiting spectra of many sequences of interest, including the sequence of hypercube graphs. The random overlapping communities (ROC) model is specified by a distribution on pairs (s,q), s∈ℤ+,q∈(0,1]. A graph on n vertices with average degree d is generated by repeatedly picking pairs (s,q) from the distribution, adding an Erdős–Rényi random graph of edge density q on a subset of vertices chosen by including each vertex with probability s/n, and repeating this process so that the expected degree is d. We also show that ROC graphs exhibit an inverse relationship between degree and clustering coefficient, a characteristic of many real‐world networks.

mathematics, applied,computer science, software engineering
Fast Approximate Spectral Clustering for Dynamic Networks

Lionel Martin,Andreas Loukas,Pierre Vandergheynst

DOI: https://doi.org/10.48550/arXiv.1706.03591

2017-06-12

Abstract:Spectral clustering is a widely studied problem, yet its complexity is prohibitive for dynamic graphs of even modest size. We claim that it is possible to reuse information of past cluster assignments to expedite computation. Our approach builds on a recent idea of sidestepping the main bottleneck of spectral clustering, i.e., computing the graph eigenvectors, by using fast Chebyshev graph filtering of random signals. We show that the proposed algorithm achieves clustering assignments with quality approximating that of spectral clustering and that it can yield significant complexity benefits when the graph dynamics are appropriately bounded.

Machine Learning,Spectral Theory
NC Algorithms for Partitioning Sparse Graphs into Induced Forests with an Application

Zhi-Zhong Chen

DOI: https://doi.org/10.1007/BFb0015449

1995-01-01

Abstract:For a simple graph G, let (G)=1+[max (G)/2], where the maximum is taken over all induced subgraphs G and (G) is the minimum degree of G. It is known that the vertex set of G can be partitioned into at most (G) subsets each of which induces a forest. We give a sufficient condition under which an NC algorithm exists for finding such a partition. From this, we obtain NC algorithms for finding such a partition for K 5-free or K 3,3-free graphs (i.e., graphs without a K 5 or K 3,3 minor). These algorithms can be used to obtain efficient NC approximation algorithms of ratio 3 for many NP-hard maximum induced-subgraph problems on K 5-free or K 3,3-free graphs.
Fast Graph Construction Using Auction Algorithm

Jun Wang,Yinglong Xia

DOI: https://doi.org/10.48550/arXiv.1210.4917

2012-10-17

Abstract:In practical machine learning systems, graph based data representation has been widely used in various learning paradigms, ranging from unsupervised clustering to supervised classification. Besides those applications with natural graph or network structure data, such as social network analysis and relational learning, many other applications often involve a critical step in converting data vectors to an adjacency graph. In particular, a sparse subgraph extracted from the original graph is often required due to both theoretic and practical needs. Previous study clearly shows that the performance of different learning algorithms, e.g., clustering and classification, benefits from such sparse subgraphs with balanced node connectivity. However, the existing graph construction methods are either computationally expensive or with unsatisfactory performance. In this paper, we utilize a scalable method called auction algorithm and its parallel extension to recover a sparse yet nearly balanced subgraph with significantly reduced computational cost. Empirical study and comparison with the state-ofart approaches clearly demonstrate the superiority of the proposed method in both efficiency and accuracy.

Machine Learning
Efficient Network Embedding by Approximate Equitable Partitions

Giuseppe Squillace,Mirco Tribastone,Max Tschaikowski,Andrea Vandin

2024-09-16

Abstract:Structural network embedding is a crucial step in enabling effective downstream tasks for complex systems that aims to project a network into a lower-dimensional space while preserving similarities among nodes. We introduce a simple and efficient embedding technique based on approximate variants of equitable partitions. The approximation consists in introducing a user-tunable tolerance parameter relaxing the otherwise strict condition for exact equitable partitions that can be hardly found in real-world networks. We exploit a relationship between equitable partitions and equivalence relations for Markov chains and ordinary differential equations to develop a partition refinement algorithm for computing an approximate equitable partition in polynomial time. We compare our method against state-of-the-art embedding techniques on benchmark networks. We report comparable -- when not superior -- performance for visualization, classification, and regression tasks at a cost between one and three orders of magnitude smaller using a prototype implementation, enabling the embedding of large-scale networks which could not be efficiently handled by most of the competing techniques.

Social and Information Networks,Machine Learning,Systems and Control
A Bayesian Approach To Graph Partitioning

Farshad Noravesh

DOI: https://doi.org/10.48550/arXiv.2204.12927

2022-04-24

Abstract:A new algorithm based on bayesian inference for learning local graph conductance based on Gaussian Process(GP) is given that uses advanced MCMC convergence ideas to create a scalable and fast algorithm for convergence to stationary distribution which is provided to learn the bahavior of conductance when traversing the indirected weighted graph. First metric embedding is used to represent the vertices of the graph. Then, uniform induced conductance is calculated for training points. Finally, in the learning step, a gaussian process is used to approximate the uniform induced conductance. MCMC is used to measure uncertainty of estimated hyper-parameters.

Machine Learning
Approximate Counting of Matchings in Sparse Hypergraphs

Marek Karpinski,Andrzej Rucinski,Edyta Szymanska

DOI: https://doi.org/10.48550/arXiv.1202.5885

2012-04-06

Abstract:In this paper we give a fully polynomial randomized approximation scheme (FPRAS) for the number of all matchings in hypergraphs belonging to a class of sparse, uniform hypergraphs. Our method is based on a generalization of the canonical path method to the case of uniform hypergraphs.

Data Structures and Algorithms,Discrete Mathematics
Scalable Nearest Neighbor Sparse Graph Approximation by Exploiting Graph Structure

Ming Shao,Xindong Wu,Yun Fu

DOI: https://doi.org/10.1109/tbdata.2016.2617883

2016-01-01

IEEE Transactions on Big Data

Abstract:We consider exploiting graph structure for sparse graph approximations. Graphs play increasingly important roles in learning problems: manifold learning, kernel learning, and spectral clustering. Specifically, in this paper, we concentrate on nearest neighbor sparse graphs which are widely adopted in learning problems due to its spatial efficiency. Nonetheless, we raise an even challenging problem...
Learn Low-dimensional Shortest-path Representation of Large-scale and Complex Graphs

Haoyu Wang,Chun Yuan,Lei Li,Jiahui Jin

2023-01-01

Abstract:Estimation of shortest-path (SP) distance lies at the heart of network analysis tasks. Along with the rapid emergence of large-scale and complex graphs, approximate SP-representing algorithms that transform a graph into compact and low-dimensional representations are critical for fast and scalable online analysis. Among different approaches, learning-based representation methods have made a breakthrough both in response time and accuracy. Several competitive works in learning-based methods heuristically leverage truncated random walk and optimization on the arbitrary linkage for SP representation learning. However, they have limitations on both exploration range and distance preservation. We propose in this paper an efficient and interpretable SP representation method called Betweenness Centrality-based Distance Resampling (BCDR). First, we prove that betweenness centrality-based random walk can occupy a wider exploration range of distance due to its awareness of high-order path structures. Second, we leverage distance resampling to simulate random shortest paths from original paths and prove that the optimization on such shortest paths preserves distance relations via implicitly decomposing SP distance-based similarity matrix. BCDR yields an average improvement of 25% accuracy and 25-30% query speed, compared to all existing approximate methods when evaluated on a broad class of real-world and synthetic graphs with diverse sizes and structures.

Fast C-K-R Partitions of Sparse Graphs

Optimal Time Complexity Algorithms for Computing General Random Walk Graph Kernels on Sparse Graphs

Heterogeneous Graph Sparsification for Efficient Representation Learning

A Fast Random Sampling Algorithm for Sparsifying Matrices

Shortcut Partitions in Minor-Free Graphs: Steiner Point Removal, Distance Oracles, Tree Covers, and More

Making Fast Graph-based Algorithms with Graph Metric Embeddings

Scalable and Robust Community Detection with Randomized Sketching

Fast and Accurate Network Embeddings via Very Sparse Random Projection

Escape times for subgraph detection and graph partitioning

Distributed Balanced Partitioning via Linear Embedding

Approximate Counting of Matchings in Sparse Uniform Hypergraphs

Distance Reconstruction of Sparse Random Graphs

Approximating sparse graphs: The random overlapping communities model

Fast Approximate Spectral Clustering for Dynamic Networks

NC Algorithms for Partitioning Sparse Graphs into Induced Forests with an Application

Fast Graph Construction Using Auction Algorithm

Efficient Network Embedding by Approximate Equitable Partitions

A Bayesian Approach To Graph Partitioning

Approximate Counting of Matchings in Sparse Hypergraphs

Scalable Nearest Neighbor Sparse Graph Approximation by Exploiting Graph Structure

Learn Low-dimensional Shortest-path Representation of Large-scale and Complex Graphs