Abstract:The Graph Convolutional Network (GCN) model and its variants are powerful graph embedding tools for facilitating classification and clustering on graphs. However, a major challenge is to reduce the complexity of layered GCNs and make them parallelizable and scalable on very large graphs -- state-of the art techniques are unable to achieve scalability without losing accuracy and efficiency. In this paper, we propose novel parallelization techniques for graph sampling-based GCNs that achieve superior scalable performance on very large graphs without compromising accuracy. Specifically, our GCN guarantees work-efficient training and produces order of magnitude savings in computation and communication. To scale GCN training on tightly-coupled shared memory systems, we develop parallelization strategies for the key steps in training: For the graph sampling step, we exploit parallelism within and across multiple sampling instances, and devise an efficient data structure for concurrent accesses that provides theoretical guarantee of near-linear speedup with number of processing units. For the feature propagation step within the sampled graph, we improve cache utilization and reduce DRAM communication by data partitioning. We prove that our partitioning strategy is a 2-approximation for minimizing the communication time compared to the optimal strategy. We demonstrate that our parallel graph embedding outperforms state-of-the-art methods in scalability (with respect to number of processors, graph size and GCN model size), efficiency and accuracy on several large datasets. On a 40-core Xeon platform, our parallel training achieves $64\times$ speedup (with AVX) in the sampling step and $25\times$ speedup in the feature propagation step, compared to the serial implementation, resulting in a net speedup of $21\times$.

Fast Hyper-walk Gridded Convolution on Graph

Variational Gridded Graph Convolution Network for Node Classification

Adaptive Sampling Towards Fast Graph Representation Learning

Deepwalk-aware graph convolutional networks

Representation Learning of Enhanced Graphs Using Random Walk Graph Convolutional Network

Hypergraph Convolutional Networks via Equivalency between Hypergraphs and Undirected Graphs

Revisiting 2D Convolutional Neural Networks for Graph-based Applications

DeepHGCN: Toward Deeper Hyperbolic Graph Convolutional Networks

Graph Sampling with Fast Random Walker on HBM-enabled FPGA Accelerators.

PCGCN: Partition-Centric Processing for Accelerating Graph Convolutional Network

Random Walks on Huge Graphs at Cache Efficiency

Adaptive frequency-based fully hyperbolic graph neural networks

DyGCN: Efficient Dynamic Graph Embedding With Graph Convolutional Network

A Hyperbolic-to-Hyperbolic Graph Convolutional Network

Towards Efficient Graph Convolutional Networks for Point Cloud Handling

Accurate, Efficient and Scalable Graph Embedding

DiffGCN: Graph Convolutional Networks via Differential Operators and Algebraic Multigrid Pooling

High-dimensional graphs convolution for quantum walks photonic applications

Permutohedral-GCN: Graph Convolutional Networks with Global Attention

Complex Exponential Graph Convolutional Networks.