Abstract:Recently, Graph Convolutional Networks (GCNs) have shown powerful learning capabilities in graph processing tasks. Computing GCNs with conventional von Neumann architectures usually suffers from limited memory bandwidth due to the irregular memory access. Recent work has proposed Processing-In-Memory (PIM) architectures to overcome the bandwidth bottleneck in Convolutional Neural Networks (CNNs) by performing in-situ matrix-vector multiplication. However, the performance improvement and computation parallelism of existing CNN-oriented PIM architectures is hindered when performing GCNs because of the large scale and sparsity of graphs. To tackle these problems, this paper presents a parallelism enhancement framework for PIM-based GCN architectures. At the software level, we propose a fixed-point quantization method for GCNs, which reduces the PIM computation overhead with little accuracy loss. We also introduce the vertex clustering algorithm to the graph, minimizing the inter-cluster links and realizing cluster-level parallel computing on multi-core systems. At the hardware level, we design a Resistive Random Access Memory (RRAM) based multi-core PIM architecture for GCN, which supports the cluster-level parallelism. Besides, we propose a coarse-grained pipeline dataflow to cover the RRAM write costs and improve the GCN computation throughput. At the software/hardware interface level, we propose a PIM-aware GCN mapping strategy to achieve the optimal tradeoff between resource utilization and computation performance. We also propose edge dropping methods to reduce the inter-core communications with little accuracy loss. We evaluate our framework on typical datasets with multiple widely-used GCN models. Experimental results show that the proposed framework achieves $698\times, 89\times$ , and $41\times$ speedup with $7108\times,255\times$ , and $31\times$ energy efficiency enhancement compared with CPUs, GPUs, and ASICs, respectively.

Rubik: A Hierarchical Architecture for Efficient Graph Neural Network Training

DyGA: A Hardware-Efficient Accelerator with Traffic-Aware Dynamic Scheduling for Graph Convolutional Networks.

Cached Operator Reordering: A Unified View for Fast GNN Training

Accurate, Efficient and Scalable Graph Embedding

An Efficient GCN Accelerator Based on Workload Reorganization and Feature Reduction

HyGCN: A GCN Accelerator with Hybrid Architecture

EnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networks

The Card Shuffling Hypotheses: Building a Time and Memory Efficient Graph Convolutional Network

GDR-HGNN: A Heterogeneous Graph Neural Networks Accelerator Frontend with Graph Decoupling and Recoupling

DyGCN: Efficient Dynamic Graph Embedding With Graph Convolutional Network

Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture

Exploiting Parallelism with Vertex-Clustering in Processing-In-Memory-based GCN Accelerators

FreshGNN: Reducing Memory Access via Stable Historical Embeddings for Graph Neural Network Training

A Task-Adaptive In-Situ ReRAM Computing for Graph Convolutional Networks

Efficient Message Passing Architecture for GCN Training on HBM-based FPGAs with Orthogonal Topology On-Chip Networks

GCNear: A Hybrid Architecture for Efficient GCN Training with Near-Memory Processing

SPA-GCN: Efficient and Flexible GCN Accelerator with an Application for Graph Similarity Computation

GCoD: Graph Convolutional Network Acceleration via Dedicated Algorithm and Accelerator Co-Design

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

DyGCN: Dynamic Graph Embedding with Graph Convolutional Network

GShuttle: Optimizing Memory Access Efficiency for Graph Convolutional Neural Network Accelerators