Abstract:Network sparsification serves as an effective technique to accelerate Deep Neural Network (DNN) inference. However, existing sparsification techniques often rely on structured sparsity, which yields limited benefits. This is primarily due to the significant memory and computational overhead introduced by numerous sparse storage formats during address generation and gradient updates. Additionally, many of these solutions are tailored solely for the inference phase, neglecting the crucial training phase. In this paper, we introduce STCO, a novel Sparse Tensor Compilation Optimization technique that significantly enhances training efficiency through structured sparse tensor compilation. Central to STCO is the Tensorization-aware Index Entity (TIE) format, which effectively represents structured sparse tensors by eliminating redundant indices and minimizing storage overhead. The TIE format plays a pivotal role in the Address-carry flow (AC flow) pass, which optimizes the data layout at the computational graph level. This pass leverages the TIE format to enhance the efficiency of tensor representations, enabling more compact and efficient sparse tensor storage. Meanwhile, a shape inference pass utilizes the AC flow to derive optimized tensor shapes, further refining the performance of sparse tensor operations. Moreover, the Address-Carry TIE Flow dynamically tracks nonzero addresses, extending the benefits of sparse optimization to both forward and backward propagation. This seamless integration into the training pipeline enables a smooth transition to sparse tensor compilation without significant modifications to existing codebases. To further boost training performance, we implement an operator-level AC flow optimization pass tailored for structured sparse tensors. This pass generates efficient addresses, ensuring minimal computational overhead during sparse tensor operations. The flexibility of STCO allows it to be efficiently integrated into various frameworks or compilers, providing a robust solution for enhancing training efficiency with structured sparse tensors. Experiments demonstrated that STCO achieved impressive speedups of 3.64 ×, 5.43 ×, 4.89 ×, and 3.91 × when compared to state-of-the-art sparse formats on VGG16, ResNet-18, MobileNetV1, and MobileNetV2, respectively. These findings underscore the efficiency and superiority of our proposed approach in leveraging unstructured sparsity for Deep Neural Network inference acceleration.

Late Breaking Results: Fast System Technology Co-Optimization Framework for Emerging Technology Based on Graph Neural Networks

Late Breaking Results: Fast System Technology Co-Optimization Framework for Emerging Technology Based on Graph Neural Networks

Fast Cell Library Characterization for Design Technology Co-Optimization Based on Graph Neural Networks

DaDianNao: A Machine-Learning Supercomputer

A Multi-Level-Optimization Framework for FPGA-Based Cellular Neural Network Implementation.

ECO-GNN: Signoff Power Prediction using Graph Neural Networks with Subgraph Approximation

New-Generation Design-Technology Co-Optimization (DTCO): Machine-Learning Assisted Modeling Framework

CoGNN: An Algorithm-Hardware Co-Design Approach to Accelerate GNN Inference With Minibatch Sampling

Towards Budget-Driven Hardware Optimization for Deep Convolutional Neural Networks Using Stochastic Computing

Benchmarking Artificial Neural Network Models for Design Technology Co-optimization

Real-Time Optimization Design of GAAFETs Based on Machine Learning Method

A Machine Learning Approach for Optimization of Channel Geometry and Source/Drain Doping Profile of Stacked Nanosheet Transistors

A 3D Structure Mapping-Based Efficient Topology Optimization Framework

STCO: Enhancing Training Efficiency Via Structured Sparse Tensor Compilation Optimization

Graph Neural Networks Automated Design and Deployment on Device-Edge Co-Inference Systems

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

An efficient leakage power optimization framework based on reinforcement learning with graph neural network

Variability-Aware Tcad Based Design-Technology Co-Optimization Platform for 7nm Node Nanowire and Beyond

Surpassing Sycamore: Achieving Energetic Superiority Through System-Level Circuit Simulation

COP: A Combinational Optimization Power Budgeting Method for Manycore Systems in Dark Silicon

GTCO: Graph and Tensor Co-Design for Transformer-Based Image Recognition on Tensor Cores