Abstract:The rapid advancement of embedded multicore and many-core systems has revolutionized computing, enabling the development of high-performance, energy-efficient solutions for a wide range of applications. As models scale up in size, data movement is increasingly the bottleneck to performance. This movement of data can exist between processor and memory, or between cores and chips. This paper investigates the impact of bottleneck size, in terms of inter-chip data traffic, on the performance of deep learning models in embedded multicore and many-core systems. We conduct a systematic analysis of the relationship between bottleneck size, computational resource utilization, and model accuracy. We apply a hardware-software co-design methodology where data bottlenecks are replaced with extremely narrow layers to reduce the amount of data traffic. In effect, time-multiplexing of signals is replaced by learnable embeddings that reduce the demands on chip IOs. Our experiments on the CIFAR100 dataset demonstrate that the classification accuracy generally decreases as the bottleneck ratio increases, with shallower models experiencing a more significant drop compared to deeper models. Hardware-side evaluation reveals that higher bottleneck ratios lead to substantial reductions in data transfer volume across the layers of the neural network. Through this research, we can determine the trade-off between data transfer volume and model performance, enabling the identification of a balanced point that achieves good performance while minimizing data transfer volume. This characteristic allows for the development of efficient models that are well-suited for resource-constrained environments.

Reducing Data Motion to Accelerate the Training of Deep Neural Networks

Reducing Data Bottlenecks in Distributed, Heterogeneous Neural Networks

AccEPT: an Acceleration Scheme for Speeding Up Edge Pipeline-parallel Training

Maintaining Performance with Less Data

Optimizing Network Performance for Distributed DNN Training on GPU Clusters: ImageNet/AlexNet Training in 1.5 Minutes

NetReduce: RDMA-Compatible In-Network Reduction for Distributed DNN Training Acceleration

Accelerating Data Loading in Deep Neural Network Training

Efficient and Robust Parallel DNN Training through Model Parallelism on Multi-GPU Platform

Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks

CATERPILLAR: Coarse Grain Reconfigurable Architecture for Accelerating the Training of Deep Neural Networks

RedSync: Reducing Synchronization Bandwidth for Distributed Deep Learning Training System

STANNIS: Low-Power Acceleration of Deep Neural Network Training Using Computational Storage

DNN Training Acceleration Via Exploring GPGPU Friendly Sparsity

Accelerating Deep Neural Networks by Combining Block-Circulant Matrices and Low-Precision Weights

A Unified Architecture for Accelerating Distributed DNN Training in Heterogeneous GPU/CPU Clusters

ACCELERATING THE TRAINING OF ARTIFICIAL NEURAL NETWORKS USING DATA PARALLELIZATION

RedSync : Reducing Synchronization Traffic for Distributed Deep Learning

Automatic Model Redundancy Reduction for Fast Back-Propagation for Deep Neural Networks in Speech Recognition

Accelerating neural network training with distributed asynchronous and selective optimization (DASO)

Near-Lossless Gradient Compression for Data-Parallel Distributed DNN Training

Layer-Wise Training To Create Efficient Convolutional Neural Networks