Abstract:Processing-in-memory (PIM) is a promising choice for accelerating deep neural networks (DNNs) featuring high efficiency and low power. However, the rapid upscaling of neural network model sizes poses a crucial challenge for the limited on-chip PIM capacity. When the PIM presumption of "pre-loading DNN weights/parameters only once before repetitive computing" is no longer practical, concurrent writing and computing techniques become necessary for PIM. Conventional methods of naive ping-pong or in~situ concurrent write/compute scheduling for PIM cause low utilization of off-chip memory bandwidth, subsequently offsetting the efficiency gain brought by PIM technology. To address this challenge, we propose an off-chip memory bandwidth centric pipelining strategy, named "generalized ping-pong", to maximize the utilization and performance of PIM accelerators toward large DNN models. The core idea of the proposed generalized ping-pong strategy is to evenly distribute the active time and fully utilize the off-chip memory bandwidth. Based on a programmable and scalable SRAM PIM architecture, we quantitatively analyze and compare the generalized ping-pong with the conventional scheduling strategies of naive ping-pong and in-situ write/compute for PIM. Experiments show that the generalized ping-pong strategy achieves acceleration of over 1.67 times when fully utilizing the off-chip memory bandwidth. When further limiting the off-chip memory bandwidth ranging in 8~256 bytes per clock cycle, the proposed generalized ping-pong strategy accelerates 1.22~7.71 times versus naive ping-pong. The developed PIM accelerator design with the generalized ping-poing strategy is open-sourced at <a class="link-external link-https" href="https://github.com/rw999creator/gpp-pim" rel="external noopener nofollow">this https URL</a>.

WPIA: Accelerating DNN Warm-Up in Web Browsers by Precompiling WebGL Programs

Empowering In-Browser Deep Learning Inference on Edge Through Just-In-Time Kernel Optimization.

Woodpecker-DL: Accelerating Deep Neural Networks via Hardware-Aware Multifaceted Optimizations

FusionRender: Harnessing WebGPU's Power for Enhanced Graphics Performance on Web Browsers

Empowering In-Browser Deep Learning Inference on Edge Devices with Just-in-Time Kernel Optimizations

Parallelizing DNN Inference in Mobile Web Browsers on Heterogeneous Hardware

Anatomizing Deep Learning Inference in Web Browsers

Accelerating Page Loads via Streamlining JavaScript Engine for Distributed Learning

Accelerating Neural Network Training with Processing-in-Memory GPU

Accelerating Image-Domain-Warping Virtual View Synthesis on GPGPU

Generalized Ping-Pong: Off-Chip Memory Bandwidth Centric Pipelining Strategy for Processing-In-Memory Accelerators

QWEB: High-Performance Event-Driven Web Architecture with QAT Acceleration.

Accelerating Adaptive IDW Interpolation Algorithm on a Single GPU

Accelerating Graph Convolutional Networks Through a PIM-Accelerated Approach

Pie: A Pipeline Energy-Efficient Accelerator for Inference Process in Deep Neural Networks

Potamoi: Accelerating Neural Rendering via a Unified Streaming Architecture

A Performance Analysis Framework for Exploiting GPU Microarchitectural Capability.

Moving Deep Learning into Web Browser: How Far Can We Go?

WELDER: Scheduling Deep Learning Memory Access Via Tile-graph

Accelerating Bandwidth-Bound Deep Learning Inference with Main-Memory Accelerators

Performance Evaluation and Optimization of HBM-Enabled GPU for Data-Intensive Applications.