Abstract:The recent advances in Artificial Intelligence (AI) achieving “better-than-human” accuracy in a variety of tasks such as image classification and the game of Go have come at the cost of exponential increase in the size of artificial neural networks. This has lead to AI hardware solutions becoming severely memory-bound and scrambling to keep-up with the ever increasing “von Neumann bottleneck”. Processing-in-Memory (PiM) architectures offer an excellent solution to ease the von Neumann bottleneck by embedding compute capabilities inside the memory and reducing the data traffic between the memory and the processor. But PiM accelerators break the standard von Neumann programming model by fusing memory and compute operations together which impedes their integration in the standard computing stack. There is an urgent requirement for system-level solutions to take full advantage of PiM accelerators for end-to-end acceleration of AI applications. This article presents AI-PiM as a solution to bridge this research gap. AI-PiM proposes a hardware, ISA and software co-design methodology which allows integration of PiM accelerators in the RISC-V processor pipeline as functional execution units. AI-PiM also extends the RISC-V ISA with custom instructions which directly target the PiM functional units resulting in their tight integration with the processor. This tight integration is especially important for edge AI devices which need to process both AI and non-AI tasks on the same hardware due to area, power, size and cost constraints. AI-PiM ISA extensions expose the PiM hardware functionality to software programmers allowing efficient mapping of applications to the PiM hardware. AI-PiM adds support for custom ISA extensions to the complete software stack including compiler, assembler, linker, simulator and profiler to ensure programmability and evaluation with popular AI domain-specific languages and frameworks like TensorFlow, PyTorch, MXNet, Keras etc. AI-PiM improves the performance for vector-matrix multiplication (VMM) kernel by 17.63x and provides a mean speed-up of 2.74x for MLPerf Tiny benchmark compared to RV64IMC RISC-V baseline. AI-PiM also speeds-up MLPerf Tiny benchmark inference cycles by 2.45x (average) compared to state-of-the-art Arm Cortex-A72 processor.

SIMDify: Framework for SIMD-Processing with RISC-V Scalar Instruction Set

SIMDive: Approximate SIMD Soft Multiplier-Divider for FPGAs with Tunable Accuracy

Extending the RISC-V ISA for exploring advanced reconfigurable SIMD instructions

An Agile Instruction Set Extension Method Based on the RISC-V Processor

Designing and Implementing a Generator Framework for a SIMD Abstraction Library

A Mixed-Precision RISC-V Processor for Extreme-Edge DNN Inference

A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference

Xuantie-910: A Commercial Multi-Core 12-Stage Pipeline Out-of-Order 64-Bit High Performance RISC-V Processor with Vector Extension : Industrial Product

Single instruction multiple data code auto generation for a very long instruction words digital signal processor in sensor-based systems

A novel multi-core processor for communication applications

SPEED: A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference

Simd Massively Parallel Processing System For Real-Time Image Processing

An Eight-Core 1.44-GHz RISC-V Vector Processor in 16-nm FinFET

A Case for a Flexible Scalar Unit in SIMT Architecture

MIMD Programs Execution Support on SIMD Machines: A Holistic Survey

Optimizing Compiler for Shared-Memory Multiple Simd Architecture

Mixed-precision Neural Networks on RISC-V Cores: ISA extensions for Multi-Pumped Soft SIMD Operations

A Heterogeneous RISC-V Processor for Efficient DNN Application in Smart Sensing System

ANDROMEDA: An FPGA Based RISC-V MPSoC Exploration Framework

Task Mapping and Scheduling on RISC-V MIMD Processor With Vector Accelerator Using Model-Based Parallelization

AI-PiM—Extending the RISC-V processor with Processing-in-Memory functional units for AI inference at the edge of IoT