Abstract:GPGPU applications exploit on-chip scratchpad memory available in the Graphics Processing Units (GPUs) to improve performance. The amount of thread level parallelism present in the GPU is limited by the number of resident threads, which in turn depends on the availability of scratchpad memory in its streaming multiprocessor (SM). Since the scratchpad memory is allocated at thread block granularity, part of the memory may remain unutilized. In this paper, we propose architectural and compiler optimizations to improve the scratchpad utilization. Our approach, Scratchpad Sharing, addresses scratchpad under-utilization by launching additional thread blocks in each SM. These thread blocks use unutilized scratchpad and also share scratchpad with other resident blocks. To improve the performance of scratchpad sharing, we propose Owner Warp First (OWF) scheduling that schedules warps from the additional thread blocks effectively. The performance of this approach, however, is limited by the availability of the shared part of scratchpad. We propose compiler optimizations to improve the availability of shared scratchpad. We describe a scratchpad allocation scheme that helps in allocating scratchpad variables such that shared scratchpad is accessed for short duration. We introduce a new instruction, relssp, that when executed, releases the shared scratchpad. Finally, we describe an analysis for optimal placement of relssp instructions such that shared scratchpad is released as early as possible. We implemented the hardware changes using the GPGPU-Sim simulator and implemented the compiler optimizations in Ocelot framework. We evaluated the effectiveness of our approach on 19 kernels from 3 benchmarks suites: CUDA-SDK, GPGPU-Sim, and Rodinia. The kernels that underutilize scratchpad memory show an average improvement of 19% and maximum improvement of 92.17% compared to the baseline approach.

Well Utilization of Cache-Aware Scratchpad Concerning the Influence of the Whole Embedded System

Efficient utilization of scratch-pad memory for embedded systems

Efficient Scratchpad Memory Management Based on Multi-thread for MPSoC Architecture.

Efficient Utilization of Scratch-Pad Memory Banks

Program Sections Allocation to Scratchpad Memory Based on Frequency Analysis

Designing Scratchpad Memory Architecture with Emerging STT-RAM Memory Technologies

Virtual I/O Based on ScratchPad Memory for Embedded System

Energy-Oriented Dynamic SPM Allocation Based on Time-Slotted Cache Conflict Graph

On-chip memory dynamic optimization for embedded Linux

Low-Power Low-Latency Data Allocation for Hybrid Scratch-Pad Memory

DPA: Demand-Based Partition and Data Allocation for Hybrid On-Chip Memory

Compiler-directed scratchpad memory data transfer optimization for multithreaded applications on a heterogeneous many-core architecture

Scratchpad Memory Assignment in Embedded Multimedia Application

A Study on Cache Mechanism in Heterogeneous Memory System

Scratchpad Sharing in GPUs

SPMTM: A Novel ScratchPad Memory Based Hybrid Nested Transactional Memory Framework

Software Managed Instruction Scratchpad Memory Optimization in Stream Architecture Based on Hot Code Analysis of Kernels.

Statistical Cache Bypassing for Non-Volatile Memory

Study on the Low Power Cache Design Technique

Fusion-Cache: A Refactored Content-Aware Host-Side SSD Cache.

Exploring Data Placement in Racetrack Memory Based Scratchpad Memory.