Abstract:3D many-core NoCs are emerging architectures for future high-performance single chips due to its integration of many processor cores and memories by stacking multiple layers. In such architecture, because processor cores and memories reside in different locations (center, corner, edge, etc.), memory accesses behave differently due to their different communication distances, and the performance (latency) gap of different memory accesses becomes larger as the network size is scaled up. This phenomenon may lead to very high latencies suffered from by some memory accesses, thus degrading the system performance. To achieve high performance, it is crucial to reduce the number of memory accesses with very high latencies. However, this should be done with care since shortening the latency of one memory access can worsen the latency of another as a result of shared network resources. Therefore, the goal should focus on narrowing the latency difference of memory accesses. In the paper, we address the goal by proposing to prioritize the memory access packets based on predicting the round-trip routing latencies of memory accesses. The communication distance and the number of the occupied items in the buffers in the remaining routing path are used to predict the round-trip latency of a memory access. The predicted round-trip routing latency is used as the base to arbitrate the memory access packets so that the memory access with potential high latency can be transferred as early and fast as possible, thus equalizing the memory access latencies as much as possible. Experiments with varied network sizes and packet injection rates prove that our approach can achieve the goal of memory access equalization and outperforms the classic round-robin arbitration in terms of maximum latency, average latency, and LSD1. In the experiments, the maximum improvement of the maximum latency, the average latency and the LSD are 80%, 14%, and 45% respectively.

Regional Cache Organization for NoC Based Many-Core Processors

Network caching for Chip Multiprocessors

Distributed Memory Management Units Architecture for NoC-based CMPs

CCNoC: Cache-Coherent Network on Chip for Chip Multiprocessors.

Group-caching for NoC Based Multicore Cache Coherent Systems.

Revisiting the Cache Effect on Multicore Multithreaded Network Processors

Reconfigurable Mpb Combined with Cache Coherence Protocol in Many-Core

Light Weight Hierarchical Cache Coherence In Noc-Based Multicore Processors

Hierarchical Cache Directory for CMP.

Distributed Shared Memory Architecture for NoC-Based Multi-Processor

Cooperative Mechanism of Local Memory and Cache in Network Processors

An optimized multicore cache coherence design for exploiting communication locality.

A Spatial and Temporal Locality-Aware Adaptive Cache Design with Network Optimization for Tiled Many-Core Architectures.

Network Victim Cache: Leveraging Network-on-Chip for Managing Shared Caches in Chip Multiprocessors

Fast Hierarchical Cache Directory: A Scalable Cache Organization for Large-Scale CMP

Online Cache Modeling for Commodity Multicore Processors

Achieving Memory Access Equalization Via Round-Trip Routing Latency Prediction in 3D Many-Core NoCs

Grouping Cores for Chip Multiprocessors Optimization

Network Main Memory Architecture for NoC-Based Chips

Set-Granular Regional Distributed Cooperative Caching.

CODA: Enabling Co-location of Computation and Data for Near-Data Processing