Abstract:Machine learning applied to architecture design presents a promising opportunity with broad applications. Recent deep reinforcement learning (DRL) techniques, in particular, enable efficient exploration in vast design spaces where conventional design strategies may be inadequate. This paper proposes a novel deep reinforcement framework, taking routerless networks-on-chip (NoC) as an evaluation case study. The new framework successfully resolves problems with prior design approaches being either unreliable due to random searches or inflexible due to severe design space restrictions. The framework learns (near-)optimal loop placement for routerless NoCs with various design constraints. A deep neural network is developed using parallel threads that efficiently explore the immense routerless NoC design space with a Monte Carlo search tree. Experimental results show that, compared with conventional mesh, the proposed deep reinforcement learning (DRL) routerless design achieves a 3.25x increase in throughput, 1.6x reduction in packet latency, and 5x reduction in power. Compared with the state-of-the-art routerless NoC, DRL achieves a 1.47x increase in throughput, 1.18x reduction in packet latency, and 1.14x reduction in average hop count albeit with slightly more power overhead.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively explore and optimize the design space of routerless Network - on - Chip (NoC) in Network - on - Chip design. Specifically, existing design methods are either unreliable due to random search or lack flexibility due to severe design space limitations. These problems lead to low design efficiency, inability to fully utilize the interconnection resources provided by modern semiconductor processes, and thus affect network performance, such as throughput, latency, and power consumption. To solve the above problems, the paper proposes a new framework based on deep reinforcement learning, specifically for the design of routerless NoC. This framework can learn (near - ) optimal loop layout strategies under different design constraints, thereby improving network performance. By using deep neural networks and Monte Carlo Tree Search (MCTS), this framework can efficiently explore and generate training data in the huge design space, and then guide the subsequent search process. Experimental results show that compared with the traditional mesh network, the proposed deep reinforcement learning design has significant improvements in throughput, packet latency, and power consumption. Compared with the state - of - the - art routerless NoC designs, this framework also shows advantages in multiple performance indicators, although there is a slight increase in power consumption. In summary, the main contributions of this paper are: 1. Identify the basic problems of applying deep reinforcement learning to routerless NoC design. 2. Propose and implement an innovative deep reinforcement learning framework for space exploration in routerless NoC design under multiple design constraints. 3. Conduct detailed cycle - level architecture simulations and circuit - level implementations to evaluate the design scheme. 4. Discuss the wide applicability of the proposed framework and give several possible application examples.

Optimizing Routerless Network-on-Chip Designs: An Innovative Learning-Based Framework

DRLAR: A Deep Reinforcement Learning-Based Adaptive Routing Framework for Network-on-chips

LARE: A Linear Approximate Reinforcement Learning Based Adaptive Routing for Network-on-Chips

Reinforcement Learning based Interconnection Routing for Adaptive Traffic Optimization

Deep Reinforcement Learning Enabled Self-Configurable Networks-on-Chip for High-Performance and Energy-Efficient Computing Systems

A Novel Reinforcement Learning Framework for Adaptive Routing in Network-on-Chips

Design and Optimization of a Fault-Tolerant Deflection Router for Networks-on-Chip

Self-optimizing Two-layer Network-on-Chip Based on Dominant Network-Flow Adaption

Machine Learning Enabled Solutions for Design and Optimization Challenges in Networks-on-Chip based Multi/Many-Core Architectures

A low overhead load balancing router for network-on-chip

A Reinforcement Learning Framework with Region-Awareness and Shared Path Experience for Efficient Routing in Networks-on-Chip

Design of a High-Throughput NoC Router with Neighbor Flow Regulation

Design And Analysis Of On-Chip Router

A Traffic-Aware Adaptive Routing Algorithm on A Highly Reconfigurable Network-On-Chip Architecture

A deep learning based latency aware predictive routing model for network‐on‐chip architectures

Lightweight Network-on-chip Router on Research and Design

A Low-Latency Virtual-Channel Router with Optimized Pipeline Stages for On-Chip Network

Ring Road: A Scalable Polar-Coordinate-based 2D Network-on-Chip Architecture

A Tightly Coupled Network-on-Chip Router Architecture

Performance Enhancement of Routers in Networks-on-Chip Using Dynamic Virtual Channels Allocation

Routing Optimization With Deep Reinforcement Learning in Knowledge Defined Networking