Abstract:A computing cluster that interconnects multiple compute nodes is used to accelerate distributed reinforcement learning that uses DQN (Deep Q-Network). In distributed reinforcement learning, actor nodes acquire experiences by interacting with a given environment and a learner node optimizes the DQN model. When distributed reinforcement learning is used in practical applications such as robotics, we can assume that actor nodes are located in edge side while the learner node is located in cloud side. In this case, the long-haul communication between them imposes significant communication overheads. However, most prior works simply assume that actors and learner are located closely, and do not take the overheads into account. In this paper, we focus on the practical environment where the actors and learner are located remotely, and they interact via a buffer node that collects information from multiple actor nodes. We implement a prototype system in which the buffer and learner nodes are connected via a 25GbE (Gigabit Ethernet) switch and a 10km optical fiber cable. Although a replay memory functionality is closely associated with the learner side, in this paper we propose to combine the replay memory into the buffer node. In our experiments using the prototype system, the proposed approach is compared with an existing approach in terms of the training efficiency (i.e., training loss) and the transfer efficiency over the long-haul communication (i.e., average priority of transferred experiences). As a result, the training loss of the proposed approach is reduced to 26% of the existing approach, and the average priority is 3.92 times higher than the existing approach after the training loss is converged. These results demonstrate that the proposed approach can improve the training/communication efficiency compared with the existing approach in a practical system that imposes long-haul communication between the actors and learner.

Automatically Reconfigurable Optical Network for HPC System Based on Deep Reinforcement Learning

Optical Circuit Switching Enabled Reconfigurable HPC Network for Traffic Pattern

ODRAD: An optical wireless DCN dynamic-bandwidth reconfiguration with AWGR and deep reinforcement learning

Hierarchical and Reconfigurable Optical/electrical Interconnection Network for High-Performance Computing

Reconfiguring Multicast Sessions in Elastic Optical Networks Adaptively with Graph-Aware Deep Reinforcement Learning

Accelerate Distributed Deep Learning with a Fast Reconfigurable Optical Network

Study on Topology Reconfiguration of Optical Interconnection Network in Data Center

Network Aware Compute and Memory Allocation in Optically Composable Data Centres with Deep Reinforcement Learning and Graph Neural Networks

A Multi-Task-Learning-Based Transfer Deep Reinforcement Learning Design for Autonomic Optical Networks

Reconfigurable Network Topology Based on Deep Reinforcement Learning in Software-Defined Data-Center Networks

Topology-Adaptive Reconfigurable Network with Optical Circuit Switch for HPC Systems

Traffic Pattern Adaptive Hybrid Electrical and Optical Switching Network for HPC System

Machine-learning-aided cognitive reconfiguration for flexible-bandwidth HPC and data center networks [Invited]

RGAIA: a Reconfigurable AWGR Based Optical Data Center Network.

Reconfigurable Topology Design with Deep Reinforcement Learning in Satellite Optical Network

SL-Hyper-FleX: a cognitive and flexible-bandwidth optical datacom network by self-supervised learning [Invited]

NegotiaToR: Towards A Simple Yet Effective On-demand Reconfigurable Datacenter Network

Experimental Assessments of a AWGR Based Optical Switching Network for Distributed Reinforcement Learning Training

Reinforcement Learning based Interconnection Routing for Adaptive Traffic Optimization

An Efficient Distributed Reinforcement Learning Architecture for Long-Haul Communication Between Actors and Learner

A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning