Abstract:To ensure the delivery of high-performance and reliable services, data center networks (DCNs) are often over-provisioned for peak workload and traffic bursts. However, in real-world data centers, network traffic seldom reaches peak capacity of the network, resulting in significant energy waste. Traditional energy conservation approaches either suffer from high computational complexity and low solution quality, or their strategies cannot be dynamically adjusted to accommodate changes in data center network traffic. Deep reinforcement learning (DRL) provides an effective way to deal with these issues. However, most of the existing DRL-based schemes only consider either a continuous action space or a discrete action space, which greatly restricts the optimality of decisions. To solve these problems, this paper proposes a novel DRL-based DCN energy optimization framework, named SmartDCN. Specifically, SmartDCN consists of a traffic prediction module (TPM) and an energy optimization module (EOM). TPM incorporates an improved LSTM model JANET with an attention mechanism providing a high prediction accuracy, while EOM integrates our newly proposed parameterized DRL algorithm, named PAS-DQN, combining with the discrete-continuous hybrid action space. PAS-DQN implements a two-level control mechanism for the network, using TPM to predict future traffic in the data center as input. It is devoted to dynamically aggregating current traffic and makes tradeoffs between energy efficiency, performance, and robustness to optimize the network's power consumption by dynamically calculating the minimum required network subset and turning off the non-involved network devices to achieve power savings. Experimental results show that SmartDCN significantly outperforms the existing state-of-the-art schemes in terms of energy savings under various network conditions.

Deep-RL: Deep Reinforcement Learning for Marking-Aware Via Per-Port in Data Centers

DeepRLB: A deep reinforcement learning‐based load balancing in data center networks

AQMDRL: Automatic Quality of Service Architecture Based on Multistep Deep Reinforcement Learning in Software-Defined Networking

Comparison of Deep Reinforcement Learning Algorithms in Data Center Cooling Management: A Case Study

DRL-PLink: Deep Reinforcement Learning with Private Link Approach for Mix-Flow Scheduling in Software-Defined Data-Center Networks

CMDRL: A Markovian Distributed Rate Limiting Algorithm in Cloud Networks

Parameterized deep reinforcement learning with hybrid action space for energy efficient data center networks

DRL-TAL: Deep Reinforcement Learning-Based Traffic-Aware Load Balancing in Data Center Networks

Towards an energy-efficient Data Center Network based on deep reinforcement learning

Multi-Dimensional Resource Allocation in Distributed Data Centers Using Deep Reinforcement Learning

Deep Reinforcement Learning Based Dynamic Flowlet Switching for DCN

DRL-D: Revenue-Aware Online Service Function Chain Deployment Via Deep Reinforcement Learning

Enabling Scalable Routing in Software-Defined Networks With Deep Reinforcement Learning on Critical Nodes

DECC: Achieving Low Latency in Data Center Networks with Deep Reinforcement Learning

Toward Packet Routing With Fully Distributed Multiagent Deep Reinforcement Learning

Deep Reinforcement Learning for Router Selection in Network with Heavy Traffic

Two-tiered Online Optimization of Region-wide Datacenter Resource Allocation via Deep Reinforcement Learning

Container stacking optimization based on Deep Reinforcement Learning

Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement Learning

Optimizing Deep-Space DTN Congestion Control Via Deep Reinforcement Learning

General multi-agent reinforcement learning integrating heuristic-based delay priority strategy for demand and capacity balancing