Reconfigurable Network Topology Based on Deep Reinforcement Learning in Software-Defined Data-Center Networks

Wen Yang,Bingli Guo,Yu Shang,Shanguo Huang

DOI: https://doi.org/10.1364/acpc.2020.m4a.196

2020-01-01

Abstract:In this paper, a Deep-Reinforcement Learning (DRL) agent is implemented and evaluated to enable dynamic topology reconfiguration according to traffic fluctuations and proposes to minimize the network delay. (C) 2020 The Author(s)

What problem does this paper attempt to address?

Enabling Scalable Routing in Software-Defined Networks With Deep Reinforcement Learning on Critical Nodes

Penghao Sun,Zehua Guo,Junfei Li,Yang Xu,Julong Lan,Yuxiang Hu

DOI: https://doi.org/10.1109/TNET.2021.3126933

2022-01-01

IEEE/ACM Transactions on Networking

Abstract:Traditional routing schemes usually use fixed models for routing policies and thus are not good at handling complicated and dynamic traffic, leading to performance degradation (e.g., poor quality of service). Emerging Deep Reinforcement Learning (DRL) coupled with Software-Defined Networking (SDN) provides new opportunities to improve network performance with automatic traffic analysis and policy generation. However, existing DRL-based routing solutions usually rely on all node information to make routing decisions for the network and hence are both hard to converge in large networks and vulnerable to topology changes. In this paper, we propose ScaleDeep, a scalable DRL-based routing scheme for SDN, which improves the routing performance and is resilient to topology changes. Essentially, ScaleDeep takes advantage of partial control on network nodes and DRL. We select a set of critical nodes from a network as driver nodes, which can simulate the entire network operation, based on the control theory. By observing the traffic variation on the driver nodes, DRL dynamically adjusts some link weights for a weighted shortest path algorithm to change the routing paths and improve the routing performance. Limiting the control on driver nodes improves the convergence ability of DRL and reduces the dependency of the DRL agent on the fixed network topology. To validate the performance of ScaleDeep, we conduct packet-level simulations on different topologies. The results show that ScaleDeep outperforms existing DRL-based schemes by reducing the average flow completion time by up to 36% and exhibiting better robustness against minor topology changes.
Reconfigurable Topology Design with Deep Reinforcement Learning in Satellite Optical Network

Yun Xiao,Bingli Guo,Hai Yang,Kuan Yan,Shanguo Huang

DOI: https://doi.org/10.1109/ACP/POEM59049.2023.10368668

2023-01-01

Abstract:To minimize delay of satellite optical network, this paper proposes an enhanced deep deterministic policy gradient scheme for reconfigurable topology design with distinct traffic intensity. Simulation results show that average delay is reduced about 28%.
A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization

Giorgio Stampa,Marta Arias,David Sanchez-Charles,Victor Muntes-Mulero,Albert Cabellos

DOI: https://doi.org/10.48550/arXiv.1709.07080

2017-09-21

Abstract:In this paper we design and evaluate a Deep-Reinforcement Learning agent that optimizes routing. Our agent adapts automatically to current traffic conditions and proposes tailored configurations that attempt to minimize the network delay. Experiments show very promising performance. Moreover, this approach provides important operational advantages with respect to traditional optimization algorithms.

Networking and Internet Architecture,Artificial Intelligence
Automatically Reconfigurable Optical Network for HPC System Based on Deep Reinforcement Learning

Yu Shang,Xingwen Guo,Bingli Guo,Haixi Wang,Jie Xiao,Shanguo Huang

DOI: https://doi.org/10.1109/acp55869.2022.10088485

2022-01-01

Abstract:High-density optical interconnects enable high-bandwidth, low-latency and energy-efficient communication. Also, in order to enhance the adaptability of High Performance Computing (HPC) networks for traffic variation, optical networking reconfiguration capability is a promising alternative to traditional fixed interconnection architecture. We propose a reconfigurable optical network for HPC systems that can automatically switch between different topologies controlled by a Deep Reinforcement Learning (DRL) agent. By varying the topology of our network, we evaluate it using an Advantage Actor-Critic (A2C) and a Deep Deterministic Policy Gradient (DDPG) algorithm-based scheme. Results show that our prototype achieves up to 1.7x performance improvement.
DROM: Optimizing the Routing in Software-Defined Networks with Deep Reinforcement Learning

Changhe Yu,Julong Lan,Zehua Guo,Yuxiang Hu

DOI: https://doi.org/10.1109/access.2018.2877686

IF: 3.9

2018-01-01

IEEE Access

Abstract:This paper proposes DROM, a deep reinforcement learning mechanism for Software-Defined Networks (SDN) to achieve a universal and customizable routing optimization. DROM simplifies the network operation and maintenance by improving the network performance, such as delay and throughput, with a black-box optimization in continuous time. We evaluate the DROM with experiments. The experimental results show that DROM has the good convergence and effectiveness and provides better routing configurations than existing solutions to improve the network performance, such as reducing the delay and improving the throughput.
A Deep Reinforcement Learning Approach for Adaptive Traffic Routing in Next-gen Networks

Akshita Abrol, Purnima Murali Mohan, Tram Truong-Huu

2024-02-07

Abstract:Next-gen networks require significant evolution of management to enable automation and adaptively adjust network configuration based on traffic dynamics. The advent of software-defined networking (SDN) and programmable switches enables flexibility and programmability. However, traditional techniques that decide traffic policies are usually based on hand-crafted programming optimization and heuristic algorithms. These techniques make non-realistic assumptions, e.g., considering static network load and topology, to obtain tractable solutions, which are inadequate for next-gen networks. In this paper, we design and develop a deep reinforcement learning (DRL) approach for adaptive traffic routing. We design a deep graph convolutional neural network (DGCNN) integrated into the DRL framework to learn the traffic behavior from not only the network topology but also link and node attributes. We adopt the Deep Q-Learning technique to train the DGCNN model in the DRL framework without the need for a labeled training dataset, enabling the framework to quickly adapt to traffic dynamics. The model leverages q-value estimates to select the routing path for every traffic flow request, balancing exploration and exploitation. We perform extensive experiments with various traffic patterns and compare the performance of the proposed approach with the Open Shortest Path First (OSPF) protocol. The experimental results show the effectiveness and adaptiveness of the proposed framework by increasing the network throughput by up to 7.8% and reducing the traffic delay by up to 16.1% compared to OSPF.

Networking and Internet Architecture,Artificial Intelligence
A Deep Reinforcement Learning Approach for Deploying SDN Switches in ISP Networks from the Perspective of Traffic Engineering

Yingya Guo,Jianshan Chen,Kai Huang,Jianping Wu

DOI: https://doi.org/10.1109/hpsr54439.2022.9831203

2022-01-01

Abstract:Nowadays, a hybrid Software-Defined Network (hybrid SDN), which combines the robustness of the distributed network and the flexibility of the centralized network, is a prevailing network architecture. The performance of Traffic Engineering (TE) in a hybrid SDN is largely influenced by the location of SDN switches. To derive the optimal location of SDN switches, the previous SDN switches deployment strategies mainly focused on manually designing heuristics to search for the SDN deployment sequence and only take a static Traffic Matrix (TM) into consideration. However, the manually-designed heuristics cannot capture the complex intrinsic relations among the location of SDN switches, network topology, and dynamic traffic demands. In addition, the SDN deployment strategy optimized under a single TM exhibits poor performance in a dynamic environment. Therefore, in this paper, we propose a Deep Reinforcement Learning (DRL)-based algorithm SEED to intelligently learn the SDN deployment strategy under multiple TMs. Specifically, to capture the dynamic traffic information, we first cluster the historical traffic demand matrices for obtaining the representative Traffic Matrices (TM) that can depict the dynamic traffic. Then, to intelligently learn the intrinsic relations between the topology, TMs, and the location of SDN switches, we design a DRL agent under multiple TMs by interacting with the environment in a trial and error manner. The extensive experiments on three real network topologies and traffic demands demonstrate that our proposed SDN switches deployment strategy can better adapt to the dynamic traffic and better improve the TE performance than the other deployment strategies.
Study on Topology Reconfiguration of Optical Interconnection Network in Data Center

YANG Wen,GUO Bing-li,YANG Hong-zhen,WANG Yan-bo,FAN Chao,MENG Ling-yu,HUANG Shan-guo

DOI: https://doi.org/10.13756/j.gtxyj.2022.01.003

2022-01-01

Abstract:Aiming at preventing the network performance deterioration caused by fixed link topology connections in optically switched networks, this paper proposes and validates an interconnection and control mechanism that can automatically optimize and reconfigure network topology according to the network traffic volume. This mechanism combines deep reinforcement learning with software defined network technology. Based on the real-time network performance parameters collected by the software-defined network control plane and the performance feedback of the network simulation system, the continuous training of the topology structure can be realized in the model of deep reinforcement learning. The automatic network topology reconfiguration is achieved with improved network performance. The experimental results show that for a given traffic intensity, the model trained by deep reinforcement learning can learn the topology reconstruction strategy and realize automatic selection. The optimal network topology structure can be achieved in one step, with reduced average network delay and average packet loss.
A Hybrid Deep Reinforcement Learning Routing Method under Dynamic and Complex Traffic with Software Defined Networking

Ziyang Zhang,Lin Guan,Qinggang Meng

DOI: https://doi.org/10.1007/978-3-031-57942-4_19

2024-01-01

Abstract:Software-defined networking (SDN) routing based on reinforcement learning (RL) is a very promising research topic in recent years, achieving better solutions comparing to traditional network routing based on mathematical models. However, with continuous increase of network complexity and scale, the RL methods show a slow convergence speed and insufficient adaptability to network changes. This leads to the major drawbacks of existing RL algorithms in modern large-scale networks, especially with complexity and dynamics features. Therefore, this paper proposed a novel RL method based on pre-trained data called PRLR, a pre-trained reinforcement learning based SDN routing method, which can effectively improve the QoS of SDN routing and improve the convergence speed of reinforcement learning. The experimental results demonstrate that our proposed PRLR outperforms the benchmarking methods in terms of multiple metrics, such as network delays, bandwidth availability, goodput ratio, as well as the convergence efficiency and works well in dynamic routing topologies.
GTD3-NET: A deep reinforcement learning-based routing optimization algorithm for wireless networks

You Lu,Zhe Zhang,Xiao Xu,Lanhui Liu,Qiming Fu,Jianping Chen,Chen Chen

DOI: https://doi.org/10.1007/s12083-024-01851-3

IF: 3.488

2024-12-01

Peer-to-Peer Networking and Applications

Abstract:Topology of Internet of Things (IoT) network topologies frequently undergo dynamic changes. Thus, robust and efficient routing technology is vital for IoT system reliability and availability. Traditional wireless routing, limited by inflexible algorithms, often can't simultaneously meet multiple goals such as low energy consumption and high connectivity. With the rise of intelligent technologies, incorporating machine learning in routing optimization algorithms has become crucial. Traditional machine learning methods like neural networks, reliant on pre-trained models, face challenges adapting to fast-changing wireless network environments. In contrast, deep reinforcement learning (DRL) methods are more adept due to their online learning and decision-making capabilities. This paper presents a new DRL-based routing optimization algorithm for wireless networks, named GTD3-NET. It utilizes a deterministic policy reinforcement learning algorithm (TD3) for a robust routing framework, improving performance and DRL convergence speed. Moreover, it replaces standard neural networks with a graph neural network (GNN) for the DRL Agent, allowing better modeling of network nodes and edges and deeper insights into node relationships and network topology characteristics. The algorithm's efficacy is validated through multiple experiments, showing GTD3-NET's capability to learn optimal routing policies and perform well in new network topologies.

computer science, information systems,telecommunications
Network Topology Optimization via Deep Reinforcement Learning

Zhuoran Li,Xing Wang,Ling Pan,Lin Zhu,Zhendong Wang,Junlan Feng,Chao Deng,Longbo Huang

DOI: https://doi.org/10.48550/arXiv.2204.14133

2022-04-19

Abstract:Topology impacts important network performance metrics, including link utilization, throughput and latency, and is of central importance to network operators. However, due to the combinatorial nature of network topology, it is extremely difficult to obtain an optimal solution, especially since topology planning in networks also often comes with management-specific constraints. As a result, local optimization with hand-tuned heuristic methods from human experts are often adopted in practice. Yet, heuristic methods cannot cover the global topology design space while taking into account constraints, and cannot guarantee to find good solutions. In this paper, we propose a novel deep reinforcement learning (DRL) algorithm, called Advantage Actor Critic-Graph Searching (A2C-GS), for network topology optimization. A2C-GS consists of three novel components, including a verifier to validate the correctness of a generated network topology, a graph neural network (GNN) to efficiently approximate topology rating, and a DRL actor layer to conduct a topology search. A2C-GS can efficiently search over large topology space and output topology with satisfying performance. We conduct a case study based on a real network scenario, and our experimental results demonstrate the superior performance of A2C-GS in terms of both efficiency and performance.

Networking and Internet Architecture,Artificial Intelligence,Machine Learning
Optimizing Deep-Space DTN Congestion Control Via Deep Reinforcement Learning

Lei Yang,Juan A. Fraire,Kanglian Zhao,Ruhai Wang,Wenfeng Li,Hong Yang

DOI: https://doi.org/10.1016/j.comnet.2024.110865

IF: 5.493

2024-01-01

Computer Networks

Abstract:This paper introduces an innovative congestion control mechanism for delay/disruption-tolerant networking (DTN) within deep-space communication systems, leveraging the nuanced capabilities of deep reinforcement learning (DRL). This approach significantly departs from traditional methods, addressing the unique challenges of deep-space data transmissions. The proposed DRL-based strategy demonstrates a superior balance of critical factors, including transmission delay, energy efficiency, and data reception integrity. We assess our approach through meticulous simulation and comparison with established benchmark schemes. The findings underscore the mechanism's enhanced performance metrics, positing it as an appealing solution in the evolving landscape of non-terrestrial networking.
Deep Reinforcement Learning for Controller Placement in Software Defined Network

Yiwen Wu,Sipei Zhou,Yunkai Wei,Supeng Leng

DOI: https://doi.org/10.1109/INFOCOMWKSHPS50562.2020.9162977

2020-01-01

Abstract:Controller placement is a critical problem in Software Defined Network (SDN), which has been identified as a potential approach to achieve a more flexible control and management of the network. To achieve an optimal placement solution, the network characters as well as flow fluctuations should be fully considered, making the problem extraordinary complicated. Deep Reinforcement Learning (DRL) has vast potential to obtain suitable results by exploring the solution space, and be adapted to the rapidly fluctuating data flow with the algorithm learning from the feedback generated during exploration. In this paper, we propose a Deep Q-Network (DQN) empowered Dynamic flow Data Driven approach for Controller Placement Problem (D4CPP). D4CPP integrates the historical network data learning into the controller deployment and real-time switch-controller mapping decision, so as to be adapted to the dynamic network environment with flow fluctuations. Specifically, D4CPP takes the flow fluctuation, data latency, and load balance into full consideration, and can reach an optimized balance among these metrics. Extensive simulations show that. D4CPP is efficient in SDN system with dynamic flow fluctuating, and outperforms traditional scheme by 13% in latency and 50% in load balance averagely when the latency and the load balance are assigned with the same weight.
Deep-Reinforcement-Learning-Based Intelligent Routing Strategy for FANETs

Deping Lin,Tao Peng,Peiliang Zuo,Wenbo Wang

DOI: https://doi.org/10.3390/sym14091787

2022-08-29

Symmetry

Abstract:Flying ad hoc networks (FANETs), which are composed of autonomous flying vehicles, constitute an important supplement to satellite networks and terrestrial networks, and they are indispensable for many scenarios including emergency communication. Unfortunately, the routing therein is largely affected by rapid topology changes, frequent disconnection of links, and a high vehicle mobility. In this paper, an intelligent routing strategy based on deep reinforcement learning (DRL) is proposed, which is decentralized and takes into account the status of symmetrical nodes in two hops. In order to perceive the local dynamics of the network as comprehensively as possible, the location, moving speed, load degree, and link quality of the nodes are considered into the setting process of state elements in the method. The nodes can select the neighboring node adaptively according to the Q values calculated by the model obtained through the training of Deep Q-Networks. The simulation and analysis show that the proposed method possesses good convergence characteristics and has obviously better performance compared with several common methods.
Opti-DeepRoute: A Topology-Adaptive Deep Reinforcement Learning Based Service Provisioning Framework for Elastic Optical Network

Zexi Zhou,Rentao Gu,Xiaoya Zhang,Yunxuan Li,Lin Bai,Yuefeng Ji

DOI: https://doi.org/10.1109/infocomwkshps61880.2024.10620793

2024-01-01

Abstract:The satisfaction of AI computing demands relies on the robust computational foundation provided by efficient optical backbone network. Deep reinforcement learning (DRL) methods have been widely adopted recently to address elastic optical network (EON) service provisioning problems with complex spectrum constraints. However, previous frameworks utilizing path-level decision spaces face challenges as path-level features cannot be directly transferred when network topology changes, making it necessary to retrain the agent, hindering the practical deployment of the algorithm. Based on the above observations, this paper proposes a novel hop-by-hop DRL service provisioning framework for EON: Opti-DeepRoute. A node-level feature engineering is employed to construct solutions incrementally through the creation of node sequences. Utilizing the topological and spectral local receptive fields provided by the Graph Attention Network (GAT), the agent acquires local network states and partial solution information to achieve topological adaptability. Preliminary evaluations demonstrate the advantages of Opti-DeepRoute in terms of topological adaptability compared with SOTA DRL approaches.
Enabling efficient routing for traffic engineering in SDN with Deep Reinforcement Learning

Xinglong Pe,Penghao Sun,Yuxiang Hu,Dan Li,Bo Chen,Le Tian

DOI: https://doi.org/10.1016/j.comnet.2024.110220

IF: 5.493

2024-01-31

Computer Networks

Abstract:Traffic Engineering (TE) is applied to optimize network transmission efficiency by managing the routing of complicated traffic. Emerging Deep Reinforcement Learning (DRL) and Software-Defined Networking (SDN) provide flexible ability for traffic management and congestion control. However, existing methods either cannot reroute the network-wide traffic in an accurate way or encounter too much calculation latency due to the inference of mathematical optimization techniques. In this paper, we propose a flexible and link-reconfigurable TE solution called EfficientTE that effectively adjusts the traffic routing based on real-time traffic demands. By analyzing the characteristics of network topology and traffic, EfficientTE selects a few links that are critical for congestion in the network. Then, we propose the idea of virtual capacity that helps the DRL algorithm adjust to different link bandwidths. Based on the traffic demand and topology information collected by the SDN controller, the DRL algorithm is used to dynamically adjust the virtual capacity of the critical links to reshape the network. To ensure network performance with low disturbance, we selectively reroute the top-K critical flows using the weighted k-short shortest path algorithm, while forwarding the major flows with default rerouting. Experiments show that EfficientTE optimizes maximum link utilization and outperforms existing TE solutions by improving the load-balancing performance ratio by at least 6.13%, 18.78%, 16.20%, and 22.81% respectively in four network topologies.

computer science, information systems,telecommunications,engineering, electrical & electronic, hardware & architecture
Traffic Engineering in Hybrid Software Defined Network Via Reinforcement Learning

Yingya Guo,Weipeng Wang,Han Zhang,Wenzhong Guo,Zhiliang Wang,Ying Tian,Xia Yin,Jianping Wu

DOI: https://doi.org/10.1016/j.jnca.2021.103116

IF: 7.574

2021-01-01

Journal of Network and Computer Applications

Abstract:The emergence of Software Defined Network (SDN) provides a centralized and flexible approach to route network flows. Due to the technical and economic challenges in upgrading to a fully SDN-enabled network, hybrid SDN, with a partial deployment of SDN switches in a traditional network, has been a prevailing network architecture. Meanwhile, Traffic Engineering (TE) in the hydbrid SDN has attracted wide attentions from academia and industry. Previous studies on TE in the hybrid SDN are either traffic-oblivious or time-consuming, which causes routing schemes failed in responding to the dynamically-changing traffic rapidly and intelligently. Therefore, in this paper, we propose a Reinforcement Learning (RL) based method, which learns a traffic-splitting agent to address the dynamically-changing traffic and achieve the link load balancing in the hybrid SDN. Specifically, to rapidly and intelligently determine a routing scheme to the new traffic demands, a traffic-splitting agent is designed and learnt offline by exploiting the RL algorithm to establish the direct relationship between traffic demands and traffic-splitting policies. Once the traffic-splitting agent is learnt, the effective traffic-splitting policies, which are used to determine the traffic-splitting ratios on SDN switches, can be generated rapidly. Additionally, to meet the interactive requirements for learning a traffic-splitting agent, a reasonable simulation environment is proposed to be constructed to avoid routing loops when traffic-splitting policies are taken. Extensive evaluations on different topologies and real traffic demands demonstrate that the proposed method achieves the comparable network performance and performs superiorities in rapidly generating the satisfying routing schemes.
Adaptive Configuration with Deep Reinforcement Learning in Software-Defined Time-Sensitive Networking

Mengjie Guo,Guochu Shou,Yaqiong Liu,Yihong Hu

DOI: https://doi.org/10.1109/noms59830.2024.10575223

2024-01-01

Abstract:Time-sensitive networking (TSN) is very appealing to industrial networks due to its support for deterministic transmission based on Ethernet. The implementation of determinism typically demands for precise configuration on each output port of a TSN switch, which is complex and time-consuming. Moreover, many emerging industrial applications bring dynamic scenarios (e.g., in real-time Internet of Things), thus the configurations should change adaptively as application requirements change to provide continued determinism. In this paper, we propose a deep reinforcement learning (DRL) based adaptive configuration scheme in Software-defined time-sensitive networking (SD-TSN). The SD-TSN is a network architecture that integrates the determinism guarantees of TSN and flexible network management of software-defined networking (SDN). Based on the capability of SD-TSN, the proposed configuration scheme exploits DRL to learn from interacting with the environment for adaptive configuration. Experimental results demonstrate the effectiveness of our scheme in dynamic scenarios.
Deep reinforcement learning based multi-level dynamic reconfiguration for urban distribution network: A cloud-edge collaboration architecture

Siyuan Jiang,Hongjun Gao,Xiaohui Wang,Junyong Liu,Kunyu Zuo

DOI: https://doi.org/10.1016/j.gloei.2023.02.001

2023-03-08

Global Energy Interconnection

Abstract:With the construction of the power Internet of Things (IoT), communication between smart devices in urban distribution networks has been gradually moving towards high speed, high compatibility, and low latency, which provides reliable support for reconfiguration optimization in urban distribution networks. Thus, this study proposed a deep reinforcement learning based multi-level dynamic reconfiguration method for urban distribution networks in a cloud-edge collaboration architecture to obtain a real-time optimal multi-level dynamic reconfiguration solution. First, the multi-level dynamic reconfiguration method was discussed, which included feeder-, transformer-, and substation-levels. Subsequently, the multi-agent system was combined with the cloud-edge collaboration architecture to build a deep reinforcement learning model for multi-level dynamic reconfiguration in an urban distribution network. The cloud-edge collaboration architecture can effectively support the multi-agent system to conduct "centralized training and decentralized execution" operation modes and improve the learning efficiency of the model. Thereafter, for a multi-agent system, this study adopted a combination of offline and online learning to endow the model with the ability to realize automatic optimization and updation of the strategy. In the offline learning phase, a Q-learning-based multi-agent conservative Q-learning (MACQL) algorithm was proposed to stabilize the learning results and reduce the risk of the next online learning phase. In the online learning phase, a multi- agent deep deterministic policy gradient (MADDPG) algorithm based on policy gradients was proposed to explore the action space and update the experience pool. Finally, the effectiveness of the proposed method was verified through a simulation analysis of a real-world 445-node system.
AQMDRL: Automatic Quality of Service Architecture Based on Multistep Deep Reinforcement Learning in Software-Defined Networking

Junyan Chen,Cenhuishan Liao,Yong Wang,Lei Jin,Xiaoye Lu,Xiaolan Xie,Rui Yao

DOI: https://doi.org/10.3390/s23010429

2022-12-30

Abstract:Software-defined networking (SDN) has become one of the critical technologies for data center networks, as it can improve network performance from a global perspective using artificial intelligence algorithms. Due to the strong decision-making and generalization ability, deep reinforcement learning (DRL) has been used in SDN intelligent routing and scheduling mechanisms. However, traditional deep reinforcement learning algorithms present the problems of slow convergence rate and instability, resulting in poor network quality of service (QoS) for an extended period before convergence. Aiming at the above problems, we propose an automatic QoS architecture based on multistep DRL (AQMDRL) to optimize the QoS performance of SDN. AQMDRL uses a multistep approach to solve the overestimation and underestimation problems of the deep deterministic policy gradient (DDPG) algorithm. The multistep approach uses the maximum value of the n-step action currently estimated by the neural network instead of the one-step Q-value function, as it reduces the possibility of positive error generated by the Q-value function and can effectively improve convergence stability. In addition, we adapt a prioritized experience sampling based on SumTree binary trees to improve the convergence rate of the multistep DDPG algorithm. Our experiments show that the AQMDRL we proposed significantly improves the convergence performance and effectively reduces the network transmission delay of SDN over existing DRL algorithms.

Reconfigurable Network Topology Based on Deep Reinforcement Learning in Software-Defined Data-Center Networks

Enabling Scalable Routing in Software-Defined Networks With Deep Reinforcement Learning on Critical Nodes

Reconfigurable Topology Design with Deep Reinforcement Learning in Satellite Optical Network

A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization

Automatically Reconfigurable Optical Network for HPC System Based on Deep Reinforcement Learning

DROM: Optimizing the Routing in Software-Defined Networks with Deep Reinforcement Learning

A Deep Reinforcement Learning Approach for Adaptive Traffic Routing in Next-gen Networks

A Deep Reinforcement Learning Approach for Deploying SDN Switches in ISP Networks from the Perspective of Traffic Engineering

Study on Topology Reconfiguration of Optical Interconnection Network in Data Center

A Hybrid Deep Reinforcement Learning Routing Method under Dynamic and Complex Traffic with Software Defined Networking

GTD3-NET: A deep reinforcement learning-based routing optimization algorithm for wireless networks

Network Topology Optimization via Deep Reinforcement Learning

Optimizing Deep-Space DTN Congestion Control Via Deep Reinforcement Learning

Deep Reinforcement Learning for Controller Placement in Software Defined Network

Deep-Reinforcement-Learning-Based Intelligent Routing Strategy for FANETs

Opti-DeepRoute: A Topology-Adaptive Deep Reinforcement Learning Based Service Provisioning Framework for Elastic Optical Network

Enabling efficient routing for traffic engineering in SDN with Deep Reinforcement Learning

Traffic Engineering in Hybrid Software Defined Network Via Reinforcement Learning

Adaptive Configuration with Deep Reinforcement Learning in Software-Defined Time-Sensitive Networking

Deep reinforcement learning based multi-level dynamic reconfiguration for urban distribution network: A cloud-edge collaboration architecture

AQMDRL: Automatic Quality of Service Architecture Based on Multistep Deep Reinforcement Learning in Software-Defined Networking