Abstract:Virtualized Radio Access Networks (vRANs) are fully configurable and can be implemented at a low cost over commodity platforms to enable network management flexibility. In this paper, a novel vRAN reconfiguration problem is formulated to jointly reconfigure the functional splits of the base stations (BSs), locations of the virtualized central units (vCUs) and distributed units (vDUs), their resources, and the routing for each BS data flow. The objective is to minimize the long-term total network operation cost while adapting to the varying traffic demands and resource availability. Testbed measurements are performed to study the relationship between the traffic demands and computing resources, which reveals high variance and depends on the platform and its load. Consequently, finding the perfect model of the underlying system is non-trivial. Therefore, to solve the proposed problem, a deep reinforcement learning (RL)-based framework is proposed and developed using model-free RL approaches. Moreover, the problem consists of multiple BSs sharing the same resources, which results in a multi-dimensional discrete action space and leads to a combinatorial number of possible actions. To overcome this curse of dimensionality, action branching architecture, which is an action decomposition method with a shared decision module followed by neural network is combined with Dueling Double Deep Q-network (D3QN) algorithm. Simulations are carried out using an O-RAN compliant model and real traces of the testbed. Our numerical results show that the proposed framework successfully learns the optimal policy that adaptively selects the vRAN configurations, where its learning convergence can be further expedited through transfer learning even in different vRAN systems. It offers significant cost savings by up to 59\% of a static benchmark, 35\% of DDPG with discretization, and 76\% of non-branching D3QN.

Energy-Efficient and Interference-Aware VNF Placement with Deep Reinforcement Learning

Fair Virtual Network Function Scheduling with Deep Reinforcement Learning

Energy-delay-aware VNF scheduling: a reinforcement learning approach with hierarchical reward enhancement

Parallel Placement of Virtualized Network Functions via Federated Deep Reinforcement Learning

Automated Function Placement and Online Optimization of Network Functions Virtualization

Online Fault-tolerant VNF Chain Placement: A Deep Reinforcement Learning Approach

Deep Learning based Wireless Resource Allocation with Application to Vehicular Networks

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Distributed Placement And Online Optimization Of Virtual Machines For Network Service Chains

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Priority-awareness VNF migration method based on deep reinforcement learning

GreenNFV: Energy-Efficient Network Function Virtualization with Service Level Agreement Constraints

A Reliable Resource Scheduling for Network Function Virtualization.

Cost Efficient State-Aware Function Placement and Flow Scheduling for NFV Networks.

Reliability-Aware VNF Placement Using a Probability-Based Approach.

Reliable Service Function Chain Deployment Method Based on Deep Reinforcement Learning

Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

A Dynamic Reliability-Aware Service Placement for Network Function Virtualization (NFV)

Virtual network function placement and resource optimization in NFV and edge computing enabled networks.

Deep Reinforcement Learning for Orchestrating Cost-Aware Reconfigurations of vRANs

A Reinforcement Learning-Based Virtual Machine Placement Strategy in Cloud Data Centers