Abstract:With an ambitious increase in the number of Internet of Things (IoT) terminals, IoT networks face a huge challenge which is providing diverse and complex network services with different requirements on a common infrastructure. To solve this challenge, Software Defined Network (SDN) and Network Function Virtualization (NFV) are adopted to build next-generation IoT networks which are softwarized and virtualized. This way, network functions are virtualized as Virtualized Network Functions (VNFs) and a network service consists of a set of VNFs. One of the main challenges for realizing this paradigm is the optimal resource allocation for VNFs. Most existing works assumed that services are represented as Service Function Chains (SFCs) which are chains. However, network services in IoT networks are more complex and diverse, therefore, more appropriate representations are Virtualized Network Function Forwarding Graphs (VNF-FGs) which are Directed Acyclic Graphs (DAGs). Previous works failed to exploit this special graph structure, which makes them sub-optimal or non-applicable for IoT networks. In this paper, we investigate the VNF-FG placing problem in dynamic IoT networks where DAG-represented services arrive and depart. To fully exploit the graph structures of services and handle the complexity of dynamic IoT networks, we combine a novel neural network structure Graph Neural Network (GNN) with Deep Reinforcement Learning (DRL) and propose an efficient algorithm for VNF-FG placing, which is called Kolin. Extensive simulation results suggest that Kolin outperforms the state-of-the-art solutions in terms of system cost, acceptance ratio, and computation complexity.

Applications of Reinforcement Learning in Virtual Network Function Placement: A Survey.

Fair Virtual Network Function Scheduling with Deep Reinforcement Learning

A Survey on Applications of Reinforcement Learning in Spatial Resource Allocation

A novel reinforcement learning algorithm for virtual network embedding

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Survey on Reinforcement Learning Applications in Communication Networks.

Network Resource Allocation Strategy Based on Deep Reinforcement Learning

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Deep Learning based Wireless Resource Allocation with Application to Vehicular Networks

Reinforcement Learning-based Dynamic Service Placement in Vehicular Networks

Reinforcement Learning Assisted Bandwidth Aware Virtual Network Resource Allocation

Energy-Efficient and Interference-Aware VNF Placement with Deep Reinforcement Learning

Deep Reinforcement Learning Approaches to Network Slice Scaling and Placement: A Survey

Reinforcement Learning Meets Wireless Networks: A Layering Perspective

Deep Reinforcement Learning (DRL)-based Resource Management in Software-Defined and Virtualized Vehicular Ad Hoc Networks

A Continuous-Decision Virtual Network Embedding Scheme Relying on Reinforcement Learning.

Efficient Function Placement in Virtual Networks: An Online Learning Approach

VNE Solution for Network Differentiated QoS and Security Requirements: From the Perspective of Deep Reinforcement Learning

A Survey on the Placement of Virtual Network Functions

Virtualized Network Function Forwarding Graph Placing in SDN and NFV-Enabled IoT Networks: A Graph Neural Network Assisted Deep Reinforcement Learning Method

A survey of deep reinforcement learning application in 5G and beyond network slicing and virtualization