Abstract:Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and generalization, especially in heterogeneous Industrial Internet of Things (IIoT) environments. In this paper, we develop a lifelong learning architecture that can integrate artificial intelligence (AI) algorithms into the heterogeneous IIoT network for efficient data transmission. Based on this, we propose an intelligent resource allocation algorithm based on dual-attention DRL (DADR) for forwarding node selection and channel access slot allocation in a specific network environment. The proposed DADR algorithm combines the advantages of multi-dimension convolutional attention and multi-head self-attention mechanisms. It can provide local- and global-feature fusion capabilities for distributed nodes while maximizing the performance of data transmission. Furthermore, we present a lifelong federated meta reinforcement learning (LFMRL) that can effectively utilize prior knowledge and enable the DRL agent quickly adapt to a new environment. Specifically, LFMRL adopts a federated meta learning-based knowledge fusion algorithm to fuse the knowledge of learned DADR algorithms and iteratively update the shared model, thereby improving the scalability and generalization of the shared model in heterogeneous IIoT environments. In addition, a simple and efficient knowledge transfer mechanism is enabled to accelerate the DRL model convergence by transferring the knowledge of the shared model to the new environment. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of energy efficiency, data transmission reliability, and network stability. Compared to DADR and FedAvg algorithms, LFMRL algorithm can further reduce the energy consumption, training time, and average forwarding node switching times, while improving packet delivery rate to 99.2%.

Deep Reinforcement Learning for Resource Demand Prediction and Virtual Function Network Migration in Digital Twin Network

Distributed Deep Reinforcement Learning for Resource Allocation in Digital Twin Networks

Deep Reinforcement Learning (DRL)-based Resource Management in Software-Defined and Virtualized Vehicular Ad Hoc Networks

Digital Twin Assisted VNF Mapping and Scheduling in SDN/NFV-Enabled Industrial IoT

Digital Twin-Enhanced Deep Reinforcement Learning for Resource Management in Networks Slicing

Resource Efficient Asynchronous Federated Learning for Digital Twin Empowered IoT Network

Collective reinforcement learning based resource allocation for digital twin service in 6G networks

DRL-D: Revenue-Aware Online Service Function Chain Deployment Via Deep Reinforcement Learning

Digital Twin-Empowered Network Planning for Multi-Tier Computing

Network Resource Allocation Strategy Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

Toward Enhanced Reinforcement Learning-Based Resource Management via Digital Twin: Opportunities, Applications, and Challenges

Digital Twin and Meta RL Empowered Fast-Adaptation of Joint User Scheduling and Task Offloading for Mobile Industrial IoT

RADiT: Resource Allocation in Digital Twin-Driven UAV-aided Internet of Vehicle Networks

Predictive Task Migration Modeling in Software Defined Vehicular Networks

Deep Reinforcement Learning for Stochastic Computation Offloading in Digital Twin Networks

Towards an energy-efficient Data Center Network based on deep reinforcement learning

Virtualized Network Function Forwarding Graph Placing in SDN and NFV-Enabled IoT Networks: A Graph Neural Network Assisted Deep Reinforcement Learning Method

Multi‐dimensional resource management with deep deterministic policy gradient for digital twin‐enabled Industrial Internet of Things in 6 generation

Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT

Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning