Abstract:Information freshness is essential to industrial wireless networks (IWNs) and can be quantified by the age-of-information (AoI) metric. This paper addresses an AoI-aware spectrum sharing (AgeS) problem in IWNs, where multiple device-to-device (D2D) links opportunistically access the spectrum to satisfy their AoI constraints while maximizing primal links’ throughput. Particularly, we orchestrate the access of D2D links in a distributed manner. Since distributed scheduling results in incomplete observation, D2D links share the spectrum with uncertainty on the transmission environment. Therefore, we propose a distributed scheduling scheme, called D-age, to deal with the transmission uncertainty in the AgeS problem, where an adaptation of actor-critic method is adopted with AoI constraints tackled in the dual domain. To address the non-stationary environment and multi-agent credit assignment issue, cooperative multi-agent reinforcement learning (MARL) approach is developed, where multiple local actors are designed to guide D2D links to make real-time decisions via distributed scheduling policies, which are evaluated by an edge-assisted global critic with action-aware advantage functions. Integrated with graph attention networks (GATs), the critic selectively learns contextual information by assigning different importances to neighboring links, which enables the evaluation of scheduling policies in a scalable and computation-efficient manner. Theoretical guarantee of the time-averaged AoI constraints is provided and the effectiveness of D-age in terms of both AoI violation ratio and the capacity of primal links is demonstrated by simulation.

Correlated Information Scheduling in Industrial Internet of Things Based on Multi-Heterogeneous-Agent-Reinforcement-Learning

Harmony or Involution: Game Inspiring Age-of-Information Optimization for Edge Data Gathering in Internet of Things

AoI-Aware Resource Scheduling for Industrial IoT with Deep Reinforcement Learning

Learning-Based Autonomous Scheduling for AoI-Aware Industrial Wireless Networks

Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

Age-of-Information Aware Scheduling for Edge-Assisted Industrial Wireless Networks

Edge-Assisted Spectrum Sharing for Freshness-Aware Industrial Wireless Networks: A Learning-Based Approach

Deep-Reinforcement-Learning-Based Production Scheduling in Industrial Internet of Things

Reinforcement Learning for Optimizing Age of Information and Energy Consumption in Misson-critical IoT

A Deep Reinforcement Learning Approach for Improving Age of Information in Mission-Critical IoT

User Scheduling for Information Freshness over Correlated Markov Channels

Intelligent Scheduling of UAVs and Sensors for Information Age Minimization at Wireless Powered Internet of Things

Real-Time Virtual Machine Scheduling in Industry IoT Network: A Reinforcement Learning Method

Age-of-Information Oriented Scheduling for Multichannel IoT Systems With Correlated Sources

Distributed Q-Learning Aided Heterogeneous Network Association for Energy-Efficient IIoT.

Deep Reinforcement Learning for Scheduling in an Edge Computing-Based Industrial Internet of Things

Scheduling Approaches for Joint Optimization of Age and Delay in Industrial Wireless Networks

Age-Based Scheduling for Monitoring and Control Applications in Mobile Edge Computing Systems

AoI-aware Energy Control and Computation Offloading for Industrial IoT.

Coexistence Between Task- and Data-Oriented Communications: A Whittle's Index Guided Multi-Agent Reinforcement Learning Approach.

QoS-Driven Distributed Cooperative Data Offloading and Heterogeneous Resource Scheduling for IIoT