Abstract:The Age of Information (AoI) has recently gained recognition as a critical quality-of-service (QoS) metric for quantifying the freshness of status updates, playing a crucial role in supporting massive ultra-reliable and low-latency communications (mURLLC) services. In mURLLC scenarios, due to the inherent system dynamics and varying environmental conditions, optimizing AoI under such multi-QoS constraints considering both delay and reliability often results in non-convex and computationally intractable problems. Motivated by the demonstrated efficacy of deep reinforcement learning (DRL) in addressing large-scale networking challenges, this work aims to apply DRL techniques to derive optimal resource allocation solutions in real time. Despite its potential, the effective integration of FBC in DRL-based AoI optimization remains underexplored, especially in addressing the challenge of simultaneously upper-bounding both delay and error-rate. To address these challenges, we propose a DRL-based framework for AoI-aware optimal resource allocation in mURLLC-driven multi-QoS schemes, leveraging AoI as a core metric within the finite blocklength regime. First, we design a wireless communication architecture and AoI-based modeling framework that incorporates FBC. Second, we proceed by deriving upper-bounded peak AoI and delay violation probabilities using stochastic network calculus (SNC). Subsequently, we formulate an optimization problem aimed at minimizing the peak AoI violation probability through FBC. Third, we develop DRL algorithms to determine optimal resource allocation policies that meet statistical delay and error-rate requirements for mURLLC. Finally, to validate the effectiveness of the developed schemes, we have executed a series of simulations.

802.11ax Uplink Scheduling Algorithm Based onReinforcement Learning

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

QoS-oriented media access control using reinforcement learning for next-generation WLANs

ReinWiFi: A Reinforcement-Learning-Based Framework for the Application-Layer QoS Optimization of WiFi Networks

Resource Allocation in Uplink NOMA-IoT Networks: A Reinforcement-Learning Approach

Joint Optimization on Uplink OFDMA and MU-MIMO for IEEE 802.11ax: Deep Hierarchical Reinforcement Learning Approach

Multi-Agent Reinforcement Learning based Uplink OFDMA for IEEE 802.11ax Networks

Toward Scalable and Efficient Hierarchical Deep Reinforcement Learning for 5G RAN Slicing

AoI-Aware Resource Scheduling for Industrial IoT with Deep Reinforcement Learning

Joint Access Point Placement and Power-Channel-Resource-Unit Assignment for 802.11ax-Based Dense WiFi with QoS Requirements

AoI-Aware Resource Allocation for Smart Multi-QoS Provisioning

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

Deep Reinforcement Learning for Optimizing Restricted Access Window in IEEE 802.11ah MAC Layer

Deep Multiagent Reinforcement-Learning-Based Resource Allocation for Internet of Controllable Things

A Deep Reinforcement Learning-Based Resource Scheduler for Massive MIMO Networks

QoE-based Deep Reinforcement Learning for Resource Allocation in Real Time XR Video Transmission

Dual Dynamic Scheduling for Hierarchical QoS in Uplink-NOMA: A Reinforcement Learning Approach

Deep-Reinforcement-Learning-Based Scheduling with Contiguous Resource Allocation for Next-Generation Cellular Systems

Reinforcement Learning Based Scheduling for Heterogeneous UAV Networking.

Resource Management for OFDMA based Next Generation 802.11ax WLANs