Abstract:The Age of Information (AoI) has recently gained recognition as a critical quality-of-service (QoS) metric for quantifying the freshness of status updates, playing a crucial role in supporting massive ultra-reliable and low-latency communications (mURLLC) services. In mURLLC scenarios, due to the inherent system dynamics and varying environmental conditions, optimizing AoI under such multi-QoS constraints considering both delay and reliability often results in non-convex and computationally intractable problems. Motivated by the demonstrated efficacy of deep reinforcement learning (DRL) in addressing large-scale networking challenges, this work aims to apply DRL techniques to derive optimal resource allocation solutions in real time. Despite its potential, the effective integration of FBC in DRL-based AoI optimization remains underexplored, especially in addressing the challenge of simultaneously upper-bounding both delay and error-rate. To address these challenges, we propose a DRL-based framework for AoI-aware optimal resource allocation in mURLLC-driven multi-QoS schemes, leveraging AoI as a core metric within the finite blocklength regime. First, we design a wireless communication architecture and AoI-based modeling framework that incorporates FBC. Second, we proceed by deriving upper-bounded peak AoI and delay violation probabilities using stochastic network calculus (SNC). Subsequently, we formulate an optimization problem aimed at minimizing the peak AoI violation probability through FBC. Third, we develop DRL algorithms to determine optimal resource allocation policies that meet statistical delay and error-rate requirements for mURLLC. Finally, to validate the effectiveness of the developed schemes, we have executed a series of simulations.

DRL-Based Channel and Latency Aware Radio Resource Allocation for 5G Service-Oriented RoF-MmWave RAN

Deep Reinforcement Learning-based Radio Resource Allocation and Beam Management under Location Uncertainty in 5G mmWave Networks

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Using Deep Reinforcement Learning for 5G RAN Slicing Resource Allocation in New Power Load Management System

DRL-Based QoS-Aware Resource Allocation Scheme for Coexistence of Licensed and Unlicensed Users in LTE and Beyond

A Reinforcement Learning Framework for Autonomous Cell Activation and Customized Energy-Efficient Resource Allocation in C-RANs.

Radio Resource and Beam Management in 5G mmWave Using Clustering and Deep Reinforcement Learning

Toward Scalable and Efficient Hierarchical Deep Reinforcement Learning for 5G RAN Slicing

Learning from Peers: Deep Transfer Reinforcement Learning for Joint Radio and Cache Resource Allocation in 5G RAN Slicing

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Deep Reinforcement Learning-Assisted Optimization for Resource Allocation in Downlink OFDMA Cooperative Systems

Deep Reinforcement Learning Based Joint Uplink and Downlink Resource Allocation for URLLC

Deep Reinforcement Learning Based Resource Allocation for Content Distribution in Fog Radio Access Networks

AoI-Aware Resource Allocation for Smart Multi-QoS Provisioning

A DRL-Based Joint Scheduling and Resource Allocation Scheme for Mixed Unicast–Broadcast Transmission in 5G

A Multi-Agent Deep Reinforcement Learning Approach for RAN Resource Allocation in O-RAN

Deep Reinforcement Learning Based Mode Selection and Resource Management for Green Fog Radio Access Networks

Deep Multi-Agent Reinforcement Learning for Resource Allocation in D2D Communication Underlaying Cellular Networks

DRL-based Joint Resource Scheduling of eMBB and URLLC in O-RAN

Joint Allocation of Radio and Fronthaul Resources in Multi-Wavelength-Enabled C-RAN Based on Reinforcement Learning

Multi-objective Resource Allocation for 5G Using Hierarchical Reinforcement Learning