Abstract:The Age of Information (AoI) has recently gained recognition as a critical quality-of-service (QoS) metric for quantifying the freshness of status updates, playing a crucial role in supporting massive ultra-reliable and low-latency communications (mURLLC) services. In mURLLC scenarios, due to the inherent system dynamics and varying environmental conditions, optimizing AoI under such multi-QoS constraints considering both delay and reliability often results in non-convex and computationally intractable problems. Motivated by the demonstrated efficacy of deep reinforcement learning (DRL) in addressing large-scale networking challenges, this work aims to apply DRL techniques to derive optimal resource allocation solutions in real time. Despite its potential, the effective integration of FBC in DRL-based AoI optimization remains underexplored, especially in addressing the challenge of simultaneously upper-bounding both delay and error-rate. To address these challenges, we propose a DRL-based framework for AoI-aware optimal resource allocation in mURLLC-driven multi-QoS schemes, leveraging AoI as a core metric within the finite blocklength regime. First, we design a wireless communication architecture and AoI-based modeling framework that incorporates FBC. Second, we proceed by deriving upper-bounded peak AoI and delay violation probabilities using stochastic network calculus (SNC). Subsequently, we formulate an optimization problem aimed at minimizing the peak AoI violation probability through FBC. Third, we develop DRL algorithms to determine optimal resource allocation policies that meet statistical delay and error-rate requirements for mURLLC. Finally, to validate the effectiveness of the developed schemes, we have executed a series of simulations.

A Self-Organizing Resource Allocation Strategy Based on Q-learning Approach in Ultra-Dense Networks

Multi-Agent Reinforcement Learning Based Unlicensed Resource Sharing for LTE-U Networks.

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Deep Q-Network Based Resource Allocation for UAV-assisted Ultra-Dense Networks

Q-learning Based Resource Allocation for Hybrid Services with Self-similar Traffic

BiLSTM Based Reinforcement Learning for Resource Allocation and User Association in LTE-U Networks

Traffic Priority-Aware Multi-User Distributed Dynamic Spectrum Access: A Multi-Agent Deep RL Approach

Multi-Agent Q-Learning for Real-Time Load Balancing User Association and Handover in Mobile Networks

AoI-Aware Resource Allocation for Smart Multi-QoS Provisioning

Federated Learning for Distributed Energy-Efficient Resource Allocation

Slicing Resource Allocation Based on Dueling DQN for eMBB and URLLC Hybrid Services in Heterogeneous Integrated Networks

Multi-Agent Deep Reinforcement Learning Based Resource Allocation for Ultra-Reliable Low-Latency Internet of Controllable Things

Deep Reinforcement Learning-Based Resource Allocation for Cellular Vehicular Network Mode 3 with Underlay Approach

Q-Learning Based Selection Strategies for Load Balance and Energy Balance in Heterogeneous Networks

A deep reinforcement learning-based D2D spectrum allocation underlaying a cellular network

Multi-objective Resource Allocation for 5G Using Hierarchical Reinforcement Learning

5G Multi-Slices Bi-Level Resource Allocation by Reinforcement Learning

DRL-Based QoS-Aware Resource Allocation Scheme for Coexistence of Licensed and Unlicensed Users in LTE and Beyond

Dynamic Spectrum Sharing Based on Deep Reinforcement Learning in Mobile Communication Systems

Deep Multiagent Reinforcement-Learning-Based Resource Allocation for Internet of Controllable Things