Abstract:The Age of Information (AoI) has recently gained recognition as a critical quality-of-service (QoS) metric for quantifying the freshness of status updates, playing a crucial role in supporting massive ultra-reliable and low-latency communications (mURLLC) services. In mURLLC scenarios, due to the inherent system dynamics and varying environmental conditions, optimizing AoI under such multi-QoS constraints considering both delay and reliability often results in non-convex and computationally intractable problems. Motivated by the demonstrated efficacy of deep reinforcement learning (DRL) in addressing large-scale networking challenges, this work aims to apply DRL techniques to derive optimal resource allocation solutions in real time. Despite its potential, the effective integration of FBC in DRL-based AoI optimization remains underexplored, especially in addressing the challenge of simultaneously upper-bounding both delay and error-rate. To address these challenges, we propose a DRL-based framework for AoI-aware optimal resource allocation in mURLLC-driven multi-QoS schemes, leveraging AoI as a core metric within the finite blocklength regime. First, we design a wireless communication architecture and AoI-based modeling framework that incorporates FBC. Second, we proceed by deriving upper-bounded peak AoI and delay violation probabilities using stochastic network calculus (SNC). Subsequently, we formulate an optimization problem aimed at minimizing the peak AoI violation probability through FBC. Third, we develop DRL algorithms to determine optimal resource allocation policies that meet statistical delay and error-rate requirements for mURLLC. Finally, to validate the effectiveness of the developed schemes, we have executed a series of simulations.

AoI-Aware Resource Allocation for Smart Multi-QoS Provisioning

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep Reinforcement Learning Based Resource Allocation for URLLC User-Centric Network

Deep Reinforcement Learning Based Joint Uplink and Downlink Resource Allocation for URLLC

Deep Reinforcement Learning-Assisted Optimization for Resource Allocation in Downlink OFDMA Cooperative Systems

Resource Allocation for Multi-service NOMA System Based on Deep Reinforcement Learning

A DRL-Driven Intelligent Optimization Strategy for Resource Allocation in Cloud-Edge-End Cooperation Environments

Intelligent Resource Allocation for IRS-Enhanced OFDM Communication Systems: A Hybrid Deep Reinforcement Learning Approach

DRL-based Joint Resource Scheduling of eMBB and URLLC in O-RAN

Deep Multiagent Reinforcement-Learning-Based Resource Allocation for Internet of Controllable Things

A Self-Organizing Resource Allocation Strategy Based on Q-learning Approach in Ultra-Dense Networks

Qoe Driven Resource Allocation In Massive Iot: A Deep Reinforcement Learning Approach

Intelligent Resource Slicing for eMBB and URLLC Coexistence in 5G and Beyond: A Deep Reinforcement Learning Based Approach

An Approach to Transmitting URLLC Data with Different Latency Requirements over Embb Services Based on Deep Reinforcement Learning

DRL-QOR: Deep Reinforcement Learning-Based QoS/QoE-Aware Adaptive Online Orchestration in NFV-Enabled Networks

Deep Reinforcement Learning Based Resource Allocation for Content Distribution in Fog Radio Access Networks

Intelligent QoS aware slice resource allocation with user association parameterization for beyond 5G ORAN based architecture using DRL

Deep Reinforcement Learning for Wireless Resource Allocation Using Buffer State Information

Resource Allocation in OFDMA Networks with Deep Reinforcement Learning

A Model-Driven Deep Reinforcement Learning Heuristic Algorithm for Resource Allocation in Ultra-Dense Cellular Networks

Cross-Layer Adaptive Resource Allocation Algorithm with Diverse QoS Requirements for Single-Cell OFDMA Systems