Abstract:The Age of Information (AoI) has recently gained recognition as a critical quality-of-service (QoS) metric for quantifying the freshness of status updates, playing a crucial role in supporting massive ultra-reliable and low-latency communications (mURLLC) services. In mURLLC scenarios, due to the inherent system dynamics and varying environmental conditions, optimizing AoI under such multi-QoS constraints considering both delay and reliability often results in non-convex and computationally intractable problems. Motivated by the demonstrated efficacy of deep reinforcement learning (DRL) in addressing large-scale networking challenges, this work aims to apply DRL techniques to derive optimal resource allocation solutions in real time. Despite its potential, the effective integration of FBC in DRL-based AoI optimization remains underexplored, especially in addressing the challenge of simultaneously upper-bounding both delay and error-rate. To address these challenges, we propose a DRL-based framework for AoI-aware optimal resource allocation in mURLLC-driven multi-QoS schemes, leveraging AoI as a core metric within the finite blocklength regime. First, we design a wireless communication architecture and AoI-based modeling framework that incorporates FBC. Second, we proceed by deriving upper-bounded peak AoI and delay violation probabilities using stochastic network calculus (SNC). Subsequently, we formulate an optimization problem aimed at minimizing the peak AoI violation probability through FBC. Third, we develop DRL algorithms to determine optimal resource allocation policies that meet statistical delay and error-rate requirements for mURLLC. Finally, to validate the effectiveness of the developed schemes, we have executed a series of simulations.

Deep Reinforcement Learning-based Dynamic Bandwidth Allocation in Weighted Fair Queues of Routers.

Freeway network traffic management based on distributed reinforcement learning

Deep Reinforcement Learning for Router Selection in Network with Heavy Traffic

QoS Routing Optimization Based on Deep Reinforcement Learning in SDN

Deep Reinforcement Learning for Wireless Resource Allocation Using Buffer State Information

DRL-TAL: Deep Reinforcement Learning-Based Traffic-Aware Load Balancing in Data Center Networks

Deep Reinforcement Learning Based Dynamic Routing Optimization for Delay-Sensitive Applications

Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control

Preferential bandwidth allocation for short flows with active queue management

Delay-Optimal Scheduling for Heavy-Tailed and Light-Tailed Flows Via Reinforcement Learning

Dynamical Weighted Scheduling Algorithm Supporting Fair Bandwidth Allocation of Virtual Networks

Deep Reinforcement Learning-Based Deterministic Routing and Scheduling for Mixed-Criticality Flows

DRL-PLink: Deep Reinforcement Learning with Private Link Approach for Mix-Flow Scheduling in Software-Defined Data-Center Networks

AoI-Aware Resource Allocation for Smart Multi-QoS Provisioning

QoS Differentiated and Fair Packet Scheduling in Broadband Wireless Access Networks

Deep Distributional Reinforcement Learning-Based Adaptive Routing with Guaranteed Delay Bounds

Integrated Resource Scheduling for User Experience Enhancement: A Heuristically Accelerated DRL

A Novel Deep Reinforcement Learning Architecture for Dynamic Power and Bandwidth Allocation in Multibeam Satellites

Deep Reinforcement Learning Based Dynamic Channel Bonding for Wi-Fi Networks

Deep Reinforcement Learning for Demand-Aware Joint VNF Placement-and-Routing.

DeepCQF: Making CQF Scheduling More Intelligent and Practicable