Abstract:A two-time-scale approximate Markov decision process (MDP) is proposed to optimize the uplink queuing performance of a millimeter wave (mmWave) communication system. Exploiting the wireless sensing techniques, the locations of signal reflectors, blockers, and mobile agents, as well as the mobility pattern of agents, become available knowledge to facilitate predictive scheduling. Notice that the state variation of the wireless channel fading and transmission queue is much faster than that of mobile agents’ locations. The joint optimization of uplink power adaptation, time allocation, and analog beamforming of all the frames is formulated as a two-time-scale MDP with the average uplink energy, queuing length, and buffer overflow rate in the minimization objective. The scheduling in the larger time scale with an infinite horizon, i.e., the analog transceiver beamforming, adapts with the random motion of mobile agents; whereas the scheduling in the smaller time scale with a finite horizon, i.e., uplink power and time allocation, adapts with both the small-scale channel fading, queue dynamics and large-scale random motion. The optimal solution of two-time-scale MDP requests iterative optimization between Bellman’s equations of both time scales, whose computation complexity is prohibitive. A novel low-complexity solution framework is then proposed to obtain the optimal larger-time-scale and sub-optimal smaller-time-scale policies, where the performance of the proposed scheme is bounded analytically. Benefiting from the motion sensing and blockage prediction, the proposed scheduling scheme outperforms existing benchmarks in the numerical simulations.

Online Multi-User Scheduling for XR Transmissions with Hard-Latency Constraint: Performance Analysis and Practical Design

Joint User Scheduling and Resource Allocation for Millimeter Wave Systems Relying on Adaptive-Resolution ADCs

Joint Spatial-Frequency Domain Message Passing Algorithm for Radio Resource Scheduling

Real-time Extended Reality Video Transmission Optimization Based on Frame-priority Scheduling

Joint Beamforming Design and Blocklength Optimization for Low-Latency Multi-User MISO URLLC Systems

On the Queue Dynamics of Multiuser Multichannel Cognitive Radio Networks

Two-Timescale Resource Management for Ultrareliable and Low-Latency Vehicular Communications

The Delay-Power Tradeoff of Low Complexity Cross-Layer Scheduling: when Lyapunov Meets Markov

Delay-Oriented Scheduling in 5G Downlink Wireless Networks Based on Reinforcement Learning With Partial Observations

Scheduling with Probabilistic Per-Packet Real-Time Guarantee for URLLC

QoE-based Deep Reinforcement Learning for Resource Allocation in Real Time XR Video Transmission

Multi-User Delay-Constrained Scheduling With Deep Recurrent Reinforcement Learning

Quality of Experience Oriented Cross-layer Optimization for Real-time XR Video Transmission

Scalable Multiuser Immersive Communications with Multi-numerology and Mini-slot

High-Resolution Cross-Layer Scheduling for Low-Latency Communications: when Infinitesimal Method Meets Stochastic Order

Delay-Aware Two-Time-Scale Scheduling for mmWave Systems with Mobility and Environment Knowledge

A Dynamic Resource Scheduling Algorithm Based on Traffic Prediction for Coexistence of eMBB and Random Arrival URLLC

Effective Multi-User Delay-Constrained Scheduling with Deep Recurrent Reinforcement Learning

Offline Learning-Based Multi-User Delay-Constrained Scheduling