Abstract:A two-time-scale approximate Markov decision process (MDP) is proposed to optimize the uplink queuing performance of a millimeter wave (mmWave) communication system. Exploiting the wireless sensing techniques, the locations of signal reflectors, blockers, and mobile agents, as well as the mobility pattern of agents, become available knowledge to facilitate predictive scheduling. Notice that the state variation of the wireless channel fading and transmission queue is much faster than that of mobile agents’ locations. The joint optimization of uplink power adaptation, time allocation, and analog beamforming of all the frames is formulated as a two-time-scale MDP with the average uplink energy, queuing length, and buffer overflow rate in the minimization objective. The scheduling in the larger time scale with an infinite horizon, i.e., the analog transceiver beamforming, adapts with the random motion of mobile agents; whereas the scheduling in the smaller time scale with a finite horizon, i.e., uplink power and time allocation, adapts with both the small-scale channel fading, queue dynamics and large-scale random motion. The optimal solution of two-time-scale MDP requests iterative optimization between Bellman’s equations of both time scales, whose computation complexity is prohibitive. A novel low-complexity solution framework is then proposed to obtain the optimal larger-time-scale and sub-optimal smaller-time-scale policies, where the performance of the proposed scheme is bounded analytically. Benefiting from the motion sensing and blockage prediction, the proposed scheduling scheme outperforms existing benchmarks in the numerical simulations.

Multi-User Multi-Application Packet Scheduling for Application-Specific QoE Enhancement Based on Knowledge-Embedded DDPG in 6G RAN

A Dynamic Packet Scheduling Method for Multipath TCP in Heterogeneous Wireless Networks

Packet Scheduling In Broadband Wireless Networks Using Neuro-Dynamic Programming

Resource Allocation and Scheduling Algorithm for Wireless Communication System with QoS Guaranteed

NDPS: Packet Scheduling Algorithm for Wireless Multimedia Networks

Learning Resource Scheduling with High Priority Users using Deep Deterministic Policy Gradients

Delay-Oriented Scheduling in 5G Downlink Wireless Networks Based on Reinforcement Learning With Partial Observations

A Multi-Server Scheduling Framework for Resource Allocation in Wireless Multi-Carrier Networks

User Scheduling and Task Offloading in Multi-Tier Computing 6G Vehicular Network

Multi - Agent Deep Deterministic Policy Gradient Based Satellite Spectrum/Code Resource Scheduling with Multi-constraint

Dual Dynamic Scheduling for Hierarchical QoS in Uplink-NOMA: A Reinforcement Learning Approach

Delay-Aware Two-Time-Scale Scheduling for mmWave Systems with Mobility and Environment Knowledge

Structural Solutions to Dynamic Scheduling for Multimedia Transmission in Unknown Wireless Environments

On-Demand Resource Management for 6G Wireless Networks Using Knowledge-Assisted Dynamic Neural Networks

Dynamic user-centric multi-dimensional resource allocation for a wide-area coverage signaling cell based on DQN

Uplink Cross-Layer Scheduling with Differential QoS Requirements in OFDMA Systems

Joint User Association and Resource Allocation for Tailored QoS Provisioning in 6G HetNets

Energy-Aware MPTCP Scheduling in Heterogeneous Wireless Networks Using Multi-Agent Deep Reinforcement Learning Techniques

EdAR: an Experience-Driven Multipath Scheduler for Seamless Handoff in Mobile Networks

A Partially Observable Deep Multi-Agent Active Inference Framework for Resource Allocation in 6G and Beyond Wireless Communications Networks

Joint QoS-Aware Scheduling and Precoding for Massive MIMO Systems via Deep Reinforcement Learning