Abstract:A two-time-scale approximate Markov decision process (MDP) is proposed to optimize the uplink queuing performance of a millimeter wave (mmWave) communication system. Exploiting the wireless sensing techniques, the locations of signal reflectors, blockers, and mobile agents, as well as the mobility pattern of agents, become available knowledge to facilitate predictive scheduling. Notice that the state variation of the wireless channel fading and transmission queue is much faster than that of mobile agents’ locations. The joint optimization of uplink power adaptation, time allocation, and analog beamforming of all the frames is formulated as a two-time-scale MDP with the average uplink energy, queuing length, and buffer overflow rate in the minimization objective. The scheduling in the larger time scale with an infinite horizon, i.e., the analog transceiver beamforming, adapts with the random motion of mobile agents; whereas the scheduling in the smaller time scale with a finite horizon, i.e., uplink power and time allocation, adapts with both the small-scale channel fading, queue dynamics and large-scale random motion. The optimal solution of two-time-scale MDP requests iterative optimization between Bellman’s equations of both time scales, whose computation complexity is prohibitive. A novel low-complexity solution framework is then proposed to obtain the optimal larger-time-scale and sub-optimal smaller-time-scale policies, where the performance of the proposed scheme is bounded analytically. Benefiting from the motion sensing and blockage prediction, the proposed scheduling scheme outperforms existing benchmarks in the numerical simulations.

Stochastic Optimization for Markov Modulated Networks with Application to Delay Constrained Wireless Scheduling

Delay Sensitive Communications over Cognitive Radio Networks

Delay-Optimal Probabilistic Scheduling for Low-Complexity Wireless Links with Fixed Modulation and Coding: A Cross-Layer Design

A Stochastic Framework for Scheduling in Wireless Packet Access Networks

On the Power of Randomization for Scheduling Real-Time Traffic in Wireless Networks

Learning to Schedule in Non-Stationary Wireless Networks With Unknown Statistics

On Delay-Power Tradeoff of Rate Adaptive Wireless Communications with Random Arrivals.

Delay-Aware Two-Time-Scale Scheduling for mmWave Systems with Mobility and Environment Knowledge

Joint Queue-Aware and Channel-Aware Delay Optimal Scheduling of Arbitrarily Bursty Traffic over Multi-State Time-Varying Channels.

Delay-Constrained Optimal Link Scheduling in Wireless Sensor Networks

Joint Channel and Queue Aware Scheduling for Wireless Links with Multiple Fading States.

Stochastic Optimization Based Dynamic User Scheduling and Hybrid Precoding for Broadband MmWave MIMO.

Toward Delay-Based Utility Maximization: Modeling and Implementation in an SDWN Platform.

Delay-optimal Probabilistic Scheduling in Green Communications with Arbitrary Arrival and Adaptive Transmission

Throughput Optimal Decentralized Scheduling of Multi-Hop Networks with End-to-End Deadline Constraints: II Wireless Networks with Interference

Time-constrained Packet Scheduling Optimization for Video Streaming in Wireless Ad-Hoc Networks

Opportunistic Scheduling Using Statistical Information of Wireless Channels

Randomized Scheduling of Real-Time Traffic in Wireless Networks Over Fading Channels

Joint Probabilistic Scheduling And Adaptive Modulation For Queue And Channel Aware Links

Fitting Empowered Cross-Layer Scheduling for Real-Time Wireless Communications