Abstract:A two-time-scale approximate Markov decision process (MDP) is proposed to optimize the uplink queuing performance of a millimeter wave (mmWave) communication system. Exploiting the wireless sensing techniques, the locations of signal reflectors, blockers, and mobile agents, as well as the mobility pattern of agents, become available knowledge to facilitate predictive scheduling. Notice that the state variation of the wireless channel fading and transmission queue is much faster than that of mobile agents’ locations. The joint optimization of uplink power adaptation, time allocation, and analog beamforming of all the frames is formulated as a two-time-scale MDP with the average uplink energy, queuing length, and buffer overflow rate in the minimization objective. The scheduling in the larger time scale with an infinite horizon, i.e., the analog transceiver beamforming, adapts with the random motion of mobile agents; whereas the scheduling in the smaller time scale with a finite horizon, i.e., uplink power and time allocation, adapts with both the small-scale channel fading, queue dynamics and large-scale random motion. The optimal solution of two-time-scale MDP requests iterative optimization between Bellman’s equations of both time scales, whose computation complexity is prohibitive. A novel low-complexity solution framework is then proposed to obtain the optimal larger-time-scale and sub-optimal smaller-time-scale policies, where the performance of the proposed scheme is bounded analytically. Benefiting from the motion sensing and blockage prediction, the proposed scheduling scheme outperforms existing benchmarks in the numerical simulations.

Delay-Optimal Scheduling for Energy Harvesting Aided Mmwave Communications with Random Blocking

Delay Optimal Scheduling for Energy Harvesting Based Communications

Low-Latency and Energy-Efficient Wireless Communications with Energy Harvesting

Delay-Aware Wireless Powered Communication Networks—Energy Balancing and Optimization

Delay-optimal Random Access in Large-Scale Energy Harvesting IoT Networks Based on Mean Field Game

Joint Queue-Aware and Channel-Aware Delay Optimal Scheduling of Arbitrarily Bursty Traffic over Multi-State Time-Varying Channels.

Multiple Timescale Energy Scheduling for Wireless Communication with Energy Harvesting Devices

Delay-Optimal Probabilistic Scheduling for Low-Complexity Wireless Links with Fixed Modulation and Coding: A Cross-Layer Design

Adaptive Harvest-then-cooperate: Delay-aware Wireless Powered Communication Networks.

Delay-Aware Two-Time-Scale Scheduling for mmWave Systems with Mobility and Environment Knowledge

Delay Optimal Random Access with Heterogeneous Device Capabilities in Energy Harvesting Networks Using Mean Field Game.

QoS-Aware Joint User Scheduling and Power Allocation for Energy Harvesting Wireless Networks.

Delay Minimal Policies in Energy Harvesting Communication Systems.

On Delay-Power Tradeoff of Rate Adaptive Wireless Communications with Random Arrivals.

Delay-Power Tradeoff of Fixed-Rate Wireless Transmission with Arbitrarily Bursty Traffics

Optimal Packet Scheduling For Delay Minimization In An Energy Harvesting System

Energy-efficient scheduling of delay constrained traffic under practical power model

Achieving the Optimal Delay-Power Tradeoff in Wireless Transmission with Arbitrarily Random Packet Arrival: A Cross-Layer Approach.

Delay-Optimal Scheduling in WPTNs with Adaptive Transmission over Fading Channels

Integrating Sub-6 GHz and Millimeter Wave to Combat Blockage: Delay-Optimal Scheduling

Delay-Optimal Buffer-Aware Scheduling with Adaptive Transmission