Abstract:In this paper, we study the design of a distributive delay-optimal cross-layer scheduling algorithm for two-hop relay communication systems over frequency selective fading channels. The complex interactions of the queues at the source node and the M relays (RSs) are modeled as an infinite horizon average reward Markov Decision Process (MDP), whose state space involves the joint queue state (QSI) of the queue at the source node and the queues at the M RSs as well as the joint channel state (CSI) of all S-R links and R-D links. As a first step to address the curse of dimensionality, we propose a reduced state MDP formulation. From the associated Bellman’s equation, we show that the delay-optimal power control (and link selection algorithm), which are functions of both the CSI and QSI, has a multi-level water-filling structure. Furthermore, using stochastic learning, we derive a distributive online learning algorithm in which each node recursively estimates a per-node potential function based on real-time observations of the local CSI and local QSI only. Based on the real-time local potential estimates and using approximate MDP, we propose an auction-based algorithm for link selection and show that the combined distributive learning converges almost surely to a global optimal solution for large arrivals. The proposed online learning algorithm is different from the conventional online learning algorithms in two ways: (1) our online iterative solution updates both the value function (potential) and the Lagrange multipliers (LM) simultaneously; and (2) we establish the technical conditions for the September 10, 2009 DRAFT September 10, 2009 2 almost sure convergence even the per-node potential update equation is no longer a contraction mapping and the existing convergence results (based on contraction mapping) cannot be applied directly to our distributive stochastic learning algorithm. Finally, we show by simulation that the delay performance of the proposed scheme is significantly better than various baselines such as the conventional CSIT-only control and the throughput optimal control (in stability sense).

Delay-Aware Online Service Scheduling in High-Speed Railway Communication Systems

A QoS-Aware Scheduling Algorithm for High-Speed Railway Communication System

Delay-aware Opportunistic Scheduling for Real-time Service in OFDMA Systems

Opportunistic Scheduling for Heterogeneous Services in Downlink OFDMA System

Scheduling Strategy for Multimedia Heterogeneous High-Speed Train Networks

Delay Constrained Optimal Power Allocation in High-Speed Railway Scenarios.

Stochastic Delay Analysis for Train Control Services in Next-Generation High-Speed Railway Communications System.

A Driving Strategy Based Integrated Rescheduling Model for High-Speed Railway by Using the Parallel Intelligent Method.

Delay-Aware Two-Hop Cooperative Relay Communications Via Approximate MDP and Stochastic Learning

Delay-Optimal Two-Hop Cooperative Relay Communications via Approximate MDP and Distributive Stochastic Learning

Optimal Selection of Heterogeneous Network Interfaces for High-Speed Rail Communications

Location-Aware Dynamic Resource Management for High-Speed Railway Wireless Communications

Resource Allocation for Millimeter-Wave Train-Ground Communications in High-Speed Railway Scenarios

Delay-Optimal User Scheduling and Inter-Cell Interference Management in Cellular Network via Distributive Stochastic Learning

Qos-Distinguished Achievable Rate Region for High Speed Railway Wireless Communications

QoS-Aware User Association and Transmission Scheduling for Millimeter-Wave Train-Ground Communications

High-speed Train Timetabling Based on Reinforcement Learning.

QoS Differential Scheduling in Cognitive-Radio-Based Smart Grid Networks: an Adaptive Dynamic Programming Approach.

Transmission Scheduling of Millimeter Wave Communication for High-Speed Railway in Space-Air-Ground Integrated Network

An Optimization Method of High-Speed Railway Rescheduling to Meet Unexpected Large Passenger Flow