Abstract:In this paper, we study the design of a distributive delay-optimal cross-layer scheduling algorithm for two-hop relay communication systems over frequency selective fading channels. The complex interactions of the queues at the source node and the M relays (RSs) are modeled as an infinite horizon average reward Markov Decision Process (MDP), whose state space involves the joint queue state (QSI) of the queue at the source node and the queues at the M RSs as well as the joint channel state (CSI) of all S-R links and R-D links. As a first step to address the curse of dimensionality, we propose a reduced state MDP formulation. From the associated Bellman’s equation, we show that the delay-optimal power control (and link selection algorithm), which are functions of both the CSI and QSI, has a multi-level water-filling structure. Furthermore, using stochastic learning, we derive a distributive online learning algorithm in which each node recursively estimates a per-node potential function based on real-time observations of the local CSI and local QSI only. Based on the real-time local potential estimates and using approximate MDP, we propose an auction-based algorithm for link selection and show that the combined distributive learning converges almost surely to a global optimal solution for large arrivals. The proposed online learning algorithm is different from the conventional online learning algorithms in two ways: (1) our online iterative solution updates both the value function (potential) and the Lagrange multipliers (LM) simultaneously; and (2) we establish the technical conditions for the September 10, 2009 DRAFT September 10, 2009 2 almost sure convergence even the per-node potential update equation is no longer a contraction mapping and the existing convergence results (based on contraction mapping) cannot be applied directly to our distributive stochastic learning algorithm. Finally, we show by simulation that the delay performance of the proposed scheme is significantly better than various baselines such as the conventional CSIT-only control and the throughput optimal control (in stability sense).

Queue-Aware Distributive Resource Control for Delay-Sensitive Two-Hop MIMO Cooperative Systems

Delay optimal power control and relay selection for two-hop cooperative OFDM systems via distributive stochastic learning

Delay-Optimal Two-Hop Cooperative Relay Communications via Approximate MDP and Distributive Stochastic Learning

Delay-Aware Two-Hop Cooperative Relay Communications Via Approximate MDP and Stochastic Learning

Distributed Mimo Network Optimization Based On Duality And Local Message Passing

Resource allocation for MIMO orthogonal relay channels with finite-rate feedback

Dynamic Partial Cooperative MIMO System for Delay-Sensitive Applications with Limited Backhaul Capacity

Distributed Extremum Seeking And Cooperative Control For Mobile Communication

Partial Channel State Information Based Cooperative Relaying And Partner Selection

Distributive Stochastic Learning for Delay-Optimal OFDMA Power and Subband Allocation

Efficient ARQ protocols for exploiting cooperative relaying in wireless sensor networks

Distributed joint optimization of relay selection and subchannel pairing in OFDM based relay networks

Distributive Subband Allocation, Power and Rate Control for Relay-Assisted OFDMA Cellular System with Imperfect System State Knowledge

On QoS-Guaranteed Downlink Cooperative OFDMA Systems with Amplify-and-Forward Relays: Optimal Schedule and Resource Allocation

Queuing Analyses and Statistically Bounded Delay Control for Two-Hop Green Wireless Relay Transmissions.

Joint channel-aware and queue-aware relay selection and resource allocation in cooperative OFDMA networks

QoS Aware Relay Selection and Subcarrier Allocation in Cooperative OFDMA Systems.

Delay-Optimal User Scheduling and Inter-Cell Interference Management in Cellular Network via Distributive Stochastic Learning

Stochastic Throughput Optimization for Two-Hop Systems with Finite Relay Buffers

Robust Energy-Efficient Power Allocation and Relay Selection for Cooperative Relay Networks

Distributed Space-Time Coding Based on Adjustable Code Matrices for Cooperative MIMO Relaying Systems