Abstract:In this paper, we propose a distributed collaborative beamforming (DCB)-based uplink communication paradigm for enabling ground-space direct communications. Specifically, DCB treats the terminals that are unable to establish efficient direct connections with the low Earth orbit (LEO) satellites as distributed antennas, forming a virtual antenna array to enhance the terminal-to-satellite uplink achievable rates and durations. However, such systems need multiple trade-off policies that variously balance the terminal-satellite uplink achievable rate, energy consumption of terminals, and satellite switching frequency to satisfy the scenario requirement changes. Thus, we perform a multi-objective optimization analysis and formulate a long-term optimization problem. To address availability in different terminal cluster scales, we reformulate this problem into an action space-reduced and universal multi-objective Markov decision process. Then, we propose an evolutionary multi-objective deep reinforcement learning algorithm to obtain the desirable policies, in which the low-value actions are masked to speed up the training process. As such, the applicability of a one-time trained model can cover more changing terminal-satellite uplink scenarios. Simulation results show that the proposed algorithm outmatches various baselines, and draw some useful insights. Specifically, it is found that DCB enables terminals that cannot reach the uplink achievable threshold to achieve efficient direct uplink transmission, which thus reveals that DCB is an effective solution for enabling direct ground-space communications. Moreover, it reveals that the proposed algorithm achieves multiple policies favoring different objectives and achieving near-optimal uplink achievable rates with low switching frequency.

Collaborative Computing in Non-Terrestrial Networks: A Multi-Time-Scale Deep Reinforcement Learning Approach

Collaborative Computing in Non-Terrestrial Networks: A Multi-Time-Scale Deep Reinforcement Learning Approach

Collaborative Deep Reinforcement Learning for Resource Optimization in Non-Terrestrial Networks

Toward Intelligent Non-Terrestrial Networks Through Symbiotic Radio: A Collaborative Deep Reinforcement Learning Scheme

Collaborative Deep Reinforcement Learning in 6G Integrated Satellite-Terrestrial Networks: Paradigm, Solutions, and Trends

Deep Reinforcement Learning for Multi-User Access Control in Non-Terrestrial Networks.

Multi-tier Collaborative Deep Reinforcement Learning for Non-terrestrial Network Empowered Vehicular Connections

Multi-Tier Deep Reinforcement Learning for Non-Terrestrial Networks

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Collaborative Ground-Space Communications via Evolutionary Multi-objective Deep Reinforcement Learning

Reinforcement Learning for Cognitive Delay/Disruption Tolerant Network Node Management in an LEO-based Satellite Constellation

Adaptive and Cooperative Resource Scheduling for Satellite-Terrestrial Networks.

Energy Efficiency Optimization for Subterranean LoRaWAN Using A Reinforcement Learning Approach: A Direct-to-Satellite Scenario

Deep Reinforcement Learning Based Resource Allocation for RSMA in LEO Satellite-Terrestrial Networks

Dual Network Computation Offloading Based on DRL for Satellite-Terrestrial Integrated Networks

Federated Deep Reinforcement Learning Assisting TT&C Mission Scheduling in Mega Satellite Networks

Multi-Agent DRL for Resource Allocation and Cache Design in Terrestrial-Satellite Networks

Joint Computation Offloading and Resource Allocation for LEO Satellite Networks Using Hierarchical Multi-Agent Reinforcement Learning

Deep Reinforcement Learning for Congestion Control and Routing Optimization in LEO Satellite Networks

Multi-Commodity Flow Routing for Large-Scale LEO Satellite Networks Using Deep Reinforcement Learning

Deep Reinforcement Learning Based Power Allocation for High Throughput Satellites.