Abstract:The integration of terrestrial and satellite wireless communication networks offers a practical solution to enhance network coverage, connectivity, and cost-effectiveness. Moreover, in today’s interconnected world, connectivity’s reliable and widespread availability is increasingly important across various domains. This is especially more crucial for applications like the Internet of Things (IoT), remote sensing, disaster management, and bridging the digital divide. However, allocating the limited network resources efficiently and ensuring seamless handover between satellite and terrestrial networks present significant challenges. Therefore, this study introduces a resource allocation framework for integrated satellite–terrestrial networks to address these challenges. The framework leverages local cache pool deployments and non-orthogonal multiple access (NOMA) to reduce time delays and improve energy efficiency. Our proposed approach utilizes a multi-agent enabled deep deterministic policy gradient algorithm (MADDPG) to optimize user association, cache design, and transmission power control, resulting in enhanced energy efficiency. The approach comprises two phases: User Association and Power Control, where users are treated as agents, and Cache Optimization, where the satellite (Bs) is considered the agent. Through extensive simulations, we demonstrate that our approach surpasses conventional single-agent deep reinforcement learning algorithms in addressing cache design and resource allocation challenges in integrated terrestrial–satellite networks. Specifically, our proposed approach achieves significantly higher energy efficiency and reduced time delays compared to existing methods. This research highlights the importance and addresses the need for efficient resource allocation and cache design in integrated terrestrial–satellite networks, paving the way for enhanced connectivity and improved network performance in various applications.

The Next Generation Heterogeneous Satellite Communication Networks: Integration of Resource Management and Deep Reinforcement Learning.

Deep Reinforcement Learning-Based Joint Satellite Scheduling and Resource Allocation in Satellite-Terrestrial Integrated Networks

Collaborative Deep Reinforcement Learning for Resource Optimization in Non-Terrestrial Networks

Multi-agent Deep Reinforcement Learning Based Resource Management in Heterogeneous V2X Networks

Semantic-Aware Resource Allocation Based on Deep Reinforcement Learning for 5G-V2X HetNets

Joint Computation Offloading and Resource Allocation for LEO Satellite Networks Using Hierarchical Multi-Agent Reinforcement Learning

Deep Reinforcement Learning Based Resource Allocation for Heterogeneous Networks

Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks

Toward Intelligent Non-Terrestrial Networks Through Symbiotic Radio: A Collaborative Deep Reinforcement Learning Scheme

Satellite-Terrestrial Coordinated Multi-Satellite Beam Hopping Scheduling Based on Multi-Agent Deep Reinforcement Learning

Collaborative Computing in Non-Terrestrial Networks: A Multi-Time-Scale Deep Reinforcement Learning Approach

Tera-SpaceCom: GNN-based Deep Reinforcement Learning for Joint Resource Allocation and Task Offloading in TeraHertz Band Space Networks

Machine Learning-Based Resource Allocation in Satellite Networks Supporting Internet of Remote Things

Spectrum-Energy-Efficient Mode Selection and Resource Allocation for Heterogeneous V2X Networks: A Federated Multi-Agent Deep Reinforcement Learning Approach

Dynamic Resource Allocation With Deep Reinforcement Learning in Multibeam Satellite Communication

+2cmFlexible Resource Management in High-throughput Satellite Communication Systems: A Two-stage Machine Learning Framework

Dynamic resource management in integrated NOMA terrestrial–satellite networks using multi-agent reinforcement learning