Abstract:Routing optimization within a low-Earth orbit (LEO) satellite broadband network (LSBN) has seen advancements through deep reinforcement learning (DRL) approaches in academia. Nonetheless, a crucial aspect often overlooked in these approaches pertains to the inference time of deep neural network (DNN) models during the routing of packets. Our investigation reveals that this oversight can significantly impair routing throughput in LSBN. In response, this article innovatively proposes a decentralized flow-centric DRL approach, shifting the focus from routing individual packets to entire traffic flows. To align with the large-scale feature of LSBN, we embrace a fully distributed architecture for flow-centric routing, which is modeled as a partially observable Markov decision process. In this construct, each satellite operates as an independent agent, locally classifying flows following a tailor-designed definition, and is responsible for forwarding a flow to an adjacent satellite based on its internal policy. Notably, the DNN inference is conducted only once on each agent to determine the route for the initial packet of a specific flow; subsequent packets are directed along the same route. Recognizing the potential impact of dynamic LSBN topologies on routing performance, we also introduce an adaptive flow routing update scheme. This scheme is completely free from LSBN environment modeling and aims to bolster the efficacy of the flow-centric approach. Comparative experiments showcase the superiority of the proposed approach over baseline algorithms across various metrics. Consequently, the flow-centric DRL approach can enable high-throughput traffic transmission for LSBN.

Integrating LEO Satellite and UAV Relaying via Reinforcement Learning for Non-Terrestrial Networks

Integrating LEO Satellites and Multi-UAV Reinforcement Learning for Hybrid FSO/RF Non-Terrestrial Networks

Federated Reinforcement Learning Based AANs with LEO Satellites and UAVs

Toward Intelligent Non-Terrestrial Networks Through Symbiotic Radio: A Collaborative Deep Reinforcement Learning Scheme

Enabling Urban MmWave Communications with UAV-Carried IRS Via Deep Reinforcement Learning

Multi-Commodity Flow Routing for Large-Scale LEO Satellite Networks Using Deep Reinforcement Learning

Energy efficiency performance in RIS-based integrated satellite–aerial–terrestrial relay networks with deep reinforcement learning

An Intelligent Routing Algorithm for LEO Satellites Based on Deep Reinforcement Learning

Collaborative Deep Reinforcement Learning for Resource Optimization in Non-Terrestrial Networks

Enabling High-Throughput Routing for LEO Satellite Broadband Networks: A Flow-Centric Deep Reinforcement Learning Approach

Deep Reinforcement Learning Based Data Offloading in Multi-Layer Ka/Q Band LEO Satellite-Terrestrial Networks

Collaborative Computing in Non-Terrestrial Networks: A Multi-Time-Scale Deep Reinforcement Learning Approach

An Intelligent Coexistence Strategy for Embb/urllc Traffic in Multi-UAV Relay Networks Via Deep Reinforcement Learning

A DRL Empowered Multipath Cooperative Routing for Ultra-Dense LEO Satellite Networks

A Hierarchical Deep Reinforcement Learning Approach for Throughput Maximization in Reconfigurable Intelligent Surface-Aided Unmanned Aerial Vehicle–Integrated Sensing and Communication Network

Learning Emergent Random Access Protocol for LEO Satellite Networks

Deep Reinforcement Learning for Aerial Data Collection in Hybrid-Powered NOMA-IoT Networks

UAV-assisted Semantic Communication with Hybrid Action Reinforcement Learning

A Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning for LEO Satellite Networks

Joint Drone Access and LEO Satellite Backhaul for a Space–Air–Ground Integrated Network: A Multi-Agent Deep Reinforcement Learning-Based Approach