Abstract:As the complexity of space missions increases, the constraints on satellite attitude control become more stringent, particularly for satellites working in orbit formation. This paper introduces a novel method, based on the categorization and modeling of different constraints, for attitude control of satellite formations under multiple constraints. The method employs the Phased Priority Reinforcement Learning (PPRL) approach, which utilizes Deep Deterministic Policy Gradient (DDPG) technology. Considering the complexity of constraints and the challenge posed by the high control dimensionality due to multi-satellite coordination, the method addresses these challenges through a two-step training strategy. The first step addresses the multi-constraint issue for individual satellites and increases the priority of single-satellite training experience data in the experience replay buffer of the second step to enhance data utilization efficiency. To address the issue of reward sparsity in complex high-dimensional constraint models, a detailed reward mechanism is proposed, incorporating both local and global constraints into the reward function, thereby achieving both efficient and effective attitude control. This approach not only meets dynamic, state, and performance constraints but also demonstrates adaptability and robustness through numerical simulations. Compared to traditional methods, this approach achieves significant improvements in control performance and constraint satisfaction, offering a novel solution pathway for high-dimensional control problems in multi-constraint satellite formations.

A Fast Approach to Satellite Range Rescheduling Using Deep Reinforcement Learning

Deep Reinforcement Learning-Based Autonomous Mission Planning Method for High and Low Orbit Multiple Agile Earth Observing Satellites

Deep Reinforcement Learning-Based Periodic Earth Observation Scheduling for Agile Satellite Constellation.

DRL-Based Dynamic Destroy Approaches for Agile-Satellite Mission Planning

Event-Triggered Deep Reinforcement Learning for Dynamic Task Scheduling in Multi-Satellite Resource Allocation

High-altitude Satellites Range Scheduling for Urgent Request Utilizing Reinforcement Learning

Learning to Construct a Solution for the Agile Satellite Scheduling Problem With Time-Dependent Transition Times

Mission Planning for Distributed Multiple Agile Earth Observing Satellites by Attention-Based Deep Reinforcement Learning Method

A Hierarchical Resource Scheduling Method for Satellite Control System Based on Deep Reinforcement Learning

Reinforcement Learning-enabled Satellite Constellation Reconfiguration and Retasking for Mission-Critical Applications

Deep Reinforcement Learning Architecture for Continuous Power Allocation in High Throughput Satellites

Deep Reinforcement Learning-Based Satellite-Ground Links Scheduling for Mega Satellite Constellations

Multi-Commodity Flow Routing for Large-Scale LEO Satellite Networks Using Deep Reinforcement Learning

Dynamic Resource Allocation With Deep Reinforcement Learning in Multibeam Satellite Communication

Data-Driven Collaborative Scheduling Method for Multi-Satellite Data-Transmission

Deep Reinforcement Learning for the Agile Earth Observation Satellite Scheduling Problem

SDDRL-SR: A High-Reliability Satellite Routing Algorithm Based on Deep Reinforcement Learning

A DRL-Based Satellite Service Allocation Method in LEO Satellite Networks

A Generic Markov Decision Process Model and Reinforcement Learning Method for Scheduling Agile Earth Observation Satellites

Reinforcement learning-based satellite formation attitude control under multi-constraint