Abstract:As the complexity of space missions increases, the constraints on satellite attitude control become more stringent, particularly for satellites working in orbit formation. This paper introduces a novel method, based on the categorization and modeling of different constraints, for attitude control of satellite formations under multiple constraints. The method employs the Phased Priority Reinforcement Learning (PPRL) approach, which utilizes Deep Deterministic Policy Gradient (DDPG) technology. Considering the complexity of constraints and the challenge posed by the high control dimensionality due to multi-satellite coordination, the method addresses these challenges through a two-step training strategy. The first step addresses the multi-constraint issue for individual satellites and increases the priority of single-satellite training experience data in the experience replay buffer of the second step to enhance data utilization efficiency. To address the issue of reward sparsity in complex high-dimensional constraint models, a detailed reward mechanism is proposed, incorporating both local and global constraints into the reward function, thereby achieving both efficient and effective attitude control. This approach not only meets dynamic, state, and performance constraints but also demonstrates adaptability and robustness through numerical simulations. Compared to traditional methods, this approach achieves significant improvements in control performance and constraint satisfaction, offering a novel solution pathway for high-dimensional control problems in multi-constraint satellite formations.

Model-based deep reinforcement learning with heuristic search for satellite attitude control

Satellite Attitude Tracking Control of Moving Targets Combining Deep Reinforcement Learning and Predefined-time Stability Considering Energy Optimization

Satellite Attitude Tracking Decision Method based on Deep Deterministic Policy Gradient for Moving Target Observation

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Spacecraft Attitude Maneuver Planning Based on Deep Reinforcement Learning under Complex Constraints

A General Technique To Combine Off-Policy Reinforcement Learning Algorithms With Satellite Attitude Control

Deep Reinforcement Learning-Based Autonomous Mission Planning Method for High and Low Orbit Multiple Agile Earth Observing Satellites

Adaptive Prediction Backstepping Attitude Control for Liquid-Filled Micro-Satellite with Flexible Appendages

Satellite fault tolerant attitude control based on expert guided exploration of reinforcement learning agent

Reinforcement learning strategy for spacecraft attitude hyperagile tracking control with uncertainties

Adaptive satellite attitude control for varying masses using deep reinforcement learning

Application of Deep Reinforcement Learning in UAVs: A Review

Reinforcement learning-based satellite formation attitude control under multi-constraint

Large-Scale Solar-Powered UAV Attitude Control Using Deep Reinforcement Learning in Hardware-in-Loop Verification

Neural network-based reinforcement learning control for combined spacecraft attitude tracking maneuvers

Robust control for satellite attitude regulation during on-orbit assembly

A Hierarchical Resource Scheduling Method for Satellite Control System Based on Deep Reinforcement Learning

Deep Reinforcement Learning with Local Attention for Single Agile Optical Satellite Scheduling Problem

Autonomous UAV Navigation with Adaptive Control Based on Deep Reinforcement Learning

Model-Reference Reinforcement Learning Control of Autonomous Surface Vehicles with Uncertainties

Reinforcement Learning for UAV Attitude Control