Abstract:As the number of spacecraft in orbit continues to increase, it is becoming more challenging for human operators to manage each mission. As a result, autonomous control methods are needed to reduce this burden on operators. One method of autonomous control is Reinforcement Learning (RL), which has proven to have great success across a variety of complex tasks. For missions with multiple controlled spacecraft, or agents, it is critical for the agents to communicate and have knowledge of each other, where this information is typically given to the Neural Network Controller (NNC) as an input observation. As the number of spacecraft used for the mission increases or decreases, rather than modifying the size of the observation, this paper develops a scalable observation space that uses a constant observation size to give information on all of the other agents. This approach is similar to a lidar sensor, where determines ranges of other objects in the environment. This observation space is applied to a spacecraft inspection task, where RL is used to train multiple deputy spacecraft to cooperate and inspect a passive chief spacecraft. It is expected that the scalable observation space will allow the agents to learn to complete the task more efficiently compared to a baseline solution where no information is communicated between agents.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to achieve autonomous control through Deep Reinforcement Learning (DRL) in multi - spacecraft collaborative tasks and ensure the safety and efficiency of the tasks. Specifically, the paper mainly focuses on the following aspects: 1. **Multi - spacecraft Cooperative Inspection Tasks**: - As the number of spacecraft in orbit increases, it becomes more and more difficult for human operators to manage each task. Therefore, an autonomous control system is required to relieve the burden on operators. - This paper studies how to use Deep Reinforcement Learning (DRL) to train multiple deputy spacecraft to work together to inspect a chief spacecraft while ensuring the safety of each deputy spacecraft. 2. **Scalable Observation Space**: - In multi - agent systems, as the number of spacecraft participating in the task changes, the size of the traditional observation space will change accordingly, which requires retraining the Neural Network Controller (NNC). To solve this problem, the paper proposes a scalable observation space method, which keeps the size of the observation space unchanged regardless of the number of other spacecraft. - This method is similar to a lidar sensor, providing information by determining the distance to other objects in the environment. 3. **Run Time Assurance (RTA)**: - To prevent unsafe behaviors, RTA is used to filter control outputs and ensure safety. RTA plays a crucial role in the DRL training process, especially in high - risk application scenarios such as In - Space Assembly and Manufacturing/On - Orbit Servicing, Assembly, and Manufacturing (ISAM/OSAM) tasks. - The paper explores the impact of different RTA configurations on multi - spacecraft inspection tasks and ensures that agents can complete tasks more efficiently under different configurations. ### Specific Problem Descriptions - **Multi - spacecraft Cooperative Inspection Tasks**: Multiple deputy spacecraft need to conduct inspections around the chief spacecraft, ensuring that the inspection coverage exceeds 95% while minimizing fuel consumption (measured by ΔV). - **Safety Constraints**: Ensure that deputy spacecraft will not collide with the chief spacecraft or other deputy spacecraft, stay within the specified distance range, and avoid unsafe behaviors such as high - speed maneuvers. - **Observation Space Design**: Develop a scalable observation space so that the same NNC can be applied to different numbers of collaborative spacecraft without having to modify the size of the observation space. ### Main Contributions 1. **Solve 6 - Degree - of - Freedom (6 - DoF) Multi - spacecraft Inspection Tasks**: Use Deep Reinforcement Learning combined with RTA to ensure the safety and efficiency of the tasks. 2. **Explore and Compare Different Scalable Observation Spaces**: Evaluate the performance of these observation spaces in cases with different numbers of agents. 3. **Evaluate Training Strategies**: Evaluate the performance of the trained strategies in cases with different numbers of agents. Through these studies, the paper aims to provide an efficient and safe autonomous control method for future multi - spacecraft collaborative tasks.

Deep Reinforcement Learning for Scalable Multiagent Spacecraft Inspection

Deep Reinforcement Learning for Autonomous Spacecraft Inspection using Illumination

Deep Reinforcement Learning-Based Autonomous Mission Planning Method for High and Low Orbit Multiple Agile Earth Observing Satellites

Demonstrating Reinforcement Learning and Run Time Assurance for Spacecraft Inspection Using Unmanned Aerial Vehicles

Safe Spacecraft Inspection via Deep Reinforcement Learning and Discrete Control Barrier Functions

Run Time Assured Reinforcement Learning for Six Degree-of-Freedom Spacecraft Inspection

A Multi-agent Reinforcement Learning Method for Swarm Robots in Space Collaborative Exploration

Deep Q-Learning for Decentralized Multi-Agent Inspection of a Tumbling Target

LLMSat: A Large Language Model-Based Goal-Oriented Agent for Autonomous Space Exploration

Investigating the Impact of Choice on Deep Reinforcement Learning for Space Controls

Spacecraft Autonomous Decision-Planning for Collision Avoidance: a Reinforcement Learning Approach

Learning Efficient Multi-Agent Cooperative Visual Exploration

Multiphase Autonomous Docking via Model-Based and Hierarchical Reinforcement Learning

Hierarchical and Stable Multiagent Reinforcement Learning for Cooperative Navigation Control

Decentralized Motion Planning for Multi-Robot Navigation using Deep Reinforcement Learning

Control and Coordination of a SWARM of Unmanned Surface Vehicles using Deep Reinforcement Learning in ROS

Space-Based Sensor Tasking Using Deep Reinforcement Learning

Reinforcement learning for multi-agent formation navigation with scalability

Decentralized Multi-Agent Reinforcement Learning with Global State Prediction

Oracle-Guided Deep Reinforcement Learning for Large-Scale Multi-UAVs Flocking and Navigation.

MASP: Scalable GNN-based Planning for Multi-Agent Navigation