Abstract:The escalating prevalence of formation flights in space missions has led researchers to intensify their focus on designing optimal control systems for satellite formation motion along reference orbits, with the aim of reducing tracking error and energy consumption. However, conventional controllers typically excel at optimizing only one of these objectives, and the manual parameter tuning of such controllers proves to be a challenging task. In this paper, we introduce a novel approach, the twin delayed deep deterministic policy gradient-based model predictive control (TD3-MPC) method. To tackle the multiobjective formation-keeping challenge, a linear model predictive controller based on the satellite's dynamics had been developed. Subsequently, a cost function is formulated to facilitate the optimization of multiple objectives, specifically tracking error and fuel consumption. In addressing the intricate issue of controller parameter tuning, we employ reinforcement learning and design a reward function reflective of the TD3 algorithm's controller performance. Simulation results underscore the superior performance of the proposed TD3-MPC algorithm compared to the linear model predictive controller, achieving a notable 27.83% reduction in tracking error and a substantial 48.30% decrease in fuel consumption under large error condition and 3.67% reduction in tracking error and a substantial 22.27% decrease in fuel consumption under small error condition. By effectively combining the strengths of reinforcement learning and model predictive control, TD3-MPC enhances the satellite's ability to adhere more precisely to its intended trajectory, thereby ensuring the stability and desired operational performance of the satellite formation.

Learning-based Parameter Optimization for a Class of Orbital Tracking Control Laws

Learning-Based Parameter Optimization for a Class of Orbital Tracking Control Laws

Autonomous Vehicle Control Through the Dynamics and Controller Learning

Adaptive Control of Mini Space Robot Based on Linear Separation of Inertial Parameters

Fuel Economy-Oriented Parameter Tuning for Platoon Distributed Model Predictive Control Based on Bayesian Optimization

Optimization of Control Parameters Based on Genetic Algorithms for Spacecraft Attitude Tracking with Input Constraints

Efficient safe learning for controller tuning with experimental validation

Optimization Of Control Parameters Based On Multi-Objective Genetic Algorithms For Spacecraft Attitude Tracking Control

A Nearly Optimal Control for Spacecraft Rendezvous with Constrained Controls

Design of Parameter-Self-Tuning Controller Based on Reinforcement Learning for Tracking Noncooperative Targets in Space

Control of an underactuated rigid spacecraft via deterministic learning

Parameter Optimization for a Quadrotor System with External Disturbance and Uncertainty Via Reinforcement Learning

Satellite Formation Keeping Via Real-Time Optimal Control and Iterative Learning Control

Constrained Feedback Control for Spacecraft Reorientation With an Optimal Gain

Stable Optimal Feedback Control for Landers Based on Machine Learning

Learning-Based Attitude Tracking Control with High-Performance Parameter Estimation

Optimization of the Model Predictive Control Meta-Parameters Through Reinforcement Learning

Iterative learning observer‐based fault‐tolerant control for on‐orbit serving spacecraft with coupled orbit and attitude dynamics

A Modified Robust Homotopic Method for the Low-Thrust Fuel-Optimal Orbital Control Problem

Safe Controller Optimization for Quadrotors with Gaussian Processes

TD3-Based Model Predictive Control for Satellite Formation-Keeping