Abstract:Rocket recycling is a crucial pursuit in aerospace technology, aimed at reducing costs and environmental impact in space exploration. The primary focus centers on rocket landing control, involving the guidance of a nonlinear underactuated rocket with limited fuel in real-time. This challenging task prompts the application of reinforcement learning (RL), yet goal-oriented nature of the problem poses difficulties for standard RL algorithms due to the absence of intermediate reward signals. This paper, for the first time, significantly elevates the success rate of rocket landing control from 8% with a baseline controller to 97% on a high-fidelity rocket model using RL. Our approach, called Random Annealing Jump Start (RAJS), is tailored for real-world goal-oriented problems by leveraging prior feedback controllers as guide policy to facilitate environmental exploration and policy learning in RL. In each episode, the guide policy navigates the environment for the guide horizon, followed by the exploration policy taking charge to complete remaining steps. This jump-start strategy prunes exploration space, rendering the problem more tractable to RL algorithms. The guide horizon is sampled from a uniform distribution, with its upper bound annealing to zero based on performance metrics, mitigating distribution shift and mismatch issues in existing methods. Additional enhancements, including cascading jump start, refined reward and terminal condition, and action smoothness regulation, further improve policy performance and practical applicability. The proposed method is validated through extensive evaluation and Hardware-in-the-Loop testing, affirming the effectiveness, real-time feasibility, and smoothness of the proposed controller.

Vanishing Bias Heuristic-guided Reinforcement Learning Algorithm

A Dynamic Adjusting Reward Function Method for Deep Reinforcement Learning with Adjustable Parameters

Deep Reinforcement Learning with Double Q-Learning

Integrated Double Estimator Architecture for Reinforcement Learning

Deep Reinforcement Learning for Unpredictability-Induced Rewards to Handle Spacecraft Landing

CM-DQN: A Value-Based Deep Reinforcement Learning Model to Simulate Confirmation Bias

State of the Art Control of Atari Games Using Shallow Reinforcement Learning

Multi-Input Autonomous Driving Based on Deep Reinforcement Learning with Double Bias Experience Replay

Data Efficient Deep Reinforcement Learning with Action-Ranked Temporal Difference Learning

A Novel Heterogeneous Actor-critic Algorithm with Recent Emphasizing Replay Memory

A Novel Adaptive Sampling Strategy for Deep Reinforcement Learning.

Double A3C: Deep Reinforcement Learning on OpenAI Gym Games

A Human Mixed Strategy Approach to Deep Reinforcement Learning

Learning Complicated Navigation Skills from Limited Experience Via Augmenting Offline Datasets

RETRACTION: Integrated Double Estimator Architecture for Reinforcement Learning (retraction of 2020)

Rocket Landing Control with Random Annealing Jump Start Reinforcement Learning

HER-PDQN: A Reinforcement Learning Approach for UAV Navigation with Hybrid Action Spaces and Sparse Rewards

Mobile Robot Sequential Decision Making Using a Deep Reinforcement Learning Hyper-Heuristic Approach

Episodic Reinforcement Learning with Expanded State-reward Space

A Methodology Based on Deep Reinforcement Learning to Autonomous Driving with Double Q-Learning

LIDAR: Learning from Imperfect Demonstrations with Advantage Rectification