Abstract:The stratospheric airship, as a near-space vehicle, is increasingly utilized in scientific exploration and Earth observation due to its long endurance and regional observation capabilities. However, due to the complex characteristics of the stratospheric wind field environment, trajectory planning for stratospheric airships is a significant challenge. Unlike lower atmospheric levels, the stratosphere presents a wind field characterized by significant variability in wind speed and direction, which can drastically affect the stability of the airship's trajectory. Recent advances in deep reinforcement learning (DRL) have presented promising avenues for trajectory planning. DRL algorithms have demonstrated the ability to learn complex control strategies autonomously by interacting with the environment. In particular, the proximal policy optimization (PPO) algorithm has shown effectiveness in continuous control tasks and is well suited to the non-linear, high-dimensional problem of trajectory planning in dynamic environments. This paper proposes a trajectory planning method for stratospheric airships based on the PPO algorithm. The primary contributions of this paper include establishing a continuous action space model for stratospheric airship motion; enabling more precise control and adjustments across a broader range of actions; integrating time-varying wind field data into the reinforcement learning environment; enhancing the policy network's adaptability and generalization to various environmental conditions; and enabling the algorithm to automatically adjust and optimize flight paths in real time using wind speed information, reducing the need for human intervention. Experimental results show that, within its wind resistance capability, the airship can achieve long-duration regional station-keeping, with a maximum station-keeping time ratio (STR) of up to 0.997.

Trajectory Planning Based on Continuous Decision Deep Reinforcement Learning for Stratospheric Airship

Autonomous Trajectory Planning Method for Stratospheric Airship Regional Station-Keeping Based on Deep Reinforcement Learning

Path planning of stratospheric airship in dynamic wind field based on deep reinforcement learning

Model-Free Control for Stratospheric Airship Based on Reinforcement Learning

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Trajectory control method of stratospheric airships based on model predictive control in wind field

Deep Reinforcement Learning Based Trajectory Real-Time Planning for Hypersonic Gliding Vehicles

Trajectory Planning for Airborne Radar in Extended Target Tracking Based on Deep Reinforcement Learning

Online Trajectory Planning Method for Midcourse Guidance Phase Based on Deep Reinforcement Learning

Trajectory Planning with Deep Reinforcement Learning in High-Level Action Spaces

Three-Dimensional Path-Following Control of a Robotic Airship with Reinforcement Learning

Trajectory Planning of Stratospheric Airship for Station-Keeping Mission Based on Improved Rapidly Exploring Random Tree

Integration of Path Planning and Following Control for the Stratospheric Airship with Forecasted Wind Field Data.

Resident Trajectory Optimization for Stratospheric Airships

Improved Twin Delayed Deep Deterministic Policy Gradient Algorithm Based Real-Time Trajectory Planning for Parafoil under Complicated Constraints

Optimization of Stratospheric Airship Station-Keeping Strategy Based on LSTM-DQN Algorithm

Hypersonic Vehicle Control Based on Deep Reinforcement Learning

Intelligent Land Vehicle Model Transfer Trajectory Planning Method Based on Deep Reinforcement Learning

Research on trajectory planning based on reinforcement learning algorithm of deep deterministic policy gradient

Prescribed Performance Event-Triggered Trajectory Tracking Control for Stratospheric Airship

Unmanned Aerial Vehicle Trajectory Planning Via Staged Reinforcement Learning