Abstract:Landing a quadrotor on a mobile platform moving with various unknown trajectories presents special challenges, including the requirements of fast trajectory planning/replanning, accurate control, and the adaptability for different target trajectories, especially when the platform is non-cooperative. However, previous works either assume the platform moves along a predefined trajectory or decouple planning from control which may cause a delay in tracking. In this work, we integrate planning and control into a unified framework and present an efficient off-policy Meta-Reinforcement Learning (Meta-RL) algorithm that enables a quadrotor (agent) to land on a mobile platform with various unknown trajectories autonomously. In our approach, we disentangle task-specific policy parameters by a separate adapter network to shared low-level parameters and learn a probabilistic encoder to extract common structures across different tasks. Specifically, during meta-training, we sample different trajectories from the task distribution, and then the probabilistic encoder accumulates the necessary statistics from past experience into the latent variables that enable the policy to perform the task. At meta-testing time, when the quadrotor is faced with an unseen trajectory, the latent variables can be sampled according to past interactions between the quadrotor and the mobile platform and held constant during an episode, enabling rapid trajectory-level adaptation. We assume similar tasks share a common low-dimensional structure in the representation of the policy network and the task-specific information is learned in the head of the policy. Accordingly, we further propose a separate adapter net as a supervised learning problem. The adapter net learns the weights of the policy’s output layer for each meta-training task given by the environment interactions from the agent. When adapting to a new task during meta-testing, we fix the shared model layers and predict the head weights for the new task using the trained adapter network. This ensures that the pretrained policy can efficiently adapt to different tasks, which boosts the out-of-distribution performance. Our method can directly control the pitch, roll, yaw angle, and thrust of the quadrotor, yielding a fast response to the trajectory change. Simulation results show the superiority of our method both in success rate and adaptation efficiency over other RL algorithms on meta-testing tasks. The real-world experimental results compared with traditional planning and control algorithms demonstrate the satisfactory performance of our autonomous landing method, especially its robustness in adapting to unknown dynamics. Note to Practitioners—Given the challenge posed by the motion uncertainty when a quadrotor lands on a mobile platform with an unknown trajectory, there hasn’t been a well-established solution, as far as we know. This paper introduces meta-reinforcement learning, incorporating a latent variable encoder to extract common features from training tasks, and designing an adapter network to enhance the ability of policy networks to adapt to new tasks, thereby enhancing the landing performance of the agent. The proposed method demonstrates promising results in both simulation and experiments.

Adaptive Guidance and Integrated Navigation with Reinforcement Meta-Learning

Integrated and Adaptive Guidance and Control for Endoatmospheric Missiles via Reinforcement Learning

UAV Maneuvering Target Tracking in Uncertain Environments Based on Deep Reinforcement Learning and Meta-Learning

Line of Sight Curvature for Missile Guidance using Reinforcement Meta-Learning

Seeker based Adaptive Guidance via Reinforcement Meta-Learning Applied to Asteroid Close Proximity Operations

Autonomous Navigation of Unmanned Vehicle Through Deep Reinforcement Learning

Learning With Stochastic Guidance for Robot Navigation

Precision Landing of Autonomous Parafoil System Via Deep Reinforcement Learning

Self-Learning Robot Autonomous Navigation with Deep Reinforcement Learning Techniques

Autonomous Learning and Navigation of Mobile Robots Based on Deep Reinforcement Learning

Double Critic Deep Reinforcement Learning for Mapless 3D Navigation of Unmanned Aerial Vehicles

Navigation in Unknown Dynamic Environments Based on Deep Reinforcement Learning

Subgoal-Driven Navigation in Dynamic Environments Using Attention-Based Deep Reinforcement Learning

Autonomous Landing of the Quadrotor on the Mobile Platform via Meta Reinforcement Learning

Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning

Parallel Distributional Deep Reinforcement Learning for Mapless Navigation of Terrestrial Mobile Robots

Intelligent Diving Guidance with Terminal Angle and Velocity Constraints Via Deep Reinforcement Learning

Reinforcement Meta-Learning for Interception of Maneuvering Exoatmospheric Targets with Parasitic Attitude Loop

Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians

Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL

Vision-Based Deep Reinforcement Learning of UAV-UGV Collaborative Landing Policy Using Automatic Curriculum