Abstract:Landing a quadrotor on a mobile platform moving with various unknown trajectories presents special challenges, including the requirements of fast trajectory planning/replanning, accurate control, and the adaptability for different target trajectories, especially when the platform is non-cooperative. However, previous works either assume the platform moves along a predefined trajectory or decouple planning from control which may cause a delay in tracking. In this work, we integrate planning and control into a unified framework and present an efficient off-policy Meta-Reinforcement Learning (Meta-RL) algorithm that enables a quadrotor (agent) to land on a mobile platform with various unknown trajectories autonomously. In our approach, we disentangle task-specific policy parameters by a separate adapter network to shared low-level parameters and learn a probabilistic encoder to extract common structures across different tasks. Specifically, during meta-training, we sample different trajectories from the task distribution, and then the probabilistic encoder accumulates the necessary statistics from past experience into the latent variables that enable the policy to perform the task. At meta-testing time, when the quadrotor is faced with an unseen trajectory, the latent variables can be sampled according to past interactions between the quadrotor and the mobile platform and held constant during an episode, enabling rapid trajectory-level adaptation. We assume similar tasks share a common low-dimensional structure in the representation of the policy network and the task-specific information is learned in the head of the policy. Accordingly, we further propose a separate adapter net as a supervised learning problem. The adapter net learns the weights of the policy’s output layer for each meta-training task given by the environment interactions from the agent. When adapting to a new task during meta-testing, we fix the shared model layers and predict the head weights for the new task using the trained adapter network. This ensures that the pretrained policy can efficiently adapt to different tasks, which boosts the out-of-distribution performance. Our method can directly control the pitch, roll, yaw angle, and thrust of the quadrotor, yielding a fast response to the trajectory change. Simulation results show the superiority of our method both in success rate and adaptation efficiency over other RL algorithms on meta-testing tasks. The real-world experimental results compared with traditional planning and control algorithms demonstrate the satisfactory performance of our autonomous landing method, especially its robustness in adapting to unknown dynamics. Note to Practitioners—Given the challenge posed by the motion uncertainty when a quadrotor lands on a mobile platform with an unknown trajectory, there hasn’t been a well-established solution, as far as we know. This paper introduces meta-reinforcement learning, incorporating a latent variable encoder to extract common features from training tasks, and designing an adapter network to enhance the ability of policy networks to adapt to new tasks, thereby enhancing the landing performance of the agent. The proposed method demonstrates promising results in both simulation and experiments.

Inverted Landing in a Small Aerial Robot Via Deep Reinforcement Learning for Triggering and Control of Rotational Maneuvers.

From Flies to Robots: Inverted Landing in Small Quadcopters with Dynamic Perching

Optimal Inverted Landing in a Small Aerial Robot with Varied Approach Velocities and Landing Gear Designs

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Precision Landing of Autonomous Parafoil System Via Deep Reinforcement Learning

A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Platform

Lander.AI: Adaptive Landing Behavior Agent for Expertise in 3D Dynamic Platform Landings

Reinforcement learning based autonomous multi-rotor landing on moving platforms

Deep Reinforcement Learning Automatic Landing Control of Fixed-Wing Aircraft Using Deep Deterministic Policy Gradient

UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy

SmoothLander: A Quadrotor Landing Control System with Smooth Trajectory Guarantee Based on Reinforcement Learning

Evolved Neuromorphic Control for High Speed Divergence-based Landings of MAVs

Learning to Fly via Deep Model-Based Reinforcement Learning

Learning to Fly Omnidirectional Micro Aerial Vehicles with an End-To-End Control Network

Air Learning: A Deep Reinforcement Learning Gym for Autonomous Aerial Robot Visual Navigation

Reinforcement Learning for Shared Autonomy Drone Landings

Towards better generalization in quadrotor landing using deep reinforcement learning

Deep Reinforcement Learning for End-to-End Local Motion Planning of Autonomous Aerial Robots in Unknown Outdoor Environments: Real-Time Flight Experiments

Cat-like Jumping and Landing of Legged Robots in Low-gravity Using Deep Reinforcement Learning

Autonomous Landing of the Quadrotor on the Mobile Platform via Meta Reinforcement Learning