Abstract:Landing a quadrotor on a mobile platform moving with various unknown trajectories presents special challenges, including the requirements of fast trajectory planning/replanning, accurate control, and the adaptability for different target trajectories, especially when the platform is non-cooperative. However, previous works either assume the platform moves along a predefined trajectory or decouple planning from control which may cause a delay in tracking. In this work, we integrate planning and control into a unified framework and present an efficient off-policy Meta-Reinforcement Learning (Meta-RL) algorithm that enables a quadrotor (agent) to land on a mobile platform with various unknown trajectories autonomously. In our approach, we disentangle task-specific policy parameters by a separate adapter network to shared low-level parameters and learn a probabilistic encoder to extract common structures across different tasks. Specifically, during meta-training, we sample different trajectories from the task distribution, and then the probabilistic encoder accumulates the necessary statistics from past experience into the latent variables that enable the policy to perform the task. At meta-testing time, when the quadrotor is faced with an unseen trajectory, the latent variables can be sampled according to past interactions between the quadrotor and the mobile platform and held constant during an episode, enabling rapid trajectory-level adaptation. We assume similar tasks share a common low-dimensional structure in the representation of the policy network and the task-specific information is learned in the head of the policy. Accordingly, we further propose a separate adapter net as a supervised learning problem. The adapter net learns the weights of the policy’s output layer for each meta-training task given by the environment interactions from the agent. When adapting to a new task during meta-testing, we fix the shared model layers and predict the head weights for the new task using the trained adapter network. This ensures that the pretrained policy can efficiently adapt to different tasks, which boosts the out-of-distribution performance. Our method can directly control the pitch, roll, yaw angle, and thrust of the quadrotor, yielding a fast response to the trajectory change. Simulation results show the superiority of our method both in success rate and adaptation efficiency over other RL algorithms on meta-testing tasks. The real-world experimental results compared with traditional planning and control algorithms demonstrate the satisfactory performance of our autonomous landing method, especially its robustness in adapting to unknown dynamics. Note to Practitioners—Given the challenge posed by the motion uncertainty when a quadrotor lands on a mobile platform with an unknown trajectory, there hasn’t been a well-established solution, as far as we know. This paper introduces meta-reinforcement learning, incorporating a latent variable encoder to extract common features from training tasks, and designing an adapter network to enhance the ability of policy networks to adapt to new tasks, thereby enhancing the landing performance of the agent. The proposed method demonstrates promising results in both simulation and experiments.

Integrated Attitude and Landing Control for Quadruped Robots in Asteroid Landing Mission Scenarios Using Reinforcement Learning

Reinforcement Learning and Sim-to-Real Transfer of Reorientation and Landing Control for Quadruped Robots on Asteroids

Reinforcement learning-based stable jump control method for asteroid-exploration quadruped robots

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Meta Reinforcement Learning of Locomotion Policy for Quadruped Robots with Motor Stuck

Nonlinear MPC-Based Control Framework for Quadruped Robots: Touch-Down in Complex Terrain

Meta-Reinforcement Learning of Hierarchical Fault-Tolerant Controller for Multiple Leg Failures in Hexapod Robots

Adaptive Locomotion Control of a Hexapod Robot Via Bio-Inspired Learning

Reinforcement Learning based Control of a Quadruped Robot

Nonlinear optimal control with disturbance rejection for asteroid landing.

Concurrent Learning-Based Trajectory Tracking Control of a Novel Multi-Node Flexible Lander for Asteroid Missions

Optimal Attitude Control for Landing on Asteroid with a Flexible Lander

Real-time optimal control for irregular asteroid landings using deep neural networks

Survey of Autonomous Guidance Methods for Powered Planetary Landing

Autonomous Landing of the Quadrotor on the Mobile Platform via Meta Reinforcement Learning

Landing Control Method of a Lightweight Four-Legged Landing and Walking Robot

An advanced reinforcement learning control method for quadruped robots in typical urban terrains

Real-time Replanning and Control of Quadruped Robots for Blind Locomotion on Uneven Terrain

Cat-like Jumping and Landing of Legged Robots in Low-gravity Using Deep Reinforcement Learning

Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured Environment