Abstract:Multi-rotor UAVs suffer from a restricted range and flight duration due to limited battery capacity. Autonomous landing on a 2D moving platform offers the possibility to replenish batteries and offload data, thus increasing the utility of the vehicle. Classical approaches rely on accurate, complex and difficult-to-derive models of the vehicle and the environment. Reinforcement learning (RL) provides an attractive alternative due to its ability to learn a suitable control policy exclusively from data during a training procedure. However, current methods require several hours to train, have limited success rates and depend on hyperparameters that need to be tuned by trial-and-error. We address all these issues in this work. First, we decompose the landing procedure into a sequence of simpler, but similar learning tasks. This is enabled by applying two instances of the same RL based controller trained for 1D motion for controlling the multi-rotor's movement in both the longitudinal and the lateral directions. Second, we introduce a powerful state space discretization technique that is based on i) kinematic modeling of the moving platform to derive information about the state space topology and ii) structuring the training as a sequential curriculum using transfer learning. Third, we leverage the kinematics model of the moving platform to also derive interpretable hyperparameters for the training process that ensure sufficient maneuverability of the multi-rotor vehicle. The training is performed using the tabular RL method Double Q-Learning. Through extensive simulations we show that the presented method significantly increases the rate of successful landings, while requiring less training time compared to other deep RL approaches. Finally, we deploy and demonstrate our algorithm on real hardware. For all evaluation scenarios we provide statistics on the agent's performance.

Reinforcement Learning and Sim-to-Real Transfer of Reorientation and Landing Control for Quadruped Robots on Asteroids

Integrated Attitude and Landing Control for Quadruped Robots in Asteroid Landing Mission Scenarios Using Reinforcement Learning

Reinforcement learning-based stable jump control method for asteroid-exploration quadruped robots

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Reinforcement Learning based Control of a Quadruped Robot

Nonlinear optimal control with disturbance rejection for asteroid landing.

An advanced reinforcement learning control method for quadruped robots in typical urban terrains

A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Platform

Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured Environment

Cat-like Jumping and Landing of Legged Robots in Low-gravity Using Deep Reinforcement Learning

Concurrent Learning-Based Trajectory Tracking Control of a Novel Multi-Node Flexible Lander for Asteroid Missions

Real-time Replanning and Control of Quadruped Robots for Blind Locomotion on Uneven Terrain

Real-time optimal control for irregular asteroid landings using deep neural networks

Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning

Reinforcement Learning based Autonomous Multi-Rotor Landing on Moving Platforms

Landing Control Method of a Lightweight Four-Legged Landing and Walking Robot

Six degree-of-freedom body-fixed hovering over unmapped asteroids via LIDAR altimetry and reinforcement meta-learning

Design and Soft-Landing Control of a Six-Legged Mobile Repetitive Lander for Lunar Exploration

Dynamic Surface Robust Guidance and Control for Landing on Asteroids