Abstract:Active tracking control is essential for UAVs to perform autonomous operations in GPS-denied environments. In the active tracking task, UAVs take high-dimensional raw images as input and execute motor actions to actively follow the dynamic target. Most research focuses on three-stage methods, which entail perception first, followed by high-level decision-making based on extracted spatial information of the dynamic target, and then UAV movement control, using a low-level dynamic controller. Perception methods based on deep neural networks are powerful but require considerable effort for manual ground truth labeling. Instead, we unify the perception and decision-making stages using a high-level controller and then leverage deep reinforcement learning to learn the mapping from raw images to the high-level action commands in the V-REP-based environment, where simulation data are infinite and inexpensive. This end-to-end method also has the advantages of a small parameter size and reduced effort requirements for parameter turning in the decision-making stage. The high-level controller, which has a novel architecture, explicitly encodes the spatial and temporal features of the dynamic target. Auxiliary segmentation and motion-in-depth losses are introduced to generate denser training signals for the high-level controller's fast and stable training. The high-level controller and a conventional low-level PID controller constitute our hierarchical active tracking control framework for the UAVs' active tracking task. Simulation experiments show that our controller trained with several augmentation techniques sufficiently generalizes dynamic targets with random appearances and velocities, and achieves significantly better performance, compared with three-stage methods.

End-to-end UAV Intelligent Training via Deep Reinforcement Learning

UAV Visual Flight Control Method Based on Deep Reinforcement Learning

End-to-End Deep Reinforcement Learning for Image-Based UAV Autonomous Control

UAV Target Tracking Method Based on Deep Reinforcement Learning

Robust Motion Control for UAV in Dynamic Uncertain Environments Using Deep Reinforcement Learning.

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV's Autonomous Motion Planning in Complex Unknown Environments

Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking

A Learning-Based Flexible Autonomous Motion Control Method for UAV in Dynamic Unknown Environments

An Improved Method towards Multi-UAV Autonomous Navigation Using Deep Reinforcement Learning

An End-to-End Deep Reinforcement Learning Method for UAV Autonomous Motion Planning

End-to-End Learning-Based Obstacle Avoidance for Fixed-Wing UAVs

Relevant experience learning: A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy

Obstacle Avoidance for UAS in Continuous Action Space Using Deep Reinforcement Learning

Deep Reinforcement Learning for UAV Intelligent Mission Planning

Autonomous Navigation of UAV in Large-Scale Unknown Complex Environment with Deep Reinforcement Learning.

Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning

Monocular vision guided deep reinforcement learning UAV systems with representation learning perception

A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidance

Representation Enhancement-Based Proximal Policy Optimization for UAV Path Planning and Obstacle Avoidance

UAV Autonomous Navigation Based on Multi-modal Perception: A Deep Hierarchical Reinforcement Learning Method