Abstract:This paper investigates an air-ground cooperative monitoring problem for spatiotemporal fields, such as air pollution, forest fires, oil spills, etc, with unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs). To fully exploit complementarities of these heterogeneous vehicles and improve efficiency of the cooperative monitoring, we design a novel cooperation scheme: each UAV is assigned to loiter over and transmit its observations to a pre-allocated UGV, and the UGV provides guidance on informative trajectories for the UAV and aims to reach a target position as fast as possible. Such a scheme brings challenges to informative trajectory planning of the UGVs, lying in the delayed observations from the UAVs and the cumulative information constraint depending on the unknown field. To overcome them, this work proposes a model-free reinforcement learning (RL)-based trajectory planning method to learn continuous policies for the UGVs, where a field estimator is designed for each UGV to recover observability of the field. In addition, we derive model predictive control (MPC)-based trajectory planners for the UAVs with tailored reference positions, where the uncertain tracking errors can be handled by the RL-based method of the UGVs. Thus, a performance coupling problem of the heterogeneous vehicles is tackled. Simulations illustrate the effectiveness of the proposed trajectory planning methods and the efficiency of the air-ground cooperative monitoring scheme. Note to Practitioners —This article is motivated by cooperative monitoring tasks with UAVs and UGVs in practical applications, such as environmental monitoring, search and rescue after disasters, etc. Due to the complex dynamics of spatiotemporal fields in these tasks, trajectory planning for the cooperative monitoring system is challenging and requires much computations. To resolve the issues, we propose a novel cooperation scheme in this article, where the large computational capability of the UGVs is utilized to solve a minimum-time trajectory planning problem under a cumulative information constraint, and the UAVs only loiter over and transmit measurements about the field to the UGVs. To achieve this scheme, RL-based and MPC-based trajectory planning methods are proposed for the UGVs and the UAVs, respectively. Simulations have validated the effectiveness of the proposed trajectory planning methods and good performance of the cooperative monitoring system.

Informative Trajectory Planning Using Reinforcement Learning for Minimum-Time Exploration of Spatiotemporal Fields

Pontryagin's Minimum Principle-Guided RL for Minimum-Time Exploration of Spatiotemporal Fields

A Novel Trajectory Planning Method Based on Trust Region Policy Optimization

Pontryagin’s Minimum Principle-Guided RL for Minimum-Time Exploration of Spatiotemporal Fields

Informative Trajectory Planning for Air-Ground Cooperative Monitoring of Spatiotemporal Fields

Reinforcement Learning Based Trajectory Planning for Autonomous Vehicles

Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning

On-Road Trajectory Planning With Spatio-Temporal Rrt* And Always-Feasible Quadratic Program

Adaptive Selection of Informative Path Planning Strategies via Reinforcement Learning

Motion Planner with Fixed-Horizon Constrained Reinforcement Learning for Complex Autonomous Driving Scenarios

Trajectory Planning for Autonomous Vehicle Using Iterative Reward Prediction in Reinforcement Learning

Off-road Autonomous Vehicles Traversability Analysis and Trajectory Planning Based on Deep Inverse Reinforcement Learning

Design and Experimental Validation of Deep Reinforcement Learning-Based Fast Trajectory Planning and Control for Mobile Robot in Unknown Environment

Trajectory Planning with Deep Reinforcement Learning in High-Level Action Spaces

Informative Path Planning for Mobile Sensing with Reinforcement Learning

Safe Trajectory Planning Using Reinforcement Learning for Self Driving

Where to Look Next: Learning Viewpoint Recommendations for Informative Trajectory Planning

Adaptive Informative Path Planning Using Deep Reinforcement Learning for UAV-based Active Sensing

Deep Reinforcement Learning Based Trajectory Planning Under Uncertain Constraints

An Efficient Spatial-Temporal Trajectory Planner for Autonomous Vehicles in Unstructured Environments

Intelligent Vehicle Decision-Making and Trajectory Planning Method Based on Deep Reinforcement Learning in the Frenet Space