DyPNIPP: Predicting Environment Dynamics for RL-based Robust Informative Path Planning

Srujan Deolasee,Siva Kailas,Wenhao Luo,Katia Sycara,Woojun Kim

2024-10-23

Abstract:Informative path planning (IPP) is an important planning paradigm for various real-world robotic applications such as environment monitoring. IPP involves planning a path that can learn an accurate belief of the quantity of interest, while adhering to planning constraints. Traditional IPP methods typically require high computation time during execution, giving rise to reinforcement learning (RL) based IPP methods. However, the existing RL-based methods do not consider spatio-temporal environments which involve their own challenges due to variations in environment characteristics. In this paper, we propose DyPNIPP, a robust RL-based IPP framework, designed to operate effectively across spatio-temporal environments with varying dynamics. To achieve this, DyPNIPP incorporates domain randomization to train the agent across diverse environments and introduces a dynamics prediction model to capture and adapt the agent actions to specific environment dynamics. Our extensive experiments in a wildfire environment demonstrate that DyPNIPP outperforms existing RL-based IPP algorithms by significantly improving robustness and performing across diverse environment conditions.

Robotics,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the lack of robustness in existing reinforcement - learning (RL) - based information path planning (IPP) algorithms in spatio - temporal varying environments. Specifically, traditional IPP methods usually require a large amount of computational time when executed, while RL - based IPP methods, although improving efficiency, perform poorly when facing dynamic environmental changes (such as changes in the spread speed of a fire). The paper points out that existing RL - based methods are mainly trained under fixed environmental dynamics, and when the environmental dynamics change, the performance of these methods will decline significantly.

To meet this challenge, the paper proposes DyPNIPP, an RL - based robust IPP framework, aiming to effectively deal with spatio - temporal environments with different dynamic characteristics. DyPNIPP improves the adaptability and robustness of the model by introducing domain randomization (DR) and a dynamic prediction model (DPM). Among them, domain randomization enables the agent to adapt to different environmental conditions by training it in diverse environments; the dynamic prediction model is used to capture and adapt to the dynamic characteristics of a specific environment, enabling the agent to adjust its behavior according to the current environmental dynamics.

The main contributions of the paper include:
1. Solving the robustness problem of RL - based information path planning in spatio - temporal environments for the first time.
2. Considering the key factors affecting robustness in the wildfire field, including fuel coefficients, vegetation coefficients and the number of fire sources.
3. Verifying the effectiveness of the proposed method under different conditions through experiments, providing an analysis of DyPNIPP, and demonstrating the actual deployment effect of the trained model.