Shrinking POMCP: A Framework for Real-Time UAV Search and Rescue

Yunuo Zhang,Baiting Luo,Ayan Mukhopadhyay,Daniel Stojcsics,Daniel Elenius,Anirban Roy,Susmit Jha,Miklos Maroti,Xenofon Koutsoukos,Gabor Karsai,Abhishek Dubey

2024-11-20

Abstract:Efficient path optimization for drones in search and rescue operations faces challenges, including limited visibility, time constraints, and complex information gathering in urban environments. We present a comprehensive approach to optimize UAV-based search and rescue operations in neighborhood areas, utilizing both a 3D AirSim-ROS2 simulator and a 2D simulator. The path planning problem is formulated as a partially observable Markov decision process (POMDP), and we propose a novel ``Shrinking POMCP'' approach to address time constraints. In the AirSim environment, we integrate our approach with a probabilistic world model for belief maintenance and a neurosymbolic navigator for obstacle avoidance. The 2D simulator employs surrogate ROS2 nodes with equivalent functionality. We compare trajectories generated by different approaches in the 2D simulator and evaluate performance across various belief types in the 3D AirSim-ROS simulator. Experimental results from both simulators demonstrate that our proposed shrinking POMCP solution achieves significant improvements in search times compared to alternative methods, showcasing its potential for enhancing the efficiency of UAV-assisted search and rescue operations.

Robotics,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to efficiently plan the drone paths when using drones for search and rescue operations in urban environments, in order to overcome the challenges of limited visibility, time constraints, and complex information - gathering. Specifically, the paper proposes a new method named "Shrinking POMCP" for optimizing drone path planning under the framework of partially observable Markov decision processes (POMDP).

### Problem Background
In search and rescue (SAR) tasks, especially in urban environments, unmanned aerial vehicles (UAVs) can provide the advantages of rapid deployment, long - flight duration, and access to hard - to - reach areas. However, the main challenges that drones face in these tasks include:
- **Limited battery life**: Drones need to complete tasks within a limited time.
- **Changing weather conditions**: Weather changes may affect the sensor performance and flight capabilities of drones.
- **Incomplete information**: Information such as the location of targets in the search area and environmental conditions is often uncertain.

### Problem Definition
The paper formalizes the drone path - planning problem as a partially observable Markov decision process (POMDP). POMDP is a mathematical framework suitable for sequential decision - making problems in uncertain environments, where the state of the system is not fully observable. Specifically, POMDP consists of the following elements:
- **State space \( S \)**: It includes the position of the drone and the positions of all targets.
- **Action space \( A \)**: Actions that the drone can perform, such as moving in four directions.
- **Transition function \( T(s' | s, a) \)**: Represents the probability of transitioning to state \( s' \) after performing action \( a \) in state \( s \).
- **Reward function \( R(s, a) \)**: Used to evaluate the goodness or badness of each action.
- **Observation space \( O \)**: Partial information obtained by the drone through sensors.
- **Observation function \( Z(o | s', a) \)**: Represents the probability of observing \( o \) after performing action \( a \) in state \( s' \).

### Solution
The paper proposes a method named "Shrinking POMCP", which optimizes path planning through the following steps:
1. **Constructing the belief tree**: Use the Monte Carlo tree search (MCTS) algorithm to construct a belief tree, where nodes represent belief states and edges represent actions.
2. **Selecting the best action**: Use the upper - confidence - bound - for - trees (UCT) strategy to select the best action, balancing exploration and exploitation.
3. **Expanding the tree**: If the selected action leads to an unexplored observation, expand the tree.
4. **Simulation**: Generate the next state, observation, and reward from the current state and action.
5. **Backtracking**: Backtrack the simulation results to the root node of the tree and update the statistical information.
6. **Dynamically shrinking the decision space**: Guide the drone to move to the next best non - sparse area and avoid target - sampling oscillation.

### Experimental Verification
The paper conducted experiments in the AirSim - ROS2 simulator and 2D simulator to verify the effectiveness of the Shrinking POMCP method. The experimental results show that, compared with traditional methods, Shrinking POMCP significantly reduces the search time and improves the efficiency of drones in search and rescue tasks.

In conclusion, this paper aims to solve the path - planning problems faced by drones in search and rescue operations in urban environments through the Shrinking POMCP method, and improve the efficiency and success rate of tasks.