Collaborative Route Planning of UAVs, Workers and Cars for Crowdsensing in Disaster Response

Lei Han,Chunyu Tu,Zhiwen Yu,Zhiyong Yu,Weihua Shan,Liang Wang,Bin Guo

2023-08-22

Abstract:Efficiently obtaining the up-to-date information in the disaster-stricken area is the key to successful disaster response. Unmanned aerial vehicles (UAVs), workers and cars can collaborate to accomplish sensing tasks, such as data collection, in disaster-stricken areas. In this paper, we explicitly address the route planning for a group of agents, including UAVs, workers, and cars, with the goal of maximizing the task completion rate. We propose MANF-RL-RP, a heterogeneous multi-agent route planning algorithm that incorporates several efficient designs, including global-local dual information processing and a tailored model structure for heterogeneous multi-agent systems. Global-local dual information processing encompasses the extraction and dissemination of spatial features from global information, as well as the partitioning and filtering of local information from individual agents. Regarding the construction of the model structure for heterogeneous multi-agent, we perform the following work. We design the same data structure to represent the states of different agents, prove the Markovian property of the decision-making process of agents to simplify the model structure, and also design a reasonable reward function to train the model. Finally, we conducted detailed experiments based on the rich simulation data. In comparison to the baseline algorithms, namely Greedy-SC-RP and MANF-DNN-RP, MANF-RL-RP has exhibited a significant improvement in terms of task completion rate.

Artificial Intelligence,Multiagent Systems

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to efficiently obtain the latest information in the disaster - stricken areas during disaster response. Specifically, it studies the path - planning problem of unmanned aerial vehicles (UAVs), workers, and cars in the collaborative completion of sensing tasks (such as data collection) to maximize the task completion rate.

### Problem Background

After a disaster occurs, obtaining the latest information in the disaster - stricken areas in a timely manner is crucial for successful rescue operations. However, traditional mobile crowdsourcing sensing (MCS) has limitations in complex and dangerous disaster - stricken environments, mainly in the following aspects:

1. **Environmental Complexity**: The environment in the disaster - stricken area is complex, which restricts the mobility of participants.
2. **Sensing Capacity Requirements**: High - precision and specific sensing capabilities are required, while traditional MCS relies on participants and their mobile devices as basic sensing units, making it difficult to meet these requirements.
3. **Limitations of UAV Autonomous Operations**:
- **Safety Hazards in Low - Altitude Environments**: UAVs face many safety issues when performing tasks in low - altitude environments and require precise manipulation.
- **Availability of Charging Stations**: Existing research assumes that UAVs can autonomously navigate to charging stations for charging, but in fact, the availability of charging stations is limited, especially in urban environments after disasters.

### Solution

To solve the above problems, this paper proposes a multi - agent reinforcement learning algorithm named MANF - RL - RP for the collaborative path - planning of UAVs, workers, and cars. The specific objectives are as follows:

1. **Maximize the Task Completion Rate**: By optimizing path - planning, ensure that as many sensing tasks as possible are completed.
2. **Overcome the Limitations of UAV Autonomous Operations**:
- Workers are responsible for precisely operating UAVs at sensing task locations to make up for the deficiencies of UAVs' autonomous operations in low - altitude environments.
- Cars are responsible for replacing UAV batteries at designated durable locations to ensure that UAVs have sufficient power to continue performing tasks.
3. **Improve Overall Efficiency**: Through global - local dual - information processing and reasonable model structure design, simplify the model and improve training efficiency.

### Main Contributions

1. **First Research**: This is the first research on the collaborative path - planning problem of UAVs, workers, and cars in disaster response, and it has been proven that this problem is NP - Hard.
2. **Propose a New Algorithm**: Propose a heterogeneous multi - agent path - planning algorithm MANF - RL - RP based on multi - agent reinforcement learning.
3. **Experimental Verification**: Detailed experiments have been carried out with rich simulation data, and the results show that MANF - RL - RP has a significant improvement in the task completion rate compared with baseline algorithms (such as Greedy - SC - RP and MANF - DNN - RP).

### Mathematical Formula Representation

To ensure the correctness and readability of the formulas, the following are the key formulas involved in the paper:

- **Relationship between UAV Remaining Power and Urgency**:
\[
urget_i=\frac{1}{e^{\lfloor uPowt_i / uCsp_i\rfloor}}
\]
where \(uPowt_i\) represents the remaining power of UAV \(i\) at time \(t\), \(uCsp_i\) represents the energy consumption of UAV \(i\) at each time step, and \(\lfloor x\rfloor\) represents the floor function.

- **UAV Power Change**:
\[
uPowt_{i + 1}=
\begin{cases}
1, & \text{if } uLoct_{i+1}=cLoct_{k + 1}\\
uPowt_i - uCsp_i, & \text{if } uLoct_{i+1}\neq cLoct_{k + 1}\text{ and } uPowt_i\geq uCsp_i\\
uPowt_i, & \text{if } uLoct_{i+1}\neq cLoct_{k + 1}\text{ and } uPowt_i < uCsp_i
\end{cases}
\]

Through these formulas, the paper describes in detail the UAVs, workers...