Abstract:Deep reinforcement learning (RL) is achieving significant success in various applications like control, robotics, games, resource management, and scheduling. However, the important problem of emergency evacuation, which clearly could benefit from RL, has been largely unaddressed. Indeed, emergency evacuation is a complex task that is difficult to solve with RL. An emergency situation is highly dynamic, with a lot of changing variables and complex constraints that make it challenging to solve. Also, there is no standard benchmark environment available that can be used to train RL agents for evacuation. A realistic environment can be complex to design. In this article, we propose the first fire evacuation environment to train RL agents for evacuation planning. The environment is modeled as a graph capturing the building structure. It consists of realistic features like fire spread, uncertainty, and bottlenecks. The implementation of our environment is in the OpenAI gym format, to facilitate future research. We also propose a new RL approach that entails pretraining the network weights of a DQN-based agent [DQN/Double-DQN (DDQN)/Dueling-DQN] to incorporate information on the shortest path to the exit. We achieved this by using tabular $Q$ -learning to learn the shortest path on the building model’s graph. This information is transferred to the network by deliberately overfitting it on the $Q$ -matrix. Then, the pretrained DQN model is trained on the fire evacuation environment to generate the optimal evacuation path under time varying conditions due to fire spread, bottlenecks, and uncertainty. We perform comparisons of the proposed approach with state-of-the-art RL algorithms like DQN, DDQN, Dueling-DQN, PPO, VPG, state-action-reward-state-action (SARSA), actor–critic method, and ACKTR. The results show that our method is able to outperform state-of-the-art models by a huge margin including the original DQN-based models. Finally, our model is tested on a large and complex real building consisting of 91 rooms, with the possibility to move to any other room, hence giving 8281 actions. In order to reduce the action space, we propose a strategy that involves one step simulation. That is, an action importance vector is added to the final output of the pretrained DQN and acts like an attention mechanism. Using this strategy, the action space is reduced by 90.1%. In this manner, the model is able to deal with large action spaces. Hence, our model achieves near optimal performance on the real world emergency environment.

Dual deep Q-learning network guiding a multiagent path planning approach for virtual fire emergency scenarios

Deep Q-Learning With Q-Matrix Transfer Learning for Novel Fire Evacuation Environment

Underground rescue path planning based on a comprehensive risk assessment approach

A path planning method based on deep reinforcement learning for crowd evacuation

A self-learning dynamic path planning method for evacuation in large public buildings based on neural networks.

Dynamic Evacuation Path Planning for Multi-Exit Building Fire: Bi-Objective Model and Algorithm

Optimal path planning in real time for dynamic building fire rescue operations using wireless sensors and visual guidance

Reinforcement Learning Based Escape Route Generation in Low Visibility Environments

A Virtual Reality Simulation Method for Crowd Evacuation in a Multiexit Indoor Fire Environment

Multi-agent navigation based on deep reinforcement learning and traditional pathfinding algorithm

Deep reinforcement learning with a particle dynamics environment applied to emergency evacuation of a room with obstacles

Rescue path planning for urban flood: A deep reinforcement learning-based approach

Path Planning for Autonomous Vehicles in Unknown Dynamic Environment Based on Deep Reinforcement Learning

Dynamic Path Planning of Unknown Environment Based on Deep Reinforcement Learning

Deep reinforcement learning and 3D physical environments applied to crowd evacuation in congested scenarios

Path optimization of integrating crowd model and reinforcement learning.

Improved Robot Path Planning Method Based on Deep Reinforcement Learning

Path Planning in Dynamic Environments through Trajectory Prediction and Reinforcement Learning

Active Wildfires Detection and Dynamic Escape Routes Planning for Humans through Information Fusion between Drones and Satellites

A Deep Reinforcement Learning Algorithm for Trajectory Planning of Swarm UAV Fulfilling Wildfire Reconnaissance

Visualization of Emergency Evacuation Physical Behavior under Multi-Agent Decision-Making