A double‐layer crowd evacuation simulation method based on deep reinforcement learning

Yong Zhang,Bo Yang,Jianlin Zhu
DOI: https://doi.org/10.1002/cav.2280
IF: 1.01
2024-06-01
Computer Animation and Virtual Worlds
Abstract:In this paper, a two‐layer crowd evacuation simulation method based on deep reinforcement learning is proposed. The overall framework of the method is divided into two layers: in the global path planning layer, a deep deterministic policy gradient algorithm based on learning curves is proposed, which utilizes the learning curve theory to dynamically change the algorithm's experience pool capacity, and utilizes a prioritized experience sampling strategy to give higher sampling probabilities to experiences with learning value, thus improving the algorithm's learning efficiency; in the layer of the crowd's microscopic behaviors, an improved social force model is applied to the crowd's In the micro‐behavioral layer of the crowd, the improved social force model is used to model the micro‐motion of the crowd, which guides the crowd to avoid obstacles and reach the safe exit quickly. The experimental results show that the deep reinforcement learning‐based two‐layer crowd evacuation simulation method proposed in this paper can improve the path planning efficiency. Existing crowd evacuation simulation methods commonly face challenges of low efficiency in path planning and insufficient realism in pedestrian movement during the evacuation process. In this study, we propose a novel crowd evacuation path planning approach based on the learning curve–deep deterministic policy gradient (LC‐DDPG) algorithm. The algorithm incorporates dynamic experience pool and a priority experience sampling strategy, enhancing convergence speed and achieving higher average rewards, thus efficiently enabling global path planning. Building upon this foundation, we introduce a double‐layer method for crowd evacuation using deep reinforcement learning. Specifically, within each group, individuals are categorized into leaders and followers. At the top layer, we employ the LC‐DDPG algorithm to perform global path planning for the leaders. Simultaneously, at the bottom layer, an enhanced social force model guides the followers to avoid obstacles and follow the leaders during evacuation. We implemented a crowd evacuation simulation platform. Experimental results show that our proposed method has high path planning efficiency and can generate more realistic pedestrian trajectories in different scenarios and crowd sizes.
computer science, software engineering
What problem does this paper attempt to address?