UAV Swarm Cooperative Target Search: A Multi-Agent Reinforcement Learning Approach
Yukai Hou,Jin Zhao,Rongqing Zhang,Xiang Cheng,Liuqing Yang
DOI: https://doi.org/10.1109/tiv.2023.3316196
IF: 8.2
2023-01-01
IEEE Transactions on Intelligent Vehicles
Abstract:The development of machine learning and artificial intelligence algorithms, as well as the progress of unmanned aerial vehicle swarm technology, has significantly enhanced the intelligence and autonomy of unmanned aerial vehicles in search missions, resulting in greater efficiency when searching unknown areas. However, as search scenarios become more complex, the existing unmanned aerial vehicle swarm search method lacks scalability and efficient cooperation. Furthermore, due to the increasing scale of search scenarios, the accuracy and real-time performance of global information are difficult to ensure, necessitating the provision of local information. This paper focuses on the large-scale search scenario and split it to provide both local and global information for running unmanned aerial vehicle swarm search algorithms. Since the search environment is often unknown, dynamic, and complex, it requires adaptive decision-making in a constantly changing environment, which is suitable for modeling as a Markov decision process. Considering the sequential-based scenario, we propose a distributed collaborative search method based on a multi-agent reinforcement learning algorithm, which can operate efficiently in complex and large-scale scenarios. Additionally, the proposed method can utilize a convolutional neural network to process high-dimensional map data with almost no loss of the structure information. Experimental results demonstrate that the proposed method can collaboratively search unknown areas, avoid collisions and repetitions, and find all targets faster compared with the benchmarks.