Multi-Ship Dynamic Weapon-Target Assignment Via Cooperative Distributional Reinforcement Learning with Dynamic Reward
Zhe Peng,Zhifeng Lu,Xiao Mao,Feng Ye,Kuihua Huang,Guohua Wu,Ling Wang
DOI: https://doi.org/10.1109/tetci.2024.3451338
2024-01-01
IEEE Transactions on Emerging Topics in Computational Intelligence
Abstract:In fleet air defense, the efficient coordination of multiple ships to complete weapon-target assignment has always been a critical challenge, primarily due to the varying combat capabilities and duties associated with each ship. Consequently, the traditional “weapon-target” assignment mode has turned into a “ship-weapon-target” assignment mode in the multi-ship dynamic weapon-target assignment (MS-DWTA) problem we proposed, with a larger solution space. In this problem, different ships possess distinct attributes, such as defense duties, weapon types, and loaded missile quantities. To solve this problem, we proposed an Attention enhanced multi-agent Distributional reinforcement learning method with Dynamic Reward (ADDR). Different from standard reinforcement learning method, ADDR learns to estimate the distribution, as opposed to only the expectation of future return, enabling better adaptation to air defense scenarios with significant randomness. The multi-head attention network integrates both the ship situation and the target situation to appropriately adjust the output of each agent, which explicitly considers the agent-level impact of ships to the whole fleet. Moreover, due to the missile fight time, ships may not immediately receive rewards after executing actions. To address this delayed phenomenon, we designed a dynamic reward mechanism to accurately adjust the delayed rewards. Through extensive simulation experiments, ADDR has demonstrated superior performance over multiple evaluation metrics.