An Auction-based Coordination Strategy for Task-Constrained Multi-Agent Stochastic Planning with Submodular Rewards

Ruifan Liu,Hyo-Sang Shin,Binbin Yan,Antonios Tsourdos

2023-08-02

Abstract:In many domains such as transportation and logistics, search and rescue, or cooperative surveillance, tasks are pending to be allocated with the consideration of possible execution uncertainties. Existing task coordination algorithms either ignore the stochastic process or suffer from the computational intensity. Taking advantage of the weakly coupled feature of the problem and the opportunity for coordination in advance, we propose a decentralized auction-based coordination strategy using a newly formulated score function which is generated by forming the problem into task-constrained Markov decision processes (MDPs). The proposed method guarantees convergence and at least 50% optimality in the premise of a submodular reward function. Furthermore, for the implementation on large-scale applications, an approximate variant of the proposed method, namely Deep Auction, is also suggested with the use of neural networks, which is evasive of the troublesome for constructing MDPs. Inspired by the well-known actor-critic architecture, two Transformers are used to map observations to action probabilities and cumulative rewards respectively. Finally, we demonstrate the performance of the two proposed approaches in the context of drone deliveries, where the stochastic planning for the drone league is cast into a stochastic price-collecting Vehicle Routing Problem (VRP) with time windows. Simulation results are compared with state-of-the-art methods in terms of solution quality, planning efficiency and scalability.

Multiagent Systems,Neural and Evolutionary Computing

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to efficiently allocate tasks in a multi - agent system while considering the possible uncertainties during the task execution process. Specifically, the paper focuses on how to rationally allocate tasks to multiple agents (such as drones) in fields such as traffic logistics, search and rescue, and collaborative monitoring, while taking into account the uncertainties during the task execution process, in order to improve the overall efficiency and quality of task completion.

The paper proposes a decentralized coordination strategy based on auctions to solve this problem by introducing a new scoring function. This scoring function is generated by modeling the problem as Markov decision processes (MDPs) under task constraints. The paper also proposes an approximation method suitable for large - scale applications - Deep Auction, which uses neural networks to avoid the trouble of constructing MDPs, and this method improves computational efficiency while ensuring a certain degree of optimality.

In addition, the paper also proves that the proposed decentralized coordination strategy can converge and achieve at least 50% optimality under the premise of sub - modular reward functions. These contributions aim to solve the problems that existing methods either ignore stochastic processes or have too high computational complexity, thereby providing a more effective and scalable solution for practical applications.