Abstract:In multiagent systems, agent factorization denotes the process of segmenting the state-action space of the environment into distinct components, each corresponding to an individual agent, and subsequently determining the interactions among these agents. Effective agent factorization significantly influences the system performance of real-world industrial applications. In this work, we try to assess the performance impact of agent factorization when using different learning algorithms in multiagent coordination settings; and thus discover the source of performance quality of the multiagent solution derived by combining different factorizations with different learning algorithms. To this end, we evaluate twelve different agent factorization instances—or agent definitions —in the warehouse traffic management domain, comparing the training performance of (primarily) three learning algorithms suitable for learning coordinated multiagent policies: the Evolutionary Strategies ( ES ), the Canonical Evolutionary Strategies ( CES ), and a genetic algorithm ( CCEA ) previously used in a similar setting. Our results demonstrate that the performance of different learning algorithms is affected in different ways by alternative agent definitions. Given this, we can conclude that many important multiagent coordination problems can eventually be solved more efficiently by a suitable agent factorization combined with an appropriate choice of a learning algorithm. Moreover, our work shows that ES and CES are effective learning algorithms for the warehouse traffic management domain, while, interestingly, celebrated policy gradient methods do not fare well in this complex real-world problem setting. As such, our work offers insights into the intrinsic properties of the learning algorithms that make them well-suited for this problem domain. More broadly, our work demonstrates the need to identify appropriate agent definitions-multiagent learning algorithm pairings in order to solve specific complex problems effectively, and provides insights into the general characteristics that such pairings must possess to address broad classes of multiagent learning and coordination problems.

Multi-Agent Optimization and Learning: A Non-Expansive Operators Perspective

Non-cooperative Multi-agent Systems with Exploring Agents

A Tutorial on Optimization for Multi-Agent Systems

Cooperative and Competitive Multi-Agent Systems: From Optimization to Games

An Algorithm for Multi-Objective Multi-Agent Optimization

Distributed Convex Nonsmooth Optimization For Multi-Agent System Based On Proximal Operator

Learning Multi-Agent Communication from Graph Modeling Perspective

Neurodynamic approaches for multi-agent distributed optimization

Optimization for Reinforcement Learning: From Single Agent to Cooperative Agents

Distributed optimization via multi-agent systems

Multiagent Learning for Competitive Opinion Optimization

Co-Optimization of Environment and Policies for Decentralized Multi-Agent Navigation

Distributed proximal‐gradient algorithms for nonsmooth convex optimization of second‐order multiagent systems

Multi-agent cooperation through learning-aware policy gradients

Option-Critic in Cooperative Multi-agent Systems

Multi-agent Black-box Optimization using a Bayesian Approach to Alternating Direction Method of Multipliers

Cooperative Learning of Multi-Agent Systems Via Reinforcement Learning

A comprehensive analysis of agent factorization and learning algorithms in multiagent systems

Communication Learning in Multi-Agent Systems from Graph Modeling Perspective

Learning to Expand/Contract Pareto Sets in Dynamic Multi-Objective Optimization With a Changing Number of Objectives

Distributed Optimization for Second-Order Discrete-Time Multiagent Systems with Set Constraints.