Abstract:There is a prevalence of multiagent reinforcement learning (MARL) methods that engage in centralized training. These methods rely on all the agents sharing various types of information, such as their actions or gradients, with a centralized trainer or each other during the learning. Subsequently, the methods produce agent policies whose prescriptions and performance are contingent on other agents engaging in behavior assumed by the centralized training. But, in many contexts, such as mixed or adversarial settings, this assumption may not be feasible. In this article, we present a new line of methods that relaxes this assumption and engages in decentralized training resulting in the agent's individual policy. The interactive advantage actor-critic (IA2C) maintains and updates beliefs over other agents' candidate behaviors based on (noisy) observations, thus enabling learning at the agent's own level. We also address MARL's prohibitive curse of dimensionality due to the presence of many agents in the system. Under assumptions of action anonymity and population homogeneity, often exhibited in practice, large numbers of other agents can be modeled aggregately by the count vectors of their actions instead of individual agent models. More importantly, we may model the distribution of these vectors and its update using the Dirichlet-multinomial model, which offers an elegant way to scale IA2C to many-agent systems. We evaluate the performance of the fully decentralized IA2C along with other known baselines on a novel Organization domain, which we introduce, and on instances of two existing domains. Experimental comparisons with prominent and recent baselines show that IA2C is more sample efficient, more robust to noise, and can scale to learning in systems with up to a hundred agents.

Multi-level agent-based modeling with the Influence Reaction principle

IRM4MLS: the influence reaction model for multi-level simulation

Modelisation a base d'Agent Augmentes par LLM pour les Simulations Sociales: Defis et Opportunites

LMRL: a Multi-Agent Reinforcement Learning Model and Algorithm

Synergistic Simulations: Multi-Agent Problem Solving with Large Language Models

Modeling and reinforcement learning in partially observable many-agent systems

Multilevel Modeling as a Methodology for the Simulation of Human Mobility

Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-Agent Reinforcement Learning

A Language for Agent-based Discrete-event Modeling and Simulation of Linked Lives

Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulations

Multiagent-based Participatory Urban Simulation through Inverse Reinforcement Learning

Autonomous Agent Behaviour Modelled in PRISM -- A Case Study

Interactions as Social Practices: towards a formalization

Agent based models and opinion dynamics as Markov chains

A Model for Multi-Agent Autonomy That Uses Opinion Dynamics and Multi-Objective Behavior Optimization

LLM-Augmented Agent-Based Modelling for Social Simulations: Challenges and Opportunities

Incorporating Inertia Into Multi-Agent Systems

CGMI: Configurable General Multi-Agent Interaction Framework

Agent cognition through micro-simulations: Adaptive and tunable intelligence with NetLogo LevelSpace

Non-cooperative Multi-agent Systems with Exploring Agents

Engineering hierarchical complex systems: an agent-based approach. The case of flexible manufacturing systems