Abstract:Coordination, i.e., multiple autonomous agents in a system to achieve a common goal, is critical for distributed systems since it can increase the overall reward among all agents. However, The dynamic environment and selfish agents pose challenges to learning coordination behavior from historical interaction data in a long-term interaction environment. Previous works mostly focus on one-shot or short-term distributed agent interaction environments, which often leads to selfish or lazy behavior in long-term interaction environments, i.e., prioritizing individual optimal strategies over cooperative strategies. This behavior is mainly due to the lack of historical memory or incomplete use of historical interaction data to guide the current interaction strategy. In this paper, we propose a hierarchical peer-rewarding mechanism, hierarchical gifting, that allows each agent to dynamically assign some of their rewards to other agents based on historical interaction data and guide the agents towards more coordinated behavior while ensuring that agents remain selfish and decentralized. Specifically, we first propose an auxiliary opponent modeling task so that agents can infer opponents’ types through historical interaction trajectories. In addition, we design a hierarchical gifting strategy that dynamically changes during execution based on known opponents’ types. We employ a theoretical framework that captures the benefit of hierarchical gifting in converging to the coordinated behavior by characterizing the equilibria’s basins of attraction in a dynamical system. With hierarchical gifting, we demonstrate increased coordinated behavior of different risk, general-sum coordination games to the prosocial equilibrium both via numerical analysis and experiments.

Coordination and Delegation under Multitasks

A Method to Coordination for Decentralized Multi- Mpc Controllers in Integrated Plant

Multi-Robot Coordination In Complex Environment With Task And Communication Constraints

Delegation Structure and Incentives:The Case of Multiple Agents

Payoff Mechanism Design for Coordination in Multi-Agent Task Allocation Games

Enhancing Coordination in Global Cooperative Software Design

Multi-agent Coordination Under Temporal Logic Tasks and Team-Wise Intermittent Communication

Governing Adaptation and Coordination: A Generalization of Multi-agent Delegation

Optimal Designment for the Incentive Mechanism among Multi-Task Principal-Agent

Coordination in Multi-Agent Systems: a Mentalistic View

(Mis)align: A Simple Dynamic Framework for Modeling Interpersonal Coordination

Coordination as inference in multi-agent reinforcement learning

Interpersonal coordination enhances brain-to-brain synchronization and influences responsibility attribution and reward allocation in social cooperation

Inducing Coordination in Multi-Agent Repeated Game Through Hierarchical Gifting Policies

Cooperation and Coordination in Heterogeneous Populations with Interaction Diversity.

Advances in Multi-Agent Coordination

The interdisciplinary study of coordination

Coordination Tactics and Team Performance:A Case Study Based on Scientific Reseach Teams

Coordinating Multi-Agent Reinforcement Learning Via Dual Collaborative Constraints

Cooperative Control-Based Task Assignments for Multiagent Systems with Intermittent Communication

Multi-Coordination of Coupled Nonidentical Agents with Partial Impulsive Control and Application to Human-Robot Interactions