Inverse Attention Agent for Multi-Agent System

Qian Long,Ruoyan Li,Minglu Zhao,Tao Gao,Demetri Terzopoulos

2024-10-29

Abstract:A major challenge for Multi-Agent Systems is enabling agents to adapt dynamically to diverse environments in which opponents and teammates may continually change. Agents trained using conventional methods tend to excel only within the confines of their training cohorts; their performance drops significantly when confronting unfamiliar agents. To address this shortcoming, we introduce Inverse Attention Agents that adopt concepts from the Theory of Mind, implemented algorithmically using an attention mechanism and trained in an end-to-end manner. Crucial to determining the final actions of these agents, the weights in their attention model explicitly represent attention to different goals. We furthermore propose an inverse attention network that deduces the ToM of agents based on observations and prior actions. The network infers the attentional states of other agents, thereby refining the attention weights to adjust the agent's final action. We conduct experiments in a continuous environment, tackling demanding tasks encompassing cooperation, competition, and a blend of both. They demonstrate that the inverse attention network successfully infers the attention of other agents, and that this information improves agent performance. Additional human experiments show that, compared to baseline agent models, our inverse attention agents exhibit superior cooperation with humans and better emulate human behaviors.

Artificial Intelligence,Multiagent Systems

What problem does this paper attempt to address?

This paper attempts to address the issue of agents in Multi-Agent Systems (MAS) adapting to different opponents and teammates in dynamic environments. Agents trained using traditional methods usually perform well within their training queue, but their performance significantly drops when facing unfamiliar agents. To solve this problem, the paper introduces "Inverse Attention Agents," which draw on the concept of Theory of Mind (ToM) and are implemented through attention mechanism algorithms. Specifically, these agents can infer the attention states of other agents and adjust their final actions accordingly. In this way, Inverse Attention Agents can perform better in complex tasks that involve cooperation, competition, or a mix of both. The key contributions of the paper include: 1. **Proposing the inverse attention mechanism**: By inferring the attention states of other agents through the inverse attention network, agents can adjust their behavior to better adapt to other agents in the environment. 2. **Enhancing agent adaptability**: Especially when collaborating with unseen agents, Inverse Attention Agents can exhibit better cooperation and behavior imitation capabilities. 3. **Experimental validation**: Through a series of experiments, including cooperation experiments with humans, it is demonstrated that Inverse Attention Agents show superior performance in various tasks. Overall, this paper aims to improve the adaptability and performance of agents in Multi-Agent Systems when facing unfamiliar opponents and teammates by introducing the inverse attention mechanism.

Inverse Attention Agent for Multi-Agent System

MAPPO method based on attention behavior network

Inner Attention Supported Adaptive Cooperation for Heterogeneous Multi Robots Teaming based on Multi-agent Reinforcement Learning

Complementary Attention for Multi-Agent Reinforcement Learning.

AATEAM: Achieving the Ad Hoc Teamwork by Employing the Attention Mechanism

Joint Attention for Multi-Agent Coordination and Social Learning

Attention or memory? Neurointerpretable agents in space and time

MAIT: Multi-agent Local Observation Interaction to Improve the Decision-Making Ability

Multi-agent Reinforcement Learning with Multi-head Attention

Multi-Agent Reinforcement Learning-Based Autonomous Intersection Management Protocol with Attention Mechanism

Multi-agent Reinforcement Learning by the Actor-Critic Model with an Attention Interface

Meta Attention for Off-Policy Actor-Critic.

Learning Attentional Communication with a Common Network for Multiagent Reinforcement Learning.

Multi-agent Attentional Activity Recognition.

Modeling Theory of Mind in Multi-Agent Games Using Adaptive Feedback Control

Cascaded Attention: Adaptive and Gated Graph Attention Network for Multiagent Reinforcement Learning

AHAC: Actor Hierarchical Attention Critic for Multi-Agent Reinforcement Learning.

Local Information Based Attentional Opponent Modelling In Multi-agent Reinforcement Learning

Attentional Policies for Cross-Context Multi-Agent Reinforcement Learning

Consciousness-Aware Multi-Agent Reinforcement Learning

A brain-inspired theory of mind spiking neural network improves multi-agent cooperation and competition