Abstract:One important approach of multiagent reinforcement learning (MARL) is equilibrium-based MARL, which is a combination of reinforcement learning and game theory. Most existing algorithms involve computationally expensive calculation of mixed strategy equilibria and require agents to replicate the other agents' value functions for equilibrium computing in each state. This is unrealistic since agents may not be willing to share such information due to privacy or safety concerns. This paper aims to develop novel and efficient MARL algorithms without the need for agents to share value functions. First, we adopt pure strategy equilibrium solution concepts instead of mixed strategy equilibria given that a mixed strategy equilibrium is often computationally expensive. In this paper, three types of pure strategy profiles are utilized as equilibrium solution concepts: pure strategy Nash equilibrium, equilibrium-dominating strategy profile, and nonstrict equilibrium-dominating strategy profile. The latter two solution concepts are strategy profiles from which agents can gain higher payoffs than one or more pure strategy Nash equilibria. Theoretical analysis shows that these strategy profiles are symmetric meta equilibria. Second, we propose a multistep negotiation process for finding pure strategy equilibria since value functions are not shared among agents. By putting these together, we propose a novel MARL algorithm called negotiation-based Q-learning (NegoQ). Experiments are first conducted in grid-world games, which are widely used to evaluate MARL algorithms. In these games, NegoQ learns equilibrium policies and runs significantly faster than existing MARL algorithms (correlated Q-learning and Nash Q-learning). Surprisingly, we find that NegoQ also performs well in team Markov games such as pursuit games, as compared with team-task-oriented MARL algorithms (such as friend Q-learning and distributed Q-learning).

KnowRU: Knowledge Reuse Via Knowledge Distillation in Multi-Agent Reinforcement Learning

KnowRU: Knowledge Reusing via Knowledge Distillation in Multi-agent Reinforcement Learning

KnowSR: Knowledge Sharing among Homogeneous Agents in Multi-agent Reinforcement Learning

Knowledge Reuse of Multi-Agent Reinforcement Learning in Cooperative Tasks

Leveraging Knowledge Distillation for Efficient Deep Reinforcement Learning in Resource-Constrained Environments

Efficient Open-world Reinforcement Learning via Knowledge Distillation and Autonomous Rule Discovery

Qauxi: Cooperative Multi-Agent Reinforcement Learning with Knowledge Transferred from Auxiliary Task

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Novelty-Guided Data Reuse for Efficient and Diversified Multi-Agent Reinforcement Learning

Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning

Learning in Multi-Agent Systems with Sparse Interactions by Knowledge Transfer and Game Abstraction

Efficient Exploration for Multi-Agent Reinforcement Learning Via Transferable Successor Features

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

KG-RL: A Knowledge-Guided Reinforcement Learning for Massive Battle Games

Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning

Hybrid knowledge transfer for MARL based on action advising and experience sharing

DGTRL: Deep graph transfer reinforcement learning method based on fusion of knowledge and data

Multi-hop Knowledge Reasoning with Deep Reinforcement Learning

An Offline-Transfer-Online Framework for Cloud-Edge Collaborative Distributed Reinforcement Learning

Multiagent Reinforcement Learning with Unshared Value Functions.

Knowledge Sharing and Transfer via Centralized Reward Agent for Multi-Task Reinforcement Learning