Abstract:Reinforcement learning has significant applications for multiagent systems, especially in unknown dynamic environments. However, most multiagent reinforcement learning (MARL) algorithms suffer from such problems as exponential computation complexity in the joint state-action space, which makes it difficult to scale up to realistic multiagent problems. In this paper, a novel algorithm named negotiation-based MARL with sparse interactions (NegoSIs) is presented. In contrast to traditional sparse-interaction-based MARL algorithms, NegoSI adopts the equilibrium concept and makes it possible for agents to select the nonstrict equilibrium-dominating strategy profile (nonstrict EDSP) or meta equilibrium for their joint actions. The presented NegoSI algorithm consists of four parts: 1) the equilibrium-based framework for sparse interactions; 2) the negotiation for the equilibrium set; 3) the minimum variance method for selecting one joint action; and 4) the knowledge transfer of local ${Q}$ -values. In this integrated algorithm, three techniques, i.e., unshared value functions, equilibrium solutions, and sparse interactions are adopted to achieve privacy protection, better coordination and lower computational complexity, respectively. To evaluate the performance of the presented NegoSI algorithm, two groups of experiments are carried out regarding three criteria: 1) steps of each episode; 2) rewards of each episode; and 3) average runtime. The first group of experiments is conducted using six grid world games and shows fast convergence and high scalability of the presented algorithm. Then in the second group of experiments NegoSI is applied to an intelligent warehouse problem and simulated results demonstrate the effectiveness of the presented NegoSI algorithm compared with other state-of-the-art MARL algorithms.

Negotiating with Unknown Opponents Toward Multi-lateral Agreement in Real-Time Domains

Toward Efficient Agreements In Real-Time Multilateral Agent-Based Negotiations

Reaching Good Agreements in Multilateral Agent-based Negotiations ∗

Detecting and Learning Against Unknown Opponents for Automated Negotiations.

Robustness Analysis of Negotiation Strategies through Multiagent Learning in Repeated Negotiation Games.

Evaluating Practical Automated Negotiation Based on Spatial Evolutionary Game Theory.

Multi-Issue Agent Automatic Negotiation Based on Fuzzy Evaluation

Tri-Training Based Bilateral Multi-Issue Negotiation Framework.

An Effective Negotiating Agent Framework Based on Deep Offline Reinforcement Learning.

A Deep Reinforcement Learning-Based Agent for Negotiation with Multiple Communication Channels

Automated Negotiation for E-Commerce Decision Making: A Goal Deliberated Agent Architecture for Multi-Strategy Selection

An autonomous agent for negotiation with multiple communication channels using parametrized deep Q-network *

ANOTO: Improving Automated Negotiation Via Offline-to-Online Reinforcement Learning.

A Novel Strategy for Complex Human-Agent Negotiation

Transfer Learning based Agent for Automated Negotiation

A Data-Driven Method for Recognizing Automated Negotiation Strategies

Multiagent Reinforcement Learning with Sparse Interactions by Negotiation and Knowledge Transfer.

Transfer Reinforcement Learning Based Negotiating Agent Framework.

An Agent Bilateral Multi-issue Alternate Bidding Negotiation Protocol Based on Reinforcement Learning and Its Application in E-commerce.

An Autonomous Negotiating Agent Framework with Reinforcement Learning Based Strategies and Adaptive Strategy Switching Mechanism

A Survey of Automated Negotiation: Human Factor, Learning, and Application