Abstract:Development of distributed Multi-Agent Reinforcement Learning (MARL) algorithms has attracted an increasing surge of interest lately. Generally speaking, conventional Model-Based (MB) or Model-Free (MF) RL algorithms are not directly applicable to the MARL problems due to utilization of a fixed reward model for learning the underlying value function. While Deep Neural Network (DNN)-based solutions perform well, they are still prone to overfitting, high sensitivity to parameter selection, and sample inefficiency. In this paper, an adaptive Kalman Filter (KF)-based framework is introduced as an efficient alternative to address the aforementioned problems by capitalizing on unique characteristics of KF such as uncertainty modeling and online second order learning. More specifically, the paper proposes the Multi-Agent Adaptive Kalman Temporal Difference (MAK-TD) framework and its Successor Representation-based variant, referred to as the MAK-SR. The proposed MAK-TD/SR frameworks consider the continuous nature of the action-space that is associated with high dimensional multi-agent environments and exploit Kalman Temporal Difference (KTD) to address the parameter uncertainty. The proposed MAK-TD/SR frameworks are evaluated via several experiments, which are implemented through the OpenAI Gym MARL benchmarks. In these experiments, different number of agents in cooperative, competitive, and mixed (cooperative-competitive) scenarios are utilized. The experimental results illustrate superior performance of the proposed MAK-TD/SR frameworks compared to their state-of-the-art counterparts.

Representation Learning For Efficient Deep Multi-Agent Reinforcement Learning

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rollouts

Towards Efficient Multi-Agent Learning Systems

Decentralized Multi-Agent Reinforcement Learning: An Off-Policy Method

Offline Multi-Agent Reinforcement Learning via In-Sample Sequential Policy Optimization

Multi-Agent Constrained Policy Optimisation

Joint-Predictive Representations for Multi-Agent Reinforcement Learning

Efficient Multi-agent Reinforcement Learning by Planning

Feudal Latent Space Exploration for Coordinated Multi-Agent Reinforcement Learning.

Proximal Policy Optimization Based Decentralized Networked Multi-Agent Reinforcement Learning

Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation

YOLO-MARL: You Only LLM Once for Multi-agent Reinforcement Learning

Sample-Efficient Multi-Agent RL: an Optimization Perspective.

Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaboration

Hierarchical Deep Multiagent Reinforcement Learning with Temporal Abstraction

A Two-Stage Multi-Objective Deep Reinforcement Learning Framework.

Multi-Agent Trust Region Policy Optimization

Multi-Agent Reinforcement Learning via Adaptive Kalman Temporal Difference and Successor Representation

Selective Learning for Sample-Efficient Training in Multi-Agent Sparse Reward Tasks

Learning Latent Representations to Influence Multi-Agent Interaction