Abstract:Multi-Agent Reinforcement Learning (MARL) has enjoyed significant recent progress thanks, in part, to the integration of deep learning techniques for modeling interactions in complex environments. This is naturally starting to benefit multi-robot systems (MRS) in the form of multi-robot RL (MRRL). However, existing infrastructure to train and evaluate policies predominantly focus on the challenges of coordinating virtual agents, and ignore characteristics important to robotic systems. Few platforms support realistic robot dynamics, and fewer still can evaluate Sim2Real performance of learned behavior. To address these issues, we contribute MARBLER: Multi-Agent RL Benchmark and Learning Environment for the Robotarium. MARBLER offers a robust and comprehensive evaluation platform for MRRL by marrying Georgia Tech's Robotarium (which enables rapid deployment on physical MRS) and OpenAI's Gym interface (which facilitates standardized use of modern learning algorithms). MARBLER offers a highly controllable environment with realistic dynamics, including barrier certificate-based obstacle avoidance. It allows anyone across the world to train and deploy MRRL algorithms on a physical testbed with reproducibility. Further, we introduce five novel scenarios inspired by common challenges in MRS and provide support for new custom scenarios. Finally, we use MARBLER to evaluate popular MARL algorithms and provide insights into their suitability for MRRL. In summary, MARBLER can be a valuable tool to the MRS research community by facilitating comprehensive and standardized evaluation of learning algorithms on realistic simulations and physical hardware. Links to our open-source framework and videos of real-world experiments can be found at <a class="link-external link-https" href="https://shubhlohiya.github.io/MARBLER/" rel="external noopener nofollow">this https URL</a>.

PantheonRL: A MARL Library for Dynamic Training Interactions

S2rl

MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library

marl-jax: Multi-Agent Reinforcement Leaning Framework

CoRL: Environment Creation and Management Focused on System Integration

Gymnasium: A Standard Interface for Reinforcement Learning Environments

On Solving Cooperative MARL Problems with a Few Good Experiences

MARLIN: Multi-Agent Reinforcement Learning Guided by Language-Based Inter-Robot Negotiation

MARL-LNS: Cooperative Multi-agent Reinforcement Learning via Large Neighborhoods Search

Continuous Coordination As a Realistic Scenario for Lifelong Learning

TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning

ChainerRL: A Deep Reinforcement Learning Library

Hierarchical Deep Multiagent Reinforcement Learning with Temporal Abstraction

MARBLER: An Open Platform for Standardized Evaluation of Multi-Robot Reinforcement Learning Algorithms

Modeling and reinforcement learning in partially observable many-agent systems

PLANRL: A Motion Planning and Imitation Learning Framework to Bootstrap Reinforcement Learning

Beyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D Sokoban

Scilab-RL: A software framework for efficient reinforcement learning and cognitive modeling research

Beyond Joint Demonstrations: Personalized Expert Guidance for Efficient Multi-Agent Reinforcement Learning

BenchMARL: Benchmarking Multi-Agent Reinforcement Learning

Scaling Population-Based Reinforcement Learning with GPU Accelerated Simulation