Abstract:A large proportion of recent studies on cooperative Multi-Agent Reinforcement Learning (MARL) focus on the policylearning process in scenarios with stationary opponents (or without opponents). This paper, instead, investigates a different challenge of achieving team superiority in dynamic competitions among competitors that evolve dynamically with MARL. We aim to enhance the competitiveness of such MARL learners by enabling them to adjust their own learning settings dynamically, so as to take quick counter-measures against the policy shift of competitor learners, or to learn faster to suppress the opponents. We propose a Competitive Auto-Multiagent Learner with Fuzzy Feedback (CALF) with two essential highlights: (1) CALF establishes feedback controllers to achieve real-time adjustments based on fuzzy logic, using human-readable fuzzy rules to provide significant explainability and flexibility; (2) CALF integrates Bayesian Optimization to search and optimize the feedback fuzzy logic rules automatically. CALF can be used to apply real-time adjustments for MARL hyperparameters and intrinsic rewards. We also give solid empirical results to show that CALF significantly promotes team competitiveness in adversarial competitions, spanning from small-scale tasks involving 2 teams to large-scale tasks involving 3 teams and hundreds of agents. Furthermore, CALF exhibits superior competitiveness when engaging in competition with established competitors like Qmix, Qtran, and Qplex in dynamic competitive environments. Moreover, the experiments also demonstrate that the integration of the fuzzy logic with Bayesian Optimization offers considerable transferability and explainability, enabling a CALF-implemented learner optimized from one scenario to be transferred to other distinct scenarios.

Multi-Scale Reinforcement Learning with Fuzzy State

Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments

Learning Obstacle Avoidance Behavior Using Multi-Agent Learning With Fuzzy States

Multiple rewards fuzzy reinforcement learning algorithm in RoboCup environment

Multi-Scale Extension in an Entorhinal-Hippocampal Model for Cognitive Map Building.

A multiagent reinforcement learning approach based on different states

Path Planning for Mobile Robot Based on Deep Reinforcement Learning and Fuzzy Control

Low-Cost Multi-Agent Navigation Via Reinforcement Learning with Multi-Fidelity Simulator.

Hierarchical reinforcement learning for handling sparse rewards in multi-goal navigation

Navigation for Mobile Robots Using Reinforcement Learning and Fuzzy Logic

Competitive Takagi-Sugeno Fuzzy Reinforcement Learning

Multilevel Graph Reinforcement Learning for Consistent Cognitive Decision-making in Heterogeneous Mixed Autonomy

Multi-objective deep reinforcement learning for crowd-aware robot navigation with dynamic human preference

Hierarchical Reinforcement Learning Algorithm Based on Structural State-Space

HELSA: Hierarchical Reinforcement Learning with Spatiotemporal Abstraction for Large-Scale Multi-Agent Path Finding

Fuzzy Feedback Multi-Agent Reinforcement Learning for Adversarial Dynamic Multi-Team Competitions

Autonomous and Adaptive Role Selection for Multi-robot Collaborative Area Search Based on Deep Reinforcement Learning

Convergence of Multi-Scale Reinforcement Q-Learning Algorithms for Mean Field Game and Control Problems

Multi-scale Fuzzy Modeling of Nonlinear Dynamical Systems

Learning to Navigate in a VUCA Environment: Hierarchical Multi-expert Approach

Reinforcement Learning in the Multi-Robot Domain