Abstract:A large proportion of recent studies on cooperative Multi-Agent Reinforcement Learning (MARL) focus on the policylearning process in scenarios with stationary opponents (or without opponents). This paper, instead, investigates a different challenge of achieving team superiority in dynamic competitions among competitors that evolve dynamically with MARL. We aim to enhance the competitiveness of such MARL learners by enabling them to adjust their own learning settings dynamically, so as to take quick counter-measures against the policy shift of competitor learners, or to learn faster to suppress the opponents. We propose a Competitive Auto-Multiagent Learner with Fuzzy Feedback (CALF) with two essential highlights: (1) CALF establishes feedback controllers to achieve real-time adjustments based on fuzzy logic, using human-readable fuzzy rules to provide significant explainability and flexibility; (2) CALF integrates Bayesian Optimization to search and optimize the feedback fuzzy logic rules automatically. CALF can be used to apply real-time adjustments for MARL hyperparameters and intrinsic rewards. We also give solid empirical results to show that CALF significantly promotes team competitiveness in adversarial competitions, spanning from small-scale tasks involving 2 teams to large-scale tasks involving 3 teams and hundreds of agents. Furthermore, CALF exhibits superior competitiveness when engaging in competition with established competitors like Qmix, Qtran, and Qplex in dynamic competitive environments. Moreover, the experiments also demonstrate that the integration of the fuzzy logic with Bayesian Optimization offers considerable transferability and explainability, enabling a CALF-implemented learner optimized from one scenario to be transferred to other distinct scenarios.

Multiple rewards fuzzy reinforcement learning algorithm in RoboCup environment

Cooperative Flocking And Learning In Multi-Robot Systems For Predator Avoidance

Multi-robot behavior adaptation to local and global communication atmosphere in humans-robots interaction

An Improved Reinforcement Learning System Using Affective Factors

Fuzzy Feedback Multi-Agent Reinforcement Learning for Adversarial Dynamic Multi-Team Competitions

LMRL: a Multi-Agent Reinforcement Learning Model and Algorithm

Learning Competition In Robot Soccer Game Based On An Adapted Neuro-Fuzzy Inference System

Heterogeneous Multi-Robot Cooperation With Asynchronous Multi-Agent Reinforcement Learning

Competitive Takagi-Sugeno Fuzzy Reinforcement Learning

Relational Q-Functionals: Multi-Agent Learning to Recover from Unforeseen Robot Malfunctions in Continuous Action Domains

Development of an algorithm for managing a multi-robot system for cargo transportation based on reinforcement learning in a virtual environment

Linguistic Reward-Oriented Takagi-Sugeno Fuzzy Reinforcement Learning

A Two-Layered Multi-Agent Reinforcement Learning Model and Algorithm

Multi-agent Reinforcement Learning with Deep Networks for Diverse Q-Vectors

Individual Reward Assisted Multi-Agent Reinforcement Learning.

Multi-State-Space Reasoning Reinforcement Learning for Long-Horizon RFID-Based Robotic Searching and Planning Tasks

Two-stage training algorithm for AI robot soccer

Reinforcement learning for encouraging cooperation in a multiagent system

Multi-goal Q-learning of Cooperative Teams

Multi-agent Collaboration for Feasible Collaborative Behavior Construction and Evaluation

Dynamic Formation Planning and Control for Robot Soccer Game with Multi-Agent Reinforcement Learning and Behavioral Model