Abstract:A large proportion of recent studies on cooperative Multi-Agent Reinforcement Learning (MARL) focus on the policylearning process in scenarios with stationary opponents (or without opponents). This paper, instead, investigates a different challenge of achieving team superiority in dynamic competitions among competitors that evolve dynamically with MARL. We aim to enhance the competitiveness of such MARL learners by enabling them to adjust their own learning settings dynamically, so as to take quick counter-measures against the policy shift of competitor learners, or to learn faster to suppress the opponents. We propose a Competitive Auto-Multiagent Learner with Fuzzy Feedback (CALF) with two essential highlights: (1) CALF establishes feedback controllers to achieve real-time adjustments based on fuzzy logic, using human-readable fuzzy rules to provide significant explainability and flexibility; (2) CALF integrates Bayesian Optimization to search and optimize the feedback fuzzy logic rules automatically. CALF can be used to apply real-time adjustments for MARL hyperparameters and intrinsic rewards. We also give solid empirical results to show that CALF significantly promotes team competitiveness in adversarial competitions, spanning from small-scale tasks involving 2 teams to large-scale tasks involving 3 teams and hundreds of agents. Furthermore, CALF exhibits superior competitiveness when engaging in competition with established competitors like Qmix, Qtran, and Qplex in dynamic competitive environments. Moreover, the experiments also demonstrate that the integration of the fuzzy logic with Bayesian Optimization offers considerable transferability and explainability, enabling a CALF-implemented learner optimized from one scenario to be transferred to other distinct scenarios.

Achieving Multiagent Coordination Through Cala-Rfmq Learning In Continuous Action Space

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

SCC-rFMQ Learning in Cooperative Markov Games with Continuous Actions

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

SCC-rFMQ: a Multiagent Reinforcement Learning Method in Cooperative Markov Games with Continuous Actions

Multi-Agent Coordination Method Based On Fuzzy Q-Learning

Cooperative Strategy Learning in Multi-Agent Environment with Continuous State Space

A Cooperative Multi-Agent Reinforcement Learning Method Based on Coordination Degree

Fuzzy Feedback Multi-Agent Reinforcement Learning for Adversarial Dynamic Multi-Team Competitions

Multi-Agent Reward-Iteration Fuzzy Q-Learning

Q-CF Multi-Agent Reinforcement Learning for Resource Allocation Problems

Entropy Enhanced Multi-Agent Coordination Based on Hierarchical Graph Learning for Continuous Action Space

Cooperative Behavior Acquisition Based Modular Q Learning in Multi-Agent System

Self-organizing Coordination of Multi-Robot Based on Monte Carlo Learning

Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces

Adaptive Individual Q-Learning-A Multiagent Reinforcement Learning Method for Coordination Optimization

Multi-domain Cooperative Action Planning Strategy Based on Reinforcement Learning

A Coordination Model Using Fuzzy Reinforcement Learning for Multi-Agent System

Fast reinforcement learning approach to cooperative behavior acquisition in multi-agent system

Mixed Q-Functionals: Advancing Value-Based Methods in Cooperative MARL with Continuous Action Domains

Multi-agent Cooperative Search Learning with Intermittent Communication