Abstract:Reinforcement Learning (RL) is an efficient learning method for solving problems that learning agents have no knowledge about the environment a priori. Ant Colony System (ACS) provides an indirect communication method among cooperating agents, which is an efficient method for solving combinatorial optimization problems. Based on the cooperating method of the indirect communication in ACS and the update policy of reinforcement values in RL, this paper proposes the Q-ACS multiagent cooperating learning method that can be applied to both Markov Decision Processes (MDPs) and combinatorial optimization problems. The advantage of the Q-ACS method is for the learning agents to share episodes beneficial to the exploitation of the accumulated knowledge and utilize the learned reinforcement values efficiently. Further, taking the visited times into account, this paper proposes the T-ACS multiagent learning method. The merit of the T-ACS method is that the learning agents share better policies beneficial to the exploration during agent's learning processes. Meanwhile, considering the Q-ACS and the T-ACS as homogeneous multiagent learning methods, in the light of indirect media communication among heterogeneous multiagent, this paper presents a heterogeneous multiagent RL method, the D-ACS that composites the learning policy of the Q-ACS and the T-ACS, and takes different updating policies of reinforcement values. The agents in our methods are given a simply cooperating way exchanging information in the form of reinforcement values updated in the common model of all agents. Owning the advantages of exploring the unknown environment actively and exploiting learned knowledge effectively, the proposed methods are able to solve both problems with MDPs and combinatorial optimization problems effectively. The results of experiments on hunter game and traveling salesman problem demonstrate that our methods perform competitively with representative methods on each domain respectively.

Cooperative Q-Learning Based On Maturity Of The Policy

An Improved Reinforcement Learning Algorithm for Cooperative Behaviors of Mobile Robots

Cooperative Q-learning based on learning automata

Multi-agent Cooperation Q-learning Algorithm Based on Constrained Markov Game

Cooperative Q Learning Based on Blackboard Architecture

The Knowledge Sharing Based Reinforcement Learning Algorithm for Collective Behaviors of Mobile Robots

Multi-Agent Path Finding Method Based on Evolutionary Reinforcement Learning

The Optimization of Path Planning for Multi-Robot System Using Boltzmann Policy Based Q-learning Algorithm

Dynamic Correlation Matrix Based Multi-Q Learning For A Multi-Robot System

Cooperative Behavior Acquisition Based Modular Q Learning in Multi-Agent System

Multi-robot Cooperation Strategy in a Partially Observable Markov Game Using Enhanced Deep Deterministic Policy Gradient

Combining Model-Based $Q$ -Learning with Structural Knowledge Transfer for Robot Skill Learning

A Multitier Reinforcement Learning Model for a Cooperative Multiagent System

Q-CP: Learning Action Values for Cooperative Planning

Study on online εnN Q-learning for multi-robot system based on community perception network

Self-organizing Coordination of Multi-Robot Based on Monte Carlo Learning

Reinforcement learning for encouraging cooperation in a multiagent system

Multi-Agent Coordination Method Based On Fuzzy Q-Learning

A Distributed Q-Learning Algorithm for Multi-Agent Team Coordination

Multiagent Cooperating Learning Methods by Indirect Media Communication.

Metropolis Policy-based Multi-step Q Learning Algorithm and Performance Simulation