Abstract:Multi-agent reinforcement learning (RL) utilizes the observations and learning experiences shared among the agents to accelerate learning speed under partial observations and the resulting learning efficiency depends on the cooperative agent selection and the RL task state formulation. In this paper, we propose an efficient communication scheme for multi-agent RL that enables each learning agent to optimize the cooperative agent selection and the task state formulation to improve the learning performance and the quality of service for RL-based applications in wireless networks. Based on the local observation, the radio channel states, the similarity of RL task with neighboring agents and previous communication cost, this scheme formulates a communication state, which is input to a neural network to estimate the communication policy distribution. The RL task state of the learning agent, which consists of the local observation such as channel states and previous task performance, as well as the correlation between the shared and the local observation extracted based on the attention mechanism, is formulated to enhance the agent receptive field. In addition, the shared learning information is also exploited to update the local learning parameters such as the task Q-values and neural network weights and further improve the RL task policy exploration. As a case study, the proposed communication scheme is implemented in the multi-agent deep Q-network based anti-jamming unmanned aerial vehicle swarm communications and the performance gain over the benchmark is verified via simulation results.

Policy evaluation for reinforcement learning over asynchronous multi-agent networks

Fully asynchronous policy evaluation in distributed reinforcement learning over networks

Observer-Based Multiagent Deep Reinforcement Learning: A Fully Distributed Training Scheme

Target-Value-Competition-Based Multi-Agent Deep Reinforcement Learning Algorithm for Distributed Nonconvex Economic Dispatch

A Multi-Agent Off-Policy Actor-Critic Algorithm for Distributed Reinforcement Learning

Multi-agent Policy Optimization with Approximatively Synchronous Advantage Estimation

Distributed Policy Evaluation Under Multiple Behavior Strategies

Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systems

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Data-Based Optimal Consensus Control for Multiagent Systems With Policy Gradient Reinforcement Learning

Asynchronous reinforcement learning algorithms for solving discrete space path planning problems

Scalable Model-based Policy Optimization for Decentralized Networked Systems

Offline Decentralized Multi-Agent Reinforcement Learning

Reinforcement learning for multi-agent with asynchronous missing information fusion method

An Effective Asynchronous Framework for Small Scale Reinforcement Learning Problems

Policy Consensus-Based Distributed Deterministic Multi-Agent Reinforcement Learning over Directed Graphs

Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning

Distributed Policy Gradient with Variance Reduction in Multi-Agent Reinforcement Learning

A Policy Gradient Algorithm to Alleviate the Multi-Agent Value Overestimation Problem in Complex Environments

Multi-Agent Reinforcement Learning in Stochastic Networked Systems

Efficient Communications for Multi-Agent Reinforcement Learning in Wireless Networks