Abstract:Self-organizing systems consist of autonomous agents that can perform complex tasks and adapt to dynamic environments without a central controller. Prior research often relies on reinforcement learning to enable agents to gain the skills needed for task completion, such as in the box-pushing environment. However, when agents push from opposing directions during exploration, they tend to exert equal and opposite forces on the box, resulting in minimal displacement and inefficient training. This paper proposes a model called Shared Pool of Information (SPI), which enables information to be accessible to all agents and facilitates coordination, reducing force conflicts among agents and enhancing exploration efficiency. Through computer simulations, we demonstrate that SPI not only expedites the training process but also requires fewer steps per episode, significantly improving the agents' collaborative effectiveness.

What problem does this paper attempt to address?

This paper attempts to solve the problems of cooperation and coordination in multi - agent reinforcement learning, especially in the box - pushing problem. Specifically, when multiple agents push a box from opposite directions, they will exert equal and opposite forces, causing the box to hardly move and thus resulting in low training efficiency. To solve this problem, the author proposes a framework named "Shared Pool of Information (SPI)". The main goal of SPI is to promote implicit coordination among agents by providing a shared information repository accessible to all agents, reduce force conflicts, and thus improve exploration efficiency and training speed. The following are the main contributions of the paper: 1. **Reduce force conflicts among agents**: Through the shared information provided by SPI, agents can better align their actions and avoid situations where they cancel each other out. 2. **Improve exploration efficiency**: SPI makes agents more efficient during the exploration process and reduces meaningless actions. 3. **No communication overhead**: Unlike traditional direct communication methods, SPI does not require an additional communication mechanism, thus reducing the complexity of the model and computational overhead. ### Core problems of the paper - **Multi - agent cooperation problem**: How to make multiple agents effectively cooperate to complete tasks without a central controller. - **Inefficient exploration in the box - pushing task**: Agents often exert equal and opposite forces during the exploration process, causing the box to hardly move and resulting in low training efficiency. ### Solutions - **Shared Information Pool (SPI)**: By providing a shared information repository, agents can cooperate without the need for direct communication, reduce ineffective actions, and improve training efficiency. ### Experimental verification Through computer simulation experiments, the author has proven that SPI not only accelerates the training process but also significantly improves the cooperation effect of agents. The experimental results show that agents using SPI can find the successful path more quickly during the training process, and can also terminate more quickly in case of failure, reducing unnecessary steps. In conclusion, this paper solves the common cooperation and coordination problems in multi - agent reinforcement learning by introducing the SPI framework, especially in the box - pushing task, greatly improving the training efficiency and the cooperation ability of agents.

Efficient Training in Multi-Agent Reinforcement Learning: A Communication-Free Framework for the Box-Pushing Problem

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Learning Effective Communication for Cooperative Pursuit with Multi-Agent Reinforcement Learning

Team-wise effective communication in multi-agent reinforcement learning

Learning Efficient Communication in Cooperative Multi-Agent Environment

Effective Master-Slave Communication On A Multi-Agent Deep Reinforcement Learning System

A Comparison of Various Approaches to Reinforcement Learning Algorithms for Multi-robot Box Pushing

A new multi-agent reinforcement learning approach

Research on Multi-Agent Communication and Collaborative Decision-Making Based on Deep Reinforcement Learning

Efficient Distributed Framework for Collaborative Multi-Agent Reinforcement Learning

Effective Communications: A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning over Noisy Channels

A Novel Multi-Agent Reinforcement Learning Approach

Fast Conflict Resolution Based on Reinforcement Learning in Multi-Agent System

Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Survey

Efficient Communication via Self-Supervised Information Aggregation for Online and Offline Multiagent Reinforcement Learning

Learning Efficient Multi-agent Communication: An Information Bottleneck Approach

Improved Reinforcement Learning in Cooperative Multi-agent Environments Using Knowledge Transfer

DESIGN OF SELF-ORGANIZING SYSTEMS USING MULTI-AGENT REINFORCEMENT LEARNING AND THE COMPROMISE DECISION SUPPORT PROBLEM CONSTRUCT

Learning Task-Oriented Channel Allocation for Multi-Agent Communication

Multi-Task Multi-Agent Shared Layers are Universal Cognition of Multi-Agent Coordination