Abstract:The literature in modern machine learning has only negative results for learning to communicate between competitive agents using standard RL. We introduce a modified sender-receiver game to study the spectrum of partially-competitive scenarios and show communication can indeed emerge in a competitive setting. We empirically demonstrate three key takeaways for future research. First, we show that communication is proportional to cooperation, and it can occur for partially competitive scenarios using standard learning algorithms. Second, we highlight the difference between communication and manipulation and extend previous metrics of communication to the competitive case. Third, we investigate the negotiation game where previous work failed to learn communication between independent agents (Cao et al., 2018). We show that, in this setting, both agents must benefit from communication for it to emerge; and, with a slight modification to the game, we demonstrate successful communication between competitive agents. We hope this work overturns misconceptions and inspires more research in competitive emergent communication.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in a competitive environment, can agents spontaneously generate effective communication? Specifically, most of the research results in modern machine - learning literature on using standard reinforcement learning (RL) methods to train competitive agents for communication are negative. This paper introduces a modified sender - receiver game to study communication in partially competitive scenarios and shows that communication can indeed emerge in a competitive environment. ### Main Problems and Goals 1. **The Relationship between Communication and Cooperation**: - Research shows that communication is directly proportional to the degree of cooperation. Communication can occur even in partially competitive situations using standard learning algorithms. 2. **Distinguishing between Communication and Manipulation**: - The author extends previous communication measurement methods to be applicable to competitive situations and emphasizes the difference between communication and manipulation. Communication requires that both parties can benefit from it, while manipulation means that one party is dominant. 3. **Communication in Negotiation Games**: - In previous work, independent agents failed to learn effective communication in a non - iterative two - player negotiation game. In this paper, by slightly modifying the game rules, it is shown that competitive agents can learn effective communication. ### Main Contributions of the Paper 1. **First Prove the Feasibility of Communication in a Competitive Environment**: - The author finds that in modern spontaneous communication, agents can effectively learn to communicate in competitive scenarios, provided that their interests are at least partially aligned. 2. **Define and Distinguish "Communication" and " "Manipulation"**: - With the help of the literature on evolutionary game theory, the author distinguishes between communication and manipulation and proposes a new method for measuring communication in a competitive environment. 3. **Analyze and Refute Previous Assumptions**: - The author re - analyzes the results of Cao et al. [6] and proposes that agents are not unable to learn communication, but have learned an ineffective communication method. They emphasize that communication can be successful only when both parties can benefit from it. 4. **Application to Negotiation Games**: - By making a small modification to the negotiation game, the author shows that competitive agents can learn effective communication and provides practical suggestions for future research. ### Conclusion This paper aims to correct misunderstandings about competitive spontaneous communication and provide a basis for future research. Through strict experimental design and dynamic analysis, the author shows that communication can spontaneously occur between agents in a competitive environment and proposes an effective method for evaluating this communication. This not only expands theoretical possibilities but also provides a new perspective for practical applications.

Emergent Communication under Competition

On Emergent Communication in Competitive Multi-Agent Teams

Emergent Communication through Negotiation

The Emergence of Adversarial Communication in Multi-Agent Reinforcement Learning

Learning Practical Communication Strategies in Cooperative Multi-Agent Reinforcement Learning

Mixed Cooperative-Competitive Communication Using Multi-Agent Reinforcement Learning

Multi-Agent Curricula and Emergent Implicit Signaling

Emergent Communication: Generalization and Overfitting in Lewis Games

Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input

Bidirectional Emergent Language in Situated Environments

Emergence of Cooperation in Two-agent Repeated Games with Reinforcement Learning

Semantics and Spatiality of Emergent Communication

Emergent Translation in Multi-Agent Communication

Learning and communication pressures in neural networks: Lessons from emergent communication

Learning Effective Communication for Cooperative Pursuit with Multi-Agent Reinforcement Learning

Emergent Communication of Generalizations

Learning multiagent coordination in the absence of communication channels

Learning Multi-Agent Communication with Contrastive Learning

Emergent Cooperation under Uncertain Incentive Alignment

Low-Bandwidth Communication Emerges Naturally in Multi-Agent Learning Systems