Abstract:Game theoretic views of convention generally rest on notions of common knowledge and hyper-rational models of individual behavior. However, decades of work in behavioral economics have questioned the validity of both foundations. Meanwhile, computational neuroscience has contributed a modernized 'dual process' account of decision-making where model-free (MF) reinforcement learning trades off with model-based (MB) reinforcement learning. The former captures habitual and procedural learning while the latter captures choices taken via explicit planning and deduction. Some conventions (e.g. international treaties) are likely supported by cognition that resonates with the game theoretic and MB accounts. However, convention formation may also occur via MF mechanisms like habit learning; though this possibility has been understudied. Here, we demonstrate that complex, large-scale conventions can emerge from MF learning mechanisms. This suggests that some conventions may be supported by habit-like cognition rather than explicit reasoning. We apply MF multi-agent reinforcement learning to a temporo-spatially extended game with incomplete information. In this game, large parts of the state space are reachable only by collective action. However, heterogeneity of tastes makes such coordinated action difficult: multiple equilibria are desirable for all players, but subgroups prefer a particular equilibrium over all others. This creates a coordination problem that can be solved by establishing a convention. We investigate start-up and free rider subproblems as well as the effects of group size, intensity of intrinsic preference, and salience on the emergence dynamics of coordination conventions. Results of our simulations show agents establish and switch between conventions, even working against their own preferred outcome when doing so is necessary for effective coordination.

An Emergence Approach to Behavior Convention in Agent Group through Propagation of Plans

Model-free conventions in multi-agent reinforcement learning with heterogeneous preferences

The Dynamics of Social Conventions in LLM populations: Spontaneous Emergence, Collective Biases and Tipping Points

The Spontaneous Emergence of Conventions: An Experimental Study of Cultural Evolution

Commitments and conventions: The foundation of coordination in multi-agent systems

Agreement problem in multi-Agent systems: nonlinear viewpoint

Can Agents Spontaneously Form a Society? Introducing a Novel Architecture for Generative Multi-Agents to Elicit Social Emergence

Emergence of common concepts, symmetries and conformity in agent groups-an information-theoretic model

Decentralised Emergence of Robust and Adaptive Linguistic Conventions in Populations of Autonomous Agents Grounded in Continuous Worlds

Generalizing meanings from partners to populations: Hierarchical inference supports convention formation on networks

Efficient Human-AI Coordination via Preparatory Language-based Convention

Diverse Conventions for Human-AI Collaboration

Improved Cooperation by Exploiting a Common Signal

Theory of Minds: Understanding Behavior in Groups Through Inverse Planning

Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi

Consensus Formation Among Mobile Agents in Networks of Heterogeneous Interaction Venues

An Agent-Based Model of Leader Emergence and Leadership Perception within a Collective

Towards Full Delegation: Designing Ideal Agentic Behaviors for Travel Planning

Selection from Multiple Equilibriums: An Evolutionary Model of Social Convention Based-on Quasi-linear Preference

Understanding Emergent Behaviours in Multi-Agent Systems with Evolutionary Game Theory

From partners to populations: A hierarchical Bayesian account of coordination and convention