Modeling Decision Process in Multi-Agent Systems: A Graphical Markov Game Based Approach

Hao Li,Yuejiang Li,H. Vicky Zhao

2020-01-01

Abstract:Multi-agent decision processes, where multiple agents interact with each other and make decisions independently, can be seen everywhere in life. Many of the multiagent systems in reality have underlying topological structures, which constraint the interactions and decision makings of agents such as social networks, computer networks, and cognitive radio networks. Some works considered the topological structure between agents, while the decision process of each agent is modeled as a simple imitation of neighbors. Thus, it is of critical importance to study and model how agents interact with each other with consideration about long-term rewards and how the system evolves when considering the topological structure between agents. In this paper, we consider the topological structure between agents and formulate the graphical Markov game. In graphical Markov game, each agent can only observe the actions of neighbors and make decisions based on the interactions with them. The goal of each agent is to maximize its long-term cumulative reward. To find the optimal policy of each agent, we implement a policy gradient based algorithm. We compare our framework with graphical evolutionary game theory where agents only consider the current rewards through experiments of different game settings.