Collaborative Multi-Agent Reinforcement Learning of Caching Optimization in Small-Cell Networks.

Xianzhe Xu,Meixia Tao
DOI: https://doi.org/10.1109/GLOCOM.2018.8647341
2018-01-01
Abstract:Previous works on learning-based caching problems often only focus on a single base station (BS) scenario. For works considering multiple BSs, the coordination among BSs only exists in the cache placement phase after the learning phase that each BS estimates the file popularity independently. In this work, we investigate the cache strategy design problem in small cell networks (SCNs) with multiple small base stations (SBSs) when user preferences are unknown. We model this multi-agent decision making problem in a multi-armed bandit (MAB) perspective. We first tackle this problem with the centralized MAB algorithm and distributed multi-agent MAB (MAMAB) algorithm. For the centralized algorithm, it considers the coordinations among SBSs but the computational complexity grows exponentially with the number of SBSs. For the distributed one, the computational complexity grows linearly with the number of SBSs but the coordinations among SBSs are totally ignored. To take both coordination among SBSs and computational complexity into account, we propose a collaborative MAMAB algorithm to learn the cache strategy directly, rather than learning the user preferences. The simulation results show that the collaborative MAMAB approaches the performance of the greedy algorithm when the user preferences are perfectly known.
What problem does this paper attempt to address?