Optimizing Resource Allocation for Multi-modal Semantic Communication in Mobile AIGC Networks: A Diffusion-based Game Approach

Jian Liu,Ming Xiao,Jinbo Wen,Jiawen Kang,Ruichen Zhang,Tao Zhang,Dusit Niyato,Weiting Zhang,Ying Liu

2024-09-26

Abstract:Mobile Artificial Intelligence-Generated Content (AIGC) networks enable massive users to obtain customized content generation services. However, users still need to download a large number of AIGC outputs from mobile AIGC service providers, which strains communication resources and increases the risk of transmission failures. Fortunately, Semantic Communication (SemCom) can improve transmission efficiency and reliability through semantic information processing. Moreover, recent advances in Generative Artificial Intelligence (GAI) further enhanced the effectiveness of SemCom through its powerful generative capabilities. However, how to strike a balance between high-quality content generation and the size of semantic information transmitted is a major challenge. In this paper, we propose a Generative Diffusion Model (GDM)-based multi-modal SemCom (GM-SemCom) framework. The framework improves the accuracy of information reconstruction by integrating GDMs and multi-modal semantic information and also adopts a controllable extraction module for efficient and controllable problems of unstable data recovery and slow decoding speed in GAI-enabled SemCom. Then, we introduce a novel metric called Age of Semantic Information (AoSI) based on the concept of Age of Information (AoI) to quantify the freshness of semantic information. To address the resource trading problem within the framework, we propose a Stackelberg game model, which integrates the AoSI with psychological factors to provide a comprehensive measure of user utility. Furthermore, we propose a GDM-based algorithm to solve the game under incomplete information. Compared with the traditional deep reinforcement learning algorithms, numerical results demonstrate that the proposed algorithm converges faster and is closer to the Stackelberg equilibrium.

Networking and Internet Architecture

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

The paper aims to address the issue of multimodal semantic communication resource allocation in Mobile Artificial Intelligence Generated Content (Mobile AIGC) networks. Specifically:

1. **Improving Transmission Efficiency and Reliability**: Enhancing data transmission efficiency and reliability by incorporating semantic information processing.
2. **Balancing Content Quality and Semantic Information Size**: Finding a balance between generating high-quality content and the amount of semantic information transmitted, to avoid slowing down transmission due to large amounts of semantic information.
3. **Optimizing Resource Allocation**: Proposing a method based on the Stackelberg game model to optimize resource allocation, ensuring users receive high-quality AIGC services while considering the Age of Semantic Information (AoSI) and the impact of psychological factors on user experience.

To achieve these goals, the paper proposes a multimodal semantic communication framework (GM-SemCom) based on the Generative Diffusion Model (GDM), and introduces a new metric, AoSI, to quantify the freshness of semantic information. Additionally, a GDM-based deep reinforcement learning algorithm is designed to solve the game problem, enabling effective resource allocation under conditions of incomplete information.