Large AI Model-Based Semantic Communications

Feibo Jiang,Yubo Peng,Li Dong,Kezhi Wang,Kun Yang,Cunhua Pan,Xiaohu You

2024-08-03

Abstract:Semantic communication (SC) is an emerging intelligent paradigm, offering solutions for various future applications like metaverse, mixed reality, and the Internet of Everything. However, in current SC systems, the construction of the knowledge base (KB) faces several issues, including limited knowledge representation, frequent knowledge updates, and insecure knowledge sharing. Fortunately, the development of the large AI model (LAM) provides new solutions to overcome the above issues. Here, we propose a LAM-based SC framework (LAM-SC) specifically designed for image data, where we first apply the segment anything model (SAM)-based KB (SKB) that can split the original image into different semantic segments by universal semantic knowledge. Then, we present an attention-based semantic integration (ASI) to weigh the semantic segments generated by SKB without human participation and integrate them as the semantic aware image. Additionally, we propose an adaptive semantic compression (ASC) encoding to remove redundant information in semantic features, thereby reducing communication overhead. Finally, through simulations, we demonstrate the effectiveness of the LAM-SC framework and the possibility of applying the LAM-based KB in future SC paradigms.

Artificial Intelligence,Networking and Internet Architecture

What problem does this paper attempt to address?

The paper aims to address issues related to the Knowledge Base (KB) in Semantic Communication (SC) systems. Specifically, the current KB in SC systems faces the following three major problems:

1. **Limited Knowledge Representation**: Traditional SC systems typically rely on Deep Neural Networks (DNN) or Knowledge Graphs (KG). However, due to parameter limitations and high annotation costs, these KBs cannot learn rich semantic knowledge from large datasets, thereby affecting their knowledge representation capabilities.

2. **Frequent Knowledge Updates**: In changing environments, existing KBs need to be continuously updated through training and sharing, which in practical applications leads to significant energy and resource consumption, thereby reducing KB efficiency.

3. **Insecure Knowledge Sharing**: Due to different users perceiving different environments, there are discrepancies between the KBs at the source and destination ends, which may cause semantic errors. Therefore, ensuring semantic alignment between the sender and receiver requires frequent transmission of knowledge models, which may contain sensitive information, posing privacy and security risks.

To address these issues, the paper proposes an SC framework based on Large AI Models (LAM), specifically for image data transmission. This framework leverages the advantages of LAM, such as accurate knowledge representation, rich prior background knowledge, and low-cost knowledge updates, to overcome the shortcomings of existing SC systems. The specific implementations include:
- **High-Precision Semantic Segmentation**: Applying large-scale semantic segmentation models as KBs to segment the original image into different semantic fragments.
- **Goal-Oriented Semantic Alignment**: Developing attention mechanisms to integrate important semantic fragments and generate new semantically aware data.
- **High-Quality Semantic Compression**: Proposing an adaptive semantic compression coding scheme to reduce redundant information and significantly lower communication overhead.

The effectiveness of the LAM-SC framework is validated through simulation experiments, demonstrating its potential in various application scenarios.