Abstract:Large language models (LLMs) have recently demonstrated state-of-the-art performance across various natural language processing (NLP) tasks, achieving near-human levels in multiple language understanding challenges and aligning closely with the core principles of semantic communication. Inspired by LLMs' advancements in semantic processing, we propose an innovative LLM-enabled semantic communication system framework, named LLM-SC, that applies LLMs directly to the physical layer coding and decoding for the first time. By analyzing the relationship between the training process of LLMs and the optimization objectives of semantic communication, we propose training a semantic encoder through LLMs' tokenizer training and establishing a semantic knowledge base via the LLMs' unsupervised pre-training process. This knowledge base aids in constructing the optimal decoder by providing the prior probability of the transmitted language sequence. Based on this foundation, we derive the optimal decoding criterion for the receiver and introduce the beam search algorithm to further reduce the complexity. Furthermore, we assert that existing LLMs can be employed directly for LLM-SC without additional re-training or fine-tuning. Simulation results demonstrate that LLM-SC outperforms classical DeepSC at signal-to-noise ratios (SNR) exceeding 3 dB, enabling error-free transmission of semantic information under high SNR, which is unattainable by DeepSC. In addition to semantic-level performance, LLM-SC demonstrates compatibility with technical-level performance, achieving approximately 8 dB coding gain for a bit error ratio (BER) of $10^{-3}$ without any channel coding while maintaining the same joint source-channel coding rate as traditional communication systems.
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is: how to use large language model (LLM) technology to improve semantic communication systems, especially applying it to physical - layer encoding and decoding to achieve more efficient and reliable semantic information transmission.
Specifically, the paper proposes a semantic communication system framework based on large language models (LLM - SC), aiming to solve the problem in the following ways:
1. **Semantic Encoding and Decoding**:
- Train the semantic encoder through the tokenizer training of the large language model.
- Use the unsupervised pre - training process of the large language model to build a semantic knowledge base, providing prior probabilities to assist decoding.
2. **Optimizing Decoding Criteria**:
- Derive the optimal decoding criteria at the receiving end and introduce the beam search algorithm in the field of natural language processing to reduce complexity and ensure efficient decoding performance.
3. **No Additional Training Required**:
- Assert that existing large language models can be directly used in the LLM - SC framework without additional re - training or fine - tuning.
4. **Performance Evaluation**:
- Simulation results show that LLM - SC outperforms the classic DeepSC system when the signal - to - noise ratio (SNR) exceeds 3 dB and can achieve error - free semantic information transmission at high SNR.
- In terms of technical - level performance, LLM - SC shows compatibility, achieving an encoding gain of about 8 dB and achieving the best bit error rate (BER) performance at the same joint source - channel coding rate.
### Mathematical Formulas
To better understand these methods, here are some key formulas involved in the paper:
- **Encoding Function**:
\[
S=\varphi(X)
\]
where \(X = (x_1, x_2,\ldots, x_n)\) is the transmitted information sequence, and \(S=(s_1, s_2,\ldots, s_t)\) is the symbol sequence after encoding and modulation.
- **Decoding Function**:
\[
\hat{X}=\varphi^{-1}(\hat{S})
\]
where \(\hat{S}\) is the symbol sequence estimated at the receiving end, and \(\hat{X}\) is the decoded information sequence.
- **Channel Transmission Relationship**:
\[
o_t = h_t\otimes s_t + n_t
\]
where \(h_t\) is the channel impulse response (CSI), \(n_t\) is noise, and \(\otimes\) represents the convolution operation.
- **Optimal Decoding Objective**:
\[
\hat{S}=\arg\max_{(s_i\in S)}P(s_1, s_2,\ldots, s_t|o_1, o_2,\ldots, o_t)
\]
- **Bayes' Formula**:
\[
P(S|O)=\frac{P(O|S)P(S)}{P(O)}
\]
- **Conditional Probability Distribution (AWGN Channel)**:
\[
P(o_i|s_i)=\frac{1}{\sqrt{2\pi\sigma}}e^{-\frac{(s_i - o_i)^2}{2\sigma^2}}
\]
- **Decoding Loss Function**:
\[
L =-\sum_{i = 1}^{t}\log P(w_i|w_{i - 1}, w_{i - 2},\ldots, w_{\max(1,i - N)})
\]
Through these formulas and methods, the paper shows how to apply large language models to semantic communication systems, thereby achieving more efficient and reliable semantic information transmission.