A Theory for Semantic Channel Coding With Many-to-one Source

Shuai Ma,Huayan Qi,Hang Li,Guangming Shi,Yong Liang,Naofal Al-Dhahir

2023-12-01

Abstract:As one of the potential key technologies of 6G, semantic communication is still in its infancy and there are many open problems, such as semantic entropy definition and semantic channel coding theory. To address these challenges, we investigate semantic information measures and semantic channel coding theorem. Specifically, we propose a semantic entropy definition as the uncertainty in the semantic interpretation of random variable symbols in the context of knowledge bases, which can be transformed into existing semantic entropy definitions under given conditions. Moreover, different from traditional communications, semantic communications can achieve accurate transmission of semantic information under a non-zero bit error rate. Based on this property, we derive a semantic channel coding theorem for a typical semantic communication with many-to-one source (i.e., multiple source sequences express the same meaning), and prove its achievability and converse based on a generalized Fano's inequality. Finally, numerical results verify the effectiveness of the proposed semantic entropy and semantic channel coding theorem.

Information Theory

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on several key challenges in the field of semantic communication:

1. **Definition of Semantic Entropy**: The paper proposes a new definition of semantic entropy to address the limitations of existing definitions in specific tasks. Traditional definitions of semantic entropy usually depend on specific tasks and lack universal applicability. The semantic entropy proposed in the paper not only considers the probability distribution of symbols, but also combines the specific values of symbols and the background knowledge base, thus providing a more general metric.

2. **Semantic Channel Coding Theory**: The paper establishes the channel coding theorem for semantic communication, especially for the case of "many - to - one" semantic sources. Traditional communication systems usually require a zero - bit error rate to achieve accurate transmission, while semantic communication can still accurately transmit semantic information with a non - zero - bit error rate. By introducing the generalized Fano inequality, the paper proves the feasibility and converse proof of this coding theorem.

Specifically, the main contributions of the paper include:

- **Universal Definition of Semantic Entropy**: A definition of semantic entropy based on the uncertainty of semantic interpretation of random variable symbols in the context of the knowledge base is proposed. This definition depends not only on the probability distribution, but also on the specific values of symbols and the background knowledge base. Different from traditional information metrics, semantic information has a certain degree of fuzziness. Different symbols can express the same semantic meaning, and the same symbol may have different semantic meanings under different knowledge bases.

- **Semantic Channel Coding Theorem**: For the semantic communication of "many - to - one" semantic sources, the semantic channel coding theorem is established, and its feasibility and converse theorem are proved. The paper uses the joint typicality tool and the many - to - one mapping relationship to prove its feasibility, and proves the converse theorem through the generalized Fano inequality. This is the first time to theoretically establish the fundamental limit of semantic communication for "many - to - one" semantic sources.

- **Simulation Verification**: The effectiveness of the proposed semantic entropy and semantic channel coding theorem is verified through simulation results. The simulation results show that the accuracy of the classification task is still high at a non - zero - bit error rate, further verifying the rationality of the theory.

These contributions provide theoretical tools for the design of semantic communication, which helps to solve the resource bottleneck problems faced in 6G communication and improve communication efficiency and transmission capacity.