Edgar Beck,Hsuan-Yu Lin,Patrick Rückert,Yongping Bao,Bettina von Helversen,Sebastian Fehrler,Kirsten Tracht,Armin Dekorsy
Abstract:As early as 1949, Weaver defined communication in a very broad sense to include all procedures by which one mind or technical system can influence another, thus establishing the idea of semantic communication. With the recent success of machine learning in expert assistance systems where sensed information is wirelessly provided to a human to assist task execution, the need to design effective and efficient communications has become increasingly apparent. In particular, semantic communication aims to convey the meaning behind the sensed information relevant for Human Decision-Making (HDM). Regarding the interplay between semantic communication and HDM, many questions remain, such as how to model the entire end-to-end sensing-decision-making process, how to design semantic communication for the HDM and which information should be provided to the HDM. To address these questions, we propose to integrate semantic communication and HDM into one probabilistic end-to-end sensing-decision framework that bridges communications and psychology. In our interdisciplinary framework, we model the human through a HDM process, allowing us to explore how feature extraction from semantic communication can best support human decision-making. In this sense, our study provides new insights for the design/interaction of semantic communication with models of HDM. Our initial analysis shows how semantic communication can balance the level of detail with human cognitive capabilities while demanding less bandwidth, power, and latency.
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is: How to integrate semantic communication with the human decision - making process (HDM) into an end - to - end perception - decision framework to improve the decision - making efficiency and accuracy in task execution. Specifically, the paper explores the following aspects:
1. **How to model the entire end - to - end perception - decision process**:
- The paper proposes a probabilistic end - to - end framework that combines the communication system with psychology, aiming to bridge the gap between the communication discipline and psychology.
- Through this framework, researchers can explore how the features extracted from semantic communication can best support human decision - making.
2. **Is semantic communication suitable for providing relevant and accurate information required for decision - making?**:
- Researchers explore what information semantic communication should provide and how much information it should provide to ensure the accuracy of human decision - making in the context of a given task.
- For example, in application scenarios such as medical image diagnosis and autonomous driving, how to select and transmit key information to assist human decision - making.
3. **The impact of the provided semantic information on the human decision - making process**:
- The research analyzes how semantic information affects the end - to - end perception - decision process and explores the balance between human cognitive ability and the amount of information.
- The results show that semantic communication can provide sufficient information to support accurate human decision - making without increasing bandwidth, power consumption, and latency.
### Main contributions
The main contributions of the paper include:
- Proposing a probabilistic end - to - end perception - decision framework based on semantic communication, which links perception data with relevant human decision - making information through wireless communication.
- Using a simple but effective HDM model, studying how to perform feature extraction through semantic communication to replace some human processing steps while adapting to the human cognitive process.
- Demonstrating how technical challenges (such as information transmission) interact with human information processing, emphasizing that semantic communication can better match human cognitive ability by adjusting the level of detail of information.
- Simulation results show that the semantic communication system can provide sufficient information for the human decision - making model to make accurate decisions while reducing bandwidth, power consumption, and latency.
- Providing a perspective on future research directions, including information presentation design (such as visualization) and the game - theory perspective of the conflict of interests between the sender and the receiver.
### Formula representation
The formulas involved in the paper are as follows:
1. **Mutual information maximization problem (InfoMax)**:
\[
\arg \max_{p_\theta(x|s)} I_\theta(z;y)=\arg \max_{\theta} E_{z,y \sim p_\theta(z,y)}\left[\ln \frac{p_\theta(z,y)}{p(z)p_\theta(y)}\right]=\arg \max_{\theta} E_{z,y \sim p_\theta(z,y)}[\ln p_\theta(z|y)]
\]
2. **Mutual information lower bound (MILBO)**:
\[
I_\theta(z;y)\geq E_{z,y \sim p_\theta(z,y)}[\ln q_\phi(z|y)] = E_{y \sim p_\theta(y)}\left[E_{z \sim p_\theta(z|y)}[\ln q_\phi(z|y)]\right]= - L_{CE}^{\theta,\phi}
\]
3. **Optimizing encoder and decoder parameters**:
\[
\{\theta^*, \phi^*\}=\arg \min_{\theta, \phi} L_{CE}^{\theta,\phi}
\]
4. **Similarity calculation in the GCM model**:
\[
\text{sim}(\nu_1, \nu_2|\theta_G)=\e