A Multi-Task Oriented Semantic Communication Framework for Autonomous Vehicles

Eslam Eldeeb,Mohammad Shehab,Hirley Alves

2024-03-06

Abstract:Task-oriented semantic communication is an emerging technology that transmits only the relevant semantics of a message instead of the whole message to achieve a specific task. It reduces latency, compresses the data, and is more robust in low SNR scenarios. This work presents a multi-task-oriented semantic communication framework for connected and autonomous vehicles (CAVs). We propose a convolutional autoencoder (CAE) that performs the semantic encoding of the road traffic signs. These encoded images are then transmitted from one CAV to another CAV through satellite in challenging weather conditions where visibility is impaired. In addition, we propose task-oriented semantic decoders for image reconstruction and classification tasks. Simulation results show that the proposed framework outperforms the conventional schemes, such as QAM-16, regarding the reconstructed image's similarity and the classification's accuracy. In addition, it can save up to 89 % of the bandwidth by sending fewer bits.

Networking and Internet Architecture,Artificial Intelligence,Information Theory,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to improve the traffic sign image transmission efficiency and accuracy between connected autonomous vehicles (CAVs) through a multi - task - oriented semantic communication framework under complex weather conditions. Specifically, the research aims to:

1. **Reduce the amount of data transmission**: By transmitting only semantic information related to specific tasks instead of the entire image, thereby reducing bandwidth usage and saving resources.
2. **Improve transmission robustness**: Ensure the accuracy and reliability of image transmission in low signal - to - noise ratio (SNR) scenarios, such as in bad weather conditions.
3. **Achieve efficient task processing**: The receiving end can complete image reconstruction or classification tasks based on the received semantic information and make decisions accordingly.

To solve these problems, the authors propose a multi - task semantic communication framework based on convolutional autoencoder (CAE), which can perform well in the following aspects:
- **Image reconstruction**: Encode traffic sign images through a semantic encoder, then transmit them to other autonomous vehicles via satellite, and the receiving end reconstructs the images through a decoder.
- **Image classification**: Classify the encoded semantic information through a convolutional neural network (CNN) to identify the categories of traffic signs.

### Specific contributions
- **Multi - task semantic communication framework**: Propose a framework that can efficiently transmit traffic sign image semantic information between autonomous vehicles, and the receiving end can perform image reconstruction or classification tasks.
- **Vehicle - satellite - vehicle link assumption**: Assume a communication link from vehicle to satellite and then to vehicle, which is especially suitable for bad weather conditions with low visibility.
- **Superior performance**: Experimental results show that this framework is superior to traditional schemes (such as QAM - 16) in terms of data transmission volume and classification accuracy, especially in low SNR scenarios.

### Summary of mathematical formulas
The main formulas involved in the paper include:

- The objective function of the optimization problem:
\[
P1: \min \sum_{i = 1}^{S}(1-\hat{p}_{ic})+\sum_{j = 1}^{J}\sum_{k = 1}^{K}|s_i(j,k)-\hat{s}_i(j,k)|
\]
where:
- \(S\) is the total number of images,
- \(C\) is the total number of categories,
- \(\hat{p}_{ic}\) is the predicted category probability,
- \(s_i(j,k)\) and \(\hat{s}_i(j,k)\) are the pixel values of the original image and the reconstructed image respectively.

- The structural similarity index (SSIM) calculation formula:
\[
SSIM(s,\hat{s})=\frac{(2\mu_s\mu_{\hat{s}} + c_1)(2\sigma_{s\hat{s}}+c_2)}{(\mu_s^2+\mu_{\hat{s}}^2 + c_1)(\sigma_s^2+\sigma_{\hat{s}}^2 + c_2)}
\]
where:
- \(\mu_s\) and \(\mu_{\hat{s}}\) are the means of the original image and the reconstructed image respectively,
- \(\sigma_s^2\) and \(\sigma_{\hat{s}}^2\) are the variances of the original image and the reconstructed image respectively,
- \(c_1\) and \(c_2\) are small constants used to stabilize the denominator.

These formulas and methods together form the core technology of this paper, aiming to improve the communication efficiency and reliability of autonomous vehicles in complex environments.