Privacy-Preserving Task-Oriented Semantic Communications Against Model Inversion Attacks

Yanhu Wang,Shuaishuai Guo,Yiqin Deng,Haixia Zhang,Yuguang Fang

2023-12-06

Abstract:Semantic communication has been identified as a core technology for the sixth generation (6G) of wireless networks. Recently, task-oriented semantic communications have been proposed for low-latency inference with limited bandwidth. Although transmitting only task-related information does protect a certain level of user privacy, adversaries could apply model inversion techniques to reconstruct the raw data or extract useful information, thereby infringing on users' privacy. To mitigate privacy infringement, this paper proposes an information bottleneck and adversarial learning (IBAL) approach to protect users' privacy against model inversion attacks. Specifically, we extract task-relevant features from the input based on the information bottleneck (IB) theory. To overcome the difficulty in calculating the mutual information in high-dimensional space, we derive a variational upper bound to estimate the true mutual information. To prevent data reconstruction from task-related features by adversaries, we leverage adversarial learning to train encoder to fool adversaries by maximizing reconstruction distortion. Furthermore, considering the impact of channel variations on privacy-utility trade-off and the difficulty in manually tuning the weights of each loss, we propose an adaptive weight adjustment method. Numerical results demonstrate that the proposed approaches can effectively protect privacy without significantly affecting task performance and achieve better privacy-utility trade-offs than baseline methods.

Information Theory

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to protect user privacy and prevent model - inversion attacks in task - oriented semantic communication. Specifically, although task - oriented semantic communication achieves low - latency inference by transmitting only task - relevant information and protects user privacy to a certain extent, attackers can still use model - inversion techniques to reconstruct the original data or extract useful information, thus violating user privacy. For this reason, the paper proposes a method based on the information bottleneck and adversarial learning (IBAL) to enhance the defense against model - inversion attacks without affecting task performance.

### Main contributions of the paper:
1. **Propose a privacy - protected task - oriented semantic communication system**: This system is based on the information bottleneck theory and adversarial learning. It can extract and transmit task - relevant features, achieve low - latency edge inference, and prevent attackers from reconstructing the original data through model - inversion techniques.
2. **Avoid directly estimating the mutual information in high - dimensional space**: Through the variational approximation method, the minimization of the IB loss is transformed into the minimization of the upper bound of the IB loss, making the problem solvable.
3. **Propose a communication scheme suitable for dynamic channel conditions**: Based on IBAL, the paper proposes a multi - objective optimization method. It uses multiple gradient descent algorithms (MGDA) to automatically adjust the weights of each loss function, adapt to channel changes, and balance privacy and task performance.
4. **Evaluate the effectiveness of the proposed scheme**: Through numerical experiments on image classification tasks, it is proved that compared with the baseline method, the proposed scheme can effectively protect privacy without significantly affecting task performance.

### Key techniques and methods:
- **Information Bottleneck theory (IB)**: Used to extract task - relevant features and reduce the amount of information between the intermediate representation and the input.
- **Adversarial Learning**: By simulating the game between the attacker and the transmitter, train the transmitter to generate features that are difficult for the attacker to reconstruct.
- **Variational approximation method**: Used to estimate the mutual information in high - dimensional space, making the optimization problem solvable.
- **Multi - objective optimization**: Use the MGDA algorithm to automatically adjust the weights of the loss functions according to the channel conditions to achieve the balance between privacy and task performance.

### Mathematical formulas:
- **IB loss function**:
\[
L_{\text{IB}}=-I(\hat{Z}; Y)+\beta I(\hat{Z}; S)
\]
where \(I(\hat{Z}; Y)\) represents the mutual information between variables \(\hat{Z}\) and \(Y\), \(I(\hat{Z}; S)\) represents the mutual information between variables \(\hat{Z}\) and \(S\), and \(\beta\) is a trade - off parameter.

- **Variational upper bound**:
\[
L_{\text{VIB}}=\mathbb{E}_{p(s, y)}\left\{\mathbb{E}_{p_\theta(\hat{z} | s)}\left[-\log q_\phi(y | \hat{z})\right]+\beta \text{KL}(p_\theta(\hat{z} | s) \| q(\hat{z}))\right\}
\]

- **Multi - objective optimization problem**:
\[
\min_{\lambda \in[0,1]}\left\|\lambda \nabla_\theta L_{\text{VIB}}-(1-\lambda) \nabla_\theta\left(\frac{1}{1+\sigma^2} L_{\text{MSE}}\right)\right\|_2^2
\]

These techniques and methods together form the core content of the paper, aiming to solve the privacy protection problem in task - oriented semantic communication.