Task-Oriented Communication with Out-of-Distribution Detection: An Information Bottleneck Framework

Hongru Li,Wentao Yu,Hengtao He,Jiawei Shao,Shenghui Song,Jun Zhang,Khaled B. Letaief

2024-01-27

Abstract:Task-oriented communication is an emerging paradigm for next-generation communication networks, which extracts and transmits task-relevant information, instead of raw data, for downstream applications. Most existing deep learning (DL)-based task-oriented communication systems adopt a closed-world scenario, assuming either the same data distribution for training and testing, or the system could have access to a large out-of-distribution (OoD) dataset for retraining. However, in practical open-world scenarios, task-oriented communication systems need to handle unknown OoD data. Under such circumstances, the powerful approximation ability of learning methods may force the task-oriented communication systems to overfit the training data (i.e., in-distribution data) and provide overconfident judgments when encountering OoD data. Based on the information bottleneck (IB) framework, we propose a class conditional IB (CCIB) approach to address this problem in this paper, supported by information-theoretical insights. The idea is to extract distinguishable features from in-distribution data while keeping their compactness and informativeness. This is achieved by imposing the class conditional latent prior distribution and enforcing the latent of different classes to be far away from each other. Simulation results shall demonstrate that the proposed approach detects OoD data more efficiently than the baselines and state-of-the-art approaches, without compromising the rate-distortion tradeoff.

Signal Processing,Information Theory,Image and Video Processing

What problem does this paper attempt to address?

The paper attempts to address the problem of handling unknown, Out-of-Distribution (OoD) data in task-oriented communication systems. Specifically, existing deep learning-based task-oriented communication systems typically assume that the training and testing data have the same distribution, or that the system has access to a large amount of OoD datasets for retraining. However, in real-world open scenarios, these systems need to handle unknown OoD data. In such cases, the strong approximation ability of learning methods may lead to overfitting to the training data (i.e., in-distribution data) and provide overly confident judgments when encountering OoD data. This not only reduces inference performance but can also lead to unacceptable consequences, especially in task-oriented communication.

To address this challenge, the paper proposes a Class Conditional Information Bottleneck (CCIB) method based on the Information Bottleneck (IB) framework. This method extracts distinguishable, compact, and informative features by introducing class-conditional prior distributions and enforcing separation between latent representations of different classes. Experimental results show that the proposed CCIB method is more effective in detecting OoD data compared to baseline methods and existing techniques, without affecting the rate-distortion trade-off.

In summary, the main contributions of the paper are:
1. **Problem Definition**: Handling unknown OoD data in task-oriented communication systems.
2. **Method Innovation**: Proposing the CCIB method, which enhances feature distinguishability through class-conditional prior distributions and contrastive learning methods.
3. **Experimental Validation**: Demonstrating the effectiveness of the proposed method through experiments on image classification tasks.