Abstract:The Information Bottleneck (IB) method is an information theoretical framework to design a parsimonious and tunable feature-extraction mechanism, such that the extracted features are maximally relevant to a specific learning or inference task. Despite its theoretical value, the IB is based on a functional optimization problem that admits a closed form solution only on specific cases (e.g., Gaussian distributions), making it difficult to be applied in most applications, where it is necessary to resort to complex and approximated variational implementations. To overcome this limitation, we propose an approach to adapt the closed-form solution of the Gaussian IB to a general task. Whichever is the inference task to be performed by a (possibly deep) neural-network, the key idea is to opportunistically design a regression sub-task, embedded in the original problem, where we can safely assume a (joint) multivariate normality between the sub-task's inputs and outputs. In this way we can exploit a fixed and pre-trained neural network to process the input data, using a tunable number of features, to trade data-size and complexity for accuracy. This approach is particularly useful every time a device needs to transmit data (or features) to a server that has to fulfil an inference task, as it provides a principled way to extract the most relevant features for the task to be executed, while looking for the best trade-off between the size of the feature vector to be transmitted, inference accuracy, and complexity. Extensive simulation results testify the effectiveness of the proposed method and encourage to further investigate this research line.

A Variational Information Bottleneck Method for Network Intrusion Detection

Visualizing Information Bottleneck through Variational Inference

Deep Variational Multivariate Information Bottleneck -- A Framework for Variational Losses

Tighter Bounds on the Information Bottleneck with Application to Deep Learning

Distributed Variational Information Bottleneck for IOT Environments

A Novel Method for Feature Learning and Network Intrusion Classification

Uncertainty in the Variational Information Bottleneck

Image-Based Ship Detection Using Deep Variational Information Bottleneck

A multiscale approach for network intrusion detection based on variance–covariance subspace distance and EQL v2

Layer-wise Learning of Stochastic Neural Networks with Information Bottleneck

On the Difference Between the Information Bottleneck and the Deep Information Bottleneck

Variational Predictive Information Bottleneck

Improving the Classification Effectiveness of Intrusion Detection by Using Improved Conditional Variational AutoEncoder and Deep Neural Network

A computationally efficient dimensionality reduction and attack classification approach for network intrusion detection

Constrained Twin Variational Auto-Encoder for Intrusion Detection in IoT Systems

Flexible Variational Information Bottleneck: Achieving Diverse Compression with a Single Training

An Intrusion Detection Model Based on Feature Reduction and Convolutional Neural Networks

Network intrusion detection based on variational quantum convolution neural network

Opportunistic Information-Bottleneck for Goal-oriented Feature Extraction and Communication

A-CAVE: Network abnormal traffic detection algorithm based on variational autoencoder

Exploring the Trade-Off in the Variational Information Bottleneck for Regression with a Single Training Run