On the benefits of robust models in modulation recognition

Javier Maroto,Gérôme Bovet,Pascal Frossard

DOI: https://doi.org/10.48550/arXiv.2103.14977

2021-03-28

Abstract:Given the rapid changes in telecommunication systems and their higher dependence on artificial intelligence, it is increasingly important to have models that can perform well under different, possibly adverse, conditions. Deep Neural Networks (DNNs) using convolutional layers are state-of-the-art in many tasks in communications. However, in other domains, like image classification, DNNs have been shown to be vulnerable to adversarial perturbations, which consist of imperceptible crafted noise that when added to the data fools the model into misclassification. This puts into question the security of DNNs in communication tasks, and in particular in modulation recognition. We propose a novel framework to test the robustness of current state-of-the-art models where the adversarial perturbation strength is dependent on the signal strength and measured with the "signal to perturbation ratio" (SPR). We show that current state-of-the-art models are susceptible to these perturbations. In contrast to current research on the topic of image classification, modulation recognition allows us to have easily accessible insights on the usefulness of the features learned by DNNs by looking at the constellation space. When analyzing these vulnerable models we found that adversarial perturbations do not shift the symbols towards the nearest classes in constellation space. This shows that DNNs do not base their decisions on signal statistics that are important for the Bayes-optimal modulation recognition model, but spurious correlations in the training data. Our feature analysis and proposed framework can help in the task of finding better models for communication systems.

Signal Processing,Machine Learning

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the vulnerability of deep neural networks (DNNs) to adversarial perturbations in modulation recognition. Specifically, the paper focuses on how to test the robustness of the current state - of - the - art modulation recognition models under different conditions, and proposes a new framework to evaluate the resistance of these models to signal - strength - dependent adversarial perturbations. Through this evaluation, the authors hope to raise awareness of the importance of security when using DNNs in communication systems, and explore the performance of these models in the face of adversarial attacks, as well as the effectiveness of the features they have learned compared to the Bayes - optimal modulation recognition model.

The main contributions of the paper include:
1. **Proposing a robustness framework**: It is used to directly measure the performance of the model against adversarial perturbations, similar to what is done in the field of computer vision.
2. **Proposing a security framework**: It simulates a more realistic scenario, in which attacks occur at the intermediate nodes of the wireless communication system rather than at the receiving end.
3. **Analyzing the features learned by the model**: By projecting the adversarial perturbations onto the constellation map space, the authors find that DNNs do not make decisions based on the signal statistical characteristics important for the modulation recognition task, but rather on accidental correlations in the training data, which makes them vulnerable to changes in channel conditions and other out - of - distribution shifts.

Through these studies, the authors aim to promote further research on the security and robustness of modulation recognition models to ensure that the system can not only resist malicious attacks, but also remain stable under other out - of - distribution conditions.