Abstract:Deep transformer neural network models have improved the predictive accuracy of intelligent text processing systems in the biomedical domain. They have obtained state-of-the-art performance scores on a wide variety of biomedical and clinical Natural Language Processing (NLP) benchmarks. However, the robustness and reliability of these models has been less explored so far. Neural NLP models can be easily fooled by adversarial samples, i.e. minor changes to input that preserve the meaning and understandability of the text but force the NLP system to make erroneous decisions. This raises serious concerns about the security and trust-worthiness of biomedical NLP systems, especially when they are intended to be deployed in real-world use cases. We investigated the robustness of several transformer neural language models, i.e. BioBERT, SciBERT, BioMed-RoBERTa, and Bio-ClinicalBERT, on a wide range of biomedical and clinical text processing tasks. We implemented various adversarial attack methods to test the NLP systems in different attack scenarios. Experimental results showed that the biomedical NLP models are sensitive to adversarial samples; their performance dropped in average by 21 and 18.9 absolute percent on character-level and word-level adversarial noise, respectively, on Micro-F1, Pearson Correlation, and Accuracy measures. Conducting extensive adversarial training experiments, we fine-tuned the NLP models on a mixture of clean samples and adversarial inputs. Results showed that adversarial training is an effective defense mechanism against adversarial noise; the models’ robustness improved in average by 11.3 absolute percent. In addition, the models’ performance on clean data increased in average by 2.4 absolute percent, demonstrating that adversarial training can boost generalization abilities of biomedical NLP systems. This study takes an important step towards revealing vulnerabilities of deep neural language models in biomedical NLP applications. It also provides practical and effective strategies to develop secure, trust-worthy, and accurate intelligent text processing systems in the biomedical domain.

Mitigating Closed-model Adversarial Examples with Bayesian Neural Modeling for Enhanced End-to-End Speech Recognition

Query-Efficient Adversarial Attack with Low Perturbation Against End-to-End Speech Recognition Systems

Characterizing Speech Adversarial Examples Using Self-Attention U-Net Enhancement

Boosting Noise Robustness of Acoustic Model via Deep Adversarial Training

Improving adversarial robustness of Bayesian neural networks via multi-task adversarial training

Attacking Bayes: On the Adversarial Robustness of Bayesian Neural Networks

Adversarial Example Devastation and Detection on Speech Recognition System by Adding Random Noise

Jointly Adversarial Enhancement Training for Robust End-to-End Speech Recognition

Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition

Bayesian Neural Network Language Modeling for Speech Recognition

Investigating Raw Wave Deep Neural Networks for End-to-End Speaker Spoofing Detection

Adversarial Separation Network for Speaker Recognition

Improving the robustness and accuracy of biomedical language models through adversarial training

Speech-enhanced and Noise-aware Networks for Robust Speech Recognition

Robustness of Speech Spoofing Detectors Against Adversarial Post-Processing of Voice Conversion

Adversarial Regularization for Attention Based End-to-End Robust Speech Recognition.

Adversarial Example Detection by Classification for Deep Speech Recognition

Bayesian Learning with Information Gain Provably Bounds Risk for a Robust Adversarial Defense

On the robustness of non-intrusive speech quality model by adversarial examples

Selective Audio Adversarial Example in Evasion Attack on Speech Recognition System

Model Access Control Based on Hidden Adversarial Examples for Automatic Speech Recognition