Abstract:Deep transformer neural network models have improved the predictive accuracy of intelligent text processing systems in the biomedical domain. They have obtained state-of-the-art performance scores on a wide variety of biomedical and clinical Natural Language Processing (NLP) benchmarks. However, the robustness and reliability of these models has been less explored so far. Neural NLP models can be easily fooled by adversarial samples, i.e. minor changes to input that preserve the meaning and understandability of the text but force the NLP system to make erroneous decisions. This raises serious concerns about the security and trust-worthiness of biomedical NLP systems, especially when they are intended to be deployed in real-world use cases. We investigated the robustness of several transformer neural language models, i.e. BioBERT, SciBERT, BioMed-RoBERTa, and Bio-ClinicalBERT, on a wide range of biomedical and clinical text processing tasks. We implemented various adversarial attack methods to test the NLP systems in different attack scenarios. Experimental results showed that the biomedical NLP models are sensitive to adversarial samples; their performance dropped in average by 21 and 18.9 absolute percent on character-level and word-level adversarial noise, respectively, on Micro-F1, Pearson Correlation, and Accuracy measures. Conducting extensive adversarial training experiments, we fine-tuned the NLP models on a mixture of clean samples and adversarial inputs. Results showed that adversarial training is an effective defense mechanism against adversarial noise; the models’ robustness improved in average by 11.3 absolute percent. In addition, the models’ performance on clean data increased in average by 2.4 absolute percent, demonstrating that adversarial training can boost generalization abilities of biomedical NLP systems. This study takes an important step towards revealing vulnerabilities of deep neural language models in biomedical NLP applications. It also provides practical and effective strategies to develop secure, trust-worthy, and accurate intelligent text processing systems in the biomedical domain.

BioADAPT-MRC: Adversarial Learning-based Domain Adaptation Improves Biomedical Machine Reading Comprehension Task

BioADAPT-MRC: Adversarial Learning-based Domain Adaptation Improves Biomedical Machine Reading Comprehension Task

Improving Biomedical Named Entity Recognition with a Unified Multi-Task MRC Framework

Adversarial Domain Adaptation for Machine Reading Comprehension

Demonstration-based learning for few-shot biomedical named entity recognition under machine reading comprehension

Multi-task Transfer Learning for Biomedical Machine Reading Comprehension.

MSCDA: Multi-level Semantic-guided Contrast Improves Unsupervised Domain Adaptation for Breast MRI Segmentation in Small Datasets

Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss

C-MADA: Unsupervised Cross-Modality Adversarial Domain Adaptation framework for medical Image Segmentation

Continual Domain Adaptation for Machine Reading Comprehension

Unsupervised Domain Adaptation from Axial to Short-Axis Multi-Slice Cardiac MR Images by Incorporating Pretrained Task Networks

A Robust Adversarial Training Approach to Machine Reading Comprehension

Deep scaled dot-product attention based domain adaptation model for biomedical question answering

A novel multi-domain machine reading comprehension model with domain interference mitigation

Automated Detection of Adverse Drug Reactions in the Biomedical Literature Using Convolutional Neural Networks and Biomedical Word Embeddings

Automatic Multi-Class Cardiovascular Magnetic Resonance Image Quality Assessment using Unsupervised Domain Adaptation in Spatial and Frequency Domains

Adversarial Transfer Learning for Biomedical Named Entity Recognition

Adapting Large Language Models to Domains via Reading Comprehension

Improving the robustness and accuracy of biomedical language models through adversarial training

Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources

Embracing the disharmony in medical imaging: A Simple and effective framework for domain adaptation