Bag of Lies: Robustness in Continuous Pre-training BERT

Ine Gevers,Walter Daelemans

2024-06-14

Abstract:This study aims to acquire more insights into the continuous pre-training phase of BERT regarding entity knowledge, using the COVID-19 pandemic as a case study. Since the pandemic emerged after the last update of BERT's pre-training data, the model has little to no entity knowledge about COVID-19. Using continuous pre-training, we control what entity knowledge is available to the model. We compare the baseline BERT model with the further pre-trained variants on the fact-checking benchmark Check-COVID. To test the robustness of continuous pre-training, we experiment with several adversarial methods to manipulate the input data, such as training on misinformation and shuffling the word order until the input becomes nonsensical. Surprisingly, our findings reveal that these methods do not degrade, and sometimes even improve, the model's downstream performance. This suggests that continuous pre-training of BERT is robust against misinformation. Furthermore, we are releasing a new dataset, consisting of original texts from academic publications in the LitCovid repository and their AI-generated false counterparts.

Computation and Language

What problem does this paper attempt to address?

The paper aims to explore the impact of Continuous Pre-Training (CPT) on the acquisition of entity knowledge and robustness of the BERT model, particularly concerning new entities not included in the original BERT pre-training data (e.g., COVID-19). Specifically, the study achieves this goal through the following points:

1. **Importance of Entity Knowledge**: The paper attempts to understand whether BERT can utilize newly introduced entity knowledge for fact verification tasks and hypothesizes that the new entity knowledge introduced through CPT can help BERT perform better in fact verification tasks.

2. **Role of Information Authenticity**: It explores the impact of the authenticity of input data (i.e., correct or incorrect information) on BERT's accuracy in fact verification. The study hypothesizes that incorrect information will negatively affect model performance.

3. **Robustness of CPT**: It tests the impact of manipulating input data during the CPT process (e.g., providing incorrect information or shuffling word order) on model performance. The study hypothesizes that CPT is not robust when faced with incorrect information but does not degrade performance when faced with meaningless data (e.g., shuffled word order).

Through these research questions, the authors hope to verify the effectiveness and stability of the CPT technique in handling domain-specific entity knowledge and provide valuable insights for future research.