COVID-19 Vaccine Misinformation in Middle Income Countries

Jongin Kim,Byeo Rhee Bak,Aditya Agrawal,Jiaxi Wu,Veronika J. Wirtz,Traci Hong,Derry Wijaya

DOI: https://doi.org/10.48550/arXiv.2311.18195

2023-11-30

Abstract:This paper introduces a multilingual dataset of COVID-19 vaccine misinformation, consisting of annotated tweets from three middle-income countries: Brazil, Indonesia, and Nigeria. The expertly curated dataset includes annotations for 5,952 tweets, assessing their relevance to COVID-19 vaccines, presence of misinformation, and the themes of the misinformation. To address challenges posed by domain specificity, the low-resource setting, and data imbalance, we adopt two approaches for developing COVID-19 vaccine misinformation detection models: domain-specific pre-training and text augmentation using a large language model. Our best misinformation detection models demonstrate improvements ranging from 2.7 to 15.9 percentage points in macro F1-score compared to the baseline models. Additionally, we apply our misinformation detection models in a large-scale study of 19 million unlabeled tweets from the three countries between 2020 and 2022, showcasing the practical application of our dataset and models for detecting and analyzing vaccine misinformation in multiple countries and languages. Our analysis indicates that percentage changes in the number of new COVID-19 cases are positively associated with COVID-19 vaccine misinformation rates in a staggered manner for Brazil and Indonesia, and there are significant positive associations between the misinformation rates across the three countries.

Computation and Language,Information Retrieval

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to solve the problem of misinformation spread about COVID - 19 vaccines in Middle Income Countries (MIC). Specifically, the research focuses on the following aspects:

1. **Constructing a multilingual dataset**:
- The researchers created a Twitter dataset from three middle - income countries (Brazil, Indonesia, and Nigeria) related to COVID - 19 vaccine - related misinformation.
- The dataset contains 5,952 tweets, and each tweet was labeled to assess its relevance to the COVID - 19 vaccine, the presence of misinformation, and the topic of misinformation.

2. **Developing an automatic detection model**:
- The researchers developed two methods to improve the COVID - 19 vaccine misinformation detection model: domain - specific pre - training and text augmentation using large - language models.
- These models improved the macro F1 - score by 2.7 to 15.9 percentage points compared to the baseline model.

3. **Large - scale data analysis**:
- The researchers used the best misinformation detection model to analyze 19 million unlabeled tweets collected from 2020 to 2022, demonstrating how to detect and analyze vaccine misinformation in multiple countries and languages.

4. **The relationship between misinformation and the change in the number of cases**:
- The study analyzed the impact of the daily percentage change in new COVID - 19 cases on the daily vaccine misinformation rate. The study shows that there is a significant positive correlation between the change in the number of new cases in Brazil and Indonesia and the vaccine misinformation rate, and this impact is lagged.

### Key formulas

To analyze the impact of the daily percentage change in new cases on the vaccine misinformation rate, the researchers used the following Distributed Lag Model:

\[ y_t=\beta_0+\beta_1 x_t+\beta_2 x_{t - 1}+\ldots+\beta_{15} x_{t - 14}+u_t \]

where:
- \( y_t \) represents the vaccine misinformation rate on date \( t \), defined as:
\[
y_t=\frac{\text{# of tweets with vaccine misinformation}}{\text{# of all tweets related to vaccines}}
\]
- \( x_t \) represents the percentage change in new cases on date \( t \), defined as:
\[
x_t = 100\times\frac{\text{# of new cases on date \( t \)}}{\text{# of new cases on date \( t - 1 \)}}
\]
- \( u_t \) is the error term.

### Conclusion

Through the above research, the paper reveals the spread pattern of vaccine misinformation in middle - income countries and its relationship with the development of the epidemic, providing an important reference for future responses to global public health crises.