Abstract:Background: Amidst the COVID-19 pandemic, misinformation on social media has posed significant threats to public health. Detecting and predicting the spread of misinformation are crucial for mitigating its adverse effects. However, prevailing frameworks for these tasks have predominantly focused on post-level signals of misinformation, neglecting features of the broader information environment where misinformation originates and proliferates. Objective: This study aims to create a novel framework that integrates the uncertainty of the information environment into misinformation features, with the goal of enhancing the model's accuracy in tasks such as misinformation detection and predicting the scale of dissemination. The objective is to provide better support for online governance efforts during health crises. Methods: In this study, we embraced uncertainty features within the information environment and introduced a novel Environmental Uncertainty Perception (EUP) framework for the detection of misinformation and the prediction of its spread on social media. The framework encompasses uncertainty at 4 scales of the information environment: physical environment, macro-media environment, micro-communicative environment, and message framing. We assessed the effectiveness of the EUP using real-world COVID-19 misinformation data sets. Results: The experimental results demonstrated that the EUP alone achieved notably good performance, with detection accuracy at 0.753 and prediction accuracy at 0.71. These results were comparable to state-of-the-art baseline models such as bidirectional long short-term memory (BiLSTM; detection accuracy 0.733 and prediction accuracy 0.707) and bidirectional encoder representations from transformers (BERT; detection accuracy 0.755 and prediction accuracy 0.728). Additionally, when the baseline models collaborated with the EUP, they exhibited improved accuracy by an average of 1.98% for the misinformation detection and 2.4% for spread-prediction tasks. On unbalanced data sets, the EUP yielded relative improvements of 21.5% and 5.7% in macro-F1-score and area under the curve, respectively. Conclusions: This study makes a significant contribution to the literature by recognizing uncertainty features within information environments as a crucial factor for improving misinformation detection and spread-prediction algorithms during the pandemic. The research elaborates on the complexities of uncertain information environments for misinformation across 4 distinct scales, including the physical environment, macro-media environment, micro-communicative environment, and message framing. The findings underscore the effectiveness of incorporating uncertainty into misinformation detection and spread prediction, providing an interdisciplinary and easily implementable framework for the field.

An Empirical Study on Uncertainty Identification in Social Media Context

The UIR Uncertainty Corpus for Chinese: Annotating Chinese Microblog Corpus for Uncertainty Identification from Social Media.

Uncertainty Visualization for Mobile and Wearable Devices Based Activity Recognition Systems

An Uncertainty-Aware Approach for Exploratory Microblog Retrieval

Uncertainty in Natural Language Processing: Sources, Quantification, and Applications

An Environmental Uncertainty Perception Framework for Misinformation Detection and Spread Prediction in the COVID-19 Pandemic: Artificial Intelligence Approach

UAMNer: uncertainty-aware multimodal named entity recognition in social media posts

A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice

Quantifying Uncertainties in Natural Language Processing Tasks

UnScientify: Detecting Scientific Uncertainty in Scholarly Full Text

On Uncertainty In Natural Language Processing

Social uncertainty in the digital world

Are you sure? Analysing Uncertainty Quantification Approaches for Real-world Speech Emotion Recognition

ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation

Ambiguity Meets Uncertainty: Investigating Uncertainty Estimation for Word Sense Disambiguation

Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness

Uncertainty over Uncertainty: Investigating the Assumptions, Annotations, and Text Measurements of Economic Policy Uncertainty

Estimating the Uncertainty in Emotion Class Labels with Utterance-Specific Dirichlet Priors

Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models

Incorporating Uncertain Segmentation Information into Chinese NER for Social Media Text

A Survey on Uncertainty Quantification Methods for Deep Learning