Abstract:Purpose Social media platforms that disseminate scientific information to the public during the COVID-19 pandemic highlighted the importance of the topic of scientific communication. Content creators in the field, as well as researchers who study the impact of scientific information online, are interested in how people react to these information resources and how they judge them. This study aims to devise a framework for extracting large social media datasets and find specific feedback to content delivery, enabling scientific content creators to gain insights into how the public perceives scientific information. Design/methodology/approach To collect public reactions to scientific information, the study focused on Twitter users who are doctors, researchers, science communicators or representatives of research institutes, and processed their replies for two years from the start of the pandemic. The study aimed in developing a solution powered by topic modeling enhanced by manual validation and other machine learning techniques, such as word embeddings, that is capable of filtering massive social media datasets in search of documents related to reactions to scientific communication. The architecture developed in this paper can be replicated for finding any documents related to niche topics in social media data. As a final step of our framework, we also fine-tuned a large language model to be able to perform the classification task with even more accuracy, forgoing the need of more human validation after the first step. Findings We provided a framework capable of receiving a large document dataset, and, with the help of with a small degree of human validation at different stages, is able to filter out documents within the corpus that are relevant to a very underrepresented niche theme inside the database, with much higher precision than traditional state-of-the-art machine learning algorithms. Performance was improved even further by the fine-tuning of a large language model based on BERT, which would allow for the use of such model to classify even larger unseen datasets in search of reactions to scientific communication without the need for further manual validation or topic modeling. Research limitations/implications The challenges of scientific communication are even higher with the rampant increase of misinformation in social media, and the difficulty of competing in a saturated attention economy of the social media landscape. Our study aimed at creating a solution that could be used by scientific content creators to better locate and understand constructive feedback toward their content and how it is received, which can be hidden as a minor subject between hundreds of thousands of comments. By leveraging an ensemble of techniques ranging from heuristics to state-of-the-art machine learning algorithms, we created a framework that is able to detect texts related to very niche subjects in very large datasets, with just a small amount of examples of texts related to the subject being given as input. Practical implications With this tool, scientific content creators can sift through their social media following and quickly understand how to adapt their content to their current user's needs and standards of content consumption. Originality/value This study aimed to find reactions to scientific communication in social media. We applied three methods with human intervention and compared their performance. This study shows for the first time, the topics of interest which were discussed in Brazil during the COVID-19 pandemic.

Investigating the dissemination of STEM content on social media with computational tools

Short-Form Videos as an Emerging Social Media Tool for STEM Edutainment

Using Social Media to Promote STEM Education: Matching College Students with Role Models

Making Science Education More Accessible: A Case Study of TikTok's Utility as a Science Communication Tool

A systematic identification and analysis of scientists on Twitter

Leveraging spark-based machine learning algorithm for audience sentiment analysis in youtube content

Modeling information diffusion in social media: data-driven observations

Junior scientists spotlight social bonds in seminars for diversity, equity, and inclusion in STEM

A Unified Seeding Framework

The science of YouTube: What factors influence user engagement with online science videos?

Moving Beyond Benchmarks and Competitions: Towards Addressing Social Media Challenges in an Educational Context

Influence of network video film and television communication based on sampling theory

#ILookLikeAnEngineer: Using Social Media Based Hashtag Activism Campaigns as a Lens to Better Understand Engineering Diversity Issues

Identifying Widely Disseminated Scientific Papers on Social Media

Analyzing social media data: A mixed-methods framework combining computational and qualitative text analysis

Engaging Politically Diverse Audiences on Social Media

Optimized discovery of discourse topics in social media: science communication about COVID-19 in Brazil

SocialML: machine learning for social media video creators

Research on the Communication Effect and Improvement of Popular Science Short Videos by Scientific Institutions

Revealing The Secret Power: How Algorithms Can Influence Content Visibility on Social Media

Public Information Sharing Behaviors Analysis over Different Social Media