Abstract:In this study, the authors present a novel methodology adept at decoding multilingual topic dynamics and identifying communication trends during crises. We focus on dialogues within Tunisian social networks during the Coronavirus Pandemic and other notable themes like sports and politics. We start by aggregating a varied multilingual corpus of comments relevant to these subjects. This dataset undergoes rigorous refinement during data preprocessing. We then introduce our No-English-to-English Machine Translation approach to handle linguistic differences. Empirical tests of this method showed high accuracy and F1 scores, highlighting its suitability for linguistically coherent tasks. Delving deeper, advanced modeling techniques, specifically LDA and HDP models are employed to extract pertinent topics from the translated content. This leads to applying ARIMA time series analysis to decode evolving topic trends. Applying our method to a multilingual Tunisian dataset, we effectively identified key topics mirroring public sentiment. Such insights prove vital for organizations and governments striving to understand public perspectives during crises. Compared to standard approaches, our model outperforms, as confirmed by metrics like Coherence Score, U-mass, and Topic Coherence. Additionally, an in-depth assessment of the identified topics revealed notable thematic shifts in discussions, with our trends identification indicating impressive accuracy, backed by RMSE-based analysis.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in a multilingual environment, especially during a crisis (such as the COVID - 19 pandemic), how to effectively decode and identify the multilingual topic dynamics and trends in social networks. Specifically, the authors focus on: 1. **Challenges in Processing Multilingual Data**: Traditional topic - modeling methods mainly focus on monolingual data, which makes it difficult to conduct a comprehensive analysis of multilingual social media content. The paper proposes a new data - driven method for multilingual topic modeling, aiming to overcome this challenge. 2. **Combination of Translation and Topic Modeling**: To deal with the complexity in a multilingual environment, the authors introduce a non - English - to - English machine translation method, uniformly translate multilingual data into English, and then apply the Latent Dirichlet Allocation (LDA) and Hierarchical Dirichlet Process (HDP) models for topic extraction. 3. **Application of Time - Series Analysis**: Through the AutoRegressive Integrated Moving Average (ARIMA) time - series analysis, the authors further decode the trend of topics changing over time to reveal the key themes of public sentiment and their evolution. ### Specific Problem Description - **Integration and Translation of Multilingual Data**: How to effectively translate text data from different languages (such as Arabic, French, English, etc.) into one language uniformly for subsequent topic modeling. - **Effectiveness of Topic Modeling**: How to ensure that in a multilingual context, topic modeling can accurately capture meaningful topics, and these topics can reflect the real situation of public opinion. - **Accuracy of Trend Identification**: How to identify and predict the trend changes of topics through time - series analysis, especially during a crisis, and these trends are crucial for governments and organizations to understand public sentiment. ### Solution Overview The authors propose a five - stage methodology: 1. **Data Collection**: Collect multilingual pandemic - related comments from multiple sources (such as ASTD, CSTA, TUNIZI corpora). 2. **Data Pre - processing**: Remove punctuation marks, special characters, stop words, etc., and perform word segmentation, lemmatization, etc. to improve data quality. 3. **Machine Translation**: Use bilingual dictionaries, Arabic dictionaries, as well as crowdsourcing translation based on the PROZ platform and the OpenAI API for translation to ensure the accuracy and consistency of translation. 4. **Topic Modeling**: Use the LDA and HDP models to extract meaningful topics from the translated data. 5. **Trend Identification**: Through ARIMA time - series analysis, decode the trend of topics changing over time. Through this method, the authors not only solve the problem of multilingual data processing but also successfully identify the key topics and their trends over time, providing valuable decision - support for governments and organizations.

Decoding Multilingual Topic Dynamics and Trend Identification through ARIMA Time Series Analysis on Social Networks: A Novel Data Translation Framework Enhanced by LDA/HDP Models

Multilingual topic modeling for tracking COVID-19 trends based on Facebook data analysis

Twitter sentiment analysis: An Arabic text mining approach based on COVID-19

Content-Localization based System for Analyzing Sentiment and Hate Behaviors in Low-Resource Dialectal Arabic: English to Levantine and Gulf

Content-Localization based Neural Machine Translation for Informal Dialectal Arabic: Spanish/French to Levantine/Gulf Arabic

Heterogeneous Ensemble Deep Learning Model for Enhanced Arabic Sentiment Analysis

Exploring a Hybrid Deep Learning Framework to Automatically Discover Topic and Sentiment in COVID-19 Tweets

Social Media Monitoring of the COVID-19 Pandemic and Influenza Epidemic With Adaptation for Informal Language in Arabic Twitter Data: Qualitative Study

Sentiment Analysis Classification for Text in Social Media: Application to Tunisian Dialect

A new sentiment analysis method to detect and Analyse sentiments of Covid-19 moroccan tweets using a recommender approach

COVID-19: Detecting Government Pandemic Measures and Public Concerns from Twitter Arabic Data Using Distributed Machine Learning

A Hybrid Multitask Learning Framework with a Fire Hawk Optimizer for Arabic Fake News Detection

Influenza-like Illness Detection from Arabic Facebook Posts Based on Sentiment Analysis and 1D Convolutional Neural Network

Enhanced Arabic disaster data classification using domain adaptation

OSN-MDAD: Machine Translation Dataset for Arabic Multi-Dialectal Conversations on Online Social Media

Enhanced Sentiment Analysis and Topic Modeling During the Pandemic Using Automated Latent Dirichlet Allocation

AraCovTexFinder: Leveraging the transformer-based language model for Arabic COVID-19 text identification

Sentiment Analysis on Arabic Public Opinions toward COVID-19 Vaccines Using Twitter Data

Sentiment Analysis of Arab Tweets: Unveiling Public Opinion Trends Using Machine Learning

The Impact of Online Social Media on Translation Pedagogy and Industry

Language Transfer for Early Warning of Epidemics from Social Media