Abstract:While resources for English language are fairly sufficient to understand content on social media, similar resources in Arabic are still immature. The main reason that the resources in Arabic are insufficient is that Arabic has many dialects in addition to the standard version (MSA). Arabs do not use MSA in their daily communications; rather, they use dialectal versions. Unfortunately, social users transfer this phenomenon into their use of social media platforms, which in turn has raised an urgent need for building suitable AI models for language-dependent applications. Existing machine translation (MT) systems designed for MSA fail to work well with Arabic dialects. In light of this, it is necessary to adapt to the informal nature of communication on social networks by developing MT systems that can effectively handle the various dialects of Arabic. Unlike for MSA that shows advanced progress in MT systems, little effort has been exerted to utilize Arabic dialects for MT systems. While few attempts have been made to build translation datasets for dialectal Arabic, they are domain dependent and are not OSN cultural-language friendly. In this work, we attempt to alleviate these limitations by proposing an online social network-based multidialect Arabic dataset that is crafted by contextually translating English tweets into four Arabic dialects: Gulf, Yemeni, Iraqi, and Levantine. To perform the translation, we followed our proposed guideline framework for content translation, which could be universally applicable for translation between foreign languages and local dialects. We validated the authenticity of our proposed dataset by developing neural MT models for four Arabic dialects. Our results have shown a superior performance of our NMT models trained using our dataset. We believe that our dataset can reliably serve as an Arabic multidialectal translation dataset for informal MT tasks.

Arabic Transliteration of Romanized Tunisian Dialect Text: A Preliminary Investigation

Normalized Orthography for Tunisian Arabic

Exploiting Dialect Identification in Automatic Dialectal Text Normalization

A Computerized Reversible Arabic Transliteration System

Automatic Standardization of Arabic Dialects for Machine Translation

Automatic Romanization of Arabic Bibliographic Records

A deep learning based approach for extracting Arabic handwriting: applied calligraphy and old cursive

Hybrid approaches for automatic vowelization of Arabic texts

Identification of arabic word from bilingual text using character features

Rule-Based Machine Translation from Tunisian Dialect to Modern Standard Arabic

Leveraging Data Collection and Unsupervised Learning for Code-switched Tunisian Arabic Automatic Speech Recognition

Sentiment Analysis Dataset in Moroccan Dialect: Bridging the Gap Between Arabic and Latin Scripted dialect

Design of Arabic Diacritical Marks

Arabic dialect identification in social media: A hybrid model with transformer models and BiLSTM

Transferring Informal Text in Arabic as Low Resource Languages: State-of-the-Art and Future Research Directions

Linguistic disparities in cross-language automatic speech recognition transfer from Arabic to Tashlhiyt

An Efficient Language-Independent Multi-Font OCR for Arabic Script

Sentiment Analysis Classification for Text in Social Media: Application to Tunisian Dialect

An Assessment of Arabic Transliteration Systems

OSN-MDAD: Machine Translation Dataset for Arabic Multi-Dialectal Conversations on Online Social Media

Content-Localization based Neural Machine Translation for Informal Dialectal Arabic: Spanish/French to Levantine/Gulf Arabic