Enhancing Semantic Similarity Understanding in Arabic NLP with Nested Embedding Learning

Omer Nacar,Anis Koubaa

2024-08-01

Abstract:This work presents a novel framework for training Arabic nested embedding models through Matryoshka Embedding Learning, leveraging multilingual, Arabic-specific, and English-based models, to highlight the power of nested embeddings models in various Arabic NLP downstream tasks. Our innovative contribution includes the translation of various sentence similarity datasets into Arabic, enabling a comprehensive evaluation framework to compare these models across different dimensions. We trained several nested embedding models on the Arabic Natural Language Inference triplet dataset and assessed their performance using multiple evaluation metrics, including Pearson and Spearman correlations for cosine similarity, Manhattan distance, Euclidean distance, and dot product similarity. The results demonstrate the superior performance of the Matryoshka embedding models, particularly in capturing semantic nuances unique to the Arabic language. Results demonstrated that Arabic Matryoshka embedding models have superior performance in capturing semantic nuances unique to the Arabic language, significantly outperforming traditional models by up to 20-25\% across various similarity metrics. These results underscore the effectiveness of language-specific training and highlight the potential of Matryoshka models in enhancing semantic textual similarity tasks for Arabic NLP.

Computation and Language

What problem does this paper attempt to address?

The paper aims to address the issue of semantic similarity understanding in Arabic Natural Language Processing (NLP) and improve processing effectiveness through a nested embedding learning approach. Specifically, the study proposes an innovative framework for training Arabic nested embedding models, utilizing multilingual, Arabic-specific, and English-based models to demonstrate the robust capabilities of nested embedding models in various Arabic NLP downstream tasks.

The main contributions include:
1. **Development of an Arabic Natural Language Inference Dataset**: Translating the English Stanford Natural Language Inference (SNLI) and MultiNLI datasets into Arabic, providing a crucial resource for Arabic natural language inference tasks.
2. **Training of Nested Embedding Models**: Training various English and Arabic embedding models and converting them into Matryoshka versions, enhancing their adaptability and performance across different tasks.
3. **Comprehensive Evaluation and Public Release**: Evaluating these trained models and providing valuable insights, while publicly releasing the datasets and models on the Hugging Face platform to promote broader research and application.

Through the aforementioned work, the paper demonstrates the superior performance of Matryoshka embedding models in capturing the unique semantic nuances of Arabic, significantly outperforming traditional models with a 20-25% improvement across various similarity metrics. These results underscore the effectiveness of language-specific training and highlight the potential of Matryoshka models in enhancing semantic text similarity tasks in Arabic NLP.