Abstract:In recent years, Natural Language Processing (NLP) models such as Generative Pre-trained Transformer 3 (GPT-3) have shown remarkable improvements in various language-related tasks, including machine translation. However, most studies that have evaluated the performance of NLP models in translation tasks have focused on general-purpose text, leaving the evaluation of their effectiveness in handling specialized text to be relatively unexplored. Therefore, this study aimed to evaluate the effectiveness of GPT-3 in translating specialized Arabic text to English and compare its performance to human translation. To achieve this goal, the study selected ten chapters from a specialized book written in Arabic, covering topics in specialized religious context. The chapters were translated by a professional human translator and by GPT-3 using its translation Application Programming Interface. The translation performance of GPT-3 to was compared to human translation using qualitative measures, specifically the Direct Assessment method. Additionally, the translations were evaluated using two different evaluation metrics, Bidirectional Encoder Representations from Transformers (BERT) score and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metric, which measure the similarity between the translated text and the reference text.The qualitative results show that GPT produced generally understandable translations but failed to capture nuances and cultural context. On the other hand, the quantitative results of the study showed that GPT-3 was able to achieve a relatively high level of accuracy in translating specialized religious text, with comparable scores to human translations in some cases. Specifically, the BERT score of GPT-3 translations was 0.83. The study also found that the Rouge score failed to fully reflect the capabilities of GPT-3 in translating specialized text.Overall, the findings of this study suggest that GPT-3 has promising potential as a translation tool for specialized religious text, but further research is needed to improve its capabilities and address its limitations.

Revolutionising Translation Technology: A Comparative Study of Variant Transformer Models - BERT, GPT, and T5

Advancements in Natural language Processing: An In-depth Review of Language Transformer Models

Transformer: A General Framework from Machine Translation to Others

Overview of the Transformer-based Models for NLP Tasks

Exploring Transformers in Natural Language Generation: GPT, BERT, and XLNet

Applications of transformer-based language models in bioinformatics: a survey

From Transformers to the Future: an In-Depth Exploration of Modern Language Model Architectures

Natural language processing with transformers: a review

Generative AI in the Era of Transformers: Revolutionizing Natural Language Processing with LLMs

Exploring the Effectiveness of GPT-3 in Translating Specialized Religious Text from Arabic to English: A Comparative Study with Human Translation

Survey of transformers and towards ensemble learning using transformers for natural language processing

Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions

How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation

GPT (Generative Pre-Trained Transformer)— A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions

AraT5: Text-to-Text Transformers for Arabic Language Generation

Rethinking the Value of Transformer Components

Transformers in Natural Language Processing: A Comprehensive Review

Investigation on task effect analysis and optimization strategy of multimodal large model based on Transformers architecture for various languages

Distinguishing Translations by Human, NMT, and ChatGPT: A Linguistic and Statistical Approach

Empirical Study on Transformer-based Techniques for Software Engineering