Abstract:Natural Language Inference (NLI) is a cornerstone of Natural Language Processing (NLP), providing insights into the entailment relationships between text pairings. It is a critical component of Natural Language Understanding (NLU), demonstrating the ability to extract information from spoken or written interactions. NLI is mainly concerned with determining the entailment relationship between two statements, known as the premise and hypothesis. When the premise logically implies the hypothesis, the pair is labeled "entailment". If the hypothesis contradicts the premise, the pair receives the "contradiction" label. When there is insufficient evidence to establish a connection, the pair is described as "neutral". Despite the success of Large Language Models (LLMs) in various tasks, their effectiveness in NLI remains constrained by issues like low-resource domain accuracy, model overconfidence, and difficulty in capturing human judgment disagreements. This study addresses the underexplored area of evaluating LLMs in low-resourced languages such as Bengali. Through a comprehensive evaluation, we assess the performance of prominent LLMs and state-of-the-art (SOTA) models in Bengali NLP tasks, focusing on natural language inference. Utilizing the XNLI dataset, we conduct zero-shot and few-shot evaluations, comparing LLMs like GPT-3.5 Turbo and Gemini 1.5 Pro with models such as BanglaBERT, Bangla BERT Base, DistilBERT, mBERT, and sahajBERT. Our findings reveal that while LLMs can achieve comparable or superior performance to fine-tuned SOTA models in few-shot scenarios, further research is necessary to enhance our understanding of LLMs in languages with modest resources like Bengali. This study underscores the importance of continued efforts in exploring LLM capabilities across diverse linguistic contexts.

A novel Data and Model Centric artificial intelligence based approach in developing high-performance Named Entity Recognition for Bengali Language

Gazetteer-Enhanced Bangla Named Entity Recognition with BanglaBERT Semantic Embeddings K-Means-Infused CRF Model

Enhancing Low Resource NER Using Assisting Language And Transfer Learning

Named Entity Recognition for Nepali Language

AsNER -- Annotated Dataset and Baseline for Assamese Named Entity recognition

DistALANER: Distantly Supervised Active Learning Augmented Named Entity Recognition in the Open Source Software Ecosystem

CL-NERIL: A Cross-Lingual Model for NER in Indian Languages

An Open-Source Dataset and A Multi-Task Model for Malay Named Entity Recognition

A Novel Approach to Enhance the Performance of Semantic Search in Bengali using Neural Net and other Classification Techniques

A deep neural network-based model for named entity recognition for Hindi language

Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages

Towards Deep Learning in Hindi NER: An approach to tackle the Labelled Data Scarcity

Named Entity Recognition for English Language Using Deep Learning Based Bi Directional LSTM-RNN

Recent Trends in Named Entity Recognition (NER)

A Named Entity Recognition Approach for Albanian Using Deep Learning

Deep learning based question answering system in Bengali

Comprehensive Overview of Named Entity Recognition: Models, Domain-Specific Applications and Challenges

Unraveling the Dominance of Large Language Models Over Transformer Models for Bangla Natural Language Inference: A Comprehensive Study

Towards Malay named entity recognition: an open-source dataset and a multi-task framework

A Survey of Named Entity Recognition in Assamese and other Indian Languages

Enriching Urdu NER with BERT Embedding, Data Augmentation, and Hybrid Encoder-CNN Architecture