Abstract:Natural Language Inference (NLI) is a task within Natural Language Processing (NLP) that holds value for various AI applications. However, there have been limited studies on Natural Language Inference in Vietnamese that explore the concept of joint models. Therefore, we conducted experiments using various combinations of contextualized language models (CLM) and neural networks. We use CLM to create contextualized work presentations and use Neural Networks for classification. Furthermore, we have evaluated the strengths and weaknesses of each joint model and identified the model failure points in the Vietnamese context. The highest F1 score in this experiment, up to 82.78\% in the benchmark dataset (ViNLI). By conducting experiments with various models, the most considerable size of the CLM is XLM-R (355M). That combination has consistently demonstrated superior performance compared to fine-tuning strong pre-trained language models like PhoBERT (+6.58\%), mBERT (+19.08\%), and XLM-R (+0.94\%) in terms of F1-score. This article aims to introduce a novel approach or model that attains improved performance for Vietnamese NLI. Overall, we find that the joint approach of CLM and neural networks is simple yet capable of achieving high-quality performance, which makes it suitable for applications that require efficient resource utilization.

Vietnamese Named Entity Recognition using Token Regular Expressions and Bidirectional Inference

End-to-end Recurrent Neural Network Models for Vietnamese Named Entity Recognition: Word-level vs. Character-level

Attentive Neural Network for Named Entity Recognition in Vietnamese

Named Entity Recognition with Bidirectional LSTM-CNNs

Improving Vietnamese Named Entity Recognition from Speech Using Word Capitalization and Punctuation Recovery Models

A Feature-Based Model for Nested Named-Entity Recognition at VLSP-2018 NER Evaluation Campaign

Error Analysis for Vietnamese Named Entity Recognition on Deep Neural Network Models

Rule-based Recognition of Vietnamese Named Entities

An Empirical Study of Discriminative Sequence Labeling Models for Vietnamese Text Processing

VTCC-NLP at NL4Opt competition subtask 1: An Ensemble Pre-trained language models for Named Entity Recognition

Vietnamese Semantic Role Labelling

Building a Semantic Role Labelling System for Vietnamese

Transformer-Based Contextualized Language Models Joint with Neural Networks for Natural Language Inference in Vietnamese

Leveraging Semantic Representations Combined with Contextual Word Representations for Recognizing Textual Entailment in Vietnamese

A Named Entity Recognition Approach for Electronic Medical Records Using BERT Semantic Enhancement and BiLSTM

A Combination of BERT and Transformer for Vietnamese Spelling Correction

Augmenting Part-of-speech Tagging with Syntactic Information for Vietnamese and Chinese

ViDeBERTa: A powerful pre-trained language model for Vietnamese

PhoNLP: A joint multi-task learning model for Vietnamese part-of-speech tagging, named entity recognition and dependency parsing

VAIS ASR: Building a conversational speech recognition system using language model combination

A BERT based Chinese Named Entity Recognition method on ASEAN News