Abstract:Visual Question Answering (VQA) has recently emerged as a potential research domain, captivating the interest of many in the field of artificial intelligence and computer vision. Despite the prevalence of approaches in English, there is a notable lack of systems specifically developed for certain languages, particularly Vietnamese. This study aims to bridge this gap by conducting comprehensive experiments on the Vietnamese Visual Question Answering (ViVQA) dataset, demonstrating the effectiveness of our proposed model. In response to community interest, we have developed a model that enhances image representation capabilities, thereby improving overall performance in the ViVQA system. Specifically, our model integrates the Bootstrapping Language-Image Pre-training with frozen unimodal models (BLIP-2) and the convolutional neural network EfficientNet to extract and process both local and global features from images. This integration leverages the strengths of transformer-based architectures for capturing comprehensive contextual information and convolutional networks for detailed local features. By freezing the parameters of these pre-trained models, we significantly reduce the computational cost and training time, while maintaining high performance. This approach significantly improves image representation and enhances the performance of existing VQA systems. We then leverage a multi-modal fusion module based on a general-purpose multi-modal foundation model (BEiT-3) to fuse the information between visual and textual features. Our experimental findings demonstrate that our model surpasses competing baselines, achieving promising performance. This is particularly evident in its accuracy of $71.04\%$ on the test set of the ViVQA dataset, marking a significant advancement in our research area. The code is available at <a class="link-external link-https" href="https://github.com/nngocson2002/ViVQA" rel="external noopener nofollow">this https URL</a>.

A Vietnamese Text-Based Conversational Agent

Teaching Machines to Converse

Uman-in-thel oop

Building a Vietnamese question answering system based on knowledge graph and distributed CNN

Understanding what the users say in chatbots: A case study for the Vietnamese language

ViConsFormer: Constituting Meaningful Phrases of Scene Texts using Transformer-based Method in Vietnamese Text-based Visual Question Answering

Dialogue Act Segmentation for Vietnamese Human-Human Conversational Texts

Exploring Chatbot AI in improving vocational students’ English pronunciation

Conversational Machine Reading Comprehension for Vietnamese Healthcare Texts

Vietnamese Legal Information Retrieval in Question-Answering System

Chatbot System Architecture

An intelligent knowledge-based chatbot for customer service

SPBERTQA: A Two-Stage Question Answering System Based on Sentence Transformers for Medical Texts

Design and implementation of a VoIP PBX integrated Vietnamese virtual assistant: a case study

A Chinese Question Answering System For Specific Domain

Advancing Vietnamese Visual Question Answering with Transformer and Convolutional Integration

VAIS ASR: Building a conversational speech recognition system using language model combination

Building a Semantic Role Labelling System for Vietnamese

Document Searching System based on Natural Language Query Processing for Vietnam Open Courseware Library

Revealing Weaknesses of Vietnamese Language Models Through Unanswerable Questions in Machine Reading Comprehension