Abstract:Visual Question Answering (VQA) has recently emerged as a potential research domain, captivating the interest of many in the field of artificial intelligence and computer vision. Despite the prevalence of approaches in English, there is a notable lack of systems specifically developed for certain languages, particularly Vietnamese. This study aims to bridge this gap by conducting comprehensive experiments on the Vietnamese Visual Question Answering (ViVQA) dataset, demonstrating the effectiveness of our proposed model. In response to community interest, we have developed a model that enhances image representation capabilities, thereby improving overall performance in the ViVQA system. Specifically, our model integrates the Bootstrapping Language-Image Pre-training with frozen unimodal models (BLIP-2) and the convolutional neural network EfficientNet to extract and process both local and global features from images. This integration leverages the strengths of transformer-based architectures for capturing comprehensive contextual information and convolutional networks for detailed local features. By freezing the parameters of these pre-trained models, we significantly reduce the computational cost and training time, while maintaining high performance. This approach significantly improves image representation and enhances the performance of existing VQA systems. We then leverage a multi-modal fusion module based on a general-purpose multi-modal foundation model (BEiT-3) to fuse the information between visual and textual features. Our experimental findings demonstrate that our model surpasses competing baselines, achieving promising performance. This is particularly evident in its accuracy of $71.04\%$ on the test set of the ViVQA dataset, marking a significant advancement in our research area. The code is available at <a class="link-external link-https" href="https://github.com/nngocson2002/ViVQA" rel="external noopener nofollow">this https URL</a>.

Utilize Transformers for translating Wikipedia category names

ViDeBERTa: A powerful pre-trained language model for Vietnamese

Contextual Categorization Enhancement through LLMs Latent-Space

Tag transformer.

A Novel Pretrained General-Purpose Vision Language Model for the Vietnamese Language

ViHateT5: Enhancing Hate Speech Detection in Vietnamese With A Unified Text-to-Text Transformer Model

A Combination of BERT and Transformer for Vietnamese Spelling Correction

Sentence Extraction-Based Machine Reading Comprehension for Vietnamese

Enhanced Transformer Architecture for Natural Language Processing

An Effective Method using Phrase Mechanism in Neural Machine Translation

Segatron: Segment-Aware Transformer for Language Modeling and Understanding

ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation

Advancing Vietnamese Visual Question Answering with Transformer and Convolutional Integration

ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing

VieSum: How Robust Are Transformer-based Models on Vietnamese Summarization?

The Evolved Transformer

Content Translation: Computer-assisted translation tool for Wikipedia articles

Transformer Utilization in Medical Image Segmentation Networks

Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation

Transformer-Based Contextualized Language Models Joint with Neural Networks for Natural Language Inference in Vietnamese

BERT-VBD: Vietnamese Multi-Document Summarization Framework