Abstract:Document classification is considered a critical element in automated document processing systems. In recent years multi-modal approaches have become increasingly popular for document classification. Despite their improvements, these approaches are underutilized in the industry due to their requirement for a tremendous volume of training data and extensive computational power. In this paper, we attempt to address these issues by embedding textual features directly into the visual space, allowing lightweight image-based classifiers to achieve state-of-the-art results using small-scale datasets in document classification. To evaluate the efficacy of the visual features generated from our approach on limited data, we tested on the standard dataset Tobacco-3482. Our experiments show a tremendous improvement in image-based classifiers, achieving an improvement of 4.64% using ResNet50 with no document pre-training. It also sets a new record for the best accuracy of the Tobacco-3482 dataset with a score of 91.14% using the image-based DocXClassifier with no document pre-training. The simplicity of the approach, its resource requirements, and subsequent results provide a good prospect for its use in industrial use cases.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenges faced by current multi - modal document classification methods in practical deployment, especially the need for large amounts of training data and computational resources. Specifically: 1. **Limitations of Existing Methods**: - Although multi - modal methods have significantly improved in performance, they usually require a large amount of training data and powerful computational resources, which makes them difficult to be widely applied in the industry. - These methods often involve multiple network streams or large multi - modal transformer networks, increasing the computational burden. - Training these models requires self - supervised learning across millions of data points, which is especially difficult for small enterprises as they usually lack sufficient computational resources and training data. - Existing multi - modal methods also need to directly input text and layout information, which may require large - scale modification of existing document processing systems. - These methods are also more difficult to extend to new languages and require additional training data for each target language. 2. **Proposed New Method**: - To overcome the above problems, this paper proposes a new lightweight document classification method - WordVIS. This method enables the image classifier to achieve state - of - the - art performance on small - scale datasets by directly embedding text semantic features into the visual space. - WordVIS encodes text information by assigning RGB colors to each word, thus allowing existing image - based classifiers to directly utilize text cues in documents. - This method does not require any self - supervised pre - training, so it is especially suitable for small - scale datasets and can be easily integrated into existing CNN document classification pipelines without additional language - specific data. 3. **Experimental Verification**: - The authors conducted experiments on the standard dataset Tobacco - 3482. The results show that after using WordVIS, the accuracy of the ResNet50 model has increased by 4.64%, and the DocXClassifier - B model has achieved the best accuracy of 91.14%, exceeding previous methods. - The experiments also show that WordVIS not only improves classification performance but also reduces the required training data and computational resources. In summary, this paper aims to solve the problem of existing document classification methods' dependence on large amounts of data and computational resources in practical applications by proposing a lightweight and efficient method (WordVIS), making it more suitable for wide - scale deployment in the industry.

WordVIS: A Color Worth A Thousand Words

Knowledge-based Document Embedding for Cross-Domain Text Classification

GlobalDoc: A Cross-Modal Vision-Language Framework for Real-World Document Image Retrieval and Classification

Document Classification Based on Word Vectors

Efficient Document Image Classification Using Region-Based Graph Neural Network

A VECTOR SPACE MODEL BASED DOCUMENT CLASSIFICATION SYSTEM [J]

Evaluating Bag-of-visual-words Representations in Scene Classification

ColPali: Efficient Document Retrieval with Vision Language Models

DUBLIN -- Document Understanding By Language-Image Network

Generating descriptive visual words and visual phrases for large-scale image applications

Multimodal Adaptive Inference for Document Image Classification with Anytime Early Exiting

Visual word coding based on difference maximization.

A Novel Approach to Document Classification using WordNet

Building Descriptive and Discriminative Visual Codebook for Large-Scale Image Applications.

A document image classification system fusing deep and machine learning models

Representing visual classification as a linear combination of words

Real-Time Document Image Classification using Deep CNN and Extreme Learning Machines

Deep Learning based Visually Rich Document Content Understanding: A Survey

Enhancing Document Information Analysis with Multi-Task Pre-training: A Robust Approach for Information Extraction in Visually-Rich Documents

VisdaNet: Visual Distillation and Attention Network for Multimodal Sentiment Classification

Learnable Visual Words for Interpretable Image Recognition