Abstract:Text classification plays an important role in many practical applications. In the real world, there are extremely small datasets. Most existing methods adopt pretrained neural network models to handle this kind of dataset. However, these methods are either difficult to deploy on mobile devices because of their large output size or cannot fully extract the deep semantic information between phrases and clauses. This paper proposes a multimodel-based deep learning framework for short-text multiclass classification with an imbalanced and extremely small dataset. Our framework mainly includes five layers: the encoder layer, the word-level LSTM network layer, the sentence-level LSTM network layer, the max-pooling layer, and the SoftMax layer. The encoder layer uses DistilBERT to obtain context-sensitive dynamic word vectors that are difficult to represent in traditional feature engineering methods. Since the transformer part of this layer is distilled, our framework is compressed. Then, we use the next two layers to extract deep semantic information. The output of the encoder layer is sent to a bidirectional LSTM network, and the feature matrix is extracted hierarchically through the LSTM at the word and sentence level to obtain the fine-grained semantic representation. After that, the max-pooling layer converts the feature matrix into a lower-dimensional matrix, preserving only the obvious features. Finally, the feature matrix is taken as the input of a fully connected SoftMax layer, which contains a function that can convert the predicted linear vector into the output value as the probability of the text in each classification. Extensive experiments on two public benchmarks demonstrate the effectiveness of our proposed approach on an extremely small dataset. It retains the state-of-the-art baseline performance in terms of precision, recall, accuracy, and F1 score, and through the model size, training time, and convergence epoch, we can conclude that our method can be deployed faster and lighter on mobile devices.

Compressing the Embedding Matrix by a Dictionary Screening Approach in Text Classification.

Embedding Compression for Text Classification Using Dictionary Screening

A Brief Summary of Dictionary Learning Based Approach for Classification (revised)

A dictionary learning approach for classification: separating the particularity and the commonality

Lightweight Conceptual Dictionary Learning for Text Classification Using Information Compression

Stacked Convolutional Deep Encoding Network for Video-Text Retrieval.

Bag-of-Embeddings for Text Classification.

A Compression Algorithm for English-Text Based on Dynamic Dictionary

Adaptive Region Embedding for Text Classification.

Efficient Image-Text Retrieval via Keyword-Guided Pre-Screening

A Multimodel-Based Deep Learning Framework for Short Text Multiclass Classification with the Imbalanced and Extremely Small Data Set

Structure-Sensitive Graph Dictionary Embedding for Graph Classification

Joint Goal for Word Embedding Compression Based on Word Frequency

Self-paced Dictionary Learning for Image Classification

Text Compression-aided Transformer Encoding

Semantic classifier based on compressed sensing for image and video annotation

Smaller Text Classifiers with Discriminative Cluster Embeddings

Jointly Learning Non-negative Projection and Dictionary with Discriminative Graph Constraints for Classification

Fast and Robust Dictionary-based Classification for Image Data

Dictionary Pair-based Data-Free Fast Deep Neural Network Compression

Dictionary Construction for Sparse Representation Classification: A Novel Cluster-Based Approach