Abstract:Text classification plays an important role in many practical applications. In the real world, there are extremely small datasets. Most existing methods adopt pretrained neural network models to handle this kind of dataset. However, these methods are either difficult to deploy on mobile devices because of their large output size or cannot fully extract the deep semantic information between phrases and clauses. This paper proposes a multimodel-based deep learning framework for short-text multiclass classification with an imbalanced and extremely small dataset. Our framework mainly includes five layers: the encoder layer, the word-level LSTM network layer, the sentence-level LSTM network layer, the max-pooling layer, and the SoftMax layer. The encoder layer uses DistilBERT to obtain context-sensitive dynamic word vectors that are difficult to represent in traditional feature engineering methods. Since the transformer part of this layer is distilled, our framework is compressed. Then, we use the next two layers to extract deep semantic information. The output of the encoder layer is sent to a bidirectional LSTM network, and the feature matrix is extracted hierarchically through the LSTM at the word and sentence level to obtain the fine-grained semantic representation. After that, the max-pooling layer converts the feature matrix into a lower-dimensional matrix, preserving only the obvious features. Finally, the feature matrix is taken as the input of a fully connected SoftMax layer, which contains a function that can convert the predicted linear vector into the output value as the probability of the text in each classification. Extensive experiments on two public benchmarks demonstrate the effectiveness of our proposed approach on an extremely small dataset. It retains the state-of-the-art baseline performance in terms of precision, recall, accuracy, and F1 score, and through the model size, training time, and convergence epoch, we can conclude that our method can be deployed faster and lighter on mobile devices.

Image-text dual neural network with decision strategy for small-sample image classification

Image-Text Dual Model for Small-Sample Image Classification.

Integrated Image-Text Based on Semi-supervised Learning for Small Sample Instance Segmentation

The image annotation algorithm using convolutional features from intermediate layer of deep learning

Small-Sample Image Classification Method of Combining Prototype and Margin Learning.

A Multimodel-Based Deep Learning Framework for Short Text Multiclass Classification with the Imbalanced and Extremely Small Data Set

Deep Neural Network with FGL for Small Dataset Classification.

Intelligent Security Image Classification on Small Sample Learning.

Improving Fine-grained Image Classification with Multimodal Information

Dual Semantic Relationship Attention Network for Image-Text Matching

Text2Model: Text-based Model Induction for Zero-shot Image Classification

Dual-stream Multi-Modal Graph Neural Network for Few-Shot Learning

Deep InterBoost networks for small-sample image classification

Dual-axial Self-Attention Network for Text Classification

Low-Sample Image Classification Based on Intrinsic Consistency Loss and Uncertainty Weighting Method

Dual-stream Reconstruction Network for Multi-label and Few-shot Learning

A text classification network model combining machine learning and deep learning

Text-Guided Mixup Towards Long-Tailed Image Categorization

Semantic Embedded Deep Neural Network: A Generic Approach to Boost Multi-Label Image Classification Performance

Transductive Semantic Decoupling Double Variational Inference for Few-Shot Classification

Meta-learning Siamese Network for Few-Shot Text Classification.