Abstract:Developing the utilized intelligent systems is increasingly important to learn effective text representations, especially extract the sentence features. Numerous previous studies have been concentrated on the task of sentence representation learning based on deep learning approaches. However, the present approaches are mostly proposed with the single task or replied on the labeled corpus when learning the embedding of the sentences. In this paper, we assess the factors in learning sentence representation and propose an efficient unsupervised learning framework with multi-task learning (USR-MTL), in which various text learning tasks are merged into the unitized framework. With the syntactic and semantic features of sentences, three different factors to some extent are reflected in the task of the sentence representation learning that is the wording, or the ordering of the neighbored sentences of a target sentence in other words. Hence, we integrate the word-order learning task, word prediction task, and the sentence-order learning task into the proposed framework to attain meaningful sentence embeddings. Here, the process of sentence embedding learning is reformulated as a multi-task learning framework of the sentence-level task and the two word-level tasks. Moreover, the proposed framework is motivated by an unsupervised learning algorithm utilizing the unlabeled corpus. Based on the experimental results, our approach achieves the state-of-the-art performances on the downstream natural language processing tasks compared to the popular unsupervised representation learning techniques. The experiments on representation visualization and task analysis demonstrate the effectiveness of the tasks in the proposed framework in creating reasonable sentence representations proving the capacity of the proposed unsupervised multi-task framework for the sentence representation learning.

An End-to-End Scalable Iterative Sequence Tagging with Multi-Task Learning.

Cross-Lingual Text Image Recognition Via Multi-Task Sequence to Sequence Learning.

Multi-Task Cross-Lingual Sequence Tagging from Scratch

SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling.

Multi-Task Learning in Natural Language Processing: An Overview

Multi-Task Learning for Front-End Text Processing in TTS

A Multi-task Framework for Named Entity Recognition

Multi-task Learning for Mongolian Morphological Analysis

A novel bundling learning paradigm for named entity recognition

Multi-task and Multi-View Training for End-to-end Relation Extraction

Enhancing Subtask Performance of Multi-modal Large Language Model

A Semi-shared Hierarchical Joint Model for Sequence Labeling

Multiple Task Learning Using Iteratively Reweighted Least Square.

Improving Sequence Tagging Using Machine-Learning Techniques

Network Clustering for Multi-task Learning

Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding.

HirMTL: Hierarchical Multi-Task Learning for dense scene understanding

Learning Multi-Task Communication with Message Passing for Sequence Learning.

Multi-task learning for natural language processing in the 2020s: where are we going?

Usr-mtl: an Unsupervised Sentence Representation Learning Framework with Multi-Task Learning