Bootstrapped Unsupervised Sentence Representation Learning.

Yan Zhang,Ruidan He,Zuozhu Liu,Lidong Bing,Haizhou Li

DOI: https://doi.org/10.18653/v1/2021.acl-long.402

2021-01-01

Abstract:Yan Zhang, Ruidan He, Zuozhu Liu, Lidong Bing, Haizhou Li. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

What problem does this paper attempt to address?

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Wenhao Zhu,Tengjun Yao,Jianyue Ni,Baogang Wei,Zhiguo Lu

DOI: https://doi.org/10.1371/journal.pone.0193919

IF: 3.9

2019-01-01

IEEE Access

Abstract:Textual representations play an important role in the field of natural language processing (NLP). The efficiency of NLP tasks, such as text comprehension and information extraction, can be significantly improved with proper textual representations. As neural networks are gradually applied to learn the representation of words and phrases, fairly efficient models of learning short text representations have been developed, such as the continuous bag of words (CBOW) and skip-gram models, and they have been extensively employed in a variety of NLP tasks. Because of the complex structure generated by the longer text lengths, such as sentences, algorithms appropriate for learning short textual representations are not applicable for learning long textual representations. One method of learning long textual representations is the Long Short-Term Memory (LSTM) network, which is suitable for processing sequences. However, the standard LSTM does not adequately address the primary sentence structure (subject, predicate and object), which is an important factor for producing appropriate sentence representations. To resolve this issue, this paper proposes the dependency-based LSTM model (D-LSTM). The D-LSTM divides a sentence representation into two parts: a basic component and a supporting component. The D-LSTM uses a pre-trained dependency parser to obtain the primary sentence information and generate supporting components, and it also uses a standard LSTM model to generate the basic sentence components. A weight factor that can adjust the ratio of the basic and supporting components in a sentence is introduced to generate the sentence representation. Compared with the representation learned by the standard LSTM, the sentence representation learned by the D-LSTM contains a greater amount of useful information. The experimental results show that the D-LSTM is superior to the standard LSTM for sentences involving compositional knowledge (SICK) data.
Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings

Jian Zhu,Zuoyu Tian,Yadong Liu,Cong Zhang,Chia-wen Lo

DOI: https://doi.org/10.48550/arXiv.2210.12857

2022-10-24

Abstract:Inducing semantic representations directly from speech signals is a highly challenging task but has many useful applications in speech mining and spoken language understanding. This study tackles the unsupervised learning of semantic representations for spoken utterances. Through converting speech signals into hidden units generated from acoustic unit discovery, we propose WavEmbed, a multimodal sequential autoencoder that predicts hidden units from a dense representation of speech. Secondly, we also propose S-HuBERT to induce meaning through knowledge distillation, in which a sentence embedding model is first trained on hidden units and passes its knowledge to a speech encoder through contrastive learning. The best performing model achieves a moderate correlation (0.5~0.6) with human judgments, without relying on any labels or transcriptions. Furthermore, these models can also be easily extended to leverage textual transcriptions of speech to learn much better speech embeddings that are strongly correlated with human annotations. Our proposed methods are applicable to the development of purely data-driven systems for speech mining, indexing and search.

Computation and Language,Sound,Audio and Speech Processing
OssCSE: Overcoming Surface Structure Bias in Contrastive Learning for Unsupervised Sentence Embedding

Zhan Shi,Guoyin Wang,Ke Bai,Jiwei Li,Xiang Li,Qingjun Cui,Belinda Zeng,Trishul Chilimbi,Xiaodan Zhu

DOI: https://doi.org/10.18653/v1/2023.emnlp-main.448

2023-01-01

Abstract:Contrastive learning has been demonstrated effective in unsupervised sentence representation learning. Given one sentence, positive pairs are obtained by passing the sentence to the encoder twice using the different dropout masks, and negative pairs are obtained by taking another sentence in the same mini-batch. However, the method suffers from the surface structure bias, i.e., sentences with similar surface structures will be regarded as close in semantics while sentences with dissimilar surface structures will be viewed as distinct in semantics. This leads to the result that paraphrasing a sentence that is dissimilar in surface structure will receive a lower semantic similarity score than inserting a negative word into the sentence. In this paper, we first verify the bias by collecting a sentence transformation testset. Then we systematically probe the existing models by proposing novel splits based on benchmark datasets in accordance with semantic and surface structure similarity. We tackle the bias in two aspects: balancing the learning target by augmenting with data that counters the bias, and meanwhile preserving word semantics by leveraging recall loss to prevent catastrophic forgetting. We evaluate our model on standard semantic textual similarity (STS) tasks using different pre-trained backbones and achieve state-of-the-art averaged performance across the STS benchmarks. Particularly, our models that are fine-tuned with RoBERTa base and RoBERTa large achieve significantly better performance on most benchmark datasets.
Ranking-Enhanced Unsupervised Sentence Representation Learning

Yeon Seonwoo,Guoyin Wang,Changmin Seo,Sajal Choudhary,Jiwei Li,Xiang Li,Puyang Xu,Sunghyun Park,Alice Oh

DOI: https://doi.org/10.48550/arXiv.2209.04333

2023-05-18

Abstract:Unsupervised sentence representation learning has progressed through contrastive learning and data augmentation methods such as dropout masking. Despite this progress, sentence encoders are still limited to using only an input sentence when predicting its semantic vector. In this work, we show that the semantic meaning of a sentence is also determined by nearest-neighbor sentences that are similar to the input sentence. Based on this finding, we propose a novel unsupervised sentence encoder, RankEncoder. RankEncoder predicts the semantic vector of an input sentence by leveraging its relationship with other sentences in an external corpus, as well as the input sentence itself. We evaluate RankEncoder on semantic textual benchmark datasets. From the experimental results, we verify that 1) RankEncoder achieves 80.07% Spearman's correlation, a 1.1% absolute improvement compared to the previous state-of-the-art performance, 2) RankEncoder is universally applicable to existing unsupervised sentence embedding methods, and 3) RankEncoder is specifically effective for predicting the similarity scores of similar sentence pairs.

Computation and Language
Self-supervised Representation Learning for Speech Processing

Hung-yi Lee,Abdelrahman Mohamed,Shinji Watanabe,Tara N. Sainath,Karen Livescu,Shang-Wen Li,Shu-Wen Yang,Katrin Kirchhoff

DOI: https://doi.org/10.18653/v1/2022.naacl-tutorials.2

2022-01-01

Abstract:Hung-yi Lee, Abdelrahman Mohamed, Shinji Watanabe, Tara Sainath, Karen Livescu, Shang-Wen Li, Shu-wen Yang, Katrin Kirchhoff. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorial Abstracts. 2022.
Unsupervised Cross-Lingual Sentence Representation Learning via Linguistic Isomorphism

Shuai Wang,Lei Hou,Meihan Tong

DOI: https://doi.org/10.1007/978-3-030-29563-9_20

2019-01-01

Abstract:Recently, many researches on learning cross-lingual word embeddings without parallel data have achieved success by utilizing word isomorphism among languages. However, unsupervised cross-lingual sentence representation, which aims to learn a unified semantic space without parallel data, has not been well explored. Though many cross-lingual tasks can be solved by learning a unified sentence representation of different languages benefiting from cross-lingual word embeddings, the performance is not competitive with their supervised counterparts. In this paper, we propose a novel framework for unsupervised cross-lingual sentence representation learning by utilizing linguistic isomorphism in both word and sentence level. After generating pseudo-parallel sentence based on the pre-trained cross-lingual word embeddings, the framework iteratively conducts sentence modeling, word embedding tuning and parallel sentences update. Our experiments show that the proposed framework achieves state-of-the-art results in many cross-lingual tasks, as well as improves the quality of cross-lingual word embeddings. The codes and pre-trained encoders will be released upon the paper publishing.
Unsupervised Learning of Structured Representations via Closed-Loop Transcription

Shengbang Tong,Xili Dai,Yubei Chen,Mingyang Li,Zengyi Li,Brent Yi,Yann LeCun,Yi Ma

DOI: https://doi.org/10.48550/arXiv.2210.16782

2022-10-30

Abstract:This paper proposes an unsupervised method for learning a unified representation that serves both discriminative and generative purposes. While most existing unsupervised learning approaches focus on a representation for only one of these two goals, we show that a unified representation can enjoy the mutual benefits of having both. Such a representation is attainable by generalizing the recently proposed \textit{closed-loop transcription} framework, known as CTRL, to the unsupervised setting. This entails solving a constrained maximin game over a rate reduction objective that expands features of all samples while compressing features of augmentations of each sample. Through this process, we see discriminative low-dimensional structures emerge in the resulting representations. Under comparable experimental conditions and network complexities, we demonstrate that these structured representations enable classification performance close to state-of-the-art unsupervised discriminative representations, and conditionally generated image quality significantly higher than that of state-of-the-art unsupervised generative models. Source code can be found at <a class="link-external link-https" href="https://github.com/Delay-Xili/uCTRL" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
Learning Distributed Representations of Sentences from Unlabelled Data

Felix Hill,Kyunghyun Cho,Anna Korhonen

DOI: https://doi.org/10.48550/arXiv.1602.03483

2016-02-10

Computation and Language

Abstract:Unsupervised methods for learning distributed representations of words are ubiquitous in today's NLP research, but far less is known about the best ways to learn distributed phrase or sentence representations from unlabelled data. This paper is a systematic comparison of models that learn such representations. We find that the optimal approach depends critically on the intended application. Deeper, more complex models are preferable for representations to be used in supervised systems, but shallow log-linear models work best for building representation spaces that can be decoded with simple spatial distance metrics. We also propose two new unsupervised representation-learning objectives designed to optimise the trade-off between training time, domain portability and performance.
RankCSE: Unsupervised Sentence Representations Learning Via Learning to Rank

Jiduan Liu,Jiahao Liu,Qifan Wang,Jingang Wang,Wei Wu,Yunsen Xian,Dongyan Zhao,Kai Chen,Rui Yan

DOI: https://doi.org/10.18653/v1/2023.acl-long.771

2023-01-01

Abstract:Unsupervised sentence representation learning is one of the fundamental problems in natural language processing with various downstream applications.Recently, contrastive learning has been widely adopted which derives highquality sentence representations by pulling similar semantics closer and pushing dissimilar ones away.However, these methods fail to capture the fine-grained ranking information among the sentences, where each sentence is only treated as either positive or negative.In many real-world scenarios, one needs to distinguish and rank the sentences based on their similarities to a query sentence, e.g., very relevant, moderate relevant, less relevant, irrelevant, etc.In this paper, we propose a novel approach, RankCSE, for unsupervised sentence representation learning, which incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework.In particular, we learn semantically discriminative sentence representations by simultaneously ensuring ranking consistency between two representations with different dropout masks, and distilling listwise ranking knowledge from the teacher.An extensive set of experiments are conducted on both semantic textual similarity (STS) and transfer (TR) tasks.Experimental results demonstrate the superior performance of our approach over several state-of-the-art baselines.
Pixel Sentence Representation Learning

Chenghao Xiao,Zhuoxu Huang,Danlu Chen,G Thomas Hudson,Yizhi Li,Haoran Duan,Chenghua Lin,Jie Fu,Jungong Han,Noura Al Moubayed

2024-02-13

Abstract:Pretrained language models are long known to be subpar in capturing sentence and document-level semantics. Though heavily investigated, transferring perturbation-based methods from unsupervised visual representation learning to NLP remains an unsolved problem. This is largely due to the discreteness of subword units brought by tokenization of language models, limiting small perturbations of inputs to form semantics-preserved positive pairs. In this work, we conceptualize the learning of sentence-level textual semantics as a visual representation learning process. Drawing from cognitive and linguistic sciences, we introduce an unsupervised visual sentence representation learning framework, employing visually-grounded text perturbation methods like typos and word order shuffling, resonating with human cognitive patterns, and enabling perturbation to texts to be perceived as continuous. Our approach is further bolstered by large-scale unsupervised topical alignment training and natural language inference supervision, achieving comparable performance in semantic textual similarity (STS) to existing state-of-the-art NLP methods. Additionally, we unveil our method's inherent zero-shot cross-lingual transferability and a unique leapfrogging pattern across languages during iterative training. To our knowledge, this is the first representation learning method devoid of traditional language models for understanding sentence and document semantics, marking a stride closer to human-like textual comprehension. Our code is available at https://github.com/gowitheflow-1998/Pixel-Linguist

Computation and Language,Computer Vision and Pattern Recognition
Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model

Mingxin Li,Richong Zhang,Zhijie Nie,Yongyi Mao

DOI: https://doi.org/10.48550/arXiv.2309.06453

2023-12-19

Abstract:Sentence Representation Learning (SRL) is a fundamental task in Natural Language Processing (NLP), with the Contrastive Learning of Sentence Embeddings (CSE) being the mainstream technique due to its superior performance. An intriguing phenomenon in CSE is the significant performance gap between supervised and unsupervised methods, with their only difference lying in the training data. Previous works attribute this performance gap to differences in two representation properties (alignment and uniformity). However, since alignment and uniformity only measure the results, they fail to answer "What aspects of the training data contribute to the performance gap?" and "How can the performance gap be narrowed?", In this paper, we conduct empirical experiments to answer these "What" and "How" questions. We first answer the "What" question by thoroughly comparing the behavior of supervised and unsupervised CSE during their respective training processes. From the comparison, we identify the similarity pattern as a key factor to the performance gap, and introduce a metric, called Relative Fitting Difficulty (RFD), to measure the complexity of the similarity pattern. Then, based on the insights gained from the "What" question, we tackle the "How" question by increasing the pattern complexity of the training data. We achieve this by leveraging the In-Context Learning (ICL) capability of the Large Language Model (LLM) to generate data that simulates complex patterns. By utilizing the hierarchical patterns in the LLM-generated data, we effectively narrow the gap between supervised and unsupervised CSE. We release our codes and appendix at <a class="link-external link-https" href="https://github.com/BDBC-KG-NLP/NGCSE" rel="external noopener nofollow">this https URL</a>.

Computation and Language,Machine Learning
Generate, Discriminate and Contrast: A Semi-Supervised Sentence Representation Learning Framework

Yiming Chen,Yan Zhang,Bin Wang,Zuozhu Liu,Haizhou Li

DOI: https://doi.org/10.48550/arxiv.2210.16798

2022-01-01

Abstract:Most sentence embedding techniques heavily rely on expensive human-annotated sentence pairs as the supervised signals. Despite the use of large-scale unlabeled data, the performance of unsupervised methods typically lags far behind that of the supervised counterparts in most downstream tasks. In this work, we propose a semi-supervised sentence embedding framework, GenSE, that effectively leverages large-scale unlabeled data. Our method include three parts: 1) Generate: A generator/discriminator model is jointly trained to synthesize sentence pairs from open-domain unlabeled corpus; 2) Discriminate: Noisy sentence pairs are filtered out by the discriminator to acquire high-quality positive and negative sentence pairs; 3) Contrast: A prompt-based contrastive approach is presented for sentence representation learning with both annotated and synthesized data. Comprehensive experiments show that GenSE achieves an average correlation score of 85.19 on the STS datasets and consistent performance improvement on four domain adaptation tasks, significantly surpassing the state-of-the-art methods and convincingly corroborating its effectiveness and generalization ability.Code, Synthetic data and Models available at https://github.com/MatthewCYM/GenSE.
Learning Semantic Lexicons Using Graph Mutual Reinforcement Based Bootstrapping

Qi ZHANG,Xi-Peng QIU,Xuan-Jing HUANG,Li-De WU

DOI: https://doi.org/10.1016/s1874-1029(08)60056-7

2008-01-01

ACTA AUTOMATICA SINICA

Abstract:This paper presents a method to learn semantic lexicons using a new bootstrapping method based on graph mutual reinforcement （GMR）.The approach uses only unlabeled data and a few seed words to learn new words for each semantic category. Different from other bootstrapping methods,we use GMR-based bootstrapping to sort the candidate words and patterns.Experi- mental results show that the GMR-based bootstrapping approach outperforms the existing algorithms both in in-domain data and out-domain data.Furthermore,it shows that the result depends on not only the size of the corpus but also the quality.
Alleviating Over-smoothing for Unsupervised Sentence Representation.

Nuo Chen,Linjun Shou,Jian Pei,Ming Gong,Bowen Cao,Jianhui Chang,Daxin Jiang,Jia Li

DOI: https://doi.org/10.18653/v1/2023.acl-long.197

2023-01-01

Abstract:Currently, learning better unsupervised sentence representations is the pursuit of many natural language processing communities. Lots of approaches based on pre-trained language models (PLMs) and contrastive learning have achieved promising results on this task. Experimentally, we observe that the over-smoothing problem reduces the capacity of these powerful PLMs, leading to sub-optimal sentence representations. In this paper, we present a Simple method named Self-Contrastive Learning (SSCL) to alleviate this issue, which samples negatives from PLMs intermediate layers, improving the quality of the sentence representation. Our proposed method is quite simple and can be easily extended to various state-of-the-art models for performance boosting, which can be seen as a plug-and-play contrastive framework for learning unsupervised sentence representation. Extensive results prove that SSCL brings the superior performance improvements of different strong baselines (e.g., BERT and SimCSE) on Semantic Textual Similarity and Transfer datasets. Our codes are available at https://github.com/nuochenpku/SSCL.
On Semi-Supervised Multiple Representation Behavior Learning

Ruqian Lu,Shengluan Hou

DOI: https://doi.org/10.48550/arXiv.1910.09292

2019-10-21

Abstract:We propose a novel paradigm of semi-supervised learning (SSL)--the semi-supervised multiple representation behavior learning (SSMRBL). SSMRBL aims to tackle the difficulty of learning a grammar for natural language parsing where the data are natural language texts and the 'labels' for marking data are parsing trees and/or grammar rule pieces. We call such 'labels' as compound structured labels which require a hard work for training. SSMRBL is an incremental learning process that can learn more than one representation, which is an appropriate solution for dealing with the scarce of labeled training data in the age of big data and with the heavy workload of learning compound structured labels. We also present a typical example of SSMRBL, regarding behavior learning in form of a grammatical approach towards domain-based multiple text summarization (DBMTS). DBMTS works under the framework of rhetorical structure theory (RST). SSMRBL includes two representations: text embedding (for representing information contained in the texts) and grammar model (for representing parsing as a behavior). The first representation was learned as embedded digital vectors called impacts in a low dimensional space. The grammar model was learned in an iterative way. Then an automatic domain-oriented multi-text summarization approach was proposed based on the two representations discussed above. Experimental results on large-scale Chinese dataset SogouCA indicate that the proposed method brings a good performance even if only few labeled texts are used for training with respect to our defined automated metrics.

Computation and Language,Artificial Intelligence
Learning Structured Representation for Text Classification via Reinforcement Learning.

Tianyang Zhang,Minlie Huang,Li Zhao

DOI: https://doi.org/10.1609/aaai.v32i1.12047

2018-01-01

Abstract:Representation learning is a fundamental problem in natural language processing. This paper studies how to learn a structured representation for text classification. Unlike most existing representation models that either use no structure or rely on pre-specified structures, we propose a reinforcement learning (RL) method to learn sentence representation by discovering optimized structures automatically. We demonstrate two attempts to build structured representation: Information Distilled LSTM (ID-LSTM) and Hierarchically Structured LSTM (HS-LSTM). ID-LSTM selects only important, task-relevant words, and HS-LSTM discovers phrase structures in a sentence. Structure discovery in the two representation models is formulated as a sequential decision problem: current decision of structure discovery affects following decisions, which can be addressed by policy gradient RL. Results show that our method can learn task-friendly representations by identifying important words or task-relevant structures without explicit structure annotations, and thus yields competitive performance.
Unsupervised Dialog Structure Learning

Weiyan Shi,Tiancheng Zhao,Zhou Yu

DOI: https://doi.org/10.48550/arXiv.1904.03736

2019-06-25

Abstract:Learning a shared dialog structure from a set of task-oriented dialogs is an important challenge in computational linguistics. The learned dialog structure can shed light on how to analyze human dialogs, and more importantly contribute to the design and evaluation of dialog systems. We propose to extract dialog structures using a modified VRNN model with discrete latent vectors. Different from existing HMM-based models, our model is based on variational-autoencoder (VAE). Such model is able to capture more dynamics in dialogs beyond the surface forms of the language. We find that qualitatively, our method extracts meaningful dialog structure, and quantitatively, outperforms previous models on the ability to predict unseen data. We further evaluate the model's effectiveness in a downstream task, the dialog system building task. Experiments show that, by integrating the learned dialog structure into the reward function design, the model converges faster and to a better outcome in a reinforcement learning setting.

Computation and Language,Artificial Intelligence
Graph Mutual Reinforcement Based Bootstrapping

Qi Zhang,Yaqian Zhou,Xuanjing Huang,Lide Wu

DOI: https://doi.org/10.1007/978-3-540-68636-1_20

2008-01-01

Abstract:In this paper, we present a new bootstrapping method based on Graph Mutual Reinforcement (GMR-Bootstrapping) to learn semantic lexicons. The novelties of this work include 1) We integrate Graph Mutual Reinforcement method with the Bootstrapping structure to sort the candidate words and patterns; 2) Pattern's uncertainty is defined and used to enhance GMR-Bootstrapping to learn multiple categories simultaneously. Experimental results on MUC4 corpus show that GMR-Bootstrapping outperforms the state-of-the-art algorithms. We also use it to extract names of automobile manufactures and models from Chinese corpus. It achieves good results too.
Effective Slot Filling via Weakly-Supervised Dual-Model Learning

Jue Wang,Ke Chen,Lidan Shou,Sai Wu,Gang Chen

DOI: https://doi.org/10.1609/aaai.v35i16.17643

2021-05-18

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:Slot filling is a challenging task in Spoken Language Understanding (SLU). Supervised methods usually require large amounts of annotation to maintain desirable performance. A solution to relieve the heavy dependency on labeled data is to employ bootstrapping, which leverages unlabeled data. However, bootstrapping is known to suffer from semantic drift. We argue that semantic drift can be tackled by exploiting the correlation between slot values (phrases) and their respective types. By using some particular weakly labeled data, namely the plain phrases included in sentences, we propose a weakly-supervised slot filling approach. Our approach trains two models, namely a classifier and a tagger, which can effectively learn from each other on the weakly labeled data. The experimental results demonstrate that our approach achieves better results than standard baselines on multiple datasets, especially in the low-resource setting.
TreeNet: Learning Sentence Representations with Unconstrained Tree Structure.

Zhou Cheng,Chun Yuan,Jiancheng Li,Haiqin Yang

DOI: https://doi.org/10.24963/ijcai.2018/557

2018-01-01

Abstract:Recursive neural network (RvNN) has been proved to be an effective and promising tool to learn sentence representations by explicitly exploiting the sentence structure. However, most existing work can only exploit simple tree structure, e.g., binary trees, or ignore the order of nodes, which yields suboptimal performance. In this paper, we proposed a novel neural network, namely TreeNet, to capture sentences structurally over the raw unconstrained constituency trees, where the number of child nodes can be arbitrary. In TreeNet, each node is learning from its left sibling and right child in a bottom-up left-to-right order, thus enabling the net to learn over any tree. Furthermore, multiple soft gates and a memory cell are employed in implementing the TreeNet to determine to what extent it should learn, remember and output, which proves to be a simple and efficient mechanism for semantic synthesis. Moreover, TreeNet significantly suppresses convolutional neural networks (CNN) and Long Short-Term Memory (LSTM) with fewer parameters. It improves the classification accuracy by 2%-5% with 42% of the best CNN’s parameters or 94% of standard LSTM’s. Extensive experiments demonstrate TreeNet achieves the state-of-the-art performance on all four typical text classification tasks.

Bootstrapped Unsupervised Sentence Representation Learning.

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings

OssCSE: Overcoming Surface Structure Bias in Contrastive Learning for Unsupervised Sentence Embedding

Ranking-Enhanced Unsupervised Sentence Representation Learning

Self-supervised Representation Learning for Speech Processing

Unsupervised Cross-Lingual Sentence Representation Learning via Linguistic Isomorphism

Unsupervised Learning of Structured Representations via Closed-Loop Transcription

Learning Distributed Representations of Sentences from Unlabelled Data

RankCSE: Unsupervised Sentence Representations Learning Via Learning to Rank

Pixel Sentence Representation Learning

Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model

Generate, Discriminate and Contrast: A Semi-Supervised Sentence Representation Learning Framework

Learning Semantic Lexicons Using Graph Mutual Reinforcement Based Bootstrapping

Alleviating Over-smoothing for Unsupervised Sentence Representation.

On Semi-Supervised Multiple Representation Behavior Learning

Learning Structured Representation for Text Classification via Reinforcement Learning.

Unsupervised Dialog Structure Learning

Graph Mutual Reinforcement Based Bootstrapping

Effective Slot Filling via Weakly-Supervised Dual-Model Learning

TreeNet: Learning Sentence Representations with Unconstrained Tree Structure.