Abstract:Contrastive learning is widely used for sentence representation learning. Despite this prevalence, most studies have focused exclusively on English and few concern domain adaptation for domain-specific downstream tasks, especially for low-resource languages like Japanese, which are characterized by insufficient target domain data and the lack of a proper training strategy. To overcome this, we propose a novel Japanese sentence representation framework, JCSE (derived from ``Contrastive learning of Sentence Embeddings for Japanese''), that creates training data by generating sentences and synthesizing them with sentences available in a target domain. Specifically, a pre-trained data generator is finetuned to a target domain using our collected corpus. It is then used to generate contradictory sentence pairs that are used in contrastive learning for adapting a Japanese language model to a specific task in the target domain. Another problem of Japanese sentence representation learning is the difficulty of evaluating existing embedding methods due to the lack of benchmark datasets. Thus, we establish a comprehensive Japanese Semantic Textual Similarity (STS) benchmark on which various embedding models are evaluated. Based on this benchmark result, multiple embedding methods are chosen and compared with JCSE on two domain-specific tasks, STS in a clinical domain and information retrieval in an educational domain. The results show that JCSE achieves significant performance improvement surpassing direct transfer and other training strategies. This empirically demonstrates JCSE's effectiveness and practicability for downstream tasks of a low-resource language.

An Adversarial Joint Learning Model for Low-Resource Language Semantic Textual Similarity.

Improving Multilingual Semantic Textual Similarity with Shared Sentence Encoder for Low-resource Languages

OssCSE: Overcoming Surface Structure Bias in Contrastive Learning for Unsupervised Sentence Embedding

Advancing Semantic Textual Similarity Modeling: A Regression Framework with Translated ReLU and Smooth K2 Loss

A Joint Model for Sentence Semantic Similarity Learning

Computing Semantic Text Similarity Using Rich Features.

Cross-lingual Transfer or Machine Translation? On Data Augmentation for Monolingual Semantic Textual Similarity

A Semi-Supervised Complementary Joint Training Approach for Low-Resource Speech Recognition

Learning Semantic Textual Similarity from Conversations

Fine-grained Semantic Textual Similarity Measurement Via a Feature Separation Network

Semantic textual similarity for modern standard and dialectal Arabic using transfer learning.

Rethinking STS and NLI in Large Language Models

End-to-End Speech Translation with Adversarial Training

MC-SLT: Towards Low-Resource Signer-Adaptive Sign Language Translation

Weakly Supervised Temporal Adjacent Network for Language Grounding

STSG: A Short Text Semantic Graph Model for Similarity Computing Based on Dependency Parsing and Pre-trained Language Models

Language-Adversarial Transfer Learning for Low-Resource Speech Recognition

Joint Spoken Language Understanding And Domain Adaptive Language Modeling

Adversarial task-specific learning.

Linguistically Conditioned Semantic Textual Similarity

JCSE: Contrastive Learning of Japanese Sentence Embeddings and Its Applications