Abstract:The neural machine translation (NMT), which relies on a large training data (bilingual parallel sentences, for NMT) to obtain the state-of-the-art performance, is similar with deep learning. In order to construct NMT systems, the number of parallel sentences is very important. However, these bilingual resources are scarce for many low-resource language pairs. Although several works attempt to obtain bilingual parallel data from Internet, the quality and quantity of mined bilingual corpus are limited for low-resource language pairs. To address this problem, we propose the multi-view knowledge distillation model (MvKD) that use the knowledge of high-resource language pairs transfer into low-resource languages by leveraging internal language invariant in different languages. In particular, we treat the mining bilingual parallel sentence pair task as classifying task and use the multi-view classifier to detect bilingual parallel sentence pair. For multi-view classifier, we use two views to recognize the semantic difference of two sentences: (i) word-level representations and (ii) sentence-level representations. We encode sentence-level representations to capture semantically similar of two sentences. Moreover, we encode word-level representations to capture word translations in a pair of parallel sentences to avoid the problem that semantically similar but non-parallel sentences. Experimental results demonstrate that our proposed method can significantly mine amount of bilingual corpus and improve the quality of parallel sentences. In particular, we carry out the experiments on several real-world low-resource situations and achieve excellent results.

A Novel Deep Learning Method for Obtaining Bilingual Corpus from Multilingual Website

Obtaining Parallel Sentences in Low-Resource Language Pairs with Minimal Supervision

Cross-Lingual Propagation for Deep Sentiment Analysis

Learning Contextualised Cross-lingual Word Embeddings and Alignments for Extremely Low-Resource Languages Using Parallel Corpora

A Recipe of Parallel Corpora Exploitation for Multilingual Large Language Models

Machine Translation Model based on Non-parallel Corpus and Semi-supervised Transductive Learning

Bilingual Lexicon Induction from Non-Parallel Data with Minimal Supervision.

Mining parallel sentences from internet with multi-view knowledge distillation for low-resource language pairs

Automatic Acquisition of Large-scale Bilingual Sentence Pair

Machine Translation With Weakly Paired Bilingual Documents

Bilingual lexicon induction from non-parallel corpora

Automatic translation of spoken English based on improved machine learning algorithm

Unsupervised Parallel Corpus Mining on Web Data

Learning Multilingual Sentence Embeddings From Monolingual Corpus

A Deep Neural Network Approach To Parallel Sentence Extraction

Sentence Alignment with Parallel Documents Helps Biomedical Machine Translation

Bilingual Corpus Mining and Multistage Fine-Tuning for Improving Machine Translation of Lecture Transcripts

Word Translation Without Parallel Data

A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning.

Unsupervised Bilingual Lexicon Induction Via Latent Variable Models.

CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB