Abstract:Neural machine translation (NMT) systems trained on clean data usually suffer from performance degradation when translating noisy inputs. Existing works attempt to improve the robustness of NMT normally via data augmentation, where synthetic noisy data are mixed with original clean data, either for training NMT with the standard NMT loss alone, or for tuning auxiliary tasks in a multi-task learning manner. Typical auxiliary tasks include detecting and correcting noises, exploiting noisy outputs for contrastive learning etc. The aforementioned two auxiliary tasks are generally designed independently, and the modules for detecting and correcting noises are heavyweight. In this article, we propose a new framework, DetTransNet ( Det ector- Trans lator Net work), aiming to detect positions of noises in the input and translate the input simultaneously. The newly introduced noise detector module is essentially a lightweight binary classifier built upon the final layer of the encoder of the original Transformer model for the translation task, which is to identify at which position of the input has potential noise. The module has a very few parameters. In order to help the model capture the relationship between clean instances and their noisy counterparts, an extra loss is further introduced to enhance the interaction between clean and noisy data. In this way, we combine noise detection and contrastive learning together. As the model is able to identify and locate noises, a heuristic method is proposed to correct detected noises, in order to achieve better translations. Experiments show that DetTransNet is robust to four types of noises (deletion, insertion, swapping, keyboard), and obtain a substantial improvement of up to 1.6 BLEU points across different datasets.

Secoco: Self-Correcting Encoding for Neural Machine Translation

An Improved Error-Correcting Output Coding Framework with Kernel-Based Decoding

Opportunistic Decoding with Timely Correction for Simultaneous Translation

Neural Machine Translation with Error Correction

Error-Correcting Neural Sequence Prediction

How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise on Machine Translation

Improving Seq2Seq Grammatical Error Correction Via Decoding Interventions

Multi-channel Encoder for Neural Machine Translation

Towards Robust k-Nearest-Neighbor Machine Translation

Synchronous Refinement for Neural Machine Translation

TransGEC: Improving Grammatical Error Correction with Translationese

DetTrans: A Lightweight Framework to Detect and Translate Noisy Inputs Simultaneously

Automatic Grammatical Error Correction Based on Edit Operations Information.

Towards Robust Neural Machine Translation with Iterative Scheduled Data-Switch Training.

Neural Machine Translation with Noisy Lexical Constraints.

Computer Assisted Translation with Neural Quality Estimation and Automatic Post-Editing

Reciprocal Supervised Learning Improves Neural Machine Translation

SemFace: Pre-training Encoder and Decoder with a Semantic Interface for Neural Machine Translation

Chunk-Based Decoder for Neural Machine Translation

Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input

Towards Reliable Neural Machine Translation with Consistency-Aware Meta-Learning