Abstract:With the rapid development of Natural Language Processing (NLP) technology, the accuracy and efficiency of machine translation have become hot topics of research. This paper proposes a novel Seq2Seq model aimed at improving translation quality while reducing the storage space required by the model. The model employs a Bidirectional Long Short-Term Memory network (Bi-LSTM) as the encoder to capture the context information of the input sequence; the decoder incorporates an attention mechanism, enhancing the model's ability to focus on key information during the translation process. Compared to the current mainstream Transformer model, our model achieves superior performance on the WMT14 machine translation dataset while maintaining a smaller size. The study first introduces the design principles and innovative points of the model architecture, followed by a series of experiments to verify the effectiveness of the model. The experimental includes an assessment of the model's performance on different language pairs, as well as comparative analysis with traditional Seq2Seq models. The results show that while maintaining translation accuracy, our model significantly reduces the storage requirements, which is of great significance for translation applications in resource-constrained scenarios. our code are available at <a class="link-external link-https" href="https://github.com/mindspore-lab/models/tree/master/research/arxiv_papers/miniformer" rel="external noopener nofollow">this https URL</a>. Thanks for the support provided by MindSpore Community.

Improve Statistical Machine Translation with Context-Sensitive Bilingual Semantic Embedding Model.

Do Multi-Sense Embeddings Improve Natural Language Understanding?

A Context-Aware Topic Model for Statistical Machine Translation.

Improving Statistical Machine Translation with monolingual collocation

Learning Bilingual Embedding Model for Cross-Language Sentiment Classification

Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context

Improved Neural Machine Translation With Smt Features

Exploiting Cross-Sentence Context for Neural Machine Translation

Improving Automatic Speech Recognition and Speech Translation Via Word Embedding Prediction

Learning Contextualized Sentence Representations for Document-Level Neural Machine Translation

Improved Learning of Chinese Word Embeddings with Semantic Knowledge.

Context-sensitive Semantic Smoothing for the Language Modeling Approach to Genomic IR

Confidence Based Bidirectional Global Context Aware Training Framework for Neural Machine Translation

HanoiT: Enhancing Context-aware Translation via Selective Context

Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax

Towards Making the Most of Context in Neural Machine Translation

Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features

Efficient Machine Translation with a BiLSTM-Attention Approach

A Novel Bilingual Word Embedding Method for Lexical Translation Using Bilingual Sense Clique

Worse WER, but Better BLEU? Leveraging Word Embedding As Intermediate in Multitask End-to-End Speech Translation

Enhancing Semantic Representations of Bilingual Word Embeddings with Syntactic Dependencies