Abstract:Visually assistive systems for the visually impaired play a pivotal role in enhancing the quality of life for the visually impaired. Assistive technologies for the visually impaired have undergone a remarkable transformation with the advent of deep learning and sophisticated assistive devices. In particular, the paper utilizes the latest machine translation models and techniques to accomplish the Chinese–Braille translation task, providing convenience for visually impaired individuals. The Traditional end-to-end Chinese–Braille translation approach incorporates Braille dots and Braille word segmentation symbols as tokens within the model’s vocabulary. However, our findings reveal that Braille word segmentation is significantly more complex than Braille dot prediction. The paper proposes a novel Two-Part Loss (TPL) method that treats these tasks distinctly, leading to significant accuracy improvements. To enhance translation performance further, we introduce a BERT-Enhanced Segmentation Transformer (BEST) method. BEST leverages knowledge distillation techniques to transfer knowledge from a pre-trained BERT model to the translate model, mitigating its limitations in word segmentation. Additionally, soft label distillation is employed to improve overall efficacy further. The TPL approach achieves an average BLEU score improvement of 1.16 and 5.42 for Transformer and GPT models on four datasets, respectively. In addition, The work presents a two-stage deep learning-based translation approach that outperforms traditional multi-step and end-to-end methods. The proposed two-stage translation method achieves an average BLEU score improvement of 0.85 across four datasets.

Improving End-to-End Text Image Translation From the Auxiliary Text Translation Task

Cross-Lingual Text Image Recognition Via Multi-Task Sequence to Sequence Learning.

Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task

Adaptive multi-task learning for speech to text translation

Rethinking and Improving Multi-task Learning for End-to-end Speech Translation

Exploring Better Text Image Translation with Multimodal Codebook

Multi-Teacher Knowledge Distillation for Text Image Machine Translation

Modal Contrastive Learning based End-to-End Text Image Machine Translation

Translation-Enhanced Multilingual Text-to-Image Generation

E2TIMT: Efficient and Effective Modal Adapter for Text Image Machine Translation

HybridVocab: Towards Multi-Modal Machine Translation Via Multi-Aspect Alignment

End-to-End Tibetan-Chinese Speech Translation Based on Multi-task and Multi-level Pre-training

An End-to-End Scalable Iterative Sequence Tagging with Multi-Task Learning.

Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Data.

Vector Quantization Knowledge Transfer for End-to-End Text Image Machine Translation.

AnyTrans: Translate AnyText in the Image with Large Scale Models

MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs

MIT-10M: A Large Scale Parallel Corpus of Multilingual Image Translation

Pre-Trained Acoustic-and-Textual Modeling for End-To-End Speech-To-Text Translation.

Transductive Auxiliary Task Self-Training for Neural Multi-Task Models

Enhancing Chinese-Braille Translation: A Two-Part Approach with Token Prediction and Segmentation Labeling