FFN: a Fine-grained Chinese-English Financial Domain Parallel Corpus

Yuxin Fu,Shijing Si,Leyi Mai,Xi-ang Li

2024-06-27

Abstract:Large Language Models (LLMs) have stunningly advanced the field of machine translation, though their effectiveness within the financial domain remains largely underexplored. To probe this issue, we constructed a fine-grained Chinese-English parallel corpus of financial news called FFN. We acquired financial news articles spanning between January 1st, 2014, to December 31, 2023, from mainstream media websites such as CNN, FOX, and China Daily. The dataset consists of 1,013 main text and 809 titles, all of which have been manually corrected. We measured the translation quality of two LLMs -- ChatGPT and ERNIE-bot, utilizing BLEU, TER and chrF scores as the evaluation metrics. For comparison, we also trained an OpenNMT model based on our dataset. We detail problems of LLMs and provide in-depth analysis, intending to stimulate further research and solutions in this largely uncharted territory. Our research underlines the need to optimize LLMs within the specific field of financial translation to ensure accuracy and quality.

Computation and Language,Artificial Intelligence,Computational Engineering, Finance, and Science

What problem does this paper attempt to address?

The paper attempts to address the issue of the effectiveness and accuracy of large language models (LLMs) in machine translation within the financial domain. Specifically:

1. **Machine Translation Needs in the Financial Sector**: With the growth of globalization and financial transactions, the demand for Chinese-English translation in the financial sector is increasing. Due to the complexity of financial concepts, professional translators need to invest a significant amount of time and effort to master the relevant knowledge, making automated machine translation particularly important.

2. **Limitations of Existing Datasets**: Existing parallel corpora are either not targeted at the financial domain or suffer from issues such as misalignment and the inclusion of HTML tags, which fail to meet the requirements for high-quality translation.

3. **Performance of Large Language Models in the Financial Domain**: Although large language models perform well in general machine translation tasks, their performance in the financial domain has not been fully explored. Therefore, it is necessary to construct specialized datasets to evaluate these models' translation capabilities in the financial sector.

To address these issues, the authors constructed a fine-grained Chinese-English financial news parallel corpus named FFN and evaluated the translation performance of two large language models (ChatGPT and ERNIE-bot) and two online translation software (DeepL and Google) using multiple evaluation metrics (such as BLEU, TER, and chrF scores). Additionally, they trained a model based on OpenNMT to further validate the dataset's effectiveness. Through this research, the authors aim to reveal the strengths and weaknesses of large language models in financial translation and provide a reference for future studies.