Multi-Task Learning for Features Extraction in Financial Annual Reports

Syrielle Montariol,Matej Martinc,Andraž Pelicon,Senja Pollak,Boshko Koloski,Igor Lončarski,Aljoša Valentinčič

2024-04-08

Abstract:For assessing various performance indicators of companies, the focus is shifting from strictly financial (quantitative) publicly disclosed information to qualitative (textual) information. This textual data can provide valuable weak signals, for example through stylistic features, which can complement the quantitative data on financial performance or on Environmental, Social and Governance (ESG) criteria. In this work, we use various multi-task learning methods for financial text classification with the focus on financial sentiment, objectivity, forward-looking sentence prediction and ESG-content detection. We propose different methods to combine the information extracted from training jointly on different tasks; our best-performing method highlights the positive effect of explicitly adding auxiliary task predictions as features for the final target task during the multi-task training. Next, we use these classifiers to extract textual features from annual reports of FTSE350 companies and investigate the link between ESG quantitative scores and these features.

Computation and Language

What problem does this paper attempt to address?

This paper discusses how to extract features from financial annual reports using a multi-task learning approach, focusing particularly on financial sentiment, objectivity, sentence predictability, and environmental, social, and governance (ESG) content detection. The study suggests that qualitative textual information in annual reports is equally important for assessing company performance, as it may contain valuable weak signals that complement quantitative data on financial performance or ESG standards.

In the paper, researchers propose different multi-task learning methods that combine the extracted information through joint training on different tasks. They find that explicitly incorporating the predictions of auxiliary tasks as features for the ultimate goal task significantly improves performance. Subsequently, they utilize these classifiers to extract text features from the annual reports of FTSE350 companies and investigate the relationship between ESG quantitative scores and these features.

The related work section mentions the literature on annual report analysis, specifically the application of multi-task learning in financial text classification. The paper employs pre-trained language models for multi-task learning to improve model efficiency and task-specific performance.

In the experimental section, the researchers conduct experiments on annotated datasets using various single-task and multi-task classification methods, using macro F1 scores as evaluation metrics. The results show that a system called ExGF-MTL performs the best, as it improves overall performance by explicitly using the predictions of auxiliary tasks as features for the training of the ultimate goal task.

Overall, this paper aims to enhance the comprehensive understanding and evaluation of companies by linking stylistic indicators (such as sentiment, objectivity, and predictability) in annual reports to ESG-related concepts through multi-task learning techniques.