Abstract:Unstructured data, especially text, continues to grow rapidly in various domains. In particular, in the financial sphere, there is a wealth of accumulated unstructured financial data, such as the textual disclosure documents that companies submit on a regular basis to regulatory agencies, such as the Securities and Exchange Commission (SEC). These documents are typically very long and tend to contain valuable soft information about a company's performance. It is therefore of great interest to learn predictive models from these long textual documents, especially for forecasting numerical key performance indicators (KPIs). Whereas there has been a great progress in pre-trained language models (LMs) that learn from tremendously large corpora of textual data, they still struggle in terms of effective representations for long documents. Our work fills this critical need, namely how to develop better models to extract useful information from long textual documents and learn effective features that can leverage the soft financial and risk information for text regression (prediction) tasks. In this paper, we propose and implement a deep learning framework that splits long documents into chunks and utilizes pre-trained LMs to process and aggregate the chunks into vector representations, followed by self-attention to extract valuable document-level features. We evaluate our model on a collection of 10-K public disclosure reports from US banks, and another dataset of reports submitted by US companies. Overall, our framework outperforms strong baseline methods for textual modeling as well as a baseline regression model using only numerical data. Our work provides better insights into how utilizing pre-trained domain-specific and fine-tuned long-input LMs in representing long documents can improve the quality of representation of textual data, and therefore, help in improving predictive analyses.

A Comparative Analysis of Temporal Long Text Similarity: Application to Financial Documents

An adaptive method for text domain similarity calculation

A New Representation and Distance Measure for Financial Time Series

Document Similarity for Texts of Varying Lengths via Hidden Topics

A novel similarity measure framework on financial data mining

Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives

A Comparison of Document Similarity Algorithms

Textual Analysis in Accounting and Finance: A Survey

Experimental analysis of similarity measurements for multivariate time series and its application to the stock market

Text analysis in financial disclosures

A survey on the techniques, applications, and performance of short text semantic similarity

Predict financial text sentiment: an empirical examination

Comparative Study of Long Document Classification

An Ensemble Semantic Textual Similarity Measure Based on Multiple Evidences for Biomedical Documents

Statutes Recommendation Based on Text Similarity.

Stock Market Analysis with Text Data: A Review

Textual sentiment in finance: A survey of methods and models

Text‐based sentiment analysis in finance: Synthesising the existing literature and exploring future directions

Measurement of Text Similarity: A Survey

FETILDA: An Effective Framework For Fin-tuned Embeddings For Long Financial Text Documents

Measuring Financial Time Series Similarity With a View to Identifying Profitable Stock Market Opportunities