Abstract:Financial risk prediction plays a crucial role in the financial sector. Machine learning methods have been widely applied for automatically detecting potential risks and thus saving the cost of labor. However, the development in this field is lagging behind in recent years by the following two facts: 1) the algorithms used are somewhat outdated, especially in the context of the fast advance of generative AI and large language models (LLMs); 2) the lack of a unified and open-sourced financial benchmark has impeded the related research for years. To tackle these issues, we propose FinPT and FinBench: the former is a novel approach for financial risk prediction that conduct Profile Tuning on large pretrained foundation models, and the latter is a set of high-quality datasets on financial risks such as default, fraud, and churn. In FinPT, we fill the financial tabular data into the pre-defined instruction template, obtain natural-language customer profiles by prompting LLMs, and fine-tune large foundation models with the profile text to make predictions. We demonstrate the effectiveness of the proposed FinPT by experimenting with a range of representative strong baselines on FinBench. The analytical studies further deepen the understanding of LLMs for financial risk prediction.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address two main issues in the field of financial risk prediction: 1. **Outdated Algorithms**: The algorithms currently used in financial risk prediction are relatively outdated, especially in the context of the rapid development of Generative AI and Large Language Models (LLMs). Existing methods have failed to fully leverage these new technologies. 2. **Lack of a Unified Benchmark Dataset**: Research in the financial field has long lacked a unified and open-source high-quality dataset, which has hindered the development of related research. To address these issues, the authors propose two main contributions: 1. **FinPT**: This is a new financial risk prediction method that generates natural language customer descriptions through "Profile Tuning" on large pre-trained foundational models and uses these descriptions for prediction. 2. **FinBench**: This is a benchmark that includes multiple high-quality financial risk datasets, covering three common types of financial risks: default, fraud, and churn. ### Specific Methods 1. **Data Processing**: - Fill tabular data into predefined instruction templates to generate natural language customer descriptions containing all tabular information. - Use large language models (such as ChatGPT) to generate these descriptions. 2. **Model Tuning**: - Fine-tune large pre-trained foundational models (such as BERT, GPT, LLaMA, etc.) using the generated natural language descriptions. - Add a small classifier (usually a feedforward neural network) on top of the hidden states of the foundational model for financial risk prediction. ### Dataset - **FinBench**: Collected hundreds of financial datasets from the Kaggle platform and screened out 10 high-quality datasets, covering three types of financial risks: default, fraud, and churn. - Each dataset provides training, validation, and test sets, and includes additional statistical information and instruction and description texts for Profile Tuning. ### Experimental Setup - **Baseline Models**: Include tree-based ensemble methods (such as Random Forest, XGBoost, CatBoost, LightGBM) and deep neural networks designed specifically for tabular data (such as DeepFM, STG, VIME, TabNet). - **Pre-trained Foundational Models**: Include BERT, FinBERT, GPT-2, T5, FLAN-T5, LLaMA, etc. - **Training Details**: Use two NVIDIA A40 GPUs, adopt mixed-precision training, batch size of 128, maximum sequence length of 128, learning rate of 5e-5, and optimizer AdamW. ### Results and Analysis - **Main Results**: Experimental results on FinBench show that FinPT significantly outperforms traditional tree-based methods and previous neural network models in financial risk prediction tasks, especially when fully fine-tuning large foundational models (such as GPT-2, T5, Flan-T5). - **Performance Comparison**: Detailed comparison of the performance of different models, demonstrating the advantages of FinPT across various datasets. Through these methods and experiments, the paper not only proposes a new financial risk prediction method but also provides a unified benchmark dataset for the field, promoting the development of financial risk prediction research.

FinPT: Financial Risk Prediction with Profile Tuning on Pretrained Foundation Models

FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets

FinGPT: Democratizing Internet-scale Data for Financial Large Language Models

A survey on deep learning for financial risk prediction

Advancing Financial Risk Prediction Through Optimized LSTM Model Performance and Comparative Analysis

Large Language Models for Financial Aid in Financial Time-series Forecasting

FinBen: A Holistic Financial Benchmark for Large Language Models

DeRisk: An Effective Deep Learning Framework for Credit Risk Prediction over Real-World Financial Data

FinBERT: A Pre-trained Financial Language Representation Model for Financial Text Mining

Application Analysis of the Machine Learning Fusion Model in Building a Financial Fraud Prediction Model

Empowering Many, Biasing a Few: Generalist Credit Scoring through Large Language Models

Estimating Financial Fraud through Transaction-Level Features and Machine Learning

Enhancing Data Quality through Self-learning on Imbalanced Financial Risk Data

FinLangNet: A Novel Deep Learning Framework for Credit Risk Prediction Using Linguistic Analogy in Financial Data

CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models

Bank Financial Risk Prediction Model Based on Big Data

Advanced User Credit Risk Prediction Model using LightGBM, XGBoost and Tabnet with SMOTEENN

Analysis of Financial Risk Behavior Prediction Using Deep Learning and Big Data Algorithms

FinGPT: Open-Source Financial Large Language Models