Credit Scoring Model for Fintech Lending: an Integration of Large Language Models and FocalPoly Loss

Yufei Xia,Zhiyin Han,Yawen Li,Lingyun He

DOI: https://doi.org/10.1016/j.ijforecast.2024.07.005

IF: 7.022

2024-01-01

International Journal of Forecasting

Abstract:Fintech lending experiences high credit risk and needs an efficient credit scoring model, but it also faces limited data sources and severe class imbalance. We develop a novel two-stage credit scoring model (called LLM-FP-CatBoost) by solving the two issues simultaneously. Large language models (LLMs) initially extract narrative data as a supplementary credit dataset. A new FocalPoly loss is then incorporated with CatBoost to handle the class imbalance problem. Extensive comparisons demonstrate that the proposed LLM-FP-CatBoost significantly outperforms the benchmarks in most circumstances. When making pairwise comparisons between LLMs on the fintech lending dataset, we found that the Chinese-specific LLM, i.e., ERNIE 4.0, achieves the best overall performance, followed by GPT-4 and BERT-based models. The performance decomposition reveals that the superiority is mainly attributed to the new data source extracted by the LLMs. The SHAP algorithm further ensures the interpretability of LLM-FP-CatBoost. The superiority of the proposed LLM-FP-CatBoost model remains robust to hyperparameters of the loss function, specific LLMs, and other extraction methods of narrative data. Finally, we discuss some managerial implications concerning credit scoring in fintech lending.