Abstract:Implementing new machine learning (ML) algorithms for credit default prediction is associated with better predictive performance; however, it also generates new model risks, particularly concerning the supervisory validation process. Recent industry surveys often mention that uncertainty about how supervisors might assess these risks could be a barrier to innovation. In this study, we propose a new framework to quantify model risk-adjustments to compare the performance of several ML methods. To address this challenge, we first harness the internal ratings-based approach to identify up to 13 risk components that we classify into 3 main categories—statistics, technology, and market conduct. Second, to evaluate the importance of each risk category, we collect a series of regulatory documents related to three potential use cases—regulatory capital, credit scoring, or provisioning—and we compute the weight of each category according to the intensity of their mentions, using natural language processing and a risk terminology based on expert knowledge. Finally, we test our framework using popular ML models in credit risk, and a publicly available database, to quantify some proxies of a subset of risk factors that we deem representative. We measure the statistical risk according to the number of hyperparameters and the stability of the predictions. The technological risk is assessed through the transparency of the algorithm and the latency of the ML training method, while the market conduct risk is quantified by the time it takes to run a post hoc technique (SHapley Additive exPlanations) to interpret the output.

Supervisory bank risk early warning modeling: an examiner’s first line of defense

Analysis of Two-phase Model of Bank's Risk-behaviors,Financial Supervision Errors and the Effectiveness of Capital Adequacy Requirements

Measuring bank risk: Forward-looking z-score

Approaching European Supervisory Risk Assessment with SupTech: A Proposal of an Early Warning System

Borrower Level Models for Stress Testing Corporate Probability of Default and the Quantification of Model Risk

Bank Supervision and Corporate Credit Supply

A Reversed Early Warning Methodology for Optimal Bank Profit Retention Recommendations

A nature inspired Ying–Yang approach for intelligent decision support in bank solvency analysis

Systemic Financial Risk Forecasting with Decomposition–Clustering-Ensemble Learning Approach: Evidence from China

Can Text-Based Statistical Models Reveal Impending Banking Crises?

Rethinking on Misleading of Machine Learning in Bank Risk Warning System

Do Firm-Level Factors Play Forward-Looking Role for Financial Systemic Risk: Evidence from China

Research on Commercial Bank Risk Early Warning Model Based on Dynamic Parameter Optimization Neural Network

Bank regulation and supervision: A symbiotic relationship

Predicting forward default probabilities of firms: a discrete-time forward hazard model with firm-specific frailty

Toward robust early-warning models: A horse race, ensembles and model uncertainty

Business model and ESG pillars: The impacts on banking default risk

Loss underreporting and the auditing role of bank exams

Enterprise risk management: coping with model risk in a large bank

Measuring the model risk-adjusted performance of machine learning algorithms in credit default prediction

A framework for assessing the systemic risk of major financial institutions