Design of Efficient Financial Big Data Processing and Analysis System Using Machine Learning Technology

Jiayu Yang
DOI: https://doi.org/10.1109/ICAIDT62617.2024.00018
2024-06-07
Abstract:In order to assist quantitative analysis models in quantitatively predicting and controlling risks through an effective credit scoring system, the author proposes using machine learning technology for an efficient financial big data processing and analysis system. In response to the shortcomings of using logistic regression to measure risk in the context of big data, based on the online cash loan data of a consumer finance company in 2017 and 2018, the XGBoost machine learning algorithm was used to establish a model, which was compared and analyzed with the model established by logistic regression. Based on this, a Stacking model was established, aiming to achieve better model performance. The results showed that the ROC curve, AUC value, and KS value of the sample data training set were 0.8707 and 0.514, respectively; Figure 3 shows the ROC curve, AUC value, and KS value of the test set, with an AUC of 0.8554 and KS of 0.5877. Conclusion: The model performance of XGBoost is better than logistic regression and slightly better than that of GBDT.
Computer Science,Business
What problem does this paper attempt to address?