A Hypothesis on Good Practices for AI-based Systems for Financial Time Series Forecasting: Towards Domain-Driven XAI Methods

Branka Hadji Misheva,Joerg Osterrieder

2023-11-14

Abstract:Machine learning and deep learning have become increasingly prevalent in financial prediction and forecasting tasks, offering advantages such as enhanced customer experience, democratising financial services, improving consumer protection, and enhancing risk management. However, these complex models often lack transparency and interpretability, making them challenging to use in sensitive domains like finance. This has led to the rise of eXplainable Artificial Intelligence (XAI) methods aimed at creating models that are easily understood by humans. Classical XAI methods, such as LIME and SHAP, have been developed to provide explanations for complex models. While these methods have made significant contributions, they also have limitations, including computational complexity, inherent model bias, sensitivity to data sampling, and challenges in dealing with feature dependence. In this context, this paper explores good practices for deploying explainability in AI-based systems for finance, emphasising the importance of data quality, audience-specific methods, consideration of data properties, and the stability of explanations. These practices aim to address the unique challenges and requirements of the financial industry and guide the development of effective XAI tools.

General Finance,Machine Learning

What problem does this paper attempt to address?

The paper primarily explores the issues of interpretability and transparency faced when applying Artificial Intelligence (AI) systems in financial time series forecasting and proposes a series of best practices to improve this situation.

The paper points out that Machine Learning (ML) and Deep Learning (DL) technologies are increasingly being applied in financial forecasting tasks. They can enhance customer experience, popularize financial services, strengthen consumer protection, and improve risk management. However, these complex models often lack transparency and interpretability, making their application in sensitive fields like finance challenging. Therefore, Explainable Artificial Intelligence (XAI) methods have emerged, aiming to create models that are easy for humans to understand.

Although existing XAI methods such as Local Interpretable Model-agnostic Explanations (LIME) and SHAP values have made some progress, they still have certain limitations, such as high computational complexity, inherent model biases, sensitivity to data sampling, and challenges in handling feature dependencies.

To address the above issues, the paper proposes several key recommendations:

1. **Data Quality**: Ensure that the data used is accurate, consistent, and complete, which is the foundation for achieving effective interpretability.
2. **Audience-specific Methods**: Customize interpretation methods based on the professional level and needs of the audience.
3. **Consider Data Characteristics**: Especially consider the dependencies between features, which is particularly important for financial data.
4. **Stability of Interpretations**: Ensure the consistency and reliability of interpretation results.

By following these best practices, AI systems in the financial field can not only improve prediction accuracy but also enhance their transparency and interpretability, thereby better meeting the industry's needs.