Abstract:This perspective paper is based on several sessions by the members of the Round Table AI at FIRM 1 , with input from a number of external and international speakers. Its particular focus lies on the management of the model risk of productive models in banks and other financial institutions. The models in view range from simple rules-based approaches to Artificial Intelligence (AI) or Machine learning (ML) models with a high level of sophistication. The typical applications of those models are related to predictions and decision making around the value chain of credit risk (including accounting side under IFRS9 or related national GAAP approaches), insurance risk or other financial risk types. We expect more models of higher complexity in the space of anti-money laundering, fraud detection and transaction monitoring as well as a rise of AI/ML models as alternatives to current methods in solving some of the more intricate stochastic differential equations needed for the pricing and/or valuation of derivatives. The same type of model is also successful in areas unrelated to risk management, such as sales optimization, customer lifetime value considerations, robo-advisory, and other fields of applications. The paper refers to recent related publications from central banks, financial supervisors and regulators as well as other relevant sources and working groups. It aims to give practical advice for establishing a risk-based governance and testing framework for the mentioned model types and discusses the use of recent technologies, approaches, and platforms to support the establishment of responsible, trustworthy, explainable, auditable, and manageable AI/ML in production. In view of the recent EU publication on AI, also referred to as the EU Artificial Intelligence Act (AIA), we also see a certain added value for this paper as an instigator of further thinking outside of the financial services sector, in particular where “High Risk” models according to the mentioned EU consultation are concerned.

The role of trust in the use of artificial intelligence for chemical risk assessment

Artificial intelligence as the new frontier in chemical risk assessment

Control Risk for Potential Misuse of Artificial Intelligence in Science

The probable future of toxicology - probabilistic risk assessment

In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes

Trust and trustworthy artificial intelligence: A research agenda for AI in the environmental sciences

Towards Risk-Free Trustworthy Artificial Intelligence: Significance and Requirements

Trust me if you can: a survey on reliability and interpretability of machine learning approaches for drug sensitivity prediction in cancer

TRUST XAI: Model-Agnostic Explanations for AI With a Case Study on IIoT Security

The Importance of Distrust in AI

Applied artificial intelligence and trust—The case of autonomous vehicles and medical assistance devices

A Risk-based Approach to Trustworthy AI Systems for Judicial Procedures

Curated Data In — Trustworthy In Silico Models Out: The Impact of Data Quality on the Reliability of Artificial Intelligence Models as Alternatives to Animal Testing

Trust but Verify: Lessons Learned for the Application of AI to Case-Based Clinical Decision-Making From Postmarketing Drug Safety Assessment at the US Food and Drug Administration

Modeling Human Trust and Reliance in AI-Assisted Decision Making: A Markovian Approach

Artificial intelligence (AI)—it's the end of the tox as we know it (and I feel fine)*

No Trust without regulation!

Trustworthiness Assurance Assessment for High-Risk AI-Based Systems

Explainable AI for clinical risk prediction: a survey of concepts, methods, and modalities

Trust, distrust, and appropriate reliance in (X)AI: a survey of empirical evaluation of user trust

Financial Risk Management and Explainable, Trustworthy, Responsible AI