Explain the Black Box for the Sake of Science: the Scientific Method in the Era of Generative Artificial Intelligence

Gianmarco Mengaldo

2024-09-25

Abstract:The scientific method is the cornerstone of human progress across all branches of the natural and applied sciences, from understanding the human body to explaining how the universe works. The scientific method is based on identifying systematic rules or principles that describe the phenomenon of interest in a reproducible way that can be validated through experimental evidence. In the era of artificial intelligence (AI), there are discussions on how AI systems may discover new knowledge. We argue that human complex reasoning for scientific discovery remains of vital importance, at least before the advent of artificial general intelligence. Yet, AI can be leveraged for scientific discovery via explainable AI. More specifically, knowing what data AI systems deemed important to make decisions can be a point of contact with domain experts and scientists, that can lead to divergent or convergent views on a given scientific problem. Divergent views may spark further scientific investigations leading to new scientific knowledge.

Artificial Intelligence,Computers and Society,Dynamical Systems

What problem does this paper attempt to address?

The core issue this paper attempts to address is how to unveil the "black box" of AI systems through explainable artificial intelligence (XAI) methods in the AI era, making their decision-making processes transparent, thereby promoting the development of scientific research. Specifically, the paper explores the following aspects:

1. **The importance of human complex reasoning in scientific discovery**: Although AI systems can discover new knowledge, the authors believe that before achieving artificial general intelligence, human complex reasoning abilities remain crucial for scientific discovery.

2. **Applications and challenges of AI in scientific discovery**: With the development of AI technology, machines can learn patterns and relationships from data, which may help solve scientific problems. However, the "black box" nature of AI systems makes their decision-making processes difficult to understand, limiting scientists' trust in their results and further applications.

3. **Promoting scientific discovery through XAI**: The authors propose that XAI methods can reveal the data or features that AI systems consider important, providing scientists with ways to interact with AI systems. By analyzing this data, scientists can verify the conclusions of AI systems or discover new scientific knowledge. This interaction can not only enhance trust in AI systems but also inspire new research directions.

4. **Re-examining the scientific method in the AI era**: The paper emphasizes the importance of the scientific method and explores how to combine the scientific method with AI technology through XAI methods in the AI era to promote the discovery of new knowledge. The authors propose several key requirements, including accuracy, causality, reproducibility, and understandability, to ensure the effectiveness and reliability of XAI methods.

In summary, this paper aims to explore how to use XAI methods to unveil the "black box" of AI systems, making the application of AI in scientific research more transparent and trustworthy, thereby advancing scientific progress.