Explain the Black Box for the Sake of Science: The Scientific Method in the Era of Generative Artificial Intelligence

Abstract. The scientific method is the cornerstone of human progress across all branches of the natural and applied sciences, from understanding the human body to explaining how the universe works. The scientific method is based on identifying systematic rules or principles that describe the phenomenon of interest in a reproducible way that can be validated through experimental evidence. In the era of generative artificial intelligence, there are discussions on how AI systems may discover new knowledge. We argue that human complex reasoning for scientific discovery remains of vital importance, at least before the advent of artificial general intelligence. Yet, AI can be leveraged for scientific discovery via explainable AI. More specifically, knowing the ‘principles’ the AI systems used to make decisions can be a point of contact with domain experts and scientists, that can lead to divergent or convergent views on a given scientific problem. Divergent views may spark further scientific investigations leading to interpretability-guided explanations (IGEs), and possibly to new scientific knowledge. We define this field as Explainable AI for Science, where domain experts – potentially assisted by generative AI – formulate scientific hypotheses and explanations based on the interpretability of a predictive AI system. To support the argument, we go through a simple practical thought experiment in the Earth Sciences related to extreme weather. This discipline is particularly sensitive to the limitations of the argument we propose, and it allows us to draw some important conclusions and potential future directions, namely the need for causality and reproducibility, among others.