Redeeming Data Science by Decision Modelling

John Mark Agosta,Robert Horton
2023-07-01
Abstract:With the explosion of applications of Data Science, the field is has come loose from its foundations. This article argues for a new program of applied research in areas familiar to researchers in Bayesian methods in AI that are needed to ground the practice of Data Science by borrowing from AI techniques for model formulation that we term ``Decision Modelling.'' This article briefly reviews the formulation process as building a causal graphical model, then discusses the process in terms of six principles that comprise \emph{Decision Quality}, a framework from the popular business literature. We claim that any successful applied ML modelling effort must include these six principles. We explain how Decision Modelling combines a conventional machine learning model with an explicit value model. To give a specific example we show how this is done by integrating a model's ROC curve with a utility model.
Machine Learning,Methodology
What problem does this paper attempt to address?
The problem this paper attempts to address is the loss of theoretical foundation in data science amidst its rapid development and widespread application. The author believes that although data science has been quickly applied in various fields, its practice often lacks solid theoretical support, especially in decision theory. The paper proposes a new research agenda by drawing on techniques from artificial intelligence (particularly Bayesian methods) to construct what is called "decision modeling," in order to strengthen the theoretical foundation of data science. Specifically, the paper emphasizes the importance of combining traditional machine learning models with explicit value models and demonstrates through a concrete example how to integrate the ROC curve of a predictive model with a utility model. Additionally, the paper proposes a framework composed of 6 principles that form the basis of decision quality, aimed at guiding how to construct effective decision models. Overall, this paper aims to make the application of data science more systematic and rational by introducing the concept of decision modeling.