Evolving machine learning workflows through interactive AutoML

Rafael Barbudo,Aurora Ramírez,José Raúl Romero
2024-02-29
Abstract:Automatic workflow composition (AWC) is a relevant problem in automated machine learning (AutoML) that allows finding suitable sequences of preprocessing and prediction models together with their optimal hyperparameters. This problem can be solved using evolutionary algorithms and, in particular, grammar-guided genetic programming (G3P). Current G3P approaches to AWC define a fixed grammar that formally specifies how workflow elements can be combined and which algorithms can be included. In this paper we present \ourmethod, an interactive G3P algorithm that allows users to dynamically modify the grammar to prune the search space and focus on their regions of interest. Our proposal is the first to combine the advantages of a G3P method with ideas from interactive optimisation and human-guided machine learning, an area little explored in the context of AutoML. To evaluate our approach, we present an experimental study in which 20 participants interact with \ourmethod to evolve workflows according to their preferences. Our results confirm that the collaboration between \ourmethod and humans allows us to find high-performance workflows in terms of accuracy that require less tuning time than those found without human intervention.
Machine Learning
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the problem of automatic workflow composition (AWC) in automated machine learning (AutoML). Specifically, the paper focuses on how to use evolutionary algorithms, especially grammar - guided genetic programming (G3P), to find appropriate sequences of pre - processing and prediction models and their optimal hyper - parameters. However, the existing G3P methods define a fixed grammar that stipulates how workflow elements are combined and which algorithms can be included, which limits the flexibility of the search space. To solve this problem, the paper proposes an interactive G3P algorithm named iEvoFlow, which allows users to dynamically modify the grammar to prune the search space and focus on the areas of their interest. This is the first research that combines the advantages of the G3P method with interactive optimization and human - guided machine learning. In this way, iEvoFlow aims to combine human knowledge and intuition, improve search efficiency, and find workflows with higher performance and in line with user preferences. The paper verifies the effectiveness of iEvoFlow through experiments, including simulated user experiments in a laboratory environment and experiments with 20 real participants. The results show that, compared with methods without human intervention, iEvoFlow significantly reduces the evaluation time while maintaining the prediction performance, and most participants consider this method useful and intuitive.