Abstract:Context. Advancements in Machine Learning (ML) are revolutionizing every application domain, driving unprecedented transformations and fostering innovation. However, despite these advances, several organizations are experiencing friction in the adoption of ML-based technologies, mainly due to the shortage of ML professionals. In this context, Automated Machine Learning (AutoML) techniques have been presented as a promising solution to democratize ML adoption. Objective. We aim to provide an overview of the evidence on the benefits and limitations of using AutoML tools. Method. We conducted a multivocal literature review, which allowed us to identify 54 sources from the academic literature and 108 sources from the grey literature reporting on AutoML benefits and limitations. We extracted reported benefits and limitations from the papers and applied thematic analysis. Results. We identified 18 benefits and 25 limitations. Concerning the benefits, we highlight that AutoML tools can help streamline the core steps of ML workflows, namely data preparation, feature engineering, model construction, and hyperparameter tuning, with concrete benefits on model performance, efficiency, and scalability. In addition, AutoML empowers both novice and experienced data scientists, promoting ML accessibility. On the other hand, we highlight several limitations that may represent obstacles to the widespread adoption of AutoML. For instance, AutoML tools may introduce barriers to transparency and interoperability, exhibit limited flexibility for complex scenarios, and offer inconsistent coverage of the ML workflow. Conclusions. The effectiveness of AutoML in facilitating the adoption of machine learning by users may vary depending on the tool and the context in which it is used. As of today, AutoML tools are used to increase human expertise rather than replace it, and, as such, they require skilled users.

AutoML from Software Engineering Perspective: Landscapes and Challenges

Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows

AutoML in The Wild: Obstacles, Workarounds, and Expectations

How far are we with automated machine learning? characterization and challenges of AutoML toolkits

Automated Machine Learning: From Principles to Practices

Assessing the Use of AutoML for Data-Driven Software Engineering

An Empirical Study on the Usage of Automated Machine Learning Tools

Techniques for Automated Machine Learning

Taking Human out of Learning Applications: A Survey on Automated Machine Learning.

A General Recipe for Automated Machine Learning in Practice

A Neophyte With AutoML: Evaluating the Promises of Automatic Machine Learning Tools

AutoML: state of the art with a focus on anomaly detection, challenges, and research directions

Human behavior in image-based Road Health Inspection Systems despite the emerging AutoML

Benchmark and Survey of Automated Machine Learning Frameworks

AutoML: A survey of the state-of-the-art

Position: A Call to Action for a Human-Centered AutoML Paradigm

A Multivocal Literature Review on the Benefits and Limitations of Automated Machine Learning Tools

Towards Green Automated Machine Learning: Status Quo and Future Directions

XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning

A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners

Benchmarking Automatic Machine Learning Frameworks