Abstract:Automated machine learning (AutoML) streamlines the creation of ML models. While most methods select the "best" model based on predictive quality, it's crucial to acknowledge other aspects, such as interpretability and resource consumption. This holds particular importance in the context of deep neural networks (DNNs), as these models are often perceived as computationally intensive black boxes. In the challenging domain of time series forecasting, DNNs achieve stunning results, but specialized approaches for automatically selecting models are scarce. In this paper, we propose AutoXPCR - a novel method for automated and explainable multi-objective model selection. Our approach leverages meta-learning to estimate any model's performance along PCR criteria, which encompass (P)redictive error, (C)omplexity, and (R)esource demand. Explainability is addressed on multiple levels, as our interactive framework can prioritize less complex models and provide by-product explanations of recommendations. We demonstrate practical feasibility by deploying AutoXPCR on over 1000 configurations across 114 data sets from various domains. Our method clearly outperforms other model selection approaches - on average, it only requires 20% of computation costs for recommending models with 90% of the best-possible quality.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to comprehensively consider these three key aspects: predictive performance, model complexity, and resource consumption when automatically selecting the optimal model in time - series prediction. Specifically: 1. **Predictive Performance** (Predictive Error, P): The accuracy of the model when predicting time - series data. 2. **Model Complexity** (Complexity, C): The structural complexity of the model, such as the number of parameters and the model size. 3. **Resource Consumption** (Resource Demand, R): The demand for computing resources during the training and inference processes of the model, such as running time and energy consumption. Existing Automated Machine Learning (AutoML) methods usually only focus on predictive performance and ignore model complexity and resource consumption. This is an important problem in practical applications, especially in resource - constrained environments (such as edge devices). Therefore, the paper proposes the AutoXPCR method, aiming to achieve automatic, interpretable, and resource - aware model selection through multi - objective optimization and meta - learning techniques. ### Main Contributions: 1. **Multi - objective Optimization**: AutoXPCR provides a more comprehensive model evaluation by simultaneously considering predictive performance, model complexity, and resource consumption through a multi - objective optimization framework. 2. **Meta - learning**: Using meta - learning techniques, it estimates the performance of the model in different aspects according to the characteristics of time - series data and the characteristics of the model. 3. **Interpretability**: It provides multi - level explanations, including the score of the recommended model, the feature importance analysis of the meta - learner, and user - controllable priority settings. 4. **Efficiency**: By estimating the model performance through meta - learning, it avoids the time - consuming exhaustive search in traditional methods and significantly reduces the computational cost. ### Experimental Verification: The paper verifies the effectiveness of AutoXPCR by conducting experiments on 114 datasets. The experimental results show that AutoXPCR can achieve near - optimal predictive performance when recommending models, while only requiring 20% of the computational cost of traditional methods. In addition, the paper also demonstrates the interpretability and interactivity of AutoXPCR, enabling users to better understand and control the model selection process. ### Conclusion: AutoXPCR is an innovative method that can automatically select the optimal model in time - series prediction while considering predictive performance, model complexity, and resource consumption. This method not only improves the efficiency of model selection but also enhances the interpretability and resource - awareness of the model, and has broad application prospects.

AutoXPCR: Automated Multi-Objective Model Selection for Time Series Forecasting

Multi-Objective Automatic Machine Learning with AutoxgboostMC

Model Selection Via Automated Machine Learning

A ranking prediction strategy assisted automatic model selection method

auto-sktime: Automated Time Series Forecasting

TSC-AutoML: Meta-learning for Automatic Time Series Classification Algorithm Selection.

AMLNet: Adversarial Mutual Learning Neural Network for Non-AutoRegressive Multi-Horizon Time Series Forecasting

Autoencoder-kNN meta-model based data characterization approach for an automated selection of AI algorithms

Developing a multivariate time series forecasting framework based on stacked autoencoders and multi-phase feature

AALF: Almost Always Linear Forecasting

Time series model selection with a meta-learning approach; evidence from a pool of forecasting algorithms

Auto-CASH: A Meta-Learning Embedding Approach for Autonomous Classification Algorithm Selection

A Time Series Forecasting Model Selection Framework using CNN and Data Augmentation for Small Sample Data

EFFECT: Explainable Framework for Meta-Learning in Automatic Classification Algorithm Selection

AutoAI-TS: AutoAI for Time Series Forecasting

Measuring the Driving Forces of Predictive Performance: Application to Credit Scoring

Fully Automated Correlated Time Series Forecasting in Minutes

An Efficient Task Implementation Modeling Framework with Multi-Stage Feature Selection and AutoML: A Case Study in Forest Fire Risk Prediction

Dissecting Advanced Time Series Forecasting Models with AICHRONOLENS

Explainable Adaptive Tree-based Model Selection for Time Series Forecasting

An Adaptive Framework for Optimization and Prediction of Air Traffic Management (Sub-)Systems with Machine Learning